Friday, June 24, 2005

lab 33 - profile lockfs


lab 33 - profile lockfs


Last lab I talked up the benefits of lockfs. I started using it in my program in place of channel locks and profiled the code using prof(1) to see it's impact on performance. The results were a little disappointing but not really suprising.

One of the test programs I'm using while developing btree application is a simple indexer. It reads a set of text files, tokenizes each file into a set of Terms, then in the database creates a document id for each document and a termid for each distinct Term and adds to the database a docid-Hit-termid relation. I then run this on a few megabytes of text data; such as all email messages for a year,

% prof -n folkfs/tok -i /n/mbox/2001/*/*

I've frequently used prof while developing the btree. Locking always presents a bottleneck. When I had the locks based on channels and a cache that also behaved as a lock, both took up large percentage of samples when profiling lines of code.

The test I did most recently was a single threaded application using lockfs and compared it with the same program without the lockfs. I obviously expected some performance loss when using lockfs but the results showed that even with no blocking for other processes the time it took to interact with lockfs completely dominated the runtime of the program. 54% of samples occurred during the statement

lock = nil;

And another 33.3% on runtime spent in the lockfs module. The total runtime with lockfs was 32 seconds, and without lockfs 11 seconds.

Even with these results, I will not discard lockfs. The benefits of a distributed, garbage collected lock make it still worth keeping. But I will selectively turn off locking when doing a bulk load of an application. A danger I noticed is that the cost of obtaining and releasing a lock started me thinking about holding locks longer to avoid as much interaction with lockfs, but this just creates a different kind of bottleneck.

The only lesson from this is always prof your code. In Inferno it's easy.

No comments: