inferno programmer's notebook

NAME lab 85 - stowage NOTES In an earlier post I defined a venti-lite based on two shell scripts, getclump and putclump, that stored files in a content addressed repository, which in that instance was just an append-only gzip tar archive with an index. After learning a little about the git SCM , this lab re-writes those scripts to use a repository layout more like git's. The key thing to know about the git repository is that it uses sha1sum(1) content addressing and that it stores the objects as regular files in a filesystem using the hash as the directory and filename, objects/hh/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh In the objects directory is 256 directories named for every 2 character prefix of the sha1hash of the object. The filename is the remaining 38 characters of the hash. Putclump calculates the hash, slices it to make the prefix and filename, tests if the file already exists, and if not writes the compressed data to the new file. Here is the important part...

NAME lab 84 - gridfs pattern (mapreduce) NOTES I've mentioned mapreduce in previous posts . It makes a good example application for thinking about grid computing. This lab is also about mapreduce although the point here is to illustrate an inferno pattern for grid computing. I'll call it here the gridfs pattern. Say you have a grid of compute nodes and you want to distribute and coordinate work among them. For the gridfs pattern you construct a synthetic file system that will get exported to all the nodes. The file system is the master process and all clients to the file system are workers. Both cpu(1) and rcmd(1) use the rstyxd(8) protocol that exports the local namespace when running remote jobs. To implement the gridfs pattern we bind our master fs into our local namespace so it gets exported when we run multiple workers across our compute grid. A very simple example of this pattern is explained in the Simple Grid Tutorial Part 2 . I export a named pipe with a pro...

NAME lab 57 - createuser NOTES I've tried setting up some services for the Inferno community. See Getting Started on the Inferno wiki for how to create a user id. There is a public signer, registry, and kfs. If you mount the registry you might find a couple more. Use the certificates for your services too. You can create an id, get a certificate and announce some of your own services using the registry then anyone else using the same signer should be able to mount your service. Acme makes a remarkable collaborative environment. With wiki and irc clients and the shared kfs, a common naming convention for mounted services, it's a wonderful thing to type paths or commands using one or these channels and have others read and execute the commands. I hope these servcies and the Acme environment mean people interested in inferno and connect in new ways. Most of the services are commands that come with the standard Inferno. What I had to create for this experiment is the create...

NAME lab 32 - lockfs DESCRIPTION I've struggled for a while trying to get multi-reader-single-writer locks (rwlock) working the way I wanted for a database I've been working on (more in a later lab). I recently realized the answer was in inferno all along. The problem was not in implementing a rwlock, which i had done for tickfs using channels and it worked fine, the problem came when I needed to kill a process that might be holding a lock. A lock based on using channels usually means that a spawned process is waiting for an event on a channel either to obtain or free the lock. However, if the process that holds a lock is killed it doesn't get to send the unlock event to the lock manager process. You might think, just don't kill the processes that might hold locks. But killing processes is sometimes hard to avoid. For example, when a Flush message arrives, or to cancel a query that launched multiple processes. Programs that depend on locking based merely on...

NAME lab 30 - wikifs for inferno DESCRIPTION I ported plan 9's wikifs to inferno. There were various aspects of the application I wanted to learn about--such as, the cache, the transforms from wiki syntax to html, the method for using html templates, and the details of creating the namespace--that it seemed worthwhile to write it in limbo to absorb it all. I also had a hunch I'd be borrowing a lot of the code for my next file system. The differences between the original and the port are small. The one significant difference is the approach to locking data structures. The original uses locks for the Map and the Cache memory as well as file locks for accessing disk. For the port I attempted to avoid the memory locks altogether. One reason being that when the Flush message is received I attempt to kill the proc processing the request, but that might leave stray locks. The f...