inferno programmer's notebook

NAME lab 24 - call google api from inferno DESCRIPTION I looked around to see what was available. Inferno has no WSDL or SOAP support, and limited xml. However, most of the SOAP messages are boiler plate, so we just need a program to handle the http. The program svc/webget/webget is something I hadn't noticed before. It serves a file /chan/webget which we can write GET and POST messages to and read back the result. With this in mind I tried writing shell scripts to read and write the SOAP messages to the google api url. I needed to debug webget a little because the POST didn't work. The changed files are attached below. The google api download came with sample SOAP messages. I inserted my key (removed from the attached files), and placed the string PHRASE where the search phrase was to appear. The shell script runs sed on the file replacing PHRASE with the actual search p

NAME lab 23 - annotating a file system DESCRIPTION i have tried out some experiments with annotating the filesystem; really, several iterations of the same idea. my latest attempt is tickfs , which i think comes close to doug mcilroy's idea of a metadata file system. each file served by tickfs is a binary relation, the key is usually an absolute pathname, and the attribute is defined by the name of the relation. each relation is also bi-temporal. it's a historical record of changes to each key, value pair and it determines whether a key, value pair is true at a given time. adding metadata is simple, % echo ${quote i . /doc/sh.ms 'the inferno shell' } > /n/tickfs/desc % echo ${quote i . /doc/sh.ms 'rogger peppe' } > /n/tickfs/author the dot indicates the current time, but any time represented as seconds since the epoch can be given. q

NAME lab 22 - news aggregator DESCRIPTION Rssget downloads an RSS xml file indexes it into tickfs . Then using tickquery , which should be copied into /dis/svc/httpd you can query the RSS aggregator using address http://localhost/magic/tickquery. % mkdir rss % cd rss % mount -c {tickfs} /n/rss % touch /n/rss/index /n/rss/desc /n/rss/chan /n/rss/title % svc/httpd/httpd& use localcron to call rssget periodically on your favourite news feeds. tickquery looks if the first keyword begins with 'date:' the rest of the word is treated as a rng(1) and restricts the results to within the date range. For example, to search for RSS entries since 1st of the current month mentioning google, date:1, google or show me all entries collected on the 21st of the current month date:21 Here is a demo of this service. more interesting examples I have yet to work out. it gets more

NAME lab 21 - popi DESCRIPTION Ported popi from the book Beyond Photography - The Digital Darkroom which is online and many of the examples from the book should work. I took the C code, translated it to limbo, then tried to add the trig functions and polar coordinates. I referenced Rich Burridge's version of popi to get ideas while adding the extra features. See the guide file here for many of the examples from chapter 3 The combination of the program and the online text makes a good example of an active essay. While reading the text the user can type in the formulas and edit them to explore further into the tool and it's possibilites. It enhances reading of the text and understanding of the tool. CONCLUSION This took longer to debug than I expected and I still have doubts about parts of it. The polar coordinates when used as a target do not work, rounding erro

NAME lab 20 - libmux in limbo DESCRIPTION To learn about mux in detail I tried to implement libmux in limbo. It actually resembles the mux from libventi more than libmux. They are pretty similar. I was going to do a detailed description of the mux in devmnt but it turns out this has already been done here by nemo. When I get this working it can be applied to the venti lib for session rpc, to build a libfs for direct client 9p interaction, bypassing devmnt, and for application protocol muxing, say for sam. This is a partial implementation. I didn't complete send , recv , gettag and settag functions, so this has not been tested. I'm just posting it here now so I don't lose track of it. I'll come back to it later, maybe writing more for the Venti library or libfs. This is more an excercise to understand the workings of libmux and venti's session rpc. FILES

NAME lab 19 - understanding wmclient DESCRIPTION In trying to understand wm (1) and its supporting libraries wmclient (2), tkclient (2), and titlebar, I modified them to resemble (if you squint) rio. I haven't adopted rio's menus or button controls. Here's a screenshot . I began with a copy of wmclient. I wanted to peel away the layers between the application and wm. I removed the dependency on titlebar and created a border that would manage the window. I removed whatever else I thought was unneccessary. I ended up with myclient which talks directly to wm. Window management is based on button clicks on the border. Button 1 moves , button 2 reshapes, and button 3 deletes the window. I liked the window management so I wanted to use that for all windows. To copy the border changes back to wmclient.b was easy. But most windows programs in inferno use the tkclient. F

NAME lab 18 - mux DESCRIPTION Much of my recent reading has led me back to various implementations of a multiplexer. This is an important and powerful abstraction and I want to understand it better. I know very little about it now. I'd like to implement one in limbo, for example to multiplex the link between sam and samterm as suggested in the protium paper. Hence my interest in sam in the earlier lab. Here is a some of the things I've been reading recently about multiplexers. See if you see a pattern. J. Hickey's Master thesis at MIT implements mux for plan9 which puts a 9p multiplexer into user space and provides asynchonous IO for user threads. Search 9fans for multiplexing where there is some interesting discussion on the use of multiplexers in plan9. For example, an idea by rog and russ's response And futher proposals by russ

NAME lab 17 - fixup inferno samterm DESCRIPTION There's an inferno version of samterm intended for connection to remote sam running on unix/nt/plan9. It looks like it hasn't been updated for the latest tkclient interface. Get it working. Simple edits to add samtk.b:/^tkcmds/ and samtk.b:/^cmd/ One of the bigger changes I made is to spawn a new proc samtk.b:/^tkproc/ for each file to handle events from the toplevel chan. This was the quickest way I could think of doing it. It might be more correct to create arrays of channels like those already in samterm.m:/Context:/ but that seemed like a lot more work. I use the idiom of returning the pid samtk.b:711 so I can kill the proc when the file is removed, otherwise the windows would be left hanging around. I added the pid to samterm.m:/pid/ Menus were placed wrongly when middle or right clicking in an edit window. Menu

NAME lab 16 - set up a styx service for active essays DESCRIPTION I created a readonly unauthenticated styx service for readers of IPN, who have inferno installed, and want to read the lab notes in acme and execute the embedded scripts. From here on I'll use the term canto for active essays authored and "read" (rather to execute or chant) in the Inferno environment. This entry in my notebook is my first attempt of the new style. It describes the files and formats of the service. Mount the service and open acme on the file for this lab. mkdir /n/canto mount -A tcp!canto.hopto.org!6700 /n/canto acme -c1 /n/canto/16/index.txt Each canto is in a numbered directory containing the following files: text the raw contents of the notes in man (6) format post.html the text file converted to html using man2html for sending to the blog. guide the

in the second post to this blog i talked about wanting this lab notebook to be more interactive, with something like mathematica notebook or squeak project in mind. after reading more about squeak, and thinking again of what i want, i'll try and describe how i'll proceed with this notebook. Alan Kay and Mitchel Resnick have used the term "active essay" to describe a publication that includes computational objects. Resnick describes them as "new forms of narrative expression, in which manipulable computational objects are integrated with text, graphics, and video." Kay in particular stresses the point that the objects in the essay can be deconstructed to learn how to put them together again. The squeak environment uses many visual elements targetted at children for building simulations of ideas they are learning in school--a constructionist view of education for math and sciences. Kay believes the most important use for modern inexpensive personal compute

been very busy of late (new baby!) and had no time for inferno programming after work. I hope to get back into some kind of routine within the next month. TODO list clean up dsp module and write more instruments try out the alphabet typesets with dsp, tickfs, grids write a general btree module for indexes port plan9 sort to inferno ... Aside from inferno I've been playing with the squeak smalltalk environment. I've been very impressed with this. The morphic interface and etoys scripting is very cool. It got me thinking about how these and other ideas could be transferred to inferno, and then I got into a kind of thought paralysis--I got too confused. another interesting paper I read yesterday was protium an infrastructure for partitioning applications. i wonder if some ideas for session persistence could be explored within inferno. i downloaded panda3d a 3d game and simulation engine with python scripting. wouldn't it be cool to link this to inferno as a new in

NAME lab 15 - cluster file DESCRIPTION I'm still curious about layering of filesystems. Ds (3) was an example I looked at before writing signalfs. Another example was cryptfs (lab 2). Ds currently resides in /os/port/devds.c. A while ago I moved it into /emu/port/devds.c but hadn't made any use of it. I have wanted to use it for a rudimentary cluster file system, so I ported ds to limbo to play with distributed files and further explore files that are built from layering of other file systems. I translated the C code and used styxservers to manage the simple, one level namespace. Here's some testing. This is really as much as I've tested it so far. % for (i in `{seq 0 9}) {zeros -v $i 1024 8 > /tmp/chunk ^$i} % echo cat c1 /tmp/chunk2 /tmp/chunk3 >ds/ctl % echo cat c0 /tmp/chunk0 /tmp/chunk1 >ds/ctl % echo mirror m0 ds/c0 ds/c1 > ds/ctl % cat ds/ctl cat c1 /tmp/chunk2 /tmp/c

NAME lab 14 - map reduce functional grid programming DESCRIPTION I read about Google's MapReduce from Rob Pike's interview at slashdot. At the same time I've been studying Inferno's alphabet-grid (1) and am wondering if I can implement map reduce using alphabet. Here's an imaginary example % - {rng , | remote | mapreduce "{tq -1rm /n/tick/tick.bt} "{tock } | /create /tmp/result } Suppose that tick.bt is a log of time spent on tasks where each record is the timestamp, task and number of seconds spent on the task that instance. Rng produces 1 or more date ranges. Remote converts type /fd to an endpoint. Mapreduce will then split a date range, such as one year, into M smaller date ranges. For each subrange it calls rexec passing it the address of an available node, the subrange and map function as parameters.

NAME lab 13 - implement the flute instrument from STK. DESCRIPTION I implemented more of the STK library but his time as a straight forward translation to a limbo module. Much of the protected classes and filters are in dsp.b as ADTs. They all share a similar interface that includes functions mk for building the object and tick for processing the next sample. The instruments are generally larger to implement but follow the same interface. They can be plugged into a signal module and then read and controlled from within signalfs. I've included a few simple modules that can be used to start a new instrument. I also tried to implement the more complicated Flute. It's close, but still doesn't sound right. It all needs a lot more debugging. To test the flute, % signalfs -a /mnt/dsp % echo add flute.dis flute > /mnt/dsp/ctl % sequencer /mnt/dsp/flute < bach.ski > /dev/aud

No code to post tonight because it's unfinished. I'm converting all of STK to limbo, but not directly into signalfs. I'm creating a module that will contain all the sound sources, filters, and effects in the STK, with one ADT for each sound. This can then be used by signalfs to serve a file, which can be a combination of any of the ADTs, or by any other limbo app. Rog has suggested an alternative application using the shell alphabet. I will try this once the library is written. Rog pointed out how inefficient signalfs is in its current form. I agree; the performance is terrible, which makes it compleletly unusable for realtime sound support. This re-implementation will improve performance. But any hardcore DSP programmer is only likely to snicker at our attempt to implement DSP in limbo. At the end of the day I'm doing this to create a framework for ease of experimenting with DSP, not to create a sound system that will out perform all others. That is the tradeoff

NAME lab 12 - implement an oscilloscope for signals from signalfs. DESCRIPTION I implemented an oscilloscope called scope to view the signals produced by signalfs, or other PCM data such as an iaf file stripped of it's header. % scope < bach.raw > /dev/null It writes it's input to it's output, but it doesn't sound very good if directed to /dev/audio . It writes small blocks of samples with small delays between writes, making it sound very choppy. Scope tries to draw 25 frames a second, getting a tick from a timer, and reads 1/25th of a second of samples from the input, then draws it on a Tk panel. This is might be useful when recording input from a microphone % scope < /dev/audio > out.raw It takes as parameter the sample rate and number of channels, stereo or mono. CONCLUSION Not being able to listen and see the waveform at the same time mak

I spent the time this evening downloading the Microsoft C/C++ toolkit and the SDK. I have only built emu for linux and plan9 so far. So it's time to start building an XP version. I also expect to be modifying the audio driver to add some more advanced features, such as 8 channels. While waiting for the downloads I found this site about audio networking interesting. A DTMF decoder would be a nice lab to do some night. I tried compiling the inferno distribution for XP. I'm missing LIB.exe. I used link /lib instead. Got most of it compiled, including emu.

NAME lab 10 - delay line DESCRIPTION I am continuing to add signal modules to signalfs copying the implementations from stk. Today I'm working on the delay line, and whatever else I can implement in two hours. The delay line does not fit the model of signals I have created so far. From the STK it looks like it is used more as a utilitly class than a standalone filter. Its used by the echo class which actually does the mix of the current input with the delayed input. I could of course do the same thing and have delay as functions with the dsp module. Trying to use the delay, or echo, brings up a number of issues. How am I going to stack multiple filters ontop one another and still be able to ctrl each one independently? To access to each ctl file I'd need to know the conversation number. This might be tricky to find out if I have multiple instruments each being built from man

NAME lab 9 - create tmap image directly instead of using Tk DESCRIPTION I modified tmap to use draw-image (2) because using tk (2) commands to create so many rectangles was slow and used a lot of main memory. The changes were straight forward. The Tk cmd TkCmd(t, sprint(".fc.c create rectangle %.1f %.1f %.1f %.1f " + " -fill #%s -outline black -width %.2f", r.x, r.y, r.x+r.w, r.y+r.h, dc[depth], 0.5)); becomes rect := Rect((int r.x, int r.y), (int(r.x+r.w), int(r.y+r.h))); img.draw(rect, t.display.color(dc[depth]), nil, (0,0)); img.border(rect, 1, t.display.black, (0,0)); where img is the global Image destination. I then update the image on the canvas tk->putimage(t, "tmap", img, nil); tk->cmd(t, ".fc.c coords tmap 0 0; update"); tmap is the name of the image previously created with image (9) and associated with canvas (9). CONCLUSION It works much faster

NAME Music band spree client and engine DESCRIPTION I have been reading through spree code and wondering what engines I could write to learn more about this type of filesystem. Here is one idea that incorporates signalfs . The aim is to create a distributed electronic music band. Each player is a client to spree and can create instrument objects and play notes on these instruments. The other players will have the instruments and sounds replicated locally by their spree client; and similarly instruments and notes they play are heard by the other musicians. Each spree client runs a separate signalfs but with a common set of signal modules. The spree action commands will be SKINI with extra commands for creating and configuring the instruments. The spree engine manages the shared configuration of instruments. Note events are broadcast to all players. The spree client is therefore a ty

Here is a new tmap.b with the memory leak removed. I just needed to call Tk cmd delete all on the canvas to remove all the old rectangles. And finally a screenshot of tmap using output from du -a / on the root of the Inferno distribution. You can see a file and it's path highlighted in red. I've played about with the coloring a bit, but haven't found anything I really liked. The current scheme is for colors to represent depth. It goes from dark blue to light blue, through greens and yellows and ends on red for the deepest nodes.

NAME lab 8 - small update to treemap. DESCRIPTION Tmap is a small util that takes the output of du and presents a treemap using algorithms from UMD HCILab To make it a bit more useful I want to zoom into the treemap and zoom out again. Button one selects a rectangle highlights it and it's path and presents information about the path and file sizes. Button two zooms one level into the selected area, where the mouse clicked. Button three zooms out one level. This was easy to add using Tk. I bind a command to the mouse events and send a message down a channel. On receiving the message I change the root of the tree, regenerate the layout and redraw the rectangles. CONCLUSION I find this a useful tool to find patterns in disk usage, such as large or duplicate files. Often the file hierachies are very deep, and so drilling down interactively is great. I should re-implement

NAME lab 7 - sequencer; create a simple sequencer that can play back a subset of SKINI messages. SETUP Inferno 4th Edition release 20040830. DESCRIPTION 2004/0928 20:37 SKINI is the sequencing language from the STK. It is a readable form of MIDI, and was designed to be "extensable and hackable"; all of which make it ideally suited to this application and Inferno. Here is a brief example // Measure number 1 =0 NoteOn 0.416667 2 72 64 NoteOff 0.208333 2 72 64 NoteOn 0 2 71 64 NoteOff 0.208333 2 71 64 NoteOn 0 2 72 64 NoteOff 0.416667 2 72 64 NoteOn 0 2 67 64 NoteOff 0.416667 2 67 64 There is one command per line. The line begins with command name followed by parameters separated by space or tabs. The second parameter is always the time delta. For the NoteOn command, the third argument is channel (or

NAME lab 6 - signal modules; implement filter (again) for signalfs and test the recursiveness of the file server. SETUP Inferno 4th Edition 20040830. See also labs 4 an 5. DESCRIPTION 2004/0922 21:24 Changed filter.b to use the clone interface. The ctl file accepts a source command which takes a filename or another signal source which may be served by the same signalfs % echo source /usr/caerwyn/lab/6/mnt/wave > mnt/filter/2/ctl The filter module then opens ctl reads the number of the conversation directory and then opens data . The filter is then responsible for setup of the signal source. It should probably pass through commands written to ctl that it does not understand, so the wave can be controlled through the filter ctl . I made a small change to signalfs.b to not fork the namespace so the mount of the server affects the current process and the filter module is abl

NAME lab 5 - signalfs version 2; generalize and enhance the signalfs still further. Create a server that can load all the signal models, and each module has a connection directory per conversation. SETUP Inferno 4th Edition 20040830. DESCRIPTION The directory hierarchy presented by signalfs is now similar to ip (3). /ctl / module /clone / module / n / module / n /ctl / module / n /data One more requirement is that the server must be recursive, a signal module must be able to open other files served by the same signalfs. (The ds (3) device does this.) To illustrate, say we implement a module to return a Fibonacci number. I use the clone mechanism so each client has it's own connection to the module. To read the fifth Fibonacci number { d=/mnt/modfs/`{read 10} echo 5 >[1=0] read 10 < $d/data }<> /mnt/modfs/clone The modu

NAME lab 4 - signalfs; generalize wavefs to a family of file servers that support a variety of sound sources and filters. SETUP Inferno 4th edition release 20040830. Using wavefs and the Synthesis Toolkit in C++. DESCRIPTION I started with the wavefs source and abstracted out an interface for modules that the fileserver can call. The signal processor implements this interface. The compiled dis module that implements a specific dsp is passed as parameter to the signalfs which exports the standard files /signal /signalctl The interface is quite simple Signal: module { configstr: string; init: fn(args: list of string); config: fn(s: string); tickFrame: fn(): array of real; }; Messages written to /signalctl are passed to config , and reads return the contents of Signal.configstr . Reads from /signal , which is readonly, are sent data generated by tickFrame . I rewrote the wave loop modu

NAME lab 3 - create a filesystem, wavefs, that serves a looped waveform. This is our staring point for playing with DSP. SETUP Inferno 4th edition release 20040830. Using styxservers (2), audio (3), audio (6). and Synthesis Toolkit in C++ stk . Emu is running hosted on Windows XP. (I haven't gotten the sound figured out yet using esd.) DESCRIPTION I've been looking for tools to experiment with DSP. The synthesis toolkit has a good all round collection of DSP routines and the code is easy enough to read. I wanted this "sound workbench" well integrated with inferno. Where possible make the resources files or file2chans. The first step was to make a sound source; an oscillator. For this I created a file server that serves two files /ctl /data I started with the sample code from styxservers-nametree (2). The server is given a raw audio file to loop. I took the

NAME lab 2 - use file2chan to create a cryptfile that encrypts/decrypts all read and writes to an underlying file. This cryptfile can then be used by kfs to create a crypt file system. SETUP Inferno 4th edition release 20040830. Using kfs(4), file2chan(2), keyring-crypt(2). DESCRIPTION Keyring-crypt contains ECB algorithms for block encryption and random access to files. I'll use the Ideaecb for this session. I setup a simple file2chan prog that layers atop another file and passes through the read/writes. cryptfile0.b Tested this. % > t1 % cryptfile0 /chan/crypt t1 % echo this is a test > /chan/crypt % cat /chan/crypt this is a test % ls -l /chan/crypt --rw-rw---- s 0 caerwyn caerwyn 0 May 27 14:41 /chan/crypt The size of the file is always 0. I checked the /appl/cmd/disk/kfs.b for calls it makes to the file. All reads and writes are blocks; blocksize can be given as a parameter. It

NAME lab 1 - implement ideas from postdate in inferno sh. write shell functions that are polymorphic along the valid timeline. SETUP Inferno 4th edition release 20040830. Using sh (1), and tickfs (1) and associated commands, rng (1), taq (1). DESCRIPTION Sh has command blocks which can be passed as params to commands, and executed. I want to exploit this to see if I can implement much of the flavor of Postdate in the shell. I already have tickfs in Inferno which supports bi-temporal binary relations. So the value for a tickfs entry could be the sh command block. Create a new relation with entry of command block % mount -c {tickfs} /n/tick % touch /n/tick/sh.bt % echo ${quote i . add {fc $1 $2 +}} > /n/tick/sh.bt % rng , |taq -1rm /n/tick/sh.bt 1094173750 add '{fc $1 $2 +}' I want shell function to return the command block from tickfs. It has to be a substitution function t

what is an open laboratory? it is an online community of computer scientists (programmers, hackers, hobbyists) experimenting with computers and software. the lab is not affilated with any corporation or university or government. the lab should provide to its members the same benefits of working in an actual laboratory; diversity of thought, collaboration, peer review ... why a lab? the distinction we'd like to make is that the lab is used for learning, discovery, and innovation. we are not developing products for market, or rewriting existing applications for open source distribution. the lab environment would encourage collaboration by making the notebook the medium for sharing work products before they reach the stage suitable for presentation in a report. coming up with new ideas is messy. but peer review, discussion and mentoring are valuable at this early stage. how would an open lab operate? we'd begin by agreeing on a common programming environment (inferno, sq

even getting started is harder than i thought. i'm having trouble deciding what tools to use to present the notebook. i already started it as a blog here thinking i'd just write html as i go and post it at the end of each session. but thinking a little more about it, i want more interaction, and i want it to integrate well within inferno. i could write it in troff man macros, which would work well in inferno. but inferno doesn't support full troff and preprocessors eqn, tbl, and pic. i considered LaTeX; but this seems more appropriate for a final printable version of a report. looking around on the web; the Mathematica Notebook is closer to what I had in mind. a interactive technical notebook. code can be run, graphics and sound presented and manipulated. another similar concept is the Squeak environment. Workspaces can be saved and exported. These can present models, simulations, experiments with objects. so what do we have in inferno. plumbing can drive the interac

this is my inferno lab notebook. each entry, when it describes a programming session in inferno, should include at least the following. purpose; what i hope to get done in the session apparatus and settings; so someone else could reproduce what i've done diary; record of actions and results analysis and conclusions i use inferno for fun. the sessions i describe take place at home, in the evenings, usually monday to friday, 9:00pm to 11:30pm. This isn't always enough time to get things done, so some of the tasks may seem quite trivial. i want to move through a wide range of subjects in computer science and elsewhere, wherever my curiosity takes me. i don't want to limit myself by false subject boundaries. i hope to experiment a little, learn a lot, and, just maybe, discover something new. inferno presents a good environment to do this work, for a variety of reasons i'll explore while i'm using it. welcome. and if you like the idea join the lab.