Monday, December 20, 2004

lab 24 - google api


lab 24 - call google api from inferno


I looked around to see what was available. Inferno has no WSDL or SOAP support, and limited xml. However, most of the SOAP messages are boiler plate, so we just need a program to handle the http. The program svc/webget/webget is something I hadn't noticed before. It serves a file /chan/webget which we can write GET and POST messages to and read back the result.

With this in mind I tried writing shell scripts to read and write the SOAP messages to the google api url.

I needed to debug webget a little because the POST didn't work. The changed files are attached below.

The google api download came with sample SOAP messages. I inserted my key (removed from the attached files), and placed the string PHRASE where the search phrase was to appear. The shell script runs sed on the file replacing PHRASE with the actual search phrase then writes the POST message and SOAP message to /chan/webget

     <> /chan/webget {
      x := "{sed 's/PHRASE/' ^ 
       $phrase ^'/' doGoogleSearch.xml}
      size=${len $x}
      echo POST∎ ^ 
       $size ^ 
       ∎0∎ ^
       ∎text/xml∎no-cache >[1=0]
      echo -n $x >[1=0]
      getlines {f = ${unquote $line}
       if { ~ ${hd $f} OK} {
        f = ${tl $f}
        read ${hd $f}
       } {exit}

The first line read from /chan/webget is the result status. Either

     OK length reqid type url


     ERROR reqid message

I use the length returned from the OK message to read the whole rest of the message to standard output.

To build, copy the limbo *.[bm] files to /svc/webget/webget and mk install. The and files should be changed to point to your .xml soap messages. Change the key in the .xml files to your key. Then for example,

     % svc/webget/webget&
     % spell britany spars
     britney spears
     % fn g {google $* |grep URL:}
     % g plan9

This also works through an http proxy if you configure the proxy in /services/webget/config.

I piped the SOAP output from through a command called rss which I originally wrote to format RSS XML. This just removes the markup and makes it a little easier to read.


How easy was that! Well it could be easier. Is all that SOAP boilerplate really needed? <Insert your rant against SOAP here>



Tuesday, December 14, 2004

lab 23 - tickfs


lab 23 - annotating a file system


i have tried out some experiments with annotating the filesystem; really, several iterations of the same idea.

my latest attempt is tickfs, which i think comes close to doug mcilroy's idea of a metadata file system.

each file served by tickfs is a binary relation, the key is usually an absolute pathname, and the attribute is defined by the name of the relation. each relation is also bi-temporal. it's a historical record of changes to each key, value pair and it determines whether a key, value pair is true at a given time.

adding metadata is simple,

      % echo ${quote i . /doc/ 'the inferno shell' } > /n/tickfs/desc
      % echo ${quote i . /doc/ 'rogger peppe' } > /n/tickfs/author

the dot indicates the current time, but any time represented as seconds since the epoch can be given.

queries are pipelines,

      % 9ls |taq /n/tickfs/author |taq -j /n/tickfs/desc

the query uses the timestamp of the files to return the currently effective values for the attributes 'author' and 'desc'.

it relies on a disciplined user to maintain a single logical namespace and to keep it consistent with the namespace of keys within tickfs. plan9 and inferno make it easy to alter the namespace; but the discipline of arranging a clear, consistent namespace is harder.

an example that includes fossil is to bind a directory from dump over the logical namespace. the currently effective values will likely vary over time, and so obtain different results from tickfs.

             % 9ls /doc |taq /n/tickfs/desc
             % bind /n/dump/2004/0101/doc /doc
             % 9ls /doc |taq /n/tickfs/desc

also, i have tried indexing file contents within tickfs, so combinations of keyword search and named descriptors can be used

      % search bug |taq /n/tickfs/author |grep caerwyn |taq /n/tickfs/desc

tickfs runs on inferno, but lookfs, which is an earlier iteration using libstyx, is very similar and should run on plan9.


The source for this lab is in the archive on my homepage.

lab 22 - news aggregator


lab 22 - news aggregator


Rssget downloads an RSS xml file indexes it into tickfs. Then using tickquery, which should be copied into /dis/svc/httpd you can query the RSS aggregator using address http://localhost/magic/tickquery.

% mkdir rss
% cd rss
% mount -c {tickfs} /n/rss
% touch /n/rss/index /n/rss/desc /n/rss/chan /n/rss/title
% svc/httpd/httpd&

use localcron to call rssget periodically on your favourite news feeds.

tickquery looks if the first keyword begins with 'date:' the rest of the word is treated as a rng(1) and restricts the results to within the date range.

For example, to search for RSS entries since 1st of the current month mentioning google,

date:1, google

or show me all entries collected on the 21st of the current month


Here is a demo of this service. more interesting examples I have yet to work out. it gets more interesting given temporal indexes of mail, mailing lists, rss feeds, stock ticker, visited web pages, images and annotations of local files and links. then a context around a particular event can be constructed...


The source for this lab is in the archive on my homepage.

lab 21 - popi


lab 21 - popi


Ported popi from the book Beyond Photography - The Digital Darkroom which is online and many of the examples from the book should work.

I took the C code, translated it to limbo, then tried to add the trig functions and polar coordinates. I referenced Rich Burridge's version of popi to get ideas while adding the extra features.

See the guide file here for many of the examples from chapter 3

The combination of the program and the online text makes a good example of an active essay. While reading the text the user can type in the formulas and edit them to explore further into the tool and it's possibilites. It enhances reading of the text and understanding of the tool.


This took longer to debug than I expected and I still have doubts about parts of it. The polar coordinates when used as a target do not work, rounding errors leave gaps in the matrix.

I was going to implement conway's life as a special function and a method for repeating a transformation so life could be run on an imported graphic. There are endless possibilities here.

I would like to implement a similar tool for sound wave editing. I could then plug the resulting waves into signalfs for looping and components of instruments.


,p> popi.b

lab 20 - libmux


lab 20 - libmux in limbo


To learn about mux in detail I tried to implement libmux in limbo. It actually resembles the mux from libventi more than libmux. They are pretty similar.

I was going to do a detailed description of the mux in devmnt but it turns out this has already been done here by nemo.

When I get this working it can be applied to the venti lib for session rpc, to build a libfs for direct client 9p interaction, bypassing devmnt, and for application protocol muxing, say for sam.

This is a partial implementation. I didn't complete send, recv, gettag and settag functions, so this has not been tested. I'm just posting it here now so I don't lose track of it. I'll come back to it later, maybe writing more for the Venti library or libfs. This is more an excercise to understand the workings of libmux and venti's session rpc.


mux.b mux.m

Monday, December 06, 2004

lab 19 - wm

     lab 19 - understanding wmclient

     In trying to understand wm(1) and its supporting
     libraries wmclient(2), tkclient(2), and titlebar,
     I modified them to resemble (if you squint) rio. I
     haven't adopted rio's menus or button controls.
     Here's a screenshot.

     I began with a copy of wmclient. I wanted to peel
     away the layers between the application and wm. I
     removed the dependency on titlebar and created a
     border that would manage the window. I removed
     whatever else I thought was unneccessary. I ended 
     up with myclient which talks directly to wm. 

     Window management is based on button clicks on the
     border. Button 1 moves , button 2 reshapes, and
     button 3 deletes the window.

     I liked the window management so I wanted to use
     that for all windows. To copy the border changes
     back to wmclient.b was easy.

     But most windows programs in inferno use the
     tkclient. For that I needed to modify the
     titlebar. I created four buttons which fill the
     four edges of the rectangle. Each button is bound
     to the three buttons which control the window

     I made the color of the borders different for
     tkclients and wmclients. There aren't many
     wmclients: acme and wm.

     The borders  change color when the window gains or
     loses keyboard focus, but it must accept kbd input
     for this to happen.

     wmclient creates a new window with a transparent
     image, so it's filled with whatever is beneath it.
     This leads to an obvious tool grab to capture and
     image of a part of the desktop. Reshape the window
     to grab the part of the picture then click button
     1 in the image and it writes it to the file named
     on the command line to grab.

     I like the results. I'll probably use this going
     forward and see what other mouse bindings I like.
     But this leads me to modify the tk for commands
     like sh etc., tweaking everything.

     For some reason wm/sh calls a Tk command which
     refers back to the titlebar assuming the .Wm_title
     exists. So I included a modified wm/sh.

     I'm tempted to make more mods to wm/sh to remove
     the menus and make it more like and acme window,
     that is, middle click sends, which is really the
     only menu option I ever use.

     grab.b myclient.b sh.b titlebar.b titlebar.m

     tkclient.b wmclient.b mkfile

Wednesday, December 01, 2004

lab 18 - mux

     lab 18 - mux

     Much of my recent reading has led me back to
     various implementations of a multiplexer. This is
     an important and powerful abstraction and I want
     to understand it better. I know very little about
     it now. I'd like to implement one in limbo, for
     example to multiplex the link between sam and
     samterm as suggested in the protium paper. Hence
     my interest in sam in the earlier lab.

     Here is a some of the things I've been reading
     recently about multiplexers. See if you see a
     pattern. J. Hickey's Master thesis at MIT
     implements mux for plan9 which puts a 9p
     multiplexer into user space and provides
     asynchonous IO for user threads.

     Search 9fans for multiplexing where there is some
     interesting discussion on the use of multiplexers
     in plan9.

     For example, an idea by rog and russ's response

     And futher proposals by russ proposal for mux, 
     local 9p multiplexing and remote 9p multiplexing.

     See also an implementation of a generic rpc
     multiplexer, libmux, in plan9port

     The various window systems by Rob Pike, mux, 8½,
     and rio are multiplexers for their environment,
     the screen, mouse and keyboard.

     The spree game engine is a multiplexer at the
     application leve for the object hierarchy managed
     by the server.

     And in inferno /emu/port/devmnt.c is the
     multiplexer for 9p.

     In the original exokernel paper the authors argued
     the principal function of an operating system is
     to multiplex access to hardware, and should do
     nothing else. Multiplexers are vital to providing
     9p service and in protium the authors argue they
     are a vital piece of infrastructure for
     distributed applications. 

     In summary, this is something I need to learn more
     about. I'll proceed in these labs with studying
     the code to libmux and devmnt and try to implement
     the mux for the sam session.