Monday, December 11, 2006

lab 67 - Inferno emu on Palm Tungsten T3

NAME

Inferno emu on Palm Tungsten T3 (and under similar palm handhelds, see notes).

DESCRIPTION

This lab collects some personal/informal notes that I took while setting up setup Inferno emu running hosted on a T3, the main reasons for doing this are:

  1. Wanted to experiment with Inferno
  2. Get tired of the 'traditional' way of using a computer and by extension a handheld (both POS and Linux), and wanting to employ my time in more useful things.
  3. after some time using Inferno i've noticed that it has all I need, and the pieces that Inferno doesn't have (ogg, mp3, etc.) can be borrowed from the host OS (Linux).

The first section covers the hardware available on the T3, the second section talks about how to setup Linux on the T3, and finishes talking about the setup of Inferno itself to perform some typical tasks, which includes brief notes on how to compile and run emu on T3.

Hardware

I'm going to center in the hardware that is currently supported under Linux, and hence available for Inferno hosted, there are some good reviews about the T3 one that it's worth mention is ppptun3

In summary we can use:

  • SD card support for storage
  • a nice 320x480 TFT LCD for graphics
  • audio support both for playing and recording
  • and buttons and touchscreen for input
  • networking: we have are usb (usbnet), IR (not tested, but should work) and BT (not working due to lack of specs).

It's important to note that T3 as the other palm handhelds have reset features that are useful to recover of some errors, you can read about at resets. Also the process of bringing back the T3 back to life, when not responding to resets is a guide to replug battery, i hope i will not need to flash it through JTAG.

Setting up Linux Booting

The procedure for booting Inferno can be sumarized in:

  1. boot POS (default from ROM) ->
  2. run Garux (from SD Card, 1st partition FAT16) ->
  3. start Linux (using rootfs from 1st partition of SD Card) ->
  4. start Inferno emu-fb (from a /etc/init.d/inferno script)

As Garux is run from Palm OS (POS), the SD card where garux is stored needs to be recognized from POS, which has a FAT16, hence maximum tested SD card size is 1Gb.

This 1Gb SD card can be partitioned the following way,

  • part1 FAT16 30Mb, to store garux and the rootfs, kernel modules and other files needed to boot Linux.
  • part2 ext3 970Mb, to store Inferno and media (audio,docs,...).

All the files and a small howto with instructions on how to boot Linux on a T3 can be found at t3 linux.

On the same place there's a guide to flash the SmallRom partition which holds the POS bootloader, to be able to suspend/resume the T3, otherwise you have to reset the handheld each time you want to power off and booting again takes it's time, this last guide talks also about networking the handheld and a computer using usbnet driver.

Setting up Inferno

Given the hardware available on T3 (and handhelds in general) it's wise to keep the system as small and simple as possible. So the hardware devices are managed by the linux kernel, which provices access to them through the /dev/ devices; /dev/fbX

for the graphics frame buffer, /dev/input/eventX for input events (touchscreen and keys) and /dev/dsp for audio. This devices can be managed by the emu-fb version written by Alexander Syschev for the iPAQ 3970 (arm) and pc platforms that was posted on the inferno-list see the 'Inferno emu with Linux framebuffer support' message and can be easily adapted to work with T3¸ see the 'using Inferno on Palm' message on inferno-list too.

The advantage is that you don't need an X server and all it's deps on such a small device you just launch the emu-fb from the console. On the drawbacks are that you loose the hability to lauch ps/pdf viewers like gv/xpdf, anyway I haven't found gv/postscript interpreters on Familiar, I guess it's too much bloated for small devices.

The audio support could be done by using the legacy OSS interface, there's emulation of it from ALSA, because many old programs still rely on it, which I send to the inferno-list with the subject 'devaudio for Linux' so audio support can be easily added.

The next thing was having a WM/GUI suitable for such small screen dimensions but by using the rio like wm/wm from caerwyn, works fine (update: after reading `Window Systems should be transparent' reverted to Inferno's wm, because it's easier to press buttons than borders).

And creating a nice guide file with the frequently launched commands, i.e. play music, raise/lower audio volume, toggle back-light, etc.

This guide can be opened with acme -c1 -l acme.dump, started by default from wmsetup.

Only recommendation is that to take advantage of the small screen space available is a good practice to keep the names of files/directories short so they can fit in one line in the Acme editor. And a nice thing is that you can even do snarfing and chording easily with the stylus.

If you haven't heard about rio, acme and plumber before, take a careful read to the Plan 9 papers in the REFERENCES section.

To finish I've put down also some notes/reminder about the state of this port on a TODO file.

Experiments

9pod: kind of 9pod, like it was said some time ago on 9fans.

voip: (idea from the beginnings) one thing that I would like to be able to do is to be able to share the audio device '#A' with a remote computer (for start just one), this will allow me to be able to talk with other people, over the net, I know that there're are a lot of projects out there that already give you that, but I wanted something simple thus, the most simple that I could think was:

% styxlisten tcp!firulillo!styx export '#A';
% mount tcp!firulillo!styx /n/remote;
% stream /dev/audio /n/remote/audio;

The above example sort of works, only drawback is that sound is played slow on a 100Mb LAN connection between two Linux hosts running Inferno emu. So the next refinement could be to compress the data before sending it and decompress on receiving it: cat /n/local/home/salva/.mpd/music/misc/mymymy.mp3 |

os madplay -Q -o wave:- - | wav2iaf | auplay -

NOTES

  1. A list of handhelds's running Linux, the status of the ports and hardware list can be consulted at supported handhelds
  2. This talk is about T3, changes to run Inferno emu on another handheld model will be left intentionally as an exercise to the interested reader.

REFERENCES

ididit, hackndev, handhelds, toolchains, Plan 9 papers.

FILES

inferno-lab

SCREENSHOTS

From inferno photos

Monday, November 13, 2006

lab 66 - useful tools

NAME

lab 66 - useful tools

NOTES

This is a light lab about writting some useful day-to-day tools, mainly language and music playing tools.

gask is a simple script for doing searches with google, which is nice when you want to consult something while writting a text, it returns the output in textual form so it can be used/plumbed etc.

the next tools are for rapidly looking definitions in english/spanish/... on a dictionary, and they're handy when you aren't a native speaker.

trgg (translate google) allows you to translate between any pair of languages that are supported by google, the usage of the command is trgg 'sl|dl' text, or |trgg 'sl|dl' where: sl: is the source language (by default english) dl: is the destination language (by default spanish)

while dorg does a search on the dictionaries using the services of www.dict.org site,

drae is is an interface to the "Real Academia de la Lengua" dictionary, so you can consult the meaning of spanish words.

And last but not least there's the google's define: keyword, that looks on the web for definitions of the arguments provided.

For interacting with google's services HTTP POST request support is needed, we can find it in webfsget as in the google-api lab, but we can also get it from hget.c and incorporate it to webgrab with a [-p post] option. That's the aproach followed.

oggplay, madplay and audio, those three scripts are used to play music in the mp3 and ogg formats, for writting them i've needed to modify wav2iaf(1) and auplay(1) to work with data piped to it's standard input and write the result to it's standard output. And last audio is a small utility to query/set the parameters of /dev/audioctl. With this tools it's very easy to set up a few plumbing rules, to handle music files.

Update: Added a rsscheck script to check updates in caerwyn.com and google-code by default.

FILES

lab 66 code

Monday, October 09, 2006

lab 65 - Man pages in Acme

NAME

lab 65 - Man pages in Acme

NOTES

This lab's purpose is to have man-pages opened in acme automatically. Easiest way to do this is to allow plumb to get it's data piped from another program. So we can do:
man2txt /man/sec/file | 
plumb -i -d edit -a action showdata -a filename '/man/file(sec)';
given that we want to plumb it to acme we have to set the destination port -d edit, and instruct acme to show the data with the tag set to the value of the filename argument. So now we can add to our plumbing rules something like:
# man pages
kind is text
data matches '([a-zA-Z0-9\-]+)\(([0-9]+)\)'
plumb alwaysstart man # nasty plumbing hack
plumb start /dis/sh.dis -c '/usr/salva/dis/man -b $2 $1'
where /usr/salva/dis/man is an adaptation of /rc/bin/man from Plan9, and in particular when invoked with the -b option does:
man2txt $file |
plumb -i -d edit -a action showdata -a filename '/man/'^${tl $*}^'('^${hd $*}^')'
But if we now start to try running it as a substitue to te default rule for opening man pages, sooner or later we will find it crashes with the following error,
'Acme: can't read /chan/plumb.edit: buffer too short',
that advice is generated from /appl/acme/acme.b:/plumbproc(). If we dig in the Msg.recv() code we will find that the data received from a channel is limited by maxdata, which is set when initializing the module plumbmsg.m, and gets it's value Dat->PLUMBSIZE in /appl/acme/acme.b:/init(.*), which is by default 1024, thus sending a file bigger that 1024 bytes will crash the plumbproc process. To sort out this we can determine the biggest data that we will send to the plumber, in particular issuing a:
du /man/*/* | sort -n | uniq | tail -n 1
will return us the size of the biggest man page that we can reach, it turns out to be 58Kb, so we set PLUMBSIZE: con 65536, with the following ed script:
% ed /appl/acme/dat.m
5192
/PLUMBSIZE : con 1024/ s/1024/65536
 PLUMBSIZE : con 65536;
w
5193
q
Now we cd /appl/acme and issue a
% mk && mk install
and we're done. And last, we can addapt wurl2txt and wdoc2txt and add a pair of rules to our $home/lib/plumbing to have url's and .docs opened inside acme. You can make and script to emulate hget using
webgrab -r -o - http://url

Update:

After using the man script i wrote for this lab, i've decided that it's enought useful to replace the default /dis/man.dis, to do this i've changed the location of /appl/cmd/man.b to be /appl/lib/man.b (and modified PATH variable in man.m accordingly):
% diff -e appl/lib/man.m.b appl/lib/man.m
37c
 PATH: con "/dis/lib/man.dis";
.
This way is possible to use wm/man also from the man script (-p option). We just have to update it cd /appl/wm and issue a
% mk man.install

FILES

caerwyn.com/lab/65

Tuesday, August 29, 2006

google hosting

NAME

google hosting

NOTES

I've created a project on http://code.google.com to host the code for acme-sac. This the first time I've used svn for a project. But I like it so far. It's nice of google to do this. I like not having to worry about maintaining the source repository.

I've uploaded most of the acme-sac distribution. Probably too much; I might be deleting portions of it, such as /appl/wm/*. I'm kinda torn about what to include and what to remove. I'll lean to removing too much to start, and add back it only the things that I find using.

I've moved the code for emu to /sys. The mkfiles now assume the build is within Inferno using wrappers 9l, 9c, 9ar to the host system. I've only tried this and got it working on Nt. When I eventually get a Linux machine to do builds on I'll get it working on that too. But if anyone wants to help...

Thursday, July 13, 2006

lab 64 - Chat

NAME

lab 64 - Chat

NOTES

Chat is an acme client that connects to the chatsrv filesystem. This is a very basic text chat service. It is considerably simpler than the Irc client. Part of the reason I did this was to have a simpler client I could use to help debug some of the acme interaction, and help me improve Irc. Another reason was a recent urge I had to setup some private communication channels, merely for privacy's sake.

Chatsrv is part of the demo code under /appl/demo. I borrowed code from /appl/demo/chat.b and the Irc client I ported from Russ.

A chatsrv service is running on tcp!canto.hopto.org!6683. But I've only allowed rc4_256 encryption to use the service, and of course you need a certificate from the canto.hopto.org signer (Getting Started).

To mount the service within acme and start Chat

Local mount -C rc4_256 tcp!canto.hopto.org!6683 /n/chat
Chat /n/chat

Note that the simplest client can be run inside wm/sh, or win:

% {cat & cat  >[1=0] < /dev/cons} <> /n/chat/msgs 

Isn't that a beautiful one liner?

FILES

caerwyn.com/lab/64

Saturday, July 08, 2006

lab 63 - Calendar

NAME

lab 63 - Calendar

NOTES

I wrote a little calendar client for acme based on Inferno's wm/calendar(1). It uses rawdbfs(1) in the same way to store calendar entries.

109412212045-Acme-SAC

The calendar window displays a month much like cal(1), and has commands Next and Prev to move between months. Days that have calendar entries have a plus superscript symbol. Right click on the days of the month to open and edit a calendar entry.

To use it make sure you have a directory /mnt/schedule, and a file in your home directory called, schedule.

mkdir /mnt/schedule
touch schedule
Calendar schedule

Or

Local rawdbfs schedule /mnt/schedule
Calendar

FILES

caerwyn.com/lab/63

Sunday, June 25, 2006

General update

NAME

general update

NOTES

I've been quiet for a while so I'll give a general update to things I've been working on. I've been doing little bits on lots of different things, but no substantial progress on any one thing. Basically nothing to merit a release of working code. I'll just itemize the stuff.

The Slashdot of acme:sac made me want to improve a lot of things. Ems setup a project on sourceforge to host the distribution when I get it completed. There were several things I want to include in the next release, such as acme debug client, an acme web client, and an acme factotum client.

  • I fixed the bug, at least running on my version of XP compile emu, that caused emu to crash. Basically XP didn't like the trap handling code that used inline asm. I removed the inline asm and registered a general trap handler, and that seemed to work fine.
  • I did some more testing on the acme debugger client. In doing so, I think I found another way of crashing acme, so that stalled me. But I also discovered the nice stackv command that rog wrote. Its not got a man page but the code is /appl/cmd/stackv.b. This simplifies the acme/debug because I shouldn't need to write stack navigation code. It can just output the stackv commands which can then be button-2 selected.
  • I got further with the acme web client that is based on charon. I got it to run within acme:sac by disabling any write to the draw device, so charon is running windowless but only writing the text to an acme window. I also included some simple forms handling so its possible to use google.
  • Npe put his Inferno GBA port aside for a while, so I tried to take it up. I'd like to get a native Inferno on at least one architecture, and ARM is far more appealing to me than x86. So I've been writing ARM code and running it on the VisualBoyAdvance emulator, and learning about the GBA architecture and the Inferno native code. A lot of fun, but I've got nothing yet to show for it.
  • I started porting the Lucene search engine library. I've convinced myself that Inferno needs a good search engine builtin. Lucene is the best I've seen. I've got some of the low level code working. A small compactfs for the compact files lucene uses, and the routines for reading all the packed data structures in the lucene file formats. But there is much, much more to do.

Sunday, April 16, 2006

lab 62 - software synth csp style

NAME

lab 62 - software synth CSP style

NOTES

This is the next iteration of my software synth code for Inferno. Of particular note is the embrace of CSP style as implementation technique. This was true of the code in lab 60. But this time much more of it actually works and the basic framework, the interfaces, are in place for me to extend it further. I think it makes a nice show case of CSP style programming. I thought my lexis database did too (lab 34) but this code is probably easier and more fun to play with.

I want to post this now before I move onto the next phase, which may add a lot more complexity but won't illustrate any better the CSP style of programming in this application. You are encouraged to edit this code to create your own synthesizer, and use that as a way into studying CSP style.

This synth comes with a basic GUI. Here is a screen shot.

synth1

The GUI is bare bones, designed just so that it very easy to add new knobs for the control of filters. (One of the things that worries me is that I'll get bogged down designing a pretty GUI.)

I'll now try and describe in more detail the code.

The interface for the processes has changed a little since lab 60. It is more general in that all the processes see the same interface and can a be plugged together in many different ways.

The interface is as follows,

 Inst: adt {
  c: Sample;
  ctl: Control;

  mk: fn(insts: Source, f: Instrument): ref Inst;
 };

 Source: type array of ref Inst;
 Sample: type chan of (array of real, chan of array of real);
 Control: type chan of (int, real);
 Instrument: type ref fn(s: Source, c: Sample, ctl: Control);

The main type here is the Instrument function. Each instrument is spawned and interacts with other processes, usually other instruments, through the source, sample, and control channels. An instrument will typically loop forever receiving on the sample or control channel. The message on the sample channel will be an array of real that the instrument should fill with samples or filter the incoming samples, or whatever it likes. It sends the array back down a receive channel that came with the request. Control requests are things such as change frequency, or gain, for example. Finally, an instrument may optionaly take as argument and array of Source instruments it can use when answering requests for an array of samples. A good example is the mixer instrument which takes an array of any other instruments and mixes their samples down to a single stream of samples.

Here's an example of a one-pole filter

# y(n) = b₀x(n) - a₁y(n-1)
onepole(nil: Source, c: Sample, ctl: Control)
{
 lastout := array[channels] of {* => 0.0};
 b := array[1] of {0.4};  #gain
 a := array[2] of {1.0, -0.9};

 for(;;) alt {
 (x, rc) := <-c =>
  for(j := 0; j < channels; j++){
   x[j] = b[0] * x[j] - a[1] * lastout[j];
   for(i := channels+j; i < len x; i += channels)
    x[i] = b[0] * x[i] - a[1] * x[i-channels];
  }
  lastout[0:] = x[len x-channels:];
  rc <-= x;
 (m, n) := <-ctl =>
  case m {
  CPOLE =>
   a[1] = -n;
   if(n > 0.0)
    b[0] = 1.0 - n;
   else
    b[0] = 1.0 + n;
  }
 }
}

Here is some example code to use this filter, and add a knob to control the filter parameter then add the filter to the main loop. See also gui.b:209


 filter := Inst.mk(nil, onepole);
 spawn knob(filter.ctl, "onepole", 0.0, 1.0, 0.01);
 
 ...

  filter.c <-= (<-wrc, wrc);
  rc <-= <-wrc;

I'll show another example that illustrates plugging the instruments together.

mypoly(nil: Source, c: Sample, ctl: Control)
{
  voice := Inst.mk(array[2] of 
     {* => Inst.mk(array[2] of 
        {Inst.mk(nil, waveloop), Inst.mk(nil, adsr)}, 
        instrument)
     }, poly);

  for(;;) alt {
    (a, rc) := <-c =>
        poly.c <-= (a, rc);
    (m, n) := <-ctl =>
        poly.ctl <-= (m, n);
}

I'll pull that first statement apart to make it clearer what's going on. Inst.mk(nil, waveloop) creates the sample and control channels and spawns the waveloop process, passing it the channels and returning an object that stores references to those channels. The same is done for the adsr process, which is an envelope filter. These two sources are passed to the instrument process, which is expecting, say, a generator and filter which it combines to form the monophonic voice that responds to keyon and keyoff control events. Finally, two instances of this instrument voice are passed to the poly process which knows how to mix the input sources into a single polyphonic sound.

Is this an example of higher order function in CSP style?

As before, you need the jit turned on to run this program for it to sound acceptable.

FILES

caerwyn.com/lab/62

Tuesday, April 11, 2006

lab 61 - javascript tool

NAME

lab 61 javascript tool

NOTES

In this lab I wanted to get a standalone javascript interpreter for general scripting use within inferno.

[Deleted a section on reasons why I'd want to do this based on the mix of languages already available--it's not that interesting and is just me trying to rationalize why I like javascript]

There is already a javascript interpreter in inferno as part of the charon web browser. All I'd need to do is add a small set of host objects to interface with the OS.

In this lab I haven't included all the host objects I'd want; not even a fraction. This is just the setup, the beginnings of a javascript tool. The only host object I've added is System which contains three functions: getline, print, and readFile. Getline reads a line from stdin and returns it as a string. Print writes a string to stdout. ReadFile reads the in the contents of a file and returns it as a string.

Suggestions on what the host objects to add are welcome.

Here is an example script showing how to call the tool, called js.

% cat t1.js
function f(n) {
 return n * 2;
}

var s;
while((s = System.getline()))
 System.print(s + f(1));

% echo a | js -f t1.js
a
2% 

FILES

caerwyn.com/lab/61

lab 60 - sequencer using channels

NAME

lab 60 - sequencer using channels

NOTES

In the comments to lab 53 Rog suggested using channels to parse buffers between processes in place of the one-sample-at-a-time technique I was using in my earlier DSP attempts.

This lab is an attempt at Rog's suggestion. It's one limbo file that acts as a simple sequencer and generates it's own voices.

Each instrument should have an interface:

f: fn(c: chan of (array of real, 
         chan of array of real), 
     ctl: chan of (int, real));

This function is spawned and control messages are sent on the ctl channel, and the request for samples and the response channel are sent down c.

It really requires jit to be turned on to sound acceptable.

Here's a sample setup,

% bind -a '#A' /dev
% echo 1 > /dev/jit
% sequencer4 < bachfugue.ski  > /dev/audio

Uncomment sequencer.b4:60 to add a little echo to the music.

I think I'm getting closer to Rog's ideas, but I'm not sure I'm still exploiting it to the fullest. I'm already liking this approach better. I do feel I can change things quicker and plug the modules together easier.

I'll try rearranging things some more.

FILES

caerwyn.com/lab/60

Sunday, March 26, 2006

lab 59 - acme SAC

NAME

lab 59 - acme stand alone complex

NOTES

A project that's been on my mind for quite a while is to package Inferno's acme as a stand alone editor. I only had Windows in mind as a target host, but the work should be quite easy to reproduce on other hosts.

I wanted the editor to blend well with the host system, and work as a substitute for other popular programmer editors such as vim, emacs, or jedit. Acme and Inferno being already stable and mature, there were only a few things I felt needed to be in place for this to work.

  • cut & paste between the host and acme (lab 55)
  • acme to resize with the host window
  • dead simple packaging and install of acme on windows.

This lab covers the code to do the acme resize with host windows.

I copied the code from /emu/port/devpointer.c and made devwmsz.c. The code is almost identical except for the name changes. This device holds a short queue of window resize events and serves a file /dev/wmsize that's the same format as /dev/pointer with x and y fields representing the width and height of the host window.

I modified acme directly to support this new device instead of modifying wm modules, which might have been more appropriate, I'm not sure. I added a new thread to /acme/gui.b to listen for resize events and resize the acme window appropriately.

startwmsize(): chan of Rect
{
 rchan := chan of Rect;
 fd := sys->open("/dev/wmsize", Sys->OREAD);
 if(fd == nil)
  return rchan;
 sync := chan of int;
 spawn wmsizeproc(sync, fd, rchan);
 <-sync;
 return rchan;
}

wmsizeproc(sync: chan of int, 
 fd: ref Sys->FD, ptr: chan of Rect)
{
 sync <-= sys->pctl(0, nil);

 b:= array[Wmsize] of byte;
 while(sys->read(fd, b, len b) > 0){
  p := bytes2rect(b);
  if(p != nil)
   ptr <-= *p;
 }
}

/appl/acme/gui.b:/^eventproc/ responds to the new event on the channel from wmsizeproc,

eventproc()
{
 wmsize := startwmsize();
 for(;;) alt{
 wmsz := <-wmsize =>
  win.image = win.screen.newwindow(wmsz, 
   Draw->Refnone, Draw->Nofill);
  p := ref zpointer;
  mainwin = win.image;
  p.buttons = Acme->M_RESIZE;
  cmouse <-= p;
...

I copied similar code into /appl/wm/wm.b so the standard window manager resizes with the host window (included in this lab's files). After using it a little I think I prefer it not resizing.

I use Acme:SAC as my main environment now, opening up another Inferno session when I want to do graphics work. I think the smoother integration with the host window system is a big win. It feels more like just an editor, neatly hiding a very powerful operating system behind it. I think Acme is one of the jewels of Inferno. Packaging it in this way may lead more people to discover Inferno through Acme. It could provide an easier introduction to the system and allow new users to start doing real work with Acme within a few clicks of downloading the package.

I've created a page for the Acme:SAC download where I'll post updates as I try to improve the packaging. The download is based on the emu from 20060227 Inferno release and my dried dung inferno tree.

FILES

caerwyn.com/lab/59
Acme: Stand Alone Complex

Monday, March 13, 2006

lab 58 - dried dung

NAME

lab 58 - dried dung

NOTES

"Even as the clock ticks, better and better computer support for the creative spirit is evolving." - Dan Ingalls.

How does computer support evolve and interact with the creative spirit?

Doug Englebart tackled a related question, "How can a computer support a problem solver's intellect?" Englebart's solution was a form of recursion called Bootstrapping. Build tools to help build better tools. The better tools would augment the programmers abilities to help them on their way to bootstrap the next system. A significant aspect of the bootstrapping philosophy was that the researchers used the tools they build.

The Smalltalk researchers took those ideas to heart. The Design Principles of Smalltalk followed this evolution cycle explicitly:

  • Build an application program within the current system (make an observation)
  • Based on that experience, redesign the language (formulate a theory)
  • Build a new system based on the new design (make a prediction that can be tested)

The point I want to emphasize is that both the system improved and the researchers learned during each cycle. This may be why constructionist learning has figured so prominently in Smalltalk's history. The Smalltalk researchers learned how to build systems by building them. But once completed they had a better system to use to start the process of building the next system: the computer support evolved with them.

Alan Kay and Dan Ingalls, original members of the Smalltalk team, went on to create Squeak. A successor to Smalltalk-80, it was built as a system for kids to build and discover their way to better understanding of math and science. Kay has written about the Story of Squeak, which closely follows the bootstrapping philosophy, including using Smalltalk to implement the Squeak system right down to the VM. And still Alan Kay doesn't see Squeak as an end in itself. Squeak is meant to be discarded, a stepping stone for the person using it to the next system that will provide even better support.

To get back to answering my original question, the creative process is a bootstrapping process. We learn by making things. And in the context of programming, there exists a kind of symbiosis between programmer and computer where the computer also learns new processes as we bootstrap new systems from old ones. The created system then becomes the context for the next system, and so it evolves, and with each evolution better supporting the niche of the creative spirit it is serving.

To further illustrate these ideas and reinforce the point that the computer is "learning" too, I point you to a fantastic talk given by Ken Thompson to the ACM, Reflections on Trusting Trust.

Inferno is several generations along the bootstrap timeline. It was bootstrapped from Plan 9, which was bootstrapped from research UNIX, which was bootstrapped from earlier, more primitive UNIX systems way back to UNIX implemented in assembly.

The question then that begs to be asked is, what's next? This is not just a question about the system but also about the person (or people) developing it.

My first steps along this path have been to start thinking about systems rather than individual tools. I've tried to create an Inferno collection of software that's more tightly aligned with the things I need. I want to reduce the overall size but keeping the same power, or even increasing it. I've been trying to consider all trade-offs in complexity versus benefit.

I'm calling this effort Dried Dung (for whatever reason to remind me to think of systems). It's an attempt to bootstrap myself and the system to get better computer support for the creative spirit, hoping both will evolve as I keep pushing on the system.

I feel I could have chosen Squeak, Plan 9, plan9ports, or a lisp environment as a starting point. But I've developed such a comfort level with Inferno I'd find it hard now to switch. I feel I've already begun to mould Inferno to my needs, even as these needs are changing.

This lab's code is a pointer to the Dried Dung file tree. See Getting Started for obtaining a certificate to logon to the tree. Once you get the certificate mount tcp!canto.hopto.org!6681 /n/dung and browse the tree. CHANGES describes some of the modifications I've made but a lot of stuff has been described in this notebook, and will continue to be described here.

FILES

tcp!canto.hopto.org!6681

Thursday, February 23, 2006

lab 57 - createuser

NAME

lab 57 - createuser

NOTES

I've tried setting up some services for the Inferno community. See Getting Started on the Inferno wiki for how to create a user id.

There is a public signer, registry, and kfs. If you mount the registry you might find a couple more.

Use the certificates for your services too. You can create an id, get a certificate and announce some of your own services using the registry then anyone else using the same signer should be able to mount your service.

Acme makes a remarkable collaborative environment. With wiki and irc clients and the shared kfs, a common naming convention for mounted services, it's a wonderful thing to type paths or commands using one or these channels and have others read and execute the commands. I hope these servcies and the Acme environment mean people interested in inferno and connect in new ways.

Most of the services are commands that come with the standard Inferno. What I had to create for this experiment is the createuser file system, which is a single file that accepts writes of username and password pairs. If the write is successful then the new id is created otherwise the write returns an error message.

The command is based on changelogin. I took that code and added a file2chan interface.

FILES

caerwyn.com/lab/57

Local mount tcp!canto.hopto.org!6669 /n/canto
/n/canto/usr/caerwyn/lab

Sunday, February 19, 2006

lab 56 - acme web

NAME

lab 56 - acme web

NOTES

I started using charon again because I wanted a web browser that started faster and used less memory than firefox. I wanted the browser to check my gmail account and scan reddit etc.

But after implementing cut & paste (lab 55) I found it frustrating all over again to not be able to cut & paste from charon. So I started looking for a solution to that. I began by just dumping the text to stdout. Since I was always starting charon from inside acme the stdout would go to an acme window. Then I thought I might as well write directly to a new acme window, so I built an acme client to charon. Then I added the links handling, which was pretty easy, and tada! a charon hack turns into an acme web browser. And a surprisingly good one too, since charon is doing all the work.

The links appear in acme as superscript numbers to the anchor text. Right-click on the number or word to follow the link. Only one link can be handled at a time. You can minimize charon once it's started and browse within acme, falling back to charon when needing to enter form input. But with plumbing turned on in acme, and the right plumbing rules, it's also easy to just type in the URL in acme and right-click on it, which then opens a new acme window on the URL.

There is no forms handling yet. But it should not be too difficult. I'll do an update to this lab, hopefully soon, with some support for forms.

Follow these instructions assuming you are in the /usr/caerwyn/lab/56 directory.

% cp acmewin.m /module
% cp acmewin.b /appl/lib
% {cd /appl/lib; mk acmewin.install}
% cp charon.b chutils.* /appl/charon
% {cd /appl/charon; mk install}

Then inside Acme run the following.

charon -doacme 1

FILES

caerwyn.com/lab/56

lab 55 - cut & paste

NAME

lab 55 - cut & paste

NOTES

this work includes some enhancements for running inferno hosted on windows. these are small details, but do seem to make a difference.

first is cut & paste between host and inferno. I copied the code from drawterm to do this. the file /dev/snarf contains the windows clipboard. typically one would say bind /dev/snarf /chan/snarf and everything works just great.

second is window resizing. in this case the host window is resized but the inferno windows all stay the same. but this still improves it. it's nice to be able to resize the window. however, for this to work the toolbar needs to be moved to the top instead of the bottom of the screen.

finally, i added /dev/fullscreen. writing anything to the file cases inferno to toggle fullscreen mode.

to use the changes copy devarch.c and win.c to /emu/Nt, and devcons.c to /emu/port.

FILES

caerwyn.com/lab/55

Wednesday, February 08, 2006

lab 54 - mux for 4th Edition

NAME

lab 54 - mux for 4th Edition

NOTES

Noah Evans is working on the GBA port. Inferno running on GBA and DS would be cool to see. It's exciting to think about the possibilities of running inferno on small mobile devices, especially if networking is included because that would draw on the strengths of inferno.

Inferno has already been ported to at least one PDA: the ipaq. But as far as I know there are no applications designed for use on a mobile device. Running acme or wm/sh on such a thing isn't worth making the port for. A GUI dedicated to the architecture has to be developed.

Let's use DS as an example, but also imagine mobile phones/PDAs, where we assume Inferno networking is available to the application developer. What applications are useful to run on the DS? (games count as useful) What's the GUI going to look like?

The device's constraints make an interesting challenge. Given a 240x160 resolution and cursor keys and A/B/L/R keys a new GUI and style of interaction different from anything else inside inferno must be developed.

Actually, maybe something suitable does exist. As a starting point to help explore the area I've updated the mux UI, which was intended for interactive TV using an IR remote. It's seems the closest thing that inferno currently has which does not assume mouse and keyboard interaction.

Mux was dropped from 4th edition. It requires the libprefab module in Inferno. To build an emu that includes do the following.

  • add the prefab.$O to the /libinterp/mkfile
  • add prefab to the lib and mod sections in /emu/Host/emu config file
  • recompile

The files for this lab include a mkfs archive that can be extracted on top of an inferno distribution. It's been mostly updated to 4th edition. Not everything works. But the demos show off apps that if scaled down to a smaller resolution would make a cool demo for the DS too.

I've put together a few screenshots.

mux4

Interaction with most of mux is using just a simulated IR remote using the keyboard keys i, m, j, k for up, down, left, right respectively. This is the kind of thing you can imagine operating with just your thumb.

I hope to come back to this, with a scaled down mux running in the smaller resolution, and maybe get the web, email, and a flickr image feed working.

FILES

caerwyn.com/lab/54

Sunday, February 05, 2006

lab 53 - granulator

NAME

lab 53 - granulator

NOTES

I want to revisit the DSP code I worked on in earlier labs (3, 4, 5, 6, 7, 10, 12, and 13). The STK C++ library it's based upon has been updated since then. The latest versions include an implementation of a granulator, which is something I've wanted to do for a while.

Granulation samples the music at tiny intervals then plays the grains back as many overlapping voices. With granulation the music speed can be dramatically slowed down but still retain its pitch. A good example of this is 9 beet stretch a granulation of Beethoven's 9th symphony played over 24 hours. I often listen to it at work. It blocks out surrounding chit-chat, and it isn't as distracting to me as most music. I can't listen to pop or classical music because I get tired of the repetition or else the music might interrupt my train of thought. With granulated music I can setup a playlist of three hours of barely interesting noise and get into "the zone". (At home I tend to listen to Drone Zone from Soma which has much the same effect as granulated Beethoven.)

I want to dump the signalfs code I did last year, and go back to just implementing the DSP library, translating the STK C++ code to limbo. This means I need to re-implement the sequencer module to load Instrument modules that are based directly on the DSP library. I think signalfs didn't work out too well. In the context of this application the whole filesystem structure and converting between real and PCM byte stream didn't seem worthwhile.

In this lab I include the implementation of the granulator. At the moment it's setup to read and play one channel at 22050 rate. It's a start. Here's an example of recording some noise from the microphone, granulating it and playing it back. Granulator takes as argument the stretch factor and the number of voices, as well as the raw PCM file to granulate. The scope just provides some visual feedback to the noise you're making.

% bind -a '#A' /dev
% echo rate 22050 > /dev/audioctl
% echo chans 1 > /dev/audioctl
% scope < /dev/audio > record1
# kill scope when done
% granulate -s 20 -v 16 record1 > record2
% stream record2 > /dev/audio

I'll be working on instruments and a new sequencer slowly over the next 40 years.

If there's an overall goal I have for this it's not to build a sequencer or synthesizer like the many already available. I'd like to build something for the computer to generate music. Implementing the STK is my starting point to learn about computer music. I like projects like 9 beet stretch, or Eigenradio. I'd like a continual stream of Inferno music generated from combinations of environmental sounds, radio, samples, keyboard playing. Because Inferno rocks!

FILES

caerwyn.com/lab/53

Friday, February 03, 2006

lab 52 - text files

NAME

lab 52 - text files

NOTES

Some limbo programs can be replaced with shell one liners, and others can be replaced with more general programs that reduce the limbo line count but increase functionality. I'll look at the few I've discovered.

The /prog filesystem exposes a textual interface that allows existing software tools to work with it. This implies I do not need a custom set of limbo tools to read and write to this filesystem. For example, ps is a 61 line limbo program, but I can do the same thing in one line of shell,

fn ps {sed '' /prog/*/status |sort -n}

This is the great power of Inferno. Simple textual interfaces exposed as files and a small set of software tools working together. Therefore, I can also try rewriting kill and broke. In this case I make the functionality more like Plan 9. Each command writes to output the commands that can be sent back to the shell to actually perform the kill action.

fn kill {
 sed -n '/' ^ $1 ^ '/s/^ +([0-9][0-9]*) +(.*)/echo '^
'kill >\/prog\/\1\/ctl # \2/p' /prog/*/status
}

fn broke {
 sed -n '/broke/s/^ +([0-9][0-9]*) +(.*)/echo '^
'kill >\/prog\/\1\/ctl # \2/p' /prog/*/status
}

This reduces the limbo line count a little more. It also is more suggestive of what can be done when filesystems are implemented with an appreciation of the encompassing system. Take this example from Plan 9 translated for Inferno which shows who is logged into the system:

fn who {
 ps | sed '/Broken/d
 /Exiting/d
 s% +[0-9]+ +[0-9]+ +([a-zA-Z0-9]+) +.*$%\1%' | 
          sort | uniq
}

The /net filesystem also exposes a textual interface. I can implement netstat as a script too (almost, it doesn't include the user name, but i wanted to keep it short).

fn netstat {
  for i in /net/*/[0-9] {
    echo  $i `{cat $i/status $i/local $i/remote}
  }
}

Lookman has bothered me because it could be more general. This is obvious if you're aware of the look command from Plan 9. Why not just port look and wrap it in a shell script to implement lookman?

fn lookman {
 look $* /man/index | sed 's/^.* //' |sort 
          |uniq |sed 's;/man/;;
           s;(.*)/(.*);man \1 \2 # \2(\1);'
}

I've again taken the idea from Plan 9. In my opinion this is better than the Inferno original. It prints the man page reference syntax, which works beautifully with Acme plumbing. I've included the source for the look port in this lab's files.

Another not-general-enough implementation is src. It is supposed to print the source file for a named dis file. The problem here is that the dis file is binary format so it needs a limbo library to extract the source file name. How is this done in Plan 9's src script? It uses adb. Well I don't have that. But I do have mdb. Maybe I should extend it with a few commands to examine a dis file. This would replace wm/rt which doesn't really fit in the system because it can't be used within scripts. So now src can be made a script, and changed slightly

fn src {
 file := $1
 if {ftest -e $file} {
  mdb $file '$s'
 } {ftest -e $file.dis} {
  mdb $file.dis '$s'
 } {ftest -e /dis/$file} {
  mdb /dis/$file '$s'
 } {ftest -e /dis/$file.dis} {
  mdb /dis/$file.dis '$s'
 } 
}

Okay, one more. This is a script to implement calendar but not as feature rich as the UNIX version. The calendar file is a list of multi-line records separated by blank lines, with any line in the record matching the date format 20060203, or just 0203 for recurring appointments, for example. This uses the xp command from lab 50.

fn calendar {
smon:='s/Jan/01/
 s/Feb/02/
 s/Mar/03/
 s/Apr/04/
 s/May/05/
 s/Jun/06/
 s/Jul/07/
 s/Aug/08/
 s/Sep/09/
 s/Oct/10/
 s/Nov/11/
 s/Dec/12/'
 (d m md tim tz yr) := `{date |sed $smon}
 xp ',x/(.+\n)+/ g/('^ $yr ^')?'^ $m ^ 
   $md  ^ '/p' $home/calendar
}

Textual manipulation with pattern-action languages is the hallmark of Plan9 and Inferno. File systems that expose textual interfaces play well with the rest of the system. However, sometimes in Inferno it feels like this isn't exploited or is hidden below the surface. Either because commands are implemented in Limbo when they don't need to be, or because a more general tool could exist based around the command line and not the wm GUI.

I have made the mistake in earlier labs of implementing tools around binary formats, or exposing binary interfaces from filesystems. After looking more closely at the elegance of /prog and /net I'd try to emulate their example.

FILES

caerwyn.com/lab/52

Monday, January 09, 2006

lab 51 - plumbing from firefox

NAME

lab 51 - plumbing from firefox

NOTES

I often want to copy and paste text from the firefox browser into my Acme session. Copy and paste isn't implemented for hosted Inferno. It could be implemented easily enough, but not in a general way for all hosted environments, and there might be more interesting ideas to explore especially when including plumber in the mix.

To solve my particular problem, getting text from the browser to Acme, I created a bookmarklet that opens a window on a URL that I serve from my Inferno session. The URL http://localhost/magic/plumb is a shell script that calls the plumb command to open the query string in Acme.

plumb -d edit -a action showdata 
 -a filename /httpd/plumb $"msg >[2=1]

I have a modified httpd that calls shell scripts and sets environment variables to the cgi data from the query string. In this case msg is a query parameter.

I install a plumb bookmarket (drag it to toolbar), then I can select text in Firefox, click on the plumb and the text pops up in Acme.

I also go the otherway by having a file /services/httpd/root/snarf.txt which I plumb to firefox by right-clicking http://localhost/snarf.txt in Acme.

I'll finish this off with a few other tricks for users running Inferno on Windows. The winstart/winplumb scripts from Plan 9 work for Inferno too. These allow you to plumb messages from Inferno to Plan 9 and have the Windows shell launch programs. I've included both in the files for this lab. See the 9fans archives for more info. I've also included my current plumbing rules that use winstart. But basically, the windows shell usually knows what program handles a file or URL. So just calling winstart http://google.com will open the URL in your default browser. Or winstart file.pdf will open Acrobat reader, for example. The plumbing rules for files local to the inferno namespace then need to be mapped to the Host namespace, which I talk about in an earlier lab.

FILES

caerwyn.com/lab/51

lab 50 - structural expressions

NAME

lab 50 - structural expressions

NOTES

Sometimes I write some code but I'll let it sit for ages because I can't do the lab write-up for it. I think it's probably better to just throw it out there and get on with the next thing. This is one of those labs. Maybe I'll come back and edit it more if I end up using the tool.

I've been reading again Rob Pike's Structural Regular Expressions paper. It suggests variations on existing tools (sort, grep, diff and awk) where applying structural expressions might make the tools more versatile or change their character entirely.

Inferno is missing awk and though it can be run externally I still feel such a tool ought to exist within Inferno. Text manipulation, software tools, filters, regular expressions, pattern-action languages: these make up the core of what I think Inferno should be about. If there's a specific user I have in mind for Inferno it's me, and my main use for it is as a programming environment; one that integrates all the tools I use but implements the core set I need most often.

I've thought about porting awk to Inferno, but the paper suggests an alternative: to consider replacing it with new tools built around the idea of structural expressions. Inferno has an implementation of the sam language described in the paper. This is the Edit builtin command in Acme. The aim of this lab was to extract the Edit command into a stand alone command but to work more like a filter.

The commands implemented are x, y, g, v, p, which behave the same, and a, c, i which don't. Since the file being operated on is not being edited, the edit commands change the text represented by dot, in memory, and print it to standard output. For example, a will append it's argument text to dot and the write to stdout the dot text.

% xp ',x a/ foo/p'

To try out the language I tried to implement the two awk scripts that are used when building emu. The two scripts are mkdevlist and mkdevc. You can compare the implementation with the awk scripts in the inferno distribution /emu/port/(mkdevc mkdevlist). Note that the change in the language semantics means that c behaves like a print command.

This tool is potentially a mistake because the same 'sam' language means something different in the new context. I think it would be better to go further and try and build the awk like language from the paper.

FILES

caerwyn.com/lab/50