Sunday, June 28, 2009

lab 103 - python content assist

NAME

lab 103 - python content assist

NOTES

Building on lab 102 I've started to write an acme client for python content-assist in python. I read the code for python's IDLE editor and pulled some pieces out of that to help with my implementation. The AutoComplete.py and CallTip.py modules are copied out of idlelib and reduced to just the functions I needed.

Just as in lab 102 the acme client is implemented using pyxp to connect to the acme file system.

In an acme editor window execute PyAssist. Start typing some python. After typing a dot the client will display available attributes for the module. Continue typing and it will reduce the list to those names with a matching prefix. Press the Tab key to complete a name or prefix depending on whether there is one or more names.

109528114135-Acme-SAC

After typing an open parenthesis the client will show the call tip for the function.

109528114325-Acme-SAC

The client implements an Import command so that more python modules can be added to the namespace. PyAssist doesn't attempt to parse the edited script. Idlelib includes code that could help to do this. But for now you need to pass a module name as argument to the Import command.

A next step for python integration to acme is to implement a debugger. Python comes with libraries for this too, so a debugger similar to the acme debugger adeb should be a future lab. I might then run edited scripts in a sub shell and inspect that process' namespace similar to the IDLE implementation, so that content-assist adapts to the script being edited.

FILES

inferno-lab/103

Tuesday, June 23, 2009

lab 102 - python acme client

NAME

lab 102 - python acme client

NOTES

A recent post to 9phackers announced Pyxp, another implementation of Styx in Python.

I immediately downloaded Pyxp and tried it out. I had no trouble using it so I started thinking about python clients I could write. Python is still new to me so writing a styx client was an excuse to get more practice.

I started with an acme client. I translated the acmewin limbo module I use for most acme-sac clients to python. Below is a simple example opening a new acme window and doing some operations on it.

from acmewin import Acmewin

win = Acmewin()
win.writebody("hello, world!\n\n\n")
win.tagwrite("Hello")
win.writebody("goodbye")
win.replace("/goodbye/", "GOODBYE")
win.select(",")
win.show()

Remember to export the namespace before trying it out.

% styxlisten -A 'tcp!*!localhost' export /
% textclient.py

I recently saw on Hacker News a repost of Peter Norvig's spelling corrector. I thought this would make an easy first trial of my python acmewin code. I implemented a client to offer spelling suggestions to an editor window. It works somewhat like my earlier acme content assist code. This client opens the event file of another window it is assisting and writes text out to its own window. In this case it offers a suggested spelling for the word currently being typed.

109523203021-Acme-SAC

Here's the implementation. Note that this is single threaded and it is not reading the event file of the second window. I haven't gotten that far in the Python book.

#!/dis/python26

import sys
from acmewin import Acmewin
import spell

win = Acmewin(int(sys.argv[1]))
outwin = Acmewin()

while True:
    (c1, c2, q0, q2, flag, nr, r) = win.getevent()
    if c2 in "xX":
        if flag & 2:
            win.getevent()
        if flag & 8:
            win.getevent()
            win.getevent()
        win.writeevent(c1, c2, q0, q2)
        if c2 == "x" and r == "Del":
            outwin.delete()
            break
    if c1 == "K" and c2 == "I":
        ch = r[0]
        if ch in " \t\r\n":
            outwin.replace(",", "")
            continue
        while q0 >= 0 and not (ch in " \t\r\n"):
            sss = win.read(q0, q0+1)
            if not sss:
                # print("empty sss %d" % q0)
                sss = " "
            ch = sss[0]
            q0 -= 1
        if q0 < 0 and not(ch in " \t\r\n"):
            q0 = 0
        else:
            q0 += 2
        ss = win.read(q0,q2)
        lastcorrect = spell.correct(ss)
        outwin.replace(",", lastcorrect)

To run this we need to know the id of the window we are assisting, so we need a wrapper to send the $acmewin environment variable as an arg to the script. For that I have a script called SpellAssist.

#!/dis/sh
$home/python/assist.py $acmewin

Now that I have a simple assist-like client working I'd like to develop it further. I'd like to try having content assist for python inside acme. It should be possible to adapt the python code that implements the IDLE editor to this purpose.

FILES

inferno-lab/102
big.txt the large text file used to train the spelling corrector. Note that the path is hardcoded in spell.py and should be changed locally.

Monday, June 15, 2009

lab 101 - limbo B+ tree

NAME

lab 101 - limbo B+ tree

NOTES

This lab is an implementation of a B+ tree. It is code from a much older lab that was overly complicated and buggy. I pulled the B+ tree code out, tried to fix the bugs and clean it up. The interface is roughly the same as dbm(2). Here is a synopsis.

include "btree.m";
btreem := load Btreem Btreem->PATH;
Datum, Btree: import Btreem;

Btree: adt {
 create: fn(file: string, perm: int): ref Btree;
 open: fn(file: string, flags: int): ref Btree;
 
 fetch: fn(b: self ref Btree, key: Datum): Datum;
 delete: fn(b: self ref Btree, key: Datum): int;
 store: fn(b: self ref Btree, key: Datum, val: Datum):int;

 firstkey: fn(b: self ref Btree): Datum;
 nextkey: fn(b: self ref Btree, key: Datum): Datum;

 flush: fn(b: self ref Btree);
 close: fn(b: self ref Btree);
};

init: fn();

Like Dbm the keys and values are stored as arrays of bytes, and being a B+tree the values are stored only in the leaf nodes. The maximum key and val size is 255 each. The block size is 8192, so the maximum leaf node size is 515 making the minimum branching factor 15. The maximum number of records in an internal nodes assuming an int as key is 630.

The delete is not fully implemented; it doesn't merge nodes.

Here is some example code listing the full contents of a btree.

sys := load Sys Sys->PATH;
btreem := load Btreem Btreem->PATH;
Datum, Btree: import btreem;

btreem->init();
bt := Btree.open("index.bt", Sys->ORDWR);
for(key := bt.firstkey(); key != nil; key = bt.nextkey(key)){
 v := bt.fetch(key);
 sys->print("%s %s\n", string key, string v);
}

Here is a rough comparison of btree vs. dbm. This simple test is more of a sanity check that the btree doesn't do anything horribly wrong in its implementation (which it did originally, by making a syscall to get the daytime for every block it tried to get).

% awk '{print $1, NR}' < /lib/words | tr -d 'cr' > t1

% >index.bt
% >dbm.pag
% >dbm.dir

% time sh -c 'btree/store < t1' 
0l 4.172r 4.172t

% time sh -c 'dbm/store dbm < t1'
0l 35.25r 35.25t

% ls -l dbm.* index.bt
--rw-rw---- M 6 xcs0998 XCS0998    8192 Jun 04 12:38 dbm.dir
--rw-rw---- M 6 xcs0998 XCS0998 1048576 Jun 04 12:38 dbm.pag
--rw-rw---- M 6 xcs0998 XCS0998 770048 Jun 04 12:36 index.bt

% time sh -c 'btree/list > /dev/null'
0l 5.187r 5.187t

% time sh -c 'dbm/list dbm > /dev/null'
0l 9.438r 9.438t

FILES

inferno-lab/101

Sunday, June 07, 2009

lab 100 - limbo tags

NAME

lab 100 - limbo tags

NOTES

I've been playing more with exuberant ctags. It's possible to make ctags recognize limbo source code using regular expressions to identify symbols we want tagged.

Here's a command I defined called ltags. It will search for functions, assuming the function name is not preceded by spaces, and, more reliably, find adt and module definitions:

#!/dis/sh
os -t ( /n/D/ctags57/ctags.exe  -n 
 '--tag-relative=yes'
 '--langdef=limbo' 
 '--langmap=limbo:.b.m' 
 '--regex-limbo=/^([a-zA-Z][a-zA-Z0-9]+)\(.*\)/\1/m/' 
 '--regex-limbo=/([a-zA-Z][a-zA-Z0-9]+) *: *adt */\1/t/'  
 '--regex-limbo=/([a-zA-Z][a-zA-Z0-9]+) *: *module */\1/c/'  
 $* ) 

Try running it under /, and use relative path names:

 
% cd /
% ltags -R appl module

And open the acme client

Ctag /tags

10957142117-Acme-SAC

I added a command "Tag" to the Ctag program that will find and print all matching symbols to the name given as argument. Mouse-chording makes this nice to use. Right-clicking on symbol names will open the source file at that location. Browsing the whole limbo source hierarchy is then made quite easy.

FILES

inferno-lab/100