Saturday, January 03, 2009

lab 92 - vxinferno


lab 92 - vxinferno


In this lab I create a new Inferno builtin module that calls the vx32 library and get a minimal system working that runs native x86 code, with system calls redirected to inferno's system calls and therefore making the inferno namespace visible to the sandboxed code.

Vx32 is a new user-level sandboxing library by Bryan Ford and Russ Cox. From the vx32 paper,

"Vx32 is a multipurpose user-level sandbox that enables any application to load and safely execute one or more guest plug-ins, confining each guest to a system call API controlled by the host application and to a restricted memory region within the host’s address space."

Inferno, being a virtual operating system, provides its own system call API to limbo applications. The same system calls are available as a C API for use by native libraries that appear as builtin modules or devices within the inferno environment. This C API is a natural fit for building a Vx32 sandbox allowing native code of all kinds to run within inferno, which has considerable flexibility over the namespace made available to the native code. This would allow inferno to be extended in new ways beyond limbo, builtins, or external processes that export styx protocol. It also extends the reach of native code that when run in this hosted environment its use of files take on even greater significance.

Please read the vx32 paper, download the code and play with it. I haven't included the vx32 code in the lab. Instead this lab is more tutorial in creating a new builtin module for inferno. This labs code, linked to in the steps below, is all the code necessary to make vx32 appear as a builtin. I've done enough to show some simple examples working, but I haven't defined the full system call interface.

So here are the steps in creating a new builtin module linkage.

module interface

Create the limbo module interface, e.g. /module/vxrun.m. I created the interface to closely resemble the vxrun application in the vx32 distribution. The module contains one function to load and run a native ELF executable.

Edit /module/runt.m to include new include the new module interface. This file includes all builtin modules and is used later to generate a runtime C struct.

incorporate library code

Copy library and header files into inferno-os tree. I copied vx32.h to /include/vx32.h. I created a new libvx32 folder at the root of the tree and created a dummy mkfile. I didn't copy all the source into the tree, I cheated and just copied libvx32.a to /Linux/386/lib. But the emu build will expect the folder and mkfile to exist. So this is a placeholder for now.

add builtin to libinterp

Implement the builtin linkage /libinterp/vxrun.c This is the bulk of the work, where we call the vx32 API and map the system calls defined in the codelet C library that comes with the vx32 distribution, libvxc, to inferno's API defined in /include/kernel.h.

The template can be generated by /dis/limbo from the vxrun.m interface

% limbo -T Vxrun /module/vxrun.m
#include <lib9.h>
#include <isa.h>
#include <interp.h>
#include "Vxrunmod.h"

 builtinmod("$Vxrun", Vxrunmodtab);

Vxrun_run(void *fp)
 F_Vxrun_run *f = fp;

Which is actually wrong, it should be,

 builtinmod("$Vxrun", Vxrunmodtab, Vxrunmodlen);

The hard work is the filling in the Vxrun_run() function. A lot of this code was taken from vx32/src/vxrun/vxrun.c.

The main work of running the native code and redirecting syscalls is in the following loop,

 for (;;) {
  int rc = vxproc_run(p);
  if (rc < 0){
   *f->ret = -4;
  if (rc == VXTRAP_SYSCALL) {
   if(dosyscall(p, f->ret))

In dosyscall we switch on the syscall number and call the Inferno method, e.g.,

static int 
dosyscall(vxproc *proc, int* fret)
 switch (NUM) {
  fd = ARG1;
  addr = ARG2;
  len = ARG3;
  if (!vxmem_checkperm(proc->mem, addr, len, VXPERM_WRITE, NULL))
   print("bad arguments to read");
  ret = kread(fd, (char*)m->base + addr, len);
... other syscalls
 RET = ret;
 return 1;

To get this to build we need to edit the /libinterp/mkfile. Add vxrun.$O to the list of OFILES, add vxrun.m to the list of MODULES, and add the following rules to ensure the module header, vxrunmod.h, is generated.

vxrunmod.h:D: $MODULES
 rm -f $target && limbo -t Vxrun -I../module ../module/runt.m > $target
vxrun.$O: vxrunmod.h

We can now compile libinterp.

edit emu config

The final step is to edit /emu/Linux/emu configuration file and add the dependencies on the vxrun module and the vx32 library. We can now build a new emu that has the vx32 vxrun as a builtin module.


We need a limbo command to call the module. I included vxinferno.b in the lab code. But it does nothing more than load the module and call it passing in any command line arguments.

init(nil:ref Draw->Context, args:list of string)
 vxrun := load Vxrun Vxrun->PATH;
 vxrun->run(tl args);

I used the vx32-gcc to compile the native code. I included one example, cat.c, that would test the system calls, open, read, write, from the inferno namespace. Note that the name of the executable to call from inside Inferno is the host pathname, because vx32 itself is not using the Inferno system calls. This could be fixed by either changing the elf loader, or by using the library call to load the ELF from memory.

$ cd ~/vx32/src/vxrun
$ vxrungcc cat.c 
$ emu -s -r ~/inferno-os
; vxinferno /home/caerwyn/vx32/src/vxrun/_a.out /dev/drivers
#/ root
#c cons
#e env
#M mnt


This lab confirmed vx32 as a builtin to inferno would work. Now it needs to be implemented in full. I'd most like to see vx32 ported to windows. There is an effort to port vx32 to windows, but it seems to have stalled.

I think this is really cool!




rog peppe said...

hey, good effort caerwyn! i was planning to do something like this, if i'd had any time. native performance for things like gzip without having to resort to os(1) will be so useful...

i guess the next-but-one step from this is to get communication with limbo code in more depth - e.g. by allowing (value-only) channel communication. how can we get code like this talking with the window system, for example. maybe it doesn't need to!

Anonymous said...

I think you can use Debug API to make the same on Windows.

caerwyn said...

Compression libraries are a great place to start. The VXA code included in vx32 dist includes decompression for bz2, zlib, vorbis, flac, and jpeg. I think this code should run inside vxinferno even with the few system calls I've implemented. This would make jukebox inside inferno doable using /dev/audio.

jas said...

As for a Windows effort, it may be easiest to start something based on NaCl--at least that project is going strong and already supported on Linux|OS X|Windows.

forsyth said...

i've changed the limbo compilers to generate the correct code (I think). it also fixes the incorrect sequence suggested for destruction of pointers.
destroy(*f->ret); *f->ret = H;
is incorrect in case of exceptions. it should be
void *r;
r = *f->ret;
*f->ret = H;

Powerman said...

You forgot to mention one more fix for generated stub: function Vxrunmodinit should be named vxrunmodinit.