Tuesday, March 31, 2015

Bare metal application builders are starting to flex their muscle.

As an application developer, security concerns me greatly. Every day there is a fresh announcement of a major hacking incident, often with the user database being compromised. I'd like to be able to develop applications that have some hope of resisting hackers.

One of the most promising ways to reduce the attack surface is to build applications that run on the bare metal of the CPU without a host operating system. More accurately, bare metal applications run directly on a virtualisation hypervisor rather than on the bare metal of a CPU a la MS-DOS.  The idea being that if there is no system to log in to then it's much harder for hackers to gain access. What can a hacker do if there's just nothing at all to log in to?

Running on bare metal is being made possible in large part by virtualisation technologies such as Xen which provide standard virtual networking and file system interfaces. These virtualised interfaces mean that bare metal solutions don't need hardware device driver support, making the core concept much easier to implement.

I'm super keen to be writing my applications to run on bare metal. There's quite a bit going on in this field but its early days; there's currently no practical way to write an application using mainstream programming languages and get it to run as bare metal. That's likely to change over the next couple of years. It is possible to build bare metal applications using  Ocaml, Haskell and Erlang.

I'd like to build bare metal applications using Python, Rust or Go. That's not possible right now. Here are the projects that I know about in the bare metal space:

MirageOS describes itself as "library operating system that constructs unikernels". The project appears to be relatively mature and actively developed. MirageOS requires that your code is written in OCaml.

Quoting from the website, HaLVM "enables developers to write high-level, lightweight virtual machines that can run directly on the Xen hypervisor". It is being developed by Galois, a U.S. company based in Portland, Oregon.  An overview presentation is here. You'll need to write your code in Haskell.

Ling allows Erlang applications to be run directly on the Xen hypervisor. It is being developed by Cloudozer, a startup company based out of Russia.

OSv appears to be a version of FreeBSD so severely stripped down that although applications still have the resources they need to run, there is little or no remaining recognisable operating system, for example there is no concept of users in OSv, and there are no drivers other than those required to run on a Xen hypervisor. OSv allows execution of JVM and Posix applications. OSv is developed by Cloudius Systems of Israel. A presentation is here.

Rump kernels
It appears that Rump kernels utilise NetBSD's user space device drivers to compile Posix applications into unikernels. Rumprun is an active project facilitating the build process. A recent tweet announced that MySQL has been built as a unikernel using Rumprun. Rumprun is described as "a wrapper for running programs that were written for a normal POSIX (NetBSD) system to run them under a rump kernel."

Determined not to leave the fun to the open source world, Microsoft Research has been doing some work on the library OS and its Drawbridge research project is described as "a research prototype of a new form of virtualization for application sandboxing. Drawbridge combines two core technologies: First, a picoprocess, which is a process-based isolation container with a minimal kernel API surface. Second, a library OS, which is a version of Windows enlightened to run efficiently within a picoprocess." Whether anything living will creep out of the bubbling green tubes of the research lab remains to be seen.

On the horizon.
The Cloudozer website mentions "Rust On Xen", "Go on Xen" and "J on Xen", exciting concepts but I couldn't find any further information. A recent post from Hajime Tazaki introduces a new project called LibOS, saying: "Our objective is to build the kernel network stack as a shared library that can be linked to by userspace programs to provide network stack personalization and testing facilities, and allow researchers to more easily simulate complex network topologies of linux routers/hosts." There's clearly more innovation and development in the works in the exciting field of bare metal application development. If you know of anything new, please let me know at andrew.stuart@supercoders.com.au

Hopefully it won't be too long until we see production quality ways to build bare metal applications using mainstream programming languages.

Monday, March 30, 2015

The continents of office and citizen computing are pulling apart.

Millions of years ago the earth had just one giant unified continent named Pangea. Over time, tectonic forces have pulled this continent apart.

For decades the worlds of office and citizen computing were unified. Visit just about anyone's house and you would see the citizen's preferred home computing environment looked very much like the office computing environment.

When most people go to work at an office, they sit down at a desktop computer for the day and that's how they get they job done. I'm using "desktop computer" loosely here - many office workers actually use laptops but the experience is effectively the same - it's a computer with a physical keyboard running Windows or OSX - general purpose operating systems.

Outside of the office world, people have moved strongly toward mobile devices - phones and tablets.

Seems to me no-one has given much thought to this growing difference between the preferred computing environment of the sit-down office world and the preferred computing environment of everyone else.

Will physical keyboards and general purpose operating systems always be the preferred computing environment in offices?

Are there unrealised opportunities to be found in the commitment of offices to desktop machines?

The continents of office and citizen computing are pulling apart. That change means opportunity for those who can see the new world through different eyes.

Thursday, March 26, 2015