The current state of operating systems is very painful. They are fundamentally flawed in multiple ways. In this multi-part series I will talk about what I think will be the next generation operating system.

With all the noise about security, why hasn’t operating system level sandboxing been done? I mean, its been done, but never done properly.

An application running in a sandbox only has permission to whatever security policy it has been given. Thus, it is possible to run arbitrary code with confidence given the right security policy.

Arbitrary code?? Yes, arbitrary code. The sandbox I have in mind will allow me to download and run any virus, keylogger, or malware.

How can sandboxing achieve this? It’s all about the default security policy. A new program will be given the minimal amount of permission by default. The default set of permission would be:

1. Run code
2. Take keyboard/mouse input
3. Manipulate user interface
4. Save/read files

“Wait a second, if by default programs have access to keyboard input, can’t it be a keylogger?” Nope. This permission only allows keyboard input to the application window. It doesn’t give permission to get keyboard input when the application is not in focus. “Wait again, if the program can save/read files, can’t it destroy all my data and phish for personal information?” Nope. The saving and reading of files will use a standard file chooser. The program doesn’t have access to any files at all without the user pointing to it.

Given those permissions, it is impossible to author malicious programs, expect for showing an offensive image. So the worst you have to worry about it those “scary popup images”. Quite the difference from complete compromization of security.

But what programs can you run under those permissions? Well, to be honest, pretty much nothing. You could run Notepad, but that’s about it. To allow for the majority of non-Internet applications to run, we need a few more permissions, namely:

1. Create threads
2. Read files in local directory tree

Now we can create powerful applications like Photoshop and Microsoft Office. If given permission to access the Internet (I mean selectively to applications - not adding it to the default permissions list), then we get the rest of the Internet applications like Firefox, World of Warcraft, etc. Given these minimal set of permissions, we can do the majority of what we use the computer for. And remember, all this is achieved without compromising security. The power of sandboxing.

More complex applications can be selectively given more permissions. The method in which extra permissions are given to an applications is a little tricky. The application will have to ask the operating system for permission, and in turn the operating system will ask the user. As much as I hate Vista’s User Access Control, that is exactly how you would give extra permissions. Of course it wouldn’t be broken like Vista’s. I imagine a standard dialog which lists the extra permissions the application is asking for. The user can then check off which permissions are allowed or denied. This dialog can even be integrated into the application. The API to this dialog simply doesn’t give access to check off permissions programmatically. Since there is no permission to move and control the mouse, the application can never give it self extra permissions.

This is so simple, so powerful, and yet there is no major operating system that does this. Well, there is a reason for that. Current operating systems cannot just turn on sandboxing, or else it would break all the applications in existence. Also, sandboxing needs to be done at the kernel level, it can’t just be patch on top of the operating system. It can be hard to imagine how sandboxing would work, since current operating systems allow you to do so much by default.

So where does this leave us? What we need is a new operating system from the ground up designed around sandboxing. This operating system will be immune from malicious programs. But of course it doesn’t stop the user from being stupid by allowing all permissions to a virus and running it.  However, this is far more difficult that running any old .exe.