Microsoft Singularity Now "Open" Source

Alex_Ionescu writes "Microsoft's Singularity operating system (covered previously by Slashdot) is now open to the public for download, under a typical Microsoft academic, non-commercial license. Inside is a fully compilable and bootable version of what could be the basis for the future of Windows, or maybe simply an experiment to demonstrate .NET's capabilities. Singularity, if you'll recall, has gained wide interest from researchers and users alike, by claiming to be a fully managed code kernel (with managed code drivers and applications as well), something that would finally revolutionize the operating system research arena. The project is available on CodePlex."

Software Isolated Processes

[parvenu74]

Singularity, if you'll recall, has gained wide interest from researchers and users alike, by claiming to be a fully managed code kernel (with managed code drivers and applications as well), something that would finally revolutionize the operating system research arena.

The impression I got by looking at what was known about the project a year ago is that it was of lesser interest that the OS was written in managed code and it was far more interesting that they had solved some problems of inter-process communication in a micro-kernel OS. As you can read at Wikipedia [wikipedia.org]:

Singularity is a microkernel operating system; however, unlike most historical microkernels, the different components do not run in separate address spaces (processes). Instead, there is only a single address space in which "Software-Isolated Processes" (SIP) reside. Each SIP has its own data and code layout, and is independent from other SIPs. These SIPs behave like normal processes, but do not require the overhead penalty of task-switches. Protection in this system is provided by a set of invariants, such as the memory-invariant which states there will be no cross-references (or memory pointers) between two SIPs. Communication between SIPs occur via higher order communication channels managed by the operating system. These rules are checked during the installation phase of the application, and must be fulfilled in order for Singularity to allow the installation (note: in Singularity, installation is managed by the operating system).

The promise of Singularity, as I understood it, was the possibility of constructing an O/S kernel with all of the modularity advantages of a microkernel without all of the process communication issues typical to this kernel type.

Re:Stability?

[Anonymous Coward]

There's a list of the applications here [codeplex.com]

No IE, but it has Pong!

Oh wow!

[AKAImBatman]

Managed code! Look at that! Microsoft has managed to prove...

What OSS developers already proved [jnode.org] years ago. :-/

Actually, I'm still pretty happy about this. Regardless of whether Microsoft was first or not, they're going to manage to market the concept far better than a conglomeration of OSS developers ever could. (Sorry, guys!) If everything goes well, perhaps the public impression of managed code being "nothing but an interpreter" can finally get turned around and Computer Science can keep moving forward. :-)

Re:Stability?

[wiggles]

Ars Technica says it all. [arstechnica.com]

This OS doesn't really run any applications at all. It's not intended for commercial use, and will not be the next Windows. All it is, is a test bed for future technologies. Think of it as an IT equivalent of a concept car. It doesn't really run, but it's nifty to look at to get ideas for future projects.

Re:Stability?

[parvenu74]

MSFT never intends to turn Singularity into a marketable product. It's simply a RESEARCH project, a breeding and proving ground for advanced O/S concepts. If they learn valuable things from the project -- like SIP, for example -- those ideas might find their way into the Windows code base in the future.

the open source definition, right here:

[erlehmann]

http://opensource.org/docs/osd [opensource.org]

also, STFU when you clearly have no clue.

there is a thing called the open source definition

[erlehmann]

http://opensource.org/docs/osd [opensource.org]

also, repost is repost [slashdot.org]

Re:!free

[peragrin]

your right. Now compare Minix and Linux. One has a license for you too look at the source code and the other one allows you to actually use the source code and ideas in it.

It's not Open Source until you can use it. BSD, MIT, Apache, GPL, allow you to actually use the code.

Re:Why are people excited about this?

[garett_spencley]

I'm not an expert on the subject, and everything I know I read just this morning (I hadn't even heard of Singularity before this slashdot article), but it appears as if everything runs inside of a SIP (software isolation process) which runs in ring 0 of the kernel's address space. Thus the creation of SIPs is extremely cheap, even less overhead than hardware enforced protection domains.

You're right, this will not eliminate bugs. But it will prevent applications from "stepping on each other's toes". SIPs can not modify their own code or write to other SIP's address space. I don't see this as so much of an abstraction layer as just a different way for the kernel to manage processes and address space.

Besides, every kernel implements abstraction layers anyway. Heck, you could even consider the kernel to be one big abstraction layer to the hardware. So abstraction layer does not always equate to "more overhead". And this isn't an abstraction layer on top of an existing high-level system. This is an entirely new kernel that implements processes and memory management in a completely different way.

NOT Open Source

[John Hasler]

This is not open source. It's just another "you can play with it but don't you dare do anything real" license.

Re:!free

[zotz]

"open source = source code is made available"

http://www.opensource.org/ [opensource.org]

They may have coinded tghe term, they certainly promoted it and made it polular. They disagree.

all the best,

drew

http://zotzbro.blogspot.com/ [blogspot.com]

technically not open source

[aepervius]

From the ars technica link below :

QUOTE:"Although the Singularity research development kit (RDK) is available for download, it is not technically open source. The source code is distributed under the terms of the restrictive Microsoft Research License rather than one of Microsoft's two OSI-approved open source licenses."

ars technica [arstechnica.com]

To be "open source" you need a tad little bit more than having the source readable in plain text, IMHO.

Re:It's open source because...

[Ctrl-Z]

Have you ever seen an obfuscator? Run your code through one of those and see how easily reversible it is.

Re:!free

[alan_dershowitz]

Why should anyone spend any time learning and working with this tool if their efforts cannot be used commercially?

Two reasons: Because it is, allegedly, a highly modern kernel design that (I've read) implements a next-generation security model that is conceptually too different to be bolted on conventional modular monolithic kernels. With an academic, noncommercial license you can use it to to learn about kernels. If you're not interested in or learning about kernels, only potentially using them, then yeah, I concede your point. However, secondly, an academic noncommercial license to the source doesn't preclude Microsoft selling an OS based on that kernel commercially, in which case having the source does have practical value for programmers even if it cannot be modified.

Re:Software Isolated Processes

[smallfries]

It sounds like a very interesting project. The idea that screams out from the wiki summary is static analysis and verification. There is a really good rundown in one of the wiki links [microsoft.com]. The really big difference from previous work is not just the use of managed code, but splitting the entire system into either trusted, or verified code. The trusted component is a tiny core, which they are working on verifying. The design of the rest of the kernel and the SIPs is a good one: instead of doing arbitrary verification, change the language design so that you can only write verifiable code. Then see how much of an O/S you can write. The progress is astounding.

For the IPC they have made some strange choices, receiving is synchronous (as in process calculi) but sending is asynchronous. As they are writing the lowest level parts (such as the schedular) in this code it may be an implementation difficulty with synchronous sends. The cheapness of the IPC seems to be routed in the transfer of ownership that communication implies. In essence you can't alias, you can only pass by value - but the low-level runtime can modify this to pass more efficiently by reference because it can verify there are no dangling references. This would (if it works over a large enough code base) solve the performance issue with IPC in a microkernel. It is (as another reply pointed out) similar to providing the semantics of heavy-weight communication to the programmer in a way that can be implemented with cheap co-routines.

Having done some (well, little) work in this area I'm really impressed by what they've achieved already.

Re:!free

[Timothy Brownawell]

My understanding is that the security model is actually pretty old, and has been around since at least 1979 [upenn.edu].

Re:Software Isolated Processes

[benjymouse]

SIPs are actually already represented in .NET where they are called AppDomains. It is leveraged in both desktop and ASP.NET applications. In the latter an application pool is a pool of processes+threads shared by web applications. Multiple applications can share a process and its threads. When the server handles a request is does so in the context of the AddDomain. So, even though different sites/apps share process (and thus memory space) the managed .NET execution environment ensures the isolation (which is why pointers are disallowed).

"Open Source" has a standard definition (OSI's)

[dwheeler]

The normal meaning of "open source software" is the one given in the Open Source Definition: http://opensource.org/docs/osd [opensource.org] This is the normal, common meaning of the term; a Google search on the term quickly proves this. Therefore, Singularity is not open source software; please don't confuse people, and submit to the Microsoft marketing engine, by incorrectly saying otherwise.

Instead, use another term when you mean "you can read the source code". I suggest "source available".

Re:!free

[rapierian]

I would point out that there are a couple other microkernels out there that have reached that point. The main one I'm familiar with is BeOS, which is currently being reborn in Haiku.

No spin required: it's kernel Erlang

[Morgaine]

Singularity also introduces a software-isolated process (SIP) mechanism, which makes it possible for lightweight processes to coexist safely with their own sealed code environments in the same memory space. All communication between SIPs is conducted through channels. The use of SIPs effectively eliminates the overhead traditionally incurred by context-switching in conventional microkernels.

In other words, Microsoft finally discovers Erlang [erlang.org].

While I wouldn't go so far as to say that the guys at Redmond lost the habit of inventing anything new a long time ago, the above concepts have been in industrial use in Erlang-powered PTT exchanges since the dawn of time.

Re:NOT Open Source

[HermMunster]

No one said they were. But as the OSI definition is the longest standing and it is a definition that really has ever been challenged except by proponents of close software, the attempt to redefine results in confusion as the the real meaning of open source and foss and the GPL. It is like working away at the chink in the armor. Sooner or later it'll bust unless you have knowledgeable folks repairing it.

OSI is not some random organization that popped up and created a website. Proponents of open source are not fascists. There's no religious ferver here. The individuals are simply protecting their homes. This is where they live when it comes to their community spirit. Soon you'll be redefining their definitions of terroristic toward Microsoft. It is insanely stupid to do so but once you attack and the open source folks defend sooner or later the battle will get much more heated and we'll begin to see terms like terrorism used in software because one party wants to ensure that their homeland is safe.

This is not an open source project and doesn't meet the established definition, one which has been long standing for years and has not been challenged except by a company that has stated they are hell bent on destroying "open source" and is a convicted monopolist. We aren't going to get a court ruling on the term "open source" and those that established it are the ones to define it. Just as I write the book I have the right to name it. BTW, did the court give Websters the right to define words? Or Blacks? I think it is that these were the first entrants and they have been accepted for years. They didn't form some world wide standards organization to create their dictionaries (legal the Blacks law dictionary, or Websters for the English language).

You can't redefine it because you disagree with the meaning given to it by those that invented it.

Re:Why are people excited about this?

[shutdown -p now]

With verifiable managed code (i.e. the one that doesn't use pointers and such), it is possible to statically prove that it will never access the address space of another process. Once you've done that, you don't need to isolate such processes from each other. This property is already used in .NET with something that MS has called "AppDomains", which allow you to isolate different parts of a single managed process from each other. I would imagine that it can also be useful in a kernel.

Re:It's open source because...

[clintp]

Is there a difference between "managed code" and "interpreted code"? They seem like two words for the same thing.

IANA Microsoft Language Lawyer, but this is what I think the distinctions are:

Managed Code is code intended for a virtual machine (like MS's CLR or Sun's JVM) that abstracts the hardware instructions away. Instead, the instruction set for the virtual machine is used. The Virtual Machine will provide "devices" and "memory" in a (hopefully) safe and portable way and take care of all of the dirty hardware business itself. Some VM's will actually take the VM instruction and turn it into actual hardware instructions as it's being executed (JIT) for speed, but that's not necessary.

Which isn't to say that Managed Code is a new thing: The USCD-Pascal p-code machine is remembered fondly by many, and the Zork games ran on a Z-Machine.

Interpreted code is a little stickier because it's been around a lot longer and has picked up some additional meanings. It can mean anything from the "Managed Code" described above to parsing (and possibly re-parsing) text lines of BASIC as they're run to process them in a giant state machine which "runs" the program.

Usually, interpreted code implies that there's no abstracted fully virtual machine underneath running the code, but possibly just a big jump-table pointing at native assembly-language (hand-coded or compiled) routines. Perl and Microsoft BASIC (basis of many of the old 8-bit BASICs) are two examples of interpreted code.

Re:Oh wow!

[VGPowerlord]

JNode's own site mentions Singularity [jnode.org] two and a quarter years ago. Singularity is not new.

Re:It's a nice system. Is this abandonment?

[ralphbecket]

Singularity runs as a collection of Software Isolated Processes (SIPs) that (normally) run in a single address space. Each SIP appears to the kernel as a thread. SIPs can have multiple threads. SIPs can provide their own memory management and thread management. SIPs communicate through a shared-memory message passing interface where precisely one SIP has ownership of a shared memory block at any given time.