'Midori' concepts materialize in .NET

'Midori' concepts materialize in .NET,

Some of Microsoft’s latest technologies could be green shoots on a migration toward its "Midori" operating system, according to analysts who are familiar with the project.

Recent additions to the .NET Framework adhere to the concurrent programming principles outlined in the Midori documents that SD Times viewed in 2008. Silverlight and the Windows Azure platform could also be complementary to a potential release of Midori, the analysts said.

Midori is a technology incubation project that was born out of Microsoft Research’s (MSR) Singularity operating system, the tools and libraries of which are completely managed code.

Microsoft has designed Midori to be Internet-centric with an emphasis on distributed concurrent systems. It also introduces a new security model that sandboxes applications.

"Midori is an attempt to create a new foundation for the operating system that runs ‘inside the box,’ on the desktop and in the rack. As such, it's willing to break with compatibility (or at least wall off compatibility to a virtual machine)," explained Larry O’Brien, a private consultant and author of the "Windows & .NET Watch" column for SD Times.

Microsoft may be laying a foundation for Midori in its existing development stack through languages and Silverlight as a runtime, O’Brien said. Microsoft Research is also increasingly focused on reasoning about concurrent programs, he added.

These major architectural transitions require developers to make a “conceptual leap” to a new model of programming, and to relearn how to program in an efficient manner, said Forrester Research principal analyst Jeffrey Hammond.

"We're seeing a gulf opening up right now between serial and parallel programming; only a small minority of rocket-scientist types can actually write code that works effectively in a parallel, multicore world,” Hammond added. “I think it's pretty clear that Midori is on the other side of that scale-out gulf. From a development point of view, those that can make the leap solidify their skills and employment opportunities for the next decade and beyond."

When asked whether there were any new developments in the Midori project, a Microsoft spokesperson said, "Microsoft is always thinking about and exploring innovative ways for people to use technology. Midori is one of many incubation projects under way at Microsoft."

Green shoots

Microsoft's F# programming language, which will ship this month with Visual Studio 2010, "hugely fits" the Midori programming model that was outlined in Microsoft’s documents, O’Brien said. F# is designed with restrictions that are intended to make it easier for developers to automatically parallelize applications, he explained.

For instance, F# is highly immutable—meaning that object states cannot be modified once created—and has an implicit type system. Midori requires developers to follow a similarly constrained model.

"Immutable variables are pretty much the opposite of how most programmers think about variables ('A variable that doesn't vary?'). So just a few years ago, the idea that functional programming was going to catch on seemed very dubious, and it was very surprising that F# became a first-class language so quickly," O'Brien wrote in an e-mail.

"Similarly, immutability and strong typing make it easier to reason about security," he added.

O'Brien questioned whether F# would become a more prominent language, or if Microsoft would evolve C# to have more of the same constructs that support automatic parallelization.

Automatic parallelization was a "big question mark" in Microsoft's Midori documents, he said. "One thing I've been noticing is that MSR is producing tons of stuff on reasoning about concurrent programs, exploiting latent parallelism ‘automatically.’ "

Microsoft must evolve the .NET Framework Common Language Runtime further to fully exploit the advantages of functional programming, O'Brien said.

Microsoft also has rapidly developed its Silverlight runtime. The Midori programming model includes Bartok, an MSR project that endeavored to create a lightweight compiled and managed runtime system that was more efficient than the  .NET Framework.

"There's no question that Microsoft is seeing Silverlight as the lightweight platform for delivering applications (Web-based and mobile). As far as Midori and [Windows] Azure go, what I can see is that a Silverlight front end is a good front end for an Azure-powered back-end system," O'Brien said.

An Azure tie-in?

It would make sense for Microsoft to use the Azure platform as a vehicle for introducing Midori, Forrester's Hammond said. "It's essentially a  .NET-centric (and Internet-centric) scale-out runtime.

"A distributed network-aware OS is the perfect thing to host in the cloud, and what better place to knock out the kinks than your own data center, where you have 100% control over the hardware and infrastructure you're testing on? This also allows them to test it underneath parts of the overall infrastructure: for example, hosting an individual service," Hammond explained.

Further, Microsoft is battling for new territory—distributed applications—with the Windows Azure platform, O'Brien said. As such, the platform has little legacy codebase, as well as ample funding in money and talent, along with new challenges, he added.

"While I don't think that we know if Midori would work as something fed ‘down the pipe’ to the consumer, the idea that Azure might ultimately benefit from its own operating system is definitely worthy of debate," O'Brien said.

O'Brien said that Microsoft might launch Midori as a new operating system for cloud data centers to up the ante against Google, which has developed new programming languages for writing distributed applications.

Midori's strong emphasis on concurrency issues, a willingness to break compatibility, and the idea of using a hypervisor "as a kind of Meta-OS" would fit that strategy, O'Brien observed. However, he noted that there is no concrete knowledge about the state of Midori or even that its design is necessarily attractive for a data center OS.

Microsoft does not have the lead in cloud computing, and it is rolling out new features for the Windows Azure platform to stay competitive with Amazon and Google, O'Brien noted. "At this stage, Microsoft cannot build Azure bottom-up. But the risks of retrofitting Azure to a new OS are vastly less than the unknowns of putting a new OS onto all the world's hardware."

The status of Midori

While the company has remained tightlipped, some information relating to the status of the project has become available. Midori team member Jonathan Shapiro departed Microsoft in March, citing personal reasons.

Microsoft recruited Shapiro from the BitC language and Coyotos operating system projects to work on Midori. He served on a team of high-profile programmers reportedly led by Microsoft senior vice president of technical strategy Eric Rudder.

Whether Rudder's focus has shifted away from Midori onto other projects in unknown. He recently presented at TechEd Dubai in early March on the topic of Microsoft's "three-screens-and-a-cloud" software-plus-services strategy for .NET.

By David Worthington