John Lam on Software

Microsoft and IronRuby

As many of you know, I recently moved my family to Redmond so that I can work on IronRuby. I was very happy with my life in Toronto, and wouldn't have made this move if I wasn't sure that I had excellent management support for our Ruby effort.

Scott Guthrie was the last guy in my interview loop. He also runs the .NET Developer Platform organization that my team (the DLR team) sits in. Scott is responsible for virtually all things related to the .NET platform, including the CLR, IIS, ASP.NET, WPF, Silverlight (and some others that escape me at this moment). Scott convinced me that Microsoft was serious about making a first class implementation of Ruby on the .NET platform when he sketched out the demo that we ultimately presented at MIX - 7 months later.

Our announcement of Silverlight at MIX is really quite significant. It makes it possible for us to deliver IronRuby to an enormous number of computers, and on many platforms (Windows, Mac OS, and Linux via Mono/Moonlight). This makes it possible for Ruby programmers to use their favorite language in many more scenarios than they can today.

However, since our announcements at MIX, there's been much speculation both publicly and privately about what our IronRuby plans are. Recently, Martin Fowler posted a well-written post that, among other things, questions our commitment to compatibility:

The big question for Microsoft's "Iron Ruby" is how compatible will it be? Will it be a full implementation on the CLR?

From where I sit, I know that I have excellent air cover from my management chain, that includes Jason Zander who runs the .NET Frameworks group at Microsoft. These guys are serious about building a first-class implementation of Ruby that runs on both CoreCLR in Silverlight, as well as the desktop/server flavors of CLR 2.0 that are shipping today. They're committed to making sure that we have what we need to get the job done.

We will ship IronRuby under the Ms-PL (our BSD-style license). If we fail to live up to your expectations, or if you don't like what we're doing, you can fork the code. I think that our track record with IronPython and the Python community serves as an example of what you can expect from our IronRuby effort.

Martin finishes off his post with a challenge to us to do the right thing:

So what I see for Ruby and Microsoft is an opportunity. The Ruby community seems eager to work with Microsoft. This provides an opportunity for Redmond to figure out how to deal with the problems of working with open source and for this effort to serve as an exemplar for future collaboration. A first class implementation of the full Ruby platform on .NET would be a wonderful product of this collaboration. Perhaps an even better result would be for this work to serve as an example of how Microsoft can collaborate with a community that's centered on openness and agility; an example that can be a springboard for further spreading of attitudes that can further help programmers and their customers throughout the Microsoft world.

I'm eager to work with the community to figure out how we can work together. An excellent opportunity to do so is at our Lang.net symposium that we will hold on campus sometime in September/October of this year. If you're interested in helping to figure out how we can collaborate better with the community I'd like to extend to you an invitation to come out to meet with us then.

In the end, I think that it's important to judge our effort based on the code that we ship. We're going to ship our first CodePlex release of IronRuby at OSCON this year. I'm looking forward to getting your feedback on that code (particularly on what we got wrong) when we ship it so that we can work together to deliver the right solution for everyone.

Skipping Tech Ed this year

I really need to spend some serious time hacking on IronRuby, so I've decided to bow out of Tech Ed this year. Mahesh Prakriya from our team (and my boss) has graciously stepped up to the plate to do my talk for me so that Tomas and I can spend more quality hacking time together in our new offices in Building 41.

One event that I'll miss doing is the Why Dynamic Languages interactive theater session (WEB11-TLC - Why Dynamic Languages?) that I was scheduled to do with Scott Hanselman (who is also missing Tech Ed this year). If you're going to Tech Ed and would be interested in leading this session, please let me know either in the comments or by email (jflam).

Wrapping up the Compiler Dev Lab

Some final thoughts about the Compiler Dev Lab that we hosted this week:

1) It's really interesting to see static languages get more dynamic, and dynamic languages get more static. An example of the former is C# adding extension methods and type inferencing. An example of the latter is IronPython adding support for manipulating CLR generic types. There's an interesting point in the middle where things are heading towards ... these ideas were inspired by Anders Hejlsberg's talk on LINQ today at the Dev Lab.

2) Miguel de Icaza spent a bunch of time lobbying us for a XamlReader.LoadFromJson() API. His reasoning was that folks don't like to type XAML, and would prefer a more wrist-friendly syntax for generating WPF/S element trees. It would be interesting to hear some feedback from folks about this. I personally prefer using a "script-driven" XAML approach where I would generate WPF/S element trees from pure Ruby code. It would be a natural fit for managed JS, of course :)

3) Mike Stall gave an excellent talk today about managed debugging. I had seen Mike give a talk in the CLR Foundations series that we have in Building 42, and really wanted him to give a talk this week. He did not disappoint; the pointer to the Managed Debugger sample was worth the price of admission alone (if you were wondering how to do IL-level debugging - this is the tool for you!)

I had to leave early to take care of the kids this afternoon after my wife got sick. I hope everyone had a good experience at the dev lab and hope to see great things come out of your work!

Compiler Dev Lab

The DLR team is hosting this year's compiler dev lab on campus this week. One of the really cool things about the lab was meeting folks who were really interested porting their languages to run on top of the DLR.

It was fun hanging out with Miguel de Icaza again - and Miguel's been posting his notes from the first two days of the dev labs on his blog.

John Gough was also there - and we had a great time talking about Ruby and some of his latest projects.

I had a fun time chatting with Rodrigo de Oliveira from the Boo project - a Python-esque statically typed language with optional dynamic typing. Rodrigo was really interested in porting Boo to run on the DLR because of the potential performance benefits of our dynamic dispatch mechanisms. Also there was JB Evain of Cecil fame, who was recently hired by Miguel to work on Mono over at Novell (congratulations on the new job!). These two guys are doing a fantastic job of building a very nice static language compiler platform that layers on top of the CLR (should you guys call it the Static Language Runtime? :))

A  nice side effect of the lab is that it forces us to start documenting parts of the DLR. I'm going to be spending some time writing up our talk from the dev lab, but for folks who want some more detailed information about the DLR, here's the slides from the talk that I gave today with Nandan Prabhu (one of the JS compiler devs) and Dino Viehland (one of the IronPython / DLR compiler devs).

Download dlr_experiences.pdf

Update: I've managed to grab a copy of Jim Hugunin's deck from his Zen of the DLR talk on Monday. We're recording all of the talks here, so I'll post them somewhere when we've done the post-production pass on them.

Download zen_of_the_dlr.pdf

DLR Running on Mono

Congratulations to the Mono team on getting the DLR up and running on Mono. And in 16 days no less!

Modularity, Ruby and Doing the Right Thing

It seems like we stirred up some controversy with these lines of code in our demo:

JS = require 'foo.js'
...
JS.initialize

Let's be clear about what this code does: it creates a ScriptModule (a DLR type) that represents the execution context of the foo.js code, and assigns it to the JS constant. The JS.initialize line calls the initialize method in foo.js. This is a user-defined method, and not some kind of DLR initialization thing.

We debated the idea of using require to import foreign modules in my office for a very short period of time before checking in the change. It seemed like a natural thing to do - monkey-patch the require method so that we can import foreign modules into IronRuby.

Nobody has argued that importing foreign modules into IronRuby is a bad idea.[1] Rather, I think that folks have taken issue with using require as the mechanism for doing so.

Point taken. We will have a different mechanism for doing so in the future.

But this brings up an important meta-level issue: our commitment to doing the right thing. It seems that our intentions in this area have been called into question. We have a fairly straightforward task here: to make a CLR implementation of the Ruby language that is as compatible as we can make it. So when we have a design decision to make, it's driven by "how does the native Ruby implementation handle it"? 

We here on the IronRuby team are under no illusion that we will be 100% compatible with the existing Ruby 1.8.x branch. But we will get as close as we can and bend over backwards to ensure that it is so. And I expect that you guys will keep us honest in this regard when it seems like we made a bad design decision.[2]

Pure Ruby code should just work, and code that depends on native platform libraries will have to be ported (either by us porting the native platform libraries or by folks modifying the code to work with our native platform libraries aka the Framework Class Libraries).

Please do speak up when we post code frags, demos (and eventually source). We're really interested in what you have to say !

[1] Keep in mind that we wanted to demo something interesting at MIX (cross-language interop) and that using Ruby (the least mature of our language implementations) as the glue to bind other code together seemed to be the right thing to do. However, at that time we couldn't solicit input from the community since we were in stealth mode.

[2] While I don't think that overloading require was a bad design decision (nobody has given us a concrete example that shows where this breaks their app / library), it's controversial enough and the fix simple enough to do that I have no problem doing it.

Rounding up DLR questions

I thought I'd answer some more questions that have shown up around the web about the DLR. I was out for a short vacation before getting back to the office middle of this week.

1. Is the CoreCLR available for a standalone download?
   
   No. It's designed to be used with Silverlight. Jamie Cansdale has figured out how to extract the assemblies out of the redist and run a console application using it. Why? Because it makes sense to be able to run your unit tests using the target runtime.
2. Is the DLR itself available for standalone download?
   
   Not at this time. Right now you can download it as part of the IronPython 2.0 Alpha release on Codeplex.
3. Will I be able to host the DLR myself in my own .NET applications?
   
   Yes. We have a generic hosting API, and we internally have a number of hosts built using it, including a console host for the desktop CLR, various test harnesses for our test team etc.
4. Does JS support generics?
   
   Not with this release, but it is on the roadmap.
5. Is the DLR just IronPython's libraries pulled out into a reusable form?
   
   At a certain level, this is true (a simple example is the BigInteger type). However, it does go considerably deeper than this. The DLR has a more ambitious goal of unifying the programming model of static languages and dynamic languages in the most efficient way possible. For details on this, see Jim Hugunin's two posts on The One True Object, particularly Part 2.
6. What about languages beyond the four (IronPython, IronRuby, JScript, and VB) that Microsoft is building? Can the DLR actually be used by folks outside the company?
   
   At the time of this writing, it's exactly 11 days since we shipped bits to the world. Peter Fisk is making good progress porting his Smalltalk implementation to the DLR.

Clearing the air about Silverlight and the DLR

It's been interesting looking at the reactions from around the blogosphere. With so many people playing telephone, I thought I'd collect a bunch of facts together in one place so that folks can get a clearer picture about what we're doing.

1. The DLR requires the CLR. So this means that it only works with the Silverlight 1.1 Alpha that was released at MIX, and not the Silverlight 1.0 Beta.

2. The source code for DLR, IronPython (and IronRuby when it is released) are being released under the Ms-PL. The Ms-PL is a BSD-style license.

3. The DLR will also run on top of the desktop CLR V2.0, not just the Silverlight CLR. We have a generic hosting API that lets us retarget the DLR to run on top of arbitrary hosts. Silverlight is only one such host.

4. The Silverlight 1.1 Alpha bits released at MIX include the DLR, IronPython and managed JScript.

5. Silverlight lets you run compiled .NET code in the browser, not just Python and JScript code. Any assembly that has been compiled to target the Silverlight libraries should just work. So if you want to write code in C#, VB.NET or Boo to target Silverlight, knock yourself out.

6. Our JScript implementation targets the ECMAScript 3.0 specification.

7. We will implement four languages for the DLR: IronPython, IronRuby, JScript, and Visual Basic. As of this writing we are only releasing the source code for IronPython and IronRuby under the Ms-PL. We have not decided whether we will release the source code for either JScript or Visual Basic.

8. Silverlight is targeted at browsers that run on the two most popular client operating systems: Windows and Mac OS X. Miguel de Icaza was quoted as saying that they will release a Mono+Linux implementation of Silverlight by the end of the year.

9. Silverlight V1.1 will only target Intel Mac OS X machines.

Is there anything else that I missed?

Silverlight: do you love it or hate it?

Wow. Technorati is normally a wonderful resource, but with > 500 posts per day on Silverlight, it's really hard to see who's saying what. So I thought that I'd try and summarize a bit of what's been going on for folks who are interested in following the conversation:

Jon Udell interviewed me and added his insightful (as always) commentary.

Mark Pilgram hates everything

Darryl Taft of eWEEK interviewed Jim and me, and he also wrote a nice article on our MIX announcement.

ArsTechnica has a nice writeup on the DLR but the forum comments are also worth a read.

Scott Hanselman has a really nice writeup on Silverlight and the DLR. I love the energy flow diagram, the calling out of the DLR, the choice of license, and the awesome quotes :)

Miguel de Icaza really liked what he saw. I love how he uses terms like "adorable" and "fantastic" in describing technology. He also comments on our choice of the Ms-PL license for the DLR and Python/Ruby:

The release for the DLR is done under the terms of the Microsoft Permissive License (MsPL) which is by all means an open source license. This means that we can use and distribute the DLR as part of Mono without having to build it from scratch. A brilliant move by Microsoft.

David Laribee really gets the "Just Glue It" theme from our talk.

Peter Fisk, the Vista Smalltalk guy, is really excited about porting to DLR

For my work, two things are of importance: * the release of a CTP of the Silverlight CLR will finally allow me to examine its capabilities and limitations, so that I can proceed with porting Vista Smalltalk to Silverlight * the DLR should allow me to build a much higher performance version of Vista Smalltalk

Over the next several weeks, I will be preparing a new release of Vst/.Net to integrate with the DLR and be better synchronized with the Flash version (Vst/Flash). Hopefully, I can reach a point where scripts can be run with minimal modifications across Flash/Apollo/Silverlight/Vista.

And after he had a chance to play with it, he was even more impressed

This is very impressive technology that is certain to impact how applications are built and delivered. I think that it will mark the beginning of the broad acceptance of Rich Internet Applications in the marketplace - and this will probably benefit Adobe as well.

My congratulations to the folks at Microsoft for some excellent software engineering.

Michael Arrington is excited by Silverlight and Nik Cubrilovic has a more detailed overview on Techcrunch. They have >383K subscribers via Feedburner. Amazing.

Alexey Gavrilov really digs the DLR Console.

This is my favorite sample so far — IronPython, JavaScript and XAML console implemented on IronPython. It has syntax highlighting and code auto-competition already so I guess it won’t take long until we see complete IDE running inside the browser!

Finally, you know you've made the big time when you have your own Wikipedia page :)

Showtime at MIX!

Where we lived for 12 hours, originally uploaded by John Lam.

The demo machines were all staged backstage to make switching between all of the different computers easier. We had a primary and backup machine for our demo (as did all of the other demos). So just in case something bad happened, we could switch to the backup machine to continue the demo.

I drove our demo backstage, mirroring what Scott did just in case we had to switch. Our demo went off without a hitch, but boy was it nerve-wracking. This was much more difficult to do than my talk at MIX, even though it only took 2 minutes here vs. the 1 hour + duration of our talk (and we were doing far more high-risk things than we were doing in the keynote).

This photo captures the setup for all of the different machines - the two 17" MacBook Pros that we used for our demos were sitting on the top shelf of this photo. You can also see one of the four giant projection screens in the background.