John Lam on Software

Want to work on languages at Microsoft?

First a public service announcement followed by a message from our sponsors ...

The videos from our Lang.net 2008 symposium are now online. For a recap of the conference, see my earlier blog post. If you like what you see in these talks, we'd love to hear from you!

Our languages team is growing fast; we have five open positions as of today. We need devs for IronRuby, IronPython and F# and testers for IronRuby and F# (the links to the F# jobs will appear soon). In the meantime, feel free to contact me (jflam at you-know-where) about any of these positions, and I'll put you in touch with the right folks.

Helping those who help you

You can't do your job all by yourself. As Gretchen Rubin says, one of the secrets of adulthood is "It's okay to ask for help".

Twice a year, folks from our offices around the world (collectively, 'the field') come to Seattle for TechReady. It's an internal conference that the product groups use to help folks in the field do their jobs better.

On Tuesday, I gave my 'Why Ruby?' talk at TechReady. It tried to answer a few questions:

• Why should you care?
• Why is Ruby interesting?
• Who is using it and were they successful?
• What is my team doing about it?
• What is our roadmap and how will we get there?
• Why is Open Source important to our future?

To some folks who read my blog, the answers to these things are obvious. But remember, not everybody is you. Not everybody cares about the same things that you do. Reaching out to folks who don't share your world view is a big part of how you can change the way things are today. And that's a big part of what I do around here.

I got the most value from giving my talk when I spent some time hanging out with folks from my talk. Over some beers I had a chance to learn what folks in the field care about, how they get measured, and how we can help each other. Next time I give this talk, I'll know exactly what to say. Talks are for your audience, and understanding what your audience cares about is the key to success.

Thanks to the good folks from DPE (our Developer and Platform Evangelism Team) for your company and picking up the check for dinner :)

*

I recently started re-reading Jeff Hawkins' most excellent book, On Intelligence. This is easily one of the best books that I've ever read. In it, he presents his model for human intelligence, which is based on what he calls a memory-predictive framework. Human brains can do complex things (like catching a ball) in about half a second. In that time, given the fact that a neuron takes 5ms to fire, your computation path is at best 100 steps long. His theory is that you don't compute the solution, but instead you recall the solution. Jeff also has a company called Numenta, which implements this algorithm in software today, and there's an SDK!

How to say 'thank you'

We live in a world where most of us say 'thank you' using an email with a single word: 'thanks'. I'm as guilty of this as the next guy. Yesterday I was reminded of the kind of world that we could live in.

When I got back to my office after a day's worth of meetings in different places, I found a package on my desk. It was postmarked from London, Ontario, Canada. I looked at the name - it didn't jump out at me. It was a fairly big, and moderately heavy package. I first thought hmmm ... I'm not the kind of guy that a Unabomber-esque person would want to take out, right?

A quick search via Google jogged my memory. The sender was Edward Ocampo-Gooding, one of the organizers of this year's CUSEC conference. I had met him at RubyConf, and he had sent me a mail out of the blue a few months ago asking for an introduction to Jon Udell. It took me maybe 30 seconds to fire off the email that introduced them to each other and I promptly forgot about it.

Once I had confidence that the package wouldn't blow up when I opened it, I looked inside. I found a small gift and a hand-written letter:

There was a link in the letter. When I clicked on the link, this is what I saw:

This was a really thoughtful and touching way to say thank-you. It says a lot about the character of this young man, and a reminder to the rest of us about how to treat others who help you out. I guess the way I'm thanking him is through this blog post :)

BTW, I heard awesome things about the CUSEC conference this year from folks that I know personally: Jeff Atwood, Zed Shaw, and Jon all wrote glowingly of their experiences.

*

Just the other day I was watching Randy Pausch's 'Other Lecture' on Time Management. Randy talks about the stack of blank thank-you cards that he keeps on his desk. He makes a point to hand-write thank-you notes to people who have helped him out. Not quite what Ed did for me, but an important thing to do nonetheless.

Surviving at Microsoft: Knowing who has the answer

You can't know everything. And in a place like Microsoft, chances are you'll need to get a fact or an opinion on something that you know very little about. Knowing who to ask is an important survival skill around here. As I've gotten better at this, it's helped me in other parts of my life too: let me tell you the story of the stripped oil pan bolt.

I drive a mildly exotic car, so repair bills can add up quickly. Last week, I took it in for a routine oil service at a local independent service center. When they removed the oil pan bolt, part of the threading in the oil pan bolt came out along with the bolt. This is usually the fault of the guy who did the previous oil service: they over-tightened the bolt. This spells trouble.

When they called, they told me that they would have to replace the oil pan. Now, for my car, this runs about $1800 before taxes (the front axle runs through the oil pan, so it's a major piece of work). I had them order the part, but since it wouldn't arrive until this week, I wound up using my Microsoft Flex Pass which lets me ride the bus for free. More importantly, this bought me some time.

Over the weekend, I had a chat with my neighbor who is another car guy and I told him my story of woe. He suggested a fix, and gave me the name and number of a guy who he trusts. I called him up and he gave me some great general advice, but he referred me to a second guy who was the real expert. After a bit of talking, he confirmed my neighbor's suggestion (for those who are curious, it uses a TimeSert to fix the problem). I also asked him the precision question[1]: "Was there any cases where you couldn't repair a stripped oil pan bolt using a TimeSert?" No.

Armed with this data, I called up my service guy. He tried to defend his position using some technical details that I couldn't confirm (they ruled out using a TimeSert since it would be too 'tall' for the pan). The precision question was a life saver since I could simply say 'Bob over at Foo motors said that he can always repair this using a TimeSert'. And since Bob is an authority in the local area, it was hard for my service guy to refute. Worst case I could tow my car to Bob and get him to fix it.

Long story short, my service guy wound up calling Bob and getting the special TimeSert that Bob uses to fix this problem. This saved me over $1700 and my car runs fine today.

It's sometimes hard for folks new to Microsoft to be effective since they often don't know who to ask. It's also hard for some folks to admit that they don't know an answer by asking, which compounds the problem. Figuring this out is one of the first things that you should do once you get here. And if you don't know who to ask, I can volunteer my help :)

If you found this interesting, my team is expanding, and we have more openings in Dev, Test and PM. More on this in a separate post.

[1] There's a class here at MSFT that used to be called "Precision Questioning" that teaches this. Execs tend to use this a lot. The trick to detecting when someone switches into "PQ mode" is hearing type information embedded in the question - eg "How many bugs are left?" (Integer) or "When will you ship?" (DateTime).

*

Distributed version control systems are all the rage with the kids these days. They are an awesome piece of technology, not so much so for their technical accomplishments (which are very cool) but more so about how they enable folks to collaborate and how they fit well into the social structure of geeks. For some entertaining insights into GIT, one of the more popular DVCS's see this talk that Linus Torvalds gave on the social implications of GIT at this talk that he gave at Google. He does spend time bashing CVS and SVN in the talk, but that made the talk more fun for me :)

The Democratization of Data

One of the great resources that you get as a Microsoft employee is access to the talks over at Microsoft Research. Microsoft really is like grad school.

Most of the talks are recorded, and if you have time, you can head on over to Building 99 and watch some in person. Last week I saw Jeffrey Heer give a talk with the provocative title: Voyagers and Voyeurs.

Jeffrey gave a fantastic talk about his work at UC Berkeley and at IBM Research as part of the Many Eyes project. I'm amazed by his work on the prefuse visualization toolkit which lets you create stunning visualizations:

The flare visualization toolkit is based on Flash (unlike prefuse which relies on Java). You should definitely check out the flash-based demo to get a feel for how well designed these animations are.

As it turns out, Jeffrey also did some work at Many Eyes. I first heard about Many Eyes from Jon Udell. Jon did a great interview with the researchers behind Many Eyes, Fernanda Viegas and Martin Wattenberg from IBM Research. One of the ideas that Jon explores is the democratization of data, where the Many Eyes visualizations open up a world of analysis to people who never had easy access to the data before. Both good and bad happen here: some people find legitimate problems with the data sets, others manipulate the data sets to support their own views.

It seems like a port of Flare would be a cool Silverlight project ...

Welcome, Wayne Kelly!

Yesterday, Wayne Kelly (the leader of the Ruby.net project) announced[1] that he was going to refocus his efforts on IronRuby. We would like to extend a warm welcome to Wayne, and we invite anyone else who wants to work on IronRuby to join our Open Source project. Microsoft Research funded a portion of the development of Ruby.net, and their parser lives on in IronRuby thanks to the excellent work that Wayne did in producing the Gardens Point Parser Generator.

Having a single implementation on the CLR makes sense for the .NET community. Ruby isn't defined just by the language. It's defined by the programs that it runs. The hardest part of our project isn't getting the language right (although it certainly isn't easy) it's making sure that IronRuby runs Ruby programs. And regardless of what the Rails-haters say these days, Rails is an important program.

Joining forces lets us get avoid duplicating effort. More people working in parallel on libraries means that folks will get a working Ruby on .NET that runs real programs sooner. And that's goodness for everyone, from the contributors who want to see their code used, to devs who want to write Ruby programs on the .NET platform.

*

I'm sure that many of you have already watched Randy Pausch's Last Lecture. If you haven't already, you should carve out about an hour and watch it when you have some quiet time alone. It's an amazing talk that you won't forget, particularly if you have kids.

[1] Here's Wayne's email in its entirety:

Ruby.NET started life in 2005 as an academic research project with the goals of learning more about the challenges of mapping a dynamic language such as Ruby onto the relatively static CLI platform. When we released our first beta in 2006, many people got excited and started blogging about the project, at which time the project took on a life of its own heading towards a production quality release that people could one day actually use. The release of IronRuby last year obviously caused us to question this unstated goal. At the time we didn't know if the IronRuby project and the DLR would succeed, so we decided to continue with Ruby.NET at that stage. Last week at the Lang.NET Symposium, I presented our work on the Ruby.NET project and also had the opportunity to learn more about the progress of the IronRuby project and the inner workings of the DLR (and also the JRuby project presented by Charles Nutter).

I've come to the conclusion that the DLR is clearly here to stay - it's becoming an even more important part of the Microsoft platform. I also believe that to obtain production quality performance, Ruby.NET would need to reinvent (or adopt) something equivalent to the DLR. If we were starting the project today, there is no way we wouldn't use the DLR. Whilst Ruby.NET initially had a good head start on the IronRuby project; by incorporating the Ruby.NET parser and scanner and by leveraging the DLR, I now believe that IronRuby is more likely to succeed as a production quality implementation of Ruby on the .NET platform. I believe that ultimately there is no need for two different implementations of Ruby on .NET. So, if Ruby.NET is ultimately not going to be that implementation, then we should not waste further developer effort fruitlessly chasing that goal. There is still a massive amount `of work required to achieve full semantic compatibility, to achieve production quality performance and to get Rails to run robustly.

There have already been a number of practical and research outcomes from the Ruby.NET project, however, at this stage, I believe we (the Ruby.NET community) can make the biggest impact by levering our experiences with Ruby.NET to contribute to the IronRuby and DLR projects. Personally, I still feel we have unfinished business - we set our selves the goal of running Rails on .NET and we haven't achieved that yet. If we can leverage our experience to help IronRuby get to that point, then I'd at least have the personal satisfaction of helping see the job completed.

These are just my views. As a researcher, my prime interest is not in developing products, but in developing innovative new ideas and having an impact by having those ideas used in the real world. I'm aware that others in the community will have different goals and so will presumably have a different take on this - I'm keen to hear what you think. If anyone wants to press ahead, then the code base is still owned and controlled by you the community, so you are free to do with it as you please with our full blessing.

I'd also like to make it very clear that this decision is entirely my own - based on research and technical considerations. Microsoft did not in any way suggest or encourage us to kill the project and we thank them again for their support of the project.

I'd like to thank all of our contributors and supporters and apologize if this decision comes as a disappointment. I hope many of you will join me in contributing to the IronRuby project and see it through to a successful completion.

Cheers, Wayne. 

IronRuby, Sun, and Lang.net

Ruby was well represented at Lang.net this year, with Charlie Nutter of JRuby, Wayne Kelly of Ruby.net and myself all giving talks.

It was great having the folks from Sun there as well, with John Rose, Charlie Nutter, and Dan Ingalls all giving interesting talks. John Rose talked about his DaVinci Machine effort to make dynamic languages run better on the JVM. Dan talked about reinventing Smalltalk in the guise of JavaScript via his Lively Kernel project. Charlie Nutter talked about the more challenging aspects of implementing JRuby.

Charlie and John did their best to keep me up to the wee hours of the morning the night before my talk. But at 9am the next morning, I managed to deliver my talk on IronRuby. It was mostly a status update on where we are today, and how we plan to get to 1.0. First, we have debugging and stack backtraces working. This was a long time coming, and it makes life much easier for folks who are building the libraries. This feature lives only in a shelveset at the moment. It's going to take some work before we're ready to integrate it into the tree.

Next, we showed an update of where we are in running the Rubinius spec suite. At Lang.net, we were passing 57% of the Rubinius specs (1418 out of 2505). Note that not all specs are enabled right now, but most are.

Finally, we talked about how we are going to get to 1.0. Right now we're going to switch to a goal-driven development process. Our next goal is to get 'gem install hoe' working. The Rakefile contains a task called 'gap', which lets you perform a gap analysis against a target application via the set_trace_proc interpreter hook.

We have folks who have signed up to get both YAML and Regex up and running. Once these two libraries are functional, we'll be much farther along towards getting gem install to work.

We also want to use the gap analysis tool as a way of automatically publishing progress from our build system. Look for a site that will boil down progress towards a goal in terms of 'n more methods to implement', along with a list of those methods.

You can download my slide deck from here.

*

I'm a fan of the writings of Rands In Repose. If you have a significant other, you must get them to spend some time reading his most excellent The Nerd Handbook. They'll understand you much better after that!

Lang.net wrap-up

The last time Lang.net was held (around 18 months ago), I made up my mind to start the process of coming to Microsoft. OK, it was the bright lights and mind conditioning in the back room that made me come but I digress ...

This time around, it was fun seeing the show from the other side of the fence. That meant, of course, that I had to sit in the cheap seats so that the 'paying' customers could have the good seats up front. The show didn't disappoint.

I'm not going to go into a blow-by-blow account of everything that happened at the show. Ted Neward does an excellent job of that in these posts: Day 1, Day 2, Day 3.

As usual, the show had a great mix of folks from inside and outside of the company, as well as folks from the .NET and JAVA communities. Even PHP was represented. The following is a random summary of things that I thought were particularly interesting / noteworthy.

Jason Zander and Don Box were the book-ends of the conference. Jason led off with a look back at the early history of the CLR, and actually dug up some old emails from that era. Don capped off the talk with a look back at how the history of [meta]data from /SUBSYSTEM:CONSOLE to XAML and a look forward to the promise of declarative programming (no secrets were harmed in the making of his talk though).

Anders Hejlsberg reviewed where C# 3.0 is today, and foreshadows the eventual convergence of languages. We see some hints of that today in F#, which combines functional, imperative and OO paradigms. It was a great look at how the set of features in C# 3.0 when combined create a whole that is greater than the sum of its parts.

Jim Hugunin did a reprise of his 'what does daddy do at work talk?'. Of course, if you've seen Jim do a talk in the last year or so you couldn't miss the robot that usually accompanies him:

The take home message in his talk is that 'sharing is good', which explains a lot of the motivation behind the DLR.

Martin Maly (the LOLCODE king) did an excellent talk on building a language on the DLR. He showed how to add nodes to the ToyScript AST, and how to use DLR features to make that code run faster. Martin is the DLR AST dev, and is blogging up a storm on how to use his baby to implement your language.

Erik Meijer did his best mad scientist imitation with a look at Volta. Volta is his project that lets you build web apps using nothing more than your favorite CLR language. Volta's big feature is 'tier splitting', where it handles the details of partitioning your app into the parts that run on the client and those that run on the server. The client parts can run on a client-side CLR (if one is installed) or it can run on JavaScript (via a CIL -> JavaScript transform). It's a cool idea, and worth taking time to look into.

Peli de Halleux talked about Pex - a very cool dynamic analysis framework that generates test cases for your app. You should carve out a chunk of time to watch their most excellent 5 minute screencast on Pex.

Jeffrey Sax gave a talk that could have been titled "N reasons why the CLR sucks for numerical computing". But Jeffrey is too polite for that, but instead presented a very critical look at the parts of the CLR that warrant improvement. He got most of the analysis correct, and a bunch of folks including Vance Morrison (the CLR perf architect) had a chance to follow up with him directly after his talk. That's a great example of the value of gatherings like this to get folks together in a room to have an open and frank conversation.

Of course, Miguel de Icaza didn't disappoint in his talk. Split into several 'themes', he talked about Moonlight, how it happened, where it is today, and where it can go. Miguel's talks are interesting because they are about people as well as technology. He tells the behind-the-scenes story which engages with a primitive part of our brains. The story about JB Evain's Rodrigo de Oliveira's 20 hour hack to create a statically typed JavaScript-esque language from Boo, and how it is used to make video games in Unity, was awesome.

*

For those of you who are wondering about Ruby and Lang.net, there will be here's a separate post about all that fun stuff :)

*

I'm a huge fan of Gretchen Rubin's The Happiness Project. If you're a parent, go visit her wonderful multimedia story: The Years Are Short. You won't be disappointed.