John Lam on Software

RubyCLR can marshal auto-layout value types

Finally.

This was a tough one. The CLR supports three types of value types: auto-layout, sequential, and explicit. A sequential value type is one whose fields are laid out in memory in the same order that they are declared in the type. An explicit value type is one whose fields are explicitly laid out in memory to achieve compatibility with some unmanaged data structure. An auto-layout value type is one where the CLR can rearrange fields to create the most efficient representation based on the target architecture.

Clearly, auto-layout value types are very useful. However, if you want to marshal these things you’re likely to run into trouble. At the very least, you need to allocate enough memory to hold the value type’s fields and then copy the value type into that buffer. The documented way of determining the size of a value type is to use System.Runtime.InteropServices.Marshal.SizeOf().

However, it refuses to report the size of an auto-layout value type. So I have to resort to using the CIL sizeof instruction that provides the same information. But sizeof requires that you specify the type of the value type. So I have to generate a small dynamic method for every auto-layout dynamic that I have to determine the size of:

sizeof  <type name>
ret

I execute and then cache the value in the Ruby class object. Yes, it’s a grotesque hack, but it does work. And for the time being, the size of auto-layout value types is fixed for the duration of the process, but this will change in the future.

dojo.storage

Wow. The dojo guys have really done an awesome job with their offline storage library. Check out this demo which shows off a simple disconnected browser-based word processor. This opens up a lot of really interesting scenarios for browser hosted apps.

Here’s a more detailed description by Brad Neuberg, the creator of dojo.storage.

RubyCLR now on RubyForge

I finally spent some time this morning checking in the source code for RubyCLR into RubyForge.

The project page is http://rubyforge.org/projects/rubyclr.

You can browse the Subversion repository as well.

If you want to check out the source code, just type this:

svn checkout svn://rubyforge.org//var/svn/rubyclr

This, of course, presumes that you have a Subversion client installed.

This is the latest source that’s on my laptop. All of the unit tests pass on this build, but it’s not a packaged release. Consider it a daily build, use at own risk etc.

Comments welcome.

Blittable vs. non-blittable types

Just the other day, I was attempting to marshal a DateTime value type from Ruby and RubyCLR blew up. Now I know why.

I should have known better, but I failed to account for blittable vs. non-blittable value types in RubyCLR. I explicitly avoid boxing value types in RubyCLR by blitting the value type into memory that I explicilty allocate for the Ruby object.

All of my value type marshaling tests were done on sequential layout types such as Point whose layout in managed memory is the same as they would be in unmanaged memory.

The trick now is figuring out how to marshal auto layout value types generically across the interop boundary.

Parallels Beta 5 Rocks

I installed Beta 5 of Parallels last night. It now supports full screen (1440×900) on my MBP. Combining it with the Virtue desktop manager, I can now toggle between OS X and Windows XP via a really cool ‘flip’ animation and live the dual-OS lifestyle. I just pre-ordered my copy today.

Coming up for air at VS Live!

The last week or so of silence was due to a nasty virus that swept across the Lam household, and we’re finally pulling out of it now. I didn’t really get it, but everyone else did, which meant that I had take care of everyone else.

I managed to get away yesterday to give my keynote on The Future of Programming Languages at VS Live! in Toronto. It was a fun time, and my first Lessig/Takahashi-style talk. Hopefully the attendees had as much fun as I had delivering it. It was also my first experience giving a talk with Keynote, and I was very happy with both the authoring and the presentation experience.

Here’s the PDF of the slides from my talk. As a Lessig/Takahashi-style talk, it’s not all that useful if you didn’t see the talk, but it’s worth a look if you’re interested in seeing one approach for motivating Ruby to a statically typed language audience.

RubyCLR now lets you implement CLR interfaces in Ruby

The first pass of the code for implementing CLR interfaces in Ruby is now working. Here’s a simple example of implementing IEnumerator in Ruby:

class RubyEnumerator
  implements IEnumerator

  def initialize(ruby_object)
    @position = -1
    @ruby_object = ruby_object
  end

  # NOTE: hack for properties - will be "current" when done
  def get__current
    @ruby_object[@position]
  end

  def move_next
    @position += 1
    @ruby_object.length != @position
  end

  def reset
    @position = -1
  end
end

Here’s the modification to the Ruby Array class to support IEnumerable:

class Array
  implements IEnumerable

  def get_enumerator
    RubyEnumerator.new(self)
  end
end

Here’s the set of unit tests that validates this behavior. Note the use of inlined C#:

class ImplicitDataBindingTests < TestCase
  inline :csharp do |compiler|
    compiler.compile <<-EOF
      using System.Collections;
      namespace RubyClr.Tests {
        public class RubyArrayDataBindingTarget {
          public static int[] EnumArray(IEnumerable obj) {
            ArrayList al = new ArrayList();
            foreach(int current in obj)
              al.Add(current);
            return (int[])al.ToArray(typeof(int));
          }
        }
      }
    EOF
  end

  def test_array_bind
    numbers = [1, 2, 3, 4, 5]
    result = RubyArrayDataBindingTarget.enum_array(numbers)
    assert_equal 1, result[0]
    assert_equal 5, result[4]
  end
end

RubyCLR progress on implementing CLR interfaces

A major new feature of RubyCLR is implementing CLR interfaces using Ruby. This will provide most of the goodness of the ‘how do I call Ruby code from my .NET code?’ scenario without having to derive a Ruby class from a .NET class (a much harder problem).

A key driver for this scenario is supporting data binding without having to hand-write CLR wrapper classes for those interfaces (right now I only support calling Ruby code via delegates). This support is there in the current drop, but will be completely rewritten in Ruby in the next drop.

However, I ran into numerous blocking issues over the past week. True, I was distracted setting up my new MacBook Pro, but I resumed work mid-week on RubyCLR and fought issues in both Ruby and Reflection.Emit (I’m pretty sure I found a bug in ILGenerator). I’ve got workarounds for all of those issues, so I’m pretty sure I can get something up and running tomorrow.

More news soon.

Configuring Parallels for Windows XP

I drive my computer using my keyboard. The major disadvantage of Macs to me is that they are missing 4 critical keys: Page Up/Down, Start and End. This makes navigating around when I’m running Windows under Parallels a royal pain.

What’s worse though is the default behavior of how Parallel’s keyboard driver maps the Mac Command keys to Windows; they’re mapped to the Windows key by default. Now, the position of the Command key is identical to where the ALT key is in most Windows keyboards so this was a major source of pain. A secondary source of pain is the fact that there is no Delete key as well, but there is a superflous Enter key where the Delete key normally lives on a Windows keyboard. Here’s a magic registry hack that I cooked up this morning that will solve this problem. Just paste this code into a .reg file and run:

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Keyboard Layout]
"Scancode Map"=hex:00,00,00,00,00,00,00,00,03,00,00,00,53,e0,1c,e0,38,00,5b,e0,\
  00,00,00,00

This will map the Command keys back to the ALT key, and the Enter key to the Delete key. Now I can undo the DoubleCommands hack that I blogged about yesterday. Which means that my Mac feels more Mac-like and my Windows feels more Windows-like. Now if only I could hack those missing keys into the keyboard …

Configuring OS X

I spent some time today setting up OS X the way I want it to be on my new MacBook Pro. At the very least I need it to be as usable as the Windows XP host OS on my M200. So here’s the apps that I need to configure and get working:

1. Mail. It took a while to get the Exchange Server support in Mail up and running, but it works fine if your Exchange server has IMAP4 support enabled (mine didn’t by default). All my mail is now synchronized inside of Apple’s Mail application, and it’s all accessible via Spotlight. The more I use Spotlight the more amazed I am. It’s better than Google Desktop Search, so +1 for the MBP so far.

2. QuickSilver. The combination of QuickSilver + Spotlight is devastating. CTRL-Space is mapped to QuickSilver and Command-Space is mapped to Spotlight. It makes me ridiculously productive with my hands never leaving the keyboard. In fact, QuickSilver is so good, that I rarely find myself using Expose or Command-Tab. I just hit CTRL-Space and type the first letter or two of the app that I want to run and it just works.

3. Adium. This is a multi-protocol IM client that supports tabbed windows, and has a beautiful interface. I can now IM folks using both MSN and Google Talk without any difficulties. File transfer works great as well.

4. NetNewsWire I really like this RSS reader. On Windows I was using Omea Pro which was OK but it had enough annoying idiosyncracies (er bugs) that I soured on it. So far the only thing I would really like to see in NetNewsWire is support for adding blog posts directly to del.icio.us and Spotlight integration. The latter would rock since I often want to find something inside of blog posts that I save in my reader.

5. FireFox 1.5.0.2 Make sure you get the Universal Binary version as the other version is way too slow to be useful. I need my extensions!

6. MenuMeters I totally love this software. I have Disk I/O and CPU thermometer + % displayed in my menu. It helps me diagose problems in a hurry.

7. Parallels Beta 3. I can now run Windows hosted inside of OS X! I was really annoyed by a number of things in Beta 2, but these guys are revving their software fast and squashing bugs flat. All of my issues with Beta 2 are now fixed except for the I-beam cursor bug in their video driver (I use black backgrounds for my text editors and the I-beam is black by default making it invisible). So you’ll need to change your text selection cursor to one of the I-beams that have a white ‘halo’.

8. DoubleCommand lets you remap keys on the keyboard. I hate the fact that under Windows the Option key is treated as the ALT key. So I changed the mapping using DoubleCommand to swap the Option / Command keys on the left side of the keyboard and the Enter / Command keys on the right side of the keyboard. This makes it a bit more awkward to use Mac applications, but it makes my Windows experience much better (more muscle memory there). It would rock if Parallels would automatically remap the Command key to ALT inside of their virtual machines that run Windows (or make this configurable).

Update: I no longer need DoubleCommand, as I am now remapping those keys in Windows using a registry hack