RubyConf: Sydney and Rubinius

Posted by Nick Sieger Fri, 20 Oct 2006 19:07:00 GMT

Update: Evan has posted code and has a page set up for the project.

Evan Phoenix (nee Webb), of Seattle.rb, is presenting on Sydney and Rubinius, an experiment in improving the ruby interpreter. Sydney has died, and Rubinius has risen from its ashes, appropriately.

Why

Why would you write a new Ruby interpreter? It’s fun, it’s a good challenge.
What’s wrong with the existing interpreter -- are you hating on Matz? Of course not.

Today’s Ruby interpreter is like a big dump truck -- sometimes a little slow, but it works for us. YARV is like the red, shiny fire truck. Both big and complex. Rubinius, by comparison, is like a dune buggy. Fast, light, but you’re going to get sand in your eyes if you drive it a lot.

The project, admittedly, is naive.

Simple architecture and implementation.
As little background magic as possible
No opaque C backend
Leverage axiom of simple == powerful
Less magic means more introspection
- More control for the developer
- Richer introspection: Backtrace, MethodTable objects

What was Sydney?

Giant patch to 1.8.2 that included reentrancy and thread-safety
Turned out to be a major PITA
CRuby uses a large number of C globals, references to which had to be tracked and fixed

Transition to Rubinius

Ruby borrowed a lot from Smalltalk, so why not try an implementation based on the same concepts?
Prototype A ported the blue-book implementation to Ruby
It worked and validated the basic concept and approach
Prototype B took ideas from A but implemented a bytecode interpreter and compiler. Used RubyInline to access raw memory operations.
At this time the goal emerged to have a translator which could take a prototype and bootstrap itself into C code.
Prototype S was a manual translation of Prototype B into C code to make the implementation quicker.
Prototype W was created to translate parts of Prototype B so that there is a maintainable core in Ruby code itself.

Questions

Q. Since you were starting over, could you use a platform-independent library to ease the process, such as APR? Yes -- currently using String and PointerArray from glib.

Q. How is performance? Too early to tell -- I hope to know by the end of the conference. Prototype S became runnable and usable on the plane here.

Q. Can you clarify the goal? To create a Ruby interpreter in Ruby that can translate itself out into a C interpreter.

Q. Have you figured out how to link in external libraries in a platform independent way? No. My hope is that the decision will be made to write a common framework for translating to system calls, e.g., SWT.

Q. Have you looked at PyPy? (similar project for Python) Yes, and it’s f-in complicated. It worries me actually.

Q. Could you have it generate backend code in another language/platform (Java bytecode, CLR)? Yes, I certainly hope so, otherwise I’m wasting my time.

Q. How will you add native thread support in a cross-platform way? I hope I won’t have to, by leveraging external tools.

Q. If you’re building a Ruby-to-C translator, why write a Ruby interpreter at all? If I didn’t, what would I translate? You still need some core engine to translate. Would it be a subset of Ruby? Yes.

Q. Looks very similar to Squeak, have you looked at Squeak code and talked to Squeak people? Looked at code a lot, I’ve really stolen all of their ideas. I haven’t talked to the folks yet because I’m afraid they might laugh at me.

Resulting Works

SydneyParser: Used parser from Sydney and stole ParseTree’s algorithm for generating a sexp that represents the Ruby code.
SegfaultProtection: detects a segfault in an extension, saves the Ruby interpreter, and raises a memory fault exception instead.

The Nitty Gritty (Red Pill)

All components separated by APIs for swappability
Garbage collector: baker two-space copy collector, and a train GC
Bytecode interpreter: small set of instructions driven by tests and need, so there are no extraneous operations
Compiler: written completely in Ruby, using ParseTree and SexpProcessor. Intended to compile itself to be used as a base compiler for Prototype S.

Future

Other backends -- Java, Smalltalk

Nick Sieger