Posted by Nick Sieger
Fri, 05 Jan 2007 05:13:00 GMT
Overheard on #rubinius, MenTaLguY, the modern-day concurrent programming guru, on memory barriers:
MenTaLguY: as far as memory barriers go, the issue is that modern processors do all kinds of crazy stuff behind the scenes
MenTaLguY: reordering reads/writes, deferring writethroughs, etc
MenTaLguY: it’s like one of those shady delis where they don’t seem to do much actual deli business, but people are always going in and out and there’s a room in back full of cigar smoke
MenTaLguY: you walk in at the wrong time, and ... well
MenTaLguY: memory barriers are like loudly announcing “WHY, HELLO THERE OFFICER, NICE DAY TODAY, I’M JUST GOING TO THE DELI NOW” before you come in
Tags concurrency | no comments | no trackbacks
Posted by Nick Sieger
Tue, 02 Jan 2007 05:32:00 GMT
Update/Disclaimer: I refer to parts of RSpec that are not blessed as an extension API. Redefining before_context_eval and using the @context_eval_module variable directly may change in the future. I’ll keep this article updated to coincide with the changes. For now, these techniques should work fine with RSpec versions up to 0.7.5.
RSpec seems to be getting more attention lately as a viable, nay, preferred, alternative to Test::Unit. It’s possible that it’s just my personal feed-reader-echo-chamber, but consider this: Rubinius has started using RSpec alongside Test::Unit as an another way to test the alternate Ruby implementation. They’re even in the midst of building some snazzy extensions to allow the same specs to be run under a Ruby implementation of your choice. (Perhaps this will point the way to a new round of executable specs to accompany the fledgling community spec? Let’s wait and see how they do and leave that topic for another day.)
But extending and customizing RSpec to add a DSL on top of RSpec’s context/specify framework doesn’t have to be the realm of experts. Here are some templates for how you can DRY up your specs by adding your own helper methods in such a way that they will be available to all your specs. But first, a little background.
Spec Helper
Most usages of RSpec that I’ve seen in the wild use a “spec helper” (spec_helper.rb). This file, following the pattern of Rails’ test_helper.rb, minimally contains require statements to pull in the RSpec code and any supporting code for running specs. By requiring the spec helper via a path relative to your spec (usually with require File.dirname(__FILE__) + '/spec_helper' or similar), it also allows you the convenience of running your specs one at a time from anywhere (say, by launching from your editor) or with rake or spec. This file is where your shared helper methods will go, and where they’ll get registered to be pulled into the contexts.
What Context in context?
context "A new stack" do
specify "should be empty" do
end
end
How do those contexts work anyway? The context method that defines a context in which specs can be defined and run takes a block to define the individual specs, but what can really go in that block?
It turns out that RSpec jumps through metaprogramming hoops (using class_eval) to make the block behave like a class definition. This means you can do things like put method definitions inside your context:
context "A new stack" do
def a_new_stack
Stack.new
end
specify "should be empty" do
a_new_stack.should_be_empty
end
end
Which is nice, but the reason we’re here is to hide that away in spec_helper.rb. So, to get back to the point of the comment in the first example above, the self inside the context block is an anonymous Module object. It’s constructed in the initialize method of a Context (condensed from spec/runner/context.rb in the RSpec codebase):
class Spec::Runner::Context
def initialize(name, &context_block)
@name = name
@context_eval_module = Module.new
@context_eval_module.extend ContextEval::ModuleMethods
@context_eval_module.include ContextEval::InstanceMethods
before_context_eval
@context_eval_module.class_eval(&context_block)
end
def before_context_eval
end
end
(Take note of that empty before_context_eval method and the fact that it’s invoked during context initialization; that’s where we can plug in our custom extensions.)
The object held by the @context_eval_module instance variable is being augmented in two ways: extension and inclusion. The object is extended with the ContextEval::ModuleMethods module; these methods are being added to the object’s singleton class. This has the effect of making these methods visible within the context block, functioning similar to “class” methods.
The object also has the ContextEval::InstanceMethods module included. This has the effect of adding these as instance methods, making them visible from within specify blocks, which are made to behave like instance methods on the same object.
Putting it together
| Technique |
Visibility |
Use |
| @context_eval_module.extend | Context block | Custom setup, shared state declaration |
| @context_eval_module.include | Specify block | Shared actions/functions, stub/expectation modification, encapsulate instance variables |
Adding specialized setup methods
spec_helper.rb snippet:
module SharedSetupMethods
def setup_new_stack
setup do
@stack = Stack.new
end
end
end
class Spec::Runner::Context
def before_context_eval
@context_eval_module.extend SharedSetupMethods
end
end
Example spec:
context "A new stack" do
setup_new_stack
specify "should be empty" do
@stack.should_be_empty
end
end
Adding shared accessors
spec_helper.rb snippet:
module StackMethods
attr_accessor :stack
def push_an_object
stack << mock("some object")
end
end
class Spec::Runner::Context
def before_context_eval
@context_eval_module.include StackMethods
end
end
Example spec:
context "A stack with an object pushed" do
setup do
@stack = Stack.new
end
specify "should not be empty" do
stack.should_be_empty
push_an_object
stack.should_not_be_empty
end
end
The examples are simple, but hopefully illustrate the techniques. For an example of some code that’s actually useful, check out my sample RSpec Selenium RC integration project, in particular the spec helper and the example spec. (More on this in the future if it proves useful, but for now if you check it out and run rake on it, it should launch Selenium RC and run the example spec in a Firefox browser.)
By mixing and matching these techniques, you can layer a mini-DSL on top of RSpec and achieve DRY-er and even more readable and intention-revealing specs. Let me know if you’re able to find uses for these tips!
Tags metaprogramming, rspec, ruby | 2 comments | no trackbacks
Posted by Nick Sieger
Fri, 22 Dec 2006 05:17:30 GMT
Mirroring the wider global trend, Ruby has seen a big uptick of growth in my hometown of Minneapolis over the past year.
A little over one year ago, the very first meeting of the Ruby Users of Minnesota (affectionately known as “Java Programmers Anonymous”) was held. I was not in attendance (until the second meeting in late December), but by the January/February timeframe we had a surprisingly good showing of 15-20 people. We’ve sustained or exceeded that number since, packing our group in the back of a Dunn Brothers Coffee on Loring Park the last Tuesday of every month.
More interesting a metric is how the number of people doing Ruby full-time has progressed. Somewhere around spring-time, guys were starting to itch for work. By summer time, Slantwise Design, formerly a mostly-web-design shop had been hired to do its first Rails contract, for what would turn out to be Sayswap. Slantwise is a Rails-exclusive shop now.
Rails Day had an entry from Bruno Bornsztein and Ben Moore, Sneakology. This prolific duo would go on to contract on YFly, and play around with a local one-page news aggregator at Acu.mn. Almost two months ago, they went live with their new startup, Curbly, a social-networking DIY design community.
We’re also happy to lay claim to RUM regulars Charles Oliver Nutter and Thomas Enebo, the two lead JRuby developers, who tirelessly give monthly updates on the lightning progress that JRuby is undergoing.
All in all, there are at least 20 full-time Rubyists in the Twin Cities, and the number is going up week by week.
Yours truly has been doing Ruby mostly-full-time for my current employer Digital River, where I’m proud to have sneaked Ruby into the system. There are at least two other programmers writing Ruby code at DR, and hopefully we’ll be hiring more in 2007. If you’re looking for work or interested in what we’re doing with Ruby, drop me a line!
So what happened this year in other user groups around the world?
Tags minneapolis, ruby, stpaul, usergroup | no comments | no trackbacks
Posted by Nick Sieger
Fri, 01 Dec 2006 18:56:00 GMT
The prospect of doing behavior-driven development in Java has just taken a step closer with the news of RSpec running on JRuby. This is already a big step that will have an impact on Ruby and Java programmers alike in a number of ways.
However, it could be even better. RSpec has a nice, intuitive mocking API, which will unfortunately, at the present time, be useless when working with java objects. It would be awesome to try to get it to work, though. Some possibilities:
- Map to JMock and use JMock under the hood. Not a very attractive option for a number of reasons, but mainly because add-on bridging layers are complex and should be avoided.
- Improve ability for JRuby to implement any number of Java interfaces dynamically.
This second option is something Charlie, Tom and I talked about on Tuesday night, that could have a much broader impact on Java integration in JRuby.
Consider this spec. It’s trivial, but bear with me.
context "A TaskRunner" do
setup do
@task = mock("Runnable")
@task_runner = TaskRunner.new(@task)
end
specify "runs a task when executed" do
@task.should_receive(:run)
@task_runner.execute
end
end
This spec might be satisfied by the following Java code:
public class TaskRunner {
private Runnable task;
public TaskRunner(Runnable r) {
this.task = r;
}
public void execute() {
task.run();
}
}
Notice how I defined the @task in the spec above. This is the normal way of mocking in RSpec, and the example illustrates how I think JRuby should handle interfaces in Java: by duck-typing them.
Basically, the RSpec mock should act like a Java Runnable because I’ve defined a run method on it (in this case implicitly with @task.should_receive(:run)). JRuby could wrap a dynamic invocation proxy around any Ruby object just before passing it into a Java method invocation. Without doing any type- or method-checking up front. Just define the proxy as implementing the interface required by the Java method signature, and let the JRuby runtime do its thing, and attempt to resolve methods as they’re invoked. Possibly falling back to method_missing, even!
Note that this would also make moot the multiple interface syntax discussion, because you’d never have to declare an object in JRuby as implementing any particular interface. Just define the appropriately named methods with the proper arity, and you’re done. Maybe you don’t even need to declare all of them, if they never get called for your usage! This is the Ruby Way, and would be a completely natural extension to the way Java objects are manipulated in JRuby today, not to mention extremely concise and powerful.
This would also allow RSpec mocking to just work, at least for Java interface types, which would be way cool.
Charlie has a Swing demo that he frequently gives when talking about JRuby. Under the new proposal, it would look more like this:
require 'java'
frame = javax.swing.JFrame.new("Hello")
frame.setSize(200,200)
frame.show
button = javax.swing.JButton.new("OK!")
frame.add(button)
frame.show
def actionPerformed(event)
event.source.text = "Pressed!"
end
button.addActionListener self
With luck, this approach will be coming to JRuby very soon.
Tags jruby, mockobjects, rspec | no comments | no trackbacks
Posted by Nick Sieger
Tue, 28 Nov 2006 04:24:00 GMT
The other day I was setting up a demo installation of FishEye at work. I don’t have root on the box where it’s running, so none of the built-in authentication schemes were acceptable. Hey, there is already user and password information stored in the CVSROOT directory of the repository (we use the pserver access method). But, ughh, I’d have to read and parse files in Java, and grab a public domain crypt implementation since it’s not included in the Java standard library. Certainly do-able, but not my idea of fun.
So why not use Ruby for this? Painless IO and text-munging? Check. Crypt? Check. In Java? Since Charlie’s Ruby-in-a-Jar, this one is a slam-dunk!
For the implementation, have a look at CVSPasswdFisheyeAuthenticator.java and authenticator.rb. The basic idea is to use the Java class as the embedding wrapper and leave all the main logic in Ruby. The Java wrapper is still a little heavy, since the actual interface is implemented by the Java class, and Ruby is invoked by evaluating a statement inside each interface method. Some notable elements:
CVSPasswdFisheyeAuthenticator#init: The interpreter is initialized here. I pass authenticator properties to the interpreter by way of globals here -- not the cleanest, but easy. The Ruby code is loaded as a classpath resource, a nice feature of JRuby’s require that allows Ruby scripts to be jarred up with the rest of your code.CVSPasswdFisheyeAuthenticator#rubyEval: This is your boilerplate-invoking-the-interpreter code here.CVS::Authenticator#initialize: Reading and parsing of the CVS passwd and users files is done here in five succinct lines, much better than you could do with Java.CVS::AuthToken: Here I’m implementing a Java interface in Ruby, and passing it back to Java! (See the CVS::Authenticator#recreate_auth method.) This is probably the nicest single aspect of JRuby’s Java integration, and it will only get better.
What would be nice for a future implementation is to make the Java class a factory and a delegate, and let JRuby implement the entire interface itself directly. This would avoid some ugly boundary-crossing issues (see the hand-crafted Ruby strings passed into the intepreter and rubyToJava conversions on the way out). As soon as JRuby gets better support for multiple interfaces and extending abstract classes, this sort of pattern could probably be templated rather easily.
If you want to build the whole thing, check out the source with Subversion and follow the README file for instructions.
For another example of this kind of embedding of JRuby in the wild, see the JRuby support in NanoContainer.
Tags fisheye, jruby | no comments | no trackbacks
Posted by Nick Sieger
Wed, 15 Nov 2006 15:46:00 GMT
Inspired by a posting on the RSpec list and recent comments stating that my Auto RSpec hack wasn’t working, I’ve bitten the bullet and upgraded to RSpec 0.7.2, and made rspec_autotest a plugin in the process. So, herewith are the necessary incantations to auto-rspec your project. If you’ve tried my hack already, please remove any bits you previously had installed.
- Install RSpec on Rails, following the original instructions. As of RSpec 0.7.3, the specific version of ZenTest is no longer required. Also, diff-lcs is required to show unified diff output on
should == failures.
gem install zentest -v 3.4.1
gem install diff-lcs
gem install rspec
script/plugin install svn://rubyforge.org/var/svn/rspec/tags/REL_0_7_2/vendor/rspec_on_rails/vendor/plugins/rspec
script/plugin install http://svn.caldersphere.net/svn/main/plugins/rspec_autotest
Please let me know if you experience any problems!
Tags plugin, rails, rspec | 15 comments | no trackbacks
Posted by Nick Sieger
Thu, 02 Nov 2006 02:12:00 GMT
Update: Koz already fixed the issue in trunk, and the changes are also going into the 1.2 release as well. Thanks!
Man, I think I’ve been reading too much Sam Ruby lately (ok, that was a year ago, but not much has changed). You have to admit, though, that XML handling in Ruby is one of those things that just doesn’t feel quite right. REXML is pretty much the standard API for Ruby, yet it suffers from two showstoppers in my opinion:
In Ruby 1.8.4 it still has the glaring hole Sam mentioned last year with well-formedness. (No exception raised below!)
irb(main):001:0> require 'rexml/document'
=> true
irb(main):002:0> d = REXML::Document.new '<div>at&t'
=> <UNDEFINED> ... </>
irb(main):003:0> d.root
=> <div> ... </>
irb(main):004:0> d.root.text
=> "at&t"
The REXML::Text#to_s method violates the principle of least surprise. In just about every other XML parser written, when you ask a
text node for its contents, it returns you the value with entities resolved. Not so Text#to_s. You have to call Text#value
instead. Unfortunately, this would be difficult to reverse in future versions of REXML without breaking existing apps.
irb(main):001:0> require 'rexml/document'
=> true
irb(main):002:0> t = REXML::Text.new('at&t')
=> "at&t"
irb(main):003:0> t.to_s
=> "at&t"
irb(main):004:0> t.value
=> "at&t"
This second problem manifests itself in subtle ways. If you’re calling Element#text (which is probably the most common way), you’re fine, because it implicitly does self.texts.first.value under the hood. But if you want to make sure you’re grabbing all the text content, you might be inclined to write element.texts.join('') to concatenate them together. But this method bypasses the value method and instead uses to_s, leaving you with unresolved entities.
It turns out this problem is exhibited in the version of XmlSimple now included with Edge Rails as of rev 4453. So if
you’re living on the edge using the newly minted ActiveResource fetching XML from remote resources like a champion, you just got benched
as soon as you tried to fetch XML that had normalized entities inside.
XmlSimple version 1.0.9 has a partial fix for this issue, but I submitted another patch to Maik Schmidt for review that he subsequently released as 1.0.10. I’ve attached the 1.0.10 version to ticket 6532 in hopes that it will be patched in Rails soon.
Posted in ruby, rails | Tags ruby, xml | no comments | no trackbacks
Posted by Nick Sieger
Fri, 27 Oct 2006 16:48:00 GMT
As part of the momentum surrounding the Ruby implementer’s summit, I have decided to take on a pet project to understand Ruby’s grammar better, with the goal of contributing to an implementation-independent specification of the grammar. Matz mentioned during his keynote how parse.y was one of the uglier parts of Ruby, but just how ugly?
Well, judge for yourself. Below is a grammar dependency graph generated using ANTLRWorks and GraphViz. The steps I took are as follows. I took parse.y, stripped all C definitions, code and actions from it to give a bare YACC definition. Next, I did the equivalent of gsub(/[kt]([A-Z]+)/, '1') (since ANTLR’s convention is to have lexer tokens named starting with a capital letter). I then used the Bison-to-ANTLR converter to generate an ANTLR 2.x grammar, which I hand-modified to produce a v3 grammar. Opening the resulting grammar in ANTLRWorks allows you to generate a DOT file from which GraphViz can then generate a jpeg image. I’ve also included visualizations of the Java 1.5 and Javascript (ECMAScript) grammars for comparison.
I haven’t even begun to absorb all the meanings from this picture, but one stark difference between Ruby and the other two is the node in the middle of the picture with a high concentration of outgoing edges. That node is called primary in the grammar definition, and it is probably one of the reasons that Ruby syntax is so flexible and forgiving. A primary node’s direct children apparently represent a large portion of the syntax, and explain why in Ruby a single statement can either be a literal, a method invocation (or series of them), a standalone expression (such as a < b), all the way up to larger syntactic groupings such as if ... else ... end and begin ... rescue ... end, among many others.
Ruby
Java 1.5
Generated from Java 1.5 grammar on antlr.org
Javascript
Generated from ECMAScript grammar on antlr.org
Posted in ruby | Tags ruby | 34 comments | no trackbacks
Posted by Nick Sieger
Tue, 24 Oct 2006 22:51:55 GMT
Whew! Back home from my first RubyConf, it’s taken me a couple days to collect some parting thoughts. As you might have noticed, I was pretty busy last weekend.
First of all, what an awesome and welcoming community. It’s going to sound cliché, but there are so many intelligent and motivated people walking around that you can’t help but be inspired to roll up your sleeves and get your hands dirty.
There were definitely some high points for me. The beauty and power of the language, even after using it for almost two years, still amazes me. Pretty much every piece of code I saw, whether in a presentation or looking over someone’s shoulder, had a clear purpose and communicated its intent better than any general-purpose machine language I have seen. The simplicity of Evan’s new Ruby-in-ruby VM, the syntax integration tricks of John’s RubyCLR project, the forthcoming RubyOSA APIs, and Geoffrey’s graphics programs, are all great testaments to Ruby’s power.
There was an implementer’s summit on Friday night, which I attended (see also coverage here and here). There are now at least 8 active implementations of Ruby (Ruby, Yarv, JRuby, Cardinal, Rubinius, MetaRuby, Ruby.NET, IronRuby), and two interop bridges (RubyCLR and RubyCocoa)! The biggest news was that there are plans to revive the Ruby testing project (formerly the Rubicon) and share as many tests as possible among the implementations.
RejectConf was a huge success, due largely to the indefatigable Adam Keys. Kevin Tew has a decent wrap-up of the talks that occurred. Charlie’s demo of NetBeans in-place refactoring feature drew a couple oohs and ahs and even one f-bomb. Heckle, in time, should be an awesome tool as well. Big thanks to zenspider for coordinating it. It’s destined to become an annual tradition. Perhaps the organizers of future RubyConfs could account for it in the budget?
On a lighter note, there were quite a few humorous moments that kept popping up. A summary may read like a list of inside jokes, so here’s some context. THAT GUY is a reference to a disclaimer in Zed’s talk about the know-it-all guy who always pipes up during your talk with skepticism. THAT GUY kept getting called out during the rest of the conference. Ani, the developer evangelist from Microsoft was pretty thick-skinned. She was heckled constantly about MS, Vista, and everything else, and still kept a smile on her face. And of course you already watched Adam’s one-act play, right?
My note-taking streak wasn’t quite perfect; I didn’t take notes Kevin’s mkmf talk nor Rich’s talk about indi, and I slept in and missed Justin’s Streamlined talk. Also, the beer was flowing for RejectConf, and despite the quality summer of code talks, I was spent. Fortunately, you can fill in the blanks by following along with Curt Hibbs and the rest of the blogosphere. Thanks for tuning in, and I hope you got something worthwhile here. See you next year!
Tags rubyconf, rubyconf2006 | 2 comments | no trackbacks
Posted by Nick Sieger
Mon, 23 Oct 2006 14:16:00 GMT
John Lam wanted to build a photo-flash-card application using Avalon and Indigo and Flickr, but also using Ruby as the implementation language. So along the way he decided to build an interop layer (a bridge) between Ruby and the CLR to do it.
Now that John has joined Microsoft, his new mission (bigger picture) is to further dynamic language implementations on the CLR.
Bridging type systems
Dynamic methods in the CLR allow you to do better than simply invoking the reflection API.
Ruby | C | CLR
============================================
shadow class | dynamic method | instance
Polymorphic inline caching -- caching method dispatches on different call sites based on the assumption that types don’t change that often
- Generate shadow classes and method stubs using
const_missing and method_missing
- Overload resolution happens in the method shims (a one time cost) to choose, e.g., which constructor to use for
System::Collections::ArrayList.new
- Integration is done to make the CLR feel more Rubyish
Implementation
This changes identity (proxied object):
ArrayList.new.as(IEnumerable)
This is less Rubyish, but identity is preserved:
IEnumerable.get_enumerator(ArrayList.new)
There are trade-offs and warts to a bridge approach to Ruby integration on top of a platform such as the CLR: there is a need to inject artificial type information occasionally to be able to construct CLR objects (e.g., arrays -- Array.of(Int32).new(3)). Generics are evil! Simple stuff doesn’t seem so bad: List.of(Int32).new, but there’s more pain to be had (see John for details). John also built a RubyInline-like implementation for the CLR languages too, to allow for getting things done (even if it’s dirty). Finally, method overloading is a problem, especially when there is no equivalent Ruby type -- this gave way to instance_shim which is a sort of aliasing method that mixes in type metadata for use by the interop layer.
On the other hand, there are many places where Ruby (even in bridged mode) can make the experience of developing on the CLR better. Implementing CLR interfaces is a feature that allows Ruby objects to cross to the CLR side, (e.g., adding IEnumerable to Ruby Array). Performance across the CLR boundary (marshalling data) is ~100 times slower than C#, but still fast (3 million calls/second). Huge benefits are gained from using DSLs in Ruby to help with the implementation of the interop layer. Also, RubyCLR allows mixing in methods into CLR types, so we can re-skin APIs that feel clunky in Ruby. This is really leveraging the power of Ruby in the best possible way.
My take is that it looks like the RubyCLR project will probably not be seeing much further development, unless John finds a willing maintainer -- but this is speculation, I haven’t confirmed with John. Yet, the problem of impedance matching between type systems is a recurring theme in the dynamic language arena, and so John’s work is valuable in helping us to understand this issue.
More Info
Posted in ruby | Tags rubyconf, rubyconf2006 | 3 comments | no trackbacks
