Nick Sieger: Tag rubytag:blog.nicksieger.com,2005:TypoTypo2007-07-13T09:45:26+00:00Nick Siegerurn:uuid:97cbdf32-ed82-430e-a874-f7fd98b2e9992007-06-11T15:35:00+00:002007-07-13T09:45:26+00:00Test Your Rake Tasks<p>Many libraries and plugins ship custom Rake tasks. Of course, as slick as Rake is for a build and configuration language, it’s still just Ruby code right? </p>
<p>Case in point: I released a version of <a href="http://caldersphere.rubyforge.org/ci_reporter"><code>ci_reporter</code></a> with a fairly careless bug in a rake task that attempted to <code><<</code> a string into an existing environment variable. It escaped me at the time that Ruby sets up the <code>ENV</code> hash with frozen strings, because my own usage of ci_reporter did not exercise the task in that way.</p>
<p>So shouldn’t that Ruby code be subjected to the rigor of automated testing just like the rest of your code? It became obvious to me that it must be so. It turns out it’s straightforward to use Rake in an embedded fashion, and invoke targeted tasks in your custom Rake recipes. The examples here use RSpec, since that’s what I use for testing <code>ci_reporter</code>, but you could apply this to <code>Test::Unit</code> as well.</p>
<p>The technique is to create a new instance of <code>Rake::Application</code>, make it the active application, and load your rake scripts into it:</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="ident">describe</span> <span class="punct">"</span><span class="string">ci_reporter ci:setup:testunit task</span><span class="punct">"</span> <span class="keyword">do</span>
<span class="ident">before</span><span class="punct">(</span><span class="symbol">:each</span><span class="punct">)</span> <span class="keyword">do</span>
<span class="attribute">@rake</span> <span class="punct">=</span> <span class="constant">Rake</span><span class="punct">::</span><span class="constant">Application</span><span class="punct">.</span><span class="ident">new</span>
<span class="constant">Rake</span><span class="punct">.</span><span class="ident">application</span> <span class="punct">=</span> <span class="attribute">@rake</span>
<span class="ident">load</span> <span class="constant">CI_REPORTER_LIB</span> <span class="punct">+</span> <span class="punct">'</span><span class="string">/ci/reporter/rake/test_unit.rb</span><span class="punct">'</span>
<span class="keyword">end</span>
<span class="ident">after</span><span class="punct">(</span><span class="symbol">:each</span><span class="punct">)</span> <span class="keyword">do</span>
<span class="constant">Rake</span><span class="punct">.</span><span class="ident">application</span> <span class="punct">=</span> <span class="constant">nil</span>
<span class="keyword">end</span>
<span class="comment"># ...</span>
<span class="keyword">end</span></code></pre></div>
<p>Notice the use of <code>#load</code> rather than <code>#require</code>, as you want to execute your rake script each time you setup the Rake application object. When tearing down your test or example, you should cleanup Rake by setting the <code>Rake.application</code> back to nil (or save the previous application and restore it, if you prefer).</p>
<p>Now, in the body of your test or example, you invoke your rake task with <code>@rake['target'].invoke</code>. Here, I’m exercising the case of an existing, frozen <code>ENV</code> value. After the task is invoked, I check the value after the task to make sure the variable was modified as expected.</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="ident">it</span> <span class="punct">"</span><span class="string">should append to ENV['TESTOPTS'] if it already contains a value</span><span class="punct">"</span> <span class="keyword">do</span>
<span class="constant">ENV</span><span class="punct">["</span><span class="string">TESTOPTS</span><span class="punct">"]</span> <span class="punct">=</span> <span class="punct">"</span><span class="string">somevalue</span><span class="punct">".</span><span class="ident">freeze</span>
<span class="attribute">@rake</span><span class="punct">["</span><span class="string">ci:setup:testunit</span><span class="punct">"].</span><span class="ident">invoke</span>
<span class="constant">ENV</span><span class="punct">["</span><span class="string">TESTOPTS</span><span class="punct">"].</span><span class="ident">should</span> <span class="punct">=~</span> <span class="punct">/</span><span class="regex">somevalue.*test_unit_loader</span><span class="punct">/</span>
<span class="keyword">end</span></code></pre></div>
<p>I was fortunate here that the tasks for which I wrote tests after the fact were simple enough to be testable on their own, which may not always be the case, especially with organic, homegrown Rake tasks that interact with the world outside of Ruby. Still, if your Rake tasks are a critical part of your application, library or plugin, they should be tested. For example, it would be nice if tests could be written for the Rake scripts in Rails’ Railties module to increase coverage there.</p>
<p>Perhaps someone out there will run with this idea and take up the challenge and write a Rakefile completely in a test-driven or behaviour-driven style. It’s always been a sore point for me with Make, Ant, Maven, and virtually every other build tool in existence that you have no other way of automatically verifying your build script is doing what you intended without manually running it and inspecting its output – it just feels so dirty! I’d expect that test-driven Rake scripts would likely have the level of granularity to match the tasks that need to be done, in a way that you can combine them in the right ways to make incremental and deconstructed builds simpler.</p>Nick Siegerurn:uuid:774d62fd-eba2-4586-8da6-cbde414851b72007-06-10T04:03:00+00:002007-07-13T09:45:26+00:00JRuby 1.0<p>I’d just like to take a moment to echo what <a href="http://ola-bini.blogspot.com/2007/06/jruby-10.html">Ola has to say</a> about the JRuby 1.0 release. This one is definitely for all of you out there. It’s been incredibly gratifying to see the growth of the community, and the increased amount of positive feedback and success stories with JRuby, and I’m honored to have been part of the team that made 1.0 happen.</p>
<p>We really feel strongly that we’ve put out a quality piece of software, a tool that will make your work more enjoyable, easier, and allow you to inject some creativity and innovation back into the Java stack.</p>
<p>We’ve got a solid base to start from. <a href="http://www.headius.com/jrubywiki/index.php/JRuby_on_Rails">Being able to run Rails</a> is no small feat, to be sure, but the best is yet to come. You can expect more performance, a complete compiler, support for more applications, and tighter integration with long-standing Java technologies. In addition, we’d like to push the envelope of what both Ruby and Java are capable of, including implementing (even driving) Ruby 2.0 features, leading the way for dynamic language support in the JVM, eased as well as novel ways of doing application deployment, better debugging and tooling, and experiments with new ways of doing concurrent and parallel computing.</p>
<p>Do join up with us – it’s never too late to hop in and enjoy the fun!</p>Nick Siegerurn:uuid:08507152-87fb-40ee-9e73-b3644c2c56192007-05-24T17:16:05+00:002007-07-13T09:45:26+00:00Rare Rubies<p>Stumbling upon a description of a <a href="http://www.mnh.si.edu/exhibits/ruby/index.htm">rare Burmese Ruby gemstone housed in the Smithsonian</a>, this line popped out at me:</p>
<blockquote>
<p>While sapphire, emerald and diamond gems weighing hundreds of carats exist, high quality Burmese rubies larger than 20 carats are exceedingly rare.</p>
</blockquote>
<p>We could rephrase that a bit:</p>
<blockquote>
<p>While Java, C++, and C# programs weighing hundreds of thousands of lines of code exist, high quality Ruby programs larger than 2000 lines are exceedingly rare.</p>
</blockquote>
<p>Isn’t it strange how you <a href="http://en.wikipedia.org/wiki/Snowclone">hardly notice a difference</a>?</p>
<p><img src="http://www.mnh.si.edu/exhibits/images/ruby/main_ruby2.jpg" alt="Carmen Lucia Ruby"/></p>Nick Siegerurn:uuid:ab53b976-cff8-410d-8047-793abbb363a22007-05-23T05:51:36+00:002007-07-13T09:45:26+00:00geekSessions I: Ruby on Rails: To Scale or Not to Scale<p>I was fortunate to be in town right after RailsConf and attended the inaugural <a href="http://www.geeksessions.com/">geekSessions</a> event on Rails scalibility. The event went off without a hitch: it was well attended, City Club is a classy place, and there was decent food and an open bar. I don’t know the SF geek/startup scene, but pretty much all of the few guys I know were there along with a ton of other folks. My only complaint would have been to let it run at least 30 minutes longer. Socializing was good too, but it seemed like the conversation was just getting started.</p>
<p>Here are some notes for you in my typical rapid-fire style – hope they’re useful to you.</p>
<h2>Ian McFarland</h2>
<p>Case study: divine caroline</p>
<p>Servers:</p>
<ul>
<li>Load balancer</li>
<li>Apache + mongrel</li>
<li>MySQL</li>
<li>SOLR</li>
</ul>
<p>Ruby is slow. Rails is slow. Unoptimized app was slow – 7 pages/sec with <code>ab</code>. So how can Rails possibly be? 150 pv/s with a simple text render. This formed a sort of upper-bound, that ruled out fragment/action/partial caching, etc. This brought the throughput to 3500 pv/s. Except for page caching limitations:</p>
<ul>
<li>Cache coherency</li>
<li>Writes are more expensive</li>
<li>Page caching is not applicable to as many pages as you think</li>
</ul>
<p>But measure first. Pivotal built a drop-in page caching extension to deal with cache coherency issues (soon to be at http://rubyforge.org/projects/pivotalrb)</p>
<h2>Jason Hoffman</h2>
<p>Jason somehow has the distinction of the first four commits in the Rails repository. Joyent/TextDrive/Strongspace.</p>
<p>If your application is successful, you’re going to have a lot of machines. What happens when you have 1000s of machines, 100s of TB, 4 locations, etc. Is this really a <em>Rails</em> issue? In a typical Joyent setup, Rails is only one of 26+ processes on the server stack. So scaling it really doesn’t mean much more than scaling any application. Object creation in Ruby is fast, sockets and threads are slow. So forget sockets and threads.</p>
<p>Instead, use DNS, load balancers, evented mongrels, JRuby/Java, DBMSes (not just RDBMS; LDAP, filesystem, etc.), Rails process doing Rails only, static assets going through a static server, federate and separate as much as you can.</p>
<h2>Jeremy LaTrasse</h2>
<p>Jeremy’s job is about safety nets; about knowing the underlying infrastructure. Is the hardware/OS/stack important? Can you build safety nets around those so that you can spare cycles when you need to intrude into the system to troubleshoot?</p>
<p>Twitter is in a unique position with the volume of traffic to be able to find some pretty tough bugs, like the recent <a href="http://dev.rubyonrails.org/changeset/6571">backtrace issue</a>.</p>
<h2>Bryan Cantrill</h2>
<p>Measure first! Like Ian said. Is software information? Or a machine? It’s both. Nothing else in human existence can claim this. 3 weeks after Bryan joined Sun, he was working with Jeff (ZFS architect) debugging an issue when Jeff retorted, “Does it bother you that none of this exists? It’s just a representation of some plastic and metal morass in a backroom” (slightly paraphrased).</p>
<p>We’ve been living with bifurcated code – “if DEBUG; print something” ad nauseum. But this has a cost. So dev code deviates from production code. But we can’t get the data we want, where it matters, in production. Bryan goes on to describe the aforementioned <a href="http://dev.rubyonrails.org/changeset/6571">backtrace issue</a> and how it saved Twitter 33% CPU. So don’t pre-optimize, but you’ve got to be prepared to go get the data. In production.</p>
<h2>Q & A</h2>
<p><em>What’s the best way to move from one database to two databases (MySQL), when you scale past the volume of reads that overwhelms one?</em></p>
<p><strong>Jason</strong> doesn’t like the replication approach, it’s not fault tolerant. Reference to <a href="http://drnicwilliams.com/2007/04/12/magic-multi-connections-a-facility-in-rails-to-talk-to-more-than-one-database-at-a-time/">Dr Nic’s magic multi-connections gem</a>. Reference to <a href="http://revolutiononrails.blogspot.com/2007/04/plugin-release-actsasreadonlyable.html">acts_as_readonly</a>. Don’t rely on things that are out of your control, start reading/writing to multiple locations, at the application level. <strong>Jeremy</strong>: So do you want to be in the business of writing SQL or C extensions to Rails? What about <a href="http://freshmeat.net/projects/mysql_proxy/">MySQL proxy</a>? Seems ok, but I might not trust it in production. <a href="http://jeremy.zawodny.com/mysql/mytop/" title="mytop - a top clone for MySQL">MyTop</a>/<a href="http://www.xaprb.com/blog/2006/07/02/innotop-mysql-innodb-monitor/">InnoTop</a> will tell you about your query volume.</p>
<p><em>Virtualization: 4 virtual servers w/ web servers on top of a single physical server? Why?</em></p>
<p><strong>Jason</strong>: Free BSD 4.9 on early pentium was the perfect balance of utilization. 18 CPUs by 64G RAM with virtual servers gets us back to that level of utilization. <strong>Bryan</strong>: Not all virtualization solutions are equivalent! (Solaris containers/zones plug.)</p>
<p><em>RDBMSes are not good for web applications? Why? Can you give some examples?</em></p>
<p><strong>Jason</strong>: It depends on when you want to join. When people are clicking, or pre-assembled. Look at your application and put the data together before people request it. Why does YouTube need an RDBMS? It serves a file that people can comment on.</p>
<p>Mention of Dabble DB, ZFS, Jabber, Atom, Atom over Jabber, etc. as ways of innovative ways of storing objects, data, etc. GData/GCal most certainly does not store its Atom files in an RDBMS.</p>
<p><em>Sell Rails apps and have the customer deploy it? What options are available?</em></p>
<p><strong>Ian</strong>: JRuby on Rails with a .war file is an interesting approach. <em>What operational issues/ways to help with scaling remote deployments?</em> <strong>Jeremy</strong>: Log files are the first line of defense. <strong>Jason</strong>: Corporate IT are comfortable with Java.</p>
<p><em>The pessimist in me says that my servers are going to fall over after 5 users. How can I be prepared/not be optimistic about a traffic spike?</em></p>
<p><strong>Ian</strong>: Load test the crap out of the app. Find out the horizontal scaling point. Use solutions like S3 for images. Make sure you can scale by throwing hardware at it. Eventually single points of failure will overcome you (such as a single database), but you can wait until you get to that point before doing something about it.</p>
<p><strong>Jason</strong>: You can benchmark your processes, and get an idea of what they can do. Most people that want to do something will be look at your stuff, and maybe signup. So front-load and optimize your signup process, possibly by taking it out of Rails.</p>
<p><strong>Jeremy</strong>: Conversations with Zed, DHH, etc. have pointed out that sometimes “Rails isn’t good at that, take it out of Rails.” Same thing for the database. Split those things out into a different application.</p>
<p><strong>Bryan</strong>: Do your dry land work, know your toolchain, so that when the moment comes, you can dive in and find the problem.</p>
<p><em>We have a migration that takes a week to run because of text processing. GC was running after every 10th DB statement. Used Rails bench GC patch to overcome the issue with the migration. Any issue running these?</em></p>
<p><strong>Jason</strong>: We run those GC modifications and a few more in production, and they’re fine.</p>
<p><em>Most comversations revolve around items like database is slow, or Ruby is slow. How can we use DTrace to streamline the process?</em></p>
<p><strong>Jeremy</strong>: We spent 20 minutes over lunch (plus some preparation) to find a Memcache issue. It’s worth it to spend a little time to learn the tool.</p>
<p><strong>Bryan</strong>: “Awk is God’s gift to all of us.” When DTrace was being reviewed inside of Sun, folks commented “This reminds us of awk.” “Thanks!”</p>
<p><strong>Jason</strong>: We’re putting a tracing plugin in Rails as a remote process to collect data from a running app. Apple has shown a commitment to get this in Leopard. Textual and graphical output are possible. I believe in DTrace a lot, and the tooling and documentation will go beyond its current state of an experts tool.</p>
<p><em>Lastly, what one closing thing would you like to say about Rails scalability?</em></p>
<p><strong>Ian</strong>: Measure.<br/>
<strong>Jason</strong>: Don’t use relational databases.<br/>
<strong>Jeremy</strong>: I thought it was a Joyent sales pitch.<br/>
<strong>Bryan</strong>: Use DTrace (with Joyent accelerators of course).<br/></p>Nick Siegerurn:uuid:0aaea16b-fda8-4754-a306-329aa9187d872006-10-27T16:48:00+00:002007-08-31T16:31:57+00:00Visualization of Ruby's Grammar<p>As part of the momentum surrounding the <a href="http://on-ruby.blogspot.com/2006/10/rubyconf-2006-implementers-summit.html">Ruby implementer’s summit</a>, I have decided to take on a pet project to understand Ruby’s grammar better, with the goal of contributing to an implementation-independent specification of the grammar. Matz mentioned during his keynote how <a href="/articles/2006/10/22/rubyconf-matz-keynote">parse.y was one of the uglier parts of Ruby</a>, but just how ugly?</p>
<p>Well, judge for yourself. Below is a grammar dependency graph generated using <a href="http://www.antlr.org/works/index.html">ANTLRWorks</a> and <a href="http://www.graphviz.org/">GraphViz</a>. The steps I took are as follows. I took parse.y, stripped all C definitions, code and actions from it to give a bare YACC definition. Next, I did the equivalent of <code>gsub(/[kt]([A-Z]+)/, '1')</code> (since ANTLR’s convention is to have lexer tokens named starting with a capital letter). I then used the <a href="http://www.antlr.org/share/list">Bison-to-ANTLR converter</a> to generate an ANTLR 2.x grammar, which I hand-modified to produce a v3 grammar. Opening the resulting grammar in ANTLRWorks allows you to generate a DOT file from which GraphViz can then generate a jpeg image. I’ve also included visualizations of the Java 1.5 and Javascript (ECMAScript) grammars for comparison.</p>
<p>I haven’t even begun to absorb all the meanings from this picture, but one stark difference between Ruby and the other two is the node in the middle of the picture with a high concentration of outgoing edges. That node is called <code>primary</code> in the grammar definition, and it is probably one of the reasons that Ruby syntax is so flexible and forgiving. A primary node’s direct children apparently represent a large portion of the syntax, and explain why in Ruby a single statement can either be a literal, a method invocation (or series of them), a standalone expression (such as <code>a < b</code>), all the way up to larger syntactic groupings such as <code>if ... else ... end</code> and <code>begin ... rescue ... end</code>, among many others.</p>
<h2>Ruby</h2>
<p><a href="http://www.flickr.com/photos/nicksieger/280661836/" title="Photo Sharing"><img src="http://static.flickr.com/93/280661836_e477a01932.jpg" width="500" height="290" alt="Ruby 1.8.4 grammar dependency graph" /></a></p>
<h2>Java 1.5</h2>
<p><em>Generated from <a href="http://antlr.org/grammar/1152141644268/java.g">Java 1.5 grammar on antlr.org</a></em></p>
<p><a href="http://www.flickr.com/photos/nicksieger/280662707/" title="Photo Sharing"><img src="http://static.flickr.com/119/280662707_5d335ac808.jpg" width="500" height="462" alt="Java 1.5 grammar dependency graph" /></a></p>
<h2>Javascript</h2>
<p><em>Generated from <a href="http://antlr.org/grammar/1153976512034/ecmascriptA3.g">ECMAScript grammar on antlr.org</a></em></p>
<p><a href="http://www.flickr.com/photos/nicksieger/280662871/" title="Photo Sharing"><img src="http://static.flickr.com/109/280662871_a53a2680ce.jpg" width="367" height="500" alt="ECMAScript (Javascript) grammar dependency graph" /></a></p>Nick Siegerurn:uuid:d060e313-e46a-49dc-9093-fcb0f2ae77832006-11-02T02:12:00+00:002007-08-31T16:41:36+00:00Ruby and XML not-so-simple?<p><em>Update: <a href="http://www.koziarski.net/">Koz</a> already <a href="http://dev.rubyonrails.org/changeset/5414">fixed the issue in trunk</a>, and the changes are also <a href="http://dev.rubyonrails.org/changeset/5415">going into the 1.2 release</a> as well. Thanks!</em></p>
<p>Man, I think I’ve been reading too much <a href="http://intertwingly.net/blog/2005/09/14/Last-Line-of-Defense/">Sam Ruby</a> lately (ok, that was a year ago, but not much has changed). You have to admit, though, that XML handling in Ruby is one of those things that just doesn’t feel quite right. REXML is pretty much the standard API for Ruby, yet it suffers from two showstoppers in my opinion:</p>
<ul>
<li><p>In Ruby 1.8.4 it still has the glaring hole Sam mentioned last year with well-formedness. (No exception raised below!)</p>
<pre><code>irb(main):001:0> require 'rexml/document'
=> true
irb(main):002:0> d = REXML::Document.new '<div>at&t'
=> <UNDEFINED> ... </>
irb(main):003:0> d.root
=> <div> ... </>
irb(main):004:0> d.root.text
=> "at&t"
</code></pre></li>
<li><p>The <code>REXML::Text#to_s</code> method violates the principle of least surprise. In just about every other XML parser written, when you ask a
text node for its contents, it returns you the value with entities resolved. Not so <code>Text#to_s</code>. You have to call <code>Text#value</code>
instead. Unfortunately, this would be difficult to reverse in future versions of REXML without breaking existing apps.</p>
<pre><code>irb(main):001:0> require 'rexml/document'
=> true
irb(main):002:0> t = REXML::Text.new('at&t')
=> "at&t"
irb(main):003:0> t.to_s
=> "at&amp;t"
irb(main):004:0> t.value
=> "at&t"
</code></pre></li>
</ul>
<p>This second problem manifests itself in subtle ways. If you’re calling <code>Element#text</code> (which is probably the most common way), you’re fine, because it implicitly does <code>self.texts.first.value</code> under the hood. But if you want to make sure you’re grabbing all the text content, you might be inclined to write <code>element.texts.join('')</code> to concatenate them together. But this method bypasses the <code>value</code> method and instead uses <code>to_s</code>, leaving you with unresolved entities.</p>
<p>It turns out this problem is exhibited in the version of <a href="http://xml-simple.rubyforge.org/">XmlSimple</a> now included with <a href="http://dev.rubyonrails.org/browser/trunk/activesupport/lib/active_support/vendor/xml_simple.rb?rev=4453&format=txt">Edge Rails as of rev 4453</a>. So if
you’re living on the edge using the newly minted ActiveResource fetching XML from remote resources like a champion, you just got benched
as soon as you tried to fetch XML that had normalized entities inside.</p>
<p><a href="http://rubyforge.org/frs/download.php/11388/xml-simple-1.0.9.gem">XmlSimple version 1.0.9</a> has a partial fix for this issue, but I submitted <a href="/files/test_xmlsimple.rb">another patch</a> to <a href="http://www.maik-schmidt.de/">Maik Schmidt</a> for review that he subsequently released as 1.0.10. I’ve attached the 1.0.10 version to ticket <a href="http://dev.rubyonrails.org/ticket/6532">6532</a> in hopes that it will be patched in Rails soon.</p>Nick Siegerurn:uuid:21755a2d-0ffa-4f0c-be7d-a2e2e01d99b42007-01-02T05:32:00+00:002007-08-31T16:45:13+00:00Customizing RSpec<p><em>Update/Disclaimer: I refer to parts of RSpec that are not blessed as an extension API. Redefining <code>before_context_eval</code> and using the <code>@context_eval_module</code> variable directly may change in the future. I’ll keep this article updated to coincide with the changes. For now, these techniques should work fine with RSpec versions up to 0.7.5.</em></p>
<p>RSpec seems to be getting more attention lately as a viable, nay, <em>preferred</em>, alternative to Test::Unit. It’s possible that it’s just my personal feed-reader-echo-chamber, but consider this: <a href="http://blog.fallingsnow.net/rubinius/">Rubinius</a> has started using RSpec alongside Test::Unit as an another way to test the alternate Ruby implementation. They’re even in the midst of building some snazzy extensions to allow the same specs to be run under a Ruby implementation of your choice. (Perhaps this will point the way to a new round of executable specs to accompany the <a href="http://www.headius.com/rubyspec/index.php/Main_Page">fledgling community spec</a>? Let’s wait and see how they do and leave that topic for another day.)</p>
<p>But extending and customizing RSpec to add a DSL on top of RSpec’s <code>context/specify</code> framework doesn’t have to be the realm of experts. Here are some templates for how you can DRY up your specs by adding your own helper methods in such a way that they will be available to all your specs. But first, a little background.</p>
<h3>Spec Helper</h3>
<p>Most usages of RSpec that I’ve seen in the wild use a “spec helper” (<code>spec_helper.rb</code>). This file, following the pattern of Rails’ <code>test_helper.rb</code>, minimally contains require statements to pull in the RSpec code and any supporting code for running specs. By requiring the spec helper via a path relative to your spec (usually with <code>require File.dirname(__FILE__) + '/spec_helper'</code> or similar), it also allows you the convenience of running your specs one at a time from anywhere (say, by launching from your editor) or with <code>rake</code> or <code>spec</code>. This file is where your shared helper methods will go, and where they’ll get registered to be pulled into the contexts.</p>
<h3>What Context in <code>context</code>?</h3>
<div class="typocode"><pre><code class="typocode_ruby "><span class="ident">context</span> <span class="punct">"</span><span class="string">A new stack</span><span class="punct">"</span> <span class="keyword">do</span>
<span class="comment"># <== What is the value of "self" here?</span>
<span class="ident">specify</span> <span class="punct">"</span><span class="string">should be empty</span><span class="punct">"</span> <span class="keyword">do</span>
<span class="keyword">end</span>
<span class="keyword">end</span></code></pre></div>
<p>How do those contexts work anyway? The <code>context</code> method that defines a context in which specs can be defined and run takes a block to define the individual specs, but what can really go in that block?</p>
<p>It turns out that RSpec jumps through metaprogramming hoops (using <code>class_eval</code>) to make the block behave like a class definition. This means you can do things like put method definitions inside your context:</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="ident">context</span> <span class="punct">"</span><span class="string">A new stack</span><span class="punct">"</span> <span class="keyword">do</span>
<span class="keyword">def </span><span class="method">a_new_stack</span>
<span class="constant">Stack</span><span class="punct">.</span><span class="ident">new</span>
<span class="keyword">end</span>
<span class="ident">specify</span> <span class="punct">"</span><span class="string">should be empty</span><span class="punct">"</span> <span class="keyword">do</span>
<span class="ident">a_new_stack</span><span class="punct">.</span><span class="ident">should_be_empty</span>
<span class="keyword">end</span>
<span class="keyword">end</span></code></pre></div>
<p>Which is nice, but the reason we’re here is to hide that away in <code>spec_helper.rb</code>. So, to get back to the point of the comment in the first example above, the <code>self</code> inside the context block is an anonymous <code>Module</code> object. It’s constructed in the <code>initialize</code> method of a <code>Context</code> (condensed from <code>spec/runner/context.rb</code> in the RSpec codebase):</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="keyword">class </span><span class="class">Spec::Runner::Context</span>
<span class="keyword">def </span><span class="method">initialize</span><span class="punct">(</span><span class="ident">name</span><span class="punct">,</span> <span class="punct">&</span><span class="ident">context_block</span><span class="punct">)</span>
<span class="attribute">@name</span> <span class="punct">=</span> <span class="ident">name</span>
<span class="attribute">@context_eval_module</span> <span class="punct">=</span> <span class="constant">Module</span><span class="punct">.</span><span class="ident">new</span>
<span class="attribute">@context_eval_module</span><span class="punct">.</span><span class="ident">extend</span> <span class="constant">ContextEval</span><span class="punct">::</span><span class="constant">ModuleMethods</span>
<span class="attribute">@context_eval_module</span><span class="punct">.</span><span class="ident">include</span> <span class="constant">ContextEval</span><span class="punct">::</span><span class="constant">InstanceMethods</span>
<span class="ident">before_context_eval</span>
<span class="attribute">@context_eval_module</span><span class="punct">.</span><span class="ident">class_eval</span><span class="punct">(&</span><span class="ident">context_block</span><span class="punct">)</span>
<span class="keyword">end</span>
<span class="keyword">def </span><span class="method">before_context_eval</span>
<span class="keyword">end</span>
<span class="keyword">end</span></code></pre></div>
<p>(Take note of that empty <code>before_context_eval</code> method and the fact that it’s invoked during context initialization; that’s where we can plug in our custom extensions.)</p>
<p>The object held by the <code>@context_eval_module</code> instance variable is being augmented in two ways: extension and inclusion. The object is <em>extended</em> with the <code>ContextEval::ModuleMethods</code> module; these methods are being added to the object’s singleton class. This has the effect of making these methods visible within the <code>context</code> block, functioning similar to “class” methods.</p>
<p>The object also has the <code>ContextEval::InstanceMethods</code> module <em>included</em>. This has the effect of adding these as instance methods, making them visible from within <code>specify</code> blocks, which are made to behave like instance methods on the same object.</p>
<h3>Putting it together</h3>
<table style="text-align: left;" summary="">
<thead>
<th>Technique</th>
<th>Visibility</th>
<th>Use</th>
</thead>
<tbody>
<tr><td>@context_eval_module.extend</td><td>Context block</td><td>Custom setup, shared state declaration</td></tr>
<tr><td>@context_eval_module.include</td><td>Specify block</td><td>Shared actions/functions, stub/expectation modification, encapsulate instance variables</td></tr>
</tbody>
</table>
<h4>Adding specialized setup methods</h4>
<p><code>spec_helper.rb</code> snippet:</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="keyword">module </span><span class="module">SharedSetupMethods</span>
<span class="keyword">def </span><span class="method">setup_new_stack</span>
<span class="ident">setup</span> <span class="keyword">do</span>
<span class="attribute">@stack</span> <span class="punct">=</span> <span class="constant">Stack</span><span class="punct">.</span><span class="ident">new</span>
<span class="keyword">end</span>
<span class="keyword">end</span>
<span class="keyword">end</span>
<span class="keyword">class </span><span class="class">Spec::Runner::Context</span>
<span class="keyword">def </span><span class="method">before_context_eval</span>
<span class="attribute">@context_eval_module</span><span class="punct">.</span><span class="ident">extend</span> <span class="constant">SharedSetupMethods</span>
<span class="keyword">end</span>
<span class="keyword">end</span></code></pre></div>
<p>Example spec:</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="ident">context</span> <span class="punct">"</span><span class="string">A new stack</span><span class="punct">"</span> <span class="keyword">do</span>
<span class="ident">setup_new_stack</span>
<span class="ident">specify</span> <span class="punct">"</span><span class="string">should be empty</span><span class="punct">"</span> <span class="keyword">do</span>
<span class="attribute">@stack</span><span class="punct">.</span><span class="ident">should_be_empty</span>
<span class="keyword">end</span>
<span class="keyword">end</span> </code></pre></div>
<h4>Adding shared accessors</h4>
<p><code>spec_helper.rb</code> snippet:</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="keyword">module </span><span class="module">StackMethods</span>
<span class="ident">attr_accessor</span> <span class="symbol">:stack</span>
<span class="keyword">def </span><span class="method">push_an_object</span>
<span class="ident">stack</span> <span class="punct"><<</span> <span class="ident">mock</span><span class="punct">("</span><span class="string">some object</span><span class="punct">")</span>
<span class="keyword">end</span>
<span class="keyword">end</span>
<span class="keyword">class </span><span class="class">Spec::Runner::Context</span>
<span class="keyword">def </span><span class="method">before_context_eval</span>
<span class="attribute">@context_eval_module</span><span class="punct">.</span><span class="ident">include</span> <span class="constant">StackMethods</span>
<span class="keyword">end</span>
<span class="keyword">end</span></code></pre></div>
<p>Example spec:</p>
<div class="typocode"><pre><code class="typocode_ruby "><span class="ident">context</span> <span class="punct">"</span><span class="string">A stack with an object pushed</span><span class="punct">"</span> <span class="keyword">do</span>
<span class="ident">setup</span> <span class="keyword">do</span>
<span class="attribute">@stack</span> <span class="punct">=</span> <span class="constant">Stack</span><span class="punct">.</span><span class="ident">new</span>
<span class="keyword">end</span>
<span class="ident">specify</span> <span class="punct">"</span><span class="string">should not be empty</span><span class="punct">"</span> <span class="keyword">do</span>
<span class="ident">stack</span><span class="punct">.</span><span class="ident">should_be_empty</span>
<span class="ident">push_an_object</span>
<span class="ident">stack</span><span class="punct">.</span><span class="ident">should_not_be_empty</span>
<span class="keyword">end</span>
<span class="keyword">end</span> </code></pre></div>
<p>The examples are simple, but hopefully illustrate the techniques. For an example of some code that’s actually useful, check out my sample <a href="http://svn.caldersphere.net/svn/main/rspec_selenium_rc/trunk/">RSpec Selenium RC integration project</a>, in particular the <a href="http://svn.caldersphere.net/svn/main/rspec_selenium_rc/trunk/lib/spec_helper.rb">spec helper</a> and the <a href="http://svn.caldersphere.net/svn/main/rspec_selenium_rc/trunk/spec/example_spec.rb">example spec</a>. (More on this in the future if it proves useful, but for now if you check it out and run <code>rake</code> on it, it should launch <a href="http://www.openqa.org/selenium-rc/">Selenium RC</a> and run the example spec in a Firefox browser.)</p>
<p>By mixing and matching these techniques, you can layer a mini-DSL on top of RSpec and achieve DRY-er and even more readable and intention-revealing specs. Let me know if you’re able to find uses for these tips!</p>Nick Siegerurn:uuid:0de6115d-57e6-482e-9c45-964a94088b5b2006-10-20T19:04:00+00:002007-08-31T16:50:37+00:00RubyConf 2006 Begins<p>The RubyConf room is filling up this morning. I’ll be doing my best to live-blog the conference here so stay tuned!</p>
<p><a href="http://www.flickr.com/photos/nicksieger/274770531/" title="Photo Sharing"><img src="http://static.flickr.com/110/274770531_01f9ba4a44_m.jpg" width="240" height="160" alt="DSCF4909.JPG" /></a></p>
<p><a href="http://www.flickr.com/photos/nicksieger/274770462/" title="Photo Sharing"><img src="http://static.flickr.com/79/274770462_957803fcd8_m.jpg" width="240" height="160" alt="DSCF4908.JPG" /></a></p>Nick Siegerurn:uuid:9785ffcc-1680-492c-8b42-6e9111cc85aa2006-10-20T19:07:00+00:002007-08-31T16:50:37+00:00RubyConf: Sydney and Rubinius<p><em>Update: Evan has posted code and has <a href="http://blog.fallingsnow.net/rubinius/">a page set up for the project</a>.</em> </p>
<p>Evan Phoenix (nee Webb), of Seattle.rb, is presenting on Sydney and Rubinius, an experiment in improving the ruby interpreter. Sydney has died, and Rubinius has risen from its ashes, appropriately.</p>
<h2>Why</h2>
<ul>
<li>Why would you write a new Ruby interpreter? It’s fun, it’s a good challenge.</li>
<li>What’s wrong with the existing interpreter – are you hating on Matz? Of course not.</li>
</ul>
<p>Today’s Ruby interpreter is like a big dump truck – sometimes a little slow, but it works for us. YARV is like the red, shiny fire truck.
Both big and complex. Rubinius, by comparison, is like a dune buggy. Fast, light, but you’re going to get sand in your eyes if you drive
it a lot.</p>
<p>The project, admittedly, is naive.</p>
<ul>
<li>Simple architecture and implementation.</li>
<li>As little background magic as possible</li>
<li>No opaque C backend</li>
<li>Leverage axiom of simple == powerful</li>
<li>Less magic means more introspection
<ul>
<li>More control for the developer</li>
<li>Richer introspection: Backtrace, MethodTable objects</li>
</ul></li>
</ul>
<h2>What was Sydney?</h2>
<ul>
<li>Giant patch to 1.8.2 that included reentrancy and thread-safety</li>
<li>Turned out to be a major PITA</li>
<li>CRuby uses a large number of C globals, references to which had to be tracked and fixed</li>
</ul>
<h2>Transition to Rubinius</h2>
<ul>
<li>Ruby borrowed a lot from Smalltalk, so why not try an implementation based on the same concepts?</li>
<li>Prototype A ported the blue-book implementation to Ruby</li>
<li>It worked and validated the basic concept and approach</li>
<li>Prototype B took ideas from A but implemented a bytecode interpreter and compiler. Used RubyInline to access raw memory operations.</li>
<li>At this time the goal emerged to have a translator which could take a prototype and bootstrap itself into C code.</li>
<li>Prototype S was a manual translation of Prototype B into C code to make the implementation quicker.</li>
<li>Prototype W was created to translate parts of Prototype B so that there is a maintainable core in Ruby code itself.</li>
</ul>
<h2>Questions</h2>
<p><em>Q. Since you were starting over, could you use a platform-independent library to ease the process, such as APR?</em> Yes – currently using String and PointerArray from glib.</p>
<p><em>Q. How is performance?</em> Too early to tell – I hope to know by the end of the conference. Prototype S became runnable and usable on the plane here.</p>
<p><em>Q. Can you clarify the goal?</em> To create a Ruby interpreter in Ruby that can translate itself out into a C interpreter.</p>
<p><em>Q. Have you figured out how to link in external libraries in a platform independent way?</em> No. My hope is that the decision will be made to write a common framework for translating to system calls, e.g., SWT.</p>
<p><em>Q. Have you looked at PyPy? (similar project for Python)</em> Yes, and it’s f-in complicated. It worries me actually.</p>
<p><em>Q. Could you have it generate backend code in another language/platform (Java bytecode, CLR)?</em> Yes, I certainly hope so, otherwise I’m wasting my time.</p>
<p><em>Q. How will you add native thread support in a cross-platform way?</em> I hope I won’t have to, by leveraging external tools.</p>
<p><em>Q. If you’re building a Ruby-to-C translator, why write a Ruby interpreter at all?</em> If I didn’t, what would I translate? You still need some core engine to translate. <em>Would it be a subset of Ruby?</em> Yes.</p>
<p><em>Q. Looks very similar to Squeak, have you looked at Squeak code and talked to Squeak people?</em> Looked at code a lot, I’ve really stolen all of their ideas. I haven’t talked to the folks yet because I’m afraid they might laugh at me.</p>
<h2>Resulting Works</h2>
<ul>
<li>SydneyParser: Used parser from Sydney and stole ParseTree’s algorithm for generating a sexp that represents the Ruby code.</li>
<li>SegfaultProtection: detects a segfault in an extension, saves the Ruby interpreter, and raises a memory fault exception instead.</li>
</ul>
<h2>The Nitty Gritty (Red Pill)</h2>
<ul>
<li>All components separated by APIs for swappability</li>
<li>Garbage collector: baker two-space copy collector, and a train GC</li>
<li>Bytecode interpreter: small set of instructions driven by tests and need, so there are no extraneous operations</li>
<li>Compiler: written completely in Ruby, using ParseTree and SexpProcessor. Intended to compile itself to be used as a base compiler for Prototype S.</li>
</ul>
<h2>Future</h2>
<ul>
<li>Other backends – Java, Smalltalk</li>
</ul>
<h2>More questions</h2>
<p><em>Q. Worried about fragmentation?</em> Yes, but I really want to make it as compatible as possible with the current interpreter.</p>
<p><em>Q. Rubinius bytecode compatibile with YARV?</em> No, but I hope to be able to write a bridge to YARV in Rubinius.</p>
<p><em>Q. Have you looked at Valgrind for the C code?</em> Yes, I have. Good possibility for future direction.</p>
<p><em>Q. Can you demo some code?</em> They’re incredibly boring. “Look I got a MethodTable object, I asked for one.”</p>Nick Siegerurn:uuid:b9b68f1b-b58c-4cf5-9976-459e559664c32007-01-25T04:23:00+00:002007-08-31T16:53:29+00:00JRuby Serial Interview 4: NetBeans<p><em>This is part 4 in our <a href="/articles/tag/jrubyserialinterview">ongoing conversation</a> tracking the development of JRuby.</em></p>
<p><em>This episode we’re pleased to have <a href="http://blogs.sun.com/tor/">Tor Norbye</a> and Martin Krauskopf from Sun with us to discuss <a href="http://netbeans.org/">NetBeans</a>.</em></p>
<p><em>If you’re a Rubyist, why should you care about NetBeans? Isn’t that one of those big honkin’ Java IDEs? Well, due to the hard work of Tor and Martin, NetBeans will soon be a world-class Ruby and Rails editor and development environment. All made possible by JRuby underneath the hood. Don’t believe me? Then read on…</em></p>
<p><strong>So, what are you hoping to accomplish with NetBeans Ruby support? Any lofty goals? Is your target audience Ruby hackers, or Java programmers looking to try something new?</strong></p>
<p><em>Tor Norbye:</em> Anybody writing code using Ruby. That would include both experienced Ruby developers as well as newbies trying out the language.</p>
<p>The lofty goal is to provide First Class Support for Ruby such that where possible, the Ruby support is as good as the Java support. There are obviously areas where Ruby’s dynamic nature makes it hard to provide the same features as those available for Java, such as the various refactoring operations and quickfix features that rely on static typing. But that doesn’t mean we won’t try. I think a Rename refactoring operation that has some limitations is still better than just Search/Replace.</p>
<p>That’s the area I’m most excited about getting into. Until now I’ve been working on getting all the basic IDE infrastructure in place such that the vital parts are there and we can start building more smarts on top.</p>
<p><em>Martin Krauskopf:</em> Simply the target is the full debugging support in NetBeans like it is in <a href="http://rubyeclipse.sourceforge.net/">RDT</a>. I contacted <a href="http://cwilliams.textdriven.com/">Chris (Williams)</a>, Markus (Barchfeld) and <a href="http://jroller.com/page/murphee">murphee (Werner Schuster)</a> from RDT regarding cooperation on the backends. Realize that actually backends are currently their effort on which they’ve spent a lot of time. I’ve started with some mini-fixes and would like to continue on the cooperation more and more so they will also get something back. But the cooperation is <em>very</em> young so I’ll have more to say later, I think. So however there are still a lot of work on NetBeans frontend I want to get as much as possible also in the backends works.</p>
<p><em>Thomas Enebo:</em> Martin just sent an <a href="http://sourceforge.net/mailarchive/forum.php?thread_id=31511654&forum_id=9198">interesting email to the RDT list</a> on a debugging specification and a cross IDE debugger project on Rubyforge….Fun times.</p>
<p><a href="http://flickr.com/photo_zoom.gne?id=368608547&size=o" target="_blank" title="Photo Sharing"><img src="http://farm1.static.flickr.com/120/368608551_ed57f9c8be_o.jpg" width="520" height="410" alt="netbeans-class-compl-snapshot" /></a><br/>
<em>NetBeans Ruby support (click through the image for a full-screen shot). There is <code>test/unit</code> output, and you can see completion of class names with an RDoc popup.</em></p>
<p><strong>When will we see a public release of NetBeans Ruby support? What are some of the features we can expect to see? Will there be Rails support?</strong></p>
<p><em>Tor Norbye:</em> I wish I could answer <em>when</em> it’s going to be released, but that’s not in my hands. I started the work in closed source, as part of the Project Semplice work. And when code doesn’t start in open source in the first place, there’s a Sun process to be followed to release it, such as a legal review, etc. etc. It’s going through that process now - and has been for quite a while, so I’m hoping it will be released soon, very soon. Without promising anything obviously, think weeks.</p>
<p>As I said earlier, the goal for the feature list is to offer the same features that are available for Java: good editing, projects support, debugging, web application support, etc. Yes, there will be Rails support.</p>
<p>The current feature set, which is what you’d see if I got the green light to commit into NetBeans CVS today, is heavy on editing support. There is semantic highlighting, code completion, various other editing features such as pair-matching, smart indent, etc. There is also some basic projects support and Rails support. My coworker Martin Krauskopf is working on a debugger and that work is coming along nicely.</p>
<p><a href="http://flickr.com/photo_zoom.gne?id=368608554&size=o" target="_blank" title="Photo Sharing"><img src="http://farm1.static.flickr.com/106/368608555_a000b91d52_o.jpg" width="520" height="426" alt="netbeans-rails-support" /></a><br/>
<em>NetBeans Rails support (click through the image for a full-screen shot). RHTML editing, project tree and generator access are shown.</em></p>
<p><strong>I’ve got this killer feature idea for a Ruby editor. How can I get it into NetBeans?</strong></p>
<p><em>Tor Norbye:</em> Well, it will all be open source, so the easiest way to do it would be to <a href="http://www.netbeans.org/community/contribute/">join the NetBeans open source project</a> and enhance the Ruby editor directly. NetBeans itself obviously has lots of extensibility APIs, and the Ruby editor may offer its own to register additional quickfixes etc. At this point that has not been my focus.</p>
<p><em>Thanks to Tor and Martin for joining us! For up-to-date progress on Ruby support in NetBeans, follow along on <a href="http://blogs.sun.com/tor/">Tor’s blog</a>. And if you have further questions, please leave a comment. What would you like to see in NetBeans Ruby support?</em></p>