Nick Sieger: Tag rails tag:blog.nicksieger.com,2005:Typo Typo 2007-07-13T09:45:26+00:00 Nick Sieger urn:uuid:ab53b976-cff8-410d-8047-793abbb363a2 2007-05-23T05:51:36+00:00 2007-07-13T09:45:26+00:00 geekSessions I: Ruby on Rails: To Scale or Not to Scale <p>I was fortunate to be in town right after RailsConf and attended the inaugural <a href="http://www.geeksessions.com/">geekSessions</a> event on Rails scalibility. The event went off without a hitch: it was well attended, City Club is a classy place, and there was decent food and an open bar. I don&#8217;t know the SF geek/startup scene, but pretty much all of the few guys I know were there along with a ton of other folks. My only complaint would have been to let it run at least 30 minutes longer. Socializing was good too, but it seemed like the conversation was just getting started.</p> <p>Here are some notes for you in my typical rapid-fire style &#8211; hope they&#8217;re useful to you.</p> <h2>Ian McFarland</h2> <p>Case study: divine caroline</p> <p>Servers:</p> <ul> <li>Load balancer</li> <li>Apache + mongrel</li> <li>MySQL</li> <li>SOLR</li> </ul> <p>Ruby is slow. Rails is slow. Unoptimized app was slow &#8211; 7 pages/sec with <code>ab</code>. So how can Rails possibly be? 150 pv/s with a simple text render. This formed a sort of upper-bound, that ruled out fragment/action/partial caching, etc. This brought the throughput to 3500 pv/s. Except for page caching limitations:</p> <ul> <li>Cache coherency</li> <li>Writes are more expensive</li> <li>Page caching is not applicable to as many pages as you think</li> </ul> <p>But measure first. Pivotal built a drop-in page caching extension to deal with cache coherency issues (soon to be at http://rubyforge.org/projects/pivotalrb)</p> <h2>Jason Hoffman</h2> <p>Jason somehow has the distinction of the first four commits in the Rails repository. Joyent/TextDrive/Strongspace.</p> <p>If your application is successful, you&#8217;re going to have a lot of machines. What happens when you have 1000s of machines, 100s of TB, 4 locations, etc. Is this really a <em>Rails</em> issue? In a typical Joyent setup, Rails is only one of 26+ processes on the server stack. So scaling it really doesn&#8217;t mean much more than scaling any application. Object creation in Ruby is fast, sockets and threads are slow. So forget sockets and threads.</p> <p>Instead, use DNS, load balancers, evented mongrels, JRuby/Java, DBMSes (not just RDBMS; LDAP, filesystem, etc.), Rails process doing Rails only, static assets going through a static server, federate and separate as much as you can.</p> <h2>Jeremy LaTrasse</h2> <p>Jeremy&#8217;s job is about safety nets; about knowing the underlying infrastructure. Is the hardware/OS/stack important? Can you build safety nets around those so that you can spare cycles when you need to intrude into the system to troubleshoot?</p> <p>Twitter is in a unique position with the volume of traffic to be able to find some pretty tough bugs, like the recent <a href="http://dev.rubyonrails.org/changeset/6571">backtrace issue</a>.</p> <h2>Bryan Cantrill</h2> <p>Measure first! Like Ian said. Is software information? Or a machine? It&#8217;s both. Nothing else in human existence can claim this. 3 weeks after Bryan joined Sun, he was working with Jeff (ZFS architect) debugging an issue when Jeff retorted, &#8220;Does it bother you that none of this exists? It&#8217;s just a representation of some plastic and metal morass in a backroom&#8221; (slightly paraphrased).</p> <p>We&#8217;ve been living with bifurcated code &#8211; &#8220;if DEBUG; print something&#8221; ad nauseum. But this has a cost. So dev code deviates from production code. But we can&#8217;t get the data we want, where it matters, in production. Bryan goes on to describe the aforementioned <a href="http://dev.rubyonrails.org/changeset/6571">backtrace issue</a> and how it saved Twitter 33% CPU. So don&#8217;t pre-optimize, but you&#8217;ve got to be prepared to go get the data. In production.</p> <h2>Q &amp; A</h2> <p><em>What&#8217;s the best way to move from one database to two databases (MySQL), when you scale past the volume of reads that overwhelms one?</em></p> <p><strong>Jason</strong> doesn&#8217;t like the replication approach, it&#8217;s not fault tolerant. Reference to <a href="http://drnicwilliams.com/2007/04/12/magic-multi-connections-a-facility-in-rails-to-talk-to-more-than-one-database-at-a-time/">Dr Nic&#8217;s magic multi-connections gem</a>. Reference to <a href="http://revolutiononrails.blogspot.com/2007/04/plugin-release-actsasreadonlyable.html">acts_as_readonly</a>. Don&#8217;t rely on things that are out of your control, start reading/writing to multiple locations, at the application level. <strong>Jeremy</strong>: So do you want to be in the business of writing SQL or C extensions to Rails? What about <a href="http://freshmeat.net/projects/mysql_proxy/">MySQL proxy</a>? Seems ok, but I might not trust it in production. <a href="http://jeremy.zawodny.com/mysql/mytop/" title="mytop - a top clone for MySQL">MyTop</a>/<a href="http://www.xaprb.com/blog/2006/07/02/innotop-mysql-innodb-monitor/">InnoTop</a> will tell you about your query volume.</p> <p><em>Virtualization: 4 virtual servers w/ web servers on top of a single physical server? Why?</em></p> <p><strong>Jason</strong>: Free BSD 4.9 on early pentium was the perfect balance of utilization. 18 CPUs by 64G RAM with virtual servers gets us back to that level of utilization. <strong>Bryan</strong>: Not all virtualization solutions are equivalent! (Solaris containers/zones plug.)</p> <p><em>RDBMSes are not good for web applications? Why? Can you give some examples?</em></p> <p><strong>Jason</strong>: It depends on when you want to join. When people are clicking, or pre-assembled. Look at your application and put the data together before people request it. Why does YouTube need an RDBMS? It serves a file that people can comment on.</p> <p>Mention of Dabble DB, ZFS, Jabber, Atom, Atom over Jabber, etc. as ways of innovative ways of storing objects, data, etc. GData/GCal most certainly does not store its Atom files in an RDBMS.</p> <p><em>Sell Rails apps and have the customer deploy it? What options are available?</em></p> <p><strong>Ian</strong>: JRuby on Rails with a .war file is an interesting approach. <em>What operational issues/ways to help with scaling remote deployments?</em> <strong>Jeremy</strong>: Log files are the first line of defense. <strong>Jason</strong>: Corporate IT are comfortable with Java.</p> <p><em>The pessimist in me says that my servers are going to fall over after 5 users. How can I be prepared/not be optimistic about a traffic spike?</em></p> <p><strong>Ian</strong>: Load test the crap out of the app. Find out the horizontal scaling point. Use solutions like S3 for images. Make sure you can scale by throwing hardware at it. Eventually single points of failure will overcome you (such as a single database), but you can wait until you get to that point before doing something about it.</p> <p><strong>Jason</strong>: You can benchmark your processes, and get an idea of what they can do. Most people that want to do something will be look at your stuff, and maybe signup. So front-load and optimize your signup process, possibly by taking it out of Rails.</p> <p><strong>Jeremy</strong>: Conversations with Zed, DHH, etc. have pointed out that sometimes &#8220;Rails isn&#8217;t good at that, take it out of Rails.&#8221; Same thing for the database. Split those things out into a different application.</p> <p><strong>Bryan</strong>: Do your dry land work, know your toolchain, so that when the moment comes, you can dive in and find the problem.</p> <p><em>We have a migration that takes a week to run because of text processing. GC was running after every 10th DB statement. Used Rails bench GC patch to overcome the issue with the migration. Any issue running these?</em></p> <p><strong>Jason</strong>: We run those GC modifications and a few more in production, and they&#8217;re fine.</p> <p><em>Most comversations revolve around items like database is slow, or Ruby is slow. How can we use DTrace to streamline the process?</em></p> <p><strong>Jeremy</strong>: We spent 20 minutes over lunch (plus some preparation) to find a Memcache issue. It&#8217;s worth it to spend a little time to learn the tool.</p> <p><strong>Bryan</strong>: &#8220;Awk is God&#8217;s gift to all of us.&#8221; When DTrace was being reviewed inside of Sun, folks commented &#8220;This reminds us of awk.&#8221; &#8220;Thanks!&#8221;</p> <p><strong>Jason</strong>: We&#8217;re putting a tracing plugin in Rails as a remote process to collect data from a running app. Apple has shown a commitment to get this in Leopard. Textual and graphical output are possible. I believe in DTrace a lot, and the tooling and documentation will go beyond its current state of an experts tool.</p> <p><em>Lastly, what one closing thing would you like to say about Rails scalability?</em></p> <p><strong>Ian</strong>: Measure.<br/> <strong>Jason</strong>: Don&#8217;t use relational databases.<br/> <strong>Jeremy</strong>: I thought it was a Joyent sales pitch.<br/> <strong>Bryan</strong>: Use DTrace (with Joyent accelerators of course).<br/></p> Nick Sieger urn:uuid:8396524a-7e00-4aad-8727-a5a0b3ec7ef3 2006-11-15T15:46:00+00:00 2007-08-31T16:57:59+00:00 RSpec Autotest now a Rails Plugin <p>Inspired by a <a href="http://rubyforge.org/pipermail/rspec-devel/2006-November/001219.html">posting on the RSpec list</a> and recent <a href="http://blog.nicksieger.com/articles/2006/09/13/auto-rspec#comment-143">comments stating that my Auto RSpec hack wasn&#8217;t working</a>, I&#8217;ve bitten the bullet and upgraded to <a href="http://rspec.rubyforge.org/upgrade.html">RSpec 0.7.2</a>, and made <code>rspec_autotest</code> <a href="http://svn.caldersphere.net/svn/main/plugins/rspec_autotest">a plugin</a> in the process. So, herewith are the necessary incantations to auto-rspec your project. If you&#8217;ve tried my hack already, please remove any bits you previously had installed.</p> <ul> <li>Install RSpec on Rails, following the <a href="http://rspec.rubyforge.org/documentation/rails/install.html">original instructions</a>. As of RSpec 0.7.3, the specific version of ZenTest is no longer required. Also, diff-lcs is required to show unified diff output on <code>should ==</code> failures.</li> </ul> <div class="typocode"><pre><code class="typocode_default ">gem install zentest -v 3.4.1 gem install diff-lcs gem install rspec script/plugin install svn://rubyforge.org/var/svn/rspec/tags/REL_0_7_2/vendor/rspec_on_rails/vendor/plugins/rspec</code></pre></div> <ul> <li>Install <code>rspec_autotest</code></li> </ul> <div class="typocode"><pre><code class="typocode_default ">script/plugin install http://svn.caldersphere.net/svn/main/plugins/rspec_autotest</code></pre></div> <ul> <li>Start autotest</li> </ul> <div class="typocode"><pre><code class="typocode_default ">rake spec:autotest</code></pre></div> <p>Please let me know if you experience any problems!</p> Nick Sieger urn:uuid:1e075610-8ab5-47a3-acae-7def4e3ab32a 2006-08-15T04:12:00+00:00 2007-08-31T17:17:11+00:00 Security Threat Last Week <p>What was the biggest security threat story for me last week? No, it was not the <a href="http://www.nytimes.com/2006/08/12/opinion/12sat1.html">disrupted liquid bomb plot</a>, it was the <a href="http://weblog.rubyonrails.com/2006/8/10/rails-1-1-6-backports-and-full-disclosure">Rails security hole</a> that caused quite a brouhaha among the Ruby community. (Guess that shows my increasing tendency to lose touch with reality. Maybe a sign of the miserable state of unrest in the world and how living in the land of the world&#8217;s only super-power makes it easy to turn the other cheek? Or&#8230;ok, ok&#8230;it&#8217;s just me.)</p> <p>From my view of the Rails security issue, there are actually quite a few interesting angles that came out of this story.</p> <h2>Rails is Growing Up</h2> <p>This is the obvious one. The first major fault to be discovered in Rails shows that Rails the codebase, Rails the core team, Rails the technology stack, and Rails the community is going through growing pains. David was both praised and criticized widely for his handling of the disclosure. Many rightly complained that the initial announcement didn&#8217;t give system maintainers enough information to decide whether the risk warranted disrupting normal operations to spend time to test and roll out the patch. This was compounded by the fact that the initial announcement did not identify versions affected and instead assumed all past versions, which turned out not to be the case.</p> <p>Others thanked the Rails team for their discretion and trusted the recommendation despite the fuzziness and lack of details. These folks either were able to perform the upgrade much more easily or had some inkling of just how serious the issue was.</p> <p>The aftermath showed that the Rails core quickly learned from the experience. A <a href="http://lists.rubyonrails.org/mailman/listinfo/rails-security">security mailing list</a> and <a href="http://groups.google.com/group/rails-security">google group</a> were set up for future incidents and David promised to apply more <a href="http://weblog.rubyonrails.com/2006/8/10/rails-1-1-6-backports-and-full-disclosure">rigor and policy</a> to future announcements.</p> <p>It seems pretty obvious that the size of the gaffe was such that to expose the details immediately would have had way too much potential to cause widespread data loss and denial of service. In fact, the nature of the bug strikes me as one of those embarrassing bugs that every software developer commits at one point in their coding life where you amaze yourself at the short-sightedness of your implementation. I think the initial message could have been dispatched with information on the severity of the threat without necessarily disclosing the exact exploit. So, essentially I agree with the approach that was taken, but the message left out details required to evaluate the threat.</p> <h2>Threat Analysis</h2> <p>Two early <a href="http://blog.koehntopp.de/archives/1367-Ruby-On-Rails-Mandatory-Mystery-Patch.html">blog</a> <a href="http://blog.evanweaver.com/articles/2006/08/10/explanation-of-the-rails-security-vulnerability-in-1-1-4-others">posts</a> came out the day after claiming to know the details of the exploit. It turned out that they didn&#8217;t quite understand what was afoot. (Although Evan Weaver has since updated <a href="http://blog.evanweaver.com/articles/2006/08/10/explanation-of-the-rails-security-vulnerability-in-1-1-4-others">his post</a> to clarify his original analysis.)</p> <p>The threat turned out to be a simple remote code execution issue. The <code>:controller</code> dynamic expansion aspect of routing contained a bug that allowed arbitrary .rb files in a Rails application to be executed undesirably. By far the most dramatic consequence would be experienced if one&#8217;s <code>db/schema.rb</code> file were to be executed with a request for <code>/db/schema</code>, causing your entire database contents to be dropped and reloaded.</p> <p>By examining the <a href="http://dev.rubyonrails.org/svn/rails/tags/rel_1-1-4/actionpack/lib/action_controller/routing.rb"><code>safe_load_paths</code></a> method defined in affected versions, it appears that the implementation tried to limit elements of the load path that matched the expanded <code>RAILS_ROOT</code> of the application. Combine this with the fact that other elements of the routing system eagerly <code>require</code>&#8216;d files with inadequate bounds-checking spells your recipe for disaster.</p> <p>Many posters and commenters quipped that a simple <code>svn diff</code> was enough to give script kiddies or other black hats the information needed to exploit the issue. Or was it? Given that the two early analyses turned out to be off the mark, were people in the know exercising more discretion by not disclosing more details? </p> <p>Personally, I spent more than an hour staring at the affected routing code trying to untangle the various metaprogramming tricks and regular expressions that make up the Rails routing system. And I consider myself fairly adept at reading and understanding code! </p> <p>The truth of the matter is that, unless you&#8217;re a member of core or have a high level of familiarity and involvement with the Rails codebase, the svn diffs provide far too little context to decode the actual problem.</p> <p>Does this speak to the obfuscated nature of the Rails codebase or to the relatively advanced nature of web programming in Ruby? If I had to pick one, it would be the latter, but I&#8217;m leaning towards neither. The Rails codebase is not the most readable, comprehensible piece of code I&#8217;ve ever seen, but it does its job remarkably well. Perhaps if the routing code in question was a bit more understandable by the masses, this rather obvious security issue wouldn&#8217;t have gone undetected for so long.</p> <h2>Post-1.1.6 Release Triage</h2> <p>A group of enthusiastic Railsers jumped onto #rails-security on freenode shortly after the 1.1.6 release, where an effort had been organized to verify all the patches across various <a href="http://wiki.caboo.se/caboose/pages/1.1.6+Matrix">combinations of web servers and Rails versions</a>. An IRC channel, a <a href="http://wiki.caboo.se/caboose/pages/1.1.6+Matrix">wiki</a>, Ruby, Zed&#8217;s <a href="http://rfuzz.rubyforge.org/design.html">RFuzz</a>, and a <a href="http://pastie.caboo.se/7993">piece of code</a> were all the tools required to get a distributed test verification process up and running. This sort of thing happens all the time in the open source world, with programmers around the globe pitching in to raise the triage tent of the MASH unit. Still, it was exciting to see and be a part of the action and to be reminded of the power of the collective whole working for a common cause.</p> <h2>Dynamic Routing Harmful?</h2> <p>Rails&#8217;s dynamic routing code came under fire too, understandably so. Maybe this is one case where the developer-friendly approach of magically recognizing URLs goes a little too far? Production-only routes that do away with the expandable path elements could easily be generated by visiting all the controllers in the codebase and generating a more static route for each &#8211; sounds like a good idea for a plugin. Perhaps the controller is the better place to store routing metadata anyway?</p> <div class="typocode"><pre><code class="typocode_ruby "><span class="keyword">class </span><span class="class">UsersController</span> <span class="punct">&lt;</span> <span class="constant">ActionController</span><span class="punct">::</span><span class="constant">Base</span> <span class="ident">map_default_route</span> <span class="comment"># could be optional</span> <span class="keyword">end</span> <span class="keyword">class </span><span class="class">PostsController</span> <span class="punct">&lt;</span> <span class="constant">ActionController</span><span class="punct">::</span><span class="constant">Base</span> <span class="ident">map_route_as_resource</span> <span class="keyword">end</span></code></pre></div> <p>Sounds like good fodder for future investigation!</p> Nick Sieger urn:uuid:9fe84534-3a40-40e8-8626-a15bb05d74b4 2007-03-16T16:55:00+00:00 2007-08-31T17:44:56+00:00 JRuby on Rails: Integration Plugin Goes 1.0 <p>On the heels of my last post, Robert <a href="http://rubyforge.org/pipermail/jruby-extras-devel/2007-March/000432.html">announced the 1.0 release of Rails Integration</a>, the bits that allow JRuby on Rails to be run out of a Java web archive (war). If you have any interest at all in trying out JRuby on Rails, do yourself a favor and <a href="http://www.headius.com/jrubywiki/index.php/Rails_Integration">try out the integration bits</a>. Even though this is 1.0, we&#8217;re still moving rapidly and would appreciate any and all feedback. (And Robert is doing his best to keep up with changes in core.)</p> <p>Related to this, Stuart Halloway <a href="http://www.relevancellc.com/2007/3/14/the-j-plugin-existing-rails-apps-on-jruby">recently announced his J plugin</a> which is a drop-in collection of Rake tasks that bridge the inherent differences between Rails running on C Ruby vs. JRuby. We still have some work to do in some areas, such as database driver configuration, test database bootstrapping and launching unit tests. For example, instead of the <a href="http://dev.rubyonrails.org/svn/rails/branches/1-2-stable/railties/lib/tasks/databases.rake">big, ugly, database-specific case statement that&#8217;s in Rails&#8217; databases.rake today</a>:</p> <div class="typocode"><pre><code class="typocode_ruby "><span class="ident">desc</span> <span class="punct">&quot;</span><span class="string">Recreate the test databases from the development structure</span><span class="punct">&quot;</span> <span class="ident">task</span> <span class="symbol">:clone_structure</span> <span class="punct">=&gt;</span> <span class="punct">[</span> <span class="punct">&quot;</span><span class="string">db:structure:dump</span><span class="punct">&quot;,</span> <span class="punct">&quot;</span><span class="string">db:test:purge</span><span class="punct">&quot;</span> <span class="punct">]</span> <span class="keyword">do</span> <span class="ident">abcs</span> <span class="punct">=</span> <span class="constant">ActiveRecord</span><span class="punct">::</span><span class="constant">Base</span><span class="punct">.</span><span class="ident">configurations</span> <span class="keyword">case</span> <span class="ident">abcs</span><span class="punct">[&quot;</span><span class="string">test</span><span class="punct">&quot;][&quot;</span><span class="string">adapter</span><span class="punct">&quot;]</span> <span class="keyword">when</span> <span class="punct">&quot;</span><span class="string">mysql</span><span class="punct">&quot;</span> <span class="constant">ActiveRecord</span><span class="punct">::</span><span class="constant">Base</span><span class="punct">.</span><span class="ident">establish_connection</span><span class="punct">(</span><span class="symbol">:test</span><span class="punct">)</span> <span class="constant">ActiveRecord</span><span class="punct">::</span><span class="constant">Base</span><span class="punct">.</span><span class="ident">connection</span><span class="punct">.</span><span class="ident">execute</span><span class="punct">('</span><span class="string">SET foreign_key_checks = 0</span><span class="punct">')</span> <span class="constant">IO</span><span class="punct">.</span><span class="ident">readlines</span><span class="punct">(&quot;</span><span class="string">db/<span class="expr">#{RAILS_ENV}</span>_structure.sql</span><span class="punct">&quot;).</span><span class="ident">join</span><span class="punct">.</span><span class="ident">split</span><span class="punct">(&quot;</span><span class="string"> </span><span class="punct">&quot;).</span><span class="ident">each</span> <span class="keyword">do</span> <span class="punct">|</span><span class="ident">table</span><span class="punct">|</span> <span class="constant">ActiveRecord</span><span class="punct">::</span><span class="constant">Base</span><span class="punct">.</span><span class="ident">connection</span><span class="punct">.</span><span class="ident">execute</span><span class="punct">(</span><span class="ident">table</span><span class="punct">)</span> <span class="keyword">end</span> <span class="keyword">when</span> <span class="punct">&quot;</span><span class="string">postgresql</span><span class="punct">&quot;</span> <span class="constant">ENV</span><span class="punct">['</span><span class="string">PGHOST</span><span class="punct">']</span> <span class="punct">=</span> <span class="ident">abcs</span><span class="punct">[&quot;</span><span class="string">test</span><span class="punct">&quot;][&quot;</span><span class="string">host</span><span class="punct">&quot;]</span> <span class="keyword">if</span> <span class="ident">abcs</span><span class="punct">[&quot;</span><span class="string">test</span><span class="punct">&quot;][&quot;</span><span class="string">host</span><span class="punct">&quot;]</span> <span class="constant">ENV</span><span class="punct">['</span><span class="string">PGPORT</span><span class="punct">']</span> <span class="punct">=</span> <span class="ident">abcs</span><span class="punct">[&quot;</span><span class="string">test</span><span class="punct">&quot;][&quot;</span><span class="string">port</span><span class="punct">&quot;].</span><span class="ident">to_s</span> <span class="keyword">if</span> <span class="ident">abcs</span><span class="punct">[&quot;</span><span class="string">test</span><span class="punct">&quot;][&quot;</span><span class="string">port</span><span class="punct">&quot;]</span> <span class="constant">ENV</span><span class="punct">['</span><span class="string">PGPASSWORD</span><span class="punct">']</span> <span class="punct">=</span> <span class="ident">abcs</span><span class="punct">[&quot;</span><span class="string">test</span><span class="punct">&quot;][&quot;</span><span class="string">password</span><span class="punct">&quot;].</span><span class="ident">to_s</span> <span class="keyword">if</span> <span class="ident">abcs</span><span class="punct">[&quot;</span><span class="string">test</span><span class="punct">&quot;][&quot;</span><span class="string">password</span><span class="punct">&quot;]</span> `<span class="ident">psql</span> <span class="punct">-</span><span class="constant">U</span> <span class="punct">&quot;</span><span class="string"><span class="expr">#{abcs[&quot;test&quot;][&quot;username&quot;]}</span></span><span class="punct">&quot;</span> <span class="punct">-</span><span class="ident">f</span> <span class="ident">db</span><span class="punct">/</span><span class="comment">#{RAILS_ENV}_structure.sql #{abcs[&quot;test&quot;][&quot;database&quot;]}`</span> <span class="keyword">when</span> <span class="punct">&quot;</span><span class="string">sqlite</span><span class="punct">&quot;,</span> <span class="punct">&quot;</span><span class="string">sqlite3</span><span class="punct">&quot;</span> <span class="ident">dbfile</span> <span class="punct">=</span> <span class="ident">abcs</span><span class="punct">[&quot;</span><span class="string">test</span><span class="punct">&quot;][&quot;</span><span class="string">database</span><span class="punct">&quot;]</span> <span class="punct">||</span> <span class="ident">abcs</span><span class="punct">[&quot;</span><span class="string">test</span><span class="punct">&quot;][&quot;</span><span class="string">dbfile</span><span class="punct">&quot;]</span> `<span class="comment">#{abcs[&quot;test&quot;][&quot;adapter&quot;]} #{dbfile} &lt; db/#{RAILS_ENV}_structure.sql`</span> <span class="keyword">when</span> <span class="punct">&quot;</span><span class="string">sqlserver</span><span class="punct">&quot;</span> `<span class="ident">osql</span> <span class="punct">-</span><span class="constant">E</span> <span class="punct">-</span><span class="constant">S</span> <span class="comment">#{abcs[&quot;test&quot;][&quot;host&quot;]} -d #{abcs[&quot;test&quot;][&quot;database&quot;]} -i db\#{RAILS_ENV}_structure.sql`</span> <span class="keyword">when</span> <span class="punct">&quot;</span><span class="string">oci</span><span class="punct">&quot;,</span> <span class="punct">&quot;</span><span class="string">oracle</span><span class="punct">&quot;</span> <span class="constant">ActiveRecord</span><span class="punct">::</span><span class="constant">Base</span><span class="punct">.</span><span class="ident">establish_connection</span><span class="punct">(</span><span class="symbol">:test</span><span class="punct">)</span> <span class="constant">IO</span><span class="punct">.</span><span class="ident">readlines</span><span class="punct">(&quot;</span><span class="string">db/<span class="expr">#{RAILS_ENV}</span>_structure.sql</span><span class="punct">&quot;).</span><span class="ident">join</span><span class="punct">.</span><span class="ident">split</span><span class="punct">(&quot;</span><span class="string">; </span><span class="punct">&quot;).</span><span class="ident">each</span> <span class="keyword">do</span> <span class="punct">|</span><span class="ident">ddl</span><span class="punct">|</span> <span class="constant">ActiveRecord</span><span class="punct">::</span><span class="constant">Base</span><span class="punct">.</span><span class="ident">connection</span><span class="punct">.</span><span class="ident">execute</span><span class="punct">(</span><span class="ident">ddl</span><span class="punct">)</span> <span class="keyword">end</span> <span class="keyword">when</span> <span class="punct">&quot;</span><span class="string">firebird</span><span class="punct">&quot;</span> <span class="ident">set_firebird_env</span><span class="punct">(</span><span class="ident">abcs</span><span class="punct">[&quot;</span><span class="string">test</span><span class="punct">&quot;])</span> <span class="ident">db_string</span> <span class="punct">=</span> <span class="ident">firebird_db_string</span><span class="punct">(</span><span class="ident">abcs</span><span class="punct">[&quot;</span><span class="string">test</span><span class="punct">&quot;])</span> <span class="ident">sh</span> <span class="punct">&quot;</span><span class="string">isql -i db/<span class="expr">#{RAILS_ENV}</span>_structure.sql <span class="expr">#{db_string}</span></span><span class="punct">&quot;</span> <span class="keyword">else</span> <span class="keyword">raise</span> <span class="punct">&quot;</span><span class="string">Task not supported by '<span class="expr">#{abcs[&quot;test&quot;][&quot;adapter&quot;]}</span>'</span><span class="punct">&quot;</span> <span class="keyword">end</span> <span class="keyword">end</span></code></pre></div> <p>we can use migrations to create the test database:</p> <div class="typocode"><pre><code class="typocode_ruby "><span class="ident">desc</span> <span class="punct">&quot;</span><span class="string">Recreate the test databases from migrations</span><span class="punct">&quot;</span> <span class="ident">task</span> <span class="symbol">:migrate_test_db</span> <span class="keyword">do</span> <span class="constant">ActiveRecord</span><span class="punct">::</span><span class="constant">Base</span><span class="punct">.</span><span class="ident">establish_connection</span><span class="punct">(</span><span class="constant">ActiveRecord</span><span class="punct">::</span><span class="constant">Base</span><span class="punct">.</span><span class="ident">configurations</span><span class="punct">['</span><span class="string">test</span><span class="punct">'])</span> <span class="constant">ActiveRecord</span><span class="punct">::</span><span class="constant">Schema</span><span class="punct">.</span><span class="ident">verbose</span> <span class="punct">=</span> <span class="ident">t</span><span class="punct">.</span><span class="ident">application</span><span class="punct">.</span><span class="ident">options</span><span class="punct">.</span><span class="ident">trace</span> <span class="constant">ActiveRecord</span><span class="punct">::</span><span class="constant">Migrator</span><span class="punct">.</span><span class="ident">migrate</span><span class="punct">(&quot;</span><span class="string">db/migrate/</span><span class="punct">&quot;)</span> <span class="keyword">end</span></code></pre></div> <p>Over time, I hope to see all these efforts coalesce and make the Rails developer experience virtually identical on either interpreter. What would be most excellent is to eventually push some of these improvements back to the Rails core.</p> Nick Sieger urn:uuid:091b1464-16fe-49ee-9d4b-ce18ddf2da0d 2006-05-15T03:12:00+00:00 2007-08-31T17:55:23+00:00 JRuby on Rails and ActiveRecord on JDBC <p><a href="http://www.bloglines.com/blog/ThomasEEnebo?id=16">Tom</a> and <a href="http://headius.blogspot.com/2006/05/and-they-said-jruby-was-dead.html">Charlie</a> have just experienced what can only be described as a watershed moment in the grand scheme of dynamic languages on the JVM. The Rails experience may soon be visiting a Java application server near you! Even though JRuby will be in perpetual catch-up mode with C Ruby, Tom and Charlie and the rest of the JRuby contributors have shown incredible perseverance in tracking the Ruby language despite <em>the lack of any formal specification</em>. Maybe Rails will never be mainstream, but the possibilities just got a whole lot more interesting. I agree with <a href="http://jroller.com/page/obie?entry=jruby_on_rails_is_born">Obie</a> that this could be a game-changer.</p> <p>Now, a few comments about the ActiveRecord JDBC adapter. This code can still be considered alpha quality at best. It&#8217;s awesome that Tom and Charlie will be able to demo a top-to-bottom, working Rails app on JRuby, but don&#8217;t jump to conclusions yet that this will be anything like a write-once, deploy-to-any-database kind of experience. But you didn&#8217;t think it would, did you? We all learned that about Java a long time ago, right?</p> <p>You can <a href="http://svn.caldersphere.net/svn/main/activerecord-jdbc/trunk/">check out the code here</a>. At the moment, I&#8217;ve only tried it with MySQL. Most of the problems with it come from the lossy mapping from ActiveRecord&#8217;s abstraction of the database to JDBC&#8217;s. (Although I suspect as more JDBC drivers are tried that there will eventually be compatibility issues with different implementations of the JDBC spec.)</p> <p>Probably the thorniest issue is the one of type conversion. ActiveRecord has a fairly simple notion of types: <code>:string, :text, :integer, :float, :datetime</code> etc. Compare this to <a href="http://java.sun.com/j2se/1.4.2/docs/api/java/sql/Types.html">JDBC&#8217;s</a>. What a mess! Right now there are arrays of <code>proc</code>s for each AR type that try to guess the best JDBC type to use. This will certainly need improvement to become a more robust solution.</p> <p>Probably the most promising approach may be to create a patch that refactors much of the AR adapters&#8217; type conversion methods into separate modules that could be included into instances of the JDBC adapter depending on the underlying database. Then the JDBC types wouldn&#8217;t be needed at all &#8211; the existing AR database metadata could be reused. Presumably this would require some petitioning of the Rails core team to accept the changes even though the changes don&#8217;t buy AR itself any additional flexibility.</p> <p>If you have a chance to try out the code or can think of any additional tricks that would help the ActiveRecord JDBC implementation along, let me know!</p> Nick Sieger urn:uuid:a88e00d4-56b6-4b75-95e9-4e21e6b0387a 2007-09-04T02:48:40+00:00 2007-09-04T02:48:40+00:00 Warbler, A Little Birdie To Introduce Your Rails App To Java <p>This week I was working on integrating the latest <a href="http://www.bloglines.com/blog/ThomasEEnebo?id=35">JRuby 1.0.1</a> and <a href="http://rubyforge.org/frs/shownotes.php?release_id=14048">Goldspike 1.3</a> releases into our environment, when my frustration hit a fever pitch.</p> <p>See, I had always thought that the .war packaging side of Goldspike was a little clunky and un-ruby-like, but I didn&#8217;t see a clear path to fixing it. I had heard little complaints about it here and there: the little configuration DSL didn&#8217;t give you enough control or wasn&#8217;t documented well enough; the fact that it downloads libraries from the internet during assembly (convenient, but not safe or reproducible for production deployments).</p> <p>Also, in my own opinion it took the wrong approach to packaging Rails in a .war file. It puts the Rails application directory structure into the root of the .war file where any web server or Java application server might mistakenly serve up your code as static content. The Java .war file spec has this special directory called <code>WEB-INF</code> expressly for the purpose of hiding that stuff away, so why not use it?</p> <p>And then, suddenly Goldspike was packaging up my entire Rails application directory, .svn directories and everything. So I set out to fix this once and for all.</p> <p>And so I present <a href="http://caldersphere.rubyforge.org/warbler/">Warbler</a>. A little bird who chirpily steps up to the task of assembling your Rails application into a Java Web Archive (.war). Here, get it:</p> <pre><code>gem install warbler </code></pre> <p>And then, in the top directory of your Rails application,</p> <pre><code>warble </code></pre> <p>Those two steps are all it takes to make a .war file, including your application and recent versions of JRuby and Goldspike, that&#8217;s deployable to your <a href="https://glassfish.dev.java.net/">favorite Java application server</a>.</p> <p>There are a number of points about Warbler worth mentioning.</p> <h3>Does one thing, well</h3> <p>Warbler only packages, and doesn&#8217;t care about anything else, like how to dispatch servlet requests to Rails. This will allow for more runtime servlet binding mechanisms to take advantage of Warbler in the future.</p> <h3>Fast and lightweight</h3> <p>50% less code than the Goldspike packaging plugin, yet does the job quickly and efficiently.</p> <h3>Sane defaults</h3> <p>Warbler only packages code that you need to run the application, omitting database migrations and tests. If your application is self-sufficient (no external dependencies), then the out-of-the-box configuration will probably work for you. Public HTML/images/javascript/stylesheets go in the root of the webapp, where Java webservers expect them to be.</p> <h3>Documented, flexible configuration</h3> <p>Need to customize your configuration? Run <code>warble config</code> and edit <code>config/warble.rb</code>. All the options are there, commented and documented.</p> <p>Need to change out the bundled JRuby/Goldspike versions? <code>warble pluginize</code> makes a copy of Warbler in the <code>vendor/plugins</code> area of your application, allowing you to change the .jar files in the <code>vendor/plugins/warbler-0.9/lib</code> directory. Warbler then makes his nest in your project&#8217;s list of rake tasks (as <code>rake -T | grep war</code> shows)</p> <pre><code>rake war # Create trunk.war rake war:app # Copy all application files into the .war rake war:clean # Clean up the .war file and the staging area rake war:gems # Unpack all gems into WEB-INF/gems rake war:jar # Run the jar command to create the .war rake war:java_libs # Copy all java libraries into the .war rake war:public # Copy all public HTML files to the root of the .war rake war:webxml # Generate a web.xml file for the webapp </code></pre> <p>Warbler even omits himself in the .war file produced when running in plugin mode, since you won&#8217;t need him at runtime. It&#8217;s the little details that matter.</p> <p>Give him a try and let me know if it makes your life deploying Rails applications to JRuby on Java appservers easier!</p> Nick Sieger urn:uuid:be1363d9-dfe0-443e-8248-93aa31622820 2007-09-14T05:33:00+00:00 2007-09-15T00:53:21+00:00 Gig: Speaking at RailsConf Europe 2007 <p>Speaking of keeping busy, <a href="http://www.railsconfeurope.com/cs/railseurope2007/view/e_sess/14961">I&#8217;ll be speaking</a> alongside my colleague <a href="http://blogs.sun.com/craigmcc/" title="Craig McClanahan's Weblog">Craig McClanahan</a> at <a href="http://www.railsconfeurope.com/" title="RailsConf Europe 2007 &#8226; September 17, 2007 - September 19, 2007 &#8226; Berlin, Germany">RailsConf Europe </a> in Berlin next week.</p> <p>Sun is a Diamond Sponsor at RailsConf again, just like in Portland last May. Part of that sponsorship money pays for a brief keynote spot (filled by Craig) as well as a session or two. So no, I didn&#8217;t get my spot through an accepted proposal submission, but that doesn&#8217;t mean that the session is going to be a big <a href="http://headius.blogspot.com/2007/09/infoworld-bossies-close-to-my-heart.html">marketing shill</a>.</p> <p>No, actually Craig and I are part of a small group at Sun that&#8217;s embracing Rails in a big way, and we&#8217;re going to be launching a site built mostly on Rails later this fall. We&#8217;re taking what we think are some novel approaches to building a Rails-based application and we thought we&#8217;d share some of those thoughts with you rather than drone on for the session about how great Sun is and what snazzy tools we make. (Although expect to see a subtle plug or two for Sun hardware and tools. Call it product placement rather than overt selling.)</p> <p>I titled the session &#8220;Rails Hydra&#8221; because the central idea of the structure of our application is not one Rails app, but many. The UI and views don&#8217;t even talk to a database; instead they make use of ActiveResource and RESTful web services, talking to the models living in other Rails applications in the backend. One key point is we&#8217;re deploying .war files to JRuby running on Glassfish, thus avoiding headaches of morbidly multiplying Mongrel math. We&#8217;ll elaborate on this arrangement and talk about some of the other tools and tricks we&#8217;re using.</p> <p>Also, Charlie, Tom and Ola will be there, so we&#8217;ll certainly have a JRuby summit at some point. Stop by and say hello!</p> Nick Sieger urn:uuid:bfda082f-fa26-41d8-aa16-fc07b41aba35 2007-10-25T03:36:00+00:00 2007-10-25T15:27:17+00:00 JRuby on Rails: Fast Enough <p>People have been asking for a while how fast JRuby runs Rails. (Of course, &#8220;fast&#8221; has always been a <a href="http://www.joelonsoftware.com/items/2006/09/12.html">relative term</a>.) We haven&#8217;t been quick to answer the question, because frankly we didn&#8217;t know. We hadn&#8217;t been building real Rails applications on JRuby ourselves yet, and there was no definitive word from the crowd either.</p> <p>Recently, several guys from <a href="http://studios.thoughtworks.com/rubyworks">ThoughtWorks</a> have been working on a <a href="http://tw-commons.rubyforge.org/svn/petstore/trunk/">Rails petstore</a> application and benchmark to get to the heart of the matter. <a href="http://www.nabble.com/forum/Search.jtp?forum=14106&amp;local=y&amp;query=petstore">Discussion has been heated</a> on the JRuby mailing list, but results have not been conclusive yet.</p> <p>In the project I&#8217;m working on, we&#8217;ve <a href="http://blog.nicksieger.com/articles/2007/10/06/railsconf-europe-hydra">committed to using and deploying on JRuby</a>. Eventually we were going to reach the point where we&#8217;d need to find out how well our application runs. So today I began running a simple single request benchmark on a relatively busy page. The numbers turned out to be rather surprising:</p> <p><img src="/files/jr-requests.png" alt="Requests" title="Requests"/></p> <p><img src="/files/jr-average.png" alt="Average" title="Average"/></p> <p>(The <a href="http://spreadsheets.google.com/pub?key=pGobleZnKsdI1zW38xpNaaw">raw data is available here</a>.)</p> <p>Now, MRI (C Ruby) will always run about the same speed no matter how many runs you give it, but it&#8217;s well known that the JVM <a href="http://www.javaworld.com/javaworld/javaqa/2003-04/01-qa-0411-hotspot.html">needs time to warm up</a>. And indeed it does; after 250 iterations, Mongrel running on JRuby finally surpasses MRI. The JRuby/Goldspike/Glassfish combo comes close as well.</p> <p>Some details about the setup:</p> <ul> <li>I ran the tests on my MacBook Pro Core 2 Duo 2.4 GHz. I didn&#8217;t disable one of the cores for the tests, which means that JRuby has an advantage over MRI because it can use both (native threads at work). However, the test script ran the requests serially, which means that the advantage was minimal.</li> <li>The application is indeed of the <a href="http://blog.nicksieger.com/articles/2007/10/06/railsconf-europe-hydra">&#8220;hydra&#8221;</a> variety; the setup is nearly identical to the second diagram on that page. So a single request is passing through <strong>not one, but two</strong> Rails applications in addition to touching the database. It rendered an HTML ERb view with data from an ActiveResource-accessed RESTful service. The applications are based on Rails 1.2.3.</li> <li>MRI version is using Ruby 1.8.6 and Mongrel 1.0.1.</li> <li>JRuby Mongrel is also version 1.0.1 (<a href="http://jxh.bingodisk.com/bingo/public/Hackdays/JRuby/">details on installing it here</a>)</li> <li>JRuby on Glassfish used Glassfish 2 and Goldspike 1.4, deployed in war files via <a href="http://caldersphere.rubyforge.org/warbler">Warbler</a>.</li> <li>The two JRuby setups used JDK 1.5 and were tweaked to <a href="http://ola-bini.blogspot.com/2007/07/objectspace-to-have-or-not-to-have.html">disable ObjectSpace</a> and use the &#8220;server&#8221; VM (-server argument to the JVM).</li> </ul> <p>The main point I wish to make with these numbers is that JRuby performance is there today, and still has room to grow. There&#8217;s no longer any doubt in my mind. Yes, this is a simplistic application benchmark run on a developer&#8217;s machine, but it&#8217;s a real application. The test may not be exacting in precision, but I see enough in the numbers to believe that this will be replicable to production environments. The plot thickens!</p> Nick Sieger urn:uuid:5fa03b7c-e168-4bd4-be31-b0237344c159 2007-11-06T15:00:00+00:00 2007-11-07T18:06:39+00:00 ActiveRecord-JDBC 0.6 Released! <p>Just out is ActiveRecord-JDBC 0.6, the post-RubyConf release.</p> <p>The sparkly new feature is Rails 2.0 support. In the soon-to-be-released Rails 2.0 (edge), Rails will automatically look for and load an adapter gem based on the name of the adapter you specify in <code>database.yml</code>. Example:</p> <pre><code>development: adapter: funkdb ... </code></pre> <p>With this database configuration, Rails will attempt to load the <code>activerecord-funkdb-adapter</code> gem, require the <code>active_record/connection_adapters/funkdb_adapter</code> library, and call the method <code>ActiveRecord::Base.funkdb_connection</code> in order to obtain a connection to the database. (This is the mechanism used to off-load non-core adapters out of the Rails codebase.)</p> <p>We can leverage this convention to make it easier than ever to get started using JRuby with your Rails application. So, the first thing new in the 0.6 release is the name. You now install <code>activerecord-jdbc-adapter</code>:</p> <pre><code>jruby -S gem install activerecord-jdbc-adapter </code></pre> <p>But wait, there&#8217;s more! We also have adapters for four open-source databases, including MySQL, PostgreSQL, and two embedded Java databases, Derby and HSQLDB. And, for your convenience, we&#8217;ve bundled the JDBC drivers in dependent gems, so you don&#8217;t have to go hunting them down if you don&#8217;t have them handy.</p> <p>Check this out. Get a <a href="http://dist.codehaus.org/jruby/jruby-bin-1.0.2.zip">fresh copy of JRuby 1.0.2</a>, unpack it, and add the <code>bin</code> directory to your path. Install the adapter:</p> <pre><code>$ jruby -S gem install activerecord-jdbcderby-adapter --include-dependencies Successfully installed activerecord-jdbcderby-adapter-0.6 Successfully installed activerecord-jdbc-adapter-0.6 Successfully installed jdbc-derby-10.2.2.0 </code></pre> <p>In your Rails application, freeze to edge Rails (soon to be Rails 2.0).</p> <pre><code>rake rails:freeze:edge </code></pre> <p>Re-run the Rails command, regenerating configuration files.</p> <pre><code>jruby ./vendor/rails/railties/bin/rails . </code></pre> <p>Currently, Rails 2.0 uses <code>openssl</code> for the HMAC digest used in the new cookie session store, so we have to install the <code>jruby-openssl</code> gem:</p> <pre><code>jruby -S gem install jruby-openssl </code></pre> <p>Now, update your config/database.yml as follows:</p> <pre><code>development: adapter: jdbcderby database: db/development </code></pre> <p>Re-run your migrations, and you should now see a Derby database footprint in the <code>db/development</code> directory.</p> <pre><code>$ ls -l db/development total 24 -rw-r--r-- 1 nicksieg nicksieg 38 Nov 6 08:24 db.lck -rw-r--r-- 1 nicksieg nicksieg 4 Nov 6 08:24 dbex.lck drwxr-xr-x 5 nicksieg nicksieg 170 Nov 6 08:24 log/ drwxr-xr-x 65 nicksieg nicksieg 2210 Nov 6 08:24 seg0/ -rw-r--r-- 1 nicksieg nicksieg 882 Nov 6 08:24 service.properties drwxr-xr-x 2 nicksieg nicksieg 68 Nov 6 08:24 tmp/ </code></pre> <p>That&#8217;s it! To <strong>re-emphasize, to make your application run under JRuby, no longer will you need to</strong> a) find and download appropriate JDBC drivers, b) wonder where they should be placed so that JRuby will find them, or c) make custom changes to <code>config/environment.rb</code>. All that&#8217;s taken care of you if you use one of the following adapters:</p> <ul> <li><code>activerecord-jdbcmysql-adapter</code> (MySQL)</li> <li><code>activerecord-jdbcpostgresql-adapter</code> (PostgreSQL)</li> <li><code>activerecord-jdbcderby-adapter</code> (Derby)</li> <li><code>activerecord-jdbchsqldb-adapter</code> (HSQLDB)</li> </ul> <p>If you need to connect to a different database, you&#8217;ll still need to place your database&#8217;s JDBC driver jar file in the appropriate place and use the straight <code>activerecord-jdbc-adapter</code>. Also note that in this case, and for Rails 1.2.x in general, you&#8217;ll still need to add that pesky <code>require</code> statement to <code>config/environment.rb</code>.</p> <p>As always, there are bug fixes too (though we haven&#8217;t been tracking exactly which ones are fixed). We&#8217;re starting to file <a href="http://jira.codehaus.org/secure/IssueNavigator.jspa?reset=true&amp;&amp;pid=11295&amp;resolution=-1&amp;component=12786&amp;sorter/field=priority&amp;sorter/order=ASC">ActiveRecord-JDBC bugs</a> in the JRuby JIRA now, and will be putting in future AR-JDBC versions to target soon too. So, please <a href="http://jira.codehaus.org/secure/CreateIssue.jspa?pid=11295">file new bugs in JIRA</a> (and select component &#8220;ActiveRecord-JDBC&#8221;) rather than in the antiquated Rubyforge tracker.</p> Nick Sieger urn:uuid:fe7e8324-82de-49dc-a132-f1e514007cdd 2008-01-17T23:48:00+00:00 2008-01-17T23:49:03+00:00 Next performance fix: Builder::XChar <p>Next up in our performance series: <code>Builder::XChar</code>. (Another fine Sam Ruby production!) While this piece of code in the Builder library strikes me as perfectly fine, it also tends to slow down quite a bit with larger documents or chunks of text.</p> <p>Our path to the bottleneck is as follows: <code>ActiveRecord::Base#to_xml =&gt; Builder::XMLMarkup#text! =&gt; String#to_xs =&gt; Fixnum#xchr</code>. Consider:</p> <div class="typocode"><pre><code class="typocode_ruby "><span class="ident">require</span> <span class="punct">'</span><span class="string">rubygems</span><span class="punct">'</span> <span class="ident">gem</span> <span class="punct">'</span><span class="string">activesupport</span><span class="punct">'</span> <span class="ident">require</span> <span class="punct">'</span><span class="string">active_support</span><span class="punct">'</span> <span class="ident">require</span> <span class="punct">'</span><span class="string">benchmark</span><span class="punct">'</span> <span class="keyword">module </span><span class="module">Benchmark</span> <span class="keyword">class </span><span class="punct">&lt;&lt;</span> <span class="constant">self</span> <span class="keyword">def </span><span class="method">report</span><span class="punct">(&amp;</span><span class="ident">block</span><span class="punct">)</span> <span class="ident">n</span> <span class="punct">=</span> <span class="number">10</span> <span class="ident">times</span> <span class="punct">=</span> <span class="punct">(</span><span class="number">1</span><span class="punct">..</span><span class="number">10</span><span class="punct">).</span><span class="ident">map</span> <span class="keyword">do</span> <span class="ident">bm</span> <span class="punct">=</span> <span class="ident">measure</span><span class="punct">(&amp;</span><span class="ident">block</span><span class="punct">)</span> <span class="ident">puts</span> <span class="ident">bm</span> <span class="ident">bm</span> <span class="keyword">end</span> <span class="ident">sum</span> <span class="punct">=</span> <span class="ident">times</span><span class="punct">.</span><span class="ident">inject</span><span class="punct">(</span><span class="number">0</span><span class="punct">)</span> <span class="punct">{|</span><span class="ident">s</span><span class="punct">,</span><span class="ident">t</span><span class="punct">|</span> <span class="ident">s</span> <span class="punct">+</span> <span class="ident">t</span><span class="punct">.</span><span class="ident">real</span><span class="punct">}</span> <span class="ident">mean</span> <span class="punct">=</span> <span class="ident">sum</span> <span class="punct">/</span> <span class="ident">n</span> <span class="ident">sumsq</span> <span class="punct">=</span> <span class="ident">times</span><span class="punct">.</span><span class="ident">inject</span><span class="punct">(</span><span class="number">0</span><span class="punct">)</span> <span class="punct">{|</span><span class="ident">s</span><span class="punct">,</span><span class="ident">t</span><span class="punct">|</span> <span class="ident">s</span> <span class="punct">+</span> <span class="ident">t</span><span class="punct">.</span><span class="ident">real</span> <span class="punct">*</span> <span class="ident">t</span><span class="punct">.</span><span class="ident">real</span><span class="punct">}</span> <span class="ident">sd</span> <span class="punct">=</span> <span class="constant">Math</span><span class="punct">.</span><span class="ident">sqrt</span><span class="punct">((</span><span class="ident">sumsq</span> <span class="punct">-</span> <span class="punct">(</span><span class="ident">sum</span> <span class="punct">*</span> <span class="ident">sum</span> <span class="punct">/</span> <span class="ident">n</span><span class="punct">))</span> <span class="punct">/</span> <span class="punct">(</span><span class="ident">n</span> <span class="punct">-</span> <span class="number">1</span><span class="punct">))</span> <span class="ident">puts</span><span class="punct">(&quot;</span><span class="string">Mean: %0.6f SDev: %0.6f</span><span class="punct">&quot;</span> <span class="punct">%</span> <span class="punct">[</span><span class="ident">mean</span><span class="punct">,</span> <span class="ident">sd</span><span class="punct">])</span> <span class="keyword">end</span> <span class="keyword">end</span> <span class="keyword">end</span> <span class="comment"># http://blog.nicksieger.com/files/page.xml</span> <span class="ident">page</span> <span class="punct">=</span> <span class="constant">File</span><span class="punct">.</span><span class="ident">open</span><span class="punct">(&quot;</span><span class="string">page.xml</span><span class="punct">&quot;)</span> <span class="punct">{|</span><span class="ident">f</span><span class="punct">|</span> <span class="ident">f</span><span class="punct">.</span><span class="ident">read</span> <span class="punct">}</span> <span class="constant">Benchmark</span><span class="punct">.</span><span class="ident">report</span> <span class="keyword">do</span> <span class="number">20</span><span class="punct">.</span><span class="ident">times</span> <span class="punct">{</span> <span class="ident">page</span><span class="punct">.</span><span class="ident">to_xs</span> <span class="punct">}</span> <span class="keyword">end</span></code></pre></div> <p>On Ruby and JRuby, this produces:</p> <pre><code>$ ruby to_xs.rb 21.430000 0.400000 21.830000 ( 22.022769) 21.530000 0.360000 21.890000 ( 22.005737) 21.540000 0.370000 21.910000 ( 22.065165) 21.530000 0.370000 21.900000 ( 22.028591) 21.500000 0.350000 21.850000 ( 21.990395) 21.550000 0.370000 21.920000 ( 22.033164) 21.520000 0.360000 21.880000 ( 21.984129) 21.550000 0.370000 21.920000 ( 22.116802) 21.550000 0.370000 21.920000 ( 22.051421) 21.520000 0.380000 21.900000 ( 22.084736) Mean: 22.038291 SDev: 0.041985 $ jruby -J-server to_xs.rb 79.112000 0.000000 79.112000 ( 79.112000) 81.480000 0.000000 81.480000 ( 81.481000) 84.745000 0.000000 84.745000 ( 84.745000) 84.384000 0.000000 84.384000 ( 84.384000) 121.933000 0.000000 121.933000 (121.933000) 85.533000 0.000000 85.533000 ( 85.532000) 82.762000 0.000000 82.762000 ( 82.763000) 82.090000 0.000000 82.090000 ( 82.090000) 81.298000 0.000000 81.298000 ( 81.299000) 80.774000 0.000000 80.774000 ( 80.773000) Mean: 86.411200 SDev: 12.635700 </code></pre> <p>(Hmm, I must have accidentally swapped in some large program in the middle of that JRuby run. The perils of benchmarking on a desktop machine. I don&#8217;t claim that the numbers are scientific, just illustrative!)</p> <p>Fortunately, the fix again is very simple, and has <a href="http://groups.google.com/group/rubyjam/browse_thread/thread/82a9ddb762019bcc">previously</a> <a href="http://dev.rubyonrails.org/changeset/7773">been acknowledged</a>. The latest (unreleased?) <a href="http://code.whytheluckystiff.net/hpricot/" title="Hpricot, a fast and delightful HTML parser">Hpricot</a> has a new native extension, <code>fast_xs</code>, which is an almost drop-in replacement for the pure-ruby <code>String#to_xs</code>. (Almost, because it creates the method <code>String#fast_xs</code> instead of <code>String#to_xs</code>. ActiveSupport 2.0.2 and later <a href="http://dev.rubyonrails.org/browser/trunk/activesupport/lib/active_support/core_ext/string/xchar.rb?rev=7773">take care of aliasing it for you</a>). Unbeknownst to me, I ported <code>fast_xs</code> recently as part of upgrading JRuby extensions that have Java code in them. And so it happens to come in handy at this time. The patch for that is <a href="http://code.whytheluckystiff.net/hpricot/ticket/131">here</a>.</p> <p>I have the latest Hpricot gems on my server, so you can install it yourself (for either Ruby or JRuby):</p> <pre><code>gem install hpricot --source http://caldersphere.net </code></pre> <p>or</p> <pre><code>jruby -S gem install hpricot --source http://caldersphere.net </code></pre> <p>With that installed, the script now produces these results:</p> <pre><code>$ ruby to_xs.rb 0.460000 0.080000 0.540000 ( 0.537793) 0.420000 0.070000 0.490000 ( 0.501965) 0.430000 0.070000 0.500000 ( 0.501359) 0.400000 0.070000 0.470000 ( 0.484495) 0.400000 0.070000 0.470000 ( 0.479995) 0.400000 0.070000 0.470000 ( 0.469118) 0.390000 0.070000 0.460000 ( 0.468864) 0.390000 0.070000 0.460000 ( 0.465009) 0.390000 0.060000 0.450000 ( 0.452902) 0.390000 0.070000 0.460000 ( 0.466881) Mean: 0.482838 SDev: 0.024926 $ jruby -J-server to_xs.rb 0.882000 0.000000 0.882000 ( 0.883000) 0.832000 0.000000 0.832000 ( 0.832000) 0.851000 0.000000 0.851000 ( 0.850000) 0.837000 0.000000 0.837000 ( 0.837000) 0.846000 0.000000 0.846000 ( 0.846000) 0.843000 0.000000 0.843000 ( 0.843000) 0.835000 0.000000 0.835000 ( 0.835000) 0.825000 0.000000 0.825000 ( 0.826000) 0.830000 0.000000 0.830000 ( 0.830000) 0.834000 0.000000 0.834000 ( 0.833000) Mean: 0.841500 SDev: 0.016379 </code></pre>