Nick Sieger: Tag ruby http://blog.nicksieger.com/articles/tag/ruby?tag=ruby en-us 40 RubyConf India 2010! <p>I&#8217;m honored to be heading to the first ever <a href="http://www.rubyconfindia.org/"><strong>RubyConf India</strong></a> in Bangalore next week&#46; I&#8217;ll be delivering an update on Rails 3, JRuby, and what&#8217;s in store for the future of the combined platform in 2010&#46; I&#8217;m looking forward to meeting you there!</p> <p><a href="http://rubyconfindia.org"><img src="http://rubyconfindia.org/stock/rubyconf-badges/RubyConf2010/270X185_speaking.jpg" alt="I'm speaking at RubyConf India 2010"></a></p> Fri, 12 Mar 2010 23:21:04 +0000 urn:uuid:b2e0444e-6cf4-468d-a222-0f5a783aed94 Nick Sieger http://blog.nicksieger.com/articles/2010/03/12/rubyconf-india-2010 ruby rubyconf rubyconfindia http://blog.nicksieger.com/articles/trackback/552 geekSessions I: Ruby on Rails: To Scale or Not to Scale <p>I was fortunate to be in town right after RailsConf and attended the inaugural <a href="http://www.geeksessions.com/">geekSessions</a> event on Rails scalibility&#46; The event went off without a hitch: it was well attended, City Club is a classy place, and there was decent food and an open bar&#46; I don&#8217;t know the SF geek/startup scene, but pretty much all of the few guys I know were there along with a ton of other folks&#46; My only complaint would have been to let it run at least 30 minutes longer&#46; Socializing was good too, but it seemed like the conversation was just getting started&#46;</p> <p>Here are some notes for you in my typical rapid&#45;fire style &#45;&#45; hope they&#8217;re useful to you&#46;</p> <h2>Ian McFarland</h2> <p>Case study: divine caroline</p> <p>Servers:</p> <ul> <li>Load balancer</li> <li>Apache + mongrel</li> <li>MySQL</li> <li>SOLR</li> </ul> <p>Ruby is slow&#46; Rails is slow&#46; Unoptimized app was slow &#45;&#45; 7 pages/sec with <code>ab</code>&#46; So how can Rails possibly be? 150 pv/s with a simple text render&#46; This formed a sort of upper&#45;bound, that ruled out fragment/action/partial caching, etc&#46; This brought the throughput to 3500 pv/s&#46; Except for page caching limitations:</p> <ul> <li>Cache coherency</li> <li>Writes are more expensive</li> <li>Page caching is not applicable to as many pages as you think</li> </ul> <p>But measure first&#46; Pivotal built a drop&#45;in page caching extension to deal with cache coherency issues (soon to be at http://rubyforge&#46;org/projects/pivotalrb)</p> <h2>Jason Hoffman</h2> <p>Jason somehow has the distinction of the first four commits in the Rails repository&#46; Joyent/TextDrive/Strongspace&#46;</p> <p>If your application is successful, you&#8217;re going to have a lot of machines&#46; What happens when you have 1000s of machines, 100s of TB, 4 locations, etc&#46; Is this really a <em>Rails</em> issue? In a typical Joyent setup, Rails is only one of 26+ processes on the server stack&#46; So scaling it really doesn&#8217;t mean much more than scaling any application&#46; Object creation in Ruby is fast, sockets and threads are slow&#46; So forget sockets and threads&#46;</p> <p>Instead, use DNS, load balancers, evented mongrels, JRuby/Java, DBMSes (not just RDBMS; LDAP, filesystem, etc&#46;), Rails process doing Rails only, static assets going through a static server, federate and separate as much as you can&#46;</p> <h2>Jeremy LaTrasse</h2> <p>Jeremy&#8217;s job is about safety nets; about knowing the underlying infrastructure&#46; Is the hardware/OS/stack important? Can you build safety nets around those so that you can spare cycles when you need to intrude into the system to troubleshoot?</p> <p>Twitter is in a unique position with the volume of traffic to be able to find some pretty tough bugs, like the recent <a href="http://dev.rubyonrails.org/changeset/6571">backtrace issue</a>&#46;</p> <h2>Bryan Cantrill</h2> <p>Measure first! Like Ian said&#46; Is software information? Or a machine? It&#8217;s both&#46; Nothing else in human existence can claim this&#46; 3 weeks after Bryan joined Sun, he was working with Jeff (ZFS architect) debugging an issue when Jeff retorted, &#8220;Does it bother you that none of this exists? It&#8217;s just a representation of some plastic and metal morass in a backroom&#8221; (slightly paraphrased)&#46;</p> <p>We&#8217;ve been living with bifurcated code &#45;&#45; &#8220;if DEBUG; print something&#8221; ad nauseum&#46; But this has a cost&#46; So dev code deviates from production code&#46; But we can&#8217;t get the data we want, where it matters, in production&#46; Bryan goes on to describe the aforementioned <a href="http://dev.rubyonrails.org/changeset/6571">backtrace issue</a> and how it saved Twitter 33% CPU&#46; So don&#8217;t pre&#45;optimize, but you&#8217;ve got to be prepared to go get the data&#46; In production&#46;</p> <h2>Q &amp; A</h2> <p><em>What&#8217;s the best way to move from one database to two databases (MySQL), when you scale past the volume of reads that overwhelms one?</em></p> <p><strong>Jason</strong> doesn&#8217;t like the replication approach, it&#8217;s not fault tolerant&#46; Reference to <a href="http://drnicwilliams.com/2007/04/12/magic-multi-connections-a-facility-in-rails-to-talk-to-more-than-one-database-at-a-time/">Dr Nic&#8217;s magic multi&#45;connections gem</a>&#46; Reference to <a href="http://revolutiononrails.blogspot.com/2007/04/plugin-release-actsasreadonlyable.html">acts_as_readonly</a>&#46; Don&#8217;t rely on things that are out of your control, start reading/writing to multiple locations, at the application level&#46; <strong>Jeremy</strong>: So do you want to be in the business of writing SQL or C extensions to Rails? What about <a href="http://freshmeat.net/projects/mysql_proxy/">MySQL proxy</a>? Seems ok, but I might not trust it in production&#46; <a href="http://jeremy.zawodny.com/mysql/mytop/" title="mytop - a top clone for MySQL">MyTop</a>/<a href="http://www.xaprb.com/blog/2006/07/02/innotop-mysql-innodb-monitor/">InnoTop</a> will tell you about your query volume&#46;</p> <p><em>Virtualization: 4 virtual servers w/ web servers on top of a single physical server? Why?</em></p> <p><strong>Jason</strong>: Free BSD 4&#46;9 on early pentium was the perfect balance of utilization&#46; 18 CPUs by 64G RAM with virtual servers gets us back to that level of utilization&#46; <strong>Bryan</strong>: Not all virtualization solutions are equivalent! (Solaris containers/zones plug&#46;)</p> <p><em>RDBMSes are not good for web applications? Why? Can you give some examples?</em></p> <p><strong>Jason</strong>: It depends on when you want to join&#46; When people are clicking, or pre&#45;assembled&#46; Look at your application and put the data together before people request it&#46; Why does YouTube need an RDBMS? It serves a file that people can comment on&#46;</p> <p>Mention of Dabble DB, ZFS, Jabber, Atom, Atom over Jabber, etc&#46; as ways of innovative ways of storing objects, data, etc&#46; GData/GCal most certainly does not store its Atom files in an RDBMS&#46;</p> <p><em>Sell Rails apps and have the customer deploy it? What options are available?</em></p> <p><strong>Ian</strong>: JRuby on Rails with a &#46;war file is an interesting approach&#46; <em>What operational issues/ways to help with scaling remote deployments?</em> <strong>Jeremy</strong>: Log files are the first line of defense&#46; <strong>Jason</strong>: Corporate IT are comfortable with Java&#46;</p> <p><em>The pessimist in me says that my servers are going to fall over after 5 users&#46; How can I be prepared/not be optimistic about a traffic spike?</em></p> <p><strong>Ian</strong>: Load test the crap out of the app&#46; Find out the horizontal scaling point&#46; Use solutions like S3 for images&#46; Make sure you can scale by throwing hardware at it&#46; Eventually single points of failure will overcome you (such as a single database), but you can wait until you get to that point before doing something about it&#46;</p> <p><strong>Jason</strong>: You can benchmark your processes, and get an idea of what they can do&#46; Most people that want to do something will be look at your stuff, and maybe signup&#46; So front&#45;load and optimize your signup process, possibly by taking it out of Rails&#46;</p> <p><strong>Jeremy</strong>: Conversations with Zed, DHH, etc&#46; have pointed out that sometimes &#8220;Rails isn&#8217;t good at that, take it out of Rails&#46;&#8221; Same thing for the database&#46; Split those things out into a different application&#46;</p> <p><strong>Bryan</strong>: Do your dry land work, know your toolchain, so that when the moment comes, you can dive in and find the problem&#46;</p> <p><em>We have a migration that takes a week to run because of text processing&#46; GC was running after every 10th DB statement&#46; Used Rails bench GC patch to overcome the issue with the migration&#46; Any issue running these?</em></p> <p><strong>Jason</strong>: We run those GC modifications and a few more in production, and they&#8217;re fine&#46;</p> <p><em>Most comversations revolve around items like database is slow, or Ruby is slow&#46; How can we use DTrace to streamline the process?</em></p> <p><strong>Jeremy</strong>: We spent 20 minutes over lunch (plus some preparation) to find a Memcache issue&#46; It&#8217;s worth it to spend a little time to learn the tool&#46;</p> <p><strong>Bryan</strong>: &#8220;Awk is God&#8217;s gift to all of us&#46;&#8221; When DTrace was being reviewed inside of Sun, folks commented &#8220;This reminds us of awk&#46;&#8221; &#8220;Thanks!&#8221;</p> <p><strong>Jason</strong>: We&#8217;re putting a tracing plugin in Rails as a remote process to collect data from a running app&#46; Apple has shown a commitment to get this in Leopard&#46; Textual and graphical output are possible&#46; I believe in DTrace a lot, and the tooling and documentation will go beyond its current state of an experts tool&#46;</p> <p><em>Lastly, what one closing thing would you like to say about Rails scalability?</em></p> <p><strong>Ian</strong>: Measure&#46;<br/> <strong>Jason</strong>: Don&#8217;t use relational databases&#46;<br/> <strong>Jeremy</strong>: I thought it was a Joyent sales pitch&#46;<br/> <strong>Bryan</strong>: Use DTrace (with Joyent accelerators of course)&#46;<br/></p> Wed, 23 May 2007 05:51:36 +0000 urn:uuid:ab53b976-cff8-410d-8047-793abbb363a2 Nick Sieger http://blog.nicksieger.com/articles/2007/05/23/geeksessions-i-ruby-on-rails-to-scale-or-not-to-scale rails ruby http://blog.nicksieger.com/articles/trackback/254 JRuby on Rails and ActiveRecord on JDBC <p><a href="http://www.bloglines.com/blog/ThomasEEnebo?id=16">Tom</a> and <a href="http://headius.blogspot.com/2006/05/and-they-said-jruby-was-dead.html">Charlie</a> have just experienced what can only be described as a watershed moment in the grand scheme of dynamic languages on the JVM&#46; The Rails experience may soon be visiting a Java application server near you! Even though JRuby will be in perpetual catch&#45;up mode with C Ruby, Tom and Charlie and the rest of the JRuby contributors have shown incredible perseverance in tracking the Ruby language despite <em>the lack of any formal specification</em>&#46; Maybe Rails will never be mainstream, but the possibilities just got a whole lot more interesting&#46; I agree with <a href="http://jroller.com/page/obie?entry=jruby_on_rails_is_born">Obie</a> that this could be a game&#45;changer&#46;</p> <p>Now, a few comments about the ActiveRecord JDBC adapter&#46; This code can still be considered alpha quality at best&#46; It&#8217;s awesome that Tom and Charlie will be able to demo a top&#45;to&#45;bottom, working Rails app on JRuby, but don&#8217;t jump to conclusions yet that this will be anything like a write&#45;once, deploy&#45;to&#45;any&#45;database kind of experience&#46; But you didn&#8217;t think it would, did you? We all learned that about Java a long time ago, right?</p> <p>You can <a href="http://svn.caldersphere.net/svn/main/activerecord-jdbc/trunk/">check out the code here</a>&#46; At the moment, I&#8217;ve only tried it with MySQL&#46; Most of the problems with it come from the lossy mapping from ActiveRecord&#8217;s abstraction of the database to JDBC&#8217;s&#46; (Although I suspect as more JDBC drivers are tried that there will eventually be compatibility issues with different implementations of the JDBC spec&#46;)</p> <p>Probably the thorniest issue is the one of type conversion&#46; ActiveRecord has a fairly simple notion of types: <code>:string, :text, :integer, :float, :datetime</code> etc&#46; Compare this to <a href="http://java.sun.com/j2se/1.4.2/docs/api/java/sql/Types.html">JDBC&#8217;s</a>&#46; What a mess! Right now there are arrays of <code>proc</code>s for each AR type that try to guess the best JDBC type to use&#46; This will certainly need improvement to become a more robust solution&#46;</p> <p>Probably the most promising approach may be to create a patch that refactors much of the AR adapters&#8217; type conversion methods into separate modules that could be included into instances of the JDBC adapter depending on the underlying database&#46; Then the JDBC types wouldn&#8217;t be needed at all &#45;&#45; the existing AR database metadata could be reused&#46; Presumably this would require some petitioning of the Rails core team to accept the changes even though the changes don&#8217;t buy AR itself any additional flexibility&#46;</p> <p>If you have a chance to try out the code or can think of any additional tricks that would help the ActiveRecord JDBC implementation along, let me know!</p> Mon, 15 May 2006 03:12:00 +0000 urn:uuid:091b1464-16fe-49ee-9d4b-ce18ddf2da0d Nick Sieger http://blog.nicksieger.com/articles/2006/05/15/jruby-on-rails-and-activerecord-on-jdbc java ruby jruby rails http://blog.nicksieger.com/articles/trackback/13 JRuby on Rails: Fast Enough <p>People have been asking for a while how fast JRuby runs Rails&#46; (Of course, &#8220;fast&#8221; has always been a <a href="http://www.joelonsoftware.com/items/2006/09/12.html">relative term</a>&#46;) We haven&#8217;t been quick to answer the question, because frankly we didn&#8217;t know&#46; We hadn&#8217;t been building real Rails applications on JRuby ourselves yet, and there was no definitive word from the crowd either&#46;</p> <p>Recently, several guys from <a href="http://studios.thoughtworks.com/rubyworks">ThoughtWorks</a> have been working on a <a href="http://tw-commons.rubyforge.org/svn/petstore/trunk/">Rails petstore</a> application and benchmark to get to the heart of the matter&#46; <a href="http://www.nabble.com/forum/Search.jtp?forum=14106&amp;local=y&amp;query=petstore">Discussion has been heated</a> on the JRuby mailing list, but results have not been conclusive yet&#46;</p> <p>In the project I&#8217;m working on, we&#8217;ve <a href="http://blog.nicksieger.com/articles/2007/10/06/railsconf-europe-hydra">committed to using and deploying on JRuby</a>&#46; Eventually we were going to reach the point where we&#8217;d need to find out how well our application runs&#46; So today I began running a simple single request benchmark on a relatively busy page&#46; The numbers turned out to be rather surprising:</p> <p><img src="/files/jr-requests.png" alt="Requests" title="Requests"/></p> <p><img src="/files/jr-average.png" alt="Average" title="Average"/></p> <p>(The <a href="http://spreadsheets.google.com/pub?key=pGobleZnKsdI1zW38xpNaaw">raw data is available here</a>&#46;)</p> <p>Now, MRI (C Ruby) will always run about the same speed no matter how many runs you give it, but it&#8217;s well known that the JVM <a href="http://www.javaworld.com/javaworld/javaqa/2003-04/01-qa-0411-hotspot.html">needs time to warm up</a>&#46; And indeed it does; after 250 iterations, Mongrel running on JRuby finally surpasses MRI&#46; The JRuby/Goldspike/Glassfish combo comes close as well&#46;</p> <p>Some details about the setup:</p> <ul> <li>I ran the tests on my MacBook Pro Core 2 Duo 2&#46;4 GHz&#46; I didn&#8217;t disable one of the cores for the tests, which means that JRuby has an advantage over MRI because it can use both (native threads at work)&#46; However, the test script ran the requests serially, which means that the advantage was minimal&#46;</li> <li>The application is indeed of the <a href="http://blog.nicksieger.com/articles/2007/10/06/railsconf-europe-hydra">&#8220;hydra&#8221;</a> variety; the setup is nearly identical to the second diagram on that page&#46; So a single request is passing through <strong>not one, but two</strong> Rails applications in addition to touching the database&#46; It rendered an HTML ERb view with data from an ActiveResource&#45;accessed RESTful service&#46; The applications are based on Rails 1&#46;2&#46;3&#46;</li> <li>MRI version is using Ruby 1&#46;8&#46;6 and Mongrel 1&#46;0&#46;1&#46;</li> <li>JRuby Mongrel is also version 1&#46;0&#46;1 (<a href="http://jxh.bingodisk.com/bingo/public/Hackdays/JRuby/">details on installing it here</a>)</li> <li>JRuby on Glassfish used Glassfish 2 and Goldspike 1&#46;4, deployed in war files via <a href="http://caldersphere.rubyforge.org/warbler">Warbler</a>&#46;</li> <li>The two JRuby setups used JDK 1&#46;5 and were tweaked to <a href="http://ola-bini.blogspot.com/2007/07/objectspace-to-have-or-not-to-have.html">disable ObjectSpace</a> and use the &#8220;server&#8221; VM (&#45;server argument to the JVM)&#46;</li> </ul> <p>The main point I wish to make with these numbers is that JRuby performance is there today, and still has room to grow&#46; There&#8217;s no longer any doubt in my mind&#46; Yes, this is a simplistic application benchmark run on a developer&#8217;s machine, but it&#8217;s a real application&#46; The test may not be exacting in precision, but I see enough in the numbers to believe that this will be replicable to production environments&#46; The plot thickens!</p> Thu, 25 Oct 2007 03:36:00 +0000 urn:uuid:bfda082f-fa26-41d8-aa16-fc07b41aba35 Nick Sieger http://blog.nicksieger.com/articles/2007/10/25/jruby-on-rails-fast-enough jruby ruby rails http://blog.nicksieger.com/articles/trackback/335 Next performance fix: Builder::XChar <p>Next up in our performance series: <code>Builder::XChar</code>&#46; (Another fine Sam Ruby production!) While this piece of code in the Builder library strikes me as perfectly fine, it also tends to slow down quite a bit with larger documents or chunks of text&#46;</p> <p>Our path to the bottleneck is as follows: <code>ActiveRecord::Base#to_xml =&gt; Builder::XMLMarkup#text! =&gt; String#to_xs =&gt; Fixnum#xchr</code>&#46; Consider:</p> <div class="typocode"><pre><code class="typocode_ruby "><span class="ident">require</span> <span class="punct">'</span><span class="string">rubygems</span><span class="punct">'</span> <span class="ident">gem</span> <span class="punct">'</span><span class="string">activesupport</span><span class="punct">'</span> <span class="ident">require</span> <span class="punct">'</span><span class="string">active_support</span><span class="punct">'</span> <span class="ident">require</span> <span class="punct">'</span><span class="string">benchmark</span><span class="punct">'</span> <span class="keyword">module </span><span class="module">Benchmark</span> <span class="keyword">class </span><span class="punct">&lt;&lt;</span> <span class="constant">self</span> <span class="keyword">def </span><span class="method">report</span><span class="punct">(&amp;</span><span class="ident">block</span><span class="punct">)</span> <span class="ident">n</span> <span class="punct">=</span> <span class="number">10</span> <span class="ident">times</span> <span class="punct">=</span> <span class="punct">(</span><span class="number">1</span><span class="punct">..</span><span class="number">10</span><span class="punct">).</span><span class="ident">map</span> <span class="keyword">do</span> <span class="ident">bm</span> <span class="punct">=</span> <span class="ident">measure</span><span class="punct">(&amp;</span><span class="ident">block</span><span class="punct">)</span> <span class="ident">puts</span> <span class="ident">bm</span> <span class="ident">bm</span> <span class="keyword">end</span> <span class="ident">sum</span> <span class="punct">=</span> <span class="ident">times</span><span class="punct">.</span><span class="ident">inject</span><span class="punct">(</span><span class="number">0</span><span class="punct">)</span> <span class="punct">{|</span><span class="ident">s</span><span class="punct">,</span><span class="ident">t</span><span class="punct">|</span> <span class="ident">s</span> <span class="punct">+</span> <span class="ident">t</span><span class="punct">.</span><span class="ident">real</span><span class="punct">}</span> <span class="ident">mean</span> <span class="punct">=</span> <span class="ident">sum</span> <span class="punct">/</span> <span class="ident">n</span> <span class="ident">sumsq</span> <span class="punct">=</span> <span class="ident">times</span><span class="punct">.</span><span class="ident">inject</span><span class="punct">(</span><span class="number">0</span><span class="punct">)</span> <span class="punct">{|</span><span class="ident">s</span><span class="punct">,</span><span class="ident">t</span><span class="punct">|</span> <span class="ident">s</span> <span class="punct">+</span> <span class="ident">t</span><span class="punct">.</span><span class="ident">real</span> <span class="punct">*</span> <span class="ident">t</span><span class="punct">.</span><span class="ident">real</span><span class="punct">}</span> <span class="ident">sd</span> <span class="punct">=</span> <span class="constant">Math</span><span class="punct">.</span><span class="ident">sqrt</span><span class="punct">((</span><span class="ident">sumsq</span> <span class="punct">-</span> <span class="punct">(</span><span class="ident">sum</span> <span class="punct">*</span> <span class="ident">sum</span> <span class="punct">/</span> <span class="ident">n</span><span class="punct">))</span> <span class="punct">/</span> <span class="punct">(</span><span class="ident">n</span> <span class="punct">-</span> <span class="number">1</span><span class="punct">))</span> <span class="ident">puts</span><span class="punct">(&quot;</span><span class="string">Mean: %0.6f SDev: %0.6f</span><span class="punct">&quot;</span> <span class="punct">%</span> <span class="punct">[</span><span class="ident">mean</span><span class="punct">,</span> <span class="ident">sd</span><span class="punct">])</span> <span class="keyword">end</span> <span class="keyword">end</span> <span class="keyword">end</span> <span class="comment"># http://blog.nicksieger.com/files/page.xml</span> <span class="ident">page</span> <span class="punct">=</span> <span class="constant">File</span><span class="punct">.</span><span class="ident">open</span><span class="punct">(&quot;</span><span class="string">page.xml</span><span class="punct">&quot;)</span> <span class="punct">{|</span><span class="ident">f</span><span class="punct">|</span> <span class="ident">f</span><span class="punct">.</span><span class="ident">read</span> <span class="punct">}</span> <span class="constant">Benchmark</span><span class="punct">.</span><span class="ident">report</span> <span class="keyword">do</span> <span class="number">20</span><span class="punct">.</span><span class="ident">times</span> <span class="punct">{</span> <span class="ident">page</span><span class="punct">.</span><span class="ident">to_xs</span> <span class="punct">}</span> <span class="keyword">end</span></code></pre></div> <p>On Ruby and JRuby, this produces:</p> <pre><code>$ ruby to_xs.rb 21.430000 0.400000 21.830000 ( 22.022769) 21.530000 0.360000 21.890000 ( 22.005737) 21.540000 0.370000 21.910000 ( 22.065165) 21.530000 0.370000 21.900000 ( 22.028591) 21.500000 0.350000 21.850000 ( 21.990395) 21.550000 0.370000 21.920000 ( 22.033164) 21.520000 0.360000 21.880000 ( 21.984129) 21.550000 0.370000 21.920000 ( 22.116802) 21.550000 0.370000 21.920000 ( 22.051421) 21.520000 0.380000 21.900000 ( 22.084736) Mean: 22.038291 SDev: 0.041985 $ jruby -J-server to_xs.rb 79.112000 0.000000 79.112000 ( 79.112000) 81.480000 0.000000 81.480000 ( 81.481000) 84.745000 0.000000 84.745000 ( 84.745000) 84.384000 0.000000 84.384000 ( 84.384000) 121.933000 0.000000 121.933000 (121.933000) 85.533000 0.000000 85.533000 ( 85.532000) 82.762000 0.000000 82.762000 ( 82.763000) 82.090000 0.000000 82.090000 ( 82.090000) 81.298000 0.000000 81.298000 ( 81.299000) 80.774000 0.000000 80.774000 ( 80.773000) Mean: 86.411200 SDev: 12.635700 </code></pre> <p>(Hmm, I must have accidentally swapped in some large program in the middle of that JRuby run&#46; The perils of benchmarking on a desktop machine&#46; I don&#8217;t claim that the numbers are scientific, just illustrative!)</p> <p>Fortunately, the fix again is very simple, and has <a href="http://groups.google.com/group/rubyjam/browse_thread/thread/82a9ddb762019bcc">previously</a> <a href="http://dev.rubyonrails.org/changeset/7773">been acknowledged</a>&#46; The latest (unreleased?) <a href="http://code.whytheluckystiff.net/hpricot/" title="Hpricot, a fast and delightful HTML parser">Hpricot</a> has a new native extension, <code>fast_xs</code>, which is an almost drop&#45;in replacement for the pure&#45;ruby <code>String#to_xs</code>&#46; (Almost, because it creates the method <code>String#fast_xs</code> instead of <code>String#to_xs</code>&#46; ActiveSupport 2&#46;0&#46;2 and later <a href="http://dev.rubyonrails.org/browser/trunk/activesupport/lib/active_support/core_ext/string/xchar.rb?rev=7773">take care of aliasing it for you</a>)&#46; Unbeknownst to me, I ported <code>fast_xs</code> recently as part of upgrading JRuby extensions that have Java code in them&#46; And so it happens to come in handy at this time&#46; The patch for that is <a href="http://code.whytheluckystiff.net/hpricot/ticket/131">here</a>&#46;</p> <p>I have the latest Hpricot gems on my server, so you can install it yourself (for either Ruby or JRuby):</p> <pre><code>gem install hpricot --source http://caldersphere.net </code></pre> <p>or</p> <pre><code>jruby -S gem install hpricot --source http://caldersphere.net </code></pre> <p>With that installed, the script now produces these results:</p> <pre><code>$ ruby to_xs.rb 0.460000 0.080000 0.540000 ( 0.537793) 0.420000 0.070000 0.490000 ( 0.501965) 0.430000 0.070000 0.500000 ( 0.501359) 0.400000 0.070000 0.470000 ( 0.484495) 0.400000 0.070000 0.470000 ( 0.479995) 0.400000 0.070000 0.470000 ( 0.469118) 0.390000 0.070000 0.460000 ( 0.468864) 0.390000 0.070000 0.460000 ( 0.465009) 0.390000 0.060000 0.450000 ( 0.452902) 0.390000 0.070000 0.460000 ( 0.466881) Mean: 0.482838 SDev: 0.024926 $ jruby -J-server to_xs.rb 0.882000 0.000000 0.882000 ( 0.883000) 0.832000 0.000000 0.832000 ( 0.832000) 0.851000 0.000000 0.851000 ( 0.850000) 0.837000 0.000000 0.837000 ( 0.837000) 0.846000 0.000000 0.846000 ( 0.846000) 0.843000 0.000000 0.843000 ( 0.843000) 0.835000 0.000000 0.835000 ( 0.835000) 0.825000 0.000000 0.825000 ( 0.826000) 0.830000 0.000000 0.830000 ( 0.830000) 0.834000 0.000000 0.834000 ( 0.833000) Mean: 0.841500 SDev: 0.016379 </code></pre> Thu, 17 Jan 2008 23:48:00 +0000 urn:uuid:fe7e8324-82de-49dc-a132-f1e514007cdd Nick Sieger http://blog.nicksieger.com/articles/2008/01/17/next-performance-fix-builder-xchar jruby performance ruby rails http://blog.nicksieger.com/articles/trackback/366 RubyConf 2006 Begins <p>The RubyConf room is filling up this morning&#46; I&#8217;ll be doing my best to live&#45;blog the conference here so stay tuned!</p> <p><a href="http://www.flickr.com/photos/nicksieger/274770531/" title="Photo Sharing"><img src="http://static.flickr.com/110/274770531_01f9ba4a44_m.jpg" width="240" height="160" alt="DSCF4909.JPG" /></a></p> <p><a href="http://www.flickr.com/photos/nicksieger/274770462/" title="Photo Sharing"><img src="http://static.flickr.com/79/274770462_957803fcd8_m.jpg" width="240" height="160" alt="DSCF4908.JPG" /></a></p> Fri, 20 Oct 2006 19:04:00 +0000 urn:uuid:0de6115d-57e6-482e-9c45-964a94088b5b Nick Sieger http://blog.nicksieger.com/articles/2006/10/20/rubyconf-2006-begins ruby rubyconf rubyconf2006 ruby http://blog.nicksieger.com/articles/trackback/74 RubyConf: Sydney and Rubinius <p><em>Update: Evan has posted code and has <a href="http://blog.fallingsnow.net/rubinius/">a page set up for the project</a>&#46;</em> </p> <p>Evan Phoenix (nee Webb), of Seattle&#46;rb, is presenting on Sydney and Rubinius, an experiment in improving the ruby interpreter&#46; Sydney has died, and Rubinius has risen from its ashes, appropriately&#46;</p> <h2>Why</h2> <ul> <li>Why would you write a new Ruby interpreter? It&#8217;s fun, it&#8217;s a good challenge&#46;</li> <li>What&#8217;s wrong with the existing interpreter &#45;&#45; are you hating on Matz? Of course not&#46;</li> </ul> <p>Today&#8217;s Ruby interpreter is like a big dump truck &#45;&#45; sometimes a little slow, but it works for us&#46; YARV is like the red, shiny fire truck&#46; Both big and complex&#46; Rubinius, by comparison, is like a dune buggy&#46; Fast, light, but you&#8217;re going to get sand in your eyes if you drive it a lot&#46;</p> <p>The project, admittedly, is naive&#46;</p> <ul> <li>Simple architecture and implementation&#46;</li> <li>As little background magic as possible</li> <li>No opaque C backend</li> <li>Leverage axiom of simple == powerful</li> <li>Less magic means more introspection <ul> <li>More control for the developer</li> <li>Richer introspection: Backtrace, MethodTable objects</li> </ul></li> </ul> <h2>What was Sydney?</h2> <ul> <li>Giant patch to 1&#46;8&#46;2 that included reentrancy and thread&#45;safety</li> <li>Turned out to be a major PITA</li> <li>CRuby uses a large number of C globals, references to which had to be tracked and fixed</li> </ul> <h2>Transition to Rubinius</h2> <ul> <li>Ruby borrowed a lot from Smalltalk, so why not try an implementation based on the same concepts?</li> <li>Prototype A ported the blue&#45;book implementation to Ruby</li> <li>It worked and validated the basic concept and approach</li> <li>Prototype B took ideas from A but implemented a bytecode interpreter and compiler&#46; Used RubyInline to access raw memory operations&#46;</li> <li>At this time the goal emerged to have a translator which could take a prototype and bootstrap itself into C code&#46;</li> <li>Prototype S was a manual translation of Prototype B into C code to make the implementation quicker&#46;</li> <li>Prototype W was created to translate parts of Prototype B so that there is a maintainable core in Ruby code itself&#46;</li> </ul> <h2>Questions</h2> <p><em>Q&#46; Since you were starting over, could you use a platform&#45;independent library to ease the process, such as APR?</em> Yes &#45;&#45; currently using String and PointerArray from glib&#46;</p> <p><em>Q&#46; How is performance?</em> Too early to tell &#45;&#45; I hope to know by the end of the conference&#46; Prototype S became runnable and usable on the plane here&#46;</p> <p><em>Q&#46; Can you clarify the goal?</em> To create a Ruby interpreter in Ruby that can translate itself out into a C interpreter&#46;</p> <p><em>Q&#46; Have you figured out how to link in external libraries in a platform independent way?</em> No&#46; My hope is that the decision will be made to write a common framework for translating to system calls, e&#46;g&#46;, SWT&#46;</p> <p><em>Q&#46; Have you looked at PyPy? (similar project for Python)</em> Yes, and it&#8217;s f&#45;in complicated&#46; It worries me actually&#46;</p> <p><em>Q&#46; Could you have it generate backend code in another language/platform (Java bytecode, CLR)?</em> Yes, I certainly hope so, otherwise I&#8217;m wasting my time&#46;</p> <p><em>Q&#46; How will you add native thread support in a cross&#45;platform way?</em> I hope I won&#8217;t have to, by leveraging external tools&#46;</p> <p><em>Q&#46; If you&#8217;re building a Ruby&#45;to&#45;C translator, why write a Ruby interpreter at all?</em> If I didn&#8217;t, what would I translate? You still need some core engine to translate&#46; <em>Would it be a subset of Ruby?</em> Yes&#46;</p> <p><em>Q&#46; Looks very similar to Squeak, have you looked at Squeak code and talked to Squeak people?</em> Looked at code a lot, I&#8217;ve really stolen all of their ideas&#46; I haven&#8217;t talked to the folks yet because I&#8217;m afraid they might laugh at me&#46;</p> <h2>Resulting Works</h2> <ul> <li>SydneyParser: Used parser from Sydney and stole ParseTree&#8217;s algorithm for generating a sexp that represents the Ruby code&#46;</li> <li>SegfaultProtection: detects a segfault in an extension, saves the Ruby interpreter, and raises a memory fault exception instead&#46;</li> </ul> <h2>The Nitty Gritty (Red Pill)</h2> <ul> <li>All components separated by APIs for swappability</li> <li>Garbage collector: baker two&#45;space copy collector, and a train GC</li> <li>Bytecode interpreter: small set of instructions driven by tests and need, so there are no extraneous operations</li> <li>Compiler: written completely in Ruby, using ParseTree and SexpProcessor&#46; Intended to compile itself to be used as a base compiler for Prototype S&#46;</li> </ul> <h2>Future</h2> <ul> <li>Other backends &#45;&#45; Java, Smalltalk</li> </ul> <h2>More questions</h2> <p><em>Q&#46; Worried about fragmentation?</em> Yes, but I really want to make it as compatible as possible with the current interpreter&#46;</p> <p><em>Q&#46; Rubinius bytecode compatibile with YARV?</em> No, but I hope to be able to write a bridge to YARV in Rubinius&#46;</p> <p><em>Q&#46; Have you looked at Valgrind for the C code?</em> Yes, I have&#46; Good possibility for future direction&#46;</p> <p><em>Q&#46; Can you demo some code?</em> They&#8217;re incredibly boring&#46; &#8220;Look I got a MethodTable object, I asked for one&#46;&#8221;</p> Fri, 20 Oct 2006 19:07:00 +0000 urn:uuid:9785ffcc-1680-492c-8b42-6e9111cc85aa Nick Sieger http://blog.nicksieger.com/articles/2006/10/20/rubyconf-sydney-and-rubinius ruby rubyconf rubyconf2006 ruby http://blog.nicksieger.com/articles/trackback/76 New Hpricot Release <p>It&#8217;s with a modicum of fanfare and a cocktail of orange peel, maraschino, bitters, bourbon and vermouth that I announce the 0&#46;8&#46;2 release of Hpricot&#46;</p> <p><img src="http://hpricot.github.com/hpricot/images/hpricot.png" alt="hpricot"/></p> <div style="clear:both;"></div> <p>The angly thingies are just two martini glasses turned on their side, see? Pouring out hpricoty goodness for you&#46; (I don&#8217;t know if <a href="http://en.wikipedia.org/wiki/Why_the_lucky_stiff">_why</a> imbibed alcoholic beverages but I feel the desire to raise a glass to him&#46;)</p> <p>This release is mostly a refresh; the previous release was way back in April&#46; From the <a href="http://github.com/hpricot/hpricot/commits/master/">Git logs</a> I can see that there were a few bug fixes since then&#46; Otherwise, the main addition is a modern JRuby release, thanks to <a href="http://olabini.com/">Ola Bini</a>&#46; (the previous was the 0&#46;6 series)&#46; This does <a href="http://jira.codehaus.org/browse/JRUBY-3732">fix a fairly old, popular JRuby bug</a>&#46;</p> <p>As for the future of Hpricot, it&#8217;s up to you&#46; I know it&#8217;s still a trusty tool for many; I have no grand plans to change it&#46; So if you encounter bugs and want to send patches, I&#8217;m happy to serve as your curator&#46;</p> Fri, 06 Nov 2009 16:06:37 +0000 urn:uuid:93e42dac-f363-41aa-bd63-24e5186d6a3b Nick Sieger http://blog.nicksieger.com/articles/2009/11/06/new-hpricot-release ruby rails jruby hpricot http://blog.nicksieger.com/articles/trackback/538 Caught redHanded <p>Ok, ok, so I&#8217;ve been caught <a href="http://redhanded.hobix.com/cult/hePaidToGoAndTypeAllDay.html">redHanded</a>&#46; Yeah I know, <a href="http://www.edwardtufte.com/">Tufte</a> hates bullets&#46; You might feel a strange sensation if you stare at my transcriptions of today too long&#46; If anyone is offended or would like to claim I&#8217;m a card&#45;carrying NRA whipping boy, let me know and I&#8217;ll try to tone it down a bit&#46; Thanks for the mirror of truth, _why!</p> Sat, 21 Oct 2006 03:22:00 +0000 urn:uuid:84f01f0c-8eff-40bc-ae21-0a093365499f Nick Sieger http://blog.nicksieger.com/articles/2006/10/21/caught-redhanded ruby http://blog.nicksieger.com/articles/trackback/84 Customizing RSpec <p><em>Update/Disclaimer: I refer to parts of RSpec that are not blessed as an extension API&#46; Redefining <code>before_context_eval</code> and using the <code>@context_eval_module</code> variable directly may change in the future&#46; I&#8217;ll keep this article updated to coincide with the changes&#46; For now, these techniques should work fine with RSpec versions up to 0&#46;7&#46;5&#46;</em></p> <p>RSpec seems to be getting more attention lately as a viable, nay, <em>preferred</em>, alternative to Test::Unit&#46; It&#8217;s possible that it&#8217;s just my personal feed&#45;reader&#45;echo&#45;chamber, but consider this: <a href="http://blog.fallingsnow.net/rubinius/">Rubinius</a> has started using RSpec alongside Test::Unit as an another way to test the alternate Ruby implementation&#46; They&#8217;re even in the midst of building some snazzy extensions to allow the same specs to be run under a Ruby implementation of your choice&#46; (Perhaps this will point the way to a new round of executable specs to accompany the <a href="http://www.headius.com/rubyspec/index.php/Main_Page">fledgling community spec</a>? Let&#8217;s wait and see how they do and leave that topic for another day&#46;)</p> <p>But extending and customizing RSpec to add a DSL on top of RSpec&#8217;s <code>context/specify</code> framework doesn&#8217;t have to be the realm of experts&#46; Here are some templates for how you can DRY up your specs by adding your own helper methods in such a way that they will be available to all your specs&#46; But first, a little background&#46;</p> <h3>Spec Helper</h3> <p>Most usages of RSpec that I&#8217;ve seen in the wild use a &#8220;spec helper&#8221; (<code>spec_helper.rb</code>)&#46; This file, following the pattern of Rails&#8217; <code>test_helper.rb</code>, minimally contains require statements to pull in the RSpec code and any supporting code for running specs&#46; By requiring the spec helper via a path relative to your spec (usually with <code>require File.dirname(__FILE__) + '/spec_helper'</code> or similar), it also allows you the convenience of running your specs one at a time from anywhere (say, by launching from your editor) or with <code>rake</code> or <code>spec</code>&#46; This file is where your shared helper methods will go, and where they&#8217;ll get registered to be pulled into the contexts&#46;</p> <h3>What Context in <code>context</code>?</h3> <div class="typocode"><pre><code class="typocode_ruby "><span class="ident">context</span> <span class="punct">&quot;</span><span class="string">A new stack</span><span class="punct">&quot;</span> <span class="keyword">do</span> <span class="comment"># &lt;== What is the value of &quot;self&quot; here?</span> <span class="ident">specify</span> <span class="punct">&quot;</span><span class="string">should be empty</span><span class="punct">&quot;</span> <span class="keyword">do</span> <span class="keyword">end</span> <span class="keyword">end</span></code></pre></div> <p>How do those contexts work anyway? The <code>context</code> method that defines a context in which specs can be defined and run takes a block to define the individual specs, but what can really go in that block?</p> <p>It turns out that RSpec jumps through metaprogramming hoops (using <code>class_eval</code>) to make the block behave like a class definition&#46; This means you can do things like put method definitions inside your context:</p> <div class="typocode"><pre><code class="typocode_ruby "><span class="ident">context</span> <span class="punct">&quot;</span><span class="string">A new stack</span><span class="punct">&quot;</span> <span class="keyword">do</span> <span class="keyword">def </span><span class="method">a_new_stack</span> <span class="constant">Stack</span><span class="punct">.</span><span class="ident">new</span> <span class="keyword">end</span> <span class="ident">specify</span> <span class="punct">&quot;</span><span class="string">should be empty</span><span class="punct">&quot;</span> <span class="keyword">do</span> <span class="ident">a_new_stack</span><span class="punct">.</span><span class="ident">should_be_empty</span> <span class="keyword">end</span> <span class="keyword">end</span></code></pre></div> <p>Which is nice, but the reason we&#8217;re here is to hide that away in <code>spec_helper.rb</code>&#46; So, to get back to the point of the comment in the first example above, the <code>self</code> inside the context block is an anonymous <code>Module</code> object&#46; It&#8217;s constructed in the <code>initialize</code> method of a <code>Context</code> (condensed from <code>spec/runner/context.rb</code> in the RSpec codebase):</p> <div class="typocode"><pre><code class="typocode_ruby "><span class="keyword">class </span><span class="class">Spec::Runner::Context</span> <span class="keyword">def </span><span class="method">initialize</span><span class="punct">(</span><span class="ident">name</span><span class="punct">,</span> <span class="punct">&amp;</span><span class="ident">context_block</span><span class="punct">)</span> <span class="attribute">@name</span> <span class="punct">=</span> <span class="ident">name</span> <span class="attribute">@context_eval_module</span> <span class="punct">=</span> <span class="constant">Module</span><span class="punct">.</span><span class="ident">new</span> <span class="attribute">@context_eval_module</span><span class="punct">.</span><span class="ident">extend</span> <span class="constant">ContextEval</span><span class="punct">::</span><span class="constant">ModuleMethods</span> <span class="attribute">@context_eval_module</span><span class="punct">.</span><span class="ident">include</span> <span class="constant">ContextEval</span><span class="punct">::</span><span class="constant">InstanceMethods</span> <span class="ident">before_context_eval</span> <span class="attribute">@context_eval_module</span><span class="punct">.</span><span class="ident">class_eval</span><span class="punct">(&amp;</span><span class="ident">context_block</span><span class="punct">)</span> <span class="keyword">end</span> <span class="keyword">def </span><span class="method">before_context_eval</span> <span class="keyword">end</span> <span class="keyword">end</span></code></pre></div> <p>(Take note of that empty <code>before_context_eval</code> method and the fact that it&#8217;s invoked during context initialization; that&#8217;s where we can plug in our custom extensions&#46;)</p> <p>The object held by the <code>@context_eval_module</code> instance variable is being augmented in two ways: extension and inclusion&#46; The object is <em>extended</em> with the <code>ContextEval::ModuleMethods</code> module; these methods are being added to the object&#8217;s singleton class&#46; This has the effect of making these methods visible within the <code>context</code> block, functioning similar to &#8220;class&#8221; methods&#46;</p> <p>The object also has the <code>ContextEval::InstanceMethods</code> module <em>included</em>&#46; This has the effect of adding these as instance methods, making them visible from within <code>specify</code> blocks, which are made to behave like instance methods on the same object&#46;</p> <h3>Putting it together</h3> <table style="text-align: left;" summary=""> <thead> <th>Technique</th> <th>Visibility</th> <th>Use</th> </thead> <tbody> <tr><td>@context_eval_module&#46;extend</td><td>Context block</td><td>Custom setup, shared state declaration</td></tr> <tr><td>@context_eval_module&#46;include</td><td>Specify block</td><td>Shared actions/functions, stub/expectation modification, encapsulate instance variables</td></tr> </tbody> </table> <h4>Adding specialized setup methods</h4> <p><code>spec_helper.rb</code> snippet:</p> <div class="typocode"><pre><code class="typocode_ruby "><span class="keyword">module </span><span class="module">SharedSetupMethods</span> <span class="keyword">def </span><span class="method">setup_new_stack</span> <span class="ident">setup</span> <span class="keyword">do</span> <span class="attribute">@stack</span> <span class="punct">=</span> <span class="constant">Stack</span><span class="punct">.</span><span class="ident">new</span> <span class="keyword">end</span> <span class="keyword">end</span> <span class="keyword">end</span> <span class="keyword">class </span><span class="class">Spec::Runner::Context</span> <span class="keyword">def </span><span class="method">before_context_eval</span> <span class="attribute">@context_eval_module</span><span class="punct">.</span><span class="ident">extend</span> <span class="constant">SharedSetupMethods</span> <span class="keyword">end</span> <span class="keyword">end</span></code></pre></div> <p>Example spec:</p> <div class="typocode"><pre><code class="typocode_ruby "><span class="ident">context</span> <span class="punct">&quot;</span><span class="string">A new stack</span><span class="punct">&quot;</span> <span class="keyword">do</span> <span class="ident">setup_new_stack</span> <span class="ident">specify</span> <span class="punct">&quot;</span><span class="string">should be empty</span><span class="punct">&quot;</span> <span class="keyword">do</span> <span class="attribute">@stack</span><span class="punct">.</span><span class="ident">should_be_empty</span> <span class="keyword">end</span> <span class="keyword">end</span> </code></pre></div> <h4>Adding shared accessors</h4> <p><code>spec_helper.rb</code> snippet:</p> <div class="typocode"><pre><code class="typocode_ruby "><span class="keyword">module </span><span class="module">StackMethods</span> <span class="ident">attr_accessor</span> <span class="symbol">:stack</span> <span class="keyword">def </span><span class="method">push_an_object</span> <span class="ident">stack</span> <span class="punct">&lt;&lt;</span> <span class="ident">mock</span><span class="punct">(&quot;</span><span class="string">some object</span><span class="punct">&quot;)</span> <span class="keyword">end</span> <span class="keyword">end</span> <span class="keyword">class </span><span class="class">Spec::Runner::Context</span> <span class="keyword">def </span><span class="method">before_context_eval</span> <span class="attribute">@context_eval_module</span><span class="punct">.</span><span class="ident">include</span> <span class="constant">StackMethods</span> <span class="keyword">end</span> <span class="keyword">end</span></code></pre></div> <p>Example spec:</p> <div class="typocode"><pre><code class="typocode_ruby "><span class="ident">context</span> <span class="punct">&quot;</span><span class="string">A stack with an object pushed</span><span class="punct">&quot;</span> <span class="keyword">do</span> <span class="ident">setup</span> <span class="keyword">do</span> <span class="attribute">@stack</span> <span class="punct">=</span> <span class="constant">Stack</span><span class="punct">.</span><span class="ident">new</span> <span class="keyword">end</span> <span class="ident">specify</span> <span class="punct">&quot;</span><span class="string">should not be empty</span><span class="punct">&quot;</span> <span class="keyword">do</span> <span class="ident">stack</span><span class="punct">.</span><span class="ident">should_be_empty</span> <span class="ident">push_an_object</span> <span class="ident">stack</span><span class="punct">.</span><span class="ident">should_not_be_empty</span> <span class="keyword">end</span> <span class="keyword">end</span> </code></pre></div> <p>The examples are simple, but hopefully illustrate the techniques&#46; For an example of some code that&#8217;s actually useful, check out my sample <a href="http://svn.caldersphere.net/svn/main/rspec_selenium_rc/trunk/">RSpec Selenium RC integration project</a>, in particular the <a href="http://svn.caldersphere.net/svn/main/rspec_selenium_rc/trunk/lib/spec_helper.rb">spec helper</a> and the <a href="http://svn.caldersphere.net/svn/main/rspec_selenium_rc/trunk/spec/example_spec.rb">example spec</a>&#46; (More on this in the future if it proves useful, but for now if you check it out and run <code>rake</code> on it, it should launch <a href="http://www.openqa.org/selenium-rc/">Selenium RC</a> and run the example spec in a Firefox browser&#46;)</p> <p>By mixing and matching these techniques, you can layer a mini&#45;DSL on top of RSpec and achieve DRY&#45;er and even more readable and intention&#45;revealing specs&#46; Let me know if you&#8217;re able to find uses for these tips!</p> Tue, 02 Jan 2007 05:32:00 +0000 urn:uuid:21755a2d-0ffa-4f0c-be7d-a2e2e01d99b4 Nick Sieger http://blog.nicksieger.com/articles/2007/01/02/customizing-rspec rspec ruby metaprogramming http://blog.nicksieger.com/articles/trackback/178