Nick Sieger

Monkey-patching is Part of the DIY Culture

Posted by Nick Sieger Fri, 14 Mar 2008 00:07:00 GMT

Recently there was a long (122 posts!) thread on Ruby-talk started by Avdi Grimm (based on a post he made on his blog). He took a risk by titling the post “Monkey-patching is Destroying Ruby,” which got a lot of attention but probably ruffled a few too many feathers before he had a chance to justify his argument.

I don’t disagree with Avdi’s main point, which is that monkey-patching isn’t always the best tool for the job. But I contend that it’s still a basic part of the Ruby programming culture, like it or not. I believe the reason for this is that monkey-patching is an extremely empowering technique. It’s part and parcel of the hacker/DIY culture.

Anyone who has had an experience in a less-flexible language or caught in a hard place trying to debug a closed-source library can appreciate how liberating it is to take matters into your own hands, fix your own problem, and to be able to do it without modifying the original library source. It’s a revelatory experience that makes you never want to go back to inflexible programming environments ever again. There’s also nothing wrong with monkey-patching a library you don’t control as long as you freeze all the code you’re using before making a working patch -- if your patch works and you don’t change or upgrade anything, you’re not likely to encounter any problems.

Of course all monkey-patches are not created equal. Some are certainly more onerous than others. Let me give you an example I ran into myself recently, which raises some questions for which I don’t yet have the answer myself.

The most prolific Rick Olson’s popular plugin attachment_fu uses Ruby’s tempfile library. It has a legitimate need to extend Tempfile to preserve the file extension on tempfiles, so that image conversion routines can use the file extension to help identify the format. How it accomplishes this, however, is not very pretty. Here’s the original Ruby code (as of Ruby 1.8.6 p111):

def make_tmpname(basename, n)
  sprintf('%s.%d.%d', basename, $$, n)
end
private :make_tmpname

Seems like about as reasonable a place as any to hook in, right? But the method is marked private -- I’m guessing the original implementor (according to svn blame, it appears to be Akira Tanaka) probably did not intend for the method to be replaced. But, hey, it’s Ruby, right? So away attachment_fu goes:

Tempfile.class_eval do
  # overwrite so tempfiles use the extension of the basename.  important for rmagick and image science
  def make_tmpname(basename, n)
    ext = nil
    sprintf("%s%d-%d%s", basename.to_s.gsub(/\.\w+$/) { |s| ext = s; '' }, $$, n, ext)
  end
end

As far as monkey-patches go, this isn’t too bad. There is no code copied from the original implementation, and it’s a fairly focused and compact method. The fact that it’s private is a bit of a smell, though. But, it works, and we forget about it happily (if we even knew it existed in the first place), and hope that tempfile.rb never changes.

Well, in my case, it did. MenTaLguY has been working on more robust, thread-safe versions of the standard libraries. And he changed the arity of make_tmpname:

@@sequence_number = 0
@@sequence_mutex = Mutex.new
def make_tmpname(basename)
  begin
    File.open("/dev/urandom", "rb") do |random|
      "#{basename}.#{random.read(16).unpack('H*')}"
    end
  rescue
    # insecure fallback
    sequence_number = @@sequence_mutex.synchronize { @@sequence_number += 1 }
    "#{basename}.#{$$}.#{sequence_number}"
  end
end

Resulting in:

Read error: #<RuntimeError: cannot generate tempfile `': wrong number of arguments(1 for 2)>
/Users/nicksieger/Projects/jruby/trunk/jruby/lib/ruby/1.8/tempfile.rb:39:in `initialize'
/Users/nicksieger/Projects/jruby/trunk/jruby/lib/ruby/gems/1.8/gems/mongrel-1.1.3-java/lib/mongrel/http_request.rb:47:in `new'
/Users/nicksieger/Projects/jruby/trunk/jruby/lib/ruby/gems/1.8/gems/mongrel-1.1.3-java/lib/mongrel/http_request.rb:47:in `initialize'
/Users/nicksieger/Projects/jruby/trunk/jruby/lib/ruby/gems/1.8/gems/mongrel-1.1.3-java/lib/mongrel.rb:149:in `new'
/Users/nicksieger/Projects/jruby/trunk/jruby/lib/ruby/gems/1.8/gems/mongrel-1.1.3-java/lib/mongrel.rb:149:in `process_client'
/Users/nicksieger/Projects/jruby/trunk/jruby/lib/ruby/gems/1.8/gems/mongrel-1.1.3-java/lib/mongrel.rb:285:in `run'

So, it took me a while before it occurred to me that something in my project might be overriding make_tmpname. But even after I found it, notified MenTaLguY, and he fixed it, it still left me wondering: who’s in the wrong here? Akira-san, for not making a better way to hook into make_tmpname, Rick for monkey-patching it, or MenTaLguY for changing the method arity in his rewrite? I can’t really point the blame at any of them.

There are certainly more egregious and offensive monkey-patches than this example (and I include myself in that camp). In any case though, I could live with just about any monkey-patch if I had better debugging tools. For example, it would be great if you could ask Ruby to track and retain references to all methods, including those that get replaced, along with the source locations where each was defined. Another possibility might be a before_method_added hook that could let you track method replacements as they’re about to happen (and maybe even veto method redefinitions!).

These are the types of tools that an enterprising Ruby programmer could look at adding to any one of the existing Ruby implementations. Both JRuby and Rubinius are becoming test beds for bleeding edge features that could help advance the state of the art. So pitch in and help make monkey-patching less painful!

Postscript: This post coming to you from the Illinois interstate courtesy of Curt and his 3G EVDO wi-fi hub!