Monkey-patching is Part of the DIY Culture

Posted by Nick Sieger Fri, 14 Mar 2008 00:07:00 GMT

Recently there was a long (122 posts!) thread on Ruby-talk started by Avdi Grimm (based on a post he made on his blog). He took a risk by titling the post “Monkey-patching is Destroying Ruby,” which got a lot of attention but probably ruffled a few too many feathers before he had a chance to justify his argument.

I don’t disagree with Avdi’s main point, which is that monkey-patching isn’t always the best tool for the job. But I contend that it’s still a basic part of the Ruby programming culture, like it or not. I believe the reason for this is that monkey-patching is an extremely empowering technique. It’s part and parcel of the hacker/DIY culture.

Anyone who has had an experience in a less-flexible language or caught in a hard place trying to debug a closed-source library can appreciate how liberating it is to take matters into your own hands, fix your own problem, and to be able to do it without modifying the original library source. It’s a revelatory experience that makes you never want to go back to inflexible programming environments ever again. There’s also nothing wrong with monkey-patching a library you don’t control as long as you freeze all the code you’re using before making a working patch -- if your patch works and you don’t change or upgrade anything, you’re not likely to encounter any problems.

Of course all monkey-patches are not created equal. Some are certainly more onerous than others. Let me give you an example I ran into myself recently, which raises some questions for which I don’t yet have the answer myself.

The most prolific Rick Olson’s popular plugin attachment_fu uses Ruby’s tempfile library. It has a legitimate need to extend Tempfile to preserve the file extension on tempfiles, so that image conversion routines can use the file extension to help identify the format. How it accomplishes this, however, is not very pretty. Here’s the original Ruby code (as of Ruby 1.8.6 p111):

def make_tmpname(basename, n)
  sprintf('%s.%d.%d', basename, $$, n)
private :make_tmpname

Seems like about as reasonable a place as any to hook in, right? But the method is marked private -- I’m guessing the original implementor (according to svn blame, it appears to be Akira Tanaka) probably did not intend for the method to be replaced. But, hey, it’s Ruby, right? So away attachment_fu goes:

Tempfile.class_eval do
  # overwrite so tempfiles use the extension of the basename.  important for rmagick and image science
  def make_tmpname(basename, n)
    ext = nil
    sprintf("%s%d-%d%s", basename.to_s.gsub(/\.\w+$/) { |s| ext = s; '' }, $$, n, ext)

As far as monkey-patches go, this isn’t too bad. There is no code copied from the original implementation, and it’s a fairly focused and compact method. The fact that it’s private is a bit of a smell, though. But, it works, and we forget about it happily (if we even knew it existed in the first place), and hope that tempfile.rb never changes.

Well, in my case, it did. MenTaLguY has been working on more robust, thread-safe versions of the standard libraries. And he changed the arity of make_tmpname:

@@sequence_number = 0
@@sequence_mutex =
def make_tmpname(basename)
  begin"/dev/urandom", "rb") do |random|
    # insecure fallback
    sequence_number = @@sequence_mutex.synchronize { @@sequence_number += 1 }

Resulting in:

Read error: #<RuntimeError: cannot generate tempfile `': wrong number of arguments(1 for 2)>
/Users/nicksieger/Projects/jruby/trunk/jruby/lib/ruby/1.8/tempfile.rb:39:in `initialize'
/Users/nicksieger/Projects/jruby/trunk/jruby/lib/ruby/gems/1.8/gems/mongrel-1.1.3-java/lib/mongrel/http_request.rb:47:in `new'
/Users/nicksieger/Projects/jruby/trunk/jruby/lib/ruby/gems/1.8/gems/mongrel-1.1.3-java/lib/mongrel/http_request.rb:47:in `initialize'
/Users/nicksieger/Projects/jruby/trunk/jruby/lib/ruby/gems/1.8/gems/mongrel-1.1.3-java/lib/mongrel.rb:149:in `new'
/Users/nicksieger/Projects/jruby/trunk/jruby/lib/ruby/gems/1.8/gems/mongrel-1.1.3-java/lib/mongrel.rb:149:in `process_client'
/Users/nicksieger/Projects/jruby/trunk/jruby/lib/ruby/gems/1.8/gems/mongrel-1.1.3-java/lib/mongrel.rb:285:in `run'

So, it took me a while before it occurred to me that something in my project might be overriding make_tmpname. But even after I found it, notified MenTaLguY, and he fixed it, it still left me wondering: who’s in the wrong here? Akira-san, for not making a better way to hook into make_tmpname, Rick for monkey-patching it, or MenTaLguY for changing the method arity in his rewrite? I can’t really point the blame at any of them.

There are certainly more egregious and offensive monkey-patches than this example (and I include myself in that camp). In any case though, I could live with just about any monkey-patch if I had better debugging tools. For example, it would be great if you could ask Ruby to track and retain references to all methods, including those that get replaced, along with the source locations where each was defined. Another possibility might be a before_method_added hook that could let you track method replacements as they’re about to happen (and maybe even veto method redefinitions!).

These are the types of tools that an enterprising Ruby programmer could look at adding to any one of the existing Ruby implementations. Both JRuby and Rubinius are becoming test beds for bleeding edge features that could help advance the state of the art. So pitch in and help make monkey-patching less painful!

Postscript: This post coming to you from the Illinois interstate courtesy of Curt and his 3G EVDO wi-fi hub!

Tags  | 2 comments


  1. Avatar ste said about 9 hours later:

    IMHO the key concept is the “contract”. Every piece of code comes with a more or less explicit contract, which says what you can or cannot do with it. In Ruby’s case, the contract is often expressed by the access-level restriction keywords. Since there’s no way to enforce them (as this case shows), they’re essentially a gentlemen’s agreement. To me it’s clear that those to “blame” are Rick’s plugin in the first place, for breaking the contract (out of necessity, of course - and he might argue that he did the “less wrong” thing by applying the minimum amount of monkey patching needed to solve his problem); and to a very small degree Akira-san for not making his code flexible enough and “hiding” this inflexibility behind a contract (well, I guess we’re all more or less guilty of this “crime”, but in the case of a core library it’s a bit more serious). I don’t think MenTaLguY is to blame: by changing the implementation of a private method he rightly didn’t consider the possible side-effects on third-party code.

  2. Avatar Assaf said about 17 hours later:
    1. YAGNI. Don’t go littering the code with hooks for anything that anyone may possibly want to use some time. Wait for people to ask for it.

    2. Fork. Like the implementation but want to use it in a different context? If the license permits, create your own variety.

    3. Don’t break contracts. Anything marked private or protected is a sign you’re breaking and entering without permission.

    4. Hack. If all else fails, fix it, don’t wait for the next working group to ratify a spec to provide you with an API for the feature you needed yesterday.

    5. Accept the dependencies. I have some code that only works on Rails 1.1, which is ok, because I only use it on Rails 1.1. I don’t consider it broken, I just don’t at the moment have a similar solution available for 2.0.

    BTW These soul-sucking whack-a-bug are not limited to monkey patching. I had the same problem with two dependencies not mixing together because of method_missing, with global namespace pollution, and with something that just works ... except on my machine.

    We definitely need better tools, some of these problems are impenetrable to a lot of developers who don’t yet have the skills to hunt them down.