Test Your Rake Tasks

Posted by Nick Sieger Mon, 11 Jun 2007 15:35:00 GMT

Many libraries and plugins ship custom Rake tasks. Of course, as slick as Rake is for a build and configuration language, it’s still just Ruby code right?

Case in point: I released a version of ci_reporter with a fairly careless bug in a rake task that attempted to << a string into an existing environment variable. It escaped me at the time that Ruby sets up the ENV hash with frozen strings, because my own usage of ci_reporter did not exercise the task in that way.

So shouldn’t that Ruby code be subjected to the rigor of automated testing just like the rest of your code? It became obvious to me that it must be so. It turns out it’s straightforward to use Rake in an embedded fashion, and invoke targeted tasks in your custom Rake recipes. The examples here use RSpec, since that’s what I use for testing ci_reporter, but you could apply this to Test::Unit as well.

The technique is to create a new instance of Rake::Application, make it the active application, and load your rake scripts into it:

describe "ci_reporter ci:setup:testunit task" do
  before(:each) do
    @rake = Rake::Application.new
    Rake.application = @rake
    load CI_REPORTER_LIB + '/ci/reporter/rake/test_unit.rb'
  after(:each) do
    Rake.application = nil
  # ...

Notice the use of #load rather than #require, as you want to execute your rake script each time you setup the Rake application object. When tearing down your test or example, you should cleanup Rake by setting the Rake.application back to nil (or save the previous application and restore it, if you prefer).

Now, in the body of your test or example, you invoke your rake task with @rake['target'].invoke. Here, I’m exercising the case of an existing, frozen ENV value. After the task is invoked, I check the value after the task to make sure the variable was modified as expected.

it "should append to ENV['TESTOPTS'] if it already contains a value" do
  ENV["TESTOPTS"] = "somevalue".freeze
  ENV["TESTOPTS"].should =~ /somevalue.*test_unit_loader/

I was fortunate here that the tasks for which I wrote tests after the fact were simple enough to be testable on their own, which may not always be the case, especially with organic, homegrown Rake tasks that interact with the world outside of Ruby. Still, if your Rake tasks are a critical part of your application, library or plugin, they should be tested. For example, it would be nice if tests could be written for the Rake scripts in Rails’ Railties module to increase coverage there.

Perhaps someone out there will run with this idea and take up the challenge and write a Rakefile completely in a test-driven or behaviour-driven style. It’s always been a sore point for me with Make, Ant, Maven, and virtually every other build tool in existence that you have no other way of automatically verifying your build script is doing what you intended without manually running it and inspecting its output -- it just feels so dirty! I’d expect that test-driven Rake scripts would likely have the level of granularity to match the tasks that need to be done, in a way that you can combine them in the right ways to make incremental and deconstructed builds simpler.

Tags ,  | 5 comments | no trackbacks


  1. Avatar Ryan Sonnek said about 1 hour later:

    I’m not sure what you’re talking about. Building maven plugins always leaves me with a warm fuzzy feeling inside.

    All that lovely “mojo”... =)

    This is really impressive and I completely agree that build scripts are code and should be tested just like anything else!

  2. Avatar Nick said about 11 hours later:

    You know, there is buildr that uses Rake and is drop-in compatible with Maven...

  3. Avatar Brittany Sieger said 5 days later:

    Actually I was just wondering how ruby is able to get away with what seems like so much less data yet still be able to handel at close to C++ capabilities. I would be interested to hear exactly how that is possible if someone could explain that in simple terms.

  4. Avatar Nick said 5 days later:

    Wow, another Sieger!

    Brittany, what do you mean by “so much less data”...there are a lot of introductory explanations of Ruby, perhaps you’d be interested in checking out the Poignant Guide.

  5. Avatar Assaf said 7 days later:

    That gives me an idea.

    I ran into problems before where the build would complete, but some files will be missing, or not in the right path, or you get the directory you expect, sans any contents.

    There really needs to be a way to test the build. I’m going to run with this and try something out in Buildr.

    Right now what I’m thinking of is some way to write expectations about things that should exist, or their contents.

    I’m wondering if this should be part of the Rakefile itself (build => test => deploy), or a separate set of tests, most likely RSpec.


Use the following link to trackback from your own site: