Nick Sieger

Test Your Rake Tasks

Posted by Nick Sieger Mon, 11 Jun 2007 15:35:00 GMT

Many libraries and plugins ship custom Rake tasks. Of course, as slick as Rake is for a build and configuration language, it’s still just Ruby code right?

Case in point: I released a version of ci_reporter with a fairly careless bug in a rake task that attempted to << a string into an existing environment variable. It escaped me at the time that Ruby sets up the ENV hash with frozen strings, because my own usage of ci_reporter did not exercise the task in that way.

So shouldn’t that Ruby code be subjected to the rigor of automated testing just like the rest of your code? It became obvious to me that it must be so. It turns out it’s straightforward to use Rake in an embedded fashion, and invoke targeted tasks in your custom Rake recipes. The examples here use RSpec, since that’s what I use for testing ci_reporter, but you could apply this to Test::Unit as well.

The technique is to create a new instance of Rake::Application, make it the active application, and load your rake scripts into it:

describe "ci_reporter ci:setup:testunit task" do
  before(:each) do
    @rake = Rake::Application.new
    Rake.application = @rake
    load CI_REPORTER_LIB + '/ci/reporter/rake/test_unit.rb'
  end
  after(:each) do
    Rake.application = nil
  end
  # ...
end

Notice the use of #load rather than #require, as you want to execute your rake script each time you setup the Rake application object. When tearing down your test or example, you should cleanup Rake by setting the Rake.application back to nil (or save the previous application and restore it, if you prefer).

Now, in the body of your test or example, you invoke your rake task with @rake['target'].invoke. Here, I’m exercising the case of an existing, frozen ENV value. After the task is invoked, I check the value after the task to make sure the variable was modified as expected.

it "should append to ENV['TESTOPTS'] if it already contains a value" do
  ENV["TESTOPTS"] = "somevalue".freeze
  @rake["ci:setup:testunit"].invoke
  ENV["TESTOPTS"].should =~ /somevalue.*test_unit_loader/
end

I was fortunate here that the tasks for which I wrote tests after the fact were simple enough to be testable on their own, which may not always be the case, especially with organic, homegrown Rake tasks that interact with the world outside of Ruby. Still, if your Rake tasks are a critical part of your application, library or plugin, they should be tested. For example, it would be nice if tests could be written for the Rake scripts in Rails’ Railties module to increase coverage there.

Perhaps someone out there will run with this idea and take up the challenge and write a Rakefile completely in a test-driven or behaviour-driven style. It’s always been a sore point for me with Make, Ant, Maven, and virtually every other build tool in existence that you have no other way of automatically verifying your build script is doing what you intended without manually running it and inspecting its output -- it just feels so dirty! I’d expect that test-driven Rake scripts would likely have the level of granularity to match the tasks that need to be done, in a way that you can combine them in the right ways to make incremental and deconstructed builds simpler.