Test Your Rake Tasks

2007-06-11T15:35:00+00:00

Many libraries and plugins ship custom Rake tasks. Of course, as slick as Rake is for a build and configuration language, it’s still just Ruby code right?

Case in point: I released a version of ci_reporter with a fairly careless bug in a rake task that attempted to << a string into an existing environment variable. It escaped me at the time that Ruby sets up the ENV hash with frozen strings, because my own usage of ci_reporter did not exercise the task in that way.

So shouldn’t that Ruby code be subjected to the rigor of automated testing just like the rest of your code? It became obvious to me that it must be so. It turns out it’s straightforward to use Rake in an embedded fashion, and invoke targeted tasks in your custom Rake recipes. The examples here use RSpec, since that’s what I use for testing ci_reporter, but you could apply this to Test::Unit as well.

The technique is to create a new instance of Rake::Application, make it the active application, and load your rake scripts into it:

describe "ci_reporter ci:setup:testunit task" do
  before(:each) do
    @rake = Rake::Application.new
    Rake.application = @rake
    load CI_REPORTER_LIB + '/ci/reporter/rake/test_unit.rb'
  end
  after(:each) do
    Rake.application = nil
  end
  # ...
end

Notice the use of #load rather than #require, as you want to execute your rake script each time you setup the Rake application object. When tearing down your test or example, you should cleanup Rake by setting the Rake.application back to nil (or save the previous application and restore it, if you prefer).

Now, in the body of your test or example, you invoke your rake task with @rake['target'].invoke. Here, I’m exercising the case of an existing, frozen ENV value. After the task is invoked, I check the value after the task to make sure the variable was modified as expected.

it "should append to ENV['TESTOPTS'] if it already contains a value" do
  ENV["TESTOPTS"] = "somevalue".freeze
  @rake["ci:setup:testunit"].invoke
  ENV["TESTOPTS"].should =~ /somevalue.*test_unit_loader/
end

I was fortunate here that the tasks for which I wrote tests after the fact were simple enough to be testable on their own, which may not always be the case, especially with organic, homegrown Rake tasks that interact with the world outside of Ruby. Still, if your Rake tasks are a critical part of your application, library or plugin, they should be tested. For example, it would be nice if tests could be written for the Rake scripts in Rails’ Railties module to increase coverage there.

Perhaps someone out there will run with this idea and take up the challenge and write a Rakefile completely in a test-driven or behaviour-driven style. It’s always been a sore point for me with Make, Ant, Maven, and virtually every other build tool in existence that you have no other way of automatically verifying your build script is doing what you intended without manually running it and inspecting its output – it just feels so dirty! I’d expect that test-driven Rake scripts would likely have the level of granularity to match the tasks that need to be done, in a way that you can combine them in the right ways to make incremental and deconstructed builds simpler.

Auto RSpec

2006-09-13T20:35:00+00:00

Update: (2 months later) If you’re reading this, you’re probably interested in my Rails plugin for this instead.

Hot off the presses, after a few hours of hacking and tweaking, may I present Auto+RSpec, otherwise known as The Mashup of RSpec on Rails and autotest. This is not an official release of any sort, but “may work for you.” It’s not a clean hack, as it exposes some areas for autotest to grow if the maintainers decide to open it up to alternatives to Test::Unit. After spending a little time looking at the autotest code, I think it would be nice to allow hooks for autotest plugins to define project conventions (i.e., @exceptions and the #tests_for_file method) as well as a result parsing API.

For now, if you’re an RSpec on Rails user, you can try this out as follows:

Install ZenTest if you haven’t already: sudo gem install ZenTest.
Download rspec_autotest.rb and put in your vendor/plugins/rspec/lib directory (you did say you’re using RSpec on Rails didn’t you?)
Download rspec_autotest.rake and put in your lib/tasks directory
Start autotest with rake by typing rake spec:autotest
Note: if you’re using RSpec 0.6, you might have better success with the files located here.

Next steps for this will be to work out whether this code should live in RSpec on Rails or autotest, or some combination of those.

Now, spec’ers, be off in search of that Red/Green/Refactor rhythm of which sage agilists speak!

Bonus tip: add the following code to your .autotest file to run spec with rcov:

  Autotest.add_hook :initialize do |at|
    # run spec with rcov
    if at.respond_to? :spec_command
      at.spec_command = %{rcov --exclude "lib/spec/.*" -Ilib --rails "/usr/lib/ruby/gems/1.8/gems/rspec-0.6.0/bin/spec" -- --diff}
    end
  end

Nick Sieger: Tag testing

Test Your Rake Tasks

Auto RSpec