Posted by Nick Sieger
Thu, 20 Jan 2011 17:47:21 GMT
I was troubleshooting some JRuby code that transforms Java camelCase method names into Ruby snake_case form. We had a bunch of specs that did this, for example:
describe "Java instance method names" do
it "should present javabean properties as attribute readers and writers" do
methods = MethodNames.instance_methods
methods.should include("getValue2")
methods.should include("get_value2")
methods.should include("value2")
methods.should include("setValue2")
methods.should include("set_value2")
methods.should include("value2=")
end
end
The problem comes when these specs fail. The default error message made by the #include matcher looks like:
Failures:
1) Java instance method names should present javabean properties as attribute readers and writers
Failure/Error: methods.should include("get_value2")
expected [...full contents of array here...] to include "get_value2"
Diff:
@@ -1,2 +1,186 @@
-get_value2
+[...all entries, one per line here...]
That’s not a terrible message, but when your array contains over 100 entries (like an array of method names), it could be a lot better. In particular, I kept scanning the failure message’s big list, unable to clearly see why the methods I was expecting weren’t there.
What I wanted to see was how my changes to the regex which splits a Java camelCase name affected the conversion. So, what I needed was a report of which method names were the closest to the ones that were not in the list. Hey, sounds like a good reason to implement a custom matcher, and take a diversion into fuzzy string matching algorithms!
I settled on porting the pseudocode in Wikipedia for the Levenshtein distance, which calculates how close in content two strings are to each other. I looked around and there are existing Levenshtein ports for Ruby, but they use native code for performance. I don’t need performance because I’m only using the Levenshtein function when there is a failure. Of course, pure Ruby code is more portable too!.
The other change I made in the specs was to pass all strings in a single matcher rather than one name per expectation, so we can see all names that fail, not just the first.
So now, the new spec looks more like this:
describe "Java instance method names" do
let(:members) { MethodNames.instance_methods }
it "should present javabean properties as attribute readers and writers" do
members.should have_strings("getValue2",
"get_value2",
"value2",
"setValue2",
"set_value2",
"value2=")
end
end
The custom RSpec matcher #have_strings is declared like so:
RSpec::Matchers.define :have_strings do |*strings|
match do |container|
@included, @missing = [], []
strings.flatten.each do |s|
if container.include?(s)
@included << s
else
@missing << s
end
end
@missing.empty?
end
failure_message_for_should do |container|
"expected array of #{container.length} elements to include #{@missing.inspect}.\n" +
"#{closest_match_message(@missing, container)}"
end
failure_message_for_should_not do |container|
"expected array of #{container.length} elements to not include #{@included.inspect}."
end
def closest_match_message(missing, container)
missing.map do |m|
groups = container.group_by {|x| levenshtein(m, x) }
" closest match for #{m.inspect}: #{groups[groups.keys.min].inspect}"
end.join("\n")
end
end
I omitted the #levenshtein function here for brevity. (You can view the full source for details.) Now our failing spec output looks like:
Failures:
1) Java instance method names should present javabean properties as attribute readers and writers
Failure/Error: members.should have_strings("getValue2",
expected array of 185 elements to include ["get_my_value", "my_value", "set_my_value", "my_value="].
closest match for "get_my_value": ["get_myvalue", "set_myvalue"]
closest match for "my_value": ["myvalue"]
closest match for "set_my_value": ["get_myvalue", "set_myvalue"]
closest match for "my_value=": ["myvalue="]
Now the failure message is giving me exactly the information I need. Much better, don’t you think?
Tags rspec, ruby
Posted by Nick Sieger
Fri, 08 Feb 2008 15:13:16 GMT
A new screen cast is up with yours truly showing off NetBeans’ RSpec support. Additionally, I tried to make it interesting to a wider audience by really showcasing RSpec’s strengths, and trying to capture some of the red-green-refactor rhythm. NetBeans does work really well for this, but in my mind, the star of the show is RSpec.
I’m pleased with how it turned out considering I hadn’t done this sort of thing before. Special thanks to Cindy Church for putting it all together, including all the production: setup, recording, editing, even the music!
A QuickTime movie version is available as well. Check it out and let me know what you think.
Tags jruby, netbeans, rspec, ruby | 1 comment
Posted by Nick Sieger
Sun, 04 Nov 2007 16:26:00 GMT
David Chelimsky and Dave Astels: RSpec
describe TestDriverDevelopment do
it "is an incremental process"
it "drives the implementation"
it "results in an exhaustive test suite"
it "should focus on design"
it "should focus on documentation"
it "should focus on behaviour"
end
class BehaviourDrivenDevelopment < TestDrivenDevelopment
include FocusOnDesign
include FocusOnDocumentation
include FocusOnBehavior
end
When doing test-driven development:
- Write your intent first. The smallest test you can that fails.
- Next, write the implementation. The simplest thing that could possibly work.
- Even though you may be tempted to think about additional edge cases, multiple requirements, etc., you should try to be disciplined and focus only on the immediate tests. Only after you’ve made one test fail, then pass, can you continue on to other tests.
RSpec history
Initially BDD was just a discussion among Aslak Hellesoy and Dan North in the ThoughtWorks London office. Dave Astels joined the conversation with a blog post stating that he thought these ideas could be easily implemented in Smalltalk or Ruby. Steven Baker jumped in with an initial implementation, and released RSpec 0.1. Later in 2006, maintenance was handed over to David Chelimsky. RSpec has evolved through a dog-fooding phase up to the present 1.0 product.
BDD is no longer just about “should instead of assert”, it’s evolving into a process. Emphasizing central concepts from extreme programming and domain-driven design, it’s moving toward focusing on customer stories and acceptance testing. It’s outside-in, starting at high levels of detail, rather than low-level like RSpec or Test::Unit.
Story Runner
Story Runner is a new feature intended for RSpec 1.1. Each story is supposed to capture a customer requirement in the following general template:
As a (role) ... I want to (some function) ... so that (some business value).
It uses a “Scenario ... Given ... When ... Then ...” format to express the high level stories. Scenarios are a series of given items, steps, and behaviour validations. Once the basic steps are established, they can be re-used. David even demonstrated a preview of an in-browser story runner that would allow the customer to play with the implementation and create new scenarios.
Pending
Pending is a nice way to mark specs as “in-progress”. You can either omit a block for your spec, or use pending inside the block to leave a placeholder to come back to.
describe Pending do
it "doesn't need a block to be pending"
it "could also be specified inside the block" do
pending("TODO")
this.should_not be_a_failure
end
it "could also use a block with pending, and you will be notified when it starts to succeed" do
pending("TODO") do
this.should_not be_a_failure
end
end
end
Behaviour-Driven Development in Ruby with RSpec is a new book David and Aslak are working on, due out early next year.
Update: David has posted his slides.
Tags rspec, rubyconf, rubyconf2007 | no comments
Posted by Nick Sieger
Tue, 30 Jan 2007 02:16:00 GMT
I know there has been some demand out there for a version of my RSpec Autotest Rails plugin that works with standalone (non-Rails) projects, so I finally caved and coded it up. I really hope this does finally get accepted as a patch to ZenTest (nudge, nudge), but until it gets a more standard release, do try it out:
- Ensure you have ZenTest installed (
gem install ZenTest).
- Download the rspec_autotest.rb file here and put it in the
spec directory of your project.
- Add this snippet of code into your
Rakefile, and rake spec:autotest as usual.
namespace :spec do
task :autotest do
require './spec/rspec_autotest'
RspecAutotest.run
end
end
Once again, happy spec’ing, and let me know of any issues you have with it.
Tags autotest, rspec, ruby | 1 comment | no trackbacks
Posted by Nick Sieger
Tue, 02 Jan 2007 05:32:00 GMT
Update/Disclaimer: I refer to parts of RSpec that are not blessed as an extension API. Redefining before_context_eval and using the @context_eval_module variable directly may change in the future. I’ll keep this article updated to coincide with the changes. For now, these techniques should work fine with RSpec versions up to 0.7.5.
RSpec seems to be getting more attention lately as a viable, nay, preferred, alternative to Test::Unit. It’s possible that it’s just my personal feed-reader-echo-chamber, but consider this: Rubinius has started using RSpec alongside Test::Unit as an another way to test the alternate Ruby implementation. They’re even in the midst of building some snazzy extensions to allow the same specs to be run under a Ruby implementation of your choice. (Perhaps this will point the way to a new round of executable specs to accompany the fledgling community spec? Let’s wait and see how they do and leave that topic for another day.)
But extending and customizing RSpec to add a DSL on top of RSpec’s context/specify framework doesn’t have to be the realm of experts. Here are some templates for how you can DRY up your specs by adding your own helper methods in such a way that they will be available to all your specs. But first, a little background.
Spec Helper
Most usages of RSpec that I’ve seen in the wild use a “spec helper” (spec_helper.rb). This file, following the pattern of Rails’ test_helper.rb, minimally contains require statements to pull in the RSpec code and any supporting code for running specs. By requiring the spec helper via a path relative to your spec (usually with require File.dirname(__FILE__) + '/spec_helper' or similar), it also allows you the convenience of running your specs one at a time from anywhere (say, by launching from your editor) or with rake or spec. This file is where your shared helper methods will go, and where they’ll get registered to be pulled into the contexts.
What Context in context?
context "A new stack" do
specify "should be empty" do
end
end
How do those contexts work anyway? The context method that defines a context in which specs can be defined and run takes a block to define the individual specs, but what can really go in that block?
It turns out that RSpec jumps through metaprogramming hoops (using class_eval) to make the block behave like a class definition. This means you can do things like put method definitions inside your context:
context "A new stack" do
def a_new_stack
Stack.new
end
specify "should be empty" do
a_new_stack.should_be_empty
end
end
Which is nice, but the reason we’re here is to hide that away in spec_helper.rb. So, to get back to the point of the comment in the first example above, the self inside the context block is an anonymous Module object. It’s constructed in the initialize method of a Context (condensed from spec/runner/context.rb in the RSpec codebase):
class Spec::Runner::Context
def initialize(name, &context_block)
@name = name
@context_eval_module = Module.new
@context_eval_module.extend ContextEval::ModuleMethods
@context_eval_module.include ContextEval::InstanceMethods
before_context_eval
@context_eval_module.class_eval(&context_block)
end
def before_context_eval
end
end
(Take note of that empty before_context_eval method and the fact that it’s invoked during context initialization; that’s where we can plug in our custom extensions.)
The object held by the @context_eval_module instance variable is being augmented in two ways: extension and inclusion. The object is extended with the ContextEval::ModuleMethods module; these methods are being added to the object’s singleton class. This has the effect of making these methods visible within the context block, functioning similar to “class” methods.
The object also has the ContextEval::InstanceMethods module included. This has the effect of adding these as instance methods, making them visible from within specify blocks, which are made to behave like instance methods on the same object.
Putting it together
| Technique |
Visibility |
Use |
| @context_eval_module.extend | Context block | Custom setup, shared state declaration |
| @context_eval_module.include | Specify block | Shared actions/functions, stub/expectation modification, encapsulate instance variables |
Adding specialized setup methods
spec_helper.rb snippet:
module SharedSetupMethods
def setup_new_stack
setup do
@stack = Stack.new
end
end
end
class Spec::Runner::Context
def before_context_eval
@context_eval_module.extend SharedSetupMethods
end
end
Example spec:
context "A new stack" do
setup_new_stack
specify "should be empty" do
@stack.should_be_empty
end
end
Adding shared accessors
spec_helper.rb snippet:
module StackMethods
attr_accessor :stack
def push_an_object
stack << mock("some object")
end
end
class Spec::Runner::Context
def before_context_eval
@context_eval_module.include StackMethods
end
end
Example spec:
context "A stack with an object pushed" do
setup do
@stack = Stack.new
end
specify "should not be empty" do
stack.should_be_empty
push_an_object
stack.should_not_be_empty
end
end
The examples are simple, but hopefully illustrate the techniques. For an example of some code that’s actually useful, check out my sample RSpec Selenium RC integration project, in particular the spec helper and the example spec. (More on this in the future if it proves useful, but for now if you check it out and run rake on it, it should launch Selenium RC and run the example spec in a Firefox browser.)
By mixing and matching these techniques, you can layer a mini-DSL on top of RSpec and achieve DRY-er and even more readable and intention-revealing specs. Let me know if you’re able to find uses for these tips!
Tags metaprogramming, rspec, ruby | 2 comments | no trackbacks
Posted by Nick Sieger
Fri, 01 Dec 2006 18:56:00 GMT
The prospect of doing behavior-driven development in Java has just taken a step closer with the news of RSpec running on JRuby. This is already a big step that will have an impact on Ruby and Java programmers alike in a number of ways.
However, it could be even better. RSpec has a nice, intuitive mocking API, which will unfortunately, at the present time, be useless when working with java objects. It would be awesome to try to get it to work, though. Some possibilities:
- Map to JMock and use JMock under the hood. Not a very attractive option for a number of reasons, but mainly because add-on bridging layers are complex and should be avoided.
- Improve ability for JRuby to implement any number of Java interfaces dynamically.
This second option is something Charlie, Tom and I talked about on Tuesday night, that could have a much broader impact on Java integration in JRuby.
Consider this spec. It’s trivial, but bear with me.
context "A TaskRunner" do
setup do
@task = mock("Runnable")
@task_runner = TaskRunner.new(@task)
end
specify "runs a task when executed" do
@task.should_receive(:run)
@task_runner.execute
end
end
This spec might be satisfied by the following Java code:
public class TaskRunner {
private Runnable task;
public TaskRunner(Runnable r) {
this.task = r;
}
public void execute() {
task.run();
}
}
Notice how I defined the @task in the spec above. This is the normal way of mocking in RSpec, and the example illustrates how I think JRuby should handle interfaces in Java: by duck-typing them.
Basically, the RSpec mock should act like a Java Runnable because I’ve defined a run method on it (in this case implicitly with @task.should_receive(:run)). JRuby could wrap a dynamic invocation proxy around any Ruby object just before passing it into a Java method invocation. Without doing any type- or method-checking up front. Just define the proxy as implementing the interface required by the Java method signature, and let the JRuby runtime do its thing, and attempt to resolve methods as they’re invoked. Possibly falling back to method_missing, even!
Note that this would also make moot the multiple interface syntax discussion, because you’d never have to declare an object in JRuby as implementing any particular interface. Just define the appropriately named methods with the proper arity, and you’re done. Maybe you don’t even need to declare all of them, if they never get called for your usage! This is the Ruby Way, and would be a completely natural extension to the way Java objects are manipulated in JRuby today, not to mention extremely concise and powerful.
This would also allow RSpec mocking to just work, at least for Java interface types, which would be way cool.
Charlie has a Swing demo that he frequently gives when talking about JRuby. Under the new proposal, it would look more like this:
require 'java'
frame = javax.swing.JFrame.new("Hello")
frame.setSize(200,200)
frame.show
button = javax.swing.JButton.new("OK!")
frame.add(button)
frame.show
def actionPerformed(event)
event.source.text = "Pressed!"
end
button.addActionListener self
With luck, this approach will be coming to JRuby very soon.
Tags jruby, mockobjects, rspec | no comments | no trackbacks
Posted by Nick Sieger
Wed, 15 Nov 2006 15:46:00 GMT
Inspired by a posting on the RSpec list and recent comments stating that my Auto RSpec hack wasn’t working, I’ve bitten the bullet and upgraded to RSpec 0.7.2, and made rspec_autotest a plugin in the process. So, herewith are the necessary incantations to auto-rspec your project. If you’ve tried my hack already, please remove any bits you previously had installed.
- Install RSpec on Rails, following the original instructions. As of RSpec 0.7.3, the specific version of ZenTest is no longer required. Also, diff-lcs is required to show unified diff output on
should == failures.
gem install zentest -v 3.4.1
gem install diff-lcs
gem install rspec
script/plugin install svn://rubyforge.org/var/svn/rspec/tags/REL_0_7_2/vendor/rspec_on_rails/vendor/plugins/rspec
script/plugin install http://svn.caldersphere.net/svn/main/plugins/rspec_autotest
Please let me know if you experience any problems!
Tags plugin, rails, rspec | 15 comments | no trackbacks
Posted by Nick Sieger
Wed, 13 Sep 2006 20:35:00 GMT
Update: (2 months later) If you’re reading this, you’re probably interested in my Rails plugin for this instead.
Hot off the presses, after a few hours of hacking and tweaking, may I present Auto+RSpec, otherwise known as The Mashup of RSpec on Rails and autotest. This is not an official release of any sort, but “may work for you.” It’s not a clean hack, as it exposes some areas for autotest to grow if the maintainers decide to open it up to alternatives to Test::Unit. After spending a little time looking at the autotest code, I think it would be nice to allow hooks for autotest plugins to define project conventions (i.e., @exceptions and the #tests_for_file method) as well as a result parsing API.
For now, if you’re an RSpec on Rails user, you can try this out as follows:
- Install ZenTest if you haven’t already:
sudo gem install ZenTest.
- Download rspec_autotest.rb and put in your
vendor/plugins/rspec/lib directory (you did say you’re using RSpec on Rails didn’t you?)
- Download rspec_autotest.rake and put in your
lib/tasks directory
- Start
autotest with rake by typing rake spec:autotest
- Note: if you’re using RSpec 0.6, you might have better success with the files located here.
Next steps for this will be to work out whether this code should live in RSpec on Rails or autotest, or some combination of those.
Now, spec’ers, be off in search of that Red/Green/Refactor rhythm of which sage agilists speak!
Bonus tip: add the following code to your .autotest file to run spec with rcov:
Autotest.add_hook :initialize do |at|
if at.respond_to? :spec_command
at.spec_command = %{rcov --exclude "lib/spec/.*" -Ilib --rails "/usr/lib/ruby/gems/1.8/gems/rspec-0.6.0/bin/spec" -- --diff}
end
end
Posted in testing, ruby, rails | Tags autotest, rspec, ruby, testing | 8 comments | no trackbacks
