Using synthesis with testunit and mocha

Jan 13 2008

Using Synthesis with Test::Unit and Mocha

Synthesis is a Ruby library that applies a Synthesized Testing strategy, aiming to reduce the number of large, slow and brittle functional tests.

Imagine an example project with the following contents:

Macintosh-4:synthesis_example gm$ ls -R
Rakefile	lib		test

./lib:
synthesis_example	synthesis_example.rb

./lib/synthesis_example:
data_brander.rb	storage.rb

./test:
data_brander_test.rb	storage_test.rb

synthesis_example.rb:

$: << File.dirname(__FILE__) + '/'
require "synthesis_example/data_brander"
require "synthesis_example/storage"

data_brander.rb:

class DataBrander
  BRAND = "METAL"

  def initialize(storage)
    @storage = storage
  end

  def save_branded(data)
    @storage.save "#{BRAND} - #{data}"
  end
end

storage.rb:

class Storage
  def initialize(filename)
    @filename = filename
  end

  def save(val)
    File.open(@filename, 'w') {|f| f << val}
  end
end

Rakefile:

require "rubygems"
require "synthesis/task"

task :default => 'synthesis:test'
Synthesis::Task.new

data_brander_test.rb:

%w(test/unit rubygems mocha).each { |l| require l }
require File.dirname(__FILE__) + "/../lib/synthesis_example"

class DataBranderTest < Test::Unit::TestCase
  def test_saves_branded_to_storage
    storage = Storage.new 'whatever'
    storage.expects(:save).with('METAL - rock')
    DataBrander.new(storage).save_branded 'rock'
  end
end

Running data_brander_test.rb produces:

Macintosh-4:synthesis_example gm$ ruby test/data_brander_test.rb
Loaded suite test/data_brander_test
Started
.
Finished in 0.000487 seconds.

1 tests, 1 assertions, 0 failures, 0 errors

Supposing we haven't written any tests for Storage yet, the outcome of the synthesis:test task is:

Macintosh-4:synthesis_example gm$ rake synthesis:test
(in /Users/gm/devel/ruby/whatever_code/synthesis_example)
[Synthesis] Collecting expectations...
Loaded suite /usr/bin/rake
Started
.
Finished in 0.00063 seconds.

1 tests, 1 assertions, 0 failures, 0 errors
[Synthesis] Verifying expectation invocations...
Loaded suite /usr/bin/rake
Started
.
Finished in 0.000575 seconds.

1 tests, 1 assertions, 0 failures, 0 errors
[Synthesis]
[Synthesis] Tested Expectations:
[Synthesis]
[Synthesis] Untested Expectations:
[Synthesis] Storage.new.save
[Synthesis]
[Synthesis] Ignoring:
[Synthesis]
[Synthesis] FAILED.

Synthesis will make a first pass at running the project's tests collecting all simulated object interaction expectations. Then, it will run the tests again, verifying that the concrete implementations of the simulated expectation members have been covered in the tests.

In this example, the Synthesis task fails, reporting that the concrete implementation of the save instance method of Storage has not been tested. Let's fix that.

storage_test.rb:

require "test/unit"
require "fileutils"
require File.dirname(__FILE__) + "/../lib/synthesis_example"

class StorageTest < Test::Unit::TestCase
  def test_saves_to_file
    Storage.new('test.txt').save('rock')
    assert_equal 'rock', File.read('test.txt')
  ensure
    FileUtils.rm_f('test.txt')
  end
end

Running rake again:

Macintosh-4:synthesis_example gm$ rake
(in /Users/gm/devel/ruby/whatever_code/synthesis_example)
[Synthesis] Collecting expectations...
Loaded suite /usr/bin/rake
Started
..
Finished in 0.002721 seconds.

2 tests, 2 assertions, 0 failures, 0 errors
[Synthesis] Verifying expectation invocations...
Loaded suite /usr/bin/rake
Started
..
Finished in 0.002342 seconds.

2 tests, 2 assertions, 0 failures, 0 errors
[Synthesis]
[Synthesis] SUCCESS.

Traditionally, and on a project of a more realistic size than the example one, we would have to perform some sort of functional testing around the integration of BrandingService and Storage. Synthesis aims to provide enough confidence in order to eliminate the need for tedious functional tests.

nutrun

Using Synthesis with Test::Unit and Mocha