Archive for January, 2008

Rack RESTful Dispatcher

Tuesday, January 29th, 2008

Quoting the authors, Rack provides a minimal, modular and adaptable interface for developing web applications in Ruby. By wrapping HTTP requests and responses in the simplest way possible, it unifies and distills the API for web servers, web frameworks, and software in between (the so-called middleware) into a single method call.

To test the theory in practice, I put together a superficial interface for writing RESTful HTTP services by implementing any of the DELETE, GET, POST, PUT, etc HTTP verbs in Ruby classes.

require "rubygems"
require "rack"

module RestfulDispatcher
  def call(env)
    request = Rack::Request.new(env)
    dispatcher = dispatcher_class.new(request)
    body = dispatcher.send(request.request_method.downcase)
    [200, {'Content-Type' => dispatcher.content_type}, body]
  end

  def dispatcher_class
    @dispatcher ||= Class.new(self.class) do
      attr_accessor :content_type

      def initialize(request)
        @request, @content_type = request, ‘text/xml’
      end
    end
  end

  module SingletonMethods
    def start(handler, host, port)
      handler.run Rack::Lint.new(self.new), :Host => host, :Port => port
    end
  end

  def self.included(receiver)
    receiver.extend SingletonMethods
  end
end

Let’s examine the example one method at a time.

All Rack applications must implement one method - call - which accepts one argument, the environment, which encapsulates data relevant to the HTTP roundtrip. The call method must return an array of 3 items: a greater than 100 integer representation of the response status, a hash holding name/value pairs representing the response’s header entries and an array of strings - the body of the response.

In the case of the RestfulDispatcher module, call first wraps env in Rack::Request, a convenient and stateless interface to the Rack environment. We then create a new instance of a dispatcher class passing it the Rack Request. We will revisit this in more detail when talking about the dispatcher_class method. Instantiating a new dispatcher to handle the request should keep things thread safe. We then call a method on the dispatcher instance corresponding to the HTTP verb included in the request. This call should return the response body. Finally, adhering to the Rack standard, we return an array containing the response status code, headers and body.

dispatcher_class creates a new class by subclassing the service we will be defining and giving it a constructor that accepts a request object. We also expose the content_type field, in case we want to override it anywhere in our service’s implementation.

Finally, we provide a start singleton method which we can call to start the service.

Mixing the RestfulDispatcher module in a class will effectively enable the class to act as a standalone RESTful service. All we need to do is implement instance methods that correspond to the HTTP verbs we want the service to respond to.

class FooService
  include RestfulDispatcher

  def get
    "<test>#{@request.GET['key']}</test>”
  end
end

require “thin”
FooService.start Rack::Handler::Thin, ‘127.0.0.1′, 2323

Rack::Request#GET conveniently returns the data received in the request query string (e.g http://127.0.0.1/?key=hello) as a hash.

Thanks to Rack’s modular nature, switching from Thin to Mongrel is as easy as replacing the last two lines of the code above with:

require "mongrel"
FooService.start Rack::Handler::Mongrel, '127.0.0.1', 2323

Synthesis 0.0.2

Sunday, January 27th, 2008

Synthesis version 0.0.2 was released a bit more than a week ago carrying one prominent new feature: Validation of simulated method call expectations takes into account the types of the arguments of the method’s signature.

Let’s revisit the example project from the Using Synthesis with Test::Unit and Mocha article and change the save method of the Storage class to take an additional argument - mode.

storage.rb:

class Storage
  def initialize(filename)
    @filename = filename
  end

  def save(val, mode)
    File.open(@filename, mode) {|f| f << val}
  end
end

We should also update the corresponding unit test.

storage_test.rb:

require "test/unit"
require "fileutils"
require File.dirname(__FILE__) + "/../lib/synthesis_example"

class StorageTest < Test::Unit::TestCase
  def test_saves_to_file
    Storage.new('test.txt').save('rock', 'w')
    assert_equal 'rock', File.read('test.txt')
  ensure
    FileUtils.rm_f('test.txt')
  end
end

Running storage_test.rb produces:

TW-MacBook-Pro:synthesis_example gmalamid$ ruby test/storage_test.rb
Loaded suite test/storage_test
Started
.
Finished in 0.004917 seconds.

1 tests, 1 assertions, 0 failures, 0 errors

Also, data_brander_test.rb still passes.

TW-MacBook-Pro:synthesis_example gmalamid$ ruby test/data_brander_test.rb
Loaded suite test/data_brander_test
Started
.
Finished in 0.00047 seconds.

1 tests, 1 assertions, 0 failures, 0 errors

At this point, all of the application’s tests are producing a green build, although there is an obvious bug. The signature of Storage#save has changed, so DataBrander#save_branded is broken.

With version 0.0.2, Synthesis attempts to address this issue and, indeed, invoking the test:synthesis task produces:

W-MacBook-Pro:synthesis_example gmalamid$ rake
(in /Users/gmalamid/devel/ruby/whatever_code/synthesis_example)
[Synthesis] Collecting expectations…
Loaded suite /usr/bin/rake
Started
..
Finished in 0.002701 seconds.

2 tests, 2 assertions, 0 failures, 0 errors
[Synthesis] Verifying expectation invocations…
Loaded suite /usr/bin/rake
Started
..
Finished in 0.002483 seconds.

2 tests, 2 assertions, 0 failures, 0 errors
[Synthesis]
[Synthesis] Tested Expectations:
[Synthesis]
[Synthesis] Untested Expectations:
[Synthesis] Storage.new.save(String)
[Synthesis]
[Synthesis] Ignoring:
[Synthesis]
[Synthesis] FAILED.

In order for the Synthesis task to be successful we need to update data_brander.rb and the corresponding test to correctly cover the interaction between DataBrander and Storage, and, conveniently, fix the associated bug.

data_brander.rb:

class DataBrander
  BRAND = "METAL"

  def initialize(storage)
    @storage = storage
  end

  def save_branded(data)
    @storage.save "#{BRAND} - #{data}", "w"
  end
end

data_brander_test.rb:

%w(test/unit rubygems mocha).each { |l| require l }
require File.dirname(__FILE__) + "/../lib/synthesis_example"

class DataBranderTest << Test::Unit::TestCase
  def test_saves_branded_to_storage
    storage = Storage.new 'whatever'
    storage.expects(:save).with('METAL - rock', 'w')
    DataBrander.new(storage).save_branded 'rock'
  end
end

This will hopefully tip the confidence scale a bit closer to the point where we feel it’s safe enough to omit having to write some functional tests that would prove the interacting members will integrate nicely when used together.

TW-MacBook-Pro:synthesis_example gmalamid$ rake
(in /Users/gmalamid/devel/ruby/whatever_code/synthesis_example)
[Synthesis] Collecting expectations…
Loaded suite /usr/bin/rake
Started
..
Finished in 0.002724 seconds.

2 tests, 2 assertions, 0 failures, 0 errors
[Synthesis] Verifying expectation invocations…
Loaded suite /usr/bin/rake
Started
..
Finished in 0.002409 seconds.

2 tests, 2 assertions, 0 failures, 0 errors
[Synthesis]
[Synthesis] SUCCESS.

rubyworks-ec2 0.1.4

Saturday, January 26th, 2008

The rubyworks-ec2 project is now hosted on RubyForge. The gem can be installed by invoking gem i rubyworks-ec2 or can be downloaded from the project’s RubyForge page.

Thanks to Markus Bengts’s input, version 0.1.4 adds a list of new features to the mix.

A capify-for-ec2 executable that will create or update the existing Capfile and config/deploy.rb files to include the EC2 Capistrano recipes. It will also copy server configuration files (e.g. config files for apache2, monit, mysql, etc) to your local working copy’s config/server directory.

Think of config/server as a local copy of the EC2 server instance’s filesystem. The files in config/server can be edited before bootstrapping the instance. They will be copied to the EC2 instance when cap instance:bootstrap is invoked.

The default configuration has been changed so that Monit manages MySQL, and MySQL is allowed to use more RAM (up to about 600MB if needed). MySQL and Monit are configured so that slow queries are written to a log file (default slow query limit is 2 seconds). Monit alerts if that file has been changed.

This release also allows specifying additional apt packages and gems to install on cap instance:bootstrap. To do so, add packages and gems to the lists that were copied to config/deploy.rb by capify-for-ec2.

Using Synthesis with Expectations

Sunday, January 13th, 2008

This example builds on the test project described in Using Synthesis with Test::Unit and Mocha. Instead of a combination of Test::Unit and Mocha, we will be using Synthesis in conjunction with the Expectations lightweight testing framework.

Synthesis comes with an Expectations adapter. The rake task looks like this:

Rakefile:

require "rubygems"
require "synthesis/task"

task :default => 'synthesis:test'
Synthesis::Task.new do |t|
  t.adapter = :expectations
end

Following are the project’s tests, rewritten with the Expectations library.

data_brander_test.rb:

require "rubygems"
require "expectations"
require File.dirname(__FILE__) + "/../lib/synthesis_example"

Expectations do
  expect Storage.new('yep').to_receive(:save).with('METAL - rock') do |s|
    DataBrander.new(s).save_branded 'rock'
  end
end

Running this test will produce:

Macintosh-4:synthesis_example gm$ ruby test/data_brander_test.rb
Expectations .
Finished in 0.001 seconds

Success: 1 fulfilled

Because we haven’t yet written a test for the simulated save method on the mocked Storage instance, the Synthesis Rake task will fail.

Macintosh-4:synthesis_example gm$ rake synthesis:test
(in /Users/gm/devel/ruby/whatever_code/synthesis_example)
[Synthesis] Collecting expectations…
Expectations .
Finished in 0.001 seconds

Success: 1 fulfilled
[Synthesis] Verifying expectation invocations…
Expectations .
Finished in 0.001 seconds

Success: 1 fulfilled
[Synthesis]
[Synthesis] Tested Expectations:
[Synthesis]
[Synthesis] Untested Expectations:
[Synthesis] Storage.new.save
[Synthesis]
[Synthesis] Ignoring:
[Synthesis]
[Synthesis] FAILED.

Writing a test for the concrete implementation of Storage.new.save fixes the issue.

storage_test.rb:

require "rubygems"
require "expectations"
require File.dirname(__FILE__) + "/../lib/synthesis_example"

Expectations do
  expect "rock" do
    begin
      Storage.new('test.txt').save('rock')
      File.read 'test.txt'
    ensure
      FileUtils.rm_f 'test.txt'
    end
  end
end
Macintosh-4:synthesis_example gm$ rake synthesis:test
(in /Users/gm/devel/ruby/whatever_code/synthesis_example)
[Synthesis] Collecting expectations…
Expectations ..
Finished in 0.00199 seconds

Success: 2 fulfilled
[Synthesis] Verifying expectation invocations…
Expectations ..
Finished in 0.00174 seconds

Success: 2 fulfilled
[Synthesis]
[Synthesis] SUCCESS.

Using Synthesis with Test::Unit and Mocha

Sunday, January 13th, 2008

Synthesis is a Ruby library that applies a Synthesized Testing strategy, aiming to reduce the number of large, slow and brittle functional tests.

Imagine an example project with the following contents:

Macintosh-4:synthesis_example gm$ ls -R
Rakefile	lib		test

./lib:
synthesis_example	synthesis_example.rb

./lib/synthesis_example:
data_brander.rb	storage.rb

./test:
data_brander_test.rb	storage_test.rb

synthesis_example.rb:

$: << File.dirname(__FILE__) + '/'
require "synthesis_example/data_brander"
require "synthesis_example/storage"

data_brander.rb:

class DataBrander
  BRAND = "METAL"

  def initialize(storage)
    @storage = storage
  end

  def save_branded(data)
    @storage.save "#{BRAND} - #{data}"
  end
end

storage.rb:

class Storage
  def initialize(filename)
    @filename = filename
  end

  def save(val)
    File.open(@filename, 'w') {|f| f << val}
  end
end

Rakefile:

require "rubygems"
require "synthesis/task"

task :default => 'synthesis:test'
Synthesis::Task.new

data_brander_test.rb:

%w(test/unit rubygems mocha).each { |l| require l }
require File.dirname(__FILE__) + "/../lib/synthesis_example"

class DataBranderTest < Test::Unit::TestCase
  def test_saves_branded_to_storage
    storage = Storage.new 'whatever'
    storage.expects(:save).with('METAL - rock')
    DataBrander.new(storage).save_branded 'rock'
  end
end

Running data_brander_test.rb produces:

Macintosh-4:synthesis_example gm$ ruby test/data_brander_test.rb
Loaded suite test/data_brander_test
Started
.
Finished in 0.000487 seconds.

1 tests, 1 assertions, 0 failures, 0 errors

Supposing we haven’t written any tests for Storage yet, the outcome of the synthesis:test task is:

Macintosh-4:synthesis_example gm$ rake synthesis:test
(in /Users/gm/devel/ruby/whatever_code/synthesis_example)
[Synthesis] Collecting expectations…
Loaded suite /usr/bin/rake
Started
.
Finished in 0.00063 seconds.

1 tests, 1 assertions, 0 failures, 0 errors
[Synthesis] Verifying expectation invocations…
Loaded suite /usr/bin/rake
Started
.
Finished in 0.000575 seconds.

1 tests, 1 assertions, 0 failures, 0 errors
[Synthesis]
[Synthesis] Tested Expectations:
[Synthesis]
[Synthesis] Untested Expectations:
[Synthesis] Storage.new.save
[Synthesis]
[Synthesis] Ignoring:
[Synthesis]
[Synthesis] FAILED.

Synthesis will make a first pass at running the project’s tests collecting all simulated object interaction expectations. Then, it will run the tests again, verifying that the concrete implementations of the simulated expectation members have been covered in the tests.

In this example, the Synthesis task fails, reporting that the concrete implementation of the save instance method of Storage has not been tested. Let’s fix that.

storage_test.rb:

require "test/unit"
require "fileutils"
require File.dirname(__FILE__) + "/../lib/synthesis_example"

class StorageTest < Test::Unit::TestCase
  def test_saves_to_file
    Storage.new('test.txt').save('rock')
    assert_equal 'rock', File.read('test.txt')
  ensure
    FileUtils.rm_f('test.txt')
  end
end

Running rake again:

Macintosh-4:synthesis_example gm$ rake
(in /Users/gm/devel/ruby/whatever_code/synthesis_example)
[Synthesis] Collecting expectations…
Loaded suite /usr/bin/rake
Started
..
Finished in 0.002721 seconds.

2 tests, 2 assertions, 0 failures, 0 errors
[Synthesis] Verifying expectation invocations…
Loaded suite /usr/bin/rake
Started
..
Finished in 0.002342 seconds.

2 tests, 2 assertions, 0 failures, 0 errors
[Synthesis]
[Synthesis] SUCCESS.

Traditionally, and on a project of a more realistic size than the example one, we would have to perform some sort of functional testing around the integration of BrandingService and Storage. Synthesis aims to provide enough confidence in order to eliminate the need for tedious functional tests.

Synthesized Testing: A primer

Saturday, January 12th, 2008

… or, reducing the volume and increasing the value of test code by connecting the dots.

Beside other objectives, test code aims to provide proof and confidence that the application code under test works as expected and as specified.

Tests are often classified under different categories, namely Unit Tests, Functional Tests, Integration Tests, Acceptance Tests, etc, each of which attempts to verify the system components’ intended functionality in various degrees of instrumentation.

Unit tests, for example, are employed for testing individual system components in isolation from their peers or environment. Unit tests rarely connect to the database, touch the filesystem or access environment resources.

A typical test code-base will contain a layer of Functional or Integration tests sitting directly above the Unit Test layer. These still don’t test the system in its whole, deployed form. They concentrate on testing the functionality of application components with their system dependencies present and wired.

At the functional testing level, testing a Service which accesses a Repository retrieving records from a database will involve opening an actual database connection, setting up and testing against real data.

dependency = DependencyRepositoty.create_dependency('Foo')
record = Repository.create_record('Bar', dependency)
assert_equal :record, Service.report(record.id)

Functional tests are often deemed necessary in order to achieve a sense of confidence that the pieces still work when put together. At the same time, because of their relative complexity, functional tests tend to become long, slow to run, difficult to write and maintain. In essence, the bulk of Functional Tests violates many of the qualities one might attribute to good test code.

The use of Mock Objects is a technique commonly found in Unit Tests aiming to verify interactions between the objects under test. Using Mocks, we concentrate on validating declared expectations of those interactions without relying on every single component to be loaded in order for the test to run.

Repository.expects(:find).with(1).returns(:record)
assert_equal :record, Service.report(1)

By examining the two code examples, it is apparent there is overlap between what they logically test. Both verify the Service’s behavior in regards to its communication with the Repository and its handling of the data involved between the call to Service.report and this interaction.

Behind the scenes, the functional test also ensures that the database connection works, the wiring between the Repository and the database adapter is functional, etc. These verifications are irrelevant under the context of what is being tested here. They also end up duplicated in all Functional tests that involve the Repository.

Tests with functional dependencies are brittle and tend to break for the wrong reasons.

Furthermore, it is common for test code-bases to involve a layer of Acceptance tests which are executed against the entire system in - or close to - its deployed form. As a result there is more testing overlap, this time between what the Functional and Acceptance tests are targeting.

The example unit test proves that a call to Service.report(1) will result to a call to Repository.find(1). It further asserts that the value returned by the call to Repository.find(1) is the one returned by Service.report(1).

The association of the Service.report method’s concrete implementation to a test attests that the actual implementation of this method has been tested.

This test doesn’t offer enough proof the components under test will work as intended as part of the deployed application. In particular, there is no evidence Repository.find has been tested to work.

A code-base with adequate test coverage must contain tests verifying the Repository’s concrete implementation’s functionality.

A programmer observing that Object A expects to receive a call on method B when method C is called on Object D, having proven that Object A’s B method works, mentally processes this information to conclude that the two Object members will work together as expected under the specified interaction.

If we could correlate the verified interaction expectations with tests against their concrete counterparts, we should be able to provide enough evidence that the dots will indeed work together once connected.

By doing so, we can significantly reduce the volume and complexity of the functional tests, achieving a leaner, more meaningful, more robust test code-base.

As proof of concept, Synthesis is a Ruby implementation of the Synthesized Testing theory. It analyzes test code by collecting Mock Object expectations and verifies that their concrete implementations have been tested.