With testing occupying a major part of our development process, we have often attempted to quantify test code quality. Like many things, it is worth considering how test code ultimately manifests itself in terms of added value. This is why Stuart and I recently tend to conclude that, stripped from technically granular details, test code must fundamentally contribute in building confidence that the system under test is complete, a proof that what we’ve built is and will continue working as intended.

A working system fulfilling its business objectives can be considered complete enough, but, if not easily extensible and maintainable, will not grant itself to the conclusion of being as good as it can be. Advancements in software development methodologies that assist in delivering working software that is easy to extend and maintain - higher level abstractions, modeling and design - have been driven by the need to reduce technical debt. Technical debt can be viewed as the cost of change.

Test code is code, too. As code bases grow more elaborate, test code also suffers from technical debt, demanding methods to eliminate the factors that hinder its maintainability and extensibility. Present procedures geared towards extensible and maintainable test code are habitually counter-proportional to the amount of confidence they achieve.

The different categories on the scale are not mutually exclusive, in fact they are commonly combined as members of a suite that exercises the system in various degrees of instrumentation. Walking the scale from left (empty) to right (full), we move from tests that are generally easier to write, understand, run and maintain but at the same time are less representative of the real system with all its components integrated.

Dependency neutral tests with all of the tested component’s dependencies stubbed are disconnected, vaguely describe how the component interacts with its environment and offer minimal proof that the component will work as specified once a member of the application ecosystem.

The fundamental difference between interaction based dependency neutral tests and their stubbed counterparts is the accurate interaction specification of collaborating components through the use of mock objects instead of stubs. Here, we concentrate on specifying the contract of communication between two components. Although much closer to how the actual system operates, these tests are still disconnected. Despite the accurate specification of the interaction, we have don’t have complete proof that the pieces fit. In particular, interaction based dependency neutral tests do not offer proof that the mocked collaborators have been tested to work.

It becomes apparent that the major flaw of interaction based dependency neutral tests is their disconnect from their peers.

As we move towards the “full” side of the confidence scale, tests tend to become larger and more complicated. Wired tests draw a picture much closer to that of the system in its entire form but suffer from poor defect localization (test failures are not always directly related to the intent of the specific test) and disrespect encapsulation (setup code often exposes the behavior of components irrelevant to the context of the current test). The dependency wired tests’ contribution to technical debt is much more significant.

Understanding the importance of confidence in our system and aiming to reduce technical debt, Synthesized Testing suggests a solution that attempts to rectify the disconnect of lightweight, interaction based dependency neutral tests and reduce the need of overarching, prone to technical debt dependency wired tests.

When using RubyGems from the command line, I almost always type sudo gem i synthesis as opposed to sudo gem install rails, the emphasis targeted at using “i” instead of “install”, of course. The gem executable happily understands what command it is being asked to execute when provided with the first few letters of the command, as long as those letters are not ambiguous, i.e. don’t clash with the names of other commands. So even though sudo gem u foo complains that Ambiguous command u matches [uninstall, unpack, update], sudo gem uni foo will uninstall the specified gem.

Here’s how this is implemented in RubyGems.

def find_command(cmd_name)
  possibilities = find_command_possibilities(cmd_name)
  if possibilities.size > 1
    raise "Ambiguous command #{cmd_name} matches [#{possibilities.join(', ')}]"
  end
  if possibilities.size < 1
    raise "Unknown command #{cmd_name}"
  end

  self[possibilities.first]
end

def find_command_possibilities(cmd_name)
  len = cmd_name.length
  self.command_names.select { |n| cmd_name == n[0,len] }
end

In the same vein, although not strictly a command abbreviation, Danilo pointed out git understands abbreviated revision hashes, so it’s possible to use something like git diff d0a..HEAD even with the hash’s complete representation being d0aa7dd4aa9a95090df1e0b9d0f426d5a5bd56ae.

Less typing is almost always a good option to have. The easy to implement Unambiguous command abbreviation trick adds a subtle usability improvement to command line interfaces and holds a nice treat to the utility’s power users.

Jabber and its underlying protocol XMPP are typically associated with instant messaging applications, although the breadth and flexibility of the technology allows for implementations that can span further from traditional online chatting.

ejabberd is a fault tolerant and clusterable Jabber/XMPP server written in Erlang and presents an interesting option as a simple, lightweight and scalable message transport for distributed applications.

EventMachine is a simple and fast library for lightweight concurrency in Ruby. Its use mainly involves, but is not limited to, spawning lightweight processes whose execution can be programatically scheduled, easy and fast socket abstractions and an implementation of the Deferrable pattern as introduced by the Twisted event-driven Python networking engine.

When a Ruby class includes the EventMachine::Deferrable module, it is provided with the ability to accept arbitrary callbacks and errbacks that will get executed when its deferred status changes, in particular when it is set to either :succeeded or :failed. Let’s look at a deferrable Worker class which performs a potentially long running operation.

class Worker
  include EM::Deferrable

  def heavy_lifting
    30.times do |i|
      puts "Lifted #{i}"
      sleep 0.1
    end
    set_deferred_status :succeeded
  end
end

Inside an EventMachine loop, we can add callbacks to a Worker instance and dispatch the expensive operation to a separate thread, or an evented process. The program’s execution will continue, with any callbacks attached to Worker executed once its deferred status is set.

EM.run do
  worker = Worker.new
  worker.callback {p "done!"}
  Thread.new {worker.heavy_lifting; EM.stop}
  puts "resuming remaining program operations"
end

Now, let’s look at combining Worker with Jabber to trigger long running jobs. For Jabber server duties, I am using ejabberd on an old laptop running Debian, but there’s no reason why a mass online Jabber service like Google Talk could not be used for playing around with the example. Also, I’m using the xmpp4r-simple Ruby library, which is a wrapper around xmpp4r.

jabber = Jabber::Simple.new("bot@thrash", "password")
at_exit{jabber.status(:away, "jabot down")}

EM.run do
  EM::PeriodicTimer.new(1) do
    jabber.received_messages do |message|
      case message.body
      when "exit" : EM.stop
      when "lift" :
        EM.spawn do
          worker = Worker.new
          worker.callback {jabber.deliver(message.from, "Done lifting")}
          worker.heavy_lifting
        end.notify
        jabber.deliver(message.from, "Scheduled heavy job...")
      else jabber.deliver(message.from, "Dunno how to #{message.body}")
      end
    end
  end
end

Inside an EventMachine loop, we check for new messages every second. The program understands two commands, exit and lift. The first quits the EventMachine loop and ultimately terminates the program’s execution. When lift is received, we instantiate a new Worker inside a spawned process and add a callback so that the Worker will notify the command issuer when the job has completed. Worth noting is the use of notify to schedule the spawned process. notify returns immediately making work dispatch non-blocking - upon issuing a lift command twice, a “Scheduled heavy job…” message will be sent to the job issuer twice before the first job completes.

I use Adium to send commands to the program - an interesting way of remote controlling or interacting with applications. Of course, the real interest lies in using the setup under discussion for inter-app communication. With multicast options, presence discovery, node status updates and more, there is lot to explore in terms of distributed application development, if simple and lightweight are two keywords to be found on the highest ranks of your list.

The common pattern for mocking class instances in Ruby involves setting an expectation on the new method of the class to return an object of the specified type. Consider the following User and Account classes.

class User
  attr_reader :name, :email

  def initialize(name, email)
    @name, @email = name, email
  end

  def contact
    send_email(@email)
  end
end

class Account
  def self.create(name, email)
    user = User.new(name, email)
    user.contact
  end
end

The test for the Account#create method would look like this:

require "rubygems"
require "mocha"
require "test/unit"

class AccountTest < Test::Unit::TestCase
  def test_emails_user_on_account_creation
    user = User.new("", "")
    user.expects(:contact)
    User.expects(:new).with("Kirk", "kirk@metallica.com").returns(user)
    Account.create("Kirk", "kirk@metallica.com")
  end
end

As an alternative, and this is the version I see most often used, a duck typed mock can be used in the place of the “real” User object.

class AccountTest < Test::Unit::TestCase
  def test_emails_user_on_account_creation
    user = mock
    user.expects(:contact)
    User.expects(:new).with("Kirk", "kirk@metallica.com").returns(user)
    Account.create("Kirk", "kirk@metallica.com")
  end
end

Both implementations have some potentially undesirable issues worth taking into consideration. None of the two is DRY, since in both cases we need to set a bespoke expectation for the object’s instantiation. In the first example, we are actually instantiating a “real” User. This compromises the dependency neutrality of the test, coupling it with User’s initialization implementation. The second example, despite its relative convenience, doesn’t fully respect the contract between Account and User. The interaction specification is inaccurate, because, as far as the application code under test is concerned, User#new doesn’t return an object of type Mocha::Mock, it returns an instance of User.

The following code can be useful in an effort to DRY up code as the one under discussion, whilst driving such interaction specifications closer to the actual contracts they are meant to express.

class Object
  def self.mock_instance(*args)
    class_eval do
      alias original_initialize initialize
      def initialize()end
    end

    instance = new
    expects(:new).with(*args).returns(instance)

    class_eval do
      alias initialize original_initialize
      undef original_initialize
    end

    return instance
  end
end

Here is how AccountTest can be written with the above Mocha extension in place.

class AccountTest < Test::Unit::TestCase
  def test_emails_user_on_account_creation
    User.mock_instance("Kirk", "kirk@metallica.com").expects(:contact)
    Account.create("Kirk", "kirk@metallica.com")
  end
end

If RSpec is your flavor, the same can be achieved with minor modifications in the Spec::Mocks::Methods module.

module Spec::Mocks::Methods
  def mock_instance(*args)
    class_eval do
      alias original_initialize initialize
      def initialize()end
    end

    instance = new
    should_receive(:new).with(*args).and_return(instance)

    class_eval do
      alias initialize original_initialize
      undef original_initialize
    end

    return instance
  end
end

With the above we can use mock_instance in specs as such.

describe Account do
  it "should email user on account creation" do
    User.mock_instance("Kirk", "kirk@metallica.com").should_receive(:contact)
    Account.create("Kirk", "kirk@metallica.com")
  end
end

During projects I’ve worked on which involved coding JavaScript, I’ve had positive experiences enjoying the event driven nature of the language and some bad, especially after code bases grew larger with events firing left and right, producing side effects which were difficult to manage or track down. During the bad days, a centralized way of managing events was often brought up in conversations with colleagues as a means of managing the issue.

I’ve long been a fan of the event driven style of programming as a particularly useful alternative promoting loose coupling of system components. Below is the code for a Ruby module which, when included, enables classes to act as event dispatchers. Other classes can subscribe to the events and be notified when an event occurs.

module EventDispatcher
  module ClassMethods
    attr_reader :listeners

    def subscribe(event, &callback)
      @listeners ||= {}
      (@listeners[event] || @listeners[event] = []) << callback
    end

    private

    def clear_listeners!
      @listeners = {}
    end
  end

  def self.included(receiver)
    receiver.extend(ClassMethods)
  end

  def notify(event, *args)
    self.class.listeners[event].each {|callback| callback[*args]}
  end
end

Let’s consider an example application where the resource management department of an organization hires recruits and upon admission notifies interested services to take relevant action.

class RM
  include EventDispatcher

  def hire(name)
    notify(:new_recruit, name)
  end
end

What is especially interesting here is how the RM class doesn’t need to know anything about any interested listeners. In order to test RM, it suffices to ensure that a notification is sent upon calling the hire method.

def test_rm_notifies_on_new_hire
  service = mock
  service.expects(:new_recruit).with("name")
  RM.subscribe(:new_recruit) {|name| service.new_recruit(name)}
  RM.new.hire("name")
end

Typically, listeners would directly register their interest to the event by subscribing to it. Let’s imagine a welcoming letter is issued to new recruits the moment they join the organization.

class WelcomeLetterService
  RM.subscribe(:new_recruit) {|name| greet(name)}

  def self.greet(name)
    "Welcome, #{name}!"
  end
end

The first thing to notice is WelcomeLetterService’s direct coupling to RM. Additionally, a code base heavily employing this strategy might suffer ill effects similar to the ones described in the JavaScript inspired first paragraph of this article. Allowing a centralized option for event registration and management could serve as possible remedy to the issue.

module EventRegistry
  def register_event_listeners
    RM.subscribe(:new_recruit) {|name| WelcomeLetterService.greet(name)}
  end

  extend self
end

The EventRegistry module acts as a centralized mechanism for declaring which listeners subscribe to which events. On top of that, the WelcomeLetterService is now completely oblivious to the RM class.

class WelcomeLetterService
  def self.greet(name)
    "Welcome, #{name}!"
  end
end

Due to the level of decoupling achieved, testing these two components becomes particularly easy.

def test_welcome_letter_service_greets_by_name
  assert_equal(WelcomeLetterService.greet("Name"), "Welcome, Name!")
end

def test_event_subscription_wiring
  WelcomeLetterService.expects(:greet).with("name")
  RM.expects(:subscribe).with(:new_recruit).yields("name")
  EventRegistry.register_event_listeners
end

All this ties well with the philosophy behind Synthesized Testing, where a coherent collection of lightweight tests becomes a major factor of confidence that the system under test is complete, reducing the need for complex overarching tests.

Sinatra doesn’t yet seem to come with built in support for Erubis. If you want the relative speed bump of Erubis over standard ERB for your Sinatra application, the code below should do the trick.

module Sinatra::Erb
  def erb(content, options={})
    begin
      require 'erubis'
      @@erb_class = Erubis::Eruby
    rescue LoadError
      require "erb"
      @@erb_class = ::ERB
    end
    render(:erb, content, options)
  end

  private

  def render_erb(content, options = {})
    @@erb_class.new(content).result(binding)
  end
end

Application code will not require any modifications, Sinatra will use Erubis if the gem is found in the load path.

Following the post on non blocking merb actions, it’s worth mentioning the Merb::ControllerMixin module, its render_deferred and render_then_call methods in particular.

render_deferred accepts a block argument, a proc that will be called outside the dispatch mutex, releasing the merb thread lock.

class MyController < Merb::Controller
  def hello
    render "hello"
  end

  def dont_wait_to_hello
    render_deferred { VerySlow.hello }
  end
end

Calls to /my_controller/hello, subsequent to requests to /my_controller/dont_wait_to_hello, will not have to wait for the latter to complete before being served.

As an interesting caveat, because execution of the proc is outside merb’s control, any exceptions raised in the block passed to render_deferred will not be rescued and redispatched, which suggests that custom error handling needs to be in place.

render_then_call accepts two arguments, a string that will be returned immediately as a response to the client and a block to be called after the string has been returned.

class MyController < Merb::Controller
  def receive_order
    render_then_call("Your pizza is in the oven.") do
      bake_pizza
      deliver_pizza
    end
  end
end

In this case, the specified message will be immediately returned, while the long running proc will be scheduled and executed by Mongrel, allowing orders to keep coming through a single application instance.

One of the most powerful features in merb is an action’s ability to return a proc, thus releasing the controller block and transferring control to the application server which will be able to handle requests in a parallel manner.

Consider the following controller.

class MyController < Application
  def hello
    render "hello"
  end

  def wait_and_hello
    sleep 10
    render "hello"
  end

  def dont_wait_to_hello
    proc { sleep 10; p "hello" }
  end
end

Start merb and access http://localhost:4000/my_controller/hello after visiting http://localhost:4000/my_controller/wait_and_hello. Ten seconds will pass before /my_controller/hello is served. A request to /my_controller/hello will be served immediately regardless of whether a preceding request to /my_controller/dont_wait_to_hello has completed.

I just ran a crud and unscientific test to satisfy my curiosity in regards to the speed difference between Thin and Mongrel.

I used Rack for convenience. This probably skews the results.

require "rubygems"
require "rack"

def run(server)
  require server
  s = Rack::Handler.const_get("#{server.capitalize}")
  app = proc {|env| [200, {"Content-Type" => "text/plain"}, "hello"]}
  s.run(app,:Host => '127.0.0.1', :Port => 2323)
end

run "thin"
# run "mongrel"

I wrote a simple HTTP client sending one thousand consecutive, non simultaneous requests.

require "net/http"

before = Time.now
1000.times {Net::HTTP.new('127.0.0.1', 2323).request_get('/')}
p "#{1000/(Time.now - before)} requests per second"

The results… Thin did around 1160 requests per second, whereas Mongrel did 1220 on average.

Interestingly, when I added Rack::Lint to the mix, Thin dropped to about 640 requests per second and Mongrel dropped to 685.

s.run(Rack::Lint.new(app),:Host => '127.0.0.1', :Port => 2323)

Bearing in mind that these are non concurrent requests and that I’m running both the server and client on my Laptop (a 2.2 GHz 2GB RAM MacBook Pro) and that Rack is included in the stack, it seems that Mongrel has outperformed Thin. The most interesting, albeit accidentally discovered, observation however is the grave hit on performance noted when Rack::Lint becomes part of the equation.

Nu is an interpreted object oriented language built on top of Objective C, with its syntax and several of its mechanics coming from Lisp, whilst being inspired by Ruby’s philosophy.

The Event Dispatcher example is one of my favorite when trying out a new language. I choose it because it’s representative of a number of programming language core topics such as lists, maps, closures, OO, etc.

I recently spent a short while porting the IO Language Event Dispatcher example in Nu and will be going over my experiment in this article.

Bellow is the code for the EventDispatcher class.

(class EventDispatcher is NSObject
  (- (id) initialize is
    (set @listeners (dict))
      self)

  (- (void) subscribe:(id)event callback:(id)callback is
    (unless (@listeners objectForKey:event)
      (@listeners setObject:(array) forKey:event))
      ((@listeners objectForKey:event) << callback))

  (- (void) notify:(id)event withArgs:(id)args is
    ((@listeners objectForKey:event) each: (do
      (callback) (callback args)))))

There are two prominent methods in this class, namely subscribe and notify.

The subscribe method takes two arguments, the event we’re subscribing to and a callback to be invoked every time the specified event takes place. The @listeners dictionary is constructed using the dict directive, which is shorthand for creating an NSCFDictionary instance. For each event, we maintain an array of callbacks. Like dict, array is available for conveniently creating NSCFArray instances.

Whenever notify is invoked with an event and the relevant arguments, we iterate invoking any callbacks registered for the given event, passing the provided arguments to each.

There are two ways that I know of that will allow classes to act as event dispatchers, inheritance being the first.

(class Factory is EventDispatcher
  (ivars)

  (- (void) produce:(id)color is
    (puts "#{color} product produced")
    (self notify:"new product" withArgs:color)))

Here, we create a Factory class which extends EventDispatcher and we define a produce method which notifies listeners that have subscribed to the “new product” event.

Composition is the second, and my preferred, option.

(class Factory is NSObject
  (ivars)

  (- (void) produce:(id)color is
    (puts "#{color} product produced")
    (self notify:"new product" withArgs:color)))

(Factory include:EventDispatcher)

Classes in Nu can be used in a manner similar to that of Module mix-ins in Ruby. Every object in Nu has an include method which will make a class’s instance methods available in another class. The advantage here is that we didn’t have to change Factory’s inheritance tree.

Following is a potential event listener that subscribes to the “new product” event and prints out a message every time the event is fired.

(class ProductWatcher is NSObject
  (- (id) watch is
    (factory subscribe:"new product" callback:(do (color)
      (puts "I see a #{color} product")))))

To run the example, we can instantiate a Factory and a ProducWatcher and fire a few events.

(set factory ((Factory new) initialize))
((ProductWatcher new) watch)

(factory produce:"black")
(factory produce:"red")
(factory produce:"blue")

The results look like this:

black product produced
I see a black product
red product produced
I see a red product
blue product produced
I see a blue product

There are a lot of reasons as to why I have time for Nu, its combining 3 of my favorite languages not included, greatly simplifying Cocoa development for native Mac and iPhone applications and Objective C’s excellent multi-core support to name but a few.