Thin, Mongrel, Rack and performance
I just ran a crud and unscientific test to satisfy my curiosity in regards to the speed difference between Thin and Mongrel.
I used Rack for convenience. This probably skews the results.
require "rubygems"
require "rack"
def run(server)
require server
s = Rack::Handler.const_get("#{server.capitalize}")
app = proc {|env| [200, {"Content-Type" => "text/plain"}, "hello"]}
s.run(app,:Host => ‘127.0.0.1′, :Port => 2323)
end
run “thin”
# run “mongrel”
I wrote a simple HTTP client sending one thousand consecutive, non simultaneous requests.
require "net/http"
before = Time.now
1000.times {Net::HTTP.new('127.0.0.1', 2323).request_get('/')}
p "#{1000/(Time.now - before)} requests per second"
The results… Thin did around 1160 requests per second, whereas Mongrel did 1220 on average.
Interestingly, when I added Rack::Lint to the mix, Thin dropped to about 640 requests per second and Mongrel dropped to 685.
s.run(Rack::Lint.new(app),:Host => '127.0.0.1', :Port => 2323)
Bearing in mind that these are non concurrent requests and that I’m running both the server and client on my Laptop (a 2.2 GHz 2GB RAM MacBook Pro) and that Rack is included in the stack, it seems that Mongrel has outperformed Thin. The most interesting, albeit accidentally discovered, observation however is the grave hit on performance noted when Rack::Lint becomes part of the equation.
April 4th, 2008 at 9:32 am
I’m pretty surprised by the results, but I got similar ones, updated my Thin gem to the most recent version and stlll saw the same.
I’m not so surprised that Rack::Lint slows things down a lot in this test - for something as simple as your test Rack::Lint is going to add a huge amount of overhead with the tests it does, and when it’s a single connection that will translate very directly to reduced requests.
I had a brief look at the strace output for both cases this morning, and nothing immediately stood out (other than the normal horrendous effects of Ruby’s green threads). I’d be interested in seeing what it’d do on a real app and multiple connectons, though - I’ll try hitting both of them with “ab” or something against my blog dev setup (it’s running purely on Rack, with several layers of Rack middleware) later today.
April 4th, 2008 at 10:03 am
Vidar,
I’m a fan of the big-framework-less approach, too and would be interested to hear the results if you go ahead with the benchmark you mention.
April 4th, 2008 at 2:03 pm
Your HTTP client doesn’t close the connection and Thin supports keep-alive, which is on by default in HTTP 1.1, that might explain the slowdown.
I’d suggest running it w/ some real benchmarking tools like ab or httperf. Try running: ab -k -n5000 127.0.0.1:2323/
Thin is not all about speed but is should be faster then mongrel when concurrency in the application is not needed (like in rails), it should use less memory then mongrel and all other Ruby servers and support a lot more concurrent connections. Check out the tuning options: run thin -h
Let me know if you have any problem w/ thin, I’d be glad to help!
April 4th, 2008 at 5:11 pm
Hey Marc,
In terms of the client closing the connection, replacing “Net::HTTP.new(’127.0.0.1′, 2323).request_get(’/')” with “Net::HTTP.new(’127.0.0.1′, 2323).start {|http| http.request_get(’/')}” should ensure the connection is closed for each request, right? I tried this, but the results didn’t change…
I totally agree with the ab/httperf suggestion, hence the “crud and unscientific” description of the example. Thanks for the cool server, by the way. It’s a viable contender.
April 4th, 2008 at 6:36 pm
Very interesting. This is a request for a static file on my blog setup using Rack::Static.
ab -k -n5000 http://www.hokstad.com:3000/static/style.css (I put it up on that port only for testing - it’s likely to be down when you see this)
Thin: 756.41 req/s
Mongrel: 104.60 req/s
ab -n5000 http://www.hokstad.com:3000/static/style.css (no keepalive):
Thin: 747 req/s
Mongrel: 211.89 req/s
With the test script above:
Thin: 138.94 req/s
Mongrel: 94.81 req/s
I first thought the -k option to “ab” caused part of the huge speedup, so that’s why I tried without it. So clearly part of it is related either to how Net::HTTP send the request or (I guess more likely) overall Net::HTTP performance. As for why Mongrel does twice as good with keepalive turned off, I have no good explanation - I reran those tests several times because I suspected I’d made a mistake.
Overall I’m thoroughly stumped, and I definitively will spend some more time looking at this…
The relevant parts of my config.ru file looks like this, btw. (the custom Rack middleware involved can be found here: http://www.hokstad.com/tag/rack )
use LatestReferers::Gather, {:exclude => [ /\/referers/, /http:\/\/www\.hokstad\.com/ ]}
use LatestReferers::View, “/referers”
use Rack::ShowStatus
use RewriteContentType, {”js” => “text/javascript”}
use CacheSettings, {
/\/static\// => { :cache_control => “max-age=86400,public”, :expires => 86400 },
/\\.html/ => { :cache_control => “max-age=86400,public”, :expires => 86400 }
}
use Rack::Static, :urls => ["/static"]