It’s a well-known stereotype that “Ruby is slow”. People say it on Slashdot. People say it on OSNews. People say it on my MSN contact list. A few quotes:

• “If you care even a little about performance, you don’t use Ruby.”
• “You thought Python is slow? Well Ruby is even slower!”

Another stereotype we hear is that “Ruby on Rails doesn’t scale”. Most of the people who say that seem to use PHP, so they imply that PHP does scale.

How much of this is true? If there’s anything I’ve learned in the past few years, it’s that I shouldn’t always take public opinion seriously. PHP is not really a great language; its interpreter and perhaps even the language specification (if that even exists) has many quirks. My friend Ninh has worked on PHP in the past. He has exposed many of those strange quirks in his blog (which is, unfortunately, down because of a recent data loss incident).

I can’t imagine that PHP is faster than Ruby (the language; I’m not talking about Rails). If Ruby on Rails is slower than PHP, then how much of that is to blame on the language, and how much on the framework? Let’s put this to the test.

Goal of and rationale behind this test

We must first agree on the following premises:
People claim that Ruby is so slow that it’s impossible to build anything in it that’s sufficiently fast for “production environments”. (1)
Yet at the same time, they accept PHP as being fast enough. (2)

From (1) and (2) follows this question: Is it possible at all to write a web framework in Ruby that’s faster than PHP?

If (1) and (2) are true, i.e. if Ruby web apps are indeed slower than PHP web apps, then the culprit might be:

• The web framework that the Ruby web app is written in. If this is the case, then it is possible to write a faster web framework that matches PHP’s performance. (3)
• Ruby itself, i.e. the language. If this is the case, then it does not matter what web framework we use or how we write it, it will always be slower than equivalent frameworks PHP. (4)
• A combination of both, in which both parts play a significant role. (5)

The goal of this benchmark is to find out whether (4) is true. (4) is what people have been arguing for years. And that’s assuming that people know what they’re talking about; they often don’t, and confuse the Ruby language with Ruby on Rails the framework, or even with web server Ruby integration modules.

Benchmarking PHP and Ruby (and Perl, Python, C++)

I want to benchmark PHP and Ruby’s code execution speed.

• I specifically do not want to benchmark code parsing speed. Ruby on Rails apps run in application servers, so all code is only parsed once. PHP has stuff like PHP Accelerator/Zend Accelerator (or whatever those products are called) which cache the parse tree.
• I’m not trying to benchmark HTTP request processing. This is only about code execution speed.

So, I decided to implemented the mergesort algorithm in PHP and Ruby. The scripts will sort an array of 100 numbers (which where randomly generated and hardcoded into the source code), and perform this sorting 3000 times. The execution times, in seconds, are as follows:

As you can see, I’ve also written a Perl, Python and C++ version just for comparison fun.

The sources code of the test programs can be found on the bottom of this blog post. A few notes:

• Operating system: Ubuntu Linux 7.10
• Hardware: Intel Core 2 Duo T5300 1.73 Ghz, 2 GB RAM
• Versions: PHP 5.2.3-1ubuntu6.2, Ruby 1.8.6 (2007-06-07 patchlevel 36), Perl 5.8.8, Python 2.5.1
• The test programs not only test raw CPU execution speed, but also memory management speed. As you can see in the merge() functions, it creates a new array, and modify the given arrays. As a result, the mergesort() functions will create a lot of objects that will have to be garbage collected.
• Ruby uses a conservative mark-and-sweep garbage collector. Perl and Python both use reference counting. The C++ test program uses Boost’s shared_ptr, and thus uses reference counting for memory management. I’m not sure how PHP manages its memory, or whether it does it at all, because all objects are freed after an HTTP request anyway.
• The C++ test program is compiled with g++ 4.1, with -O2.
• By using Psyco, a JIT optimizer for Python, the Python test program’s execution time went down to a whopping 0.640 seconds. That’s even faster than my C++ version (!)

But as you can see in the graph, Ruby beats PHP even with a full-blown mark-and-sweep garbage collector. Ruby’s garbage collector was run several dozens times while running the test program, and Ruby still beats PHP in execution time.

I have no choice but to conclude that all the “Ruby is slow” stereotypes are bollocks. PHP the language is slower, yet we do not see anybody complaining about PHP “scalability”. The Rails framework could be faster, yes, but at least now we know the language is not as big as a bottleneck as some people might think.
Wait! Do I see a flame thrower there? Read on before you reply!

“But but but… you didn’t benchmark the web environment!”

Yes. I did not benchmark the HTTP request handling, and this is intentional. People claim that Ruby is so slow, that it is impossible to write anything in it that’s acceptably fast. This benchmark proves that, as a language, Ruby is faster than PHP. Therefore, if people can write frameworks in PHP that’s acceptably fast, then it is also possible to write frameworks in Ruby that’s acceptably fast.

Note that I do not make any claims about Ruby on Rails’s speed compared to typical PHP applications.

“Why don’t you use the language’s builtin sort function? This benchmark is crap!”

Implementing sort in the languge instead of using the builtin sort function is intentional. The goal of the benchmark is to test *code execution speed*, not to find out which language can sort faster. This benchmark is a good test because it tests different language primitives, like function calls, comarison operators, arrays, etc. I might as well have implemented A* graph searching, but mergesort is easier.

Appendix: Source codes

• PHP
• Ruby
• Perl
• Python
• C++