Deathmatch may be an overstatement but here are the results from some performance benchmarking.
The Setup:
Server:
- CENTOS 5.1
- Dual 2.4GHz Xeon CPUs
- 4GB RAM
- RAID5 (4 x 15k disks)
- Server and test client were connected via a consumer grade 10/100 switch
Configurations:
- Basic static vhost
- Keepalive turned on and with timeout of 15 seconds
- GZIP turned on
I used autobench to perform the tests. Basically this is a perl script that sits on top of httperf and will run multiple tests in succession outputing the results to CSV. Awfully convenient.
All the tests were run against the same robots.txt file. Autobench ran the following command 20 times incrementing the request rate by 10 each time. I started at 10 requests per second and went up to 200.
httperf –timeout=5 –client=0/1 –server=HOST –port=80 –uri=/robots.txt –rate=X –send-buffer=4096 –recv-buffer=16384 –num-conns=5000 –num-calls=10
I performed two samples and arbitrarily used the second as the results shown here. At the bottom of this post I will have spreadsheet containing the data from these tests so you can check out all the results.
The Results:
Both web servers performed well in all the tests and had no issues completing the requests. So I will not mention the metrics that they finished very closely on, only the ones that they did not have similar results.
There were three httperf related tests that Nginx and Apache differed on more than small amount, reply rate, network I/O and response time.
This one really piqued my interest. It seems strange to me why we would see such a result from Apache. In both tests there was a big difference at the 700 request mark. Statistically the difference was only on the max reply rate. The average and minimum are within a few tenths of a percent consistent through the tests. The max for Apache in the first test was 734.7 and in the second 758.7, standard deviations of 13.9 and 22.9 respectively. I suppose the real question here is whether or not this is my test or how Apache acts. If it is the later it seems strange that dealing with 700 requests would be any different than dealing with 800. From 800 requests to 2000, the larger differences in these results seems more realistic, controlled and gradual.
The network I/O graph I find interesting mostly because I don’t know how to take it. On one hand it seems Apache is simply using more bandwidth to do the same number of requests as Nginx. Which would seem bad. On the other it could just mean that Apache does a better job of consuming and using the available pipe. Which would seem good. My hunch is that it is the former.
The response times are also interesting since Nginx responds consistently at 0.4 ms. I am not sure why this is since I don’t know the internals to Nginx but I imagine that it is something that is built into how it deals with requests.
While the httperf tests were running I collected sar data from that time. The results show that Nginx uses a good amount less CPU and produces equally less load.
Apache:
CPU:
Load:
Nginx:
CPU:
Load:
Thats all I got, pretty cool. Nginx seems to compete pretty well with Apache and there doesn’t seem like there is a good reason not to use it especially in CPU usage constrained situations (ie huge traffic, slow machines and etc).
Here’s my results spreadsheet for the detailed results of each httperf metric.
11 responses so far ↓
1 Uncle ED // Mar 5, 2008 at 6:34 pm
Hey Joe,
Did your test system have dual core, I hear Nginx will see both cores, I have been playing around with Apache and can only get it to see one.
2 joe // Mar 5, 2008 at 7:13 pm
Hi Uncle Ed,
It was a dual CPU (single core) system.
I think nginx you can setup multiple workers to deal with requests if you have more than one core or cpu.
Check out (search for “worker_proceses”):
http://wiki.codemongers.com/NginxMainAndEventModules
As far as Apache goes it shouldn’t be worried about the type of CPU. It will just spin off child processes on whatever core or CPU is available at that time. So you can have numerous Apache processes running and have them on multiple cores/cpus.
For more info check out: http://httpd.apache.org/docs/2.0/mod/prefork.html#how-it-works
Hope this helps.
3 Thoughts From Inside The Ether » Blog Archive » Apache vs. nginx : Web Server Performance Deathmatch // Mar 7, 2008 at 3:42 pm
[...] engineers, Joe Williams, decided to put both system to the test with a Battle Royale. Check out the results. Posted by matthew.porterTags: apache, mongrel, nginx, rails, ruby in Tech [...]
4 Dan P // Mar 8, 2008 at 10:45 am
On the net_io graph, I’m wondering if Apache is sending more headers in the response than Nginx. That might account for the extra (and increasing from more requests) bandwidth usage.
5 Anynymous // Mar 8, 2008 at 12:02 pm
You used the Prefork MPM? Why not Worker or Event?
Where are the config files anyway? What version of Nginx and Apache? Are these packages or builds, and if builds what options were used?
6 joe // Mar 8, 2008 at 2:06 pm
Dan P : Thats a good point, it is something I was wondering myself. Since I used a small file to test with the headers would constitute a large percent of the traffic which may be the cause of the difference.
Anynymous: The default redhat/centos configuration packages and versions were used for the tests. The only change was to turn on gzip in both cases and turn on keepalives in Apache. Nginx has these on by default. Hope this helps.
7 Anynymous // Mar 8, 2008 at 2:29 pm
joe: thanks for the reply.
It’d be appreciated if you’d post the configs in the future. A lot of us don’t have Redhat boxes laying around. Even if we did, it’s a lot easier to read exactly what YOU had if you post it.
And of course this blog post will probably last longer than the current version of a distro. When someone visits in one or two years time, will the stock versions and configs be the same?
8 Andreas Schipplock // Mar 8, 2008 at 3:20 pm
Hey Joe,
I appreciate your work but you are comparing a Suzuki Swift with a Lexus LS and that’s simply because Apache is dynamic in any respect and that’s what lighttpd and nginx are missing though it makes sense they exist. In fact you can even trim Apache down to the state of lighttpd and nginx but then again you would loose a lot of functionality which is mandatory in most setups.
If you read the wiki or irc channel of lighttpd e.g. you might notice that there are situations where lighttpd and probably nginx are slower because of their nature. Apache in contrast tries to serve the needs of anyone in the default configuration and this of course can’t be the fastest one though you _can_ configure apache for your needs and that’s where apache’s flexibility comes in.
My experiences with nginx and lighttpd are very positive because on slow machines I really could save the cpu’s life 2 years ago but today I changed my setup in the way that I use Apache for dynamic content (for webapps) and lighttpd and nginx for proxying and static content. There is nothing easier to proxy with lighttpd and nginx to an apache instance that serves dynamic content and especially for slow clients it would save your ass and even primitive loadbalancing is possible in minutes without anyone taking your whole money :).
But whenever you plan to use lighttpd and nginx standalone keep in mind that configuring lighttpd and nginx in the first run appears very easy … just because it _is_ easy but whenever you need something special you need to get in touch with the wiki or the developer team to ask _if_ it’s possible and if it works.
Another possible problem with nginx especially is the fact that the development is done by only one person so even if companies are using nginx to proxy to their apache or iis farm (:P) I recommend lighttpd as its community is much more involved and reports bugs very fast.
All these things you have to take into account and my advice is to use the best of both nginx and apache or lighttpd and apache.
But if we are honest anything I said to be an advantage of lighttpd and nginx can also be done with apache itself. It’s just a matter of configuring and compiling your apache for your special needs. The default configuration most linux distributions ship with is for the broad mass and not for the one being in the need of maintaining a high-load webserver.
Still I prefer lighttpd and nginx for proxying simply because I can’t sleep well with two different apache instances fiddling around ;)…
9 joe // Mar 8, 2008 at 3:28 pm
Andreas: Thanks for your comment. I couldn’t agree more. This is why I attempted to shy away from making a suggestion or conclusion on which is better. Each situation, work load and environment may have a use for any of these three web servers. Testing is the only true way to find out which will work best in a specific case.
10 Marco Fang // Mar 15, 2008 at 11:37 am
Joe, your test result simply helped me a lot and saved a lot of time for me not to build up another test lab to compare nginx and apache. I just want to know more about comparison between nginx and lighttpd but someone had done this before, but the report is a little bit old (at http://blog.kovyrin.net/2006/08/22/high-performance-rails-nginx-lighttpd-mongrel/), thanks all of you.
11 joe // Mar 15, 2008 at 12:06 pm
Thanks, I am glad it was helpful. I haven’t had a chance to do an in depth comparison between lighttpd and nginx like this one. I will say that I have liked working with nginx better than lighttpd (manageability, config files, config options and etc).
Leave a Comment