I look after a site that serves up a lot of static content, early on already I ran into issues with Apache coping on one machine, also my bandwidth at my main site is pretty expensive so I started farming off my static content to a number of machines hosted at other ISPs, typically paying around 100 pounds per machine per month, as long as I push out less than around 100Gb/month it’s a pretty good deal.
The only problem I have is I get tons of SYN_RECV connections on each of my machines, around 300 of them at any given time. Typically these indicate a lot of connections waiting to be served but the servers handle new requests immediately, there are no time spent waiting for IO on the servers, in fact the CPU’s are 98% idle always.
In an effort to try and resolve this (after much tuning of server sysctl’s) I asked Jaco if he’s seen it before and he suggested giving Lighttpd a try.
I installed it on one of my 3 static servers and ran it for a few days now and the results are encouraging. I still have 300 SYN_RECV’s but the machine is performing much faster than its siblings. On average before Lighttpd I was getting 30 requests/sec out of each of my machines now this one is doing around 50/sec, it is also pushing out about 30Kb/sec more than the other two. Comparing the Lighttpd machine to 2 apache machines on a graph shows it consistently out performs the others by about 20%.
| Uptime | 4 days 21 hours 19 min 20 s |
| Requests | 18 Mreq |
| Traffic | 20.77 Gbyte |
| Requests | 105 req/s |
| Traffic | 89.99 kbyte/s |
There has been some discussion about Lighttpd and Apache benchmarks, one Apache user has written a debunk of the benchmarks. This is linked too from the Lighttpd home page so might be worth investigating some more, I’ve done a lot of the typical things that Apache people recommend but they didn’t help much. I think I’ll try and tune my one Apache server to hit the same performance as the Lighttpd and see if it’s possible, for now though I’m quite happy with the results of a quick 30 minutes spent upgrading to Lighttpd.
