Lessons in Optimizing a Really Huge Blog

Up until now, I haven’t had any close experience doing server-side optimization of a highly-trafficked blog. When I say “highly-trafficked”, I mean it’s like being on the front page of Digg all day, 7 days a week.

I have had some experiences with vBulletin forums getting concurrent connections in the vicinity of 500 but a WordPress blog clocking in 1,000 concurrent connections is something else. Btw, I’m talking about Rickey’s blog on Americal Idol.

The blog peaks at 1,161 concurrent users during peak hours.

Here are some stuff I’ve learned so far and I hope will be of good use to other bloggers who reaches this level of traffic:

• If you can afford it, get two separate dedicated servers to host Apache and mySQL. So we’ve got a Pentium 4 2.8GHz Dual Core with 3GB of RAM to run Apache and all static files and another Pentium 2.8GHz with 4GB of RAM to run mySQL. This set up seems better than my own Dual Xeon server with 4GB of RAM.
• Upgrade to PHP5USD 0.09INR 7EUR 0.08CNY 0.62 and mySQL5 if you haven’t done so. Both newer versions have general performance benefits over the older ones.
• Optimize server configurations. Most new owners of dedicated servers will have to face the challenge of having a good mix of settings in their php.ini, httpd.conf and my.cnf. Here is where you set connection limits, time outs, and memory caching/allocation. Making the most of your available server memory will determine whether your server will conk out or not when the moment strikes and the barrage of traffic comes in.
• PHP Opcode Caching. There are several free ones available such as eAccelerator, Alternative PHP Cache (APC), and the XCache. These are basically frameworks for caching and optimizing PHP intermediate code and really helps if you run a lot of WP plugins.
• For WordPress blogs, running WP Super Cache is highly recommended. You can also use PHP Speedy to compress and speed up page deliveries.
• Monitor. Tweak. Optimize. Wash, rinse and repeat. You won’t hit it the first time and it’s gonna be frustrating. Just watch out for the peak hours, log the volume of traffic, watch their behavior (commenters vs. viewers) and tweak again based on that volume.

As for the cost, it depends and will vary between a hundred dollars to a couple hundred. Server management is the most expensive IMO as they charge as much as $75PHP 4,401INR 6,356EUR 71CNY 546 per hour just to take a look at your server.

And no, a 500GB + 5 terabyte $20PHP 1,174INR 1,695EUR 19CNY 146/month shared hosting will not cut it. They’ll kick you out before your blog can even reach 1TB.