Lessons in Optimizing a Really Huge Blog

Up until now, I haven’t had any close experience doing server-side optimization of a highly-trafficked blog. When I say “highly-trafficked”, I mean it’s like being on the front page of Digg all day, 7 days a week.

I have had some experiences with vBulletin forums getting concurrent connections in the vicinity of 500 but a WordPress blog clocking in 1,000 concurrent connections is something else. Btw, I’m talking about Rickey’s blog on Americal Idol.

Whos Online
The blog peaks at 1,161 concurrent users during peak hours.

Here are some stuff I’ve learned so far and I hope will be of good use to other bloggers who reaches this level of traffic:

If you can afford it, get two separate dedicated servers to host Apache and mySQL. So we’ve got a Pentium 4 2.8GHz Dual Core with 3GB of RAM to run Apache and all static files and another Pentium 2.8GHz with 4GB of RAM to run mySQL. This set up seems better than my own Dual Xeon server with 4GB of RAM.
Upgrade to PHP5USD 0.09INR 7EUR 0.08CNY 0.62 and mySQL5 if you haven’t done so. Both newer versions have general performance benefits over the older ones.
Optimize server configurations. Most new owners of dedicated servers will have to face the challenge of having a good mix of settings in their php.ini, httpd.conf and my.cnf. Here is where you set connection limits, time outs, and memory caching/allocation. Making the most of your available server memory will determine whether your server will conk out or not when the moment strikes and the barrage of traffic comes in.
PHP Opcode Caching. There are several free ones available such as eAccelerator, Alternative PHP Cache (APC), and the XCache. These are basically frameworks for caching and optimizing PHP intermediate code and really helps if you run a lot of WP plugins.
For WordPress blogs, running WP Super Cache is highly recommended. You can also use PHP Speedy to compress and speed up page deliveries.
Monitor. Tweak. Optimize. Wash, rinse and repeat. You won’t hit it the first time and it’s gonna be frustrating. Just watch out for the peak hours, log the volume of traffic, watch their behavior (commenters vs. viewers) and tweak again based on that volume.

As for the cost, it depends and will vary between a hundred dollars to a couple hundred. Server management is the most expensive IMO as they charge as much as $75PHP 4,401INR 6,356EUR 71CNY 546 per hour just to take a look at your server.

And no, a 500GB + 5 terabyte $20PHP 1,174INR 1,695EUR 19CNY 146/month shared hosting will not cut it. They’ll kick you out before your blog can even reach 1TB.

Abe Olandres

Abe is the founder and Editor-in-Chief of YugaTech with over 20 years of experience in the technology industry. He is one of the pioneers of blogging in the country and considered by many as the Father of Tech Blogging in the Philippines. He is also a technology consultant, a tech columnist with several national publications, resource speaker and mentor/advisor to several start-up companies.

https://www.yugatech.com

23 Comments

ChrisMo says:

17 years ago

Thanks for the tips Abe, I was contemplating the same thing… Got a directory site before with huge traffic which 90% are bots. I didn’t receive any notice from a Phil. host, they just closed the account. So it is always good to have a dedicated hosting…
- Reply
LiNTEK says:

17 years ago

Nice server optimization tips. If you are “earning much” from your blogs then I would like to add the idea of co-locating your servers to a nice colo-company where you can get a bigger bandwidth (T1/E1/T3/DS3/ATM etc) at a lower price.
- Reply
Dennis says:

17 years ago

– Use a software/hardware Load Balancer.
– If your memory is ok, the bottleneck usually is MySQL. Do a MySQL replication to distribute the mysql load to multiple servers without compromising data integrity.
– Cache ,cache and cache. Opcode caching is ok. Memcache would be best if you set up a clustered server.
- Reply
Jonathan Concepcion says:

17 years ago

Very nice tip! I agree to Lintek for his comment. I know there are few hosting company that host highly traffic website like media temple. In hostmonster they host 15terabyte of hosting space and transfer.

By the way nice mobile website this post via mobile device.

Keep up the good post abe!
- Reply
Christian says:

17 years ago

Rickey is the man
- Reply
Alex says:

17 years ago

I use rdns between 4 quad xeons for mp3-codes.com. consuming about 200TB’s of bandwidth/month.
- Reply
GM Tristan says:

17 years ago

Ey Abe, am clocking in at around 3K per day now. Got a bandwith limit error already on my host. Got suggestions on which hosting plan has most bang for the buck?

Thanks
- Reply
skiper says:

17 years ago

For people who cant afford a dedicated server you can try a host that uses a grid server technology or a something like split shared technology, actually im planning to move to a split shared host but im still reviewing this host its $25/month compared to my current host which is $9/month so theres a big difference.
- Reply
Abe Olandres says:

17 years ago

@GMTristan – have you tried doing some caching for your WP like the SuperCache? Or if you hosta lot of media files, you may want to remote host most of them to save up on bandwidth. You may want to look into MediaTemple later on though.
- Reply
skiper says:

17 years ago

yuga/gmtristan – check webhostingtalk.com before getting your host, check for their review, there are threads about media temple thats why I didnt get a host from media temple yet, im stil checking their reviews.
- Reply
GM Tristan says:

17 years ago

Thanks for the tips, guys. Will check them out and give you updates.

Much appreciated. More power to your blogs and sites!

Peace
- Reply
Connie says:

17 years ago

Get a dedicated server, o, di ba Yuga? I love mine hehehehhe
- Reply
Carlo says:

17 years ago

thanks for the post. i hope to read some more.
Best regards from Sebbi
- Reply
franz says:

16 years ago

the idea of separating apache and mysql into two different server seems to be not that good when it comes to server security…
- Reply
How to Get Six Pack Fast says:

16 years ago

I follow your posts for a long time and must tell you that your articles are always valuable to readers.
- Reply
Rai Ciego says:

16 years ago

Dedicated Server/ Shared hosting is OK!. But for a student like me..It’s hard for me to order those hosting. Coz I have no credit card.. Good thing I found a hosting and domain that accepts gcash as payment.
- Reply
Tiyo Paeng says:

15 years ago

Is There super cache also for blogger or blogspot?
- Reply
Jhunz says:

15 years ago

What should be the best OS for that server.

I wondering if Centos OS will be good for such a dedicated server.

Actually I have post some Ideas in this site http://www.ctrzone.com forums.
- Reply
Movementsame says:

15 years ago

Detail Consumer,colleague action normally previous discuss touch victim share path suffer more state put wine set broad internal object your assessment word imply therefore double project heavy display signal spend mean trip past sentence health king advantage indeed gun possibly still route need large vast chair traditional brother important row pretty energy product carefully annual reality capable buy join appropriate ticket technique divide flight then live fail difference lot as coffee independent deliver win clear become definition supply wash close existing hear instruction people acid long key island lip grant knowledge ready indicate computer seem detail
- Reply
Asndrew Pelst says:

15 years ago

When looking for top hosting -I thought all the money made from internet are scams. Now, i know and im enlightened.
- Reply
Jhunz says:

15 years ago

Good to know that high traffic blogging needs you to have a server optimization. Actually server optimization is kinda hard thing to do. But if you have an experience doing some linux server installation and configuration it will be fine..
- Reply
Jhunz says:

15 years ago

For those who want a blog site. Please let me know. If you want a domain and a hosting I can help you with this with or w/out credit card. Php2500 annual for a blog site is not to much..
- Reply
John Alvero says:

14 years ago

This post is old already, to keep it current and keep up with the latest, Clouding Computing technology such as the one offered by Amazon EC2 is worth looking at. It may not be the cheapest but some might go this way.

This way, it is very easy to scale-up or scale-out your infrastructure. Just choose what type of server instance depending on your memory and cpu needs. You can also choose small instances and and add more small instances during high traffic — the best part is you can even script this process. Here you pay by the hour, so during low traffic you can terminate some of the services instances. Cool huh.

Oh btw, did I mention that we already have an Amazon Asia Pacific? It’s in Singapore and just a few hops away from most Philippine based ISPs.

Do I sound like a salesman now?
- Reply