Skip to content
November 13, 2009

How to Effectively Measure Broadband Speeds?

There will always be claims of product/service superiority among competing companies and the one between Smart and Globe is a classic example. The latest one is the battle between mobile broadband services Globe Tattoo and Smart Bro.

I’m sure a lot of you have seen SmartBro’s latest commercial with Mo Twister in it. If you haven’t, check this YouTube clip:

In the commercial, Smart claims that 9 out of 10 times, SmartBro is faster than the competing brand. In this case it, was implying Globe Tattoo (the blurred footage of the Tattoo USB dongle is pretty obvious). In the trials made by Smart they used Speedtest.net as their benchmark tool of choice.

Not to be outdone, Globe Telecom wants to rebut the claim and says it isn’t so. So Globe commissioned NESIC to do a broadband test for them and showed the results to the media proving otherwise.

The methodology includes both an HTPP Speed Test using 2Wire.com as well as Direct HTTP Download from Download.com and an FTP Speed Test using a 12MB file from a local server. I am sure you know what the results of the study are and who came on top.

What is more interesting is the methodology of the benchmarking they’ve done. In my opinion, using SpeedTest.net is not a reliable way to test broadband speeds, despite it being a popular tool among users. My understanding is that Speedtest.net measures the burst speed of your connection and not the sustained speed or average speed. It means that during the test window, the tool will see variable speeds (that’s while the dial meter fluctuates) and will pick up the fastest it has detected. In some sense, it is a good way to measure the maximum achievable speed of an internet connection but that’s all it does.

Here’s an analogy — imagine a Javelin Thrower in a competition is allowed 3 attempts to get the javelin the farthest. His 3 attempts were 55 feet, 125 feet and 185 feet. If SpeedTest.net were the judge, it will score the athlete a 185 feet rating. If 2Wire.com were the judge, it will give the athlete just 121.67 feet (average of 3 attempts).

So why is Globe saying that Speedtest.net is not reliable? Because it does not take into account consistency. Let’s go back to the Javelin tournament — a second athlete throws with the following distances: 145 feet, 130 feet, 155 feet. If Speedtest.net were the judge, it will give the 2nd athlete a score of 155 feet while 2Wire.com would give it 143.3 feet.

Now, which of the two athletes is the better javelin thrower?

However, there could be a number of external factors that may affect speed test for mobile broadband:

  • Location of the Test Site. The distance of the cell tower from the test site can affect the performance of the benchmark as signal degrades over distance.
  • Signal Strength. This is self-explanatory. Factors include distance, elevation and barriers like buildings or thick walls.
  • Connection Mode. Connection mode (GPRS, EDGE, 3G, HSDPA) sometimes switches depending on which tower is servicing the connection.
  • Time of the Day. Networks could get congested depending on the time of the day when a lot of users are using the network.

Rather than using benchmark tools, I’d rather propose some real-world measurements:

  • Download a movie file using Bittorent. A 600MB file downloaded in about 4 hours will give you a speed of 44KB/s or 350Kbps. Longer download times will give us a better picture of the network’s consistency.
  • Streaming with YouTube HD. Streaming a 10-minute HD video on YouTube will also provide live bandwidth performance of the connection. This can be augmented with a YouTube upload of a 100MB file to see upstream speeds.
  • iTunes Download. What I used to do is download 3 video podcasts simultaneously on iTunes and record the incoming bandwidth using NetMeter. This will show sustained download speeds (like the one shown below).

Who has the biggest subscriber base? Who has the widest coverage? Who has the fastest mobile broadband service? Any one of them can claim these but ultimately it is the users’ commulative experience that will decide which is more reliable.


76 Responses to “How to Effectively Measure Broadband Speeds?”

  1. The Teknisyan
    Twitter: Teknisyan
    says:

    very well explained… plus the commercial of smartbro… still has a lot of factor to consider… like signal and location.

    Anyhow… thanks for a very informative post!!!

  2. simplynice93
    Twitter: simplynice93
    says:

    I have working as a network and computer technician for quite some time now. Let me advice you all about the truth in broadband internet here in the Philippines.
    When ISPs (Smart, PLDT, Globe, etc.) offer up to 2Mbps or 5 Mbps, they are actually telling you is the PIR (Peak Information Rate). But what you really need to know is the CIR (Committed Information Rate). What is the CIR? It is the average bandwidth guaranteed by your ISP under “normal” conditions. “Normal” means peak hours internet speed (usually daytime). Ask your ISP about your CIR speed. Magugulat kayo lahat! It’s usually 128kbps (or 258kbps) if you choose the basic plan. Sobrang bagal! Hope that helps! :-)

  3. biggie9385 says:

    i really like the congress to investigate this “up to #mbps” style of the ISPs. This is another big scam in the level of the lagacy banks. Hehehehe! I noticed that everytime i call Smart hotline, they will try to have me test using speedtest.net. But last time, I let them have their choice. I tested my speed thru speedtest.net connected to QC server and it really reported a very slow connection. The next thing the tech support guys (not the robot script reading support) asked me to do was test using the Singapore server. I was shocked because, QC is nearer to my place than Singapore. :D Luckily, very slow pa rin. :D Right now, I can ask Smart to call me the next day when I have a slow connection. And they really call me. Hehehehe!

  4. llewon17 says:

    Agree to the fact that Speedtest.net gets the maximum bandwidth obtained. BUT USING 2WIRE.COM OTHERWISE???? I DON’T THINK SO… 2Wire.com is much more not reliable in checking for speedtests. Our company has a 12Mbps speed yet 2wire measured it @ 25Mbps. Is that what you called reliable as the article suggests??? THINK AGAIN, MEN!!!

  5. Neil says:

    I think the better way to test the speed of the connection is to do an actual test on both smartbro and globetatoo and see which one is faster and better

  6. Ernie says:

    I got dizzy with all the comments regarding broadband speed. What I did gather from all those comments is the importance of location. So eto na lng, I live in Project 8, QC. Has anyone tried all three telcos (Sun, Smart, Globe)in this area? Ano pinakamabilis? I’m planning to get Sun, but I want to be sure. Can anyone help me out?

  7. Cmdr73 says:

    para makakuha ako ng pinaka accurate bandwidth test gumagamit ako ng bandwidth monitoring + by using the testmy.net set at 50mb download test then compare the result of bandwidth monitoring and testmy.net that they are too close almost 95% accurate i think

  8. Kyle
    Twitter: technokyletwit
    says:

    Iba kac ang Broadband Speed sa Transfer Rate or download speed. I’ve read a comment before na 200-250kbps lang something. I think may point dun ung sun cell kac most users nagsasabi or iniisip na kung 2mbps ang broadband speed eh 2mbps din ung speed na makikita pag nag dodownload sila. Which is wrong, Transfer rate po un hndi un ung broadband speed. Hindi ko lang talga alam kung bakit ba talga 2mbps ang pinapakita tapos ang transfer rate eh hndi naman tugma sa broadband speed,napaka misleadng.Regarding sa post na to, agree ako sa sinsabi ni sir yuga. hndi talga reliable ang speedtest.net when it comes to getting the average speed of an internet connection

  9. v3xat10n says:

    in my case, smart is the fastest due to strong signal, but i hate smart, mag nanakaw kasi ng load… how i wish na malakas signal ng sun dito

  10. SpeedDemon says:

    Sana Eastern Communications would come up with a USB broadband access too. You will never go wrong with them.

  11. Jake says:

    im using Smart Bro prepaid..im very satisfied! mabilis sya..i can download 800mb torrent movies in 4 hours..sulit tlga..mabilis din s INTERNET BROWSING..

  12. john says:

    If you need real bandwidth test. Try iperf. It’s is the usual tool in the wireless world for bandwidth testing. You have an option to send tcp or udp, a single stream of data, or multiple stream (think torrent).

    The catch? both ends of the link must be running iperf.

  13. Jmoi says:

    SMART user here. cool ang connection sken. way better than Globe. :P

  14. terra says:

    what would i say? GO SMARTBRO. im so so satisfied with their service.:p

  15. karren says:

    same here. good flow of connection with smartbro.:)

  16. terra says:

    yeh yeh.:)

  17. mykel says:

    choos d les poisonous frm d 3 known telcos. choos telco with less subscribers & ul get hi-spid. if ur on mobile, just take d risk, expect worst. u can hir frm dem but u wil not always get wat u want, inconsistent. anyway it wil be al d same bananas, all poisons, but different kind.
    ;-)

  18. rhye 18 says:

    in my own opinion, i dont agree with smartbro is saying that they have the fastest broadband against other competitors… i have a cousin that switched over from smart bro to suncell and finally stayed with globe tatoo. coz, one rainy morning ( not that heavy rain ) smart and suncell has the same result, slow connection, and also they have a huge pine tree right next to their backyard and sometimes they say that it is the pine tree that is blocking the signal but when she started using the globe tatoo, even that big pine tree has no match because she can connect with no hassle, she is also downloading a song that time… and why is smart saying that they are the top choice by some computer brands and yet there is no ad that those computer brands are choosing smartbro is their top choice and not the other competitors? i bought an asus laptop with a broadband as a part of the package. i saw in the advertisement that asus is one of the brands that uses smartbro but they gave me globe broadband? so can smart explain these scenario? so if smart is saying that they has the widest coverage, well better think twice…

  19. Glenn says:

    I tried using Sun broadband (prepaid) here in Albay with 1 day unlimited. It works! but what I got is EDGE signal :( it is worst than GPRS 15-30 kbps and is intermittent. I have to reconnect after 2-3 minutes because the connection speed dips to 0 kbps. SUn is truly useless here in the province. I’ll switch back when 3G / HSDPA signal is already available meantime SMARTBro would still be the best option.

  20. magoo says:

    waaaah….im here at venezia in subic….lakas ng WCDNA signal ng globe tatoo…..pero sobrang bagal……..nelp globe….fastest to drain my load…waaah.goota go..baka maubos losd ko…

  21. nalcolm says:

    actually wala yan sa speedtest. nasa ping yan. kung pldt ka type mu sa run http://www.yahoo.com -t ang regular ms mu dapat is 220ms to 250ms. Kung more than that may problem si pldt.

  22. Albert says:

    Globe Tattoo or Smart Bro? I think its better to resort to a wired internet connection if you wanted consistent speed. Based on experience, these things are useful only on e-mails, not on videos.

  23. jaesiz says:

    connected ako sa postpaid ng globe, ung 999 maayos naman ang speed sa una, kaya lang habang tumatagal, nagiiba ang speed depende sa oras, sa gabi na lang siya mabilis. mga 700 mb na lang pag mid day… kung technical problem un, tagal naman na ayusin.. 5 mbps ang speed na allocated para dito… paano ko po kaya gawing consistent ang internet speed kahit peak or off peak hours?,.. any idea there please?

Leave a Reply

*
*