fbpx

How to Effectively Measure Broadband Speeds?

There will always be claims of product/service superiority among competing companies and the one between Smart and Globe is a classic example. The latest one is the battle between mobile broadband services Globe Tattoo and Smart Bro.

I’m sure a lot of you have seen SmartBro’s latest commercial with Mo Twister in it. If you haven’t, check this YouTube clip:

In the commercial, Smart claims that 9 out of 10 times, SmartBro is faster than the competing brand. In this case it, was implying Globe Tattoo (the blurred footage of the Tattoo USB dongle is pretty obvious). In the trials made by Smart they used Speedtest.net as their benchmark tool of choice.

Not to be outdone, Globe Telecom wants to rebut the claim and says it isn’t so. So Globe commissioned NESIC to do a broadband test for them and showed the results to the media proving otherwise.

The methodology includes both an HTPP Speed Test using 2Wire.com as well as Direct HTTP Download from Download.com and an FTP Speed Test using a 12MB file from a local server. I am sure you know what the results of the study are and who came on top.

What is more interesting is the methodology of the benchmarking they’ve done. In my opinion, using SpeedTest.net is not a reliable way to test broadband speeds, despite it being a popular tool among users. My understanding is that Speedtest.net measures the burst speed of your connection and not the sustained speed or average speed. It means that during the test window, the tool will see variable speeds (that’s while the dial meter fluctuates) and will pick up the fastest it has detected. In some sense, it is a good way to measure the maximum achievable speed of an internet connection but that’s all it does.

Here’s an analogy — imagine a Javelin Thrower in a competition is allowed 3 attempts to get the javelin the farthest. His 3 attempts were 55 feet, 125 feet and 185 feet. If SpeedTest.net were the judge, it will score the athlete a 185 feet rating. If 2Wire.com were the judge, it will give the athlete just 121.67 feet (average of 3 attempts).

So why is Globe saying that Speedtest.net is not reliable? Because it does not take into account consistency. Let’s go back to the Javelin tournament — a second athlete throws with the following distances: 145 feet, 130 feet, 155 feet. If Speedtest.net were the judge, it will give the 2nd athlete a score of 155 feet while 2Wire.com would give it 143.3 feet.

Now, which of the two athletes is the better javelin thrower?

However, there could be a number of external factors that may affect speed test for mobile broadband:

  • Location of the Test Site. The distance of the cell tower from the test site can affect the performance of the benchmark as signal degrades over distance.
  • Signal Strength. This is self-explanatory. Factors include distance, elevation and barriers like buildings or thick walls.
  • Connection Mode. Connection mode (GPRS, EDGE, 3G, HSDPA) sometimes switches depending on which tower is servicing the connection.
  • Time of the Day. Networks could get congested depending on the time of the day when a lot of users are using the network.

Rather than using benchmark tools, I’d rather propose some real-world measurements:

  • Download a movie file using Bittorent. A 600MB file downloaded in about 4 hours will give you a speed of 44KB/s or 350Kbps. Longer download times will give us a better picture of the network’s consistency.
  • Streaming with YouTube HD. Streaming a 10-minute HD video on YouTube will also provide live bandwidth performance of the connection. This can be augmented with a YouTube upload of a 100MB file to see upstream speeds.
  • iTunes Download. What I used to do is download 3 video podcasts simultaneously on iTunes and record the incoming bandwidth using NetMeter. This will show sustained download speeds (like the one shown below).

Who has the biggest subscriber base? Who has the widest coverage? Who has the fastest mobile broadband service? Any one of them can claim these but ultimately it is the users’ commulative experience that will decide which is more reliable.

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 985 other subscribers
Avatar for Abe Olandres

Abe is the founder and Editor-in-Chief of YugaTech with over 20 years of experience in the technology industry. He is one of the pioneers of blogging in the country and considered by many as the Father of Tech Blogging in the Philippines. He is also a technology consultant, a tech columnist with several national publications, resource speaker and mentor/advisor to several start-up companies.

76 Responses

  1. Avatar for Vance Vance says:

    @sorry about the first post. I meant download a movie using torrent I meant.

  2. Avatar for Vance Vance says:

    I doubt that the downloading a movie file utorrent will be reliable source for data, remember it will depedent on how many seeders and peers..

    @abe, out of topic: are you going to CEL 09? this weekend.

  3. Avatar for Reel Advice Reel Advice says:

    Honestly, I have tried both Smart and Globe and both are huge wastes of time and money. 9 out of 10 times, I need to refresh a webpage for it to load properly albeit very slowly

    Anyway, have they even tried going against Sun? I think Sun is the fastest and most consistent here in the Metro – going outside Metro Mabnila is a different case though.

  4. Avatar for movie poster movie poster says:

    great move for smart bro but fail for 2 factor. speedtest.net and Mo Twister lol

  5. Avatar for Carl Carl says:

    Just in time when Globe is under proxy again. :/

  6. Avatar for Abe Olandres Abe Olandres says:

    @herbert – at the time of posting, there’s none for all 3 of them although Smart and Globe have been advertisers before with Smart being the biggest advertiser so far. However, there are also two upcoming ad deals with Smart and Globe.

  7. Avatar for Patrick Patrick says:

    I agree on the factors involved but most especially the location. This is why I still prefer wired broadband for my home connection. If I need wireless internet when I’m on-the-go, there’s bound to be a free WIFI hotspot somewhere nearby.

    And shame on that Smart Bro commercial! It’s misleading and sadly, I’m sure many non-techie consumers would be won over by it. <_<

  8. Avatar for jill jill says:

    Flatly, they’re both the same. Tooth and nails over this speed thing: Duh and Hisshh! I agree with response #3.

  9. Avatar for JohnLlot JohnLlot says:

    totally agree on the factors involve. ath the end of the day users experience will still matter.

  10. Avatar for Jeff Jeff says:

    Liked the post sir yuga. Ãœ it’s pretty obvious that SMART came out with this solely for advertising purposes, of course it’s part of the marketing strategy that Globe cannot accept. Reality bites, its a competetion. it really depends on each customer’s experience because nothing is constant.

  11. Avatar for francis francis says:

    The more subscribers, the more that are using the networks respectively, the slower the connection. Of course these telcos want to maximize their earnings by overcrowding their subscribers….. bahala na si batman basta may kita tayo. Don’t be fooled by their commercials, pare pareho lang sila. Typical inefficient utility providers

  12. Avatar for berkano berkano says:

    I’ve been using a Smart Bro canopy for over a year and i can the speeds have been consistent especially for torrent downloads and video streams.

    USB users will always have a different opinion since they are mobile and certainly the experience will depend on their location and the density of subscribers.

    Globe on the other hand..well I’ve used their USB modems before they were tattoed and they flat out SUCKED. Multiple areas, nada. Sorry I couldn’t have said more gently.

  13. Avatar for herbert herbert says:

    After missing cellphone credits, this is another techy thing that the Congress should investigate. They should clarify what “speed up to…” is all about. There should be a minimum speed that subscribers should have in any given package.

    Abe, suggestion though. Please disclose in this post any business linkage you have with Globe, Smart, or Sun.

  14. Avatar for ROEL_V ROEL_V says:

    have any one tried the crack USB modem, im using it now, if the area have strong smart signal, I use smart sim vice versa for globe . . . . . sorry for my english

  15. Avatar for Ric Ric says:

    People who get suckered by that ad will be in for a really big disappointment. Smart should be ashamed of themselves. The only time I can consider Smart Bro usable is during 2am – 7 am in the morning. Sheesh!

  16. Avatar for Jhay Jhay says:

    If it’s about their mobile internet speed, three words: location, location, location.

    3G signal is strong in malls and cities, while very spotty in the provinces for both networks. So the commercial by Smart? It’s just another PR stunt.

  17. Avatar for Paul Pisig Paul Pisig says:

    I had a friend who got an apartment unit somewhere in JP Rizal (Makati City). He’s currently using Globe Tattoo. He’s always upset because even if the UI of Globe Tattoo indicates my friend has HSDPA as the device’s signal, he’s experiencing a really SLOW connection. I tried to explain to him that internet speed is different from internet bandwidth. His Tattoo may show him the datacard is receiving a very strong signal but the question is the bandwidth available in his location (network congestion). And he’s been arguing with me about this analogy the whole night. Hahaha. Was I right or not? I hope somebody will clear this out for us. If I’m right, I’ll be very excited to show this thread to my friend. If not, oh well, I’ll admit to him that I’m wrong. :p

  18. Avatar for lolipown lolipown says:

    Another point that should be added when looking at broadband speed tests is coverage. Which network has better coverage AND network throughput overall?

    They can run speed tests all they want but what matters to everyone are:
    1.) Can they avail of the service (meaning connect to the network) with the same consistency that they’re touting?
    2.) Can the infrastructure handle all those connections? Or will we get to point 3..
    3.) Are we getting throttled? -> Probably going to raise red flags to the most active users :D

  19. Avatar for marvin marvin says:

    I agree… I used speedtest.net last few days measuring the if the upgrade on my PLDT DSL subscription is working…

    Seems like I have to test it again using other methods to get the real speed.

  20. Avatar for Brian_B Brian_B says:

    Good post. Hopes this gets the link love it deserves.

    Providers, take notice.

Leave a Reply
JOIN OUR TELEGRAM DISCUSSION

Your email address will not be published. Required fields are marked *