Not all speedtesters up to the job - still!
Virgin Media started off looking at the ability of speedtesters to accurately test its 50Mbps XXL service back in 2008. The results surprised it, and has led to the provider carrying out further checks to consider how well checkers handle 10Mbps, 20Mbps in addition to 50Mbps.
"We’re dismayed to find that for years, many speedtest and comparison sites have been misleading the public on ISP performance. Consumers rely on these sites for clear advice and honest results. The fact that many of them cannot accurately measure speeds of 20Mb, and some cannot even accurately measure speeds of 10Mb, demonstrates there is a clear need for more transparency. As ultrafast services such as 50Mb become more mainstream, this issue will need to be addressed quickly"Jon James, Director of Broadband at Virgin Media
To ensure the testing was as accurate as possible Virgin Media used its Next Generation platform for all the tests, this ensured that contention and capacity issues were not a problem. The report highlights the importance of also having a computer that is optimised for the broadband speeds you are able to receive, Vista automatically tunes itself, but for Windows XP you may have to manually optimise the RWIN value to get the best performance.
One aspect not covered in the report is the testing methodology used, there are some testers in the market where you can actually unplug your ethernet cable and plug it back in, and the result is the same as if you had left the cable plugged in. These testers appear to work by taking a peak reading, which if your service has a highly variable speed due to congestion or some other issue can give you a false impression of high speeds. Our tbbMeter tool is useful to allow you to monitor your speeds without wasting your allowance on constant speedtests and it will show how large downloads are varying in speed, as well as on speedtesters.
A table showing the sites tested and the results is below:
|speedtest.net (London)||Yes||Yes||Yes||Has been 50Mb ready since start of campaign, however have made improvements to testing algorithm and plan to introduce Northern server|
|speedtest.net (Maidenhead)||Yes||Yes||Yes||Hosting/Server change at Maidenhead site coupled with changes to algorithm have improved results.|
|thinkbroadband.com||Yes||Yes||Yes||Has been 50Mb ready since start of campaign|
|broadband-expert.co.uk||Yes||-||Yes||Site was unavailable during 20Mb testing, however use same technology as speedtest.net (by Ookla)|
|ispreview.co.uk||Yes||Yes||Yes||Now using thinkbroadband’s technology.|
|numion.com||Yes*||Yes||Yes*||*Inconsistent results on 10Mb and 50Mb|
|speedchecker.org||Yes*||Yes||No||*Unable to measure 50Mb, with inconsistent results during 10Mb tests|
|broadband.co.uk||Yes||No||No||Unable to measure over 10Mb accurately. Releasing monthly results using inaccurate data.|
|Yes||No||Yes*||*Inconsistent results on speeds above 10Mb. Currently working on improvements to test.|
|broadbandchoices.co.uk||No||No||No||Working on improvements to testing system.|
|broadbandgenie.co.uk||No||No||No||Working on improvements to testing system.|
|broadbandwatchdog.co.uk||No||No||No||Working on improvements to testing system.|