Skip Navigation

Ofcom release ISP breakdown on Code of Practice Compliance
Thursday 29 April 2010 02:43:31 by John Hunt

Ofcom have released the full ISP breakdown (PDF) of the mystery shopping they carried out to check compliance with the Broadband Speeds Voluntary Code of Practice (COPBB). The main finding from the initial report was that more accurate information of broadband speed estimates were needed and work toward this would be carried out by trying to standardise how service providers report this.

Ofcom have identified within the full data where statistically significant differences have occurred between ISP's. This is where the results are significantly higher or lower than other results and are deemed to be measured with high confidence, not down to chance. The breakdown covers the ISPs with enough samples which could be deemed accurate. These were BT, O2, Orange, Plusnet, Sky, TalkTalk, Virgin DSL and 'Other' providers.

Access line speeds provided via telephone contact with ISP
BT underperformed most other providers at giving access line speeds without being prompted by the caller (28% vs 43% average). Ofcom clarify the main reason for this through the data which indicates that some services providers ask for a MAC code or other information such as bank details early in the sales process. For BT, the MAC had been asked for in 37% of cases where the mystery shopper had to prompt for speeds (average 22%). O2 provided this data in 54% of cases without being prompted, but they did did only manage to do this in 80% of calls, slightly below other providers (average 85%).

Provision of an access line speed estimate via telephone shop
Sky varied compared to other providers in this aspect by giving a range (of greater than 1Mbps difference) of speeds in 28% of cases (the average being 10%). One reason given for providing a range is that ISP's believe that line checkers used are not necessarily accurate and a range would give a more realistic expectation of speed. Sky also gave the headline speed (the speed the product is marketed at) in 21% of cases (average 10%).

Access line speed provided via ISP website
Virgin DSL products seemed to perform worst in this category giving a headline speed in 35% of cases where the average was only 8%. They also failed to give an answer in 29%. Sky again gave a range for the speed in 21% (average 5%) and no value was given in 30% of cases (average 19%). Plusnet and TalkTalk failed to give a figure in only 12% of cases. Virgin also failed to identify that the speed was an estimate in 79% of calls (average 60%). This is particularly worrying where they are giving the headline speed of the product, which is rarely (if ever) actually achievable on a DSL based broadband service.

Consistency of access line speed provided via telephone shops and ISP website
The average where figures matched exactly was only 51%, but the best performing was O2 who met this in 66% of cases. Virgin DSL products had a 1Mbps difference in 48% of checks (below the average of 69%) whilst Sky's data only had an exact match in 25% of cases and averaged a 1Mbps difference in 57%.

Consistency of access line speed provided via telephone shops and BT Wholesale line checker estimate
BT provided the most consistency with the Wholesale checker with 39% of values given matching, more than double the average of 17%. In terms of ADSL2+ speeds matching those given by the BT Wholesale checker, BT again outperformed with 46% matching compared with the average of 17%. In 74% of cases O2 reported more than 1Mbps higher than the BT Wholesale checker.

The other area investigated is based on perceptions of the ISP staff and uses a scale of 1-5 where box 1 is a poor rating, and box 5 is a good rating. BT customer agents were deemed to be less knowledgeable scoring only 30% (average 43%) in the top two boxes where callers felt the staff were knowledgeable. Plusnet came top in perceptions of helpfulness with 60% scoring the top two boxes of 5 to indicate helpfulness (average 38%). Plusnet also came top in clearly explaining that actual speeds are likely to be lower than access speeds with 37% of callers indicating this in the top 2 boxes (average 27%).

The data is interesting in areas with the full breakdown showing that some ISPs have particular failings which will need to be addressed by these service providers. Many of the large ISPs averaged similarly in most categories. Smaller ISPs will be more likely to give a more personal experience and so may excel in being knowledgeable and helpful, but the data provided indicating speeds may not necessarily be more accurate than the larger providers.


Posted by WalterWillcox over 6 years ago
BT's line speed estimator is often inaccurate as it monitors the line's current performance. If a line suffers from noise, including if the client has poor internal wiring,** then a much lower estimate is produced than is possible in practice.

** Many users seem quite unaware of simple ways of improving their internal wiring.
Posted by herdwick over 6 years ago
Thanks to me for making the FoI request that resulted in OFCOM publishing the data :-)

Measuring current performance isn't inaccurate, WW, it shows what speed the customer can expect to get. Whether they can improve it is a different matter.
Posted by WalterWillcox over 6 years ago
@ Herdwick,

Agreed, but if you wish to change to e.g. O2 (who seem to have a rather strange marketing policy) and your line is under-performing (which I believe many are) the O2 web site will then tell you they do not have a service available in your area; however your "lucky" neighbour is accepted automatically. O2 do provide a phone number to call but that is one more hurdle for the disgruntled user to jump.
Posted by chrysalis over 6 years ago
seems a poor test to me, ofcom only care about if an estimate is given rather than improving actual speeds? mind you maybe I can ask them if it matters if VM didnt tell me my cable service can only download at 0.5mbit prior to ordering.
Posted by herdwick over 6 years ago
"ofcom only care about if an estimate is given" - in terms of the code of practice, yes. It's about informing customers of the service their about to buy.
Posted by CARPETBURN over 6 years ago
I was going to launch into a rant about ISPs, then i thought about it..... Why are ofcom moaning about speeds any ISP quote when they were the tools that allowed the use of that magical phrase "UP TO" in the first place?

Why should any ISP even have to give an estimate when these clowns have allowed them not to?
Posted by WalterWillcox over 6 years ago
A purchaser requires -

1. The peak-time contention of the ISP's network.

2. An estimate of the line's speed performance. Line length is often blamed for poor performance when the culprit is line quality.

This is difficult for a novice to understand. If a newcomer obtains the modem's statistics, & the attenuation is entered e.g. at

a better estimate will result. The tricky bit is to obtain a satisfactory conclusion, even with a co-operative ISP.

If industry & Ofcom would address these points we would all be more contented with our services.
Posted by tommy45 over 6 years ago
One key obstacle they failed completely to address is the fact is there is no way any one can tell what speed their line will support until adsl is enabled on that line , only then do you can any stats such as attenuation levels ,there should be a test that any potential isp could carry out prior to sign up that would tell them at least what the attenuation is then they could give better details
Posted by WalterWillcox over 6 years ago

Agreed !

You can use the BT linecheck but the results it produces are quite random, and rather bewildering for a novice.

One house I helped had one of its two lines "false greened" and the other no-dsl line told us we would achieve 6 Mbps. After four months of struggle it has an IP profile of 135, and even then drops out sometimes for days at a time.
Posted by greemble over 6 years ago
According to the BT checker, I was able to get 9Mb/s - signed up & actually get 2.2Mb/s. Two BT engineer visits later, no change. BT checker now shows I can get 2.5Mb/s

Seems Ofcom are encouraging the ISPs to make up figures. The ISPs that deliberately give a high estimate are likely to get more people signing up & then locked in for the next 12 months or more.

Maybe the ISPs should give estimates in writing and if the actual stable speed is less than 75%, then the customer should be able to leave, without penalty at any time

Cannot see Ofcom even considering such an idea, though
Posted by jakthelad over 6 years ago
Am I alone in finding this topic extremely opaque ? The general drift however sems to be that our ISPs are screwqing uds in some way. Is this something new ?
Posted by kijoma over 6 years ago
I see complaints about "Up to" , to be fair that is what should be used as ultimately many factors outside of an ISP's control affect end to end speed to a destination. for the ADSL guys its BT's network, your phone line, the weather, what the line is made of , where the sun lies with respect to saturn etc... We use "up to" to define the top speed you WILL get on our network. we do not have BT to deal with but there will always be the internet cloud itself to "contend" with.. Unlike ADSL though when we say "up to" you are capable of it.
Posted by tommy45 over 6 years ago
Well just for a giggle i ran the bt checker, it rambles about my line should support 2mb,stable, and currently supports 6mb max and adsl2+est of 8mb,lol i currently sync at over 18mb
You must be logged in to post comments. Click here to login.