Are the average speeds providers now listing correct?
We have looked at the average peak time speeds for the various broadband services a couple of times now and generally from month to month there is not much variation in our figures and while in the past our results have been slightly lower than Ofcom monitoring via its SamKnows system this is to be expected as we are not using dedicated hardware, but actual users devices so the impact of Wi-Fi, device performance and a myriad of other features affect the result, but the advantage is that end to end testing also reflects what the public see. In short our speed test analysis is a warts and all analysis.
We have now had to broadband providers claim the average peak time speed of their ADSL2+ service is 11 Mbps, entry level FTTC 34 or 35 Mbps and the top end FTTC service 57 or 63 Mbps, so how does this stack up with our numbers and the theory.
ADSL2+ and ADSL should be fairly easy to predict and the line length theory suggests that across the UK the median line length is around 3.3km (i.e. 50% are on shorter lines to the cabinet, 50% are on longer lines) and the models suggest this should give a download speed of 9 Mbps.
In practice for ADSL/ADSL2+ we see a median download speed of 6.2 Mbps and 33% of speed tests are above 9 Mbps and this is across all providers in Q1/2018 and is eliminating tests with a poor quality score, i.e. this will exclude most bad Wi-Fi and line faults (we call this our clean figure). Our reporting of a median value during the individual test will also account for some of the difference, a number of other testers use the mean or a something approaching the top quartile speed of the individual test. We do not split out ADSL/ADSL2+ in our analysis, but the medians do not change significantly when we look at LLU (ADSL2+ only) providers compared to ADSL and ADSL2+ providers.
Ofcom has the largest batch of SamKnows monitoring and for ADSL2+ at peak times their most recent report was 9.8 Mbps for ADSL2+ and 4.1 Mbps for ADSL. So what Post Office and Sky are saying is that they are outperforming the Ofcom average and thus it will be interesting to see what the other major providers say, and if they all out perform Ofcom there may be some questions.
Onto VDSL2 for which there are three variants, 40/10, 55/10 and 80/20 which up until now were advertised as up to 38 Mbps, up to 52 Mbps and up to 76 Mbps.
Sky has opted for a peak time average of 34 Mbps on the 40/10, with Post Office at 35 Mbps. At initial glance these figures look impossible since VDSL2 has a fairly quick speed drop off over distance, so what is going on?
The median distance to a street cabinet in the on the Openreach network is around 450m and with our standard model for VDSL2 reach this would give a download speed of 40 Mbps so the 34 and 35 Mbps figures used do look achievable. For FTTC 40/10 services we have varying medians from our analysis that run around the 25 to 29 Mbps from speed test analysis and ignoring product splits we get a clean FTTC median of 32.1 Mbps. Ofcom reports a peak time average for 40/10 services as 32.7 Mbps with a 24 hour average of 33.1 Mbps.
So why are providers quoting figures higher than expected? It is possible that that the spread of line lengths is different e.g. some providers will have very few people buying the top speed service even though they can go that fast but this may well shift their median speed a couple of Mbps and this highlights a concern we have voiced all along that careful customer selection may mean some see their provider choice restricted as those with slower VDSL2 estimates may be refused the advertised product and shifted to a 'secret menu' service or refused service at all.
For the 80/20 top speed VDSL2 service things are much more complicated as the selection process of paying more for higher upload speeds can skew the line length profile away from the expected model. The theory model suggests that to bother with the fastest VDSL2 service you want to be within 450 metres of the cabinet and the median in that range will be around 60 Mbps, but with the sharp drop speed vs distance graphs in this range small shifts in the maths and real world performance can produce very different results.
The spread of results from speed testing on the 80/20 service is much wider than other services at around 45 Mbps to 66 Mbps and while we have just two samples from providers of 57 and 63 Mbps this looks like it is going to be reflected in what they advertise. The Ofcom peak time average for the 80/20 services is 59.6 Mbps.
Where providers locate their monitoring hardware is critical, since we are tracking a consistent around 4 Mbps difference in median speeds between Huawei and ECI VDSL2 cabinets every month, so if provider wants to be clever it will use testers for VDSL2 on the faster Huawei cabinets.
So is there a conclusion? YES
Since for services based around the Openreach network are based on the same identical lines your connection speeds are going to be pretty much the same and the difference you see will be down to how their network performs and while the ASA/CAP move to peak time figures was meant to reflect that what we believe is going to happen is an arms race to be just 1 Mbps higher than competitors. So expect things like availability checkers being more selective, free speed upgrades for some even though remaining on the same product and many other marketing trickery.
The complications around different providers selecting or attracting different profiles of customers to different products makes comparing services for their peak time difference potentially misleading now. The other option as some seem to do now is not talk about product speed at all in advertising or on product pages.
The rule of ignoring the advertised speeds and instead look at the personal estimate during sign-up remains the golden rule and if others with the same provider on the same street as you have problems with video streaming at night then avoid the provider.
The collective excitement that average peak time speeds would expose the problems of line length issues for ADSL2+ and VDSL2 services look set to fade away.