Skip Navigation


Packet loss problems continue for users of Be and O2 networks
Wednesday 15 February 2012 13:10:47 by Andrew Ferguson

Customers of O2 and Be have been having a difficult few days, with packet loss impacting on many services, and while web browsing can generally function with packet loss present, online gaming becomes almost impossible, protocols such as POP3 and SMTP are usually more tolerant to packet loss.

Customers are becoming increasingly frustrated, as it seems multiple fixes to the problem have been attempted, but as with a status update today, fixes are not always working.

Extract from Be Wholesale status page
Click image for larger version

The situation is also one that is not common to all destinations on the internet, or all users of the services. This has resulted in Be support staff having to get customers to collate ping and tracert results to gain an understanding of where the problem lies. Our own Broadband Quality Monitor provides some insight into the issue.

BQM data for four sample O2 and Be connections
Click image for larger version
Graphs from other providers
Click image for larger version

The graphs show the variability of the experience users will receive, with almost no packet loss (red area on graphs) for some, and significant levels for others. The comb tooth effect with regularly spaced spikes is thought to be an effect due to how the standard Be modems handle ICMP traffic. The fact that on the badly affected connection the packet loss was still present at off-peak times, and the connection did not appear to be busy with downloads/uploads suggests a network problem. The three BQM graphs on the right show what you should normally see on a connection which is not suffering packet loss.

A more traditional trace route shows the problem clearly too. With the * representing a dropped packet, the non-repsonse from the final server is most likely down to guardian.co.uk not responding to ICMP requests at all, this is an unfriendly but common practice.

1 <1 ms <1 ms <1 ms 192.168.1.254
2 * * * Request timed out.
3 8 ms 22 ms 8 ms 10.1.3.178
4 * * 12 ms Xe7-1-0-0-grtlontl3.red.telefonica-wholesale.net .7.16.84.in-addr.arpa [84.16.7.241]
5 * 72 ms * Xe2-0-0-0-grtmadno1.red.telefonica-wholesale.net [213.140.49.181]
6 41 ms 44 ms 41 ms Level3-1-0-0-grtmadno1.red.telefonica-wholesale.net [213.140.55.178]
7 * 42 ms * ae-0-11.bar1.Madrid2.Level3.net [4.69.141.45]
8 42 ms 50 ms 49 ms ae-5-5.ebr1.Paris1.Level3.net [4.69.141.42]
9 41 ms 42 ms * ae-45-45.ebr1.London1.Level3.net [4.69.143.101]
10 53 ms * * ae-58-113.csw1.London1.Level3.net [4.69.153.122]
11 54 ms 48 ms 110 ms ae-15-51.car5.London1.Level3.net [4.69.139.70]
12 209 ms * 50 ms GUARDIAN-UN.car5.London1.Level3.net [217.163.45.90]
13 * 54 ms * 77.91.255.142
14 59 ms 53 ms 76 ms 77.91.255.146
15 63 ms 53 ms * 77.91.255.202
16 * * * Request timed out.
17 * * * Request timed out.

A new network was announced in 2011, with it being built in 2012, alas it seems the new network cannot come quickly enough. Be has over the years since it started as one of the first ADSL2+ providers in the UK has built a loyal following, and while many accept the occasional evening of disruption, when problems go on for a period of time people may start voting with their wallet and move to other providers. The assumption at this time is that this is not just an issue with the volumes of traffic, but rather kit actually failing, since if it was traffic volumes then emergency measures such as traffic management could reduce packet loss to an acceptable level.

These problems also show the level of investment that is required to keep a network running smoothly 24/7, there is one large LLU provider that provides redundancy and capacity to the level that a major node can vanish, and the remaining network absorb the traffic without increasing contention significantly. This does not rule out congestion issues at a local exchange, but does help to avoid widespread issues that Be/O2 appear to have currently.

We will keep an eye on our Broadband Quality Monitoring system, as once a final fix is in place, the reduction in packet loss should be obvious.

Comments

Posted by audioslim over 5 years ago
Cant say I've noticied anything on my O2 line recently, but havent done any gaming last few days.

Slim
Posted by MrTAToad2 over 5 years ago
It might have been better stating that it doesn't affect everyone in the first sentence - currently it reads as though it affects everyone, and then you find out it doesn't after going through a couple of sentences and looking at a picture...

I, for one, am not affected - Be here is just as good as it was last year...
Posted by andrew (Favicon staff member) over 5 years ago
But this way I made you read it :-)

Low levels of packet loss should not really affect HTTP, so even those who don't see anything might be affected.

Once spent five days with a 90% packet loss (line card fault) line, and could still do some basic browsing.
Posted by simondup over 5 years ago
This might explain why BBC Iplayer has been nearly impossible to use at peak times recently.
Posted by andywhy over 5 years ago
I don't appear to be affected that badly. I run a server at home hence the pings being all over the place but there is little to no packet loss.

http://www.thinkbroadband.com/ping/share/360f3efce5fa352fd69a288011ba69cb-15-02-2012.html
Posted by tommy45 over 5 years ago
For some of us this has been ongoing since Friday 3rd feb 12,for me it's mainly been the whole of the BBC news,weather,sport, ect and Iplayer, all loaded really slowly,sometimes they would fail ,as would the loading of I player content, and even then the stream would frequently pause or display "insufficient available bandwidth" messages problems started at around 8 A.M thought till 11:30p.m.12:30am everyday
Posted by tommy45 over 5 years ago
This could be due to those links being overloaded,if it is that it's not completely fixed
Posted by groobs over 5 years ago
I'm in the same position as Tommy45, web access has been very flaky and intermittent for a couple of weeks.
Posted by stanman24 over 5 years ago
Xe2-0-0-0-grtlontl3.red.telefonica-wholesale.net.7.16.84.in-addr.arpa

is spiking to 180ms

here is the ink to my monitor

http://www.thinkbroadband.com/ping/share/a3149e7e03aae6cf7aef94960409efc3.html
Posted by Scubaholic over 5 years ago
Problems started on 4th. Mainly bbc inc iPlayer. After 2 days improvement iPlayer is borked again. Ping to BB has 96% loss & tracert times out on las 20 odd legs.
No indication fro Be what the actual problem is!
Posted by ionic over 5 years ago
I have several lines across the country with A&A over Be wholesale and I'm not seeing any of these issues on them. Further A&A don't seem to be reporting any issues on their status pages.
It looks like this isn't necessarily a problem with the carrier part of the network, more to do with O2/Be transit / peering?
Posted by darrenb73 over 5 years ago
First started having problems last Thursday webpages really slow to load i really dont know what the problem is but it needs sorting out, im just dissapointed with the way Be has communicated this with its Customers. Been telling Customers the problem was fixed when it wasnt. I understand Some people arnt affected by this but for the People that are its so frustrating.
Posted by tommy45 over 5 years ago
Peering/ routing are supplied by the wholesale isp not BE , also some isp's who supply the be wholesale service also use their own backhaul too, so none of the issue that are effecting be'o2 retail will affect the wholesale side

here is an example of problems to bbc sites http://oi43.tinypic.com/2afeatg.jpg
Posted by otester over 5 years ago
@andrew

So now it's all about the traffic not content?
Posted by andrew (Favicon staff member) over 5 years ago
@otester

Well packet loss can affect all traffic can it not. If it were just a HTTP proxy issue (web traffic) then HTTPS should not be affected, and ICMP should not be affected.
Posted by MrTAToad2 over 5 years ago
I've been playing a lot online (mainly the bestest online FPS - Delta Force Xtreme) and as that has been fine, I do think this area hasn't been affected...
Posted by blueacid over 5 years ago
OK, I have no idea what's happened to the routing from MRARD to www.guardian.co.uk - seems to go via Madrid, Paris then finally London.

http://pastebin.com/rm2Frn87
Posted by freder over 5 years ago
I fetch Usenet from News.Individual.Net. That's been badly disrupted since Saturday, being able to fetch some news but mostly timing out. Fetching mail from Demon through O2 works but is slow and sometimes times out. Web browsing is possible but frustrating.
Posted by otester over 5 years ago
@andrew

What I meant is the title should have said "some" Be/O2 users.
Posted by nredwood over 5 years ago
@blueacid

That has been an ongoing issue for some time and I believe unrelated
Posted by nredwood over 5 years ago
Some workarounds here
http://beusergroup.co.uk/?id=825
Posted by simondup over 5 years ago
still looks unhappy

http://www.thinkbroadband.com/ping/share/e7a9ee36b7496c41f675dbf180d7a7fe-16-02-2012.html
Posted by rachann over 5 years ago
Must say I have been looking at other ISP's, my speed seems to be getting slower.

http://www.thinkbroadband.com/ping/share/490cea9a9daf94614888b068442f8905-16-02-2012.html
Posted by drummerjohn over 5 years ago
My BE connection is fine - I do lots of online gaming. Check out how my BE compares to my crappy Virgin Media...
BE...
http://www.thinkbroadband.com/ping/share/4a0afa092840451bf43b535bafc412db.html
VM...
http://www.thinkbroadband.com/ping/share/de3811d21d78ed71e24862e8b7d54924.html
Posted by nredwood over 5 years ago
Reports coming in that packet loss has subsided after a few seconds loss of connectivity to the gateway

Perhaps someone has finally idenitifed the dodgy VLANs / tunnels and reconfigured
Posted by freder over 5 years ago
Don't want to speak too soon but my impression is that my access is back to normal. No timeouts with mail, usenet or slow web.
Posted by Scubaholic over 5 years ago
I saw some improvement yesterday but despite a message from Be this a.m. saying issue fixed still getting 96% ping failure, tracert timing out & can't access SD programmes via iPlayer.
I have a static iP.
Posted by scopio over 5 years ago
Would this problem be related to the trouble I and my friend are having with o2 download speeds?
My download speed with o2 has dramatically reduced from around 14MBps to 6, 7,8 1MBps! It is surprising because I only live half a mile from the exchange and have a relative powerful pc.Would be interesting if anyone else is having similar experiences. Please post them.

My friend is with o2 aswell, leaves in near by Worcester is even worse! She can hardly get enough download speed to log on!
Posted by David-Park over 5 years ago
Well this is the first I have heard of Be having problems, use my internet most of the day and all evening and nothing as been different for me, same excellent service from Be Unlimited.
Posted by nailersofbelper over 5 years ago
I've been having problems like this for a about 3 weeks now, and after raising a ticket with Be Support, they asked me to run a variety of ping and traceroute tests. This is the reply I've now had from Be:

We have reviewed your attachments and as you can see the issue is only appearing while you are trying to reach bbc.co.uk.

... cont
Posted by nailersofbelper over 5 years ago
Please let us inform you that the support team of the BBC web site has confirmed that they are currently making major network
changes on their end which appear to be the cause of the intermittent
packet loss for some customers. Also there are similar complaints from other ISPs.

What we can assure you that the issue is not related with our service and everything is happening when the internet packets are leaving our network.

Is Thinkbroadband aware of these problems at the BBC?
Posted by r1m0 over 5 years ago
This issue is still affecting me, to any website.

Tracing route to www.l.google.com []
over a maximum of 30 hops:

1 68 ms 99 ms 99 ms O2wirelessbox.lan []
2 * * * Request timed out.
3 18 ms 16 ms 15 ms 10.1.3.178
4 * * 16 ms 10.1.2.158
5 * * * Request timed out.
6 157 ms 36 ms 17 ms 209.85.240.61
7 16 ms 15 ms 15 ms 209.85.253.90
8 25 ms 24 ms 27 ms 66.249.95.173
9 22 ms 21 ms 21 ms 72.14.236.191
..cont
Posted by r1m0 over 5 years ago
I'm an O2 customer routing via BE equipment.
O2 are happy to keep running me through the diagnostic sheet that the support engs all have and are reluctant to even raise a case.They even said they don't support packet loss and as long as I'm shown as connected, they don't see what else they can do.
You must be logged in to post comments. Click here to login.