Skip Navigation


Major cable cut in East London tunnel
Sunday 05 April 2009 14:28:01 by Sebastien Lahtinen

Over the last 24 hours, a serious incident has taken place in East London (near Dartford) where a service tunnel containing communications cables has been damaged by a contractor, forcing many broadband users in East London to remember what it was like to browse the Internet before broadband was available. The tunnel in question is over 30 metres below ground and due to the extensive structural damage, the repair process is likely to be quite lengthy.

It appears to be affecting a number of different services, including BT Wholesale's DSL platform. BT have indicated to service providers that in order to fix the issue, they will need to bring the cables to the surface and route them through new ducting, a process that is likely to take days, although BT doesn't appear to be providing more precise estimates.

In the mean time, Openreach have managed to work around some of the problems to get more users online, although we are seeing reports of several problems including high latency and slow speeds indicating that these are running over very congested links. Still, slow Internet is better than no Internet. At least you can Twitter about it ;)

07/04/09 Update from BT: "Service has been restored to a significant number of customers, including emergency services, all 75,000 telephony lines impacted, as well as a number of business and other communications provider circuits using alternative methods of connection. Restoring some services is a complex engineering task, but BT engineering teams are working around the clock to restore service to all remaining customers without service as soon as possible. [..] Given the complexity of the incident, it is not yet possible to accurately predict when all services will be restored. [..] BT will issue further updates as the situation changes." - Customers wishing to get updated information can call 0800 169 0199 to hear the latest.

Comments

Posted by cyberdoyle over 5 years ago
I sincerely hope they get it fixed soon, and I sympathise with the many people who will suffer and the business lost, but on the other hand I wish the houses of parliament are on that pipe and that they have a full week of poor access to t'internet and then they may just realise what many of the people in the rest of the country have to contend with. It might bring new perception into the final Carter report.
Every cloud has a silver lining. somewhere.
Posted by Dixinormous over 5 years ago
This is a bit of a bummer.

A Tsar Bomba size nuclear explosion wouldn't bring new perception into the final Carter report, only a total replacement of the politicians and industry whores with vested interests who compose it with people who are not interested in political expediency, not interested in commercial expediency and have some clue.
Posted by phoenixseven over 5 years ago
A source of mine from BT Wholesale says over 70,000 lines have been affected, and they're not expecting it to be fixed before Tuesday.
Posted by _TRIaXOR_ over 5 years ago
What is it with Contactors and communications cabling in London? sheesh..
Posted by Spectre_01 over 5 years ago
Contractors... you get what you pay for
Posted by rmillwood over 5 years ago
What happened to the idea of a 'network'? Seems to me its always a 'tree'. Surely the original ARPANET idea was to allow for links to be knocked out and data would just route around the failed link - how often do you see that design in practice?
Posted by amonra over 5 years ago
I always thought BT had a "figure of eight" network on all the main routes so that this sort of thing couldn't happen ? More economies methinks, "it cant possibly happen".
Posted by rogan8 over 5 years ago
http://www.openreach.co.uk/orpg/news/generalbriefings/gen02709.do
Posted by g-bhxu over 5 years ago
FAO cyberdoyle

Parliament is on Easter break
Posted by webchem over 5 years ago
This is even affecting us down in deepest Dorset...

For once I am able to get a decent 3G connection though - hmmm...

Lets hope they fix it soon.
Posted by jokily over 5 years ago
Happened 32 meters beneath street level. Contractor wasn't working for BT. Since then, emergency services have been restored; half the end users restored. Engineers working through the night to restore further services
Posted by indigobananajam over 5 years ago
Affecting us here in Cumbria as well! My internet went down last night, thankfully came back up after a few hours but next door is still getting a BT Openreach error page "your service should be resumed shortly..."
Posted by ben_phd over 5 years ago
"forcing many broadband users in East London" - ha - not quite right there!

I'm way up north of Aberdeen and am affected too. Namesco seem to have been hit quite badly by this one...

Hopefully it'll be back soon...
Posted by Groovehound over 5 years ago
Well, at least they'll take it more seriously than the paltry 70 homes cut-off totally land-line wise, for about a MONTH in N.E. Wales. Contractors. NOT 30M underground though, so a little easier to fix. Did they care or give compensation? NO! Bunch of ignorant ****s, BT and the contractors. When you live in the sticks, broadband is even more important.
Posted by Daytona2 over 5 years ago
I'm a NDO/Namesco customer and was affected from about 2pm Saturday and all Sunday. It's working normally since login at 7am today and the ping and speed is normal according to -
http://www.speedtest.bbmax.co.uk/
BT were slow to acknowledge the problem on their status page, taking nearly 7 hours -
http://www.btbroadbandoffice.com/business/help/ss/1,,Broadband_0_5_BTCTB_Home,00.html?com.bea.event.type=linkclick&oLName=link.searchresults&oLDesc=KB_127
NDO had better info. -
http://www.ndo.com/network_status.html
As already mentioned, this incident highlights a lack of resilience in BTs network.
Posted by Spectre_01 over 5 years ago
"As already mentioned, this incident highlights a lack of resilience in BTs network."

Yep its an every day event having cables 30mtrs+ below street level damaged by contractors...
Posted by Daytona2 over 5 years ago
Incident over - duration Sat 4 April 14:30 - Mon 6 April 03:00
Posted by chrysalis over 5 years ago
I admit I am shocked a cut in london the capital has caused outages, I would have expected some redundancy. To be fair to BT tho BT backhaul outages seem rare.
Posted by Daytona2 over 5 years ago
A laughable response to a refund of broadband and dial-up costs -
"As the issue was caused by BT and is no fault of Namesco we will be unable to offer a refund."
Using the excuse of a deficient subcontractor has no legal basis.
Ho hum; off to the small claims court we go....
http://www.hmcourts-service.gov.uk/infoabout/claims/index.htm
http://www.consumerdirect.gov.uk/after_you_buy/making-complaint/template-letters/letter_before_action/
Posted by Daytona2 over 5 years ago
"Yep its an every day event having cables 30mtrs+ below street level damaged by contractors..."

Which doesn't address the issue.

Now try and explain it given all the cabling routes BT has around London.
Posted by rogan8 over 5 years ago
AIUI, the incident occured near to Ilford exchange which is a network point for BT and and other operators. The extent of the damage which apparently severed or partly damaged every cable in the tunnel meant it was impossible to re-route every affect service as quickly as would normally occur in such incidents
Posted by rogan8 over 5 years ago
Priority was given to restoring essential services, blue light services, PSTN and other priority services. Everything else will be restored as when new routes are made available.
Posted by KevinR over 5 years ago
The complaints about resilience and multiple routes are valid BUT the problem is it costs money to have multiple transit links even if some are only contingent. Modern business hates investment so you get this problem. Also multiple fibres in one duct is not resilience since fibre usually only fail with JCB syndrome! They cannot configure london's traffic lights either. Assume national broadband affected as lost link to radius servers to allow logins.
Posted by Daytona2 over 5 years ago
"Assume national broadband affected as lost link to radius servers to allow logins."

My router appeared to log in fine, I just got the BT Wholesale error page or document contains no data.
Posted by Dawn_Falcon over 5 years ago
Daytona2 - And if you have a SLA which has been breached, you might have a case... most domestic services are "best effort", though.
Posted by AndrueC over 5 years ago
@Daytona2:I doubt you have a case. The most you could reasonably expect is a couple of days subscription. That's what, a penny? Tuppence?

You wouldn't get call charges for dial-up because that's your choice to use it rather than live without an Internet connection for a couple of days.

Sorry.
Posted by johndc over 5 years ago
Enta have posted an update on their NOC from BT
http://noc.enta.net/2009/04/outage-framestream-leased-lines/comment-page-1/#comment-15783
Posted by MarcusJClifford over 5 years ago
The recovery work is taking place in 2 stages. We have installed ducting and surface level cables (3 x 276 fibre cables installed, 4 more are being installed) to divert cables away from the damaged area of the tunnel

The second stage is to retrieve cables from the tunnel to connect to the newly installed surface cables

The first 96 x fibre cable has been successfully retrieved and jointed, restoring service to multiple transmission systems.
Posted by MarcusJClifford over 5 years ago
Services to Poplar exchange have started to restore with full restoration expected imminently. This will mean that 999 service has been restored to all PSTN customers with contended access to normal voice service

The second 96 x fibre cable retrieval from manhole 144 is progressing slowly due to asbestos and poor air quality. Engineers in breathing apparatus have entered the manhole and are extracting the cable for jointing over ground. This cable will be completed around 05.00. on 7 April.

Other restoration work using make-good capacity is continuing.
Posted by MarcusJClifford over 5 years ago
In total there are 1836 fibres are to be repaired. The first 96 fibres represents 5% of the total.

Post split into 3 because of 600 char limit
Posted by CARPETBURN over 5 years ago
quote"The second 96 x fibre cable retrieval from manhole 144 is progressing slowly due to asbestos and poor air quality."

So like most BT stuff its moldy and old hehe
Posted by Grinzola over 5 years ago
Just received a picture of the Tunnel and the Damage
Looks very Bad
Have no where to upload it to or i would
Posted by SPOTTY over 5 years ago
A late comment I know...perhaps BT should ask to see the contractors risk assessment? It takes real expertise to hit a tunnel at that depth! Especially a shared service duct of considerable significance? "Failing to plan is planning to fail"
Posted by Dawn_Falcon over 5 years ago
No, perhaps BT should be sueing. Which I expect they will be.
Posted by CARPETBURN over 5 years ago
quote"No, perhaps BT should be sueing. Which I expect they will be."

LOL sueing who... Their own contractors which are fixing it or the people that went through the cable in the first place? Some how i dont think thats going to happen. Please remove BT fanboy blinkers, things like cutting a electric cable, comms cable and even water and gas pipes are common, nobody will get sued over this LOL
Posted by adriandaz over 5 years ago
Grinzola, Rapidshare?
Posted by Dawn_Falcon over 5 years ago
Um, so you're saying that the gross incompetence involved in this isn't actionable, Carpet? Of course it is.

Please try actually thinking of a change. People get sued over this sort of thing all the time, yes.
Posted by Somerset over 5 years ago
Maybe not sued but who pays?

CB - using silly terms and text speak will get you banned (again).
Posted by brcooperman over 5 years ago
Check out the pictures of the cable tunnel damage!:-

http://www.flickr.com/photos/23919135@N00/
Posted by CARPETBURN over 5 years ago
quote"Please try actually thinking of a change. People get sued over this sort of thing all the time, yes."

Ill tell my sister to sue BT then which went through a water main about a year ago while running cables to a new lot of housing near her. Her entire road and another a street away were without water for almost 48 hours. The water company obviously just fixed it rather than sue and act like spoilt brats because they had to do work!
Posted by CARPETBURN over 5 years ago
Further more, Virgin Media cables because they are so close to the service are regularly damaged from road works and other contracting works, ever heard of them sueing people cos they chopped one of their cables? No i didnt think so
Posted by rogan8 over 5 years ago
In common with other utilities, BT operates a cost recovery program when their is damage to their network or infrastructure.
The damage incurred in this incident and is considerable .
Posted by Dawn_Falcon over 5 years ago
You don't hear about it, perhaps, Carpet. But either there are payouts (you can insure builders against it, incidentally), or lawsuits. In this case, it looks like gross negligence and that probably means lawsuit...
Posted by mhc over 5 years ago
Carpetburn - you obviously know NOTHING about the processes.
It would not have been necessary for the water co. to sue BT if there was evidence that BT caused the damage. BT would review the claim and pay out if they were at fault.

The BT network protection teams issue many claims every day to individuals, insurance co.s, contractors, utility companies & others for network damage. In this case BT will work on the repairs and issue progress claims. If the contractor/utility refuses to pay then legal action will follow.
Posted by CARPETBURN over 5 years ago
quote"It would not have been necessary for the water co. to sue BT if there was evidence that BT caused the damage. BT would review the claim and pay out if they were at fault."

LMFAO oh yeah BT review there self to see if they are to blame, that makes perfect sense.

Posted by CARPETBURN over 5 years ago
The person that is closest to correct here in how things work is rogan8. There will be loads to look at before BT can claim anything from a contractor, they will have to prove the damage was due to negligence and also prove the damage was avoidable... Legal things are possible but BT will have to prove its purely the contractor at fault to win anything, something believe it or not is highly unlikely. As i said Virgins cables get dug up and damaged all the time, a road not far from me has regular road works and probably every utility company you can think of has damaged Virgins cables
Posted by CARPETBURN over 5 years ago
Even if BT took legal action to recoup repair costs they would have to prove the amount they attempt to claim from this utility company. Any damages award would be based on what a court thinks they are entitled to, not BTs daydream jacked up figures.
Posted by metalhead41 over 5 years ago
Hang on, BT know who caused the damage, they have the evidence that it occured and they know how much it cost to rectify the issue since they rectified it.
Considering the amount of damage, and the compensation BT will be paying out to their customers, they will be claiming the money back from the contractors. It's not as if it was some cowboy firm who cut a line then filled the hole up before anyone noticed...
Posted by JoHubb over 5 years ago
I live in N. Ireland and lost connection a week ago. My ISP is Tiscali. I have had no indication from either Tiscali or BT if/when my broadband will ever be restored. The latest update from either of them was on 7th April.
I will use Virgin as my ISP, cancel my dedicated business phone line with BT, and buy a satellite connection as backup.
Posted by Somerset over 5 years ago
CB - why do you disagree with mhc who knows what happens?

Are you saying utilities just accept damage, no matter what the cost?
Posted by rogan8 over 5 years ago
It would have to be established the contractor was negligent (didnt make the necessary enquiries or deviated off course), or if they were given incorrect info e.g. BT said the tunnel was at a different location or depth. We know what happend, we dont know why
Posted by rogan8 over 5 years ago
Openreach has declared MBORC so there will be minimal compensation for CP's
Posted by rogan8 over 5 years ago
The problem Virgin and other cable co.s have is that mush of their network was laid close to the surface, often half the depth or less than that recommended by the NJUC. This would mae it difficult to enforce damage claims
Posted by Somerset over 5 years ago
See http://www.njug.org.uk/
Posted by CARPETBURN over 5 years ago
quote"It would have to be established the contractor was negligent (didnt make the necessary enquiries or deviated off course), or if they were given incorrect info e.g. BT said the tunnel was at a different location or depth. We know what happend, we dont know why"

Exactly none of us know the circumstances, its even possible the thing the contractor had to repair required removal (or ripping up) of the communications cables. Before BT or any company thinks about claiming any type of damages they have to prove the carnage caused by the contractor was un-necessary (or negligent).
Posted by CARPETBURN over 5 years ago
BT there self could even be to blame for not giving the contractor accurate measurements in terms of depth or similar of where the cable is/was located. Just assuming some company has broke another companys cable and its either parties fault is stupid without knowing the full story (a story of which starts before any holes were even dug).
Posted by rogan8 over 5 years ago
The contractor was Thrust Boring, they were not repairing or removing anything (other than some earth/clay). Its not just cables but an underground tunnel big enough to walk through (well it was until one of the Thunderbird machines went through it)
Posted by Dawn_Falcon over 5 years ago
rogan - Quite (on Virgin). And the fact that a tunnel was breached...no, there's really no excuse here and it's highly unlikely insurance will pay out for gross negligence.

Carpetburn's rabid hatred is amusing, but as usual not focused on reality.
Posted by CARPETBURN over 5 years ago
quote"Carpetburn's rabid hatred is amusing, but as usual not focused on reality"

Oh and your blatant BT humping is reality?

Neither you or anyone here knows the full circumstances. If you could remove your BT blinkers for a split second you would realise that. People must think you own 90% of BT shares or work for them, as nobody else on planet Earth would be so stupid as to defend a company so fruitlessly with no evidence to show the other party concerned are entirely to blame.
Posted by Kper over 5 years ago
FINALLY back online at about midday on 15th April!

11 days without internet at home really makes you realize what a utility it has become.

Quite revealing that didn't miss the landline for PHONECALLS at all (except for a couple of long international calls which would have been much more expensive on a mobile).
Posted by CARPETBURN over 5 years ago
quote"11 days without internet at home...."

Oh but its all one parties fault nothing to do with BT taking that darn long to fix things..... Well that applies if you believe the BT humpers.
Posted by KarlAustin over 5 years ago
Yeah because you could repair that many fibres/re-route them quicker couldn't you?
Posted by reggy666 over 5 years ago
Kper glad to see you're back online :)
would you know which exchange your connected to and who is your isp

i;ve been without broadband since that sat afternoon and just trying to figure out how long it will take to get back online. it's been a proper pain in the ..
Posted by Dawn_Falcon over 5 years ago
So you blame BT for cutting the cable, Carpet? Hmm, that's reasonable. Not.
Posted by CARPETBURN over 5 years ago
Quote"Yeah because you could repair that many fibres/re-route them quicker couldn't you?"

Give me the same amount of staff and resources of one of the worlds richest telecom companies and yes i probably could. My god BT still take a week or longer to go to the exchange pull out a cable and plug it into others equipment when you switch to LLU. If they work at the same speed to repair cables Im not shocked it took them 11 days... BT ethos, if its done quicker than a week it looks like they tried.
Posted by CARPETBURN over 5 years ago
Quote"So you blame BT for cutting the cable, Carpet? Hmm, that's reasonable. Not."

Nah i point the blame where its needed and to whoever is truely at fault..... Something you and your BT buddies cant comprehend.

Maybe this contractor did just hack up the tunnel and cables if thats the case they should in some form be punished. At the same time maybe they did contact BT first who gave they measurements of where the cable is/was located incorrectly or there was no way to avoid cutting the cable to complete the work they had to do...
Posted by CARPETBURN over 5 years ago
Unless you can prove the contractor was 'negligent' (WHICH YOU CANT and never will because you dont even understand what the word means) then I suggest you stop humping praise on BT like a rabid dog humps peoples legs, pause and for once try to think, before you defend BTs share price, or whatever it is you love or try to save about them.
You must be logged in to post comments. Click here to login.