Broadband News

Major cable cut in East London tunnel

Over the last 24 hours, a serious incident has taken place in East London (near Dartford) where a service tunnel containing communications cables has been damaged by a contractor, forcing many broadband users in East London to remember what it was like to browse the Internet before broadband was available. The tunnel in question is over 30 metres below ground and due to the extensive structural damage, the repair process is likely to be quite lengthy.

It appears to be affecting a number of different services, including BT Wholesale's DSL platform. BT have indicated to service providers that in order to fix the issue, they will need to bring the cables to the surface and route them through new ducting, a process that is likely to take days, although BT doesn't appear to be providing more precise estimates.

In the mean time, Openreach have managed to work around some of the problems to get more users online, although we are seeing reports of several problems including high latency and slow speeds indicating that these are running over very congested links. Still, slow Internet is better than no Internet. At least you can Twitter about it ;)

07/04/09 Update from BT: "Service has been restored to a significant number of customers, including emergency services, all 75,000 telephony lines impacted, as well as a number of business and other communications provider circuits using alternative methods of connection. Restoring some services is a complex engineering task, but BT engineering teams are working around the clock to restore service to all remaining customers without service as soon as possible. [..] Given the complexity of the incident, it is not yet possible to accurately predict when all services will be restored. [..] BT will issue further updates as the situation changes." - Customers wishing to get updated information can call 0800 169 0199 to hear the latest.

Comments

I sincerely hope they get it fixed soon, and I sympathise with the many people who will suffer and the business lost, but on the other hand I wish the houses of parliament are on that pipe and that they have a full week of poor access to t'internet and then they may just realise what many of the people in the rest of the country have to contend with. It might bring new perception into the final Carter report.
Every cloud has a silver lining. somewhere.

  • cyberdoyle
  • over 8 years ago

This is a bit of a bummer.

A Tsar Bomba size nuclear explosion wouldn't bring new perception into the final Carter report, only a total replacement of the politicians and industry whores with vested interests who compose it with people who are not interested in political expediency, not interested in commercial expediency and have some clue.

  • Dixinormous
  • over 8 years ago

A source of mine from BT Wholesale says over 70,000 lines have been affected, and they're not expecting it to be fixed before Tuesday.

  • phoenixseven
  • over 8 years ago

What is it with Contactors and communications cabling in London? sheesh..

  • _TRIaXOR_
  • over 8 years ago

Contractors... you get what you pay for

  • Spectre_01
  • over 8 years ago

What happened to the idea of a 'network'? Seems to me its always a 'tree'. Surely the original ARPANET idea was to allow for links to be knocked out and data would just route around the failed link - how often do you see that design in practice?

  • rmillwood
  • over 8 years ago

I always thought BT had a "figure of eight" network on all the main routes so that this sort of thing couldn't happen ? More economies methinks, "it cant possibly happen".

  • amonra
  • over 8 years ago

http://www.openreach.co.uk/orpg/news/generalbriefings/gen02709.do

  • rogan8
  • over 8 years ago

FAO cyberdoyle

Parliament is on Easter break

  • g-bhxu
  • over 8 years ago

This is even affecting us down in deepest Dorset...

For once I am able to get a decent 3G connection though - hmmm...

Lets hope they fix it soon.

  • webchem
  • over 8 years ago

Happened 32 meters beneath street level. Contractor wasn't working for BT. Since then, emergency services have been restored; half the end users restored. Engineers working through the night to restore further services

  • jokily
  • over 8 years ago

Affecting us here in Cumbria as well! My internet went down last night, thankfully came back up after a few hours but next door is still getting a BT Openreach error page "your service should be resumed shortly..."

  • indigobananajam
  • over 8 years ago

"forcing many broadband users in East London" - ha - not quite right there!

I'm way up north of Aberdeen and am affected too. Namesco seem to have been hit quite badly by this one...

Hopefully it'll be back soon...

  • ben_phd
  • over 8 years ago

Well, at least they'll take it more seriously than the paltry 70 homes cut-off totally land-line wise, for about a MONTH in N.E. Wales. Contractors. NOT 30M underground though, so a little easier to fix. Did they care or give compensation? NO! Bunch of ignorant ****s, BT and the contractors. When you live in the sticks, broadband is even more important.

  • Groovehound
  • over 8 years ago

I'm a NDO/Namesco customer and was affected from about 2pm Saturday and all Sunday. It's working normally since login at 7am today and the ping and speed is normal according to -
http://www.speedtest.bbmax.co.uk/
BT were slow to acknowledge the problem on their status page, taking nearly 7 hours -
http://www.btbroadbandoffice.com/business/help/ss/1,,Broadband_0_5_BTCTB_Home,00.html?com.bea.event.type=linkclick&oLName=link.searchresults&oLDesc=KB_127
NDO had better info. -
http://www.ndo.com/network_status.html
As already mentioned, this incident highlights a lack of resilience in BTs network.

  • Daytona2
  • over 8 years ago

"As already mentioned, this incident highlights a lack of resilience in BTs network."

Yep its an every day event having cables 30mtrs+ below street level damaged by contractors...

  • Spectre_01
  • over 8 years ago

Incident over - duration Sat 4 April 14:30 - Mon 6 April 03:00

  • Daytona2
  • over 8 years ago

I admit I am shocked a cut in london the capital has caused outages, I would have expected some redundancy. To be fair to BT tho BT backhaul outages seem rare.

  • chrysalis
  • over 8 years ago

A laughable response to a refund of broadband and dial-up costs -
"As the issue was caused by BT and is no fault of Namesco we will be unable to offer a refund."
Using the excuse of a deficient subcontractor has no legal basis.
Ho hum; off to the small claims court we go....
http://www.hmcourts-service.gov.uk/infoabout/claims/index.htm
http://www.consumerdirect.gov.uk/after_you_buy/making-complaint/template-letters/letter_before_action/

  • Daytona2
  • over 8 years ago

"Yep its an every day event having cables 30mtrs+ below street level damaged by contractors..."

Which doesn't address the issue.

Now try and explain it given all the cabling routes BT has around London.

  • Daytona2
  • over 8 years ago

AIUI, the incident occured near to Ilford exchange which is a network point for BT and and other operators. The extent of the damage which apparently severed or partly damaged every cable in the tunnel meant it was impossible to re-route every affect service as quickly as would normally occur in such incidents

  • rogan8
  • over 8 years ago

Priority was given to restoring essential services, blue light services, PSTN and other priority services. Everything else will be restored as when new routes are made available.

  • rogan8
  • over 8 years ago

The complaints about resilience and multiple routes are valid BUT the problem is it costs money to have multiple transit links even if some are only contingent. Modern business hates investment so you get this problem. Also multiple fibres in one duct is not resilience since fibre usually only fail with JCB syndrome! They cannot configure london's traffic lights either. Assume national broadband affected as lost link to radius servers to allow logins.

  • KevinR
  • over 8 years ago

"Assume national broadband affected as lost link to radius servers to allow logins."

My router appeared to log in fine, I just got the BT Wholesale error page or document contains no data.

  • Daytona2
  • over 8 years ago

Daytona2 - And if you have a SLA which has been breached, you might have a case... most domestic services are "best effort", though.

  • Dawn_Falcon
  • over 8 years ago

@Daytona2:I doubt you have a case. The most you could reasonably expect is a couple of days subscription. That's what, a penny? Tuppence?

You wouldn't get call charges for dial-up because that's your choice to use it rather than live without an Internet connection for a couple of days.

Sorry.

  • AndrueC
  • over 8 years ago

Enta have posted an update on their NOC from BT
http://noc.enta.net/2009/04/outage-framestream-leased-lines/comment-page-1/#comment-15783

  • johndc
  • over 8 years ago

The recovery work is taking place in 2 stages. We have installed ducting and surface level cables (3 x 276 fibre cables installed, 4 more are being installed) to divert cables away from the damaged area of the tunnel

The second stage is to retrieve cables from the tunnel to connect to the newly installed surface cables

The first 96 x fibre cable has been successfully retrieved and jointed, restoring service to multiple transmission systems.

  • MarcusJClifford
  • over 8 years ago

Services to Poplar exchange have started to restore with full restoration expected imminently. This will mean that 999 service has been restored to all PSTN customers with contended access to normal voice service

The second 96 x fibre cable retrieval from manhole 144 is progressing slowly due to asbestos and poor air quality. Engineers in breathing apparatus have entered the manhole and are extracting the cable for jointing over ground. This cable will be completed around 05.00. on 7 April.

Other restoration work using make-good capacity is continuing.

  • MarcusJClifford
  • over 8 years ago

In total there are 1836 fibres are to be repaired. The first 96 fibres represents 5% of the total.

Post split into 3 because of 600 char limit

  • MarcusJClifford
  • over 8 years ago

quote"The second 96 x fibre cable retrieval from manhole 144 is progressing slowly due to asbestos and poor air quality."

So like most BT stuff its moldy and old hehe

  • CARPETBURN
  • over 8 years ago

Just received a picture of the Tunnel and the Damage
Looks very Bad
Have no where to upload it to or i would

  • Grinzola
  • over 8 years ago

A late comment I know...perhaps BT should ask to see the contractors risk assessment? It takes real expertise to hit a tunnel at that depth! Especially a shared service duct of considerable significance? "Failing to plan is planning to fail"

  • SPOTTY
  • over 8 years ago

No, perhaps BT should be sueing. Which I expect they will be.

  • Dawn_Falcon
  • over 8 years ago

quote"No, perhaps BT should be sueing. Which I expect they will be."

LOL sueing who... Their own contractors which are fixing it or the people that went through the cable in the first place? Some how i dont think thats going to happen. Please remove BT fanboy blinkers, things like cutting a electric cable, comms cable and even water and gas pipes are common, nobody will get sued over this LOL

  • CARPETBURN
  • over 8 years ago

Grinzola, Rapidshare?

  • adriandaz
  • over 8 years ago

Um, so you're saying that the gross incompetence involved in this isn't actionable, Carpet? Of course it is.

Please try actually thinking of a change. People get sued over this sort of thing all the time, yes.

  • Dawn_Falcon
  • over 8 years ago

Maybe not sued but who pays?

CB - using silly terms and text speak will get you banned (again).

  • Somerset
  • over 8 years ago

Check out the pictures of the cable tunnel damage!:-

http://www.flickr.com/photos/23919135@N00/

  • brcooperman
  • over 8 years ago

quote"Please try actually thinking of a change. People get sued over this sort of thing all the time, yes."

Ill tell my sister to sue BT then which went through a water main about a year ago while running cables to a new lot of housing near her. Her entire road and another a street away were without water for almost 48 hours. The water company obviously just fixed it rather than sue and act like spoilt brats because they had to do work!

  • CARPETBURN
  • over 8 years ago

Further more, Virgin Media cables because they are so close to the service are regularly damaged from road works and other contracting works, ever heard of them sueing people cos they chopped one of their cables? No i didnt think so

  • CARPETBURN
  • over 8 years ago

In common with other utilities, BT operates a cost recovery program when their is damage to their network or infrastructure.
The damage incurred in this incident and is considerable .

  • rogan8
  • over 8 years ago

You don't hear about it, perhaps, Carpet. But either there are payouts (you can insure builders against it, incidentally), or lawsuits. In this case, it looks like gross negligence and that probably means lawsuit...

  • Dawn_Falcon
  • over 8 years ago

Carpetburn - you obviously know NOTHING about the processes.
It would not have been necessary for the water co. to sue BT if there was evidence that BT caused the damage. BT would review the claim and pay out if they were at fault.

The BT network protection teams issue many claims every day to individuals, insurance co.s, contractors, utility companies & others for network damage. In this case BT will work on the repairs and issue progress claims. If the contractor/utility refuses to pay then legal action will follow.

  • mhc
  • over 8 years ago

quote"It would not have been necessary for the water co. to sue BT if there was evidence that BT caused the damage. BT would review the claim and pay out if they were at fault."

LMFAO oh yeah BT review there self to see if they are to blame, that makes perfect sense.

  • CARPETBURN
  • over 8 years ago

The person that is closest to correct here in how things work is rogan8. There will be loads to look at before BT can claim anything from a contractor, they will have to prove the damage was due to negligence and also prove the damage was avoidable... Legal things are possible but BT will have to prove its purely the contractor at fault to win anything, something believe it or not is highly unlikely. As i said Virgins cables get dug up and damaged all the time, a road not far from me has regular road works and probably every utility company you can think of has damaged Virgins cables

  • CARPETBURN
  • over 8 years ago

Even if BT took legal action to recoup repair costs they would have to prove the amount they attempt to claim from this utility company. Any damages award would be based on what a court thinks they are entitled to, not BTs daydream jacked up figures.

  • CARPETBURN
  • over 8 years ago

Hang on, BT know who caused the damage, they have the evidence that it occured and they know how much it cost to rectify the issue since they rectified it.
Considering the amount of damage, and the compensation BT will be paying out to their customers, they will be claiming the money back from the contractors. It's not as if it was some cowboy firm who cut a line then filled the hole up before anyone noticed...

  • metalhead41
  • over 8 years ago

I live in N. Ireland and lost connection a week ago. My ISP is Tiscali. I have had no indication from either Tiscali or BT if/when my broadband will ever be restored. The latest update from either of them was on 7th April.
I will use Virgin as my ISP, cancel my dedicated business phone line with BT, and buy a satellite connection as backup.

  • JoHubb
  • over 8 years ago

CB - why do you disagree with mhc who knows what happens?

Are you saying utilities just accept damage, no matter what the cost?

  • Somerset
  • over 8 years ago

It would have to be established the contractor was negligent (didnt make the necessary enquiries or deviated off course), or if they were given incorrect info e.g. BT said the tunnel was at a different location or depth. We know what happend, we dont know why

  • rogan8
  • over 8 years ago

Openreach has declared MBORC so there will be minimal compensation for CP's

  • rogan8
  • over 8 years ago

The problem Virgin and other cable co.s have is that mush of their network was laid close to the surface, often half the depth or less than that recommended by the NJUC. This would mae it difficult to enforce damage claims

  • rogan8
  • over 8 years ago

See http://www.njug.org.uk/

  • Somerset
  • over 8 years ago

quote"It would have to be established the contractor was negligent (didnt make the necessary enquiries or deviated off course), or if they were given incorrect info e.g. BT said the tunnel was at a different location or depth. We know what happend, we dont know why"

Exactly none of us know the circumstances, its even possible the thing the contractor had to repair required removal (or ripping up) of the communications cables. Before BT or any company thinks about claiming any type of damages they have to prove the carnage caused by the contractor was un-necessary (or negligent).

  • CARPETBURN
  • over 8 years ago

BT there self could even be to blame for not giving the contractor accurate measurements in terms of depth or similar of where the cable is/was located. Just assuming some company has broke another companys cable and its either parties fault is stupid without knowing the full story (a story of which starts before any holes were even dug).

  • CARPETBURN
  • over 8 years ago

The contractor was Thrust Boring, they were not repairing or removing anything (other than some earth/clay). Its not just cables but an underground tunnel big enough to walk through (well it was until one of the Thunderbird machines went through it)

  • rogan8
  • over 8 years ago

rogan - Quite (on Virgin). And the fact that a tunnel was breached...no, there's really no excuse here and it's highly unlikely insurance will pay out for gross negligence.

Carpetburn's rabid hatred is amusing, but as usual not focused on reality.

  • Dawn_Falcon
  • over 8 years ago

quote"Carpetburn's rabid hatred is amusing, but as usual not focused on reality"

Oh and your blatant BT humping is reality?

Neither you or anyone here knows the full circumstances. If you could remove your BT blinkers for a split second you would realise that. People must think you own 90% of BT shares or work for them, as nobody else on planet Earth would be so stupid as to defend a company so fruitlessly with no evidence to show the other party concerned are entirely to blame.

  • CARPETBURN
  • over 8 years ago

FINALLY back online at about midday on 15th April!

11 days without internet at home really makes you realize what a utility it has become.

Quite revealing that didn't miss the landline for PHONECALLS at all (except for a couple of long international calls which would have been much more expensive on a mobile).

  • Kper
  • over 8 years ago

quote"11 days without internet at home...."

Oh but its all one parties fault nothing to do with BT taking that darn long to fix things..... Well that applies if you believe the BT humpers.

  • CARPETBURN
  • over 8 years ago

Yeah because you could repair that many fibres/re-route them quicker couldn't you?

  • KarlAustin
  • over 8 years ago

Kper glad to see you're back online :)
would you know which exchange your connected to and who is your isp

i;ve been without broadband since that sat afternoon and just trying to figure out how long it will take to get back online. it's been a proper pain in the ..

  • reggy666
  • over 8 years ago

So you blame BT for cutting the cable, Carpet? Hmm, that's reasonable. Not.

  • Dawn_Falcon
  • over 8 years ago

Quote"Yeah because you could repair that many fibres/re-route them quicker couldn't you?"

Give me the same amount of staff and resources of one of the worlds richest telecom companies and yes i probably could. My god BT still take a week or longer to go to the exchange pull out a cable and plug it into others equipment when you switch to LLU. If they work at the same speed to repair cables Im not shocked it took them 11 days... BT ethos, if its done quicker than a week it looks like they tried.

  • CARPETBURN
  • over 8 years ago

Quote"So you blame BT for cutting the cable, Carpet? Hmm, that's reasonable. Not."

Nah i point the blame where its needed and to whoever is truely at fault..... Something you and your BT buddies cant comprehend.

Maybe this contractor did just hack up the tunnel and cables if thats the case they should in some form be punished. At the same time maybe they did contact BT first who gave they measurements of where the cable is/was located incorrectly or there was no way to avoid cutting the cable to complete the work they had to do...

  • CARPETBURN
  • over 8 years ago

Unless you can prove the contractor was 'negligent' (WHICH YOU CANT and never will because you dont even understand what the word means) then I suggest you stop humping praise on BT like a rabid dog humps peoples legs, pause and for once try to think, before you defend BTs share price, or whatever it is you love or try to save about them.

  • CARPETBURN
  • over 8 years ago

Post a comment

Login Register