Mailing List Archive

Fiber cut in NoVA
We have some Fiberlight circuits impacted as of about 9:45 AM Eastern.
Still down. Here's what they said:

FiberLight NOC detected multiple alarms between devices in the greater WDC
area, including links down between Vienna VA and our WDC datacenters.*Current
Status:*
FiberLight technician was dispatched to Vienna VA to isolate the issue.
Initial readings put the fiber break less than 1km from that location.
Troubleshooting is ongoing.
*Preliminary RFO:*
Fiber cut near 7990 Science Applications Ct, Vienna VA
Re: Fiber cut in NoVA [ In reply to ]
We also have some Wave (RCN) circuits impacted by this cut. A little more
info:

As of ~7pm Eastern:

Damage traced to construction at Gallows Rd and Saic Ct. Damage is due
to construction boring and is 10 feet deep. Crews on site. No ETR at
this time.

Then at ~9pm Eastern (this one we got over the phone from Wave who got it
from Fiberlight):
they are "routing a spare conduit to check for damages" and continuing to
do escavation [sic], no ETR, 432-count fiber

Nothing since. Anyone have a picture of the damage? Or live close enough to
go take one?

On Tue, Oct 20, 2020 at 6:36 PM Ross Tajvar <ross@tajvar.io> wrote:

> We have some Fiberlight circuits impacted as of about 9:45 AM Eastern.
> Still down. Here's what they said:
>
> FiberLight NOC detected multiple alarms between devices in the greater WDC
> area, including links down between Vienna VA and our WDC datacenters.*Current
> Status:*
> FiberLight technician was dispatched to Vienna VA to isolate the issue.
> Initial readings put the fiber break less than 1km from that location.
> Troubleshooting is ongoing.
> *Preliminary RFO:*
> Fiber cut near 7990 Science Applications Ct, Vienna VA
>
>
Re: Fiber cut in NoVA [ In reply to ]
Sorry, we got an update that I missed around 10pm Eastern:

Safety Shoring is in, and excavation is underway.

That's referring to this previous update:
Trench shoring taking place due to the depth of the damage
(10 foot deep trench). Damaged fiber is 432ct.


So, looks like it's still going to be a while.

On Tue, Oct 20, 2020 at 10:47 PM Ross Tajvar <ross@tajvar.io> wrote:

> We also have some Wave (RCN) circuits impacted by this cut. A little more
> info:
>
> As of ~7pm Eastern:
>
> Damage traced to construction at Gallows Rd and Saic Ct. Damage is due to construction boring and is 10 feet deep. Crews on site. No ETR at this time.
>
> Then at ~9pm Eastern (this one we got over the phone from Wave who got it
> from Fiberlight):
> they are "routing a spare conduit to check for damages" and continuing to
> do escavation [sic], no ETR, 432-count fiber
>
> Nothing since. Anyone have a picture of the damage? Or live close enough
> to go take one?
>
> On Tue, Oct 20, 2020 at 6:36 PM Ross Tajvar <ross@tajvar.io> wrote:
>
>> We have some Fiberlight circuits impacted as of about 9:45 AM Eastern.
>> Still down. Here's what they said:
>>
>> FiberLight NOC detected multiple alarms between devices in the greater
>> WDC area, including links down between Vienna VA and our WDC datacenters.*Current
>> Status:*
>> FiberLight technician was dispatched to Vienna VA to isolate the issue.
>> Initial readings put the fiber break less than 1km from that location.
>> Troubleshooting is ongoing.
>> *Preliminary RFO:*
>> Fiber cut near 7990 Science Applications Ct, Vienna VA
>>
>>
Re: Fiber cut in NoVA [ In reply to ]
The North American Fiber-Seeking Backhoe strikes again.

We need to start an annual culling of that herd.


--
Christopher Conley
Systems Administrator | Fors Marsh Group
1010 N. Glebe Rd, Suite 510
Arlington, VA 22201
________________________________
From: Ross Tajvar via Outages <outages@outages.org>
Sent: Tuesday, October 20, 2020 10:54 PM
To: OUTAGES Mailing List
Subject: Re: [outages] Fiber cut in NoVA

We also have some Wave (RCN) circuits impacted by this cut. A little more info:

As of ~7pm Eastern:

Damage traced to construction at Gallows Rd and Saic Ct. Damage is due to construction boring and is 10 feet deep. Crews on site. No ETR at this time.

Then at ~9pm Eastern (this one we got over the phone from Wave who got it from Fiberlight):
they are "routing a spare conduit to check for damages" and continuing to do escavation [sic], no ETR, 432-count fiber

Nothing since. Anyone have a picture of the damage? Or live close enough to go take one?

On Tue, Oct 20, 2020 at 6:36 PM Ross Tajvar <ross@tajvar.io<mailto:ross@tajvar.io>> wrote:
We have some Fiberlight circuits impacted as of about 9:45 AM Eastern. Still down. Here's what they said:

FiberLight NOC detected multiple alarms between devices in the greater WDC area, including links down between Vienna VA and our WDC datacenters.Current Status:
FiberLight technician was dispatched to Vienna VA to isolate the issue. Initial readings put the fiber break less than 1km from that location. Troubleshooting is ongoing.
Preliminary RFO:
Fiber cut near 7990 Science Applications Ct, Vienna VA
Re: Fiber cut in NoVA [ In reply to ]
Its was a duct bank seeking directional drill actually... :)


On Tue, Oct 20, 2020 at 11:00 PM Christopher Conley via Outages <
outages@outages.org> wrote:

> The North American Fiber-Seeking Backhoe strikes again.
>
> We need to start an annual culling of that herd.
>
>
> --
> Christopher Conley
> Systems Administrator | Fors Marsh Group
> 1010 N. Glebe Rd, Suite 510
> Arlington, VA 22201
> ------------------------------
> *From:* Ross Tajvar via Outages <outages@outages.org>
> *Sent:* Tuesday, October 20, 2020 10:54 PM
> *To:* OUTAGES Mailing List
> *Subject:* Re: [outages] Fiber cut in NoVA
>
> We also have some Wave (RCN) circuits impacted by this cut. A little more
> info:
>
> As of ~7pm Eastern:
>
> Damage traced to construction at Gallows Rd and Saic Ct. Damage is due to construction boring and is 10 feet deep. Crews on site. No ETR at this time.
>
> Then at ~9pm Eastern (this one we got over the phone from Wave who got it
> from Fiberlight):
> they are "routing a spare conduit to check for damages" and continuing to
> do escavation [sic], no ETR, 432-count fiber
>
> Nothing since. Anyone have a picture of the damage? Or live close enough
> to go take one?
>
> On Tue, Oct 20, 2020 at 6:36 PM Ross Tajvar <ross@tajvar.io> wrote:
>
>> We have some Fiberlight circuits impacted as of about 9:45 AM Eastern.
>> Still down. Here's what they said:
>>
>> FiberLight NOC detected multiple alarms between devices in the greater
>> WDC area, including links down between Vienna VA and our WDC datacenters.*Current
>> Status:*
>> FiberLight technician was dispatched to Vienna VA to isolate the issue.
>> Initial readings put the fiber break less than 1km from that location.
>> Troubleshooting is ongoing.
>> *Preliminary RFO:*
>> Fiber cut near 7990 Science Applications Ct, Vienna VA
>>
>> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
>


--


[image: photo]
*Stephen Klenert, CSMP*
Chief Strategy Officer, Atlantic Metro
212.792.9950 x 113 | 609.510.3665 | steve@atlanticmetro.net |
www.atlanticmetro.net |
300 Blvd East Weehawken, NJ 07086
<https://www.facebook.com/atlanticmetro/>
<http://linkedin.com/in/getklenert> <https://twitter.com/atlanticmetro>
*COVID-19 Response -* https://www.atlanticmetro.net/covid-19-response/
Re: Fiber cut in NoVA [ In reply to ]
If you have a better address, I'm about a 10 minute drive from there.

Saic Drive does not cross Gallows.

Gallows Road ends at Route 7, turns into International Drive.

I could not find an Saic Ct.


On Tue, 20 Oct 2020, Ross Tajvar via Outages wrote:

> We also have some Wave (RCN) circuits impacted by this cut. A little more
> info:
>
> As of ~7pm Eastern:
>
> Damage traced to construction at Gallows Rd and Saic Ct. Damage is due
> to construction boring and is 10 feet deep. Crews on site. No ETR at
> this time.
>
> Then at ~9pm Eastern (this one we got over the phone from Wave who got it
> from Fiberlight):
> they are "routing a spare conduit to check for damages" and continuing to
> do escavation [sic], no ETR, 432-count fiber
>
> Nothing since. Anyone have a picture of the damage? Or live close enough to
> go take one?
>
> On Tue, Oct 20, 2020 at 6:36 PM Ross Tajvar <ross@tajvar.io> wrote:
>
>> We have some Fiberlight circuits impacted as of about 9:45 AM Eastern.
>> Still down. Here's what they said:
>>
>> FiberLight NOC detected multiple alarms between devices in the greater WDC
>> area, including links down between Vienna VA and our WDC datacenters.*Current
>> Status:*
>> FiberLight technician was dispatched to Vienna VA to isolate the issue.
>> Initial readings put the fiber break less than 1km from that location.
>> Troubleshooting is ongoing.
>> *Preliminary RFO:*
>> Fiber cut near 7990 Science Applications Ct, Vienna VA
>>
>>
>

---------------------------------------------------------------------------
Peter Beckman Internet Guy
beckman@angryox.com http://www.angryox.com/
---------------------------------------------------------------------------
Re: Fiber cut in NoVA [ In reply to ]
Hmm, that's all I have for an address. They did say it was "near 7990
Science Applications Ct, Vienna VA". I bet if you start there and drive
north on Gallows you'll see it. Might be after it turns into International
Dr even - I wouldn't be surprised if they got it wrong.

On Wed, Oct 21, 2020, 2:09 AM Peter Beckman via Outages <outages@outages.org>
wrote:

> If you have a better address, I'm about a 10 minute drive from there.
>
> Saic Drive does not cross Gallows.
>
> Gallows Road ends at Route 7, turns into International Drive.
>
> I could not find an Saic Ct.
>
>
> On Tue, 20 Oct 2020, Ross Tajvar via Outages wrote:
>
> > We also have some Wave (RCN) circuits impacted by this cut. A little more
> > info:
> >
> > As of ~7pm Eastern:
> >
> > Damage traced to construction at Gallows Rd and Saic Ct. Damage is due
> > to construction boring and is 10 feet deep. Crews on site. No ETR at
> > this time.
> >
> > Then at ~9pm Eastern (this one we got over the phone from Wave who got it
> > from Fiberlight):
> > they are "routing a spare conduit to check for damages" and continuing to
> > do escavation [sic], no ETR, 432-count fiber
> >
> > Nothing since. Anyone have a picture of the damage? Or live close enough
> to
> > go take one?
> >
> > On Tue, Oct 20, 2020 at 6:36 PM Ross Tajvar <ross@tajvar.io> wrote:
> >
> >> We have some Fiberlight circuits impacted as of about 9:45 AM Eastern.
> >> Still down. Here's what they said:
> >>
> >> FiberLight NOC detected multiple alarms between devices in the greater
> WDC
> >> area, including links down between Vienna VA and our WDC
> datacenters.*Current
> >> Status:*
> >> FiberLight technician was dispatched to Vienna VA to isolate the issue.
> >> Initial readings put the fiber break less than 1km from that location.
> >> Troubleshooting is ongoing.
> >> *Preliminary RFO:*
> >> Fiber cut near 7990 Science Applications Ct, Vienna VA
> >>
> >>
> >
>
> ---------------------------------------------------------------------------
> Peter Beckman Internet Guy
> beckman@angryox.com
> http://www.angryox.com/
>
> ---------------------------------------------------------------------------_______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
>
Re: Fiber cut in NoVA [ In reply to ]
12am Eastern:
Crews are having to dig 10 feet in both directions along fiber path due to
more
damage being identified. No ETR available.

1am Eastern:
Crews have repaired half the duct, and have 1 of 2 cables placed to the
first
splice point. Splicers are preparing cable while construction crews
continue to
repair the other half of the duct. No ETR available.

2am Eastern:
Crews have the second section of duct while continuing to prepare for
splicing. No ETR available.

On Tue, Oct 20, 2020, 11:03 PM Stephen Klenert <sklenert@atlanticmetro.net>
wrote:

> Its was a duct bank seeking directional drill actually... :)
>
>
> On Tue, Oct 20, 2020 at 11:00 PM Christopher Conley via Outages <
> outages@outages.org> wrote:
>
>> The North American Fiber-Seeking Backhoe strikes again.
>>
>> We need to start an annual culling of that herd.
>>
>>
>> --
>> Christopher Conley
>> Systems Administrator | Fors Marsh Group
>> 1010 N. Glebe Rd, Suite 510
>> Arlington, VA 22201
>> ------------------------------
>> *From:* Ross Tajvar via Outages <outages@outages.org>
>> *Sent:* Tuesday, October 20, 2020 10:54 PM
>> *To:* OUTAGES Mailing List
>> *Subject:* Re: [outages] Fiber cut in NoVA
>>
>> We also have some Wave (RCN) circuits impacted by this cut. A little more
>> info:
>>
>> As of ~7pm Eastern:
>>
>> Damage traced to construction at Gallows Rd and Saic Ct. Damage is due to construction boring and is 10 feet deep. Crews on site. No ETR at this time.
>>
>> Then at ~9pm Eastern (this one we got over the phone from Wave who got it
>> from Fiberlight):
>> they are "routing a spare conduit to check for damages" and continuing to
>> do escavation [sic], no ETR, 432-count fiber
>>
>> Nothing since. Anyone have a picture of the damage? Or live close enough
>> to go take one?
>>
>> On Tue, Oct 20, 2020 at 6:36 PM Ross Tajvar <ross@tajvar.io> wrote:
>>
>>> We have some Fiberlight circuits impacted as of about 9:45 AM Eastern.
>>> Still down. Here's what they said:
>>>
>>> FiberLight NOC detected multiple alarms between devices in the greater
>>> WDC area, including links down between Vienna VA and our WDC datacenters.*Current
>>> Status:*
>>> FiberLight technician was dispatched to Vienna VA to isolate the issue.
>>> Initial readings put the fiber break less than 1km from that location.
>>> Troubleshooting is ongoing.
>>> *Preliminary RFO:*
>>> Fiber cut near 7990 Science Applications Ct, Vienna VA
>>>
>>> _______________________________________________
>> Outages mailing list
>> Outages@outages.org
>> https://puck.nether.net/mailman/listinfo/outages
>>
>
>
> --
>
>
> [image: photo]
> *Stephen Klenert, CSMP*
> Chief Strategy Officer, Atlantic Metro
> 212.792.9950 x 113 | 609.510.3665 | steve@atlanticmetro.net |
> www.atlanticmetro.net |
> 300 Blvd East Weehawken, NJ 07086
> <https://www.facebook.com/atlanticmetro/>
> <http://linkedin.com/in/getklenert> <https://twitter.com/atlanticmetro>
> *COVID-19 Response -* https://www.atlanticmetro.net/covid-19-response/
>
Re: Fiber cut in NoVA [ In reply to ]
That would make sense.

On Wed, Oct 21, 2020, 7:19 AM Ben Burwell <ben@benburwell.com> wrote:

> On Oct 21, 2020, at 02:29, Ross Tajvar via Outages <outages@outages.org>
> wrote:
>
> Hmm, that's all I have for an address. They did say it was "near 7990
> Science Applications Ct, Vienna VA". I bet if you start there and drive
> north on Gallows you'll see it. Might be after it turns into International
> Dr even - I wouldn't be surprised if they got it wrong.
>
> On Wed, Oct 21, 2020, 2:09 AM Peter Beckman via Outages <
> outages@outages.org> wrote:
>
>> If you have a better address, I'm about a 10 minute drive from there.
>>
>> Saic Drive does not cross Gallows.
>>
>> Gallows Road ends at Route 7, turns into International Drive.
>>
>> I could not find an Saic Ct.
>>
>
> Looking at a map online, it looks like there is a SAIC office at 1953
> Gallows Rd, about 600 meters northwest of 7990 Science Applications Ct.
> Maybe they just added an extra “court” to the address?
>
Re: Fiber cut in NoVA [ In reply to ]
9am Eastern:
Crews have spliced the first 216 of the first 432.
Working to pull 2nd 432. Some alarms have started to clear. No ETR
available

On Wed, Oct 21, 2020 at 2:32 AM Ross Tajvar <ross@tajvar.io> wrote:

> 12am Eastern:
> Crews are having to dig 10 feet in both directions along fiber path due to
> more
> damage being identified. No ETR available.
>
> 1am Eastern:
> Crews have repaired half the duct, and have 1 of 2 cables placed to the
> first
> splice point. Splicers are preparing cable while construction crews
> continue to
> repair the other half of the duct. No ETR available.
>
> 2am Eastern:
> Crews have the second section of duct while continuing to prepare for
> splicing. No ETR available.
>
> On Tue, Oct 20, 2020, 11:03 PM Stephen Klenert <sklenert@atlanticmetro.net>
> wrote:
>
>> Its was a duct bank seeking directional drill actually... :)
>>
>>
>> On Tue, Oct 20, 2020 at 11:00 PM Christopher Conley via Outages <
>> outages@outages.org> wrote:
>>
>>> The North American Fiber-Seeking Backhoe strikes again.
>>>
>>> We need to start an annual culling of that herd.
>>>
>>>
>>> --
>>> Christopher Conley
>>> Systems Administrator | Fors Marsh Group
>>> 1010 N. Glebe Rd, Suite 510
>>> Arlington, VA 22201
>>> ------------------------------
>>> *From:* Ross Tajvar via Outages <outages@outages.org>
>>> *Sent:* Tuesday, October 20, 2020 10:54 PM
>>> *To:* OUTAGES Mailing List
>>> *Subject:* Re: [outages] Fiber cut in NoVA
>>>
>>> We also have some Wave (RCN) circuits impacted by this cut. A little
>>> more info:
>>>
>>> As of ~7pm Eastern:
>>>
>>> Damage traced to construction at Gallows Rd and Saic Ct. Damage is due to construction boring and is 10 feet deep. Crews on site. No ETR at this time.
>>>
>>> Then at ~9pm Eastern (this one we got over the phone from Wave who got
>>> it from Fiberlight):
>>> they are "routing a spare conduit to check for damages" and continuing
>>> to do escavation [sic], no ETR, 432-count fiber
>>>
>>> Nothing since. Anyone have a picture of the damage? Or live close enough
>>> to go take one?
>>>
>>> On Tue, Oct 20, 2020 at 6:36 PM Ross Tajvar <ross@tajvar.io> wrote:
>>>
>>>> We have some Fiberlight circuits impacted as of about 9:45 AM Eastern.
>>>> Still down. Here's what they said:
>>>>
>>>> FiberLight NOC detected multiple alarms between devices in the greater
>>>> WDC area, including links down between Vienna VA and our WDC datacenters.*Current
>>>> Status:*
>>>> FiberLight technician was dispatched to Vienna VA to isolate the issue.
>>>> Initial readings put the fiber break less than 1km from that location.
>>>> Troubleshooting is ongoing.
>>>> *Preliminary RFO:*
>>>> Fiber cut near 7990 Science Applications Ct, Vienna VA
>>>>
>>>> _______________________________________________
>>> Outages mailing list
>>> Outages@outages.org
>>> https://puck.nether.net/mailman/listinfo/outages
>>>
>>
>>
>> --
>>
>>
>> [image: photo]
>> *Stephen Klenert, CSMP*
>> Chief Strategy Officer, Atlantic Metro
>> 212.792.9950 x 113 | 609.510.3665 | steve@atlanticmetro.net |
>> www.atlanticmetro.net |
>> 300 Blvd East Weehawken, NJ 07086
>> <https://www.facebook.com/atlanticmetro/>
>> <http://linkedin.com/in/getklenert> <https://twitter.com/atlanticmetro>
>> *COVID-19 Response -* https://www.atlanticmetro.net/covid-19-response/
>>
>