Mailing List Archive: Major L3 issues on east coast?

Major L3 issues on east coast?

Sep 19, 2007, 10:32 AM

Post #1 of 4 (686 views)

Hey everyone,

Seems to be a major issues with Broadwing/Level3. Our BGP session went kaput
about fifteen minutes ago. Traceroutes from Sprint and AT&T make it
one or two hops into their network. NOC line is busy... has anyone heard
any info?

border1.dc3>traceroute 216.140.65.1

Type escape sequence to abort.
Tracing the route to 216.140.65.1

1 208.24.180.58 [AS 1239] 0 msec 4 msec 4 msec
2 144.232.16.105 [AS 1239] 4 msec 0 msec 4 msec
3 144.232.20.145 [AS 1239] 8 msec 4 msec 4 msec
4 144.232.14.122 [AS 1239] 4 msec 4 msec 8 msec
5 144.232.20.189 [AS 1239] 8 msec 4 msec 8 msec
6 4.68.111.169 [AS 3356] 8 msec 8 msec 8 msec

route-server>traceroute 216.140.65.73

Type escape sequence to abort.
Tracing the route to 216.140.65.73

1 white-dwarf.cbbtier3.att.net (12.0.1.1) [AS 7018] 0 msec 0 msec 0 msec
2 ar13.s10-0-0.n54ny.ip.att.net (12.124.182.17) 0 msec 4 msec 4 msec
3 tbr1.n54ny.ip.att.net (12.123.0.42) [MPLS: Label 31292 Exp 0] 0 msec 4 msec 4 msec
4 ggr2.n54ny.ip.att.net (12.123.0.89) 4 msec 0 msec 4 msec
5 so-8-0-0.car3.NewYork1.Level3.net (4.68.127.149) [AS 3356] 4 msec 0 msec 4 msec
6 ae-2-79.edge5.NewYork1.Level3.net (4.68.16.73) [AS 3356] 4 msec 4 msec 4 msec

Thanks,
Ross Vandegrift

Major L3 issues on east coast? [ In reply to ]

lazlor

Sep 19, 2007, 10:36 AM

Post #2 of 4 (671 views)

Permalink

We lost a broadwing DS3 in SF.

Back now, mostly.

Major L3 issues on east coast? [ In reply to ]

Kevin

Sep 19, 2007, 10:39 AM

Post #3 of 4 (673 views)

Permalink

Ditto here, our OC12 BGP session to Bwing/L3 died along the same
timeline...
Thank goodness for proper design and backup capacity !

======================
W. Kevin Hunt
CCIE #11841

-----Original Message-----
From: Ross Vandegrift [mailto:ross@kallisti.us]
Sent: Wednesday, September 19, 2007 12:33 PM
To: outages at isotf.org
Subject: [Outages] Major L3 issues on east coast?

Hey everyone,

Seems to be a major issues with Broadwing/Level3. Our BGP session went
kaput
about fifteen minutes ago. Traceroutes from Sprint and AT&T make it
one or two hops into their network. NOC line is busy... has anyone
heard
any info?

Major L3 issues on east coast? [ In reply to ]

jason

Sep 19, 2007, 10:41 AM

Post #4 of 4 (674 views)

Permalink

Same here. Number of prefixes from Broadwing in Cleveland Ohio went to
~2200 and latency to next hop ICMP RTT ~300ms. We also have a lot of
Broadwing DIA resale lines in Chicago area that we can not get to while
other Broadwing circuits elsewhere are reachable. BW/L3 NOC lines are
jammed but did get through. Got the usual "We know something is wrong but
not exactly what."

Looks like a major routing blowup somewhere.

Jason

On 9/19/07 1:32 PM, "Ross Vandegrift" <ross at kallisti.us> wrote:

> Hey everyone,
>
> Seems to be a major issues with Broadwing/Level3. Our BGP session went kaput
> about fifteen minutes ago. Traceroutes from Sprint and AT&T make it
> one or two hops into their network. NOC line is busy... has anyone heard
> any info?
>
>
> border1.dc3>traceroute 216.140.65.1
>
> Type escape sequence to abort.
> Tracing the route to 216.140.65.1
>
> 1 208.24.180.58 [AS 1239] 0 msec 4 msec 4 msec
> 2 144.232.16.105 [AS 1239] 4 msec 0 msec 4 msec
> 3 144.232.20.145 [AS 1239] 8 msec 4 msec 4 msec
> 4 144.232.14.122 [AS 1239] 4 msec 4 msec 8 msec
> 5 144.232.20.189 [AS 1239] 8 msec 4 msec 8 msec
> 6 4.68.111.169 [AS 3356] 8 msec 8 msec 8 msec
>
>
> route-server>traceroute 216.140.65.73
>
> Type escape sequence to abort.
> Tracing the route to 216.140.65.73
>
> 1 white-dwarf.cbbtier3.att.net (12.0.1.1) [AS 7018] 0 msec 0 msec 0 msec
> 2 ar13.s10-0-0.n54ny.ip.att.net (12.124.182.17) 0 msec 4 msec 4 msec
> 3 tbr1.n54ny.ip.att.net (12.123.0.42) [MPLS: Label 31292 Exp 0] 0 msec 4
> msec 4 msec
> 4 ggr2.n54ny.ip.att.net (12.123.0.89) 4 msec 0 msec 4 msec
> 5 so-8-0-0.car3.NewYork1.Level3.net (4.68.127.149) [AS 3356] 4 msec 0 msec 4
> msec
> 6 ae-2-79.edge5.NewYork1.Level3.net (4.68.16.73) [AS 3356] 4 msec 4 msec 4
> msec
>
>
> Thanks,
> Ross Vandegrift
> _______________________________________________
> Outages mailing list
> Outages at isotf.org
> http://isotf.org/mailman/listinfo/outages

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://isotf.org/pipermail/outages/attachments/20070919/31ae7392/attachment.htm