Mailing List Archive: Internap/XO/Savvis problems?

Internap/XO/Savvis problems?

Jul 2, 2008, 5:56 PM

Post #1 of 3 (176 views)

Greetings Outageers,

One of our customers is seeing some problems between themselves and our DC,
and both-way traceroutes are showing problems just corewards of our
upstream, Internap:

Customer to us:

traceroute to 74.201.255.113 (74.201.255.113), 64 hops max, 40 byte packets
1 192.168.1.1 (192.168.1.1) 3.188 ms 7.275 ms 5.663 ms
2 ATM1-0-60.win17.Melbourne.telstra.net (165.228.176.49) 12.096 ms 9.410 ms 13.803 ms
3 TenGigE0-8-0-5.win-core1.Melbourne.telstra.net (203.50.80.129) 8.397 ms 8.633 ms 11.843 ms
4 Bundle-Pos1.ken-core4.Sydney.telstra.net (203.50.6.21) 22.303 ms 26.382 ms 29.742 ms
5 Port-Channel1.pad-gw2.Sydney.telstra.net (203.50.6.29) 21.379 ms 26.954 ms 31.464 ms
6 10GigabitEthernet1-0.sydp-core02.Sydney.reach.com (203.50.13.46) 115.516 ms 24.861 ms 204.107 ms
7 i-0-0.wil-core02.net.reach.com (202.84.144.101) 167.032 ms 170.378 ms 175.605 ms
8 p9_0-ashc1.net.reach.com (202.84.140.90) 182.026 ms 184.921 ms 181.750 ms
9 g4_14-pax06.net.reach.com (202.84.251.106) 181.076 ms 183.144 ms 181.113 ms
10 206.111.12.173.ptr.us.xo.net (206.111.12.173) 192.088 ms 324.819 ms 194.243 ms
11 * * te-11-0-0.rar3.sanjose-ca.us.xo.net (207.88.12.69) 196.075 ms
12 207.88.14.133.ptr.us.xo.net (207.88.14.133) 261.502 ms 259.664 ms 266.501 ms
13 207.88.14.134.ptr.us.xo.net (207.88.14.134) 264.905 ms 257.116 ms 258.146 ms
(oh so many stars)

Us back to the customer:

traceroute to 165.228.176.49 (165.228.176.49), 30 hops max, 40 byte packets
1 33.254.201.74.static.ey03.engineyard.com (74.201.254.33) 0.907 ms 0.908 ms 0.928 ms
2 76.254.201.74.static.ey03.engineyard.com (74.201.254.76) 0.210 ms 0.241 ms 0.273 ms
3 core1.te6-1-bbnet1.nym007.pnap.net (216.52.95.24) 1.559 ms core1.te6-2-bbnet2.nym007.pnap.net (216.52.95.88) 1.338 ms core1.te6-1-bbnet1.nym007.pnap.net (216.52.95.24) 1.369 ms
4 208.173.129.61 (208.173.129.61) 1.441 ms 1.525 ms 1.658 ms
5 * * *
6 cr2-loopback.sfo.savvis.net (206.24.210.71) 98.457 ms 94.617 ms 101.634 ms
7 pr1-so-0-0-0.PaloAltoPaix.savvis.net (204.70.200.193) 89.389 ms 89.397 ms 89.478 ms
(constellations from here on)

I'm fine from my connection on the same side of the pond, since it looks
like my ISP has a direct peering with Internap:

traceroute to 74.201.255.113 (74.201.255.113), 30 hops max, 40 byte packets
1 172.14.17.1 (172.14.17.1) 34.982 ms 35.987 ms 48.326 ms
2 lns10.syd6.internode.on.net (150.101.197.88) 50.219 ms 51.812 ms 53.293 ms
3 vl14.cor2.syd6.internode.on.net (150.101.197.83) 91.320 ms 92.807 ms 94.274 ms
4 po3.bdr1.syd6.internode.on.net (150.101.199.232) 226.619 ms 231.256 ms 234.897 ms
5 pos2-1.bdr1.lax1.internode.on.net (203.16.213.158) 228.912 ms 233.132 ms 236.411 ms
6 cr1-eqix-peer.lax009.internap.net (206.223.123.29) 245.254 ms 208.194 ms *
7 cr1.wdc005.inappnet-65.cr1.lax009.internap.net (66.79.146.141) 279.374 ms 283.716 ms 289.151 ms
8 cr1.nym009.inappnet-84.cr1.wdc005.internap.net (66.79.146.209) 309.854 ms 316.382 ms 318.303 ms
9 core3.nym.inappnet-82.cr1.nym007.internap.net (66.79.152.130) 298.775 ms 300.648 ms 307.357 ms
10 border2.pc2-bbnet2.nyj001.pnap.net (216.52.95.74) 323.780 ms 325.643 ms 331.222 ms
11 74.201.254.73 (74.201.254.73) 348.343 ms 350.727 ms *
12 113.255.201.74.static.ey03.engineyard.com (74.201.255.113) 305.041 ms 303.767 ms 305.289 ms

Wondering if anyone else is seeing similar problems, or if anyone knows of
any widespread problems with Internap/XO/Savvis that I can pass on to my
customer to let them know what's going on.

Thanks,
- Matt

Internap/XO/Savvis problems? [ In reply to ]

lists at outsourcenow

Jul 2, 2008, 7:15 PM

Post #2 of 3 (181 views)

Permalink

We received the following notice regarding scheduled maintenance from XO.
It is apparently due to some hardware upgrades. While this notice only
refers to AZ, they may have scheduled similar outages in other areas today
as well.

Dear Customer,

This email notice is to inform you of scheduled maintenance occurring in the
Phoenix, AZ area affecting your XO Data/Access service.

On 7/2/2008, between the hours of 11:00 PM MST to 6:00 AM MST, you will
experience a 120 minute interruption of service. XO Communications Customer
Care and Network Teams will be monitoring this activity to ensure necessary
downtime is kept to a minimum while the maintenance is completed.

At your convenience, feel free to reply to this message or call us at the
toll free number listed on your XO bill, as our Customer Care teams will be
available to handle any questions, comments, or concerns.

Please reference Maintenance Request 53135 when contacting Customer Care.

Thank You,
XO Communications

On 7/2/08 5:56 PM, "Matt Palmer" <mpalmer at hezmatt.org> wrote:

> Greetings Outageers,
>
> One of our customers is seeing some problems between themselves and our DC,
> and both-way traceroutes are showing problems just corewards of our
> upstream, Internap:
>
> Customer to us:
>
> traceroute to 74.201.255.113 (74.201.255.113), 64 hops max, 40 byte packets
> 1 192.168.1.1 (192.168.1.1) 3.188 ms 7.275 ms 5.663 ms
> 2 ATM1-0-60.win17.Melbourne.telstra.net (165.228.176.49) 12.096 ms 9.410
> ms 13.803 ms
> 3 TenGigE0-8-0-5.win-core1.Melbourne.telstra.net (203.50.80.129) 8.397 ms
> 8.633 ms 11.843 ms
> 4 Bundle-Pos1.ken-core4.Sydney.telstra.net (203.50.6.21) 22.303 ms 26.382
> ms 29.742 ms
> 5 Port-Channel1.pad-gw2.Sydney.telstra.net (203.50.6.29) 21.379 ms 26.954
> ms 31.464 ms
> 6 10GigabitEthernet1-0.sydp-core02.Sydney.reach.com (203.50.13.46) 115.516
> ms 24.861 ms 204.107 ms
> 7 i-0-0.wil-core02.net.reach.com (202.84.144.101) 167.032 ms 170.378 ms
> 175.605 ms
> 8 p9_0-ashc1.net.reach.com (202.84.140.90) 182.026 ms 184.921 ms 181.750
> ms
> 9 g4_14-pax06.net.reach.com (202.84.251.106) 181.076 ms 183.144 ms
> 181.113 ms
> 10 206.111.12.173.ptr.us.xo.net (206.111.12.173) 192.088 ms 324.819 ms
> 194.243 ms
> 11 * * te-11-0-0.rar3.sanjose-ca.us.xo.net (207.88.12.69) 196.075 ms
> 12 207.88.14.133.ptr.us.xo.net (207.88.14.133) 261.502 ms 259.664 ms
> 266.501 ms
> 13 207.88.14.134.ptr.us.xo.net (207.88.14.134) 264.905 ms 257.116 ms
> 258.146 ms
> (oh so many stars)
>
> Us back to the customer:
>
> traceroute to 165.228.176.49 (165.228.176.49), 30 hops max, 40 byte packets
> 1 33.254.201.74.static.ey03.engineyard.com (74.201.254.33) 0.907 ms 0.908
> ms 0.928 ms
> 2 76.254.201.74.static.ey03.engineyard.com (74.201.254.76) 0.210 ms 0.241
> ms 0.273 ms
> 3 core1.te6-1-bbnet1.nym007.pnap.net (216.52.95.24) 1.559 ms
> core1.te6-2-bbnet2.nym007.pnap.net (216.52.95.88) 1.338 ms
> core1.te6-1-bbnet1.nym007.pnap.net (216.52.95.24) 1.369 ms
> 4 208.173.129.61 (208.173.129.61) 1.441 ms 1.525 ms 1.658 ms
> 5 * * *
> 6 cr2-loopback.sfo.savvis.net (206.24.210.71) 98.457 ms 94.617 ms 101.634
> ms
> 7 pr1-so-0-0-0.PaloAltoPaix.savvis.net (204.70.200.193) 89.389 ms 89.397
> ms 89.478 ms
> (constellations from here on)
>
> I'm fine from my connection on the same side of the pond, since it looks
> like my ISP has a direct peering with Internap:
>
> traceroute to 74.201.255.113 (74.201.255.113), 30 hops max, 40 byte packets
> 1 172.14.17.1 (172.14.17.1) 34.982 ms 35.987 ms 48.326 ms
> 2 lns10.syd6.internode.on.net (150.101.197.88) 50.219 ms 51.812 ms 53.293
> ms
> 3 vl14.cor2.syd6.internode.on.net (150.101.197.83) 91.320 ms 92.807 ms
> 94.274 ms
> 4 po3.bdr1.syd6.internode.on.net (150.101.199.232) 226.619 ms 231.256 ms
> 234.897 ms
> 5 pos2-1.bdr1.lax1.internode.on.net (203.16.213.158) 228.912 ms 233.132 ms
> 236.411 ms
> 6 cr1-eqix-peer.lax009.internap.net (206.223.123.29) 245.254 ms 208.194 ms
> *
> 7 cr1.wdc005.inappnet-65.cr1.lax009.internap.net (66.79.146.141) 279.374 ms
> 283.716 ms 289.151 ms
> 8 cr1.nym009.inappnet-84.cr1.wdc005.internap.net (66.79.146.209) 309.854 ms
> 316.382 ms 318.303 ms
> 9 core3.nym.inappnet-82.cr1.nym007.internap.net (66.79.152.130) 298.775 ms
> 300.648 ms 307.357 ms
> 10 border2.pc2-bbnet2.nyj001.pnap.net (216.52.95.74) 323.780 ms 325.643 ms
> 331.222 ms
> 11 74.201.254.73 (74.201.254.73) 348.343 ms 350.727 ms *
> 12 113.255.201.74.static.ey03.engineyard.com (74.201.255.113) 305.041 ms
> 303.767 ms 305.289 ms
>
> Wondering if anyone else is seeing similar problems, or if anyone knows of
> any widespread problems with Internap/XO/Savvis that I can pass on to my
> customer to let them know what's going on.
>
> Thanks,
> - Matt
> _______________________________________________
> outages mailing list
> outages at isotf.org
> https://puck.nether.net/mailman/listinfo/outages

Internap/XO/Savvis problems? [ In reply to ]

tlyons at ivenue

Jul 2, 2008, 8:01 PM

Post #3 of 3 (177 views)

Permalink

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, Jul 03, 2008 at 10:56:21AM +1000, Matt Palmer wrote:

>traceroute to 165.228.176.49 (165.228.176.49), 30 hops max, 40 byte packets
> 1 33.254.201.74.static.ey03.engineyard.com (74.201.254.33) 0.907 ms 0.908 ms 0.928 ms
> 2 76.254.201.74.static.ey03.engineyard.com (74.201.254.76) 0.210 ms 0.241 ms 0.273 ms
> 3 core1.te6-1-bbnet1.nym007.pnap.net (216.52.95.24) 1.559 ms core1.te6-2-bbnet2.nym007.pnap.net (216.52.95.88) 1.338 ms core1.te6-1-bbnet1.nym007.pnap.net (216.52.95.24) 1.369 ms
> 4 208.173.129.61 (208.173.129.61) 1.441 ms 1.525 ms 1.658 ms
> 5 * * *
> 6 cr2-loopback.sfo.savvis.net (206.24.210.71) 98.457 ms 94.617 ms 101.634 ms
> 7 pr1-so-0-0-0.PaloAltoPaix.savvis.net (204.70.200.193) 89.389 ms 89.397 ms 89.478 ms
>(constellations from here on)

Same here, with a Savvis peering:

Router: Ivenue bgp1
Command: traceroute 165.228.176.49

Tracing the route to ATM1-0-60.win17.Melbourne.telstra.net (165.228.176.49)

1 64.70.17.121 [AS 3561] 4 msec 4 msec 0 msec
2 204.70.203.90 [AS 3561] 0 msec 8 msec 0 msec
3 * *
cr2-loopback.sfo.savvis.net (206.24.210.71) [AS 3561] 12 msec
4 pr1-so-0-0-0.PaloAltoPaix.savvis.net (204.70.200.193) [AS 3561] 12
msec 12 msec 12 msec
5 * * * repeated ad nauseum

>Wondering if anyone else is seeing similar problems, or if anyone knows of
>any widespread problems with Internap/XO/Savvis that I can pass on to my
>customer to let them know what's going on.

Don't know what the issues are, but the gist from tech support is
"Eastern Australia can't connect".

Savvis case number is 1336683, no ETA yet, but engineers are on it.
Savvis confirmed that it is a connectivity issue to Asia/Pac region.
- --
Regards... Todd
They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety. --Benjamin Franklin
Linux kernel 2.6.24-18-generic 3 users, load average: 0.00, 0.00, 0.00
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFIbEDuY2VBGxIDMLwRAkyJAJsHqeeMg4PmQ0NwgoZnnlum5ewgywCfdjhP
1Xhg7q00y5/+TG0GuWC/8C8=
=Okzj
-----END PGP SIGNATURE-----