Mailing List Archive

1. Are you able to use a tool like mtr (instead of traceroute), so the
loss can be tracked back to a specific hop easier? mtr is also
ICMP-based unless otherwise specified. I want to believe it's between
hop #6 and #7 but I cannot rule out what's shown at hop #1, nor do I
know if the loss shown at #7 is normal or not. If you need to know what
mtr flags to use, I recommend -4 -z -b --order=LSRNABW

2. This is only half the picture due to asymmetric routing being common.
Are you able to run mtr on/from the destination (208.79.93.154) going
back to the source (whatever your IP is)?

Ideally, start both mtrs at the same time + let them run for a full
5-10 minutes; anything less isn't helpful.

--
| Jeremy Chadwick jdc@koitsu.org |
| UNIX Systems Administrator PGP 0x2A389531 |
| Making life hard for others since 1977. |

On Mon, Jan 15, 2024 at 11:12:28AM -0800, Lyndon Nerenberg (VE7TFX/VE6BBM) via Outages wrote:
> For the last hour I've been seeing high packet loss rates inside
> Shaw's western Canada network. E.g.:
>
> --- orthanc.ca ping statistics ---
> 152 packets transmitted, 8 packets received, 94.7% packet loss
> round-trip min/avg/max/stddev = 109.441/285.678/696.621/194.853 ms
>
> That's from Vancouver residential to a colo server in L.A. A typical
> traceroute shows the choke point seems to be Calgary:
>
> traceroute to orthanc.ca (208.79.93.154), 64 hops max, 72 byte packets
> 1 * 192.168.0.1 (192.168.0.1) 20.742 ms 4.571 ms
> 2 24.86.192.1 (24.86.192.1) 15.915 ms 12.080 ms 16.014 ms
> 3 rc1bb-be147-1.vc.shawcable.net (64.59.150.185) 11.852 ms 16.537 ms 31.073 ms
> 4 24.244.59.217 (24.244.59.217) 12.509 ms 12.295 ms 12.035 ms
> 5 24.83.252.66 (24.83.252.66) 11.529 ms 24.933 ms 197.457 ms
> 6 rc1st-be214.vc.shawcable.net (24.244.58.65) 13.587 ms 10.923 ms 104.439 ms
> 7 * * rc3no-be11-1.cg.shawcable.net (66.163.72.69) 58.938 ms
> 8 rc3so-be23.cg.shawcable.net (66.163.71.118) 50.948 ms 27.276 ms 25.755 ms
> 9 66.163.71.42 (66.163.71.42) 24.937 ms 30.755 ms 23.542 ms
> 10 rc4ec-be13.il.shawcable.net (66.163.65.18) 96.292 ms * 90.125 ms
> 11 equinix-ch2.cr2.ord1.amcbb.net (208.115.136.124) 53.652 ms * *
> 12 ae1-4.cr1.ord1.atlanticmetro.net (174.128.183.2) 173.172 ms * 248.078 ms
> 13 ve521.cr2.lax0.atlanticmetro.net (208.78.26.210) 126.758 ms * *
> 14 xe-0-0-3-0.cr2.lax2.atlanticmetro.net (208.78.31.37) 104.574 ms 234.672 ms *
> 15 * ae3-4.cr2.lax3.atlanticmetro.net (173.205.176.40) 114.315 ms *
> 16 arpnetworks-dmarc.lax3.atlanticmetro.net (173.205.184.132) 179.245 ms * *
> 17 * orthanc.ca (208.79.93.154) 191.936 ms 102.680 ms
>
> Anyone else seeing this? Or know what's going on?
>
> --lyndon
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
>
_______________________________________________
Outages mailing list
Outages@outages.org
https://puck.nether.net/mailman/listinfo/outages

I didn't have mtr installed on those hosts. The path is definitely
asym: 17 hops YVR-LAX, 10 hops coming back. But both ways the
traffic routes through Chicago, giving me those spectacular 100ms
RTTs :-P

I ran several traceroutes from the YVR side, and they all pointed
to Calgary as the choke point. But whatever it was has cleared up
now.

.--lyndon
_______________________________________________
Outages mailing list
Outages@outages.org
https://puck.nether.net/mailman/listinfo/outages

Is this maybe due to the cooling issues in Equinix CH1?

On Mon, Jan 15, 2024 at 2:28?PM Lyndon Nerenberg (VE7TFX/VE6BBM) via
Outages <outages@outages.org> wrote:

> I didn't have mtr installed on those hosts. The path is definitely
> asym: 17 hops YVR-LAX, 10 hops coming back. But both ways the
> traffic routes through Chicago, giving me those spectacular 100ms
> RTTs :-P
>
> I ran several traceroutes from the YVR side, and they all pointed
> to Calgary as the choke point. But whatever it was has cleared up
> now.
>
> .--lyndon
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
>


--

Adair Winter
AW Broadband
A 203 SW 8Th Ave. Ste 601, Amarillo, Texas
O 806.412.0888 <806-412-0888>M 806.231.7180 <806-231-7180>E
adairw@awbb.net <adairw@awbb.net> W www.awbb.net
Click Here to Schedule a Meeting with me.
<https://calendar.app.google/XNeZDG6FUtcc8uRY8>
<http://facebook.com/awbroadband>

fwiw,

This shows up in an interesting way looking at the v4 vs. v6 paths for
this. I'm also coming from Shaw in Western Canada (BC).

The v6 path heads from Shaw through to a quick handoff to 1299 via Seattle
and then down the west coast:


Packets Pings
Host
Loss% Snt Drop Last Avg Best Wrst StDev Jttr
1. AS6327 2604:3d08:427f:3c03:2e0:67ff:fe22:3565
0.0% 254 0 2.7 2.8 2.1 7.0 0.4 0.2
2. AS6327 2604:3d08:1:2::1
0.0% 254 0 17.7 15.1 10.5 44.8 3.5 4.1
3. AS6327 2604:3d00:f040::186
35.6% 254 90 14.9 15.4 8.8 25.6 2.5 0.4
4. (waiting for reply)
5. (waiting for reply)
6. (waiting for reply)
7. AS6327 2001:4e8:0:8150::2
18.1% 254 46 22.1 22.8 18.7 85.9 5.3 0.1
8. AS6327 2001:4e8:0:80b4::1
16.9% 254 43 22.5 21.9 16.5 90.5 5.6 1.2
9. AS1299 sea-b2-link.ip.twelve99.net
88.9% 254 225 16.4 21.3 16.4 26.6 2.2 5.2
10. AS1299 sjo-b23-v6.ip.twelve99.net
0.0% 254 0 45.0 41.4 34.3 65.2 3.6 5.2
11. AS1299 lax-b23-v6.ip.twelve99.net
0.0% 254 0 46.1 48.3 42.7 90.8 5.1 0.9
12. AS1299 las-b24-link.ip.twelve99.net
73.4% 253 185 46.6 49.5 42.6 107.6 10.1 0.0
13. AS1299 atlantic-ic336064-las-b24.ip.twelve99-cust.net
0.0% 253 0 49.9 54.0 48.2 91.2 5.1 0.9
14. AS29838 2607:f108:d01:8::4
0.0% 253 0 47.8 50.4 44.3 71.5 3.9 1.6
15. AS25795 orthanc.ca
0.0% 253 0 50.8 48.4 42.3 83.7 3.6 1.0

Whereas the v4 path heads east to Chicago for a handoff between Shaw and
"Atlantic Metro" (looks like "365 Data Centers
<https://www.peeringdb.com/org/9314>"?) before coming back west:


Packets Pings
Host
Loss% Snt Drop Last Avg Best Wrst StDev Jttr
1. AS??? 10.128.0.65
0.0% 270 0 2.7 2.8 2.0 4.4 0.3 0.7
2. AS6327 70.78.192.1
0.0% 270 0 14.1 14.9 9.0 29.1 2.7 0.1
3. AS6327 rc1bb-be131-1.vc.shawcable.net
0.0% 270 0 15.1 15.5 10.3 36.3 3.1 0.6
4. AS6327 24.244.60.85
0.0% 269 0 21.3 16.9 12.5 35.7 3.3 4.8
5. AS6327 24.83.252.130
27.1% 269 73 17.3 16.8 10.4 54.2 3.7 0.8
6. AS6327 rc1bb-be214.vc.shawcable.net
0.0% 269 0 15.9 16.7 10.7 90.1 5.1 0.9
7. AS6327 rc3so-be6-1.cg.shawcable.net
0.0% 269 0 26.1 28.3 23.1 95.7 4.8 7.9
8. AS6327 66.163.71.42
27.2% 269 73 28.5 28.8 23.7 40.4 2.3 0.2
9. AS6327 rc4ec-be13.il.shawcable.net
0.0% 269 0 57.3 57.2 51.9 77.8 2.7 1.2
10. AS46887 equinix-ch2.cr2.ord1.amcbb.net
0.0% 269 0 54.5 57.7 52.5 94.4 4.4 0.8
11. AS29838 ae1-4.cr1.ord1.atlanticmetro.net
0.0% 269 0 56.4 64.0 51.4 166.3 18.7 0.9
12. AS29838 ve521.cr2.lax0.atlanticmetro.net
0.0% 269 0 109.1 128.4 105.3 472.6 42.3 0.3
13. AS29838 xe-0-0-3-0.cr2.lax2.atlanticmetro.net
0.0% 269 0 107.9 109.8 103.8 166.1 5.3 0.2
14. AS29838 ae3-4.cr2.lax3.atlanticmetro.net
0.0% 269 0 107.1 109.5 105.5 171.7 4.5 9.1
15. AS29838 arpnetworks-dmarc.lax3.atlanticmetro.net
0.0% 269 0 107.9 110.2 105.0 187.6 5.7 0.5
16. AS25795 orthanc.ca
0.0% 269 0 105.8 106.5 100.9 142.7 3.1 0.4

My guess would be the scenic route via Chicago is probably based on some of
"Atlantic Metro's" routing policy on the v4 end, or quite possibly some of
Shaw's "selective" routing policy cutting off a more direct path through
Seattle or e.g. SIX.

--
Hugo Slabbert

I would like to acknowledge that the land on which I work and make my home
is the traditional and unceded territory of Coast Salish Peoples,
specifically the Kwantlen, Semiahmoo, and other Stó:l? Nations.


On Mon, Jan 15, 2024 at 12:29?PM Lyndon Nerenberg (VE7TFX/VE6BBM) via
Outages <outages@outages.org> wrote:

> I didn't have mtr installed on those hosts. The path is definitely
> asym: 17 hops YVR-LAX, 10 hops coming back. But both ways the
> traffic routes through Chicago, giving me those spectacular 100ms
> RTTs :-P
>
> I ran several traceroutes from the YVR side, and they all pointed
> to Calgary as the choke point. But whatever it was has cleared up
> now.
>
> .--lyndon
> _______________________________________________
> Outages mailing list
> Outages@outages.org
> https://puck.nether.net/mailman/listinfo/outages
>