On 2014-11-09 20:18, Job Snijders wrote:
> On Sun, Nov 09, 2014 at 08:03:01PM +0100, Jeroen Massar wrote:
>>> No. I feel that 250+ successes vs 10 failures is enough to conclude
>>> that Akamai and Google are *not* universally broken, far from it.
>>
>> Testing from colod boxes on well behaved networks (otherwise they would
>> not know or be part of the RING), while the problem lies with actual
>> home users is quite a difference.
>
> I can't comment on the validaty of the tests performed, but I'd like to
> point out one thing: I like that the NLNOG RING is very diverse,
> especially in terms of the node's IPv6 connectivity.
As they are distributed amongst a large rang of ISPs, that is correct.
But that primarily affects routing paths, not their link types.
> Some hosts are behind exotic 6to4 NATted tunnels,
I am a bit surprised by such a statement, or the need for it, everybody
knows of the positive value of the one true RING. But most of these
nodes nicely sitting on colod boxes and thus while it excludes any
problem in that setup for that single query made, it does not exclude
the problems reported.
Also making a claim like that they are 6to4 that is known to be false,
is a bit weird as there is no 2002::/16 address in the participants hosts:
$ wget https://ring.nlnog.net/images/ring-logos/participants.js
..
$ (for i in `cat participants.js |grep http |grep -v ris.ripe.net | cut
-f8 -d\' | sed 's/,//g'`; do dig +short aaaa $i.ring.nlnog.net; done) >>
aaaa
$ grep 2002: aaaa
2607:f0d0:2002:6e::2
The below MTU tracepaths also do not reveal anything in 2002::/16
And also "6to4 NAT" does not work unless you hack a kernel AND the NAT
box to understand that they need to use fake addresses in those 6in4
packets. Unless you mean "The NAT is terminated on a CPE that does NAT
and 6to4 but forwards packets to the CPE's internal network". In that
case though there is no NAT involved.
Hence, I can only assume that you meant "6in4" instead of "6to4", common
mistake it seems. But lets not confuse the two as they are totally
dislike from a "working" perspective.
But lets test MTU according to PMTU on those nodes:
(for i in `cat aaaa`; do echo "tracepath6 $i"; tracepath6 -n $i; done)
>>mtu.txt
See attached.
Hmm, indeed what is odd about those traces is the "hops xx, back
xx+~40", indicating quite some asymmetric routing; but doing tracepath6s
from two sides indicates the same on both sides of a trace, something to
dig into one day.
Except for:
tracepath6 2a02:78:d443:8314::1
Resume: pmtu 1480 hops 11 back 53
tracepath6 2001:4830:123:1::d85d:f0b3
Resume: pmtu 1480 hops 18 back 52
tracepath6 2001:4c48:2:71::ffff
Resume: pmtu 1476 hops 13 back 54
tracepath6 2001:67c:105c:800::2
Resume: pmtu 1480 hops 14 back 54
tracepath6 2001:4830:ed::2
Resume: pmtu 1480 hops 16 back 51
Also, broken pMTU/traceroute for:
2a02:58:3:110::23:1
2a01:310:8312:1001::19
2a00:1f00:dc06:11::10
2001:48c8:3:2::2
2607:fcc0:2:1:208:70:247:50
2001:67c:2274:4021::101
2001:67c:2814:60::7c:1
2001:4ce8:c134:1019::1 <--- routing loop
2001:8d8:580:400::1:0
2a03:3400:3:2501::150:255
2a00:1798:fff0::4
2001:4b28:198:0:21c:42ff:fe2a:d38a <-- intermediary nodes broken
2a01:528:2000:0:21c:42ff:fe4a:c3a0 <-- "" (same network it seems)
2001:500:a:500::229
2a01:8e40:1:12::2
2a03:b980:c037::1
2a02:e0:1::2
2001:12f0:500:1dc::1916:12 <-- routing loop
2001:67c:2a0:8::beef <-- intermediary nodes broken
2405:fc00::a001
2a02:74c0::232:52
2620:0:2d0:203::158
2620:0:2830:203::158
2a02:c10:3f00:5::2
2a02:c10:3f00:7::2
2001:fb0:100:ffff:211:25ff:fe40:9468 <-- intermediates broken
2a00:1c18:0:101::5
2a00:59c0:1:3::130
2a02:480:3411:6::2
etc...
Actually, what that demonstrates is that out of a set of well-managed
nodes, there are issues with PMTU. Hence, please contact these networks
(they are part of the ring, thus that should be easy) and make them
behave properly. That solves another set of problems.
>From the above, it is actually pure magic that the majority has working
connectivity. Now, the above nodes likely are not affected by PMTUD as
they are likely on full 1500-MTU networks...
You might want to run a similar one from all the nodes, thus making a
cross-RING MTU determination to get even better results, as I did it
only from one vantage point. Maybe time for ring-mtu? :)
And of course, contacting those nodes as they obviously have broken
connectivity; as you have the contacts directly, it should be quick to
get resolved.
> others behind regular
> tunnels, some inadvertently block useful ICMPv6 messages, some networks
> are just broken.
Yes, there where 10 broken nodes in the list. but apparently quite a few
more nodes have issues.
> For NLNOG RING applications we mandate that there is 1 globally unique
> IPv6 address on the host, we do not specify how this should be
> accomplished. This leads to some variety, not all of those
> implementations I would describe as "well behaved".
While that is absolutely true, most of those boxes ARE well behaved, as
the providers involved will take sure that there are no problems.
And they definitely are not located in an access network on a dingy home
DSL line...
Greets,
Jeroen
> On Sun, Nov 09, 2014 at 08:03:01PM +0100, Jeroen Massar wrote:
>>> No. I feel that 250+ successes vs 10 failures is enough to conclude
>>> that Akamai and Google are *not* universally broken, far from it.
>>
>> Testing from colod boxes on well behaved networks (otherwise they would
>> not know or be part of the RING), while the problem lies with actual
>> home users is quite a difference.
>
> I can't comment on the validaty of the tests performed, but I'd like to
> point out one thing: I like that the NLNOG RING is very diverse,
> especially in terms of the node's IPv6 connectivity.
As they are distributed amongst a large rang of ISPs, that is correct.
But that primarily affects routing paths, not their link types.
> Some hosts are behind exotic 6to4 NATted tunnels,
I am a bit surprised by such a statement, or the need for it, everybody
knows of the positive value of the one true RING. But most of these
nodes nicely sitting on colod boxes and thus while it excludes any
problem in that setup for that single query made, it does not exclude
the problems reported.
Also making a claim like that they are 6to4 that is known to be false,
is a bit weird as there is no 2002::/16 address in the participants hosts:
$ wget https://ring.nlnog.net/images/ring-logos/participants.js
..
$ (for i in `cat participants.js |grep http |grep -v ris.ripe.net | cut
-f8 -d\' | sed 's/,//g'`; do dig +short aaaa $i.ring.nlnog.net; done) >>
aaaa
$ grep 2002: aaaa
2607:f0d0:2002:6e::2
The below MTU tracepaths also do not reveal anything in 2002::/16
And also "6to4 NAT" does not work unless you hack a kernel AND the NAT
box to understand that they need to use fake addresses in those 6in4
packets. Unless you mean "The NAT is terminated on a CPE that does NAT
and 6to4 but forwards packets to the CPE's internal network". In that
case though there is no NAT involved.
Hence, I can only assume that you meant "6in4" instead of "6to4", common
mistake it seems. But lets not confuse the two as they are totally
dislike from a "working" perspective.
But lets test MTU according to PMTU on those nodes:
(for i in `cat aaaa`; do echo "tracepath6 $i"; tracepath6 -n $i; done)
>>mtu.txt
See attached.
Hmm, indeed what is odd about those traces is the "hops xx, back
xx+~40", indicating quite some asymmetric routing; but doing tracepath6s
from two sides indicates the same on both sides of a trace, something to
dig into one day.
Except for:
tracepath6 2a02:78:d443:8314::1
Resume: pmtu 1480 hops 11 back 53
tracepath6 2001:4830:123:1::d85d:f0b3
Resume: pmtu 1480 hops 18 back 52
tracepath6 2001:4c48:2:71::ffff
Resume: pmtu 1476 hops 13 back 54
tracepath6 2001:67c:105c:800::2
Resume: pmtu 1480 hops 14 back 54
tracepath6 2001:4830:ed::2
Resume: pmtu 1480 hops 16 back 51
Also, broken pMTU/traceroute for:
2a02:58:3:110::23:1
2a01:310:8312:1001::19
2a00:1f00:dc06:11::10
2001:48c8:3:2::2
2607:fcc0:2:1:208:70:247:50
2001:67c:2274:4021::101
2001:67c:2814:60::7c:1
2001:4ce8:c134:1019::1 <--- routing loop
2001:8d8:580:400::1:0
2a03:3400:3:2501::150:255
2a00:1798:fff0::4
2001:4b28:198:0:21c:42ff:fe2a:d38a <-- intermediary nodes broken
2a01:528:2000:0:21c:42ff:fe4a:c3a0 <-- "" (same network it seems)
2001:500:a:500::229
2a01:8e40:1:12::2
2a03:b980:c037::1
2a02:e0:1::2
2001:12f0:500:1dc::1916:12 <-- routing loop
2001:67c:2a0:8::beef <-- intermediary nodes broken
2405:fc00::a001
2a02:74c0::232:52
2620:0:2d0:203::158
2620:0:2830:203::158
2a02:c10:3f00:5::2
2a02:c10:3f00:7::2
2001:fb0:100:ffff:211:25ff:fe40:9468 <-- intermediates broken
2a00:1c18:0:101::5
2a00:59c0:1:3::130
2a02:480:3411:6::2
etc...
Actually, what that demonstrates is that out of a set of well-managed
nodes, there are issues with PMTU. Hence, please contact these networks
(they are part of the ring, thus that should be easy) and make them
behave properly. That solves another set of problems.
>From the above, it is actually pure magic that the majority has working
connectivity. Now, the above nodes likely are not affected by PMTUD as
they are likely on full 1500-MTU networks...
You might want to run a similar one from all the nodes, thus making a
cross-RING MTU determination to get even better results, as I did it
only from one vantage point. Maybe time for ring-mtu? :)
And of course, contacting those nodes as they obviously have broken
connectivity; as you have the contacts directly, it should be quick to
get resolved.
> others behind regular
> tunnels, some inadvertently block useful ICMPv6 messages, some networks
> are just broken.
Yes, there where 10 broken nodes in the list. but apparently quite a few
more nodes have issues.
> For NLNOG RING applications we mandate that there is 1 globally unique
> IPv6 address on the host, we do not specify how this should be
> accomplished. This leads to some variety, not all of those
> implementations I would describe as "well behaved".
While that is absolutely true, most of those boxes ARE well behaved, as
the providers involved will take sure that there are no problems.
And they definitely are not located in an access network on a dingy home
DSL line...
Greets,
Jeroen
Attached Files:
-
mtu.txt (307 KB)
