Mailing List Archive

On 2021-09-10 18:27, Owen DeLong wrote:
>
>
>> On Sep 10, 2021, at 01:39 , Jeroen Massar <jeroen@massar.ch> wrote:
>>
>>
>>
>>> On 20210909, at 21:55, Owen DeLong via NANOG <nanog@nanog.org> wrote:
>>>> [..]
>>>> Awful lot of red spots even in the top 100. Hell, even amazon.com
>>>> isn't IPv6 yet. And the long tail is going to be the death of a thousand
>>>> cuts for the call center unless you have a way to deal with those sites.
>>>
>>> This is my point… That is why I think an announcement of “On X date,
>>> we will begin charging extra for IPv4 services and define Internet Access
>>> to be IPv6” by a couple of the larger eyeball ISPs would light a pretty
>>> big fire under those laggards.
>>
>> You mean like: https://docs.hetzner.com/general/others/ipv4-pricing/ ?
>>
>> yes, a /24 was 0 setup, now now close to 5000 of currency.... and the monthly fee also doubled.
>> Noting that 20 for an IPv4 IP is effectively quite cheap with current prices going towards 50+...
>>
>> There are thus already a few places that are doing the squish....
>>
>> Greets,
>> Jeroen
>>
>
> Yes, but it needs to come from a major eyeball ISP to be the motivating
> factor that we need here. A minor virtual server host isn’t going to do it.

"minor virtual server host". I think you are underestimating things...

https://bgp.he.net/AS24940 + AS213230

That is not a toy network... but hey, I guess 'everything is bigger' on
the other side of the big old lake eh.

I am sure, considering the news and rumors, that that pricing change
made, that it made an impact at a lot of companies, that things are
changing, and that is a win for IPv6...

But hey, YMMV.

Greets,
Jeroen

> On Sep 10, 2021, at 13:22 , John Levine <johnl@iecc.com> wrote:
>
> It appears that Owen DeLong via NANOG <owen@delong.com> said:
>> This is my point… That is why I think an announcement of “On X date,
>> we will begin charging extra for IPv4 services and define Internet Access
>> to be IPv6” by a couple of the larger eyeball ISPs would light a pretty
>> big fire under those laggards.
>
> Indeed. They would send postcards to all their customers saying
> "Comcast has said they will cut off your access to Netflix on April 1,
> Call their president's office at 1-800-xxx-xxxx and tell them what you think."

Nope… Netflix is fully available on IPv6 and actually looks forward to ISPs
doing this.

Now Amazon might send post cards saying “Comcast has said that they
will cut off your access to our store…”, but I suspect that the cost of such
a campaign and the collateral damage would be well in excess of the
cost of just adding IPv6 to their service.

Owen

> On Sep 10, 2021, at 14:17 , Jeroen Massar <jeroen@massar.ch> wrote:
>
> On 2021-09-10 18:27, Owen DeLong wrote:
>>> On Sep 10, 2021, at 01:39 , Jeroen Massar <jeroen@massar.ch> wrote:
>>>
>>>
>>>
>>>> On 20210909, at 21:55, Owen DeLong via NANOG <nanog@nanog.org> wrote:
>>>>> [..]
>>>>> Awful lot of red spots even in the top 100. Hell, even amazon.com
>>>>> isn't IPv6 yet. And the long tail is going to be the death of a thousand
>>>>> cuts for the call center unless you have a way to deal with those sites.
>>>>
>>>> This is my point… That is why I think an announcement of “On X date,
>>>> we will begin charging extra for IPv4 services and define Internet Access
>>>> to be IPv6” by a couple of the larger eyeball ISPs would light a pretty
>>>> big fire under those laggards.
>>>
>>> You mean like: https://docs.hetzner.com/general/others/ipv4-pricing/ ?
>>>
>>> yes, a /24 was 0 setup, now now close to 5000 of currency.... and the monthly fee also doubled.
>>> Noting that 20 for an IPv4 IP is effectively quite cheap with current prices going towards 50+...
>>>
>>> There are thus already a few places that are doing the squish....
>>>
>>> Greets,
>>> Jeroen
>>>
>> Yes, but it needs to come from a major eyeball ISP to be the motivating
>> factor that we need here. A minor virtual server host isn’t going to do it.
>
> "minor virtual server host". I think you are underestimating things...
>
> https://bgp.he.net/AS24940 + AS213230
>
> That is not a toy network... but hey, I guess 'everything is bigger' on the other side of the big old lake eh.

Not everything, but compare to (e.g. 15169, 16509).

Also, the key here is more that we need large eyeballs, such as 7922, 21928, 6167, 6298, etc.).

I wasn’t meaning to disparage Hetzner in any way, but they’re a cloud host, not an eyeball provider. While they’re not a toy network, I also wouldn’t call them a major cloud provider in the same league with (e.g. AWS, Azure, Google Cloud).

> I am sure, considering the news and rumors, that that pricing change made, that it made an impact at a lot of companies, that things are changing, and that is a win for IPv6...

Agreed… I’m glad to see what they’ve done, but I’m advocating for someone like Comcast to follow suit in hopes that it will drive other large content providers to make the move.

Owen

>> Indeed. They would send postcards to all their customers saying
>> "Comcast has said they will cut off your access to Netflix on April 1,
>> Call their president's office at 1-800-xxx-xxxx and tell them what you think."
>
> Nope… Netflix is fully available on IPv6 and actually looks forward to ISPs
> doing this.

OK, then Disney+ or Hulu or whoever. Peering wars never end well. Don't
even need postcards, just stick the flyer in with the bill.

> Now Amazon might send post cards saying “Comcast has said that they
> will cut off your access to our store…”, but I suspect that the cost of such
> a campaign and the collateral damage would be well in excess of the
> cost of just adding IPv6 to their service.

AWS has perfectly good support for IPv6 so they must have business reasons
to stick with v4. But I expect the cost of putting up banners on the
homepage saying to call your ISP, for people coming through the relevant
ISP, would be pretty cheap, and Amazon's customers appear to like their
accounts and their Prime quite a lot. Again, peering wars never end well.

This ignores the question of what the advantage to the ISP would be other
than bloody-mindedness. The world is not going to abandon IPv4 any time
soon and the last time I looked, the cost of routing packets is the same
regardless of which header they have.

Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

> On Sep 8, 2021, at 1:26 AM, Bjørn Mork <bjorn@mork.no> wrote:
>
>
> I started wondering if there are areas where we can disable IPv4 today.

There are places in which it is disabled today, for at least some services.

Sent using a machine that autocorrects in interesting ways...

> On Sep 8, 2021, at 1:31 AM, Saku Ytti <saku@ytti.fi> wrote:
>
> If the mid size eyeballs knew ipv4 is going away in 10, 15, 20 years
> whichever it is, then they'd of course have to start moving too,
> because no upstream.

And they would fight it tooth and nail, just like they do now, and if they found an address they could NAT to, they would argue that that one address gave them the ability to avoid the transition -just like they do now.

> On Sep 11, 2021, at 9:04 PM, Fred Baker <fredbaker.ietf@gmail.com> wrote:
>
>
> Sent using a machine that autocorrects in interesting ways...
>
>> On Sep 8, 2021, at 1:31 AM, Saku Ytti <saku@ytti.fi> wrote:
>>
>> If the mid size eyeballs knew ipv4 is going away in 10, 15, 20 years
>> whichever it is, then they'd of course have to start moving too,
>> because no upstream.
>
> And they would fight it tooth and nail, just like they do now, and if they found an address they could NAT to, they would argue that that one address gave them the ability to avoid the transition -just like they do now.

Speaking for the smaller providers, there is enough of the Internet that is only accessible via IPv4 out there that CGN solutions are a reasonable way to manage the situation. There is also enough legacy equipment out there that doesn’t accommodate IPv6 that this process will still take several decades.

Edicts never work. More carrot, less stick.

>> If the mid size eyeballs knew ipv4 is going away in 10, 15, 20 years
>> whichever it is, then they'd of course have to start moving too,
>> because no upstream.
>
> And they would fight it tooth and nail, just like they do now

you speak as if it was religion or they are bad people. neither is the
case. it is what they see as their best business interest.

being from the pacific northwest, i have learned not to try to push
water uphill. so, until we can tilt the hill, it's probably bast for
one's health if one gets over it.

randy

> Edicts never work. More carrot, less stick.

but the ipv6 stick has worked so well over the last 25 years

> On Sep 12, 2021, at 11:35 , Brian Johnson <brian.johnson@netgeek.us> wrote:
>
>
>
>> On Sep 11, 2021, at 9:04 PM, Fred Baker <fredbaker.ietf@gmail.com> wrote:
>>
>>
>> Sent using a machine that autocorrects in interesting ways...
>>
>>> On Sep 8, 2021, at 1:31 AM, Saku Ytti <saku@ytti.fi> wrote:
>>>
>>> If the mid size eyeballs knew ipv4 is going away in 10, 15, 20 years
>>> whichever it is, then they'd of course have to start moving too,
>>> because no upstream.
>>
>> And they would fight it tooth and nail, just like they do now, and if they found an address they could NAT to, they would argue that that one address gave them the ability to avoid the transition -just like they do now.
>
> Speaking for the smaller providers, there is enough of the Internet that is only accessible via IPv4 out there that CGN solutions are a reasonable way to manage the situation. There is also enough legacy equipment out there that doesn’t accommodate IPv6 that this process will still take several decades.
>
> Edicts never work. More carrot, less stick.

They did with ATSC.

Owen

On 9/12/21 1:08 PM, Randy Bush wrote:
>>> If the mid size eyeballs knew ipv4 is going away in 10, 15, 20 years
>>> whichever it is, then they'd of course have to start moving too,
>>> because no upstream.
>> And they would fight it tooth and nail, just like they do now
> you speak as if it was religion or they are bad people. neither is the
> case. it is what they see as their best business interest.
>
> being from the pacific northwest, i have learned not to try to push
> water uphill. so, until we can tilt the hill, it's probably bast for
> one's health if one gets over it.
>
If vendors actually cared they could make the CGNAT's and other hacks
ridiculously buggy and really expensive to deploy and maintain. I doubt
many vendors were chomping at the bit to support CGNAT and are probably
wondering what fresh hell is next to keep ipv4 limping along.

Mike

> I doubt many vendors were chomping at the bit to support CGNAT

definitely. they hate to sell big expensive boxes.

randy

On 9/12/21 4:59 PM, Randy Bush wrote:
>> I doubt many vendors were chomping at the bit to support CGNAT
> definitely. they hate to sell big expensive boxes.

Back in the early 2000's the first rumblings of what would eventually
turn into CGN started popping up at Cablelabs. I went to the EVP of
Service Provider and basically told him that he had a choice between
that mess or developing ipv6. I doubt he was interested in doing
anything at all, but he chose ipv6, at least in the abstract. Steve
Deering and I then went around to all of the BU's trying to figure out
what it would take for them to implement ipv6 in the routing plane.
Cablelabs was also pretty ipv6-focused too making a similar calculation.

So no, they weren't interested in it either. They were completely driven
by what the providers wanted and what a large group of providers have
since made pretty clear is that horrible hacks are fine by them if it
gets them out of a short term bind. But it's hardly uniform across the
industry. This is a classic reverse-tragedy of the commons.

Mike

On 9/13/21 01:00, Michael Thomas wrote:

>
> If vendors actually cared they could make the CGNAT's and other hacks
> ridiculously buggy and really expensive to deploy and maintain. I
> doubt many vendors were chomping at the bit to support CGNAT and are
> probably wondering what fresh hell is next to keep ipv4 limping along.
>

10's of millions of dollars being spent by African mobile operators,
every year, in CG-NAT hardware and licenses, with the vendors.

They will make sure that code is bug-free :-).

Mark.

On 9/13/21 02:16, Michael Thomas wrote:

>
> But it's hardly uniform across the industry. This is a classic
> reverse-tragedy of the commons.

The problem is it's uniform in the corners that contain scale and the
money to make a difference at vendor-land.

7 million mom & pop ISP's vs. 10 large mobile operators... yeah, $vendor
will struggle to make that decision :-).

Mark.

> On Sep 13, 2021, at 05:17 , Mark Tinka <mark@tinka.africa> wrote:
>
>
>
> On 9/13/21 01:00, Michael Thomas wrote:
>
>>
>> If vendors actually cared they could make the CGNAT's and other hacks ridiculously buggy and really expensive to deploy and maintain. I doubt many vendors were chomping at the bit to support CGNAT and are probably wondering what fresh hell is next to keep ipv4 limping along.
>>
>
> 10's of millions of dollars being spent by African mobile operators, every year, in CG-NAT hardware and licenses, with the vendors.
>
> They will make sure that code is bug-free :-).

ROFLMAO

100s of millions of dollars are spent every year on major backbone router kit. That code has never been bug free.

Your assertion is provably false.

Owen

< rant >

ipv6 was designed at a time where the internet futurists/idealists had
disdain for operators and vendors, and thought we were evil money
grabbers who had to be brought under control.

the specs as originally RFCed by the ietf is very telling. for your
amusement, take a look at rfc 2450. it took five years of war to get
rid of the tla/sla crap. and look at the /64 religion today[0].

real compatibility with ipv4 was disdained. the transition plan was
dual stack and v4 would go away in a handful of years. the 93
transition mechanisms were desperate add-ons when v4 did not go away.
and dual stack does not scale, as it requires v4 space proportional to
deployed v6 space.

we are left to make the mess work for the users, while being excoriated
for not doing it quickly or well enough, and for trying to make ends
meet financially.

randy, trying to deal with the mess since the early '90s

---

[0] https://datatracker.ietf.org/doc/html/draft-bourbaki-6man-classless-ipv6-05

Randy Bush wrote on 13/09/2021 19:22:
> the specs as originally RFCed by the ietf is very telling. for your
> amusement, take a look at rfc 2450. it took five years of war to get
> rid of the tla/sla crap. and look at the /64 religion today[0].

architectural decisions were made because of a mixture of actual and
perceived problems at the time, but several of the outcomes make little
sense now or in some cases, actively cause problems. E.g. using mcast
for address resolution because large flat l2 networks were the order of
the day, that "privacy addresses" would give privacy, that client
self-selected addresses should be the only game in town for
auto-addressing (it took years to get any form of dhcp through), that
extension headers were a great idea, that "transition mechanisms" would
be viable for fundamentally incompatible protocols, etc. That said,
it's easy to be critical of design decisions with 25y of hindsight, and
even easier to understate how difficult it is to dislodge ipv4 which
took 40 years of evolution to cement itself into its current position.

Nick

> it's easy to be critical of design decisions with 25y of hindsight

there was a good number of senior implementors and ops who screamed
loudly at the time. to no avail.

randy

On Mon, Sep 13, 2021 at 8:22 PM Randy Bush <randy@psg.com> wrote:

> real compatibility with ipv4 was disdained. the transition plan was
> dual stack and v4 would go away in a handful of years. the 93
> transition mechanisms were desperate add-ons when v4 did not go away.
> and dual stack does not scale, as it requires v4 space proportional to
> deployed v6 space.
>

What I find most peculiar about this whole rant (not just yours but the
whole thread) is that I may be the only one who found implementing IPv6
with dual stack completely trivial and a non issue? There is no scale issue
nor any of the other rubbish.

Some say what we have is not a true dual stack setup because we run MPLS
and the IPv6 mostly lives inside L2VPN or L3VPN tunnels. Most of our
network gear has no idea what an IPv6 address is. But neither does that
equipment touch the public IPv4 internet. Nevertheless we configure our
IPv4 and IPv6 both only on our few edge Juniper MX devices and that is it.
I do not believe IPv6 has 100 config lines in my network total.

For all I care we already have a perfect working system with IPv4+CGN+IPv6.
The CGN part was the most troublesome, not the IPv6.

Regards,

Baldur

On 9/13/21 11:22 AM, Randy Bush wrote:
> < rant >
>
> ipv6 was designed at a time where the internet futurists/idealists had
> disdain for operators and vendors, and thought we were evil money
> grabbers who had to be brought under control.
>
> the specs as originally RFCed by the ietf is very telling. for your
> amusement, take a look at rfc 2450. it took five years of war to get
> rid of the tla/sla crap. and look at the /64 religion today[0].
>
> real compatibility with ipv4 was disdained. the transition plan was
> dual stack and v4 would go away in a handful of years. the 93
> transition mechanisms were desperate add-ons when v4 did not go away.
> and dual stack does not scale, as it requires v4 space proportional to
> deployed v6 space.
>
> we are left to make the mess work for the users, while being excoriated
> for not doing it quickly or well enough, and for trying to make ends
> meet financially.
>
This is really easy to say in hindsight. 30 years ago it wasn't even
vaguely a given that the Internet would even win and the size of the IP
universe was still tiny. The main problem is that the internet was a
classic success disaster where you're going as fast as possible and
falling farther and farther behind. All of the gripes about particulars
strike me as utterly irrelevant in the global scheme of things. As I
mentioned, if they did nothing more than bolted on two more address
bytes it still would have been just as impossible to get vendors and
providers to care because everybody was heads down trying to deal with
the success disaster. It's really easy to say that ipv6 suffers from
second system syndrome -- which it does -- but that doesn't provide any
concrete strategy for what would have been "better" in both getting
vendors and providers to care. None of them wanted to do anything other
than crank out kit that could be sold in the here and now that providers
were willing to buy. That was certainly my experience at Cisco. As I
said, the exec I talked to didn't actually want to do anything at all
but was willing to let a couple of engineers navel gaze if it gave him
something to talk about were the subject to actually come up with
customers and a bludgeon against 3COM (iirc) at the time.

None of this is technical. It was which short term hack is going to keep
the gravy train flowing? I was a developer at the time keeping an eye on
the drafts as they were coming out. They didn't strike me was overly
difficult to implement nor did they strike me as particularly
overwrought. From a host standpoint, i didn't think it would take too
much effort to get something up and running but I waited until somebody
started asking for it. That never came. Nothing ever came. Then NAT's
came and kicked the can down the roads some more. Now we have mega-yacht
NAT's to kick it down the road even farther. Tell us what else would
have prevented that?

Mike

In resource challenged regions we have been using IPv4+CGN+IPv6 dual stack for the last ten or so years. For 20K subs you can use one /24 of ipv4 and a /40 or so of ipv6. There have been available RGW’s and sufficient vendor support throughout this time. The only issues I have ever really seen have been with Sony PSN doing random ipv4 /32 blocks. Apart from that it has been working just fine. Over 50% of traffic now flows on the V6 side and in general end customers have no clue, it just works.



>For all I care we already have a perfect working system with IPv4+CGN+IPv6. The CGN part was the most troublesome, not the IPv6.

Baldur Norddahl wrote:
>
>
> On Mon, Sep 13, 2021 at 8:22 PM Randy Bush <randy@psg.com
> <mailto:randy@psg.com>> wrote:
>
> real compatibility with ipv4 was disdained. the transition plan was
> dual stack and v4 would go away in a handful of years. the 93
> transition mechanisms were desperate add-ons when v4 did not go away.
> and dual stack does not scale, as it requires v4 space proportional to
> deployed v6 space.
>
>
> What I find most peculiar about this whole rant (not just yours but
> the whole thread) is that I may be the only one who found implementing
> IPv6 with dual stack completely trivial and a non issue? There is no
> scale issue nor any of the other rubbish.
>
> Baldur
>
The essential point is that your dual stack is barely relevant until
every stack is dual. Which is impossible without CGNAT.

It also turns out that is barely relevant how easy it may have been for you.

Joe

On 9/13/21 2:52 PM, Baldur Norddahl wrote:
>
>
> On Mon, Sep 13, 2021 at 8:22 PM Randy Bush <randy@psg.com
> <mailto:randy@psg.com>> wrote:
>
> real compatibility with ipv4 was disdained.  the transition plan was
> dual stack and v4 would go away in a handful of years.  the 93
> transition mechanisms were desperate add-ons when v4 did not go away.
> and dual stack does not scale, as it requires v4 space proportional to
> deployed v6 space.
>
>
> What I find most peculiar about this whole rant (not just yours but
> the whole thread) is that I may be the only one who found implementing
> IPv6 with dual stack completely trivial and a non issue? There is no
> scale issue nor any of the other rubbish.

I agree on the host side. It didn't even occur to me at the time I was
looking at it that it would be any sort of issue -- we had all kinds of
other protocols on our boxes like SMB, Netware, DEC LAT, etc. We would
have done it if customers told us they wanted it, just like we
implemented ACL's not realizing why they were especially important. Back
in the early days all routing was done in software so it wouldn't have
been hard to squeeze v6 in. All of that changed when the forwarding
plane got cast in silicon though which made it far, far more difficult
to get anybody to stick their necks out vs a skunk works software
project. But before that it would have been completely doable if
somebody was willing to throw money at it.

Mike

On 13.09.21 20:22, Randy Bush wrote:
> < rant >

> ipv6 was designed at a time where the internet futurists/idealists had
> disdain for operators and vendors, and thought we were evil money
> grabbers who had to be brought under control.

and...

> <snip>

> real compatibility with ipv4 was disdained.


I'm not claiming IPv6 is any sort of perfect, but let's not revise history.

To quote Lee Hayes' adaptation of Will Rogers:

“Things aren't what they used to be, what's more they never were.”

There were four proposals for the IPng:

* NIMROD, PIP, SIP, and TUBA

SIP was the one that was chosen, supported by endpoint manufacturers
such as Sun and SGI, and it was the MOST compatible.  Operators and
router manufacturers at the time pushed TUBA, which was considerably
less compatible with the concepts used in v4 because of variable length
addressing.   If we endpoints had some notion that v6 would take as long
as it has to diffuse, perhaps we all might have thought differently. I
don't know.

There is no evidence that any other design choices on the table at the
time would have gotten us transitioned any faster, and a lot of evidence
and analysis that the exact opposite is more likely.  There are many
reasons v6 has taken this long to deploy, but one prediction model
(Elmore/Camp) dating back to 2008 have said it would take to the end of
the century to get to 80%, barring a paradigm shift.  We may have that
paradigm shift with IoT, but the jury is still out.

Eliot