Mailing List Archive

On Tue, Nov 26, 2019 at 12:15 AM Sabri Berisha <sabri@cluecentral.net>
wrote:

> ----- On Nov 26, 2019, at 1:36 AM, Doug Barton dougb@dougbarton.us wrote:
>
> > I get that some people still don't like it, but the answer is IPv6. Or,
> > folks can keep playing NAT games, etc. But one wonders at what point
> > rolling out IPv6 costs less than all the fun you get with [CG]NAT.
>
> When the MBAs start realizing the risk of not deploying it.
>

Hey, i have an mba. That and $5 will get me cup of coffee.


> I have some inside knowledge about the IPv6 efforts of a large eyeball
> network.


Me too.

In that particular case, the cost of deploying IPv6 internally is not
> simply configuring it on the network gear; that has already been done. The
> cost of fully supporting IPv6 includes (but is probably not limited to):
>
> - Support for deploying IPv6 across more than 20 different teams;


Wow. I support 80M mobile subscribers, 90% of which are ipv6-only. I
think 20 people in the company can spell ipv6, but somehow you need 20
teams.... how many teams speak ipv4 ?


> - Modifying old (ancient) internal code;


Ancient in 2019 means what? Is this code not in security compliance ?


> - Modifying old (ancient) database structures (think 16 character fields
> for IP addresses);


Hash 128 bits into 240/4 is how i heard Google handled it early on


> - Upgrading/replacing load balancers and other legacy crap that only
> support IPv4 (yeah, they still exist);


Again, with all the CVEs, this code is always moving in the real world.


> - Modifying the countless home-grown tools that automate firewalls etc;


Home grown means it can be fixed instead of replaced.


> - Auditing the PCI infrastructure to ensure it is still compliant after
> deploying IPv6;
>

Ah, so you are keeping up with compliance / cve and are upgrading at
regular intervals?



> If it was as simple as upgrading a few IP stacks here and there, it would
> be a non-issue.
>

Usually is just a few edge stacks to start and scale the edge


> Don't get me wrong, I'm not advocating against IPv6 deployment; on the
> contrary. But it is not that simple in the real corporate world. Execs have
> bonus targets.


Why would an exec care? Ipv6 is just normal work like ipv4.

IPv6 is not yet important enough to become part of that bonus target:


The bonus target was normal business continuity planning... in 2008. Sorry
you missed that one. Here you go, just put 1 in 2009 to make it 2019 so
you dont look so bad

https://www.arin.net/vault/knowledge/about_resources/ceo_letter.pdf


there is no ROI at this point. In this kind of environment there needs to
> be a strong case to invest the capex to support IPv6.
>
> IPv6 must be supported on the CxO level in order to be deployed.
>
> Thanks,
>
> Sabri, (Badum tsss) MBA


I see....well let me translate it you MBA-eese for you:

FANG deployed ipv6 nearly 10 years ago. Since deploying ipv6, the cohort
experienced 300% CAGR. Also, everything is mobile, and all mobile providers
in the usa offer ipv6 by default in most cases. Latency! Scale! As your
company launches its digital transformation iot 2020 virtualization
container initiatives, ipv6 will be an integral part of staying relevant on
the blockchain. Also, FANG did it nearly 10 years ago. Big content and
big eyeballs are on ipv6, ipv4 is a winnowing longtail of irrelevance and
iot botnets.


>

Top posting...

---------------------------------
:: But it is not that simple in the real corporate world.
:: Execs have bonus targets.

Why would an exec care? Ipv6 is just normal work like ipv4.
---------------------------------

No, you have to make purchases and have folks across the
company do work to get everything going. Refocusing folks
work on deploying IPv6 to *everything* (rather than, say,
getting that shiny new Nokia 7750 deployed so we can sell
more services) costs money. Ancient boxen are out here
and don't support aye pee vee six well or at all. Getting
ones that do costs money. Training lower level folks takes
them away from their current work and costs money. Etc.

:: Ancient in 2019 means what? Is this code not in security
:: compliance ?

I recently started back with a company after being gone nine
years. My code was still running and no one in neteng had
the knowledge of how to do anything with it much less to try
to write in IPv6 sections. To take an SA and look into the
networking code I wrote takes them away from things they
need to do to sell services. That costs money.

What Sabri wrote hit home here. Folks are not looking into
it and will wait until forced to do so. Then said companies
will be behind the ball in a big way, but that it what it is
here and in the other companies I worked for.

A lot of this read to me as flippant. You don't seem to be
willing to listen to those of us out here on the raggedy
edges. I've said what Sabri said at least a few times on this
list.

scott





--- cb.list6@gmail.com wrote:

From: Ca By <cb.list6@gmail.com>
To: Sabri Berisha <sabri@cluecentral.net>
Cc: nanog <nanog@nanog.org>
Subject: Re: RIPE our of IPv4
Date: Tue, 26 Nov 2019 15:11:40 -0800

On Tue, Nov 26, 2019 at 12:15 AM Sabri Berisha <sabri@cluecentral.net>
wrote:

> ----- On Nov 26, 2019, at 1:36 AM, Doug Barton dougb@dougbarton.us wrote:
>
> > I get that some people still don't like it, but the answer is IPv6. Or,
> > folks can keep playing NAT games, etc. But one wonders at what point
> > rolling out IPv6 costs less than all the fun you get with [CG]NAT.
>
> When the MBAs start realizing the risk of not deploying it.
>

Hey, i have an mba. That and $5 will get me cup of coffee.


> I have some inside knowledge about the IPv6 efforts of a large eyeball
> network.


Me too.

In that particular case, the cost of deploying IPv6 internally is not
> simply configuring it on the network gear; that has already been done. The
> cost of fully supporting IPv6 includes (but is probably not limited to):
>
> - Support for deploying IPv6 across more than 20 different teams;


Wow. I support 80M mobile subscribers, 90% of which are ipv6-only. I
think 20 people in the company can spell ipv6, but somehow you need 20
teams.... how many teams speak ipv4 ?


> - Modifying old (ancient) internal code;


Ancient in 2019 means what? Is this code not in security compliance ?


> - Modifying old (ancient) database structures (think 16 character fields
> for IP addresses);


Hash 128 bits into 240/4 is how i heard Google handled it early on


> - Upgrading/replacing load balancers and other legacy crap that only
> support IPv4 (yeah, they still exist);


Again, with all the CVEs, this code is always moving in the real world.


> - Modifying the countless home-grown tools that automate firewalls etc;


Home grown means it can be fixed instead of replaced.


> - Auditing the PCI infrastructure to ensure it is still compliant after
> deploying IPv6;
>

Ah, so you are keeping up with compliance / cve and are upgrading at
regular intervals?



> If it was as simple as upgrading a few IP stacks here and there, it would
> be a non-issue.
>

Usually is just a few edge stacks to start and scale the edge


> Don't get me wrong, I'm not advocating against IPv6 deployment; on the
> contrary. But it is not that simple in the real corporate world. Execs have
> bonus targets.


Why would an exec care? Ipv6 is just normal work like ipv4.

IPv6 is not yet important enough to become part of that bonus target:


The bonus target was normal business continuity planning... in 2008. Sorry
you missed that one. Here you go, just put 1 in 2009 to make it 2019 so
you dont look so bad

https://www.arin.net/vault/knowledge/about_resources/ceo_letter.pdf


there is no ROI at this point. In this kind of environment there needs to
> be a strong case to invest the capex to support IPv6.
>
> IPv6 must be supported on the CxO level in order to be deployed.
>
> Thanks,
>
> Sabri, (Badum tsss) MBA


I see....well let me translate it you MBA-eese for you:

FANG deployed ipv6 nearly 10 years ago. Since deploying ipv6, the cohort
experienced 300% CAGR. Also, everything is mobile, and all mobile providers
in the usa offer ipv6 by default in most cases. Latency! Scale! As your
company launches its digital transformation iot 2020 virtualization
container initiatives, ipv6 will be an integral part of staying relevant on
the blockchain. Also, FANG did it nearly 10 years ago. Big content and
big eyeballs are on ipv6, ipv4 is a winnowing longtail of irrelevance and
iot botnets.


>

On Tue, Nov 26, 2019 at 05:26:44PM -0500, bzs@theworld.com wrote:
> If the commitment really was to spread IPv6 far and wide IPv6 blocks
> would be handed out for free, one per qualified customer (e.g., if you
> have an IPv4 allocation you get one IPv6 block free), or perhaps some
> trivial administrative fee like $10 per year.

It has been some time since I had to deal with RIRs directly, but my
understanding was that if you had an IPv4 allocation, you got a reasonably
sized chunk of IPv6 alongside for free. Not even an extra $10/year. FREE!

Looking at ARIN's fee schedule
(https://www.arin.net/resources/fees/fee_schedule/), it does seem like that
is still the case:

> For organizations holding both ARIN-issued IPv4 and IPv6 allocations, the
> fee is based on the larger of the two service categories.

So you only need to pay extra for your IPv6 numbers if you've got a lot more
of them than you've got IPv4. The only situation in which I could imagine
that happening is if you were a (very) late-start eyeball network that had a
tiny IPv4 allocation (and a *lot* of CGNAT), but were planning on handing
out IPv6 /48s to every customer.

- Matt

----- On Nov 26, 2019, at 7:59 AM, Willy Manga mangawilly@gmail.com wrote:

Hi,

> I would have said the very very minimum could be to invest in a
> dual-stack 'proxy' for public-facing services; internal or external
> solution, you have the choice.
>
> And why even do that ? Because the other side is not only on IPv4.

Using a dual-stack proxy is not always an option. Source IP information may be needed on the app level for risk analysis, OFAC compliance, and copyright purposes. For example, Paypal will definitely use IP address information in its fraud risk analysis.

That said, there are of course ways to do that while using a proxy. However, that will now require some for of development. Dev time better used to properly implement v6.

Unfortunately, I've been part of way to many discussions where the only thing a beancounter wants to know is: what is the short term effect of not doing it?

Short term exec bonuses, short term decisions.

Thanks,

Sabri

On Tue, Nov 26, 2019 at 3:47 PM Scott Weeks <surfer@mauigateway.com> wrote:

>
>
> Top posting...
>
> ---------------------------------
> :: But it is not that simple in the real corporate world.
> :: Execs have bonus targets.
>
> Why would an exec care? Ipv6 is just normal work like ipv4.
> ---------------------------------
>
> No, you have to make purchases and have folks across the
> company do work to get everything going. Refocusing folks
> work on deploying IPv6 to *everything* (rather than, say,
> getting that shiny new Nokia 7750 deployed so we can sell
> more services) costs money. Ancient boxen are out here
> and don't support aye pee vee six well or at all. Getting
> ones that do costs money. Training lower level folks takes
> them away from their current work and costs money. Etc.
>

This is known as “too hungry to eat” or something similar about failing to
help yourself

https://www.google.com/amp/s/amp.businessinsider.com/marc-andreessen-advice-to-startups-raise-prices-2016-6



> ::> - Modifying old (ancient) internal code;
> :: Ancient in 2019 means what? Is this code not in security
> :: compliance ?
>
> I recently started back with a company after being gone nine
> years. My code was still running and no one in neteng had
> the knowledge of how to do anything with it much less to try
> to write in IPv6 sections. To take an SA and look into the
> networking code I wrote takes them away from things they
> need to do to sell services. That costs money.
>
> What Sabri wrote hit home here. Folks are not looking into
> it and will wait until forced to do so. Then said companies
> will be behind the ball in a big way, but that it what it is
> here and in the other companies I worked for.
>

We agree, neglecting ipv6 is a bad business decision


> A lot of this read to me as flippant. You don't seem to be
> willing to listen to those of us out here on the raggedy
> edges. I've said what Sabri said at least a few times on this
> list.
>

Sabri volunteered the information that they are an MBA at a large eyeball
network with 20 teams... , not the “raggedy edge”, they said something
about executive bonus alignment being the key problem....

That said, speaking of not being listened too, this artifact is useful as
it squarely raises the business risk in no uncertain terms.

https://www.arin.net/vault/knowledge/about_resources/ceo_letter.pdf

Business risk is mitigated or accepted ... for the last 10 years. Folks /
orgs make decisions and deal with the consequence.

My reality is that, at scale, ipv4 is winnowing longtail. The majority of
real bits/s and dollars are in ipv6. Ymmv. But i reject vehemently the
notion that v6 vanity project with no obvious business case / roi (Another
misstatement by Sabri).

If your business is dysfunctional, that is a different issue from ipv6
being dysfunctional.


> scott
>
>
>
>
>
> --- cb.list6@gmail.com wrote:
>
> From: Ca By <cb.list6@gmail.com>
> To: Sabri Berisha <sabri@cluecentral.net>
> Cc: nanog <nanog@nanog.org>
> Subject: Re: RIPE our of IPv4
> Date: Tue, 26 Nov 2019 15:11:40 -0800
>
> On Tue, Nov 26, 2019 at 12:15 AM Sabri Berisha <sabri@cluecentral.net>
> wrote:
>
> > ----- On Nov 26, 2019, at 1:36 AM, Doug Barton dougb@dougbarton.us
> wrote:
> >
> > > I get that some people still don't like it, but the answer is IPv6. Or,
> > > folks can keep playing NAT games, etc. But one wonders at what point
> > > rolling out IPv6 costs less than all the fun you get with [CG]NAT.
> >
> > When the MBAs start realizing the risk of not deploying it.
> >
>
> Hey, i have an mba. That and $5 will get me cup of coffee.
>
>
> > I have some inside knowledge about the IPv6 efforts of a large eyeball
> > network.
>
>
> Me too.
>
> In that particular case, the cost of deploying IPv6 internally is not
> > simply configuring it on the network gear; that has already been done.
> The
> > cost of fully supporting IPv6 includes (but is probably not limited to):
> >
> > - Support for deploying IPv6 across more than 20 different teams;
>
>
> Wow. I support 80M mobile subscribers, 90% of which are ipv6-only. I
> think 20 people in the company can spell ipv6, but somehow you need 20
> teams.... how many teams speak ipv4 ?
>
>
> > - Modifying old (ancient) internal code;
>
>
> Ancient in 2019 means what? Is this code not in security compliance ?
>
>
> > - Modifying old (ancient) database structures (think 16 character fields
> > for IP addresses);
>
>
> Hash 128 bits into 240/4 is how i heard Google handled it early on
>
>
> > - Upgrading/replacing load balancers and other legacy crap that only
> > support IPv4 (yeah, they still exist);
>
>
> Again, with all the CVEs, this code is always moving in the real world.
>
>
> > - Modifying the countless home-grown tools that automate firewalls etc;
>
>
> Home grown means it can be fixed instead of replaced.
>
>
> > - Auditing the PCI infrastructure to ensure it is still compliant after
> > deploying IPv6;
> >
>
> Ah, so you are keeping up with compliance / cve and are upgrading at
> regular intervals?
>
>
>
> > If it was as simple as upgrading a few IP stacks here and there, it would
> > be a non-issue.
> >
>
> Usually is just a few edge stacks to start and scale the edge
>
>
> > Don't get me wrong, I'm not advocating against IPv6 deployment; on the
> > contrary. But it is not that simple in the real corporate world. Execs
> have
> > bonus targets.
>
>
> Why would an exec care? Ipv6 is just normal work like ipv4.
>
> IPv6 is not yet important enough to become part of that bonus target:
>
>
> The bonus target was normal business continuity planning... in 2008. Sorry
> you missed that one. Here you go, just put 1 in 2009 to make it 2019 so
> you dont look so bad
>
> https://www.arin.net/vault/knowledge/about_resources/ceo_letter.pdf
>
>
> there is no ROI at this point. In this kind of environment there needs to
> > be a strong case to invest the capex to support IPv6.
> >
> > IPv6 must be supported on the CxO level in order to be deployed.
> >
> > Thanks,
> >
> > Sabri, (Badum tsss) MBA
>
>
> I see....well let me translate it you MBA-eese for you:
>
> FANG deployed ipv6 nearly 10 years ago. Since deploying ipv6, the cohort
> experienced 300% CAGR. Also, everything is mobile, and all mobile providers
> in the usa offer ipv6 by default in most cases. Latency! Scale! As your
> company launches its digital transformation iot 2020 virtualization
> container initiatives, ipv6 will be an integral part of staying relevant on
> the blockchain. Also, FANG did it nearly 10 years ago. Big content and
> big eyeballs are on ipv6, ipv4 is a winnowing longtail of irrelevance and
> iot botnets.
>
>
> >
>
>
>

> On 27 Nov 2019, at 10:58, Sabri Berisha <sabri@cluecentral.net> wrote:
>
> ----- On Nov 26, 2019, at 7:59 AM, Willy Manga mangawilly@gmail.com wrote:
>
> Hi,
>
>> I would have said the very very minimum could be to invest in a
>> dual-stack 'proxy' for public-facing services; internal or external
>> solution, you have the choice.
>>
>> And why even do that ? Because the other side is not only on IPv4.
>
> Using a dual-stack proxy is not always an option. Source IP information may be needed on the app level for risk analysis, OFAC compliance, and copyright purposes. For example, Paypal will definitely use IP address information in its fraud risk analysis.

And existing proxies don’t already pass through the connecting IP address? There are even header fields that are dedicated for this purpose [1].

Most web sites could be dual stacked today with zero issues. Web site analytic tools already deal with IPv6 and have for years.

> That said, there are of course ways to do that while using a proxy. However, that will now require some for of development. Dev time better used to properly implement v6.

And the difference in time between reading the address from X-Forwarded-For: vs directly is negligible.

> Unfortunately, I've been part of way to many discussions where the only thing a beancounter wants to know is: what is the short term effect of not doing it?
>
> Short term exec bonuses, short term decisions.
>
> Thanks,
>
> Sabri


[1] https://en.wikipedia.org/wiki/X-Forwarded-For
--
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: marka@isc.org

> Scott Weeks wrote :
> A lot of this read to me as flippant. You don't seem to be willing to listen to those of us out here on the raggedy edges.

And there are lots of us.

> I've said what Sabri said at least a few times on this list.

+1

Michel.

TSI Disclaimer: This message and any files or text attached to it are intended only for the recipients named above and contain information that may be confidential or privileged. If you are not the intended recipient, you must not forward, copy, use or otherwise disclose this communication or the information contained herein. In the event you have received this message in error, please notify the sender immediately by replying to this message, and then delete all copies of it from your system. Thank you!...

Sounds like your company is about to go offline. So I will say bye bye
for now just in case it happens faster than you expected.

C

On 26/11/2019 23:46, Scott Weeks wrote:
>
> Top posting...
>
> ---------------------------------
> :: But it is not that simple in the real corporate world.
> :: Execs have bonus targets.
>
> Why would an exec care? Ipv6 is just normal work like ipv4.
> ---------------------------------
>
> No, you have to make purchases and have folks across the
> company do work to get everything going. Refocusing folks
> work on deploying IPv6 to *everything* (rather than, say,
> getting that shiny new Nokia 7750 deployed so we can sell
> more services) costs money. Ancient boxen are out here
> and don't support aye pee vee six well or at all. Getting
> ones that do costs money. Training lower level folks takes
> them away from their current work and costs money. Etc.
>
> ::> - Modifying old (ancient) internal code;
> :: Ancient in 2019 means what? Is this code not in security
> :: compliance ?
>
> I recently started back with a company after being gone nine
> years. My code was still running and no one in neteng had
> the knowledge of how to do anything with it much less to try
> to write in IPv6 sections. To take an SA and look into the
> networking code I wrote takes them away from things they
> need to do to sell services. That costs money.
>
> What Sabri wrote hit home here. Folks are not looking into
> it and will wait until forced to do so. Then said companies
> will be behind the ball in a big way, but that it what it is
> here and in the other companies I worked for.
>
> A lot of this read to me as flippant. You don't seem to be
> willing to listen to those of us out here on the raggedy
> edges. I've said what Sabri said at least a few times on this
> list.
>
> scott
>
>
>
>
>
> --- cb.list6@gmail.com wrote:
>
> From: Ca By <cb.list6@gmail.com>
> To: Sabri Berisha <sabri@cluecentral.net>
> Cc: nanog <nanog@nanog.org>
> Subject: Re: RIPE our of IPv4
> Date: Tue, 26 Nov 2019 15:11:40 -0800
>
> On Tue, Nov 26, 2019 at 12:15 AM Sabri Berisha <sabri@cluecentral.net>
> wrote:
>
>> ----- On Nov 26, 2019, at 1:36 AM, Doug Barton dougb@dougbarton.us wrote:
>>
>>> I get that some people still don't like it, but the answer is IPv6. Or,
>>> folks can keep playing NAT games, etc. But one wonders at what point
>>> rolling out IPv6 costs less than all the fun you get with [CG]NAT.
>> When the MBAs start realizing the risk of not deploying it.
>>
> Hey, i have an mba. That and $5 will get me cup of coffee.
>
>
>> I have some inside knowledge about the IPv6 efforts of a large eyeball
>> network.
>
> Me too.
>
> In that particular case, the cost of deploying IPv6 internally is not
>> simply configuring it on the network gear; that has already been done. The
>> cost of fully supporting IPv6 includes (but is probably not limited to):
>>
>> - Support for deploying IPv6 across more than 20 different teams;
>
> Wow. I support 80M mobile subscribers, 90% of which are ipv6-only. I
> think 20 people in the company can spell ipv6, but somehow you need 20
> teams.... how many teams speak ipv4 ?
>
>
>> - Modifying old (ancient) internal code;
>
> Ancient in 2019 means what? Is this code not in security compliance ?
>
>
>> - Modifying old (ancient) database structures (think 16 character fields
>> for IP addresses);
>
> Hash 128 bits into 240/4 is how i heard Google handled it early on
>
>
>> - Upgrading/replacing load balancers and other legacy crap that only
>> support IPv4 (yeah, they still exist);
>
> Again, with all the CVEs, this code is always moving in the real world.
>
>
>> - Modifying the countless home-grown tools that automate firewalls etc;
>
> Home grown means it can be fixed instead of replaced.
>
>
>> - Auditing the PCI infrastructure to ensure it is still compliant after
>> deploying IPv6;
>>
> Ah, so you are keeping up with compliance / cve and are upgrading at
> regular intervals?
>
>
>
>> If it was as simple as upgrading a few IP stacks here and there, it would
>> be a non-issue.
>>
> Usually is just a few edge stacks to start and scale the edge
>
>
>> Don't get me wrong, I'm not advocating against IPv6 deployment; on the
>> contrary. But it is not that simple in the real corporate world. Execs have
>> bonus targets.
>
> Why would an exec care? Ipv6 is just normal work like ipv4.
>
> IPv6 is not yet important enough to become part of that bonus target:
>
>
> The bonus target was normal business continuity planning... in 2008. Sorry
> you missed that one. Here you go, just put 1 in 2009 to make it 2019 so
> you dont look so bad
>
> https://www.arin.net/vault/knowledge/about_resources/ceo_letter.pdf
>
>
> there is no ROI at this point. In this kind of environment there needs to
>> be a strong case to invest the capex to support IPv6.
>>
>> IPv6 must be supported on the CxO level in order to be deployed.
>>
>> Thanks,
>>
>> Sabri, (Badum tsss) MBA
>
> I see....well let me translate it you MBA-eese for you:
>
> FANG deployed ipv6 nearly 10 years ago. Since deploying ipv6, the cohort
> experienced 300% CAGR. Also, everything is mobile, and all mobile providers
> in the usa offer ipv6 by default in most cases. Latency! Scale! As your
> company launches its digital transformation iot 2020 virtualization
> container initiatives, ipv6 will be an integral part of staying relevant on
> the blockchain. Also, FANG did it nearly 10 years ago. Big content and
> big eyeballs are on ipv6, ipv4 is a winnowing longtail of irrelevance and
> iot botnets.
>
>
>

--- cdel@firsthand.net wrote:
From: Christian <cdel@firsthand.net>

Sounds like your company is about to go offline. So I will
say bye bye for now just in case it happens faster than you
expected.
---------------------------------------------


Speaking of flippant... No the ILEC has been here since the
1800s. I don't think it's going anywhere fast.

scott

--- cb.list6@gmail.com wrote:
From: Ca By <cb.list6@gmail.com>

If your business is dysfunctional, that is a different
issue from ipv6 being dysfunctional.
-----------------------------------------


I was just expressing the problems eyeball networks are
having getting this done. Shittons of stuff is out there
in the CPE that mobile and DC networks do not have to deal
with. The suits are looking at the short term cost/risk.

scott

> On 27 Nov 2019, at 11:40, Scott Weeks <surfer@mauigateway.com> wrote:
>
>
>
> --- cb.list6@gmail.com wrote:
> From: Ca By <cb.list6@gmail.com>
>
> If your business is dysfunctional, that is a different
> issue from ipv6 being dysfunctional.
> -----------------------------------------
>
>
> I was just expressing the problems eyeball networks are
> having getting this done. Shittons of stuff is out there
> in the CPE that mobile and DC networks do not have to deal
> with. The suits are looking at the short term cost/risk.
>
> scott

Eyeball networks can still deliver IPv6 even if most of their
gear isn’t IPv6 ready. 6rd [1] allows you to give every customer
a /48 over a IPv4 access network. You just need to record the
6rd DHCPv4 Option being returned over time so you can map from
IPv6 address to the IPv4 address your customer was using.

You bill on the IPv4 packets.

This is 10+ years old at this stage and quite frankly just works.
Yes, you need a few BRs and a IPv6 path from them to the rest of
the world. Lots of ISP’s deliver IPv6 to their customers today
using 6rd.

Lots of CPE routers support 6rd already and if the CPE router doesn’t
there is no harm so no foul. If you are supplying CPE devices just
replace ones that don’t support 6rd with ones that support 6rd (and
native IPv6) as they break. If a customer requests a new CPE router
post it to them. If you aren’t supplying CPE devices just tell your
customers that 6rd is supported.

6rd works with 50 year replacement time frames.

This is really no different to what HE has been doing for the last
20 years, it just moves the encapsulation / decapsulation closer to
the customer.

Mark

[1] https://tools.ietf.org/html/rfc5969
--
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: marka@isc.org

On Tuesday 2019-11-26 00:13, Sabri Berisha wrote:

>Don't get me wrong, I'm not advocating against IPv6 deployment; on the
>contrary. But it is not that simple in the real corporate world. Execs
>have bonus targets. IPv6 is not yet important enough to become part of
>that bonus target: there is no ROI at this point.

Though eyeballs need to change, so does content. And eyeballs will
invest if the content were to demand it. So, perhaps Google will give
IPv6 hosted content the same tiny boost they gave HTTPS content.


/mark

Speaking as being a trifle self-entitled?


On 27/11/2019 00:35, Scott Weeks wrote:
>
> --- cdel@firsthand.net wrote:
> From: Christian <cdel@firsthand.net>
>
> Sounds like your company is about to go offline. So I will
> say bye bye for now just in case it happens faster than you
> expected.
> ---------------------------------------------
>
>
> Speaking of flippant... No the ILEC has been here since the
> 1800s. I don't think it's going anywhere fast.
>
> scott

Telcos looking at the short term is why telcos have largely turned into dumpster fires in the last 20 years.




-----
Mike Hammett
Intelligent Computing Solutions
http://www.ics-il.com

Midwest-IX
http://www.midwest-ix.com

----- Original Message -----

From: "Scott Weeks" <surfer@mauigateway.com>
To: nanog@nanog.org
Sent: Tuesday, November 26, 2019 6:40:11 PM
Subject: Re: RIPE our of IPv4



--- cb.list6@gmail.com wrote:
From: Ca By <cb.list6@gmail.com>

If your business is dysfunctional, that is a different
issue from ipv6 being dysfunctional.
-----------------------------------------


I was just expressing the problems eyeball networks are
having getting this done. Shittons of stuff is out there
in the CPE that mobile and DC networks do not have to deal
with. The suits are looking at the short term cost/risk.

scott

On 2019-11-26 17:11, Ca By wrote:
> On Tue, Nov 26, 2019 at 12:15 AM Sabri Berisha <sabri@cluecentral.net>
> wrote:
>
>> ----- On Nov 26, 2019, at 1:36 AM, Doug Barton dougb@dougbarton.us
>> wrote:
>>

[snip]
>> there is no ROI at this point. In this kind of environment there needs
>> to
>> be a strong case to invest the capex to support IPv6.
>>
>> IPv6 must be supported on the CxO level in order to be deployed.
>>
>> Thanks,
>>
>> Sabri, (Badum tsss) MBA
>
>
> I see....well let me translate it you MBA-eese for you:
>
> FANG deployed ipv6 nearly 10 years ago. Since deploying ipv6, the
> cohort
> experienced 300% CAGR. Also, everything is mobile, and all mobile
> providers
> in the usa offer ipv6 by default in most cases. Latency! Scale! As your
> company launches its digital transformation iot 2020 virtualization
> container initiatives, ipv6 will be an integral part of staying
> relevant on
> the blockchain. Also, FANG did it nearly 10 years ago. Big content
> and
> big eyeballs are on ipv6, ipv4 is a winnowing longtail of irrelevance
> and
> iot botnets.

None of which matters a damn to almost all of my business eyeball
customers. They can still get from our network to 100% of all Internet
content & services via IPv4 in 2019. I regularly vet deals for our
sales team, and out of the hundreds of deals we sold this year, I can
count on one hand the number of deals where customers wanted IPv6. We
sold them IPv6 access, but we didn't put it on our own network, because
we face the same internal challenges Sabri mentioned. (SD-WAN, OTOH,
was far more popular. I'll give you three guesses why. Hint - it's not
because tunnel technology is awesome and allows us to scale our networks
further and everyone is doing it.)

Though their participation has been key in making IPv6 more useful for
eyeballs, content hasn't driven adoption. The only thing eyeballs care
about is getting to 100% of what they need and want at minimal cost.
Until eyeball networks start charging eyeballs for IPv4, IPv4 will
linger. The day eyeballs start bitching on forums, opening tickets,
complaining on Twitter, etc. because they have only IPv6 is when IPv4
will start to lose relevance.

As an aside, I would guess that it's the corporate eyeball customers
with servers, not resi/mobile behind CGNAT, that will bear the brunt of
the IPv4 cost first. But what enterprise wants to tell its non-IPv6
customers "your Internet needs to be upgraded, come back to us when
you're done?" That doesn't bode well for the short-term future.

-Brian

----- On Nov 26, 2019, at 4:16 PM, Ca By cb.list6@gmail.com wrote:

> Sabri volunteered the information that they are an MBA at a large eyeball
> network with 20 teams...

You drew the wrong conclusions. I wrote: "I have some inside knowledge about the IPv6 efforts of a large eyeball network". I also have similar knowledge of a large worldwide e-commerce enterprise, with similar challenges.

I've been a JNCIE since 2007 and have worked for 3 different network gear vendors in a technical capacity. The MBA (which I earned this year) just helps understand the business side of things. I can recommend it to anyone, even if you have no aspirations for management roles.

Thanks,

Sabri,

On Wed Nov 27, 2019 at 01:08:04PM -0600, Brian Knight wrote:
> None of which matters a damn to almost all of my business eyeball
> customers. They can still get from our network to 100% of all Internet
> content & services via IPv4 in 2019. I regularly vet deals for our
> sales team, and out of the hundreds of deals we sold this year, I can
> count on one hand the number of deals where customers wanted IPv6. We
> sold them IPv6 access, but we didn't put it on our own network, because
> we face the same internal challenges Sabri mentioned. (SD-WAN, OTOH,
> was far more popular

A few year later customer wakes up:

"wait you sold us all those toys we didn't need but didn't include
the basic transport capabilites everyone apparently has been saying
for over a decade are required minimum?"

"and now you want us to pay you to rebuild it again and trust that
you got the basics right this time?"

If you're an internet professional you are a negligent one if by
now you are not ensuring all you build quietly includes IPv6, no
customer should need to know to ask for it. It's not like it
needs different kit.

> As an aside, I would guess that it's the corporate eyeball customers
> with servers, not resi/mobile behind CGNAT, that will bear the brunt of
> the IPv4 cost first. But what enterprise wants to tell its non-IPv6
> customers "your Internet needs to be upgraded, come back to us when
> you're done?" That doesn't bode well for the short-term future.

"all that multi natted into same address space VPN firewall
complicated knitting we never got right wasn't needed if you'd
told us to use IPv6?"

brandon

--- brandon@rd.bbc.co.uk wrote:
From: Brandon Butterworth <brandon@rd.bbc.co.uk>

If you're an internet professional you are a negligent one if by
now you are not ensuring all you build quietly includes IPv6, no
customer should need to know to ask for it. It's not like it
needs different kit.
-----------------------------------------------------


No, it's just that (at least in my case at several different
companies) we're so focused by management on getting the sale
done by augmenting the existing network there is not enough
time to devote to **planning an entire network from the
ground up**, then working your plan. The other way (just
start configuring stuff) is replete with troubles.

BTW, I have been the IPv6 loudmouth every time, but I don't
get any traction at all in any of the companies I've worked
for. Eyes gloss over and someone quickly changes the
conversation. Then we talk about sizing subnets and stuff...

scott

> Brian Knight wrote :
> None of which matters a damn to almost all of my business eyeball customers. They
> can still get from our network to 100% of all Internet content & services via IPv4 in 2019.

And will for the foreseable future. I am not one of your customers, but I like your realistic views. I vote with my wallet and buy my transit from the ISP who understands my needs.

> I regularly vet deals for our sales team, and out of the hundreds of deals we sold this year,
> I can count on one hand the number of deals where customers wanted IPv6.

Won't change any time soon. For the vast majority of business eyballs and entreprises, IPv6 is not even on the agenda.

> But what enterprise wants to tell its non-IPv6 customers "your Internet needs to be upgraded,
> come back to us when you're done?" That doesn't bode well for the short-term future.

None of my customers has IPv6. None of my suppliers has IPv6. Nobody wants The business / enterprise ecosystem is and will remain of a size large enough to keep the IPv4 services for the foreseeable future.
Facebook going IPv6-only ? that would be a blessing. That is not what we pay employees to do at the office.

As Sabri would have said, why should I look like an idiot and go to the board and the investors for money to invest in something that has zero ROI ?
I was on the 6bone. I heard the IPv6 FUD for 20 years.

Michel.

TSI Disclaimer: This message and any files or text attached to it are intended only for the recipients named above and contain information that may be confidential or privileged. If you are not the intended recipient, you must not forward, copy, use or otherwise disclose this communication or the information contained herein. In the event you have received this message in error, please notify the sender immediately by replying to this message, and then delete all copies of it from your system. Thank you!...

--- surfer@mauigateway.com wrote:
From: "Scott Weeks" <surfer@mauigateway.com>

No, it's just that (at least in my case at several different
companies) we're so focused by management on getting the sale
done by augmenting the existing network there is not enough
time to devote to **planning an entire network from the
ground up**, then working your plan. The other way (just
start configuring stuff) is replete with troubles.

BTW, I have been the IPv6 loudmouth every time, but I don't
get any traction at all in any of the companies I've worked
for. Eyes gloss over and someone quickly changes the
conversation. Then we talk about sizing subnets and stuff...
---------------------------------------------


BTW, what Mark Andrews said about 6rd fixes (I'm assuming
a relatively low level of network architecturing work is
necessary to get it done) what I am saying, but it feels
so dirty. I would like to go straight to dual stack.

scott

> On 28 Nov 2019, at 06:08, Brian Knight <ml@knight-networks.com> wrote:
>
> On 2019-11-26 17:11, Ca By wrote:
>> On Tue, Nov 26, 2019 at 12:15 AM Sabri Berisha <sabri@cluecentral.net>
>> wrote:
>>> ----- On Nov 26, 2019, at 1:36 AM, Doug Barton dougb@dougbarton.us wrote:
>
> [snip]
>>> there is no ROI at this point. In this kind of environment there needs to
>>> be a strong case to invest the capex to support IPv6.
>>> IPv6 must be supported on the CxO level in order to be deployed.
>>> Thanks,
>>> Sabri, (Badum tsss) MBA
>> I see....well let me translate it you MBA-eese for you:
>> FANG deployed ipv6 nearly 10 years ago. Since deploying ipv6, the cohort
>> experienced 300% CAGR. Also, everything is mobile, and all mobile providers
>> in the usa offer ipv6 by default in most cases. Latency! Scale! As your
>> company launches its digital transformation iot 2020 virtualization
>> container initiatives, ipv6 will be an integral part of staying relevant on
>> the blockchain. Also, FANG did it nearly 10 years ago. Big content and
>> big eyeballs are on ipv6, ipv4 is a winnowing longtail of irrelevance and
>> iot botnets.
>
> None of which matters a damn to almost all of my business eyeball customers. They can still get from our network to 100% of all Internet content & services via IPv4 in 2019.

No you can’t. You can’t reach the machine I’m typing on via IPv4 and it is ON THE INTERNET. It is directly reachable via IPv6. Selling Internet connectivity without IPv6 should be considered fraud these days. Don’t
you believe in “Truth in Advertising”?

> I regularly vet deals for our sales team, and out of the hundreds of deals we sold this year, I can count on one hand the number of deals where customers wanted IPv6. We sold them IPv6 access, but we didn't put it on our own network, because we face the same internal challenges Sabri mentioned. (SD-WAN, OTOH, was far more popular. I'll give you three guesses why. Hint - it's not because tunnel technology is awesome and allows us to scale our networks further and everyone is doing it.)

> Though their participation has been key in making IPv6 more useful for eyeballs, content hasn't driven adoption. The only thing eyeballs care about is getting to 100% of what they need and want at minimal cost. Until eyeball networks start charging eyeballs for IPv4, IPv4 will linger. The day eyeballs start bitching on forums, opening tickets, complaining on Twitter, etc. because they have only IPv6 is when IPv4 will start to lose relevance.
>
> As an aside, I would guess that it's the corporate eyeball customers with servers, not resi/mobile behind CGNAT, that will bear the brunt of the IPv4 cost first. But what enterprise wants to tell its non-IPv6 customers "your Internet needs to be upgraded, come back to us when you're done?" That doesn't bode well for the short-term future.
>
> -Brian

--
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742 INTERNET: marka@isc.org

>> Brian Knight wrote :
>> None of which matters a damn to almost all of my business eyeball customers. They can still get from our network to 100% of all Internet content & services via IPv4 in 2019.

> Mark Andrews wrote :
> No you can’t. You can’t reach the machine I’m typing on via IPv4 and it is ON THE INTERNET.

Why should I care ? it contains zero content of value to me. It's on a subset of the Internet that contains zero content that interests me.

Michel.

TSI Disclaimer: This message and any files or text attached to it are intended only for the recipients named above and contain information that may be confidential or privileged. If you are not the intended recipient, you must not forward, copy, use or otherwise disclose this communication or the information contained herein. In the event you have received this message in error, please notify the sender immediately by replying to this message, and then delete all copies of it from your system. Thank you!...

>> On Nov 27, 2019, at 2:54 PM, Brandon Butterworth <brandon@rd.bbc.co.uk> wrote:
>>
>> ?On Wed Nov 27, 2019 at 01:08:04PM -0600, Brian Knight wrote:
>> None of which matters a damn to almost all of my business eyeball
>> customers. They can still get from our network to 100% of all Internet
>> content & services via IPv4 in 2019. I regularly vet deals for our
>> sales team, and out of the hundreds of deals we sold this year, I can
>> count on one hand the number of deals where customers wanted IPv6. We
>> sold them IPv6 access, but we didn't put it on our own network, because
>> we face the same internal challenges Sabri mentioned. (SD-WAN, OTOH,
>> was far more popular
>
> A few year later customer wakes up:
>
> "wait you sold us all those toys we didn't need but didn't include
> the basic transport capabilites everyone apparently has been saying
> for over a decade are required minimum?"
>
> "and now you want us to pay you to rebuild it again and trust that
> you got the basics right this time?"
>
> If you're an internet professional you are a negligent one if by
> now you are not ensuring all you build quietly includes IPv6, no
> customer should need to know to ask for it. It's not like it
> needs different kit.

Possibly some customers may react this way, but I’m thinking many more would ask “what does it take to enable it?” Most are reasonable and show good faith, even if an equipment swap is needed. And if the demand for IPv6 is there, the providers will get the work prioritized.

>> As an aside, I would guess that it's the corporate eyeball customers
>> with servers, not resi/mobile behind CGNAT, that will bear the brunt of
>> the IPv4 cost first. But what enterprise wants to tell its non-IPv6
>> customers "your Internet needs to be upgraded, come back to us when
>> you're done?" That doesn't bode well for the short-term future.
>
> "all that multi natted into same address space VPN firewall
> complicated knitting we never got right wasn't needed if you'd
> told us to use IPv6?"

IPv6 doesn’t help anyone get access to their IPv4-only customers. (Too bad that it doesn’t.)

My point was that, if eyeball networks start charging a premium for IPv4, their likely first customers to be charged are business customers not behind CGNAT. Those that don’t wish to pay the IPv4 premium would have to force *their* customers to go IPv6. That would be a much more difficult conversation than simply paying the premium. So out of all the forces at work, which gives way first?

>
> brandon

Thanks,

-Brian

IPv6 significantly offloads the CGN servers. If you are not yet using CGN
you probably won't care, but sooner or later you will.

Thanks to the content providers that make this possible by offering enough
content by volume available on the IPv6 internet.

Regards

Baldur

On 11/26/19 12:13 AM, Sabri Berisha wrote:
> ----- On Nov 26, 2019, at 1:36 AM, Doug Barton dougb@dougbarton.us wrote:
>
>> I get that some people still don't like it, but the answer is IPv6. Or,
>> folks can keep playing NAT games, etc. But one wonders at what point
>> rolling out IPv6 costs less than all the fun you get with [CG]NAT.
>
> When the MBAs start realizing the risk of not deploying it.
>
> I have some inside knowledge about the IPv6 efforts of a large eyeball network.

For what it's worth, I have extensive experience in both eyeball and
content networks.

> In that particular case, the cost of deploying IPv6 internally is not simply configuring it on the network gear;

We're rehashing old ground here. Perhaps you weren't on the list the
last N times this has come up. My short answer, I didn't say it would be
easy, I said it is less expensive than the alternatives over time.

> that has already been done. The cost of fully supporting IPv6 includes (but is probably not limited to):
>
> - Support for deploying IPv6 across more than 20 different teams;

I don't understand how you're using "teams" here. For the most part you
turn it on, and end-user systems pick up the RA and do the right thing.
If you want something fancier, you can do that with DHCP, static
addressing, etc. In other words, this works the exact same way that IPv4
does.

> - Modifying old (ancient) internal code;

What code? IPv4 isn't going away on the inside, so what needs to be
modified? If you're talking monitoring software, etc., if you're still
using software that doesn't understand IPv6, you're way overdue for an
upgrade already.

> - Modifying old (ancient) database structures (think 16 character fields for IP addresses);

Either see above, or much more likely you'd be adding a field, not
modifying the existing one.

> - Upgrading/replacing load balancers and other legacy crap that only support IPv4 (yeah, they still exist);

If we're talking about an enterprise that is seriously still using stuff
this old, it's more likely than not that IPv6 is the least of their
worries. And I'm not being flippant or disrespectful here. For at least
the last 10 years or so, and definitely in the last 5, all of the
enterprise level network gear sold has had support for IPv6. So again,
way overdue for an update, but if this is all you have available, then
you likely have bigger fish to fry. (And feel free to save the
obligatory, "My favorite network widget that I use in my 100%
enterprise-class network does not support IPv6." Yes, I realize that
there are exceptions, but they are the exceptions, not the rule.)

> - Modifying the countless home-grown tools that automate firewalls etc;

Yes, this is actually a legitimate point.

> - Auditing the PCI infrastructure to ensure it is still compliant after deploying IPv6;

Also legit, where it applies, although you also have the option of not
deploying on the network with the PCI data. For internal-only things,
it's great to have IPv6, and will become increasingly important as time
goes on, but it's not required.

> Execs have bonus targets. IPv6 is not yet important enough to become part of that bonus target: there is no ROI at this point.

That depends heavily on what enterprise you're talking about.

The point I'm trying to make is that there IS an ROI here. For content
providers it's the ability to create a stable network architecture
across all of your sites, and connect directly to the many eyeballs that
are already on IPv6 (cell networks, many ISPs, etc.). There is also the
much harder to define ROI for future-proofing the network, but that's
part of the master class. :)

For eyeball networks the same stable network architecture argument
applies. The immediate ROI is harder to define, but similar, in the
sense that connect directly to the many content networks that have
already deployed IPv6 and future-proofing are both relevant.

Much harder for the eyeball networks to quantify are the savings related
to NOT having to do [CG]NAT, etc. To create that slide you need an exec
who truly understands the (rising over time) costs of twiddling around
with the NATs, as well as the realistic costs involved in rolling out
IPv6 balanced by the long term support. Then you also need an executive
team and board that can understand those slides when they see them.

But it's not all in vain. I'm on Spectrum here at home, and I have
native IPv6 that "just worked" from the moment I plugged my router into
my cable modem.

So there are a non-trivial number of both eyeball and content networks
that already get it. The value proposition obviously does exist, we just
need more people in the right places with the right knowledge and
experience to make it happen.

Doug