Mailing List Archive

Hi,

On Wed, Jun 10, 2020 at 07:19:18PM +0200, Mark Tinka wrote:
> Just want to sample the room and find out if anyone here - especially
> those running an LDP-based BGPv4-free core (or something close to it) -
> would be interested in LDPv6, in order to achieve the same for BGPv6?
>
> A discussion I've been having with Cisco on the matter is that they do
> not "see any demand" for LDPv6, and thus, won't develop it (on IOS XE).
> Meanwhile, it is actively developed, supported and maintained on IOS XR
> since 5.3.0, with new features being added to it as currently as 7.1.1.

To be honest, I do not think we're going to buy any IOS XE gear in the
foreseeable future. But if we did, LDPv6 would be nice to have - to get
rid of IPv4 in the backbone network.

We do not intend to run SRv6 any time soon.

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de

I would take either LDPv6 or SRv6 - but also need L3VPN (and now EVPN)
re-wired to use IPv6 NH.

I have requested LDPv6 and SRv6 many times from Cisco to migrate the
routing control plane from IPv4 to IPv6

I have lots of IPv6 address space. I don't have a lot of IPv4
address space. RFC1918 is not as big as it seems. Apparently this is hard
to grasp...

(This is primarily IOS-XE - can't afford the IOS-XR supercars)

On Wed, Jun 10, 2020 at 1:20 PM Mark Tinka <mark.tinka@seacom.mu> wrote:

> Hi all.
>
> Just want to sample the room and find out if anyone here - especially
> those running an LDP-based BGPv4-free core (or something close to it) -
> would be interested in LDPv6, in order to achieve the same for BGPv6?
>
> A discussion I've been having with Cisco on the matter is that they do not
> "see any demand" for LDPv6, and thus, won't develop it (on IOS XE).
> Meanwhile, it is actively developed, supported and maintained on IOS XR
> since 5.3.0, with new features being added to it as currently as 7.1.1.
>
> Needless to say, a bunch of other vendors have been supporting it for a
> while now - Juniper, Nokia/ALU, Huawei, even HP.
>
> IOS XR supporting LDPv6 notwithstanding, Cisco's argument is that "the
> world" is heavily focused on deploying SRv6 (Segment Routing). While I know
> of one or two questionable deployments, I'm not entirely sure much of the
> world is clamouring to deploy SR, based on all the polls we've done at
> various NOG meetings and within the general list-based operator community
>
> So I just wanted to hear from this operator community on whether you would
> be interested in having LDPv6 support to go alongside your LDPv4
> deployments, especially if you run native dual-stack backbones. Or if your
> focus is totally on SRv6. Or if you don't care either way :-). Thanks.
>
> Mark.
>


--
Tim:>
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 10/Jun/20 20:10, Gert Doering wrote:

> To be honest, I do not think we're going to buy any IOS XE gear in the
> foreseeable future. But if we did, LDPv6 would be nice to have - to get
> rid of IPv4 in the backbone network.

We have LDPv6 working beautifully between IOS XR (6.4.2) and Junos
(17.4). But we can't make the core BGPv6-free because downstream
ASR1000's and ASR920's don't have it.

So if you don't have IOS XE today, but have the others, then you're good
to go :-).

Nobody is going to demand LDPv6, because it's not about LDPv6... it's
about IPv6, and wanting IPv4 parity. But, guess that doesn't much a
business case write.


>
> We do not intend to run SRv6 any time soon.

As I thought :-).

Mark.

I'm pretty sure that one or more of Mark, Gert or Tim are thinking
SR/MPLS IPv6 when they say SRv6?

No one in their right minds thinks SRv6 is a good idea, terrible snake
oil and waste of NRE. SR/MPLS IPv6 of course is terrific.

LDPv6 and SRv6 seem like an odd couple, LDPv6 SR/MPLS IPv6 seem far
more reasonable couple to choose from. I have my favorite.


On Wed, 10 Jun 2020 at 21:32, Tim Durack <tdurack@gmail.com> wrote:
>
> I would take either LDPv6 or SRv6 - but also need L3VPN (and now EVPN) re-wired to use IPv6 NH.
>
> I have requested LDPv6 and SRv6 many times from Cisco to migrate the routing control plane from IPv4 to IPv6
>
> I have lots of IPv6 address space. I don't have a lot of IPv4 address space. RFC1918 is not as big as it seems. Apparently this is hard to grasp...
>
> (This is primarily IOS-XE - can't afford the IOS-XR supercars)
>
> On Wed, Jun 10, 2020 at 1:20 PM Mark Tinka <mark.tinka@seacom.mu> wrote:
>>
>> Hi all.
>>
>> Just want to sample the room and find out if anyone here - especially those running an LDP-based BGPv4-free core (or something close to it) - would be interested in LDPv6, in order to achieve the same for BGPv6?
>>
>> A discussion I've been having with Cisco on the matter is that they do not "see any demand" for LDPv6, and thus, won't develop it (on IOS XE). Meanwhile, it is actively developed, supported and maintained on IOS XR since 5.3.0, with new features being added to it as currently as 7.1.1.
>>
>> Needless to say, a bunch of other vendors have been supporting it for a while now - Juniper, Nokia/ALU, Huawei, even HP.
>>
>> IOS XR supporting LDPv6 notwithstanding, Cisco's argument is that "the world" is heavily focused on deploying SRv6 (Segment Routing). While I know of one or two questionable deployments, I'm not entirely sure much of the world is clamouring to deploy SR, based on all the polls we've done at various NOG meetings and within the general list-based operator community
>>
>> So I just wanted to hear from this operator community on whether you would be interested in having LDPv6 support to go alongside your LDPv4 deployments, especially if you run native dual-stack backbones. Or if your focus is totally on SRv6. Or if you don't care either way :-). Thanks.
>>
>> Mark.
>
>
>
> --
> Tim:>



--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 10/Jun/20 20:29, Tim Durack wrote:
> I would take either LDPv6 or SRv6 - but also need L3VPN (and now EVPN)
> re-wired to use IPv6 NH.

At the moment, LDPv6 doesn't have what I call "service" support, i.e.,
l3vpn's, l2vpn's, MPLSv6-TE, mLDP, CsC, e.t.c. To be honest, I don't
mind those so much on my end because you can still run a BGP-free core
and still deliver those services.

Granted, if your goal is a single-stack MPLS-based IPv6 network, then
yes, it would be good for LDPv6 to support the "services".

But if it's just vanilla MPLSv6 switching you're after, then LDPv6 will
do just fine.


>
> I have requested LDPv6 and SRv6 many times from Cisco to migrate the
> routing control plane from IPv4 to IPv6

Well, according to them, SRv6 is winning customers over, and nobody
wants LDPv6. Then again, they have LDPv6 in IOS XR; figures.


>
> I have lots of IPv6 address space. I don't have a lot of IPv4
> address space. RFC1918 is not as big as it seems. Apparently this is
> hard to grasp...
>
> (This is primarily IOS-XE - can't afford the IOS-XR supercars)

LDPv6 must be a business case :-). Well, wonder how they sell so many
CGN's, then :-).

Mark.

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Hi,

On Wed, Jun 10, 2020 at 08:45:31PM +0200, Mark Tinka wrote:
> On 10/Jun/20 20:10, Gert Doering wrote:
>
> > To be honest, I do not think we're going to buy any IOS XE gear in the
> > foreseeable future. But if we did, LDPv6 would be nice to have - to get
> > rid of IPv4 in the backbone network.
>
> We have LDPv6 working beautifully between IOS XR (6.4.2) and Junos
> (17.4). But we can't make the core BGPv6-free because downstream
> ASR1000's and ASR920's don't have it.
>
> So if you don't have IOS XE today, but have the others, then you're good
> to go :-).

We do have IOS XEs today (ASR920), and based on that, we're not going
to buy new IOS XE devices any time soon.

The amount of... strangeness... that this BU considers acceptable
is... not.

gert

--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de

Hi,

On Wed, Jun 10, 2020 at 09:45:55PM +0300, Saku Ytti wrote:
> I'm pretty sure that one or more of Mark, Gert or Tim are thinking
> SR/MPLS IPv6 when they say SRv6?
>
> No one in their right minds thinks SRv6 is a good idea, terrible snake
> oil and waste of NRE. SR/MPLS IPv6 of course is terrific.
>
> LDPv6 and SRv6 seem like an odd couple, LDPv6 SR/MPLS IPv6 seem far
> more reasonable couple to choose from. I have my favorite.

Oh. Indeed, sorry for being unclear here.

SR/MPLS sounds like a good idea (reducing label state).

SR/IPv6 does not sound convincing.

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de

Ah yes, I would say LDPv6 and/or SR/MPLS IPv6. SRv6 reads like a science
project.

Either way, I would like to achieve a full IPv6 control plane.


On Wed, Jun 10, 2020 at 2:46 PM Saku Ytti <saku@ytti.fi> wrote:

> I'm pretty sure that one or more of Mark, Gert or Tim are thinking
> SR/MPLS IPv6 when they say SRv6?
>
> No one in their right minds thinks SRv6 is a good idea, terrible snake
> oil and waste of NRE. SR/MPLS IPv6 of course is terrific.
>
> LDPv6 and SRv6 seem like an odd couple, LDPv6 SR/MPLS IPv6 seem far
> more reasonable couple to choose from. I have my favorite.
>
>
> On Wed, 10 Jun 2020 at 21:32, Tim Durack <tdurack@gmail.com> wrote:
> >
> > I would take either LDPv6 or SRv6 - but also need L3VPN (and now EVPN)
> re-wired to use IPv6 NH.
> >
> > I have requested LDPv6 and SRv6 many times from Cisco to migrate the
> routing control plane from IPv4 to IPv6
> >
> > I have lots of IPv6 address space. I don't have a lot of IPv4 address
> space. RFC1918 is not as big as it seems. Apparently this is hard to
> grasp...
> >
> > (This is primarily IOS-XE - can't afford the IOS-XR supercars)
> >
> > On Wed, Jun 10, 2020 at 1:20 PM Mark Tinka <mark.tinka@seacom.mu> wrote:
> >>
> >> Hi all.
> >>
> >> Just want to sample the room and find out if anyone here - especially
> those running an LDP-based BGPv4-free core (or something close to it) -
> would be interested in LDPv6, in order to achieve the same for BGPv6?
> >>
> >> A discussion I've been having with Cisco on the matter is that they do
> not "see any demand" for LDPv6, and thus, won't develop it (on IOS XE).
> Meanwhile, it is actively developed, supported and maintained on IOS XR
> since 5.3.0, with new features being added to it as currently as 7.1.1.
> >>
> >> Needless to say, a bunch of other vendors have been supporting it for a
> while now - Juniper, Nokia/ALU, Huawei, even HP.
> >>
> >> IOS XR supporting LDPv6 notwithstanding, Cisco's argument is that "the
> world" is heavily focused on deploying SRv6 (Segment Routing). While I know
> of one or two questionable deployments, I'm not entirely sure much of the
> world is clamouring to deploy SR, based on all the polls we've done at
> various NOG meetings and within the general list-based operator community
> >>
> >> So I just wanted to hear from this operator community on whether you
> would be interested in having LDPv6 support to go alongside your LDPv4
> deployments, especially if you run native dual-stack backbones. Or if your
> focus is totally on SRv6. Or if you don't care either way :-). Thanks.
> >>
> >> Mark.
> >
> >
> >
> > --
> > Tim:>
>
>
>
> --
> ++ytti
>


--
Tim:>
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 10/Jun/20 20:45, Saku Ytti wrote:

> I'm pretty sure that one or more of Mark, Gert or Tim are thinking
> SR/MPLS IPv6 when they say SRv6?

Oh, not at all, Saku.

> No one in their right minds thinks SRv6 is a good idea, terrible snake
> oil and waste of NRE. SR/MPLS IPv6 of course is terrific.
>
> LDPv6 and SRv6 seem like an odd couple, LDPv6 SR/MPLS IPv6 seem far
> more reasonable couple to choose from. I have my favorite.

I've been tracking SR since it began making "waves" in Paris at a
previous MPLS/SDN/IPv6 Congress meeting a a 2013, or thereabouts. Plenty
of promise about being a decent alternative to LDPv6 (which had been on
my mind since 2008).

In the end, as you rightly point out, it's been much ado about nothing.

To this day, I am yet to hear from a single operator about all the
chanting I hear from vendors - and not just SRv6, but SR in general. 7
years and counting.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 10/Jun/20 21:17, Gert Doering wrote:

>
> We do have IOS XEs today (ASR920), and based on that, we're not going
> to buy new IOS XE devices any time soon.
>
> The amount of... strangeness... that this BU considers acceptable
> is... not.

It's been a week of trying to get them to see reason.

Tough for their "unified" case with the IOS XR team are delivering the
goods.

Mark.

On 10/Jun/20 21:20, Gert Doering wrote:

> Oh. Indeed, sorry for being unclear here.
>
> SR/MPLS sounds like a good idea (reducing label state).
>
> SR/IPv6 does not sound convincing.

+1.

2010 - 2019 has been a decade of "pushing stuff".

2020 is the year of deciding what snake oil no longer deserves energy,
the Coronavirus notwithstanding.

Mark.

On 10/Jun/20 21:36, Phil Bedard wrote:
> In its simplest form without TE paths, there isn't much to SRv6. You use a v6 address as an endpoint and a portion of the address to specify a specific VPN service. You completely eliminate the label distribution protocol.

A BGPv6-free core is a decent use-case for us.

Much simplicity has been enjoyed by removing that in the IPv4 world.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Thu, 11 Jun 2020 at 00:48, Mark Tinka <mark.tinka@seacom.mu> wrote:

> On 10/Jun/20 21:36, Phil Bedard wrote:
> > In its simplest form without TE paths, there isn't much to SRv6. You use a v6 address as an endpoint and a portion of the address to specify a specific VPN service. You completely eliminate the label distribution protocol.
>
> A BGPv6-free core is a decent use-case for us.

100% Eliminating label forwarding in core is not an asset, it is a
liability. Label forwarding is fast, cheap and simple[0]. You can do
it with on-chip memory in constant time. IP lookups are slow,
expensive and complex[0]. SRv6 marketing is false, bordering dishonest
marketing of an unclean abomination of a protocol. Every HW designer
has sighed in relief when I've said I don't care about it, because
it's also very HW unfriendly, like IPv6 generally. Unfortunately SRv6
is somewhat easy to market with the whole 'it's simple, just IP'
spiel.

[0] None of this is hard to measure, it is a known fact. And all of it
matters, you can measure lower jitter for MPLS than IP, you can better
carry DDoS traffic when using MPLS compared to IP and you can have
more ports in front-plate for the same money, by spending external
memory SERDES for WAN ports.



--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Wed, 10 Jun 2020 at 22:36, Phil Bedard <bedard.phil@gmail.com> wrote:

> In its simplest form without TE paths, there isn't much to SRv6. You use a v6 address as an endpoint and a portion of the address to specify a specific VPN service. You completely eliminate the label distribution protocol.

Then do IPv6-in-IPv6, and attach the inner IPv6 header to VRF,
pseudowire, what-have-you.

It is clear market needs tunnelling, and we should all understand that
colour of tunneling doesn't matter, what matters is how many bytes of
overhead does the tunnel add (the more bytes, the more bps leverage
attacker gets) and what is the cost of looking up the headers.
Evaluating 40B IPv6 and 4B MPLS tunneling headers based on objective
desirable qualities of tunneling, MPLS is blatantly better. But if
someone does not like MPLS, fair-game, they should have ability to do
IPV6 in IPv6 in IPv6 in IPv6, go crazy.

I'm not saying we can't improve over MPLS header, we can. But IPv6 is
just objectively inferior by key metrics of 'goodness' of tunneling.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/Jun/20 06:51, Saku Ytti wrote:

> Every HW designer
> has sighed in relief when I've said I don't care about it, because
> it's also very HW unfriendly, like IPv6 generally. Unfortunately SRv6
> is somewhat easy to market with the whole 'it's simple, just IP'
> spiel.

Mine have sighed in disbelief that I don't share their vision of an
SR(v6) world :-).

What's funny is that the ASR920 does not support SRv6, which I think is
more due to hardware limitations than a lack of coding kiddies.
Conversely, you don't need new bits of hardware to support LDPv6.

Today, there is no box that supports LDPv4 that cannot support LDPv6, by
just extending the code. No further hardware needed.

Instead, I'm being asked to "upgrade" to the NCS540 so that I can get
LDPv6. You, sometimes, have to wonder in what world these folk live.

It's a Coronavirus era now - people want to hold on to all the $$ they
don't have, and only spend it where they will extract the most value.
Boeing and Airbus are struggling to reach customers that have pending
deliveries; now isn't the time for vendors to posture. We need value,
not product.

Mark.

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Thu, 11 Jun 2020 at 10:37, Mark Tinka <mark.tinka@seacom.mu> wrote:

> Mine have sighed in disbelief that I don't share their vision of an
> SR(v6) world :-).

I don't like to conflate these two; SR is great, SRv6 is horrible
abomination. SR is what MPLS should have been day1, but it probably
was easier to market LDP than to say 'we need to change all IGP
protocols'.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Wed, Jun 10, 2020, at 20:51, Mark Tinka wrote:

> Well, according to them, SRv6 is winning customers over, and nobody
> wants LDPv6. Then again, they have LDPv6 in IOS XR; figures.

Well, given their (Cisco's) braindead policy regarding non-implementation of LDPv6 on XE, no wonder people are looking for alternatives, and SRv6 is one of them. And don't forget SRv6 is also heavily associated (marketing-wise) with 5G....

Back to our friends and their policy: It happens that in certain regions of the world, if you want to be an ISP other than the "establishment" (== incumbent + "first alternatives" that started 20-25 years ago), you MUST have LNS (if you want to stay in business). If like many, you are kind of stuck with Cisco because it's Cisco, the only decent solution to have LNS is ASR1K (running XE). Also add ASR920 which has a number of uses. Also, in order to stay in business, you may want to offer L3VPN services, which brings you to doing MPLS. You say MPLS, you say LDP, and there you go, your backbone remains v4-based for the next eternity.

There also seems to be a lack of global vision. Tyry to ask your favourite vendor what do you need in order to be able to offer IPv4-L3VPN, IPv6-L3VPN and L2VPN (mainly point-to-point - NO MAC learning) over a backbone that does NOT use any single IPv4 address (backbone-side). Because you can do it on a backbone that does not use any single IPv*6* address, but you may want to go forwards, not backwards. Add a LNS in the mix (the v4 addresses for the LNS go in VRFs - that's not backbone). Add a money, rack space and power needed constraints in the mix. This exercise looks challenging with other vendors too, but with Cisco it's just impossible.

Of course, Cisco says there is no demand for one simple reason : the people talking with Cisco account managers (or whatever they are called) are only rarely those that care about technical stuff. They may want some features on the CPEs (like "ui uant SDWAN"), but for anything else (including backbone equipment) they only want lower prices. You end up with everybody having to deal with a specific platform in real life to dream about a specific feature, yet the vendor to consider that "nobody wants it".

--
R.-A. Feurdean
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/Jun/20 09:42, Saku Ytti wrote:

>
> I don't like to conflate these two; SR is great, SRv6 is horrible
> abomination. SR is what MPLS should have been day1, but it probably
> was easier to market LDP than to say 'we need to change all IGP
> protocols'.

Fair point.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> Mark Tinka
> Sent: Wednesday, June 10, 2020 6:19 PM
>
> Hi all.
>
> Just want to sample the room and find out if anyone here - especially
those
> running an LDP-based BGPv4-free core (or something close to it) - would be
> interested in LDPv6, in order to achieve the same for BGPv6?
>
> A discussion I've been having with Cisco on the matter is that they do not
> "see any demand" for LDPv6, and thus, won't develop it (on IOS XE).
> Meanwhile, it is actively developed, supported and maintained on IOS XR
> since 5.3.0, with new features being added to it as currently as 7.1.1.
>
> Needless to say, a bunch of other vendors have been supporting it for a
> while now - Juniper, Nokia/ALU, Huawei, even HP.
>
> IOS XR supporting LDPv6 notwithstanding, Cisco's argument is that "the
> world" is heavily focused on deploying SRv6 (Segment Routing). While I
know
> of one or two questionable deployments, I'm not entirely sure much of the
> world is clamouring to deploy SR, based on all the polls we've done at
various
> NOG meetings and within the general list-based operator community
>
> So I just wanted to hear from this operator community on whether you
> would be interested in having LDPv6 support to go alongside your LDPv4
> deployments, especially if you run native dual-stack backbones. Or if your
> focus is totally on SRv6. Or if you don't care either way :-). Thanks.
>
Hey Mark,
My stance is that should I go with anything "new" for label distribution the
MPLS SR/SPRING is getting to a point where it might be mature enough.
Also "BGP free core" means internet won't talk to your core -i.e. free to
use private addressing -so no need for v6 at all in the "underlay" (as
hipsters call it these days).
Alternatively using public "infrastructure subnet" (i.e. not advertised to
the Internet) for a "BGP free core", the aim is to make money of the core
-what additional revenue stream am I getting by enabling v6 in the
underlay/management plane that would offset the pain of dealing with the
increased bug surface?

And with regards to the XE/XR discrepancies, I mentioned my prophecy a
number of times, I think XE future in SP products portfolio is next to none.



adam

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Saku Ytti wrote on 11/06/2020 05:51:
> Unfortunately SRv6 is somewhat easy to market with the whole 'it's
> simple, just IP' spiel.
it's not "just IP": it's ipv6 with per-router push / pop operations on
ipv6 extension headers, i.e. high touch in areas which are known to be
deeply troublesome on hardware.

In this regard alone, the specification is problematic enough that it's
unearthed a bug in the IPv6 standard (rfc8200).

Nick
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/Jun/20 10:32, adamv0025@netconsultings.com wrote:

> Hey Mark,
> My stance is that should I go with anything "new" for label distribution the
> MPLS SR/SPRING is getting to a point where it might be mature enough.

"Getting to a point" doesn't really work if you are actively running a
network today :-).

While I do agree that going with the new thing is always a good plan,
one has to truly consider the overall gain vs. labour required to get there.

Going from static routing to an IGP + BGP makes sense when you scale up.

Switching from Distribute Lists to Prefix Lists makes sense when you
scale up.

Route summarization after you dump your old Cisco 2501 for an ASR9901
doesn't add value any longer.

You get the idea.

The position about not needing a label distribution protocol in SR is
actually quite sexy. But considering how powerful router control planes
are nowadays, especially when they are being virtualized on or off
chassis, I just don't see the gains expected by removing LDP and going
SR, on a box and code that supports both. Yes, if you are talking about
dumping a spaghetti of RSVP tunnels, that makes sense as there is a gain
in day-to-day network administration. But in this case, we are just
speaking about LDP.

10 years ago, we worried about how well an IP network would scale
running OSPF or IS-IS. With control planes what they are today, who
really cares anymore? You may recall we've been running CSR1000v for
route reflection since 2014 - millions of routes being carried everyday,
converging in seconds. We've never had to think about those boxes until
last year when we did the server hardware refresh as a matter of course,
not because anything was struggling.

What I'm saying is not all new tech. NEEDS to get deployed just because
it's new. If that was the case, we'd all be running Inter-AS DSCP over
IPoDWDM :-).


> Also "BGP free core" means internet won't talk to your core -i.e. free to
> use private addressing -so no need for v6 at all in the "underlay" (as
> hipsters call it these days).

Careful with that one - Cisco's proposal to me was to run my IPv6
network on link-local :-). Don't encourage them, hehe.


> Alternatively using public "infrastructure subnet" (i.e. not advertised to
> the Internet) for a "BGP free core", the aim is to make money of the core
> -what additional revenue stream am I getting by enabling v6 in the
> underlay/management plane that would offset the pain of dealing with the
> increased bug surface?

I don't know about you, but my BGP-free core is inaccessible from the
world even if it lives in public-IPv4 land. That's how pure MPLS
forwarding works. You'd have be "inside" to reach it (IGP).

Also, if you link every feature to a revenue stream, you'll never deploy
RPKI or DNSSEC :-).


>
> And with regards to the XE/XR discrepancies, I mentioned my prophecy a
> number of times, I think XE future in SP products portfolio is next to none.

Which is fine - but customers are spending real money and need to keep
real networks running with real costs for real years. If Cisco want to
kick IOS XE to the side, let customers know so we can make informed
decisions about where to purchase gear.

The current state-of-the-art is that kit you buy today is probably good
years after standard depreciation policies, probably longer. If Cisco's
model is to throw boxes away sooner than that like they did in the old
days, that is not consistent with where the tech. has gotten to in the
past 2 decades.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/Jun/20 09:58, Radu-Adrian FEURDEAN wrote:

> Well, given their (Cisco's) braindead policy regarding non-implementation of LDPv6 on XE, no wonder people are looking for alternatives, and SRv6 is one of them.

Which doesn't track because there are a number of IOS XE boxes that do
not support SRv6, and probably never will. Meanwhile, no issue with
those boxes supporting LDPv6, as they already support LDPv4.


> And don't forget SRv6 is also heavily associated (marketing-wise) with 5G....

Well, we all know the farce that is 5G.

Also, not all of us run mobile networks, and even for those that do,
there are probably a few that don't buy into the snake oil, especially
those that offer native IPv6 to their mobile customers :-).


> Back to our friends and their policy: It happens that in certain regions of the world, if you want to be an ISP other than the "establishment" (== incumbent + "first alternatives" that started 20-25 years ago), you MUST have LNS (if you want to stay in business). If like many, you are kind of stuck with Cisco because it's Cisco, the only decent solution to have LNS is ASR1K (running XE). Also add ASR920 which has a number of uses. Also, in order to stay in business, you may want to offer L3VPN services, which brings you to doing MPLS. You say MPLS, you say LDP, and there you go, your backbone remains v4-based for the next eternity.

Which I would understand if Cisco's strategy, as a company, was anti-LDP.

But when you have the IOS XR team actively supporting LDP and adding new
features with each major release, the split brain is obvious. You can't
come to me under a "unified" organizational banner, and then tell me you
are a-thousand companies internally. If you choose to fork code into
IOS, IOS XE, IOS XR, NX OS, or whatever new flavour of the decade, don't
make that my problem. Customers don't buy code; they buy boxes that are
built for the task. The code that comes with it is the code that comes
with it.

Moreover, IOS XE also has its own split BU's. So getting IOS XE fixed
for the ASR920 doesn't necessarily mean you get it fixed for the
ASR1000. I mean, how much more complicated does a business have to be?


>
> There also seems to be a lack of global vision. Tyry to ask your favourite vendor what do you need in order to be able to offer IPv4-L3VPN, IPv6-L3VPN and L2VPN (mainly point-to-point - NO MAC learning) over a backbone that does NOT use any single IPv4 address (backbone-side). Because you can do it on a backbone that does not use any single IPv*6* address, but you may want to go forwards, not backwards. Add a LNS in the mix (the v4 addresses for the LNS go in VRFs - that's not backbone). Add a money, rack space and power needed constraints in the mix. This exercise looks challenging with other vendors too, but with Cisco it's just impossible.

That's because it's not about l3vpn6 or l2vpn6; it's about IPv6. And you
can't business-case IPv6.

Some vendors fail to understand that IPv6 is not the money-maker. IPv6
is what attracts the customer to the network so that the money can flow.
It's a means to an end, not the end in itself.


>
> Of course, Cisco says there is no demand for one simple reason : the people talking with Cisco account managers (or whatever they are called) are only rarely those that care about technical stuff. They may want some features on the CPEs (like "ui uant SDWAN"), but for anything else (including backbone equipment) they only want lower prices. You end up with everybody having to deal with a specific platform in real life to dream about a specific feature, yet the vendor to consider that "nobody wants it".

One of the reasons I've never been keen on working for a vendor is
tunnel vision becomes easy. As an operator, you have a much wider view
of what's happening in the real world. When a vendor decides to home-in
on EoMPLS and VPLS being better signaled by LDP vs. BGP, you end up with
situations where they track user-demand through the number of TAC cases
the feature caused. The fewer the TAC cases, the lower the demand. Very
scientific.

Nobody is going to demand LDPv6 because the over-arching problem is a
lack of IPv6 deployment at global scale. If folk don't deploy IPv6, they
will not think about LDPv6, RSVPv6, e.t.c. But to make it worse, if the
vendors keep encouraging the "big-spending" mobile operators to maintain
IPv4 by selling CGN licenses, it's kind of back-to-front, isn't it? No
operator unwilling to deploy IPv6 but favours CGN is ever going to ask a
vendor about LDPv6, especially if the vendor is in charge of building
and operating that backbone as a "professional, managed service".

Ultimately, we are not asking for an ASIC re-spin. We are not asking for
a doubling of forwarding capacity. We are not asking for a greener
chassis. We are not asking for 100Gbps ports. All of which are
physically impossible. We are asking for LDP to extended to support
IPv6. Really, how hard is that?

Mark.


_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

>
> I don't like to conflate these two; SR is great, SRv6 is horrible
> abomination. SR is what MPLS should have been day1, but it probably
> was easier to market LDP than to say 'we need to change all IGP
> protocols'.
>

Nope that was not the main reason.

Main reason was the belief that labels MUST be locally significant - and
not domain wide unique. Just look at Juniper's SRm6 or now SRH ... they
keep this notion of locally significant SIDs. It is deep in their DNA ...
still.

We argued about it a lot in cisco back in TDP days - and we lost.

- - -

Now to your runt that MPLS is great because of exact match perhaps you
missed it but number of solutions on the table (including RbR[**] I
recently proposed) use exact match 4B locator based lookup in the v6
packets to get from segment end to segment end.

On the other hand your comments about greatness of MPLS ... simplified data
plane and depending on the hardware difference in jitter (in sub ms ranges
- if that even matters) comes up with a lot of control plane complexity
when you want to build a network across all continents, yet keep it scoped
from IGP to areas or levels. No summarization in MPLS in FECs is something
we should not sweep under the carpet.

Best,
R.

[**] -
https://mailarchive.ietf.org/arch/msg/ipv6/Ef05LFFij45mm8fM8hLFXknxoIA/
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Mark Tinka wrote on 11/06/2020 10:48:
> We are asking for LDP to extended to support IPv6. Really, how hard
> is that?
Nearly impossible, apparently.

It would require a change of mindset.

Nick
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/Jun/20 11:58, Nick Hilliard wrote:

> Nearly impossible, apparently.
>
> It would require a change of mindset.

I'll leave that to the Coronavirus - it will open eyes.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/Jun/20 11:57, Robert Raszuk wrote:

>
> Nope that was not the main reason. 
>
> Main reason was the belief that labels MUST be locally significant -
> and not domain wide unique. Just look at Juniper's SRm6 or now SRH ...
> they keep this notion of locally significant SIDs. It is deep in their
> DNA ... still. 
>
> We argued about it a lot in cisco back in TDP days - and we lost.

I get this for VLAN's, being only 4,096 per broadcast domain and all.

But are we struggling with scaling label space?

Just my 1+1, since I may be over-simplifying the issue.


>
> - - -
>
> Now to your runt that MPLS is great because of exact match perhaps you
> missed it but number of solutions on the table (including RbR[**] I
> recently proposed) use exact match 4B locator based lookup in the v6
> packets to get from segment end to segment end. 
>
> On the other hand your comments about greatness of MPLS ... simplified
> data plane and depending on the hardware difference in jitter (in sub
> ms ranges - if that even matters) comes up with a lot of control plane
> complexity when you want to build a network across all continents, yet
> keep it scoped from IGP to areas or levels. No summarization in MPLS
> in FECs is something we should not sweep under the carpet.

I found multi-level IS-IS to be useless in an MPLS network because you
still need to leak routes between L2 and L1 in order to form MPLS FEC's.
So you simplify the network by having a single L2 (or just Area 0 in
OSPF), because today's control planes can handle it. And yes, some are
brave enough to run RFC 3107 if it becomes a problem, but if you can
afford to string a network together across all continents, I doubt an
x86-based control plane with 64GB of RAM is topping the list of your
problems.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Thu, 11 Jun 2020 at 12:57, Robert Raszuk <robert@raszuk.net> wrote:

> Nope that was not the main reason.
>
> Main reason was the belief that labels MUST be locally significant - and not domain wide unique. Just look at Juniper's SRm6 or now SRH ... they keep this notion of locally significant SIDs. It is deep in their DNA ... still.

Seems weird, because neither LDP or SR implies globally significant
labels, implementation choice. What SR does imply is a continuous
block of labels of equal size in domain.

> Now to your runt that MPLS is great because of exact match perhaps you missed it but number of solutions on the table (including RbR[**] I recently proposed) use exact match 4B locator based lookup in the v6 packets to get from segment end to segment end.

Which is making a simple thing a complex thing.

> On the other hand your comments about greatness of MPLS ... simplified data plane and depending on the hardware difference in jitter (in sub ms ranges - if that even matters) comes up with a lot of control plane complexity when you want to build a network across all continents, yet keep it scoped from IGP to areas or levels. No summarization in MPLS in FECs is something we should not sweep under the carpet.

MPLS complexity is trivial in control-plane and forwarding-plane
compared to IPV6 or SRv6 or RbR[**]. I'm not saying we can't do better
than MPLS, but the proposal here is blatantly worse.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

/* removing nanog list as I am not subscribed there and it bounces back */

> I found multi-level IS-IS to be useless in an MPLS network because you
still need to leak routes between L2 and L1 in order to form
> MPLS FEC's. So you simplify the network by having a single L2 (or just
Area 0 in OSPF), because today's control planes can handle it.

Spot on ! OSPF is not any better.

And yes you can build a global flat IGP. But this is not a design I would
endorse in most networks.

Reason is that most networks do not have latest connectivity restoration
techniques and still wait for typical "IGP convergence" So if you flood
globally you need to adjust your IGP & SPF timers with the notion of global
flooding.

If you however scope your flooding domains to be relatively small (say per
1-few regions in a continent) you can easily and safely make those timers
much more aggressive hence significantly reducing connectivity restoration
times upon failures.

Many thx,
R.




On Thu, Jun 11, 2020 at 12:15 PM Mark Tinka <mark.tinka@seacom.mu> wrote:

>
>
> On 11/Jun/20 11:57, Robert Raszuk wrote:
>
>
> Nope that was not the main reason.
>
> Main reason was the belief that labels MUST be locally significant - and
> not domain wide unique. Just look at Juniper's SRm6 or now SRH ... they
> keep this notion of locally significant SIDs. It is deep in their DNA ...
> still.
>
> We argued about it a lot in cisco back in TDP days - and we lost.
>
>
> I get this for VLAN's, being only 4,096 per broadcast domain and all.
>
> But are we struggling with scaling label space?
>
> Just my 1+1, since I may be over-simplifying the issue.
>
>
>
> - - -
>
> Now to your runt that MPLS is great because of exact match perhaps you
> missed it but number of solutions on the table (including RbR[**] I
> recently proposed) use exact match 4B locator based lookup in the v6
> packets to get from segment end to segment end.
>
> On the other hand your comments about greatness of MPLS ... simplified
> data plane and depending on the hardware difference in jitter (in sub ms
> ranges - if that even matters) comes up with a lot of control plane
> complexity when you want to build a network across all continents, yet keep
> it scoped from IGP to areas or levels. No summarization in MPLS in FECs is
> something we should not sweep under the carpet.
>
>
> I found multi-level IS-IS to be useless in an MPLS network because you
> still need to leak routes between L2 and L1 in order to form MPLS FEC's. So
> you simplify the network by having a single L2 (or just Area 0 in OSPF),
> because today's control planes can handle it. And yes, some are brave
> enough to run RFC 3107 if it becomes a problem, but if you can afford to
> string a network together across all continents, I doubt an x86-based
> control plane with 64GB of RAM is topping the list of your problems.
>
> Mark.
>
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> Seems weird, because neither LDP or SR implies globally significant
> labels, implementation choice. What SR does imply is a continuous
> block of labels of equal size in domain.
>

LDP or MPLS LSPs require hop by hop label swapping (directly connected or
over say IP tunnels). So labels in LDP are always locally significant.

SR-MPLS implies globally (domain wide) label significance. Index in the IGP
extensions is there just to relax requirement to use the same block on all
nodes - so the block does not need to be continuous in an SR domain. It is
recommended by Cisco but not required by standard.

Thx
R.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/Jun/20 12:28, Robert Raszuk wrote:

>
> Spot on ! OSPF is not any better. 
>
> And yes you can build a global flat IGP. But this is not a design I
> would endorse in most networks. 
>
> Reason is that most networks do not have latest connectivity
> restoration techniques and still wait for typical "IGP convergence" So
> if you flood globally you need to adjust your IGP & SPF timers with
> the notion of global flooding. 
>
> If you however scope your flooding domains to be relatively small (say
> per 1-few regions in a continent) you can easily and safely make those
> timers much more aggressive hence significantly reducing connectivity
> restoration times upon failures.

Well, we operate a single IS-IS L2 domain across 3 continents.

We use what-I'd-call aggressive IS-IS detection and convergence timers,
in addition to BFD and LFA/IP-FRR.

We do very okay.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> Well, we operate a single IS-IS L2 domain across 3 continents.
>
> We use what-I'd-call aggressive IS-IS detection and convergence timers,
> in addition to BFD and LFA/IP-FRR.
>
> We do very okay.
>

No doubt.

However one network is not equal the other. Especially SP/ISP network
requirements and any to any traffic patterns there are very different from
typical hub and spoke connectivity in the content or service serving
enterprise.

So if I am to put 1000 routers in the flat network, reduce OSPF timers and
inject all 1000 BGP next hops with label to all 999 PEs while I only need
999 PEs to ever reach 10 next hops I would keep to argue flat IGP is not
the right choice.

Moreover as one of the best industry IGP developer and expert I highly
respect stated very recently the end to end flooding time should stay below
200 ms. That is not your ICMP RTT ... that is time to receive the LSA/LSP
to local RE, install in LSDB and send it back. That determines the flooding
radius you should try not to exceed. And the time packet takes to even get
from LC to RE is very platform dependent as I am sure most of you know very
well :)

Cheers,
R.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

We're deploying some new POPs with a mix of IOS-XR as borders, BNG and
PEs and IOS-XE for LNSs. LDPv6 would be awesome to have availabke on
IOS-XE alongside LDPv4. We're being pushed by Cisco to go one flavour
or another of SR as well by Cisco, but i'd much prefer to have LDPv4 &
LDPv6 right now.

We have some buying power with Cisco at the moment, so let me know how
I can help from the land 'down under'.

Drikus.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/Jun/20 12:54, Robert Raszuk wrote:

> No doubt. 
>
> However one network is not equal the other. Especially SP/ISP network
> requirements and any to any traffic patterns there are very different
> from typical hub and spoke connectivity in the content or service
> serving enterprise.

Totally agree, which is why I said if you are building a backbone across
all continents, the ability of your hardware is the least of your problems.

You'll probably spend more time dealing with backbone and license issues.


>
> So if I am to put 1000 routers in the flat network, reduce OSPF timers
> and inject all 1000 BGP next hops with label to all 999 PEs while I
> only need 999 PEs to ever reach 10 next hops I would keep to argue
> flat IGP is not the right choice.

While I don't use OSPF, that almost sounds like something you'd do at an
EBC with a vendor in the lab.

Real networks may operate slightly less over-zealously :-).

But who knows, only time will tell. The problem is each time we seem to
reach the peak of what control planes can do, they up the ante.

If I was having this discussion with you 10 years ago, I'd be in total
agreement.


>
> Moreover as one of the best industry IGP developer and expert I highly
> respect stated very recently the end to end flooding time should stay
> below 200 ms. That is not your ICMP RTT ... that is time to receive
> the LSA/LSP to local RE, install in LSDB and send it back. That
> determines the flooding radius you should try not to exceed. And the
> time packet takes to even get from LC to RE is very platform dependent
> as I am sure most of you know very well :)

I won't diss the IGP developers, they help us to eat. But after being in
the game this long, I've often found that caution and real life don't
always align.

Naturally, we will take the considerations, and see what feedback we get
from the actual network, and adjust accordingly.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/Jun/20 13:03, Drikus Brits wrote:
> We're deploying some new POPs with a mix of IOS-XR as borders, BNG and
> PEs and IOS-XE for LNSs. LDPv6 would be awesome to have availabke on
> IOS-XE alongside LDPv4. We're being pushed by Cisco to go one flavour
> or another of SR as well by Cisco, but i'd much prefer to have LDPv4 &
> LDPv6 right now.
>
> We have some buying power with Cisco at the moment, so let me know how
> I can help from the land 'down under'.

Thanks, Drikus.

If you can get your AM to get in touch with the ASR920 and ASR1000 BU's
to ask for LDPv6, that will help a great deal. Cisco are saying "no one"
is asking for LDPv6, forgetting that it's more about IPv6, than other
lower/higher-level services it can support, LDPv6 included.

As with you, they are trying to shove SRv6 down our throats, which can
only come on an NCS540 for the moment, which is odd since that runs IOS
XR, which supports LDPv6. So basically, swap out tons of boxes to get a
feature that can easily be coded for in existing hardware... makes you
wonder about the thought-process;

It's been a crazy two weeks with our Cisco AM's and the platform TME's
on heated calls about this that it's almost old testament biblical :-).
It's at the stage where it's going up the chain to the BU's who will
make the final call. So if they can get some noise from you as well, it
certainly won't hurt.

You don't need new hardware features on the ASR920 or ASR1000 to support
LDPv6. You might do for SRv6. Either way, both platforms would need both
features developed in code, and I think it's safe to assume which one
will be practically easier to build and roll-out to the entire MPLS
customer base, today.

Mark.

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Thu, 11 Jun 2020 at 13:33, Robert Raszuk <robert@raszuk.net> wrote:

> SR-MPLS implies globally (domain wide) label significance. Index in the IGP extensions is there just to relax requirement to use the same block on all nodes - so the block does not need to be continuous in an SR domain. It is recommended by Cisco but not required by standard.

As all need to agree on having label at a given offset for the
application, the range either needs to be continuous or have the same
holes which are not used by anyone. It is slightly more convenient to
an equal size continuous label.
This is not unlike VPLS.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> From: Mark Tinka <mark.tinka@seacom.mu>
> Sent: Thursday, June 11, 2020 10:21 AM
>
>> -what additional revenue stream am I getting by enabling v6 in the
>> underlay/management plane that would offset the pain of dealing with the
>> increased bug surface?
>
> Also, if you link every feature to a revenue stream, you'll never deploy
> RPKI or DNSSEC :-).
>
Well RPKI or DNSSEC is at least adding a value, even something you can brag about to your customers (we are part of the solution not part of the problem etc...).

But running MPLS over IPv6 in addition to already running it over IPv4, gaining new functionality or features in the process that could be ultimately monetized in providing better service to customers? -maybe even exposing the network to a new set of bugs? -I don't know, that doesn’t sound like a good use of company resources especially in these uncertain times . Maybe I'm being overly harsh here maybe if you could please explain the drivers or expected net value out of this exercise please?


adam

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Hi,

On Thu, Jun 11, 2020 at 01:17:55PM +0100, adamv0025@netconsultings.com wrote:
> But running MPLS over IPv6 in addition to already running it over IPv4,

Not "in addition to" but "to throw out the IPv4 part" :-)

More relevant to strongly growing networks that do not need to roll out
IPv4 to new parts, though.

(If I had LDPv6 today, I wouldn't actually change the existing network
today. But for the next round of rebuilding things, it would be something
to consider...)

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de

On 11/Jun/20 14:17, adamv0025@netconsultings.com wrote:

> Well RPKI or DNSSEC is at least adding a value, even something you can brag about to your customers (we are part of the solution not part of the problem etc...).

Bragging doesn't bring in income, it's just PR :-).


>
> But running MPLS over IPv6 in addition to already running it over IPv4, gaining new functionality or features in the process that could be ultimately monetized in providing better service to customers? -maybe even exposing the network to a new set of bugs? -I don't know, that doesn’t sound like a good use of company resources especially in these uncertain times . Maybe I'm being overly harsh here maybe if you could please explain the drivers or expected net value out of this exercise please?

Oh dear, you sound like our Finance department now; "drivers" and "net
value" :-).

If I take your line of reasoning, deploying SRv6 likely requires new
hardware, which means $$. How much money will I charge customers for my
shiny new SRv6?

LDPv6 builds on LDPv4. Just like IPv6 builds on IPv4. At best, you can
remove BGPv6 from your core, which lowers your administration costs in
that part of the network even further, costing you less in human time
running it, resources you can otherwise re-deploy to other time- and
money-saving activities. At worst, you get IPv4/IPv6 feature parity, and
who doesn't like that :-).

And how much money did LDPv6 cost you to deploy? $0.

Mark.

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/Jun/20 14:43, Gert Doering wrote:

>
> Not "in addition to" but "to throw out the IPv4 part" :-)

That's another view-point, yes.

There are plenty of networks that can't afford to keep buying IPv4 on
the open market. They want to go single-stack IPv6.

Today, you would need to build a 6PE network if you want MPLS support in
your IPv6-only network, which still requires IPv4. Private IPv4 address
space works, but some people like it, some don't, and more importantly,
you still carry around IPv4.

You could try the SRv6 gravy, but now you probably need new kit. What's
the SRv6 business-case? Well, one would say you save millions of $$ not
buying IPv4, but now you've spent money supporting SRv6.

LDPv6 can be supported by any platform that supports MPLS today. No
money out the door. Cisco would need to develop code for both LDPv6 and
SRv6 anyway, so no harm done on that side.

As we used to say in Vladivostok, "It's simple physics :-)".


> (If I had LDPv6 today, I wouldn't actually change the existing network
> today. But for the next round of rebuilding things, it would be something
> to consider...)

Give your favorite Cisco AM a shout-out. I promise, they are probably
too young to remember your 6500/7600 tickets :-).

Mark.

On Thu, 11 Jun 2020 at 15:58, Mark Tinka <mark.tinka@seacom.mu> wrote:

> > Well RPKI or DNSSEC is at least adding a value, even something you can brag about to your customers (we are part of the solution not part of the problem etc...).
>
> Bragging doesn't bring in income, it's just PR :-).

Unsure of others, but we didn't implement RPKI for shit and giggles,
we implemented it, because customers wanted us to do RPKI.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/Jun/20 15:34, Saku Ytti wrote:

>
> Unsure of others, but we didn't implement RPKI for shit and giggles,
> we implemented it, because customers wanted us to do RPKI.

I'll be honest, none of our customers ever asked us to deploy RPKI and
ROV. Will they benefit from it, sure? Is it about to become part of an
RFP requirements document? Probably not.

Then again, the typical NTT customer probably has an engineering
department that understands and demands RPKI, because they are a
sizeable operator themselves. We only have a handful of customers that
technically appreciate that we've deployed RPKI. For the rest, it
doesn't register.

Not sure if beating up on Job for months qualifies as "a customer
wanting RPKI from NTT" :-).

Mark.


_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> From: Mark Tinka <mark.tinka@seacom.mu>
> Sent: Thursday, June 11, 2020 1:56 PM
>
> On 11/Jun/20 14:17, adamv0025@netconsultings.com wrote:
>
> > Well RPKI or DNSSEC is at least adding a value, even something you can
> brag about to your customers (we are part of the solution not part of the
> problem etc...).
>
> Bragging doesn't bring in income, it's just PR :-).
>
Good PR might ;)

>
> >
> > But running MPLS over IPv6 in addition to already running it over IPv4,
> gaining new functionality or features in the process that could be ultimately
> monetized in providing better service to customers? -maybe even exposing
> the network to a new set of bugs? -I don't know, that doesn’t sound like a
> good use of company resources especially in these uncertain times . Maybe
> I'm being overly harsh here maybe if you could please explain the drivers or
> expected net value out of this exercise please?
>
> Oh dear, you sound like our Finance department now; "drivers" and "net
> value" :-).
>
I thought I sound like a network architect :(

> If I take your line of reasoning, deploying SRv6 likely requires new hardware,
> which means $$. How much money will I charge customers for my shiny new
> SRv6?
>
No, my line of reasoning is if you have MPLS LSPs signalled over v4 I see no point having them signalled also over v6 in parallel.
Or if your full-mesh RSVP-TE is killing your RSVP-TE or you're in need of TE, then might want to look at SR MPLS.

> LDPv6 builds on LDPv4. Just like IPv6 builds on IPv4. At best, you can remove
> BGPv6 from your core,
I'm using VPNv6 & VPNv4 so not sure I follow how LDPv6 allows for removal of BGPv6 (is it BGPv6 over v4 NHs/transport?) but then again if it works over v4...

> which lowers your administration costs in that part of
> the network even further, costing you less in human time running it,
> resources you can otherwise re-deploy to other time- and money-saving
> activities. At worst, you get IPv4/IPv6 feature parity, and who doesn't like
> that :-).
>
I knew there's a bit of OCD hidden in this at some level :)

> And how much money did LDPv6 cost you to deploy? $0.
>
Apart from X months worth of functionality, performance, scalability and interworking testing -network wide code upgrades to address the bugs found during the testing process and then finally rollout across the core and possibly even migration from LDPv4 to LDPv6, involving dozens of people from Arch, Design, OPS, Project management, etc... with potential for things to break while making changes in live network.


adam

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Thu, 11 Jun 2020 at 16:45, Mark Tinka <mark.tinka@seacom.mu> wrote:

> Not sure if beating up on Job for months qualifies as "a customer
> wanting RPKI from NTT" :-).

I'm sure we can blame Job for this, why not. But probably because of
his lobbying some customers are _requiring_, i.e. flat out told they
will stop accepting transit offers from providers who don't do RPKI.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> From: Saku Ytti <saku@ytti.fi>
> Sent: Thursday, June 11, 2020 3:29 PM
>
> On Thu, 11 Jun 2020 at 16:45, Mark Tinka <mark.tinka@seacom.mu> wrote:
>
> > Not sure if beating up on Job for months qualifies as "a customer
> > wanting RPKI from NTT" :-).
>
> I'm sure we can blame Job for this, why not. But probably because of his
> lobbying some customers are _requiring_, i.e. flat out told they will stop
> accepting transit offers from providers who don't do RPKI.
>
Case in point.

On the other hand not sure if any of the customers would care whether LSPs are signalled over v4 as opposed to v6.

adam


_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/Jun/20 16:28, Saku Ytti wrote:

> I'm sure we can blame Job for this, why not. But probably because of
> his lobbying some customers are _requiring_, i.e. flat out told they
> will stop accepting transit offers from providers who don't do RPKI.

As my Chad friend would say, "I like the sound of this".

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/Jun/20 16:36, adamv0025@netconsultings.com wrote:

> Case in point.
>
> On the other hand not sure if any of the customers would care whether LSPs are signalled over v4 as opposed to v6.

They care if your core router CPU doesn't struggle from dealing with
churning BGP routes at scale, taking the network down.

Not every bit of good network operation can be attributed to direct
revenue. BCP-38 is a great example, and that is still poorly deployed
despite being supported.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/Jun/20 16:25, adamv0025@netconsultings.com wrote:

> Good PR might ;)

I'm old school - build something customers want to use, and the money
follows.

Care to guess how much PR Tik Tok do :-)? But I digress.


> No, my line of reasoning is if you have MPLS LSPs signalled over v4 I see no point having them signalled also over v6 in parallel.

It's not about signaling IPv4 LSP's over IPv6.

LDPv4 creates IPv4 FEC's.

LDPv6 creates IPv6 FEC's.

The idea is to create IPv6 FEC's so that IPv6 traffic can be
label-switched in the network natively, allowing you to remove BGPv6 in
a native dual-stack core.


> Or if your full-mesh RSVP-TE is killing your RSVP-TE or you're in need of TE, then might want to look at SR MPLS.

No RSVP-TE here, thank the Lord :-).


> I'm using VPNv6 & VPNv4 so not sure I follow how LDPv6 allows for removal of BGPv6 (is it BGPv6 over v4 NHs/transport?) but then again if it works over v4...

Nothing like the real thing:

In IPv4-land, you get this:

24005  49017       105.16.Y.Z/32     Te0/0/0/2    105.16.A.B    8614773
            49017       105.16.Y.Z/32     Te0/1/0/0    105.16.C.D   8121753
            49017       105.16.Y.Z/32     Te0/7/0/5    105.16.E.F    9964543

In IPv6-land, you get this:

2c0f:feb0:X:Y::Z/128 25071   25458          BE1          Link-local
                                              25458         
BE2          Link-local


In Junos land, it looks like this:

7967               *[LDP/9] 1w5d 01:21:05, metric 1
                        >  to fe80::de38:e1ff:fe15:dc02 via ae2.0, Swap
145674

If I run an IPv6 traceroute toward a host that sits behind a router
doing LDPv6, this is what happens:

run traceroute 2c0f:feb0:2f:ff00:WWWW:XXXX:YYYY:ZZZZ
traceroute6 to 2c0f:feb0:2f:ff00:WWWW:XXXX:YYYY:ZZZZ
(2c0f:feb0:2f:ff00:WWWW:XXXX:YYYY:ZZZZ) from 2c0f:feb0:1:2::112, 64 hops
max, 12 byte packets
 1  ae-5-0.er6-01-fra.de.seacomnet.com (2c0f:feb0:1:2::111)  165.567 ms 
166.158 ms  166.924 ms
     MPLS Label=145786 CoS=0 TTL=1 S=1
 2  xe-0-0-0-0.cr6-02-mrs.fr.seacomnet.com (2c0f:feb0:1:2::f9)  165.682
ms 2c0f:feb0:1:2::279 (2c0f:feb0:1:2::279)  165.817 ms  168.123 ms
     MPLS Label=25494 CoS=0 TTL=1 S=1
<snip>

As you can see, just as with IPv4, IPv6 packets are now being
MPLS-switched in the core, allowing you to remove BGPv6 in the core and
simplify operations in that area of the network.

So this is native MPLSv6. It's not 6PE or 6VPE.

Your IPv4 domain could fall over and your MPLSv6 network will still be
alive, because it's neither 6PE nor 6VPE. And vice versa - your IPv6
network could die, and your MPLSv4 will be unaffected.


> I knew there's a bit of OCD hidden in this at some level :)

Safe to say the Internet is one big OCD project :-).

My OCD is sleeping at 3AM, in peace, lockdown or not, hehe.


> Apart from X months worth of functionality, performance, scalability and interworking testing -network wide code upgrades to address the bugs found during the testing process and then finally rollout across the core and possibly even migration from LDPv4 to LDPv6, involving dozens of people from Arch, Design, OPS, Project management, etc... with potential for things to break while making changes in live network.

Which you wouldn't have to do with SRv6, because you trust the vendors?

And no, LDPv6 does not call for one to migrate from LDPv4. They can live
together, side-by-side, just as native dual-stack IPv4/IPv6 does.

We are running LDPv4 and LDPv6 side-by-side, with no problems, between
IOS XR and Junos, as you can see above. This is a live network, carrying
revenue-generating, production traffic. It's not a fantasy.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

please remove me from this mailing list

On 11/06/2020 15:58, Mark Tinka wrote:
>
> On 11/Jun/20 16:25, adamv0025@netconsultings.com wrote:
>
>> Good PR might ;)
> I'm old school - build something customers want to use, and the money
> follows.
>
> Care to guess how much PR Tik Tok do :-)? But I digress.
>
>
>> No, my line of reasoning is if you have MPLS LSPs signalled over v4 I see no point having them signalled also over v6 in parallel.
> It's not about signaling IPv4 LSP's over IPv6.
>
> LDPv4 creates IPv4 FEC's.
>
> LDPv6 creates IPv6 FEC's.
>
> The idea is to create IPv6 FEC's so that IPv6 traffic can be
> label-switched in the network natively, allowing you to remove BGPv6 in
> a native dual-stack core.
>
>
>> Or if your full-mesh RSVP-TE is killing your RSVP-TE or you're in need of TE, then might want to look at SR MPLS.
> No RSVP-TE here, thank the Lord :-).
>
>
>> I'm using VPNv6 & VPNv4 so not sure I follow how LDPv6 allows for removal of BGPv6 (is it BGPv6 over v4 NHs/transport?) but then again if it works over v4...
> Nothing like the real thing:
>
> In IPv4-land, you get this:
>
> 24005  49017       105.16.Y.Z/32     Te0/0/0/2    105.16.A.B    8614773
>             49017       105.16.Y.Z/32     Te0/1/0/0    105.16.C.D   8121753
>             49017       105.16.Y.Z/32     Te0/7/0/5    105.16.E.F    9964543
>
> In IPv6-land, you get this:
>
> 2c0f:feb0:X:Y::Z/128 25071   25458          BE1          Link-local
>                                               25458
> BE2          Link-local
>
>
> In Junos land, it looks like this:
>
> 7967               *[LDP/9] 1w5d 01:21:05, metric 1
>                         >  to fe80::de38:e1ff:fe15:dc02 via ae2.0, Swap
> 145674
>
> If I run an IPv6 traceroute toward a host that sits behind a router
> doing LDPv6, this is what happens:
>
> run traceroute 2c0f:feb0:2f:ff00:WWWW:XXXX:YYYY:ZZZZ
> traceroute6 to 2c0f:feb0:2f:ff00:WWWW:XXXX:YYYY:ZZZZ
> (2c0f:feb0:2f:ff00:WWWW:XXXX:YYYY:ZZZZ) from 2c0f:feb0:1:2::112, 64 hops
> max, 12 byte packets
>  1  ae-5-0.er6-01-fra.de.seacomnet.com (2c0f:feb0:1:2::111)  165.567 ms
> 166.158 ms  166.924 ms
>      MPLS Label=145786 CoS=0 TTL=1 S=1
>  2  xe-0-0-0-0.cr6-02-mrs.fr.seacomnet.com (2c0f:feb0:1:2::f9)  165.682
> ms 2c0f:feb0:1:2::279 (2c0f:feb0:1:2::279)  165.817 ms  168.123 ms
>      MPLS Label=25494 CoS=0 TTL=1 S=1
> <snip>
>
> As you can see, just as with IPv4, IPv6 packets are now being
> MPLS-switched in the core, allowing you to remove BGPv6 in the core and
> simplify operations in that area of the network.
>
> So this is native MPLSv6. It's not 6PE or 6VPE.
>
> Your IPv4 domain could fall over and your MPLSv6 network will still be
> alive, because it's neither 6PE nor 6VPE. And vice versa - your IPv6
> network could die, and your MPLSv4 will be unaffected.
>
>
>> I knew there's a bit of OCD hidden in this at some level :)
> Safe to say the Internet is one big OCD project :-).
>
> My OCD is sleeping at 3AM, in peace, lockdown or not, hehe.
>
>
>> Apart from X months worth of functionality, performance, scalability and interworking testing -network wide code upgrades to address the bugs found during the testing process and then finally rollout across the core and possibly even migration from LDPv4 to LDPv6, involving dozens of people from Arch, Design, OPS, Project management, etc... with potential for things to break while making changes in live network.
> Which you wouldn't have to do with SRv6, because you trust the vendors?
>
> And no, LDPv6 does not call for one to migrate from LDPv4. They can live
> together, side-by-side, just as native dual-stack IPv4/IPv6 does.
>
> We are running LDPv4 and LDPv6 side-by-side, with no problems, between
> IOS XR and Junos, as you can see above. This is a live network, carrying
> revenue-generating, production traffic. It's not a fantasy.
>
> Mark.
> _______________________________________________
> cisco-nsp mailing list cisco-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Respectfully, that is deployment dependent. In a traditional SP topology that focuses on large do everything boxes, where the topology is fairly point-to-point and you only have a small handful of nodes at a PoP, labels can be fast, cheap and easy. Given the lack of ECMP/WECMP, they remain fairly efficient within the hardware.

However if you move away from large multi-chip systems, which hide internal links which can only be debugged and monitored if you know the the obscure, often different ways in which they are partially exposed to the operator, and to a system of fixed form-factor, single chip systems, labels fall apart at scale with high ECMP. Needing to enumerate every possible path within the network or having to have a super-deep label stack removes all of the perceived benefits of cheap and simple. The arguments about IP lookups being slow is one best left to the 1990's when it was true. Fixed pipeline systems have proven this to be false.

And if the argument is coming down to the lookup delay between the two and your talking about the WAN, speed of light is dominate so the ns differences are in the noise, baring the monetary exchanges.

Your follow on statement about more ports and smaller silicon is in abstract somewhat correct but is practice not. Across the silicon manufactures no one in the open market is making a purely MPLS capable chip. With the cost of chips today, you must have multiple customers for the silicon, or, if you are using wholly internally, have enough volume to justify the $100M's that it costs to make. So everyone is optimizing around re-use and maximal customer overlap for execution. So they all do MPLS and IP and SR and IPIP and ... making the argument moot.

But please to not take this as saying SRv6 is great. The community got v6 wrong in a number of areas and SR is not helping that. v4 as a transport vs. MPLS is a useful conversation to be had, again depending on deployment and philosophy around large vs. small network nodes.

David

> On Jun 10, 2020, at 9:51 PM, Saku Ytti <saku@ytti.fi> wrote:
>
> On Thu, 11 Jun 2020 at 00:48, Mark Tinka <mark.tinka@seacom.mu> wrote:
>
>> On 10/Jun/20 21:36, Phil Bedard wrote:
>>> In its simplest form without TE paths, there isn't much to SRv6. You use a v6 address as an endpoint and a portion of the address to specify a specific VPN service. You completely eliminate the label distribution protocol.
>>
>> A BGPv6-free core is a decent use-case for us.
>
> 100% Eliminating label forwarding in core is not an asset, it is a
> liability. Label forwarding is fast, cheap and simple[0]. You can do
> it with on-chip memory in constant time. IP lookups are slow,
> expensive and complex[0]. SRv6 marketing is false, bordering dishonest
> marketing of an unclean abomination of a protocol. Every HW designer
> has sighed in relief when I've said I don't care about it, because
> it's also very HW unfriendly, like IPv6 generally. Unfortunately SRv6
> is somewhat easy to market with the whole 'it's simple, just IP'
> spiel.
>
> [0] None of this is hard to measure, it is a known fact. And all of it
> matters, you can measure lower jitter for MPLS than IP, you can better
> carry DDoS traffic when using MPLS compared to IP and you can have
> more ports in front-plate for the same money, by spending external
> memory SERDES for WAN ports.
>
>
>
> --
> ++ytti
> _______________________________________________
> cisco-nsp mailing list cisco-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Thu, 11 Jun 2020 at 18:32, David Sinn <dsinn@dsinn.com> wrote:

> However if you move away from large multi-chip systems, which hide internal links which can only be debugged and monitored if you know the the obscure, often different ways in which they are partially exposed to the operator, and to a system of fixed form-factor, single chip systems, labels fall apart at scale with high ECMP. Needing to enumerate every possible path within the network or having to have a super-deep label stack removes all of the perceived benefits of cheap and simple. The arguments about IP lookups being slow is one best left to the 1990's when it was true. Fixed pipeline systems have proven this to be false.

It continues to be very much true. IP lookups require external memory,
which takes SERDES, which could be used for revenue otherwise. IP
lookups are slow, expensive and complex, fundamentally, no amount of
advancement will change this fundamental nature.
Sure we can come up with all kind of implementations which bridge the
gap, but the gap is there.

If we take say JNPR MX, your lookup speed isn't limited by the
instruction count on the PPE, the PPE spends most of its time
sleeping, when the platform is fully PPS congested, the PPE is waiting
for the memory to return!

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Thu, 11 Jun 2020 at 19:49, Phil Bedard <bedard.phil@gmail.com> wrote:

> As for normal v6 forwarding, the way most higher speed routers made recently work there is little difference in latency since the encapsulation for the packet is done in a common function at the end of the pipeline and the lookups are often in the same memory space. NPUs are also being built today with enough on-package memory to hold larger routing tables. Whether a packet has to be buffered on-chip vs. off-chip has a much larger impact on latency/PDV than a forwarding lookup.

On-package is not important, on-chip or off-chip is what matters, i.e.
do you eat SERDES to connect memory or not.

Of course you could always implement a software feature that says
these 32b/32 or 128b/128 addresses are blessed and need to live on
tiny on-chip memory and from this CIDR we guarantee all are host
routes. To achieve similar-to-MPLS performance, with few more bytes
per number.

The demand is, we need tunneling, then the question is what are the
metrics of a good tunneling solution. By answering this honestly, MPLS
is better. We could do better surely, but IP is not that.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Phil Bedard wrote on 11/06/2020 17:49:
> Just to clarify the only routers who potentially need to inspect or
> do anything with those headers are endpoints who require information
> in the extension header or hops in an explicit path. In the simple
> example I gave, there are no extension headers at all.

perhaps, but no-one planning to use srv6 is going to invest in kit which
can handle srv6 but not the TE component. Or deploy srv6 on existing
kit which can't handle TE.

Nick
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> On Jun 11, 2020, at 8:41 AM, Saku Ytti <saku@ytti.fi> wrote:
>
> On Thu, 11 Jun 2020 at 18:32, David Sinn <dsinn@dsinn.com> wrote:
>
>> However if you move away from large multi-chip systems, which hide internal links which can only be debugged and monitored if you know the the obscure, often different ways in which they are partially exposed to the operator, and to a system of fixed form-factor, single chip systems, labels fall apart at scale with high ECMP. Needing to enumerate every possible path within the network or having to have a super-deep label stack removes all of the perceived benefits of cheap and simple. The arguments about IP lookups being slow is one best left to the 1990's when it was true. Fixed pipeline systems have proven this to be false.
>
> It continues to be very much true. IP lookups require external memory,
> which takes SERDES, which could be used for revenue otherwise. IP
> lookups are slow, expensive and complex, fundamentally, no amount of
> advancement will change this fundamental nature.
> Sure we can come up with all kind of implementations which bridge the
> gap, but the gap is there.

But now you are comparing apples and oranges. You're asserting that all IP lookups require external memory. But your talking about comparing a lite-core to a heavy-core. As I said, it depend on deployments. If you have a lite-IP core you don't need external memories. So there isn't a lag question going out to massive memories needed for 2M+ entires. So it is not always that lookups are slow, expensive, complex. Sure, you can build a network around a heavy core but you can also build one without. Sweeping generalizations that MPLS is always better than all other technologies is just that, a sweeping generalization. It misses a ton of points.

Rewrites on MPLS is horrible from a memory perspective as maintaining the state and label transition to explore all possible discrete paths across the overall end-to-end path you are trying to take is hugely in-efficient. Applying circuit switching to a packet network was bad from the start. SR doesn't resolve that, as you are stuck with a global label problem and the associated lack of being able to engineer your paths, or a label stack problem on ingress that means you need a massive ASIC's and memories there.

IP at least gives you rewrite sharing, so in a lite-core you have way better trade-off on resources, especially in a heavily ECMP'ed network. Such as one build of massive number of open small boxes vs. a small number of huge opaque ones. Pick your poison but saying one is inheriantly better then another in all cases is just plane false.

> If we take say JNPR MX, your lookup speed isn't limited by the
> instruction count on the PPE, the PPE spends most of its time
> sleeping, when the platform is fully PPS congested, the PPE is waiting
> for the memory to return!

You've made my point for me. If you are building the core of your network out of MX's, to turn a phrase, in a past life "I fully support my competitors to do so". Large numbers of small boxes, as they have already shown in the data-center, have major cost, control and operational advantages over a small number of large ones. They also expose the inherent problems of label-switching and where IP has it's merits.

David

>
> --
> ++ytti

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Thu, 11 Jun 2020 at 21:04, David Sinn <dsinn@dsinn.com> wrote:

> You've made my point for me. If you are building the core of your network out of MX's, to turn a phrase, in a past life "I fully support my competitors to do so". Large numbers of small boxes, as they have already shown in the data-center, have major cost, control and operational advantages over a small number of large ones. They also expose the inherent problems of label-switching and where IP has it's merits.

Except this implementation does not exist, but we can argue that is
missing feature. We can argue we should be able to tell the lookup
engine this CIDR is on-chip and it's host routes only. This is
certainly doable, and would make IP tunnels like MPLS tunnels for
lookup cost, just larger lookup key, which is not significant cost.

But even if we had this (we don't, we have for MPLS) IP would be still
inferior, it is more tunneling overhead, i.e. I need more overspeed.
Technically MPLS is just better tunneling header. I can understand
sentimental arguments for IPv4 and market seems to appreciate those
arguments particularly well.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> From: David Sinn
> Sent: Thursday, June 11, 2020 4:32 PM
>
> However if you move away from large multi-chip systems,
> to a system of fixed form-factor, single chip systems, labels fall
> apart at scale with high ECMP. Needing to enumerate every possible path
> within the network or having to have a super-deep label stack removes all
of
> the perceived benefits of cheap and simple.
>
Looks like the deployments you describe are large DC Clos/Benes fabric, then
the potentially deep label imposition would be done by the VMs right?
On transit nodes the 64x ECMP or super-deep labels is no problem for
NPU/lookup process as it's always just top label lookup and resolving to a
single egress interface.

adam

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/Jun/20 17:32, David Sinn wrote:

> Respectfully, that is deployment dependent. In a traditional SP topology that focuses on large do everything boxes, where the topology is fairly point-to-point and you only have a small handful of nodes at a PoP, labels can be fast, cheap and easy. Given the lack of ECMP/WECMP, they remain fairly efficient within the hardware.
>
> However if you move away from large multi-chip systems, which hide internal links which can only be debugged and monitored if you know the the obscure, often different ways in which they are partially exposed to the operator, and to a system of fixed form-factor, single chip systems, labels fall apart at scale with high ECMP.

I'm curious about this statement - have you hit practical ECMP issues
with label switching at scale?

We have ECMP'ed label switch paths with multiple paths for a single FEC
all over the place, and those work fine both on Cisco and Junos (of all
sizes), both for IPv4 and IPv6 FEC's. Have done for years.

Unless I misunderstand your concern.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> From: Mark Tinka <mark.tinka@seacom.mu>
> Sent: Thursday, June 11, 2020 3:59 PM
>
> > No, my line of reasoning is if you have MPLS LSPs signalled over v4 I see no
> point having them signalled also over v6 in parallel.
>
> It's not about signaling IPv4 LSP's over IPv6.
> LDPv4 creates IPv4 FEC's.
> LDPv6 creates IPv6 FEC's.
>
> The idea is to create IPv6 FEC's so that IPv6 traffic can be label-switched in
> the network natively, allowing you to remove BGPv6 in a native dual-stack
> core.
>
Right I see what you are striving to achieve is migrate from BGP in a core to a BGP free core but not leveraging 6PE or 6VPE?

>
> As you can see, just as with IPv4, IPv6 packets are now being MPLS-switched
> in the core, allowing you to remove BGPv6 in the core and simplify
> operations in that area of the network.
>
> So this is native MPLSv6. It's not 6PE or 6VPE.
>
So considering you already had v4 FECs wouldn't it be simpler to do 6PE/6VPE, what do you see as drawbacks of these compared to native MPLSv6 please?

> > Apart from X months worth of functionality, performance, scalability and
> interworking testing -network wide code upgrades to address the bugs
> found during the testing process and then finally rollout across the core and
> possibly even migration from LDPv4 to LDPv6, involving dozens of people
> from Arch, Design, OPS, Project management, etc... with potential for things
> to break while making changes in live network.
>
> Which you wouldn't have to do with SRv6, because you trust the vendors?
>
Well my point was that if v4 FECs would be enough to carry v6 traffic then I wouldn't need SRv6 nor LDPv6, hence I'm curious to hear from you about the benefits of v6 FEC over v4 FEC (or in other words MPLSv6 vs 6PE/6VPE).

adam

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/Jun/20 23:45, adamv0025@netconsultings.com wrote:

> Right I see what you are striving to achieve is migrate from BGP in a core to a BGP free core but not leveraging 6PE or 6VPE?

Yes sir.


> So considering you already had v4 FECs wouldn't it be simpler to do 6PE/6VPE, what do you see as drawbacks of these compared to native MPLSv6 please?

Because 6PE, for us, adds a lot more complexity in how we design the
network.

But most importantly, it creates a dependency for the success of IPv6 on
IPv4. If my IPv4 network were to break, for whatever reason, it would
take my IPv6 network down with it.

Years back, there was a nasty bug in the ASR920 that set an upper limit
on the MPLS label space it created FEC's for. Since Juniper sometimes
uses higher label numbers than Cisco, traffic between that ASR920 and
our Juniper network was blackholed. It took weeks to troubleshoot, Cisco
sent some engineering code, I confirmed it fixed the issue, and it was
rolled out generally. During that time when the ASR920 was unavailable
on IPv4, it was still reachable on IPv6.

Other issues are also with the ASR920 and ME3600X/3800X routers, where
0/0 and ::/0 are the last routes to be programmed into FIB when you run
BGP-SD. It can be a while until those boxes can reach the rest of the
world via default. IPv6 will get there faster.

I also remember another issue, back in 2015, where a badly-written IPv4
ACL kicked one of our engineers out of the box. Thankfully, he got back
in via IPv6.

I guess what I'm saying is we don't want to fate-share. IPv4 and IPv6
can operate independently. A failure mode in one of them does not
necessarily propagate to the other, in a native, dual-stack network. You
can deploy something in your IPv6 control/data plane without impacting
IPv4, and vice versa, if you want to roll out gracefully, without
impacting the other protocol.

6PE simply has too many moving parts to setup, comparing to just adding
an IPv6 address to a router interface and updating your IGP. Slap on
LDPv6 for good measure, and you've achieved MPLSv6 forwarding without
all the 6PE faffing.


> Well my point was that if v4 FECs would be enough to carry v6 traffic then I wouldn't need SRv6 nor LDPv6, hence I'm curious to hear from you about the benefits of v6 FEC over v4 FEC (or in other words MPLSv6 vs 6PE/6VPE).

No need for 6PE deployment and day-to-day operation complexity.

A simplified and more native tunneling for IPv6-in-MPLSv6, rather than
IPv6-in-MPLSv4-on-IPv4.

No inter-dependence between IPv6 and IPv4.

Easier troubleshooting if one of the protocols is misbehaving, because
then you are working on just one protocol, and not trying to figure if
IPv4 or MPLSv4 are breaking IPv6, or vice versa.

For me, those 4 simple points help me sleep well at 3AM, meaning I can
stay up longer having more wine, in peace :-).

Mark.

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> On Jun 11, 2020, at 12:39 PM, Saku Ytti <saku@ytti.fi> wrote:
>
> On Thu, 11 Jun 2020 at 21:04, David Sinn <dsinn@dsinn.com> wrote:
>
>> You've made my point for me. If you are building the core of your network out of MX's, to turn a phrase, in a past life "I fully support my competitors to do so". Large numbers of small boxes, as they have already shown in the data-center, have major cost, control and operational advantages over a small number of large ones. They also expose the inherent problems of label-switching and where IP has it's merits.
>
> Except this implementation does not exist, but we can argue that is
> missing feature. We can argue we should be able to tell the lookup
> engine this CIDR is on-chip and it's host routes only. This is
> certainly doable, and would make IP tunnels like MPLS tunnels for
> lookup cost, just larger lookup key, which is not significant cost.

I'm not sure what implementation you are saying doesn't exist. The Broadcom XGS line is all on-die. The two largest cloud providers are using them in their transport network (to the best of my understanding). So I'm not sure if your saying that no one is using small boxes like I'm describing or what. And it doesn't have to be MPLS over IP. That is one option, but IPIP is another.

> But even if we had this (we don't, we have for MPLS) IP would be still
> inferior, it is more tunneling overhead, i.e. I need more overspeed.
> Technically MPLS is just better tunneling header. I can understand
> sentimental arguments for IPv4 and market seems to appreciate those
> arguments particularly well.

Again, feel free to look at only one small aspect and say that it is completely better in all cases. MPLS is not better in wide ECMP cases, full stop. SR doesn't help that when you actually look at the problems at massive scale as I have done. You continually are on the trade-off spectrum of irrationally deep label stacks or enumeration of all of the possible paths through the network and burn all of your next-hop re-writes. At least if you want high-radiux, single chip systems. So this is not sentimentally around a protocol, it's the practical reality when you look at the problems at scale using commodity components. So if you want to optimize for costs and power (which is operational costs), MPLS is not where it is at.

David

> --
> ++ytti

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> On Jun 11, 2020, at 2:02 PM, Mark Tinka <mark.tinka@seacom.mu> wrote:
>
>
>
> On 11/Jun/20 17:32, David Sinn wrote:
>
>> Respectfully, that is deployment dependent. In a traditional SP topology that focuses on large do everything boxes, where the topology is fairly point-to-point and you only have a small handful of nodes at a PoP, labels can be fast, cheap and easy. Given the lack of ECMP/WECMP, they remain fairly efficient within the hardware.
>>
>> However if you move away from large multi-chip systems, which hide internal links which can only be debugged and monitored if you know the the obscure, often different ways in which they are partially exposed to the operator, and to a system of fixed form-factor, single chip systems, labels fall apart at scale with high ECMP.
>
> I'm curious about this statement - have you hit practical ECMP issues
> with label switching at scale?

Yes. Path enumeration when you use mult-tier Clos topologies within a PoP causes you many, many problem.

> We have ECMP'ed label switch paths with multiple paths for a single FEC
> all over the place, and those work fine both on Cisco and Junos (of all
> sizes), both for IPv4 and IPv6 FEC's. Have done for years.

The protocols will work fine. And if you are still buying SP class chips, you're fine. But you are paying a lot for those chips in cost and power. If you look to move to the class of single-chip systems, which gives you lower costs and higher radix, you have to pay the trade-offs somewhere. MPLS at high-radix ECMP exposes this.

David

> Unless I misunderstand your concern.
>
> Mark.

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Fri, 12 Jun 2020 at 18:16, David Sinn <dsinn@dsinn.com> wrote:

> I'm not sure what implementation you are saying doesn't exist. The Broadcom XGS line is all on-die. The two largest cloud providers are using them in their transport network (to the best of my understanding). So I'm not sure if your saying that no one is using small boxes like I'm describing or what. And it doesn't have to be MPLS over IP. That is one option, but IPIP is another.

I'm saying implementation which has off-chip and supports putting some
on-chip. So that you could have full table lookup for edge packets and
and fast exact lookup for others. Of course we do have platforms which
do have large LEM tables off-chip.

> Again, feel free to look at only one small aspect and say that it is completely better in all cases. MPLS is not better in wide ECMP cases, full stop. SR doesn't help that when you actually look at the problems at massive scale as I have done. You continually are on the trade-off spectrum of irrationally deep label stacks or enumeration of all of the possible paths through the network and burn all of your next-hop re-writes. At least if you want high-radiux, single chip systems. So this is not sentimentally around a protocol, it's the practical reality when you look at the problems at scale using commodity components. So if you want to optimize for costs and power (which is operational costs), MPLS is not where it is at.

I'm not sure why this deep label stack keeps popping, if we need
multiple levels of tunneling, we need it in IP too, and it's almost
more expensive in IP. Cases I can think of in SR, you'll only loop top
label or two, even if you might have 10 labels.
For every apples to apples cases MPLS tunnels are superior to IP
tunnels. If you want cheap very small FIB backbone, then all traffic
will need to be IP tunneled to egress, and you get all the MPLS
problems, and you get more overhead and larger keys (larger keys is
not a big deal).

Now if discussion is do we need tunnelling at all, the discussion is
very different.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/Jun/20 19:19, Saku Ytti wrote:

> The demand is, we need tunneling, then the question is what are the
> metrics of a good tunneling solution. By answering this honestly, MPLS
> is better. We could do better surely, but IP is not that.

One unexpected benefit, I will say, with going native LDPv6 is that
MTR's for IPv6 destinations no longer report packet loss on the
intermediary core routers (CRS-X).

I know this was due to the control plane, and nothing to do with the
actual data plane, but it was always a tool explaining to customers why
MTR's for IPv4 destinations have 0% packet loss in our core, while IPv6
ones have 30% - 50% (in spite of the final end-host reporting 0% packet
loss).

Since going LDPv6, IPv6 traffic is now label-switched in the core, in
lieu of hop-by-hop IPv6 forwarding. The unforeseen-but-welcome side
effect is that customer packet loss MTR's for IPv6 destinations that
traverse the CRS-X core are as 0% as they are for IPv4 (even though we
haven't yet removed BGPv6 from the core due to IOS XE platforms that
don't yet run LDPv6).

One less trouble ticket to have to explain for our NOC; I'll gladly take
that...

As my Swedish friend would say, "That gives me an avenue of pleasure and
joy" :-).

Mark.

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> On Jun 12, 2020, at 8:26 AM, Saku Ytti <saku@ytti.fi> wrote:
>
> On Fri, 12 Jun 2020 at 18:16, David Sinn <dsinn@dsinn.com> wrote:
>
>> I'm not sure what implementation you are saying doesn't exist. The Broadcom XGS line is all on-die. The two largest cloud providers are using them in their transport network (to the best of my understanding). So I'm not sure if your saying that no one is using small boxes like I'm describing or what. And it doesn't have to be MPLS over IP. That is one option, but IPIP is another.
>
> I'm saying implementation which has off-chip and supports putting some
> on-chip. So that you could have full table lookup for edge packets and
> and fast exact lookup for others. Of course we do have platforms which
> do have large LEM tables off-chip.

But why do you need full table lookup in the middle of the network? Why place that class of gear where it's not needed?

>> Again, feel free to look at only one small aspect and say that it is completely better in all cases. MPLS is not better in wide ECMP cases, full stop. SR doesn't help that when you actually look at the problems at massive scale as I have done. You continually are on the trade-off spectrum of irrationally deep label stacks or enumeration of all of the possible paths through the network and burn all of your next-hop re-writes. At least if you want high-radiux, single chip systems. So this is not sentimentally around a protocol, it's the practical reality when you look at the problems at scale using commodity components. So if you want to optimize for costs and power (which is operational costs), MPLS is not where it is at.
>
> I'm not sure why this deep label stack keeps popping, if we need
> multiple levels of tunneling, we need it in IP too, and it's almost
> more expensive in IP. Cases I can think of in SR, you'll only loop top
> label or two, even if you might have 10 labels.
> For every apples to apples cases MPLS tunnels are superior to IP
> tunnels. If you want cheap very small FIB backbone, then all traffic
> will need to be IP tunneled to egress, and you get all the MPLS
> problems, and you get more overhead and larger keys (larger keys is
> not a big deal).

The label stack question is about the comparisons between the two extremes of SR that you can be in. You either label your packet just for it's ultimate destination or you apply the stack of the points you want to pass through.

In the former case you are, at the forwarding plane, equal to what you see with traditional MPLS today, with every node along the path needing to know how to reach the end-point. Yes, you have lowered label space from traditional MPLS, but that can be done with site-cast labels already. And, while the nodes don't have to actually swap labels, when you look at commodity implementations (across the last three generations since you want to do this with what is deployed, not wholesale replace the network) a null swap still ends up eating a unique egress next-hop entry. So from a hardware perspective, you haven't improved anything. Your ECMP group count is high.

In the extreme latter case, you have to, on ingress, place the full stack of every "site" you want to pass through. That has the benefits of "sites" only need labels for their directly connected sites, so you have optimized the implications on commodity hardware. However, you now have a label stack that can be quite tall. At least if you want to walk the long-way around the world, say due to failure. On top, that depth of label stack means devices in the middle can't look at the original payload to make ECMP decisions. So you can turn to entropy labels, but that sort of makes matters worse.

The practical reality is somewhere in the middle. At least you probably want some form of path engineering, so the first model really doesn't work or is at least equal to just doing traditional MPLS-TE. The closer you get to the latter, the higher your stack goes. So then you have to look at the hardware you want at the edge of your network and how many labels it can impose. If you want a all commodity network, that doesn't work.

So, yes, MPLS works fine if you want to buy big iron boxes. But that come at a cost. So the point about MPLS is always better is not accurate. Engineering is about trade-offs and there are trade-offs to be made when you optimize in a different direction and that points away from MPLS and back to IPIP

David

> Now if discussion is do we need tunnelling at all, the discussion is
> very different.
>
> --
> ++ytti

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Fri, 12 Jun 2020 at 18:42, David Sinn <dsinn@dsinn.com> wrote:

> But why do you need full table lookup in the middle of the network? Why place that class of gear where it's not needed?

Some people do collapsed core. But this is getting bit theoretical,
because we definitely could do this on IP, we could do some lookups on
on-chip and some off-chip to do both, should market want it.

> The label stack question is about the comparisons between the two extremes of SR that you can be in. You either label your packet just for it's ultimate destination or you apply the stack of the points you want to pass through.

Quite, but transit won't inspect the stack, it doesn't have to care
about it, so it can be very deep.

> In the former case you are, at the forwarding plane, equal to what you see with traditional MPLS today, with every node along the path needing to know how to reach the end-point. Yes, you have lowered label space from traditional MPLS, but that can be done with site-cast labels already. And, while the nodes don't have to actually swap labels, when you look at commodity implementations (across the last three generations since you want to do this with what is deployed, not wholesale replace the network) a null swap still ends up eating a unique egress next-hop entry. So from a hardware perspective, you haven't improved anything. Your ECMP group count is high.

I don't disagree. What I'm trying to say, however you tunnel, you have
the same issues. If you need to tunnel, then MPLS is better tunnel
than IP. Ultimately both can be made LEM on-chip should market want
it, so difference left is what is the overhead of the tunnel, and MPLS
wins here handsdown. This is objectively true, now what practical
market reality is, that may be different, because market doesn't
optimise for best solution.

> So, yes, MPLS works fine if you want to buy big iron boxes. But that come at a cost. So the point about MPLS is always better is not accurate. Engineering is about trade-offs and there are trade-offs to be made when you optimize in a different direction and that points away from MPLS and back to IPIP

Always if you need tunnel. Because the fundamental question is how
much overhead we have, and what is our key width. In both IP tunnel
and MPLS tunnel cases we will assume LEM lookup, to keep the lookup
cheap.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

>> The label stack question is about the comparisons between the two extremes of SR that you can be in. You either label your packet just for it's ultimate destination or you apply the stack of the points you want to pass through.
>
> Quite, but transit won't inspect the stack, it doesn't have to care
> about it, so it can be very deep.

Unless you want ECMP then it VERY much matters. But I guess since we are only talking about theoretical instead of building an actual practical network, it doesn't matter.

David
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Salve,

On Thu, Jun 11, 2020 at 8:08 PM David Sinn <dsinn@dsinn.com> wrote:

Rewrites on MPLS is horrible from a memory perspective as maintaining the
> state and label transition to explore all possible discrete paths across
> the overall end-to-end path you are trying to take is hugely in-efficient.
> Applying circuit switching to a packet network was bad from the start. SR
> doesn't resolve that, as you are stuck with a global label problem and the
> associated lack of being able to engineer your paths, or a label stack
> problem on ingress that means you need a massive ASIC's and memories there.
>

I don't think rewrites are horrible, but just very flexible and this *can*
come up with a certain price. Irt to your memory argument that path
engineering takes in vanilla TE a lot of forwarding slots we should remind
us that this is not a design principle of MPLS. Discrete paths could also
be signalled in MPLS with shared link-labels so that you will end up with
the same big instructional headend packet as in SR. There are even
implementations offering this.

IP at least gives you rewrite sharing, so in a lite-core you have way
> better trade-off on resources, especially in a heavily ECMP'ed network.
> Such as one build of massive number of open small boxes vs. a small number
> of huge opaque ones. Pick your poison but saying one is inheriantly better
> then another in all cases is just plane false.
>

If I understand this argument correctly then it shouldn't be one because of
"rewrite sharing" being irrelevant for the addressability of single nodes
in a BGP network. Why a header lookup depth of 4B per label in engineered
and non-engineered paths should be a bad requisite for h/w designers of
modern networks is beyond me. In most MPLS networks (unengineered L3VPN)
you need to read less of headers than in a eg. VXLAN fabric to make ECMP
work (24B vs. 20B).
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Fri, 12 Jun 2020 at 18:52, David Sinn <dsinn@dsinn.com> wrote:

> Unless you want ECMP then it VERY much matters. But I guess since we are only talking about theoretical instead of building an actual practical network, it doesn't matter.

Well blatantly we are, because in the real world most of the value of
MPLS tunnels is not available as IP tunnels. Again technically
entirely possible to replace MPLS tunnels with IP tunnels, just
question how much overhead you have in transporting the tunnel key and
how wide they are.

Should we design a rational cost-efficient solution, we should choose
the lowest overhead and narrowest working keys.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Saku Ytti ????? 2020-06-12 12:10:
> On Fri, 12 Jun 2020 at 18:52, David Sinn <dsinn@dsinn.com> wrote:
>
>> Unless you want ECMP then it VERY much matters. But I guess since we
>> are only talking about theoretical instead of building an actual
>> practical network, it doesn't matter.
>
> Well blatantly we are, because in the real world most of the value of
> MPLS tunnels is not available as IP tunnels. Again technically
> entirely possible to replace MPLS tunnels with IP tunnels, just
> question how much overhead you have in transporting the tunnel key and
> how wide they are.
>
> Should we design a rational cost-efficient solution, we should choose
> the lowest overhead and narrowest working keys.

Sorry for jumping in in the mddle of discussion, as a side note, in case
of IPIP tunneling, shouldn't another protocol type be utilized in MAC
header? As I understand, in VXLAN VTEP ip is dedicated for this purpose,
so receiving a packet with VTEP DST IP already means "decapsulate and
lookup the next header". But in traditional routers loopback IPs are
used for multiple purposes and usually receiving a packet with lo0 IP
means punt it to control plane. Isn't additional differentiator is
needed here to tell a router which type of action it has to do? Or, as
alternative, if dedicated stack of IPs is used for tunneling, then
another lookup table is needed for it, isn't it? And now looks like we
are coming to the header structure and forwarding process similar that
we already have in MPLS, only with different label format. Please
correct me if I went off track somewhere in this logical chain.

To David's point about ECMP I'd like to mention that in WAN networks
number of diverse paths is always limited, so having multiple links
taking same path doesn't make much sense. With current economics 4x10G
and 1x100G are usually close from price POV. Obviously, there are
different situations when multiple links are the only option, but how
many, usually 4-8. Sure if you need multiple 400G then there is
currently no option to go to higher speeds, but that's more DC use case
than WAN network. So ECMP in WAN network isn't that big scale problem
imho, also there are existing and proposed solutions, like SR, for it.

Kind regards,
Andrey
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Fri, 12 Jun 2020 at 20:12, Andrey Kostin <ankost@podolsk.ru> wrote:

> Sorry for jumping in in the mddle of discussion, as a side note, in case
> of IPIP tunneling, shouldn't another protocol type be utilized in MAC
> header? As I understand, in VXLAN VTEP ip is dedicated for this purpose,
> so receiving a packet with VTEP DST IP already means "decapsulate and
> lookup the next header". But in traditional routers loopback IPs are
> used for multiple purposes and usually receiving a packet with lo0 IP
> means punt it to control plane. Isn't additional differentiator is
> needed here to tell a router which type of action it has to do? Or, as
> alternative, if dedicated stack of IPs is used for tunneling, then
> another lookup table is needed for it, isn't it? And now looks like we
> are coming to the header structure and forwarding process similar that
> we already have in MPLS, only with different label format. Please
> correct me if I went off track somewhere in this logical chain.

I don't think new etherType is mandatory by any means. Biggest gain is
security. SRv6 will necessarily have a lot of issues where
unauthorised packet gets treated as SRv6, which is much harder in MPLS
network. Many real-life devices work very differently with EH chains
(with massive performance drop, like can be 90%!). JNPR Trio will
parse up-to N EH, then drop if it cannot finish parsing. NOK FP wil
parse up-to N EH, then forward if it cannot finish parsing (i.e. now
it bypasses TCP/UDP denies, as it didn't know it is TCP/IP, or it
could have SRv6 EH, which it couldn't drop, as it didn't know it had
it).

But in terms of the big MPLS advantage, of having guaranteed exact
match lookups on small space, compared to LPM lookups on large space.
We could guarantee this in IPIP tunnels too, without having any
difference in the headers, other than obligation/guarantee that all
LSR packets are IPIP encapsulated with a small amount of outer packet
DADDRs.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

>
> I'm not sure why this deep label stack keeps popping, if we need
> multiple levels of tunneling, we need it in IP too, and it's almost
> more expensive in IP.
>

Well imagine you need only one level of tunneling but rich ECMP.

Then with IP encap (even MPLS app demux carried in UDP) you just make sure
src UDP port is random and voila works very nicely.

In contrast to achieve the same with MPLS you need Entropy label (which is
two - first the marker/indicator and then the actual value) + hardware
capable of reading it.

R.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Fri, 12 Jun 2020 at 21:41, Robert Raszuk <robert@raszuk.net> wrote:

> Well imagine you need only one level of tunneling but rich ECMP.
>
> Then with IP encap (even MPLS app demux carried in UDP) you just make sure src UDP port is random and voila works very nicely.
>
> In contrast to achieve the same with MPLS you need Entropy label (which is two - first the marker/indicator and then the actual value) + hardware capable of reading it.

Extending MPLS is byte expensive, we could certainly improve that on
new tunneling headers. But it's still cheaper than looking up UDP, and
potentially impossible (IPv6, EH). Of course if you are working with
existing pipeline which has support for IIP UDP parsing but not
entropy parsing then it's easy to say in that special case UDP is a
better solution. But all things equal, MPLS still wins, but we could
definitely improve upon it.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> Rewrites on MPLS is horrible from a memory perspective as maintaining the state and label transition to explore all possible discrete paths across the overall end-to-end path you are trying to take is hugely in-efficient. Applying circuit switching to a packet network was bad from the start. SR doesn't resolve that, as you are stuck with a global label problem and the associated lack of being able to engineer your paths, or a label stack problem on ingress that means you need a massive ASIC's and memories there.
>
> I don't think rewrites are horrible, but just very flexible and this *can* come up with a certain price. Irt to your memory argument that path engineering takes in vanilla TE a lot of forwarding slots we should remind us that this is not a design principle of MPLS. Discrete paths could also be signalled in MPLS with shared link-labels so that you will end up with the same big instructional headend packet as in SR. There are even implementations offering this.

Except that is actually the problem if you look at it in hardware. And to be very specific, I'm talking about commodity hardware, not flexible pipelines like you find in the MX and a number of the ASR's. I'm also talking about the more recent approach of using Clos in PoP's instead of "big iron" or chassis based systems. On those boxes, it's actually better to not do shared labels, as this pushes the ECMP decision to the ingress node. That does mean you have to enumerate every possible path (or some approximate) through the network, however the action on the commodity gear is greatly reduced. It's a pure label swap, so you don't run into any egress next-hop problems. You definitely do on the ingress nodes. Very, very badly actually.

So you can move to a shared label mode. Now the commodity boxes have to perform ECMP. That means they also have to have a unique ECMP group for every site/any-cast label passing through them, as every label is being swapped differently. You get no reuse for two labels that are on identical paths because the "swaps" are not identical. So you hit up against ECMP next-hop group starvation, forcing you to lower radix and limiting total any-/site-cast count.

> IP at least gives you rewrite sharing, so in a lite-core you have way better trade-off on resources, especially in a heavily ECMP'ed network. Such as one build of massive number of open small boxes vs. a small number of huge opaque ones. Pick your poison but saying one is inheriantly better then another in all cases is just plane false.
>
> If I understand this argument correctly then it shouldn't be one because of "rewrite sharing" being irrelevant for the addressability of single nodes in a BGP network. Why a header lookup depth of 4B per label in engineered and non-engineered paths should be a bad requisite for h/w designers of modern networks is beyond me. In most MPLS networks (unengineered L3VPN) you need to read less of headers than in a eg. VXLAN fabric to make ECMP work (24B vs. 20B).

What I'm getting at is that IP allows re-write sharing in that what needs to change on two IP frames taking the same paths but ultimately reaching different destinations are re-written (e.g. DMAC, egress-port) identically. And, at least with IPIP, you are able to look at the inner-frame for ECMP calculations. Depending on your MPLS design, that may not be the case. If you have too deep of a label stack (3-5 depending on ASIC), you can't look at the payload and you end up with polarization.

David

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

>
>> Unless you want ECMP then it VERY much matters. But I guess since we are only talking about theoretical instead of building an actual practical network, it doesn't matter.
>
> Well blatantly we are, because in the real world most of the value of
> MPLS tunnels is not available as IP tunnels. Again technically
> entirely possible to replace MPLS tunnels with IP tunnels, just
> question how much overhead you have in transporting the tunnel key and
> how wide they are.

You may be, I am not. I'm talking about practical networks and the use-case that multiple large networks are going down around commodity ASIC's. And it is a practial question about the total solution not point-specific ones. Overhead is only one part. Lookup delay is not at all for the class I'm referring to.

> Should we design a rational cost-efficient solution, we should choose
> the lowest overhead and narrowest working keys.

In the abstract, sure. But if you want a practical, deployable, production network, it's multi-dimensioned.

David
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> On Jun 12, 2020, at 11:41 AM, Robert Raszuk <robert@raszuk.net> wrote:
>
> I'm not sure why this deep label stack keeps popping, if we need
> multiple levels of tunneling, we need it in IP too, and it's almost
> more expensive in IP.
>
> Well imagine you need only one level of tunneling but rich ECMP.
>
> Then with IP encap (even MPLS app demux carried in UDP) you just make sure src UDP port is random and voila works very nicely.

In reality you don't even need to do UDP and consume those extra bytes. All of the commodity pipelines as well as the big-iron ones will look at the inner-header of a IPIP frame for entropy calculations if you configure them to do so. You also have the problem that most of the very high-performance commodity ASIC's are moving away from VXLAN/UDP encap since they are in the middle of the classic core/edge feature split and deem VXLAN as edge.

David
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 12/Jun/20 17:19, David Sinn wrote:

> Yes. Path enumeration when you use mult-tier Clos topologies within a PoP causes you many, many problem.

Okay, got you. I thought you were running into these problems on the
"usual suspect" platforms.

Yes, commodity hardware certainly has a number of trade-offs for the
cost benefit. We've been evaluating this path since 2013, and for our
use-case, it will actually make less sense because we are more large
capacity, small footprint, i.e., typical network service provider.

If we were a cloud provider operating at scale in several data centres,
our current model of running Cisco's, Juniper's, Arista's, e.t.c., may
not necessarily be the right choice in 2020, particularly in the edge.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Fri, 12 Jun 2020 at 23:25, David Sinn <dsinn@dsinn.com> wrote:

> > Should we design a rational cost-efficient solution, we should choose
> > the lowest overhead and narrowest working keys.
>
> In the abstract, sure. But if you want a practical, deployable, production network, it's multi-dimensioned.

We have probably largely converged to the same place. Your vantage
point sees practical offerings where IPIP may make more sense to you
than MPLS, my vantage point definitely only implements the rich
features I need in MPLS tunnels (RSVP-TE, L2 pseudowires, FRR, L3 MPLS
VPN, all of which technically could be done of course with IPIPIP
tunnels) And the theory we agree that less is more.

ECMP appears to be your main pain point, the rich features are not
relevant, and you mentioned commodity hardware being able to hash on
IPIP. I feel this may be a very special case where HW can do IPIP hash
but not MPLSIP hash. Out of curiosity, what is this hardware? Jericho
can do MPLSIP, I know JNPR's pipeline offering, Paradise, can. Or
perhaps it's not even that the underlaying hardware you have, cannot
do, it's that the NOS you are being offered is so focused on your
use-case, it doesn't do anything else reasonably, then of course that
use-case is best by default.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 13/Jun/20 08:00, Saku Ytti wrote:

>
> ECMP appears to be your main pain point, the rich features are not
> relevant, and you mentioned commodity hardware being able to hash on
> IPIP. I feel this may be a very special case where HW can do IPIP hash
> but not MPLSIP hash. Out of curiosity, what is this hardware? Jericho
> can do MPLSIP, I know JNPR's pipeline offering, Paradise, can. Or
> perhaps it's not even that the underlaying hardware you have, cannot
> do, it's that the NOS you are being offered is so focused on your
> use-case, it doesn't do anything else reasonably, then of course that
> use-case is best by default.

One of the biggest challenges we found of leveraging commodity hardware
was locating a suitable OS that is not only fit-for-purpose in our
service provider environment, but that could also leverage the hardware
at its disposal to its fullest potential.

It's hard enough for one vendor to get both their own hardware and
software right most of the time. We posited it would be doubly hard for
an operator to marry hardware and software vendors that do not
necessarily co-ordinate with one another, if your goal is to run a
profit-oriented operational network.

Sure, the idea is great on paper, but there aren't that many shops that
can throw warm bodies at this problem like some of the more established
content and cloud folk.

If it was easy, I certainly wouldn't have started this thread in the
first place :-).

Mark.

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Fri, Jun 12, 2020 at 10:22 PM David Sinn <dsinn@dsinn.com> wrote:

> Except that is actually the problem if you look at it in hardware. And to
> be very specific, I'm talking about commodity hardware, not flexible
> pipelines like you find in the MX and a number of the ASR's. I'm also
> talking about the more recent approach of using Clos in PoP's instead of
> "big iron" or chassis based systems.
>

TE gives you the most powerful traffic engineering tool kit available.
Naturally it has a bit more weight than just a single screwdriver. It can
you build nearly any kind of multipath transport while that Clos thing is
just one architecture hunting for the cheapest implementation of
IP/LDP-style ECMP.

On those boxes, it's actually better to not do shared labels, as this
> pushes the ECMP decision to the ingress node. That does mean you have to
> enumerate every possible path (or some approximate) through the network,
> however the action on the commodity gear is greatly reduced. It's a pure
> label swap, so you don't run into any egress next-hop problems. You
> definitely do on the ingress nodes. Very, very badly actually.
>

Actually shared links are not a swap but just a pop similar to SR. But
indeed this would shift your ECMP issue just to the headend. So for your
ECMP scaling there would still be an option left to use an implementation
which offers you a merge-point with a single label to all upstreams for a
certain equal-cost multipath downstream. This does exist, so would
certainly fix your ECMP scaling problem. But advanced control-plane code is
certainly not cheap so in the end, like it was already said before, if a
simple and cheap platform can solve all your needs then it might be the
better one. Let‘s see what problems we need to solve in five years again.

What I'm getting at is that IP allows re-write sharing in that what needs
> to change on two IP frames taking the same paths but ultimately reaching
> different destinations are re-written (e.g. DMAC, egress-port) identically.
> And, at least with IPIP, you are able to look at the inner-frame for ECMP
> calculations. Depending on your MPLS design, that may not be the case. If
> you have too deep of a label stack (3-5 depending on ASIC), you can't look
> at the payload and you end up with polarization.
>

Not really as you are still forced to rewrite on imposition for the
simplest form of tunneling, and for TE as often as you need to go against
your SPT as well, it‘s just happening on IP (and IP rewrites are more
expensive than MPLS rewrites / forwarding operations).
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> From: David Sinn
> Sent: Friday, June 12, 2020 4:19 PM
>
> > On Jun 11, 2020, at 2:02 PM, Mark Tinka <mark.tinka@seacom.mu> wrote:
> >
> > On 11/Jun/20 17:32, David Sinn wrote:
> >
> >> Respectfully, that is deployment dependent. In a traditional SP
topology
> that focuses on large do everything boxes, where the topology is fairly
point-
> to-point and you only have a small handful of nodes at a PoP, labels can
be
> fast, cheap and easy. Given the lack of ECMP/WECMP, they remain fairly
> efficient within the hardware.
> >>
> >> However if you move away from large multi-chip systems, which hide
> internal links which can only be debugged and monitored if you know the
the
> obscure, often different ways in which they are partially exposed to the
> operator, and to a system of fixed form-factor, single chip systems,
labels fall
> apart at scale with high ECMP.
> >
> > I'm curious about this statement - have you hit practical ECMP issues
> > with label switching at scale?
>
> Yes. Path enumeration when you use mult-tier Clos topologies within a PoP
> causes you many, many problem.
>
Hi David,

Can you be more specific please? Maybe some examples with numbers.

I can see how you might run out of L2 rewrite/adjacency table space on a
particular node if you enumerate every possible path downstream of it
(especially on leaf nodes), cause that number is has a dependency on the
size of the fabric in terms of the total number of links in the fabric
(which balloons quickly).
Let's focus on the alternate case then, (the deep label-stack one)
At each node in the multi-tier Clos (which I assume is the Russ White's
butterfly model? But any Clos or Benes fabric needs the same) you need to
program a label to uniquely identify each egress interface ,so now there's
this nice one-to-one relationship between label and egress interface. Now
the depth of the label-stack depends on the number of hops the packet needs
to traverse across the fabric. But how deep could the fabric realistically
be? Even in the "butterfly" model with separate pods instead of leaf nodes I
counted 9 hops (that's not ultra-deep is it)? It's the VM doing label
imposition as programmed by the fabric controller (all in SW so can go as
deep as you want) and all fabric nodes are just popping top label so no big
deal.

adam



_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> David Sinn
> Sent: Friday, June 12, 2020 4:42 PM
>
> > On Jun 12, 2020, at 8:26 AM, Saku Ytti <saku@ytti.fi> wrote:
> >
> > On Fri, 12 Jun 2020 at 18:16, David Sinn <dsinn@dsinn.com> wrote:
> >
> The label stack question is about the comparisons between the two
> extremes of SR that you can be in. You either label your packet just for
it's
> ultimate destination or you apply the stack of the points you want to pass
> through.
>
> In the former case you are, at the forwarding plane, equal to what you see
> with traditional MPLS today, with every node along the path needing to
> know how to reach the end-point. Yes, you have lowered label space from
> traditional MPLS, but that can be done with site-cast labels already. And,
> while the nodes don't have to actually swap labels, when you look at
> commodity implementations (across the last three generations since you
> want to do this with what is deployed, not wholesale replace the network)
a
> null swap still ends up eating a unique egress next-hop entry. So from a
> hardware perspective, you haven't improved anything. Your ECMP group
> count is high.
>
Yes this is where each node needs to have a label uniquely identifying every
LSP passing through it.
Saku,
With IP header you don't need this,
Consider this:
PE1 to PE2 via 3 P-core nodes
With ECMP in IP, then PE1 just needs single FEC the DST-IP of PE2, which
will be load-shared across all 3 paths.
Using MPLS If you need to uniquely identify each path you need 3 FECs (3
LSPs one via each P core node), now imagine you have 100K possible paths
across the fabric
-that's a lot of FECs on PE1 or any node in the fabric where each has to
have a unique label for every possible unique path via the core that the
particular node is part of.

> In the extreme latter case, you have to, on ingress, place the full stack
of
> every "site" you want to pass through. That has the benefits of "sites"
only
> need labels for their directly connected sites, so you have optimized the
> implications on commodity hardware. However, you now have a label stack
> that can be quite tall. At least if you want to walk the long-way around
the
> world, say due to failure. On top, that depth of label stack means devices
in
> the middle can't look at the original payload to make ECMP decisions. So
you
> can turn to entropy labels, but that sort of makes matters worse.
>
David,
You can use hierarchy of tunnels to help with the deep label-stack
imposition problem.

adam

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Mon, 15 Jun 2020 at 12:24, <adamv0025@netconsultings.com> wrote:


> Yes this is where each node needs to have a label uniquely identifying every
> LSP passing through it.
> Saku,
> With IP header you don't need this,
> Consider this:
> PE1 to PE2 via 3 P-core nodes
> With ECMP in IP, then PE1 just needs single FEC the DST-IP of PE2, which
> will be load-shared across all 3 paths.
> Using MPLS If you need to uniquely identify each path you need 3 FECs (3
> LSPs one via each P core node), now imagine you have 100K possible paths
> across the fabric
> -that's a lot of FECs on PE1 or any node in the fabric where each has to
> have a unique label for every possible unique path via the core that the
> particular node is part of.

Are we talking about specific implementations of fundamentals? It
sounds like we are talking about a specific case where IP next-hop is
unilist of N next-hops, and MPLS next-hop is a single item without
indirection? This is not a fundamental difference, this is
implementation detail.
There is no particular reason MPLS next-hop couldn't be unilist of N
destinations.

I think people are too focused on thinking IP and MPLS have some
inherent magical property differences, they don't. We only care about
lookup cost (again IP can be made cheap with IPinIPinIP tunnels and
telling LSR devices all lookups are LEM host lookups) and we care
about key width. Rest is implementation detail.

Yes on typical case there are some biases in IP and MPLS, but these
can be rendered away leaving the fundamental differences. Briding the
gap from MPLS to IP is far smaller than bridging the gap from IP to
MPLS, there is so much added value which depends on MPLS tunnels.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> From: Saku Ytti <saku@ytti.fi>
> Sent: Monday, June 15, 2020 10:31 AM
>
> On Mon, 15 Jun 2020 at 12:24, <adamv0025@netconsultings.com> wrote:
>
>
> > Yes this is where each node needs to have a label uniquely identifying
> > every LSP passing through it.
> > Saku,
> > With IP header you don't need this,
> > Consider this:
> > PE1 to PE2 via 3 P-core nodes
> > With ECMP in IP, then PE1 just needs single FEC the DST-IP of PE2,
> > which will be load-shared across all 3 paths.
> > Using MPLS If you need to uniquely identify each path you need 3 FECs
> > (3 LSPs one via each P core node), now imagine you have 100K possible
> > paths across the fabric -that's a lot of FECs on PE1 or any node in
> > the fabric where each has to have a unique label for every possible
> > unique path via the core that the particular node is part of.
>
> Are we talking about specific implementations of fundamentals? It sounds
> like we are talking about a specific case where IP next-hop is unilist of N next-
> hops, and MPLS next-hop is a single item without indirection? This is not a
> fundamental difference, this is implementation detail.
> There is no particular reason MPLS next-hop couldn't be unilist of N
> destinations.
>
Yes it can indeed, and that's moving towards the centre between the extreme cases that David laid out.
It's about how granular one wants to be in identifying an end-to-end path between a pair of edge nodes.
I agree with you that MPLS is still better than IP,
and I tried to illustrate that even enumerating every possible paths using deep label stack is not a problem (and even that can be alleviated using hierarchy of LSPs).

adam


_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Mon, 15 Jun 2020 at 12:46, <adamv0025@netconsultings.com> wrote:

> Yes it can indeed, and that's moving towards the centre between the extreme cases that David laid out.
> It's about how granular one wants to be in identifying an end-to-end path between a pair of edge nodes.
> I agree with you that MPLS is still better than IP,
> and I tried to illustrate that even enumerating every possible paths using deep label stack is not a problem (and even that can be alleviated using hierarchy of LSPs).

The entirety of my point is, if we were rational, we'd move towards
increasingly efficient solutions. And technically everything we do in
MPLS tunnels, we can do in IP tunnels and converse. Should we imagine
a future where all features and functions are supported in both, it's
clear we should want to do MPLS tunnels. Just the [IGP][BGP-LU] 8B
overhead, compared to IP 40B overhead should drive the point home, and
ultimately, that's the only difference, rest is implementation.

And I'm saddened we've been marketed snake-oil like SRv6 with fake
promises of inherent advantages or simplicity 'just IP'.

We can do better than MPLS, absolutely. But IP is worse.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> From: Saku Ytti <saku@ytti.fi>
> Sent: Monday, June 15, 2020 11:02 AM
>
> On Mon, 15 Jun 2020 at 12:46, <adamv0025@netconsultings.com> wrote:
>
> > Yes it can indeed, and that's moving towards the centre between the
> extreme cases that David laid out.
> > It's about how granular one wants to be in identifying an end-to-end path
> between a pair of edge nodes.
> > I agree with you that MPLS is still better than IP, and I tried to
> > illustrate that even enumerating every possible paths using deep label
> stack is not a problem (and even that can be alleviated using hierarchy of
> LSPs).
>
> The entirety of my point is, if we were rational, we'd move towards
> increasingly efficient solutions. And technically everything we do in MPLS
> tunnels, we can do in IP tunnels and converse. Should we imagine a future
> where all features and functions are supported in both, it's clear we should
> want to do MPLS tunnels. Just the [IGP][BGP-LU] 8B overhead, compared to
> IP 40B overhead should drive the point home, and ultimately, that's the only
> difference, rest is implementation.
>
> And I'm saddened we've been marketed snake-oil like SRv6 with fake
> promises of inherent advantages or simplicity 'just IP'.
>
> We can do better than MPLS, absolutely. But IP is worse.
>
Yes I absolutely agree,

Not to mention this whole thread is focused solely on next-hop identification -which is just the lowest of the layers of abstraction in the vertical stack.
We haven’t talked about other "entities" that need identification like: VPNs, applications, policies (yes I'm looking at you VXLAN!) etc... - all of which are way better identified by a simple label rather than IPinIPinIP....

adam

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> ECMP appears to be your main pain point, the rich features are not
> relevant, and you mentioned commodity hardware being able to hash on
> IPIP. I feel this may be a very special case where HW can do IPIP hash
> but not MPLSIP hash. Out of curiosity, what is this hardware? Jericho
> can do MPLSIP, I know JNPR's pipeline offering, Paradise, can. Or
> perhaps it's not even that the underlaying hardware you have, cannot
> do, it's that the NOS you are being offered is so focused on your
> use-case, it doesn't do anything else reasonably, then of course that
> use-case is best by default.

I'm referring to the DC not SP class chips. So XGS if you are a Broadcom fan or any of the half a dozen other options. Because if you really are going to go commodity, why go commodity when there is only one option (DNX) vs. multiple with more coming in at a decent pace in the DC class. Your really trading out lock-in if stick to SP class chips in the commodity space, at least until there are more alternatives.

David
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> What I'm getting at is that IP allows re-write sharing in that what needs to change on two IP frames taking the same paths but ultimately reaching different destinations are re-written (e.g. DMAC, egress-port) identically. And, at least with IPIP, you are able to look at the inner-frame for ECMP calculations. Depending on your MPLS design, that may not be the case. If you have too deep of a label stack (3-5 depending on ASIC), you can't look at the payload and you end up with polarization.
>
> Not really as you are still forced to rewrite on imposition for the simplest form of tunneling, and for TE as often as you need to go against your SPT as well, it‘s just happening on IP (and IP rewrites are more expensive than MPLS rewrites / forwarding operations).

Sure, but not following SPT in IP isn't rocket science. You can do it using traditional protocols if you really want to, or you can write a controller to do it for you. And it doesn't take 100's of people to do so. It doesn't even take 10. So, yes, you need to justify the funding of those people, so milage will vary based on size and scope of your network.

David
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 15/Jun/20 12:13, adamv0025@netconsultings.com wrote:

> Not to mention this whole thread is focused solely on next-hop identification -which is just the lowest of the layers of abstraction in the vertical stack.
> We haven’t talked about other "entities" that need identification like: VPNs, applications, policies (yes I'm looking at you VXLAN!) etc... - all of which are way better identified by a simple label rather than IPinIPinIP....

The problem is if you want to have MPLS services signaled over IPv6, you
first need an IPv6 control plane that will signal the labels that will
support those services, i.e., LDPv6 and RSVPv6.

Right now, all vendors that support LDPv6 do so only for pure MPLS-IPv6
forwarding. If you want l2vpnv6, l3vpnv6, EVPNv6, 4PE, 4VPE, TE-FRRv6,
e.t.c., you can't get that today. You'd have to signal those services
over LDPv4 or RSVPv4.

The guys did a great gap analysis back in 2015, when LDPv6 began to
appear in IOS XR and Junos, that showed the challenges toward an
IPv6-only MPLS network. I believe much of this is still relevant in 2020:

    https://tools.ietf.org/html/rfc7439

I have to commend Vishwas, together with both Rajiv's, for kicking this
off way back in 2008. The road has been long and hard.

Mark.

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 15/Jun/20 12:13, adamv0025@netconsultings.com wrote:

> Not to mention this whole thread is focused solely on next-hop
> identification -which is just the lowest of the layers of abstraction
> in the vertical stack. We haven’t talked about other "entities" that
> need identification like: VPNs, applications, policies (yes I'm
> looking at you VXLAN!) etc... - all of which are way better identified
> by a simple label rather than IPinIPinIP....

The problem is if you want to have MPLS services signaled over IPv6, you
first need an IPv6 control plane that will signal the labels that will
support those services, i.e., LDPv6 and RSVPv6.

Right now, all vendors that support LDPv6 do so only for pure MPLS-IPv6
forwarding. If you want l2vpnv6, l3vpnv6, EVPNv6, 4PE, 4VPE, TE-FRRv6,
e.t.c., you can't get that today. You'd have to signal those services
over LDPv4 or RSVPv4.

The guys did a great gap analysis back in 2015, when LDPv6 began to
appear in IOS XR and Junos, that showed the challenges toward an
IPv6-only MPLS network. I believe much of this is still relevant in 2020:

    https://tools.ietf.org/html/rfc7439

I have to commend Vishwas, together with both Rajiv's, for kicking this
off way back in 2008. The road has been long and hard.

Mark.


_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> From: Mark Tinka <mark.tinka@seacom.mu>
> Sent: Monday, June 15, 2020 4:07 PM
>
> On 15/Jun/20 12:13, adamv0025@netconsultings.com wrote:
>
> > Not to mention this whole thread is focused solely on next-hop
> identification -which is just the lowest of the layers of abstraction in the
> vertical stack.
> > We haven’t talked about other "entities" that need identification like:
> VPNs, applications, policies (yes I'm looking at you VXLAN!) etc... - all of
> which are way better identified by a simple label rather than IPinIPinIP....
>
> The problem is if you want to have MPLS services signaled over IPv6, you first
> need an IPv6 control plane that will signal the labels that will support those
> services, i.e., LDPv6 and RSVPv6.
>
> Right now, all vendors that support LDPv6 do so only for pure MPLS-IPv6
> forwarding. If you want l2vpnv6, l3vpnv6, EVPNv6, 4PE, 4VPE, TE-FRRv6,
> e.t.c., you can't get that today. You'd have to signal those services over LDPv4
> or RSVPv4.
>
Hence my earlier comment on why I think it's not commercially feasible to switch to v6 control plane, (the only thing I'd be getting is MPLS-IPv6 which I already have (or have had) via L3VPN-6VPE), time will tell if I ever need to make the switch.
But I'm thankful to you for doing the "ice breaking" for the rest of the community.

adam

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 16/Jun/20 12:00, adamv0025@netconsultings.com wrote:

> Hence my earlier comment on why I think it's not commercially feasible to switch to v6 control plane,

Personally, I've never been a fan of a single-stack backbone. I can,
however, understand the use-case where a new or growing network is
unable to obtain anymore IPv4 space and don't want to use RFC 1918 space
(yuck!).


> (the only thing I'd be getting is MPLS-IPv6 which I already have (or have had) via L3VPN-6VPE),

Well, not quite.

What you currently have with 6PE is IPv6 tunneled inside MPLSv4 which
runs over IPv4. While you get the benefits of MPLS forwarding for your
IPv6 traffic, you now create a condition where your IPv6 network is in
the hands of your IPv4 network. Granted, there are many folk that run
6PE, so whether the fate-sharing is of concern to you or not is an
exercise left up to the reader. Personally, I'd rather avoid
fate-sharing whenever I can.

On the other hand, MPLSv6 is native, runs over IPv6 and does not depend
on IPv4 at all.

Ultimately, plenty of energy will need to go into supporting the
additional VPN services that go beyond plain-old MPLSv6 switching. But
that can only be promoted with the vendors after we jump the first
hurdle of deploying the 1st application, basic MPLSv6 switching; get
that widely adopted, and create more awareness within the vendor
community about its overall viability.

80% of our network currently runs LDPv6 and switches IPv6 traffic in
MPLSv6. Our immediate task is to get the remaining 20% supported (IOS
XE) as well.


> But I'm thankful to you for doing the "ice breaking" for the rest of the community.

As Eriq La Salle unashamedly claimed in the 1988 Eddie Murphy picture,
Coming to America, "You know me, anything for the kids :-)".

Mark.

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> From: Mark Tinka <mark.tinka@seacom.mu>
> Sent: Tuesday, June 16, 2020 12:09 PM
>
> On 16/Jun/20 12:00, adamv0025@netconsultings.com wrote:
>
> > Hence my earlier comment on why I think it's not commercially feasible
> > to switch to v6 control plane,
>
> Personally, I've never been a fan of a single-stack backbone. I can, however,
> understand the use-case where a new or growing network is unable to
> obtain anymore IPv4 space and don't want to use RFC 1918 space (yuck!).
>
Actually I was exactly I that situation and v4 RFC 1918 space worked out just fine.

>
> > (the only thing I'd be getting is MPLS-IPv6 which I already have (or
> > have had) via L3VPN-6VPE),
>
> Well, not quite.
>
> What you currently have with 6PE is IPv6 tunneled inside MPLSv4 which runs
> over IPv4. While you get the benefits of MPLS forwarding for your
> IPv6 traffic, you now create a condition where your IPv6 network is in the
> hands of your IPv4 network. Granted, there are many folk that run 6PE, so
> whether the fate-sharing is of concern to you or not is an exercise left up to
> the reader. Personally, I'd rather avoid fate-sharing whenever I can.
>
I've been dependent solely on v4 all my life and I still am.
But I see your fate-sharing argument, similar to my argument around separate iBGP infrastructure (Route-Reflector plane) for Internet vs for other customer private VPN prefixes.
But in the multiplanar iBGP case one plane is statistically more likely to fail than the other, whereas in case of v4 vs v6 control plane I'd say it's actually the NEW v6 that's more likely hiding some unforeseen bug.
So let me ask the following "devil's advocate" type of question, under the assumption that the LDPv6 is a new thing (not as proven as LDPv4), are you taking dependency away by splitting control plane to v4 and v6 or actually adding dependency - where the NEW v6 control plane components could negatively affect the existing v4 control plane components after all they share a common RE (or even RDP in Junos).

> On the other hand, MPLSv6 is native, runs over IPv6 and does not depend on
> IPv4 at all.
>
> Ultimately, plenty of energy will need to go into supporting the additional
> VPN services that go beyond plain-old MPLSv6 switching. But that can only be
> promoted with the vendors after we jump the first hurdle of deploying the
> 1st application, basic MPLSv6 switching; get that widely adopted, and create
> more awareness within the vendor community about its overall viability.
>
> 80% of our network currently runs LDPv6 and switches IPv6 traffic in MPLSv6.
> Our immediate task is to get the remaining 20% supported (IOS
> XE) as well.
>
>
> > But I'm thankful to you for doing the "ice breaking" for the rest of the
> community.
>
> As Eriq La Salle unashamedly claimed in the 1988 Eddie Murphy picture,
> Coming to America, "You know me, anything for the kids :-)".
>
That was a good movie :D :D :D

adam


_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 16/Jun/20 14:24, adamv0025@netconsultings.com wrote:

> Actually I was exactly I that situation and v4 RFC 1918 space worked out just fine.

In that way, you are braver than me. But hey, if you need IPv4 and can't
get the public stuff, I won't fault you for going with the private stuff
:-).


> I've been dependent solely on v4 all my life and I still am.
> But I see your fate-sharing argument, similar to my argument around separate iBGP infrastructure (Route-Reflector plane) for Internet vs for other customer private VPN prefixes.
> But in the multiplanar iBGP case one plane is statistically more likely to fail than the other, whereas in case of v4 vs v6 control plane I'd say it's actually the NEW v6 that's more likely hiding some unforeseen bug.
> So let me ask the following "devil's advocate" type of question, under the assumption that the LDPv6 is a new thing (not as proven as LDPv4), are you taking dependency away by splitting control plane to v4 and v6 or actually adding dependency - where the NEW v6 control plane components could negatively affect the existing v4 control plane components after all they share a common RE (or even RDP in Junos).

Well, that's a bottomless rabbit hole that go could all to the way to
the data centre providing A+B power feeds, but connected to a single
grid on the outside. At some point, redundancy stops making sense and
eats into your margins as much as it does your sanity :-).

But back to the question at hand, even with 6PE, you can't avoid running
a dual-stack network... you'd need to at the edge. So if your goal is to
use 6PE to avoid running IPv6 in some native form anywhere in your
network, that won't work out, as I'm sure you know :-).

But more importantly, I, as have many others on this group, have been
running IPv6 since about 2003 (others, even longer, I'm sure). IPv6, in
and of itself, has never been an issue for me. The problems have always
been the ancillary services that need to run on top of it in order to
work. For the past 17 years, my IPv6 headaches have been about feature
parity with IPv4, mostly, in routers and switches (in general-purpose
server OS's too, but those got fixed much more quickly):

* DNS over IPv6 took a while to arrive.
* TACACS+ over IPv6 took a while to arrive.
* IPv6 ACL's took a while to get proper support.
* SNMP over IPv6 took a while to arrive.
* NTP over IPv6 took a while to arrive.
* SSH over IPv6 took a while to arrive.
* OSPFv3 Authentication was very clunky.
* Multi Topoloy IS-IS support was very clunky.

You get the idea.

I've always operated a native dual-stack network, so having to go back
and upgrade routers every so often when one of the above limitations got
fixed in a later revision of code was tiresome, but worthwhile. We take
a lot of these things for granted in 2020, but it was no joke more than
a decade ago.

So for me, I've never really experienced any problems from basic IPv6
that have negatively impacted IPv4.

The corner case I am aware of that didn't even bother IPv4 was Ethernet
switches and some popular Chinese GPON AN's that silently dropped
Ethernet frames carrying IPv6 packets because they did not know how to
handle the 0x86DD EtherType. But AFAIK, these have all been since fixed.

So based on pure experience, I don't expect this "32-year old new IPv6"
thing to be hiding some unforeseen bug that will break our IPv4 network :-).

LDPv6 was first implemented in IOS XR, Junos and SR-OS in 2015/2016, so
it has been around for a while. The biggest challenge was with IOS XR in
2015 (5.3.0) which didn't support dual-stack TLV's. So if the LDP
neighbor negotiated LDPv4 and LDPv6 in the same LDP session, IOS XR
didn't know what to do. It could do LDPv4-only or LDPv6-only, and not
both. That issue was fixed in IOS XR 6.0.1, when the Dual-Stack
Capability TLV feature was introduced. That was May of 2016, so also not
that new.

Mark.

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/