Mailing List Archive: Rant: ASR1000 MPLS (not) load-balancing

Rant: ASR1000 MPLS (not) load-balancing

Dec 30, 2019, 6:24 AM

Post #1 of 15 (3326 views)

Dear list,

tl;dr: this is a rant about ASR1000 not load-balancing (Eo-)MPLS traffic.

So the common approach to load-balance MPLS packets at the LSR
(ingress MPLS, egress MPLS) is to either look at the payload
(IPv4/IPv6 source/destination address if that is detected) or to
fallback to load-balancing based on the bottom of stack label (for
example in the EoMPLS case). At least the latter should be a common
denominator in all load-balancing MPLS implementations because it's
just so basic and for the EoMPLS case it basically means per-VC
load-balancing.

Or so I thought naively ...

In a LSR configuration (ingress MPLS, egress MPLS) with port-channels,
the ASR1000:

- does load-balance when detecting vpnv4/vpnv6 traffic (first octet 4
or 6 in the mpls payload), based on the L3 source/destination address
in the MPLS payload
- does not load-balance at all when not detecting 4 or 6 traffic
(*not* considering the bottom of stack label to load-balance this
traffic)

So what does that mean for:

- EoMPLS traffic with or without control word?
- EoMPLS traffic with FAT or entropy labels?

It's not getting load-balanced at all [1]. Bucked-ID 0 is assigned
always, so the *ENTIRE* EoMPLS traffic (whatever the number of VC's)
is egressing on a single link (the one assigned to Bucked-ID 0).

But this can't be right I hear you saying ... I mean load-balancing
based on the MPLS payload is much more complex than load-balancing
based on the bottom of stack label, so it must just be an oversight,
right?

The response by the BU is of course the usual "status quo is by design":

> It is by design from ASR1K implementation

and:

> BU has updated the documentation [2]:
>
> “This feature achieves load balancing of MPLS traffic only by using source IP address and destination IP address. The MPLS label is not considered for load balancing.”

The ASR1000 is of course not designed to be a P box. But sometimes you
do have MPLS traffic transiting a box and load-balancing works fine
for much more complicated use-cases (vpnv4/6 address, or EoMPLS in
ECMP configurations). I personally find it ridiculous to drop the
D-bomb in this case, when clearly it is just an oversight. The fact
that it was an oversight some 10 years ago does not make it suddenly
"by design". Neither does lack of specification in the actual design.

Dropping the port-channels for ECMP now, hurray.

That was my rant for the day, I wish you all a happy new year (and may
it be "by design"),
lukas

[1] Except of course erroneous load-balancing of EoMPLS traffic
without control-word, if the first octet of the payload is 4 or 6
[2] https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/lanswitch/configuration/xe-16/lanswitch-xe-16-book/lnsw-flow-portchannel-load.html#GUID-2FB36A3D-6603-48DC-8FAA-CA3F604F462E
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: Rant: ASR1000 MPLS (not) load-balancing [ In reply to ]

adamv0025 at netconsultings

Dec 31, 2019, 5:58 AM

Post #2 of 15 (3322 views)

Permalink

Thanks for sharing,
This sucks, but at least it's limited to load-sharing over bundle members only (as opposed to ECMP load-sharing) right?

I was going to object that this can't possibly be by design cause what if I have 100 labels on the packet, then I realized that ASR1k's QFP gets the whole packets not just the first N bytes, so I guess I can see the thought process like "since we are getting the whole packet we might as well look into IP header and balance based on what we'll find there".
And I'm not sure if at the time of ASR1k design the methods to indicate flow using label/label-stack were around, but I would have thought that the problems related to PW traffic load-sharing were known already, suggesting that making the parser look for v4/v6 in bundle scenario as a hard and fast rule might not be the right way forward? (After all they got it right in the ECMP scenario right?...)

adam

> -----Original Message-----
> From: cisco-nsp <cisco-nsp-bounces@puck.nether.net> On Behalf Of Lukas
> Tribus
> Sent: Monday, December 30, 2019 2:24 PM
> To: Cisco-nsp <cisco-nsp@puck.nether.net>
> Subject: [c-nsp] Rant: ASR1000 MPLS (not) load-balancing
>
> Dear list,
>
>
> tl;dr: this is a rant about ASR1000 not load-balancing (Eo-)MPLS traffic.
>
>
> So the common approach to load-balance MPLS packets at the LSR (ingress
> MPLS, egress MPLS) is to either look at the payload
> (IPv4/IPv6 source/destination address if that is detected) or to fallback to
> load-balancing based on the bottom of stack label (for example in the
> EoMPLS case). At least the latter should be a common denominator in all
> load-balancing MPLS implementations because it's just so basic and for the
> EoMPLS case it basically means per-VC load-balancing.
>
> Or so I thought naively ...
>
>
> In a LSR configuration (ingress MPLS, egress MPLS) with port-channels, the
> ASR1000:
>
> - does load-balance when detecting vpnv4/vpnv6 traffic (first octet 4 or 6 in
> the mpls payload), based on the L3 source/destination address in the MPLS
> payload
> - does not load-balance at all when not detecting 4 or 6 traffic
> (*not* considering the bottom of stack label to load-balance this
> traffic)
>
>
> So what does that mean for:
>
> - EoMPLS traffic with or without control word?
> - EoMPLS traffic with FAT or entropy labels?
>
>
> It's not getting load-balanced at all [1]. Bucked-ID 0 is assigned always, so the
> *ENTIRE* EoMPLS traffic (whatever the number of VC's) is egressing on a
> single link (the one assigned to Bucked-ID 0).
>
>
> But this can't be right I hear you saying ... I mean load-balancing based on the
> MPLS payload is much more complex than load-balancing based on the
> bottom of stack label, so it must just be an oversight, right?
>
>
> The response by the BU is of course the usual "status quo is by design":
>
> > It is by design from ASR1K implementation
>
> and:
>
> > BU has updated the documentation [2]:
> >
> > “This feature achieves load balancing of MPLS traffic only by using source IP
> address and destination IP address. The MPLS label is not considered for load
> balancing.”
>
> The ASR1000 is of course not designed to be a P box. But sometimes you do
> have MPLS traffic transiting a box and load-balancing works fine for much
> more complicated use-cases (vpnv4/6 address, or EoMPLS in ECMP
> configurations). I personally find it ridiculous to drop the D-bomb in this case,
> when clearly it is just an oversight. The fact that it was an oversight some 10
> years ago does not make it suddenly "by design". Neither does lack of
> specification in the actual design.
>
> Dropping the port-channels for ECMP now, hurray.
>
>
> That was my rant for the day, I wish you all a happy new year (and may it be
> "by design"), lukas
>
>
> [1] Except of course erroneous load-balancing of EoMPLS traffic without
> control-word, if the first octet of the payload is 4 or 6 [2]
> https://www.cisco.com/c/en/us/td/docs/ios-
> xml/ios/lanswitch/configuration/xe-16/lanswitch-xe-16-book/lnsw-flow-
> portchannel-load.html#GUID-2FB36A3D-6603-48DC-8FAA-CA3F604F462E
> _______________________________________________
> cisco-nsp mailing list cisco-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: Rant: ASR1000 MPLS (not) load-balancing [ In reply to ]

saku at ytti

Dec 31, 2019, 6:11 AM

Post #3 of 15 (3322 views)

Permalink

On Tue, 31 Dec 2019 at 15:59, <adamv0025@netconsultings.com> wrote:

> I was going to object that this can't possibly be by design cause what if I have 100 labels on the packet, then I realized that ASR1k's QFP gets the whole packets not just the first N bytes, so I guess I can see the thought process like "since we are getting the whole packet we might as well look into IP header and balance based on what we'll find there".

Based on context I believe 'this' here means ability to look past
labels into IP headers. This is the norm, not exception. And it is
typically followed by restriction such as 'up tp 5 labels'.

There is absolutely no way to know for sure what is carried by MPLS,
so fast and simple rule of 4/6 is fine, as it's the common case to
follow MPLS label by IP header, with implied promise that any other
protocol you send does not start with 4 or 6, i.e. code-word is
non-optional if you plan to do any transit heuristics, no matter how
smart the heuristics is, it is heuristics, not guarantee.

I agree with Lukas that not including labels in balancing hash
calculation is broken behaviour.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: Rant: ASR1000 MPLS (not) load-balancing [ In reply to ]

adamv0025 at netconsultings

Jan 2, 2020, 5:28 AM

Post #4 of 15 (3310 views)

Permalink

Hey Saku, happy new year!

> From: Saku Ytti <saku@ytti.fi>
> Sent: Tuesday, December 31, 2019 2:12 PM
>
> On Tue, 31 Dec 2019 at 15:59, <adamv0025@netconsultings.com> wrote:
>
> > I was going to object that this can't possibly be by design cause what if I
> have 100 labels on the packet, then I realized that ASR1k's QFP gets the
> whole packets not just the first N bytes, so I guess I can see the thought
> process like "since we are getting the whole packet we might as well look
> into IP header and balance based on what we'll find there".
>
> Based on context I believe 'this' here means ability to look past labels into IP
> headers. This is the norm, not exception. And it is typically followed by
> restriction such as 'up tp 5 labels'.
>
I thought that what you wrote is obvious from the one liner comment quoted.
Nevertheless yes correct that's what I meant, that on majority of the NPUs used in routers there are limits as to how many labels a platform can skip in order to inspect IP header contents and I was going to use this against ASR1k and the hard and fast rule to always look for IP header wondering how could this rule work if there's a limit of say 5 labels and my packet has more -what will the platform do then in terms of load-sharing if it can do such decision based solely on contents of the IP header but can't get those during parsing operation (due to up to 5 labels limit for instance) possibly leading to some kind of exception, however then I realized that QFP used in ASR9k is different in this regard and always has access to all the contents of a packet (very much like a FW NPUs for instance), therefore the "usual" router limits (e.g. lookup up to 5 labels deep) do not apply in case of asr1k so I was playing devil's advocate arguing how I could see ASR1k eng. Falling for this trap
.
So I guess this is the long version of my thought process summed in that one liner there.

> There is absolutely no way to know for sure what is carried by MPLS,
Exactly

> so fast
> and simple rule of 4/6 is fine,
Yes fine as to usually does the right thing, but completely oblivious to any guidance on load-sharing carried by the label stack

> as it's the common case to follow MPLS label by
> IP header, with implied promise that any other protocol you send does not
> start with 4 or 6, i.e. code-word is non-optional if you plan to do any transit
> heuristics, no matter how smart the heuristics is, it is heuristics, not
> guarantee.
>
Hence I'd always prefer transit nodes to use solely the MPLS stack for any clues on how to load-share.
As the PE has much better chance of having an un-skewed view on what is or is not part of the same flow therefore my preference is for PEs to record the flow information in the flow stack and having P nodes blindly follow that (i.e. the use of entropy labels).
For what it's worth nowadays it could be an application or a controller with true knowledge of what constitutes a flow programming the label stack a PE should impose on a set of packets.
And it seems this ASR1k acting as MPLS transit node and using port-channel interfaces, will happily ignore any clues carried by the label stack.

> I agree with Lukas that not including labels in balancing hash calculation is
> broken behaviour.
So do I.
I'd have Cisco to update ASR1k documentation also on use of Entropy and Flow labels to put a note there that these features are not supported in combination with port-channels.

adam

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: Rant: ASR1000 MPLS (not) load-balancing [ In reply to ]

robert at raszuk

Jan 2, 2020, 5:45 AM

Post #5 of 15 (3310 views)

Permalink

> Hence I'd always prefer transit nodes to use solely the MPLS stack for any
> clues on how to load-share.

That may not be a good idea.

Think about SR-MPLS and global labels with say 5 TE segment nodes (hops).
As MPLS header would be identical all flows travelling via such TE path
would get zero load balancing across ECMP paths even parallel L3 links.

Even if you get fancy and stick there EL as described
in draft-ietf-mpls-spring-entropy-label-12 the limited ERLD in most
platforms will not even reach it in the middle of the network.

Not to mention that P routes may be receiving both IP and MPLS traffic. But
I guess your point was that ".. if packets is mpls packet".

Thx,
R.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: Rant: ASR1000 MPLS (not) load-balancing [ In reply to ]

saku at ytti

Jan 2, 2020, 6:45 AM

Post #6 of 15 (3308 views)

Permalink

On Thu, 2 Jan 2020 at 15:46, Robert Raszuk <robert@raszuk.net> wrote:

>> Hence I'd always prefer transit nodes to use solely the MPLS stack for any clues on how to load-share.
>
> That may not be a good idea.
>
> Think about SR-MPLS and global labels with say 5 TE segment nodes (hops). As MPLS header would be identical all flows travelling via such TE path would get zero load balancing across ECMP paths even parallel L3 links.
>
> Even if you get fancy and stick there EL as described in draft-ietf-mpls-spring-entropy-label-12 the limited ERLD in most platforms will not even reach it in the middle of the network.

Adam implied presence of entropy indeed, and of course this would then
be design constraint in choosing the LSR. Most platforms seems
ambiguous, Trio, FP4, Jericho, Pacific should be good to +dozen
labels, which seems very much reasonable for SR+EL. No perfect answer,
but certainly if you want to design network to never do payload
heuristics, it's entirely possible and there is no significant CAPEX
penalty to it.

> Not to mention that P routes may be receiving both IP and MPLS traffic. But I guess your point was that ".. if packets is mpls packet".

And if you only balance on labels and include EL, then it doesn't
matter what it carries. There is absolutely no way to be sure that
MPLS payload heuristics is interpreting bytes correctly, it cannot
know what it is carrying. The more verification you do (like IP packet
length match) the rarer and harder to troubleshoot the problems are,
the problems quickly become so esoteric you'll never even hear of
them, because even customer can't believe it would be caused by the
service provider's pseudowire. Like 'all hosts work, then we added
ipsec, and one host stopped working' could very well be transit
heuristics problem, but going from customer => noc => ops => dev =>
vendor would be unlikely.
IMHO, if you MUST do payload heuristics, simple heuristic 4 == IPv4, 6
== IPv6, anything else is unknown is the correct heuristics. And then
you must design your network to honor that assumption.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: Rant: ASR1000 MPLS (not) load-balancing [ In reply to ]

robert at raszuk

Jan 2, 2020, 7:08 AM

Post #7 of 15 (3308 views)

Permalink

All true.

But for me from the perspective of number of years passed by I think using
MPLS transport is just a one big mistake.

MPLS for service demux is great, but I really see no advantages in using
LDP or even concept of global labels for transport. IP transport be it IPv4
or IPv6 offers much more advantages, native summarization UDP or flow
label adds a lot of entropy while MPLS service label can still sit happy
behind.

If you need TE - it seems pretty trivial to do that with either IPv6 or
IPv4 encapsulation too.

Originally when we started with tag switching and then mpls GSRs did not
support IP encap (except of CPU based Engine 0 LCs) and hence to offer any
SP services - to allow them to move from ATM and F/R to IP yet still
interconnect customers new transport was rolled out. RSVP-TE came along as
natural extension too, but as we all know it has its own set of problems.

Sev 1 bug free 2020,
R.

On Thu, Jan 2, 2020 at 3:46 PM Saku Ytti <saku@ytti.fi> wrote:

> On Thu, 2 Jan 2020 at 15:46, Robert Raszuk <robert@raszuk.net> wrote:
>
>
> >> Hence I'd always prefer transit nodes to use solely the MPLS stack for
> any clues on how to load-share.
> >
> > That may not be a good idea.
> >
> > Think about SR-MPLS and global labels with say 5 TE segment nodes
> (hops). As MPLS header would be identical all flows travelling via such TE
> path would get zero load balancing across ECMP paths even parallel L3 links.
> >
> > Even if you get fancy and stick there EL as described in
> draft-ietf-mpls-spring-entropy-label-12 the limited ERLD in most platforms
> will not even reach it in the middle of the network.
>
> Adam implied presence of entropy indeed, and of course this would then
> be design constraint in choosing the LSR. Most platforms seems
> ambiguous, Trio, FP4, Jericho, Pacific should be good to +dozen
> labels, which seems very much reasonable for SR+EL. No perfect answer,
> but certainly if you want to design network to never do payload
> heuristics, it's entirely possible and there is no significant CAPEX
> penalty to it.
>
> > Not to mention that P routes may be receiving both IP and MPLS traffic.
> But I guess your point was that ".. if packets is mpls packet".
>
> And if you only balance on labels and include EL, then it doesn't
> matter what it carries. There is absolutely no way to be sure that
> MPLS payload heuristics is interpreting bytes correctly, it cannot
> know what it is carrying. The more verification you do (like IP packet
> length match) the rarer and harder to troubleshoot the problems are,
> the problems quickly become so esoteric you'll never even hear of
> them, because even customer can't believe it would be caused by the
> service provider's pseudowire. Like 'all hosts work, then we added
> ipsec, and one host stopped working' could very well be transit
> heuristics problem, but going from customer => noc => ops => dev =>
> vendor would be unlikely.
> IMHO, if you MUST do payload heuristics, simple heuristic 4 == IPv4, 6
> == IPv6, anything else is unknown is the correct heuristics. And then
> you must design your network to honor that assumption.
>
> --
> ++ytti
>
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: Rant: ASR1000 MPLS (not) load-balancing [ In reply to ]

saku at ytti

Jan 2, 2020, 7:19 AM

Post #8 of 15 (3308 views)

Permalink

On Thu, 2 Jan 2020 at 17:08, Robert Raszuk <robert@raszuk.net> wrote:

> But for me from the perspective of number of years passed by I think using MPLS transport is just a one big mistake.

Pistols at dawn, I drank the kool aid and it's good.

> MPLS for service demux is great, but I really see no advantages in using LDP or even concept of global labels for transport. IP transport be it IPv4 or IPv6 offers much more advantages, native summarization UDP or flow label adds a lot of entropy while MPLS service label can still sit happy behind.

We need tunneling, that much is clear, what the tunnel bytes exactly
are, is not very interesting to me, as long as there are as few of
them as possible. Toy idea from some 6 years back
https://ytti.fi/mplsv2.txt

> If you need TE - it seems pretty trivial to do that with either IPv6 or IPv4 encapsulation too.

I see no upside. Exact match MPLS is on-chip LEM, which is cheap, this
is good, tunneling should be exact match LEM lookup. IP is LPM, it's
fundamentally more expensive. Not to mention too many bytes, so you
need more overspeed for given edge rate.
Minimum amount of excess bytes, LEM table lookup, these are
fundamental requirements for ideal tunneling mechanism, tunneling is
needed. If it's MPLS or what ever is not important, but IP is
blatantly worse.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: Rant: ASR1000 MPLS (not) load-balancing [ In reply to ]

robert at raszuk

Jan 2, 2020, 7:41 AM

Post #9 of 15 (3308 views)

Permalink

First most of the ASICs or NPs you get do just fine LEM or LPM at their
line rate. And while processing wise they may be more cycles to do LPM your
invoice amount for box is still the same :) I guess one vendor tried to do
pure LEM box ... but you know the story there.

Then consider smart LPMs like Arista FlexRoute .. the advantages of LEM
here are getting very marginal for the significant cost of control plane
complexity.

Best,
R.

On Thu, Jan 2, 2020 at 4:19 PM Saku Ytti <saku@ytti.fi> wrote:

> On Thu, 2 Jan 2020 at 17:08, Robert Raszuk <robert@raszuk.net> wrote:
>
> > But for me from the perspective of number of years passed by I think
> using MPLS transport is just a one big mistake.
>
> Pistols at dawn, I drank the kool aid and it's good.
>
> > MPLS for service demux is great, but I really see no advantages in using
> LDP or even concept of global labels for transport. IP transport be it IPv4
> or IPv6 offers much more advantages, native summarization UDP or flow
> label adds a lot of entropy while MPLS service label can still sit happy
> behind.
>
> We need tunneling, that much is clear, what the tunnel bytes exactly
> are, is not very interesting to me, as long as there are as few of
> them as possible. Toy idea from some 6 years back
> https://ytti.fi/mplsv2.txt
>
> > If you need TE - it seems pretty trivial to do that with either IPv6 or
> IPv4 encapsulation too.
>
> I see no upside. Exact match MPLS is on-chip LEM, which is cheap, this
> is good, tunneling should be exact match LEM lookup. IP is LPM, it's
> fundamentally more expensive. Not to mention too many bytes, so you
> need more overspeed for given edge rate.
> Minimum amount of excess bytes, LEM table lookup, these are
> fundamental requirements for ideal tunneling mechanism, tunneling is
> needed. If it's MPLS or what ever is not important, but IP is
> blatantly worse.
>
> --
> ++ytti
>
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: Rant: ASR1000 MPLS (not) load-balancing [ In reply to ]

saku at ytti

Jan 2, 2020, 8:07 AM

Post #10 of 15 (3308 views)

Permalink

On Thu, 2 Jan 2020 at 17:42, Robert Raszuk <robert@raszuk.net> wrote:

> First most of the ASICs or NPs you get do just fine LEM or LPM at their line rate. And while processing wise they may be more cycles to do LPM your invoice amount for box is still the same :) I guess one vendor tried to do pure LEM box ... but you know the story there.

No box does line rate, there is no market for it and it's expensive.
Only way to do line rate is to stop using half or more of the ports.

> Then consider smart LPMs like Arista FlexRoute .. the advantages of LEM here are getting very marginal for the significant cost of control plane complexity.

Unsure how we evaluate complexity here and of what. But LPM/LEM
difference is shrinking in high end because there is strong market for
high performance LPM, doesn't change the fact that small table fast
LEM is fundamentally cheaper to implement.
Now when it comes to control-plane complexity, that doesn't care at
all on the bytes imposed, so what ever tunneling headers we come up
with, has no inherent bearing to complexity of control-plane. There is
no reason for IP to simpler or more complex in control-plane compared
to MPLS.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: Rant: ASR1000 MPLS (not) load-balancing [ In reply to ]

robert at raszuk

Jan 2, 2020, 8:14 AM

Post #11 of 15 (3307 views)

Permalink

>
> There is no reason for IP to simpler or more complex in control-plane
> compared
> to MPLS.

Disagree.

WIth flat MPLS transport labels must be present to reach all of your 100s
or 1000s of LSP endpoints in another IGP area or your other global AS.
Think VPN Option-C

With IP transport all I need is a *single* IP summary route to my other
area or AS.

I hope control plane difference is clear now.

Thx
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: Rant: ASR1000 MPLS (not) load-balancing [ In reply to ]

saku at ytti

Jan 3, 2020, 12:23 AM

Post #12 of 15 (3299 views)

Permalink

On Thu, 2 Jan 2020 at 18:14, Robert Raszuk <robert@raszuk.net> wrote:

>> There is no reason for IP to simpler or more complex in control-plane compared
>> to MPLS.
>
>
> Disagree.
>
> WIth flat MPLS transport labels must be present to reach all of your 100s or 1000s of LSP endpoints in another IGP area or your other global AS. Think VPN Option-C
>
> With IP transport all I need is a *single* IP summary route to my other area or AS.
>
> I hope control plane difference is clear now.

It is not, you are looking particular implementations, which have no
implicit correlation to the actual bytes imposed, either could do
either. Only thing which matters is what are the lookup keys and how
many are there. What algorithmic solution do I need in HW to find
rewrite information for those lookup keys.

But we may be beyond discussion of value to the list audience.
--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: Rant: ASR1000 MPLS (not) load-balancing [ In reply to ]

adamv0025 at netconsultings

Jan 3, 2020, 3:30 AM

Post #13 of 15 (3299 views)

Permalink

> From: Saku Ytti <saku@ytti.fi>
> Sent: Thursday, January 2, 2020 4:07 PM
>
> On Thu, 2 Jan 2020 at 17:42, Robert Raszuk <robert@raszuk.net> wrote:
>
> > First most of the ASICs or NPs you get do just fine LEM or LPM at their line
> rate. And while processing wise they may be more cycles to do LPM your
> invoice amount for box is still the same :) I guess one vendor tried to do pure
> LEM box ... but you know the story there.
>
> No box does line rate, there is no market for it and it's expensive.
> Only way to do line rate is to stop using half or more of the ports.
>
Well there are a few routers/line-cards that can deliver line-rate with a following disclaimer,

On run to completion NPUs there's really no such thing as just line-rate (it's always line-rate with what combination and scale of features enabled).
One can make any run to completion NPU fail miserably revealing interesting facts about the NPU or whole box design choices.
So the "line-rate-ness" is just a good indication of how much headroom one has in terms of enabling additional features while expecting a certain throughput -which with advent of 100G very little line-card variants deliver good headroom for additional features (with "additional" I mean above basic IP/MPLS forwarding).

On pipeline NPUs there is such a thing as line-rate.
There’s a max time (number of clock ticks) the packet can spend in the pipeline - and if my u-code is too complex that the packet would spend more, then the compiler won’t allow me to compile such a u-code.
That's why they claim you get line-rate for any combination of features on these chips. (obviously there's a limited number of supported feature combinations compared to run to completion chips where one get carte la blanche in terms of time the packet can spend in the assigned packet processing engine).

But there's a big HOWEVER,
Line-rate is a very ambiguous term these days, so the question is what packet size ergo packet rate did they use as line-rate baseline.
In my book it's lowest possible packet size 64B@L2 hence fastest possible packet rate is line-rate, but some conveniently use 128B (half the pps rate compared to 64B) and even higher.

And of course in both cases it's about how many ports a vendor hooks up to an NPU -and as Saku mentioned they tend to overdo it in majority of the cases and that's perfectly fine as we need holes first and foremost the more the better on the bases that all these pipes will be half empty anyways.
Unfortunately the modern licensing is screwing us big time in this regard.

adam

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: Rant: ASR1000 MPLS (not) load-balancing [ In reply to ]

adamv0025 at netconsultings

Jan 3, 2020, 3:59 AM

Post #14 of 15 (3299 views)

Permalink

> From: Saku Ytti <saku@ytti.fi>
> Sent: Friday, January 3, 2020 8:24 AM
>
> On Thu, 2 Jan 2020 at 18:14, Robert Raszuk <robert@raszuk.net> wrote:
>
> >> There is no reason for IP to simpler or more complex in control-plane
> >> compared to MPLS.
> >
> >
> > Disagree.
> >
> > WIth flat MPLS transport labels must be present to reach all of your
> > 100s or 1000s of LSP endpoints in another IGP area or your other
> > global AS. Think VPN Option-C
> >
> > With IP transport all I need is a *single* IP summary route to my other area
> or AS.
> >
> > I hope control plane difference is clear now.
>
> It is not, you are looking particular implementations, which have no implicit
> correlation to the actual bytes imposed, either could do either. Only thing
> which matters is what are the lookup keys and how many are there. What
> algorithmic solution do I need in HW to find rewrite information for those
> lookup keys.
>
Hey Robert, Happy new year!
On a theoretical fundamental level I have to agree with Saku here,
Think SDN controller implementation of CP for instance, the control-plane complexity of Intra-AS vs Inter-AS LSP programming is exactly the same in this example.

However on practical level,
You're right in the complexity argument in a sense that we have to work with/support technology we currently have installed -which isn't pure SDN controlled Inter-AS setup, but more like ISIS + LDP/RSVP at the intra-AS layer and BGP-LU at the Inter-AS layer. And all the headaches (and elaborate hacks) around scaling in absence of summarization especially in setups with MPLS all the way to the access layer (e.g. cell-tower eNode B in RAN).

adam

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: Rant: ASR1000 MPLS (not) load-balancing [ In reply to ]

mark.tinka at seacom

Jan 7, 2020, 4:34 AM

Post #15 of 15 (3266 views)

Permalink

On 30/Dec/19 16:24, Lukas Tribus wrote:

>
> Dropping the port-channels for ECMP now, hurray.

So this particular issue was the very reason we dumped the use of LACP
to create aggregate bandwidth in the core, and instead, run individual N
x 10Gbps links on IP/MPLS, and let them ECMP with one another.

As I speak, we are now in a position to move several backbone links to
native 100Gbps, which will be better, but at scale, LACP is a real problem.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/