Mailing List Archive

802.3ad LAG between ASR 1002-X and Juniper MX204
Hi all,
I need to bring up a 2x10G LAG between an MX204 and a customer's ASR 1002-X and I want to make sure the links get load balanced as closely and reliably as possible. Junos docs say, "The hash-computation for the forward and reverse flow must be identical." They go on to detail how to configure a link index to each physical port and that Trio chipsets require symmetrical load-balancing. The LAG will be bridged through to one of the 40G uplinks on the MX204. Here's my config for the Juniper side:

chassis {
aggregated-devices {
ethernet {
device-count 1;
}
}
fpc 0 {
pic 1 {
hash-key {
family {
multiservice {
payload {
ip {
layer-3;
}
}
}
}
}
}
}
}
interfaces {
et-0/0/0 {
per-unit-scheduler;
flexible-vlan-tagging;
encapsulation flexible-ethernet-services;
}
unit 2 {
encapsulation vlan-bridge;
vlan-id 3190;
family bridge;
}
}
xe-0/1/0 {
gigether-options {
802.3ad {
ae0;
link-index 0;
}
}
}
xe-0/1/1 {
gigether-options {
802.3ad {
ae0;
link-index 1;
}
}
}
ae0 {
encapsulation ethernet-bridge;
aggregated-ether-options {
no-flow-control;
minimum-links 1;
link-speed 10g;
}
unit 0 {
family bridge;
}
}
}
forwarding-options {
hash-key {
family multiservice {
payload {
ip {
layer-3;
}
}
symmetric-hash;
}
}
enhanced-hash-key {
family multiservice {
no-mac-addresses;
}
symmetric;
}
}
On the Cisco, I am going to suggest:
port-channel load-balance-hash-algo src-dst-ip
!
interface TenGigabitEthernet0/0/0
no ip address
channel-group 1 link 1
!
interface TenGigabitEthernet0/0/1
no ip address
channel-group 1 link 2
!
interface Port-channel1
no negotiation auto
ip address 10.45.98.10 255.255.255.0
load-balance flow
!
Can anyone tell me if there is anything I'm missing here? I did not include here my CoS config to help with serialization delay issues. I don't have an ASR 1002-X to test with and the ASR 920s I do have available to me don’t have full command parity with the 1002-X. I’m also not sure of the chipset differences between the 1002-X and the 920 model. Any suggestions appreciated.
Thanks,
evt


_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: 802.3ad LAG between ASR 1002-X and Juniper MX204 [ In reply to ]
Hi,

On Fri, Jul 19, 2019 at 07:33:59PM +0000, Eric Van Tol wrote:
> Hi all,
> I need to bring up a 2x10G LAG between an MX204 and a customer's ASR 1002-X and I want to make sure the links get load balanced as closely and reliably as possible. Junos docs say, "The hash-computation for the forward and reverse flow must be identical." They go on to detail how to configure a link index to each physical port and that Trio chipsets require symmetrical load-balancing.

That sounds a bit weird... why should the device care how the other
end balances its packets? Never heard anyone state this, and I can't
come up with a reason why.

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de
Re: 802.3ad LAG between ASR 1002-X and Juniper MX204 [ In reply to ]
On 7/19/19, 3:40 PM, "Gert Doering" <gert@greenie.muc.de> wrote:
That sounds a bit weird... why should the device care how the other
end balances its packets? Never heard anyone state this, and I can't
come up with a reason why.

*sigh*

I'd been focusing way too much on the config portion of the documentation that I completely skimmed over the very first paragraph:

"MX Series routers with Aggregated Ethernet PICs support symmetrical load balancing on an 802.3ad LAG. This feature is significant when two MX Series routers are connected transparently through deep packet inspection (DPI) devices over an LAG bundle. DPI devices keep track of flows and require information of a given flow in both forward and reverse directions. Without symmetrical load balancing on an 802.3ad LAG, the DPIs could misunderstand the flow, leading to traffic disruptions. By using this feature, a given flow of traffic (duplex) is ensured for the same devices in both directions."

Carry on, nothing to see here...


_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: 802.3ad LAG between ASR 1002-X and Juniper MX204 [ In reply to ]
Hi,

On Fri, Jul 19, 2019 at 07:56:47PM +0000, Eric Van Tol wrote:
> On 7/19/19, 3:40 PM, "Gert Doering" <gert@greenie.muc.de> wrote:
> That sounds a bit weird... why should the device care how the other
> end balances its packets? Never heard anyone state this, and I can't
> come up with a reason why.
>
> *sigh*
>
> I'd been focusing way too much on the config portion of the documentation that I completely skimmed over the very first paragraph:
>
> "MX Series routers with Aggregated Ethernet PICs support symmetrical
> load balancing on an 802.3ad LAG. This feature is significant when
> two MX Series routers are connected transparently through deep
> packet inspection (DPI) devices over an LAG bundle.

Yes, *that* makes total sense :-) (I was thinking about "is it something
with stateful inspection?" but since this - inside MX or Cisco - usually
operates "on the ae/port-channel level" and not the individual member,
it didn't make sense either)

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de
Re: 802.3ad LAG between ASR 1002-X and Juniper MX204 [ In reply to ]
Yes, you'd better drop all the hash+loadbalance+linkindex conf (by the way, on MX the "hash-key" knob is only for DPC cards, 10+ years old).

However about the LAG itself, if you want something reliable you really should use LACP instead of static LAG.
Static LAGs, a good way to get your traffic lost...

> Le 19 juil. 2019 à 22:02, Gert Doering <gert@greenie.muc.de> a écrit :
>
> On Fri, Jul 19, 2019 at 07:56:47PM +0000, Eric Van Tol wrote:
>> On 7/19/19, 3:40 PM, "Gert Doering" <gert@greenie.muc.de> wrote:
>> That sounds a bit weird... why should the device care how the other
>> end balances its packets? Never heard anyone state this, and I can't
>> come up with a reason why.
>>
>> *sigh*
>>
>> I'd been focusing way too much on the config portion of the documentation that I completely skimmed over the very first paragraph:
>>
>> "MX Series routers with Aggregated Ethernet PICs support symmetrical
>> load balancing on an 802.3ad LAG. This feature is significant when
>> two MX Series routers are connected transparently through deep
>> packet inspection (DPI) devices over an LAG bundle.
>
> Yes, *that* makes total sense :-) (I was thinking about "is it something
> with stateful inspection?" but since this - inside MX or Cisco - usually
> operates "on the ae/port-channel level" and not the individual member,
> it didn't make sense either)

_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: 802.3ad LAG between ASR 1002-X and Juniper MX204 [ In reply to ]
On 19/Jul/19 21:40, Gert Doering wrote:

> That sounds a bit weird... why should the device care how the other
> end balances its packets? Never heard anyone state this, and I can't
> come up with a reason why.

Because they assume both sides is a Juniper. I mean, why would you
possibly be running anything else :-)...

Mark.
Re: 802.3ad LAG between ASR 1002-X and Juniper MX204 [ In reply to ]
On 19/Jul/19 22:33, Olivier Benghozi wrote:

> Yes, you'd better drop all the hash+loadbalance+linkindex conf (by the way, on MX the "hash-key" knob is only for DPC cards, 10+ years old).
>
> However about the LAG itself, if you want something reliable you really should use LACP instead of static LAG.
> Static LAGs, a good way to get your traffic lost...

We now restrict LAG's to router-switch 802.1Q trunks.

On backbone links, we've found regular IP ECMP to be more reliable than
LAG's.

Mark.
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: 802.3ad LAG between ASR 1002-X and Juniper MX204 [ In reply to ]
Using BFD instead of LACP for link monitoring?

/Per


On 20 Jul 2019, at 12:06, Mark Tinka wrote:

> We now restrict LAG's to router-switch 802.1Q trunks.
>
> On backbone links, we've found regular IP ECMP to be more reliable than
> LAG's.
>
> Mark.
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: 802.3ad LAG between ASR 1002-X and Juniper MX204 [ In reply to ]
We usually prefer LAGs here (with microBFD on backbone links) ; but, any horror stories to share?


> Le 20 juil. 2019 à 12:06, Mark Tinka <mark.tinka@seacom.mu> a écrit :
>
> We now restrict LAG's to router-switch 802.1Q trunks.
> On backbone links, we've found regular IP ECMP to be more reliable than
> LAG's.

_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: 802.3ad LAG between ASR 1002-X and Juniper MX204 [ In reply to ]
On 20/Jul/19 18:56, Per Westerlund wrote:
> Using BFD instead of LACP for link monitoring?

I wouldn't say BFD is an alternative to LACP, but yes, we do use BFD on
all backbone links.

Also, BFD is broken in LAG scenarios.

Mark.
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: 802.3ad LAG between ASR 1002-X and Juniper MX204 [ In reply to ]
On 20/Jul/19 19:10, Olivier Benghozi wrote:

> We usually prefer LAGs here (with microBFD on backbone links) ; but, any horror stories to share?

BFD on LAG's is broken.

Hashing on LACP is less predictable than hashing on IP.

Mark.
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: 802.3ad LAG between ASR 1002-X and Juniper MX204 [ In reply to ]
> On 21/07/2019, at 6:26 AM, Mark Tinka <mark.tinka@seacom.mu> wrote:
>
>
>
> On 20/Jul/19 18:56, Per Westerlund wrote:
>> Using BFD instead of LACP for link monitoring?
>
> I wouldn't say BFD is an alternative to LACP, but yes, we do use BFD on
> all backbone links.
>
> Also, BFD is broken in LAG scenarios.

I agree with you generally - that multiple links is better than LAG in the core for a bunch of reasons.

However, the issues I’m aware of with BFD on LAGs are dealt with by Micro BFD on each LAG member. Is there something more that I’m not aware of which Micro BFD doesn’t solve?

--
Nathan Ward

_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: 802.3ad LAG between ASR 1002-X and Juniper MX204 [ In reply to ]
On 21/Jul/19 04:09, Nathan Ward wrote:

>
>
> However, the issues I’m aware of with BFD on LAGs are dealt with by Micro BFD on each LAG member. Is there something more that I’m not aware of which Micro BFD doesn’t solve?

So we run BFD both on p2p and mp2mp (LAN) links.

Micro-BFD assumes all LAG's p2p in nature, which is just silly. What
solution do you have for a LAN environment?

Mark.
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: 802.3ad LAG between ASR 1002-X and Juniper MX204 [ In reply to ]
> On 21/07/2019, at 9:41 PM, Mark Tinka <mark.tinka@seacom.mu> wrote:
>
>
>
> On 21/Jul/19 04:09, Nathan Ward wrote:
>
>>
>>
>> However, the issues I’m aware of with BFD on LAGs are dealt with by Micro BFD on each LAG member. Is there something more that I’m not aware of which Micro BFD doesn’t solve?
>
> So we run BFD both on p2p and mp2mp (LAN) links.
>
> Micro-BFD assumes all LAG's p2p in nature, which is just silly. What
> solution do you have for a LAN environment?

A LAG is p2p - it’s a bundle of links directly between two devices.

Are you now talking about a scenario where you have a switched (or similar) ethernet network with several L2 hops between your devices running BFD, and there being a LAG somewhere between those L2 hops?

--
Nathan Ward

_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: 802.3ad LAG between ASR 1002-X and Juniper MX204 [ In reply to ]
On 21/Jul/19 13:53, Nathan Ward wrote:

> A LAG is p2p - it’s a bundle of links directly between two devices.
>
> Are you now talking about a scenario where you have a switched (or similar) ethernet network with several L2 hops between your devices running BFD, and there being a LAG somewhere between those L2 hops?

I'm referring to a switched LAN which runs LAG's to connected devices.

Mark.
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp