Mailing List Archive

Hi Joe,

There are some documents on Junipers website describing principles and
including configurations, like this:
https://www.juniper.net/us/en/training/jnbooks/day-one/data-center-technologies/data-center-deployment-evpn-vxlan/

Some parameters can vary, so it depends on what your requirements are.

You can also try to use this scrips to generate configs for your
specific configuration:
https://github.com/JNPRAutomate/ansible-junos-evpn-vxlan/

Kind regards,
Andrey

Joe Freeman ????? 2019-09-16 15:52:
> Does anyone have a working example for EVPN configuration on the QFX
> 5200's
> that they'd be willing to share?
>
> I've got four 5200's split between two DC's with 2x100G links between
> each
> pair in a mesh. I'd like to run EVPN on them such that my network
> infrastructure between the sites is transparent to the servers.
> _______________________________________________
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

? 19 septembre 2019 16:25 -04, Andrey Kostin <ankost@podolsk.ru>:

> You can also try to use this scrips to generate configs for your
> specific configuration:
> https://github.com/JNPRAutomate/ansible-junos-evpn-vxlan/

I would stay away from most of the random examples available on Internet
(even ones from Juniper). For example, the above is using
ingress-node-replication in the "vlan" directive. This will bring havoc
in your network.

Start with the following documentation which is correct:
<https://www.juniper.net/documentation/en_US/release-independent/solutions/information-products/pathway-pages/sg-005-cloud-data-center.pdf>

Also, be sure to read the following page to know the limitations:
<https://www.juniper.net/documentation/en_US/junos/topics/concept/vxlan-constraints-qfx-series.html>

Notably, the QFX5200 is not able to route VXLANs.
--
Write clearly - don't sacrifice clarity for "efficiency".
- The Elements of Programming Style (Kernighan & Plauger)
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Hi,

I'm running VXLAN with ingress-node-replication in prod, can you
explain what you mean by havoc?

e.g.

show vlans

VLAN-NAME {

interface xe-0/0/2.2;

no-arp-suppression;

vxlan {

vni 561;

encapsulate-inner-vlan;

ingress-node-replication;

}

}

show interfaces xe-0/0/2

flexible-vlan-tagging;

mtu 9216;

encapsulation flexible-ethernet-services;

unit 2 {

encapsulation vlan-bridge;

vlan-id 2;

input-vlan-map pop;

output-vlan-map push;

}



Cheers

Liam

On Fri, 20 Sep 2019 at 08:49, Vincent Bernat <bernat@luffy.cx> wrote:

> ? 19 septembre 2019 16:25 -04, Andrey Kostin <ankost@podolsk.ru>:
>
> > You can also try to use this scrips to generate configs for your
> > specific configuration:
> > https://github.com/JNPRAutomate/ansible-junos-evpn-vxlan/
>
> I would stay away from most of the random examples available on Internet
> (even ones from Juniper). For example, the above is using
> ingress-node-replication in the "vlan" directive. This will bring havoc
> in your network.
>
> Start with the following documentation which is correct:
> <
> https://www.juniper.net/documentation/en_US/release-independent/solutions/information-products/pathway-pages/sg-005-cloud-data-center.pdf
> >
>
> Also, be sure to read the following page to know the limitations:
> <
> https://www.juniper.net/documentation/en_US/junos/topics/concept/vxlan-constraints-qfx-series.html
> >
>
> Notably, the QFX5200 is not able to route VXLANs.
> --
> Write clearly - don't sacrifice clarity for "efficiency".
> - The Elements of Programming Style (Kernighan & Plauger)
> _______________________________________________
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
>


--
Kind Regards


Liam Farr

Maxum Data
+64-9-950-5302
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

? 20 septembre 2019 11:55 +12, Liam Farr <liam@maxumdata.com>:

> I'm running VXLAN with ingress-node-replication in prod, can you
> explain what you mean by havoc?

When using EVPN, prefer using "set protocols evpn multicast-mode
ingress-replication". Using "set vlans XXX vxlan
ingress-node-replication" will send replicated packets to all VTEP,
including the ones not advertising the Type 3 route. See
<https://www.juniper.net/documentation/en_US/junos/topics/example/evpn-vxlan-collapsed-topology.html>:

> Retains the QFX10000 switch’s default setting of disabled for ingress
> node replication for EVPN-VXLAN. With this feature disabled, if a
> QFX10000 switch that functions as a VTEP receives a BUM packet
> intended, for example, for a physical server in a VLAN with the VNI of
> 1001, the VTEP replicates and sends the packet only to VTEPs on which
> the VNI of 1001 is configured. If this feature is enabled, the VTEP
> replicates and sends this packet to all VTEPs in its database,
> including those that do not have VNI 1001 configured. To prevent a
> VTEP from needlessly flooding BUM traffic throughout an EVPN-VXLAN
> overlay network, we strongly recommend that if not already disabled,
> you disable ingress node replication on each of the leaf devices by
> specifying the delete vlans vlan-name vxlan ingress-node-replication
> command.

In turn, this may exhaust the resources of the Broadcom
chipset (Trident2 or Trident2+) if you have a lot of VLANs and/or a lot
of VTEPs.
--
Talkers are no good doers.
-- William Shakespeare, "Henry VI"
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Hi Vincent,

Thank you for elaborating on this, I had the same question when read
your reply.
It may be not an issue for a small deployment but definitely should be
considered in terms of BCP.

Could you advise about various external connectivity options for
EVPN-VXLAN fabric? Let's say there are two spines that centrally route
VXLAN vnis and some leaves. Spines are CEs from core MPLS network
perspective. I understand that EVPN can be extended to the PE router and
L3-gateways run on them, but probably not right now. What is a proper
way to connect spines to PE router or pair of PE routers? I'm looking
into running EBGP from each spine to [each] PE router over routed P2P
interface. Are there possible flaws in this topology? Is direct
connection needed between spines in this case?

Kins regards,
Andrey


Vincent Bernat ????? 2019-09-20 02:25:
> ? 20 septembre 2019 11:55 +12, Liam Farr <liam@maxumdata.com>:
>
>> I'm running VXLAN with ingress-node-replication in prod, can you
>> explain what you mean by havoc?
>
> When using EVPN, prefer using "set protocols evpn multicast-mode
> ingress-replication". Using "set vlans XXX vxlan
> ingress-node-replication" will send replicated packets to all VTEP,
> including the ones not advertising the Type 3 route. See
> <https://www.juniper.net/documentation/en_US/junos/topics/example/evpn-vxlan-collapsed-topology.html>:
>
>> Retains the QFX10000 switch’s default setting of disabled for ingress
>> node replication for EVPN-VXLAN. With this feature disabled, if a
>> QFX10000 switch that functions as a VTEP receives a BUM packet
>> intended, for example, for a physical server in a VLAN with the VNI of
>> 1001, the VTEP replicates and sends the packet only to VTEPs on which
>> the VNI of 1001 is configured. If this feature is enabled, the VTEP
>> replicates and sends this packet to all VTEPs in its database,
>> including those that do not have VNI 1001 configured. To prevent a
>> VTEP from needlessly flooding BUM traffic throughout an EVPN-VXLAN
>> overlay network, we strongly recommend that if not already disabled,
>> you disable ingress node replication on each of the leaf devices by
>> specifying the delete vlans vlan-name vxlan ingress-node-replication
>> command.
>
> In turn, this may exhaust the resources of the Broadcom
> chipset (Trident2 or Trident2+) if you have a lot of VLANs and/or a lot
> of VTEPs.
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

? 20 septembre 2019 11:47 -04, Andrey Kostin <ankost@podolsk.ru>:

> Could you advise about various external connectivity options for
> EVPN-VXLAN fabric? Let's say there are two spines that centrally route
> VXLAN vnis and some leaves. Spines are CEs from core MPLS network
> perspective. I understand that EVPN can be extended to the PE router
> and L3-gateways run on them, but probably not right now. What is a
> proper way to connect spines to PE router or pair of PE routers? I'm
> looking into running EBGP from each spine to [each] PE router over
> routed P2P interface. Are there possible flaws in this topology? Is
> direct connection needed between spines in this case?

I am not familiar with MPLS. You need to use QFX10k for the spines as
the QFX5k are not able to route VXLAN outside (or not able to route at
all).
--
Avoid unnecessary branches.
- The Elements of Programming Style (Kernighan & Plauger)
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Thank you for reply.
I meant a slightly different thing. Currently my setup is in lab stage
with QFX5110 as spines and QFX5000 as leaves. I need to connect vlans
running in EVPN-VXLAN fabric to an aggregation router, ideally two of
them for redundancy. To have a redundant gateway for hosts sitting in
VNIs I need to run EVPN L3 gateway somewere. It can be done either on
aggregation routers or on QFX5110. Putting L3GW on routers means they
have to run EVPN as well and effectively become leaves for VXLAN fabric.
It may be a feasible solution in the future but for now we don't want to
put EVPN-VXLAN in prod network. So, the another option is to run L3
gateways on spines and somehow route them to agg routers. Possible
connectivity options between edge routers and spines could be:
- have individual P2P routed links Spine-RTR and run BGP session between
them. Balancing and redundancy in this case will be provided by BGP+ECMP
and also limited by their capabilities.
- have LACP to both Spines from each RTR and then L3 interface on each
spine, BGP from each spine to each RTR. Load balancing is provided by
BGP multipath+ECMP+LACP. In this case LACP bundle from spines POV is
switched. Direct connection between spines is necessary in this case.
ROuters in this topology play CE role for VXLAN fabric but connected to
spines instead of leaves.

Any recommendations or links to BCP are appreciated.

Kind regards,
Andrey

Vincent Bernat ????? 2019-09-21 01:34:
> ? 20 septembre 2019 11:47 -04, Andrey Kostin <ankost@podolsk.ru>:
>
>
> I am not familiar with MPLS. You need to use QFX10k for the spines as
> the QFX5k are not able to route VXLAN outside (or not able to route at
> all).

_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Hello,

The QFX5110 is unable to route between a VXLAN and a layer 3 interface.
There is a hack documented here:

<https://www.juniper.net/documentation/en_US/junos/topics/concept/evpn-vxlan-qfx5110-l2-vxlan-l3-logical.html>

Such a setup is quite fragile. Only the QFX10k is able to act as a L3
gateway for VXLAN and be connected to non-VXLAN stuff. QFX5110 is only
able to act as a L3 gateway when routing between VXLANs.
--
Watch out for off-by-one errors.
- The Elements of Programming Style (Kernighan & Plauger)

??????? Original Message ???????
From: Andrey Kostin <ankost@podolsk.ru>
Sent: 25 septembre 2019 11:37 -04
Subject: Re: [j-nsp] EVPN on QFX5200
To: Vincent Bernat
Cc: Liam Farr; juniper-nsp@puck.nether.net

> Thank you for reply.
> I meant a slightly different thing. Currently my setup is in lab stage
> with QFX5110 as spines and QFX5000 as leaves. I need to connect vlans
> running in EVPN-VXLAN fabric to an aggregation router, ideally two of
> them for redundancy. To have a redundant gateway for hosts sitting in
> VNIs I need to run EVPN L3 gateway somewere. It can be done either on
> aggregation routers or on QFX5110. Putting L3GW on routers means they
> have to run EVPN as well and effectively become leaves for VXLAN
> fabric. It may be a feasible solution in the future but for now we
> don't want to put EVPN-VXLAN in prod network. So, the another option
> is to run L3 gateways on spines and somehow route them to agg routers.
> Possible connectivity options between edge routers and spines could
> be:
> - have individual P2P routed links Spine-RTR and run BGP session
> between them. Balancing and redundancy in this case will be provided
> by BGP+ECMP and also limited by their capabilities.
> - have LACP to both Spines from each RTR and then L3 interface on each
> spine, BGP from each spine to each RTR. Load balancing is provided by
> BGP multipath+ECMP+LACP. In this case LACP bundle from spines POV is
> switched. Direct connection between spines is necessary in this case.
> ROuters in this topology play CE role for VXLAN fabric but connected
> to spines instead of leaves.
>
> Any recommendations or links to BCP are appreciated.
>
> Kind regards,
> Andrey
>
> Vincent Bernat ????? 2019-09-21 01:34:
>> ? 20 septembre 2019 11:47 -04, Andrey Kostin <ankost@podolsk.ru>:
>>
>>
>> I am not familiar with MPLS. You need to use QFX10k for the spines as
>> the QFX5k are not able to route VXLAN outside (or not able to route at
>> all).
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Hi Vincent,

Thank you for a good advice. I saw this page before, but now reviewed
it. According to it only second opton could qualify and I'm going to
test it. Anyway, for final solution QFX10K will be in consideration.

Kind regards,
Andrey

Vincent Bernat ????? 2019-09-26 02:49:
> Hello,
>
> The QFX5110 is unable to route between a VXLAN and a layer 3 interface.
> There is a hack documented here:
>
>
> <https://www.juniper.net/documentation/en_US/junos/topics/concept/evpn-vxlan-qfx5110-l2-vxlan-l3-logical.html>
>
> Such a setup is quite fragile. Only the QFX10k is able to act as a L3
> gateway for VXLAN and be connected to non-VXLAN stuff. QFX5110 is only
> able to act as a L3 gateway when routing between VXLANs.
> --
> Watch out for off-by-one errors.
> - The Elements of Programming Style (Kernighan & Plauger)
>
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp