Mailing List Archive

MX VRRP on VXLAN enviroment
I have a scenario here where I use EVPN-VXLAN with qfx5120 switches
and until then I was using the gateways on the switches, but as the
switch does not have the possibility to use any kind of firewall on
the irb interfaces, I had the idea to migrate the networks to two
routers MX80.
But I caught a problem with these routers, when using VRRP over VXLAN.
I configured the two MX80 routers with VRRP in IPv4 and IPv6,
sometimes IPv4 dies and the virtual IP stops responding, generating
timeout in network accesses.
Apparently it seems that the mac address of the virtual IP and the
table of mac's of the VXLAN are lost, causing the problem.
Does anyone happen to have a scenario like this and faced this problem?
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: MX VRRP on VXLAN enviroment [ In reply to ]
Hi,

> On 20/07/2021, at 12:23 AM, Cristian Cardoso via juniper-nsp <juniper-nsp@puck.nether.net> wrote:
>
> I have a scenario here where I use EVPN-VXLAN with qfx5120 switches
> and until then I was using the gateways on the switches, but as the
> switch does not have the possibility to use any kind of firewall on
> the irb interfaces, I had the idea to migrate the networks to two
> routers MX80.
> But I caught a problem with these routers, when using VRRP over VXLAN.
> I configured the two MX80 routers with VRRP in IPv4 and IPv6,
> sometimes IPv4 dies and the virtual IP stops responding, generating
> timeout in network accesses.
> Apparently it seems that the mac address of the virtual IP and the
> table of mac's of the VXLAN are lost, causing the problem.
> Does anyone happen to have a scenario like this and faced this problem?

I’m not sure exactly what is causing your problem, though there are certainly a lot of curly edge cases in EVPN where this sort of thing can happen.
Do you have a specific need to run VRRP?

One of the benefits of EVPN is “virtual-gateway-address” which advertises the gateway address from all IRBs with that configured - and they are all “active” rather than VRRP’s active/standby.
If you don’t have a need for VRRP specifically, this might be a better solution for you.

Incidentally, by default it uses the VRRP group 1 MAC, but it is not running VRRP at all. You can configure it to use a different MAC, if required (i.e. if you have other devices on that broadcast domain running VRRP group 1 perhaps).

--
Nathan Ward

_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: MX VRRP on VXLAN enviroment [ In reply to ]
I had several problems using the virtual gateway via EVPN on the
switches, even the function of being switches and not routers. In my
scenario it is important to have a minimal firewall on the interfaces,
and in the models I have here, this is not possible.
My idea of using VRRP on the routers above the VXLAN switches, is just
to let the switches do only the VXLAN and thus remove the functions of
the layer 3 gateway from the qfx.
Using a more "simple" Layer 3 scenario for routing.

Em seg., 19 de jul. de 2021 às 09:41, Nathan Ward
<juniper-nsp@daork.net> escreveu:
>
> Hi,
>
> > On 20/07/2021, at 12:23 AM, Cristian Cardoso via juniper-nsp <juniper-nsp@puck.nether.net> wrote:
> >
> > I have a scenario here where I use EVPN-VXLAN with qfx5120 switches
> > and until then I was using the gateways on the switches, but as the
> > switch does not have the possibility to use any kind of firewall on
> > the irb interfaces, I had the idea to migrate the networks to two
> > routers MX80.
> > But I caught a problem with these routers, when using VRRP over VXLAN.
> > I configured the two MX80 routers with VRRP in IPv4 and IPv6,
> > sometimes IPv4 dies and the virtual IP stops responding, generating
> > timeout in network accesses.
> > Apparently it seems that the mac address of the virtual IP and the
> > table of mac's of the VXLAN are lost, causing the problem.
> > Does anyone happen to have a scenario like this and faced this problem?
>
> I’m not sure exactly what is causing your problem, though there are certainly a lot of curly edge cases in EVPN where this sort of thing can happen.
> Do you have a specific need to run VRRP?
>
> One of the benefits of EVPN is “virtual-gateway-address” which advertises the gateway address from all IRBs with that configured - and they are all “active” rather than VRRP’s active/standby.
> If you don’t have a need for VRRP specifically, this might be a better solution for you.
>
> Incidentally, by default it uses the VRRP group 1 MAC, but it is not running VRRP at all. You can configure it to use a different MAC, if required (i.e. if you have other devices on that broadcast domain running VRRP group 1 perhaps).
>
> --
> Nathan Ward
>
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: MX VRRP on VXLAN enviroment [ In reply to ]
In theory when the VRRP falls over, the promoted MX80 should send a
frame with the VRRP virtual MAC as source, causing it to be learnt on a
different switch.  That switch should then send an EVPN type-2 UPDATE
for the MAC and the remaining switches will update their local MAC
tables with VXLAN-encap next-hop.  Can you verify any of that?  Like
does the MAC get learnt from the other MX80 properly?  Does the EVPN
update get generated?


On 19/07/2021 13:56, Cristian Cardoso via juniper-nsp wrote:
> I had several problems using the virtual gateway via EVPN on the
> switches, even the function of being switches and not routers. In my
> scenario it is important to have a minimal firewall on the interfaces,
> and in the models I have here, this is not possible.
> My idea of using VRRP on the routers above the VXLAN switches, is just
> to let the switches do only the VXLAN and thus remove the functions of
> the layer 3 gateway from the qfx.
> Using a more "simple" Layer 3 scenario for routing.
>
> Em seg., 19 de jul. de 2021 às 09:41, Nathan Ward
> <juniper-nsp@daork.net> escreveu:
>> Hi,
>>
>>> On 20/07/2021, at 12:23 AM, Cristian Cardoso via juniper-nsp <juniper-nsp@puck.nether.net> wrote:
>>>
>>> I have a scenario here where I use EVPN-VXLAN with qfx5120 switches
>>> and until then I was using the gateways on the switches, but as the
>>> switch does not have the possibility to use any kind of firewall on
>>> the irb interfaces, I had the idea to migrate the networks to two
>>> routers MX80.
>>> But I caught a problem with these routers, when using VRRP over VXLAN.
>>> I configured the two MX80 routers with VRRP in IPv4 and IPv6,
>>> sometimes IPv4 dies and the virtual IP stops responding, generating
>>> timeout in network accesses.
>>> Apparently it seems that the mac address of the virtual IP and the
>>> table of mac's of the VXLAN are lost, causing the problem.
>>> Does anyone happen to have a scenario like this and faced this problem?
>> I’m not sure exactly what is causing your problem, though there are certainly a lot of curly edge cases in EVPN where this sort of thing can happen.
>> Do you have a specific need to run VRRP?
>>
>> One of the benefits of EVPN is “virtual-gateway-address” which advertises the gateway address from all IRBs with that configured - and they are all “active” rather than VRRP’s active/standby.
>> If you don’t have a need for VRRP specifically, this might be a better solution for you.
>>
>> Incidentally, by default it uses the VRRP group 1 MAC, but it is not running VRRP at all. You can configure it to use a different MAC, if required (i.e. if you have other devices on that broadcast domain running VRRP group 1 perhaps).
>>
>> --
>> Nathan Ward
>>
> _______________________________________________
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: MX VRRP on VXLAN enviroment [ In reply to ]
Take a look on this KB
https://kb.juniper.net/InfoCenter/index?page=content&id=KB35676&cat=EX9208&actp=LIST

The default duplicate-mac-detection settings are far to high

Nitzan

On Mon, Jul 19, 2021 at 4:50 PM Cathal Mooney via juniper-nsp <
juniper-nsp@puck.nether.net> wrote:

> In theory when the VRRP falls over, the promoted MX80 should send a
> frame with the VRRP virtual MAC as source, causing it to be learnt on a
> different switch. That switch should then send an EVPN type-2 UPDATE
> for the MAC and the remaining switches will update their local MAC
> tables with VXLAN-encap next-hop. Can you verify any of that? Like
> does the MAC get learnt from the other MX80 properly? Does the EVPN
> update get generated?
>
>
> On 19/07/2021 13:56, Cristian Cardoso via juniper-nsp wrote:
> > I had several problems using the virtual gateway via EVPN on the
> > switches, even the function of being switches and not routers. In my
> > scenario it is important to have a minimal firewall on the interfaces,
> > and in the models I have here, this is not possible.
> > My idea of using VRRP on the routers above the VXLAN switches, is just
> > to let the switches do only the VXLAN and thus remove the functions of
> > the layer 3 gateway from the qfx.
> > Using a more "simple" Layer 3 scenario for routing.
> >
> > Em seg., 19 de jul. de 2021 às 09:41, Nathan Ward
> > <juniper-nsp@daork.net> escreveu:
> >> Hi,
> >>
> >>> On 20/07/2021, at 12:23 AM, Cristian Cardoso via juniper-nsp <
> juniper-nsp@puck.nether.net> wrote:
> >>>
> >>> I have a scenario here where I use EVPN-VXLAN with qfx5120 switches
> >>> and until then I was using the gateways on the switches, but as the
> >>> switch does not have the possibility to use any kind of firewall on
> >>> the irb interfaces, I had the idea to migrate the networks to two
> >>> routers MX80.
> >>> But I caught a problem with these routers, when using VRRP over VXLAN.
> >>> I configured the two MX80 routers with VRRP in IPv4 and IPv6,
> >>> sometimes IPv4 dies and the virtual IP stops responding, generating
> >>> timeout in network accesses.
> >>> Apparently it seems that the mac address of the virtual IP and the
> >>> table of mac's of the VXLAN are lost, causing the problem.
> >>> Does anyone happen to have a scenario like this and faced this problem?
> >> I’m not sure exactly what is causing your problem, though there are
> certainly a lot of curly edge cases in EVPN where this sort of thing can
> happen.
> >> Do you have a specific need to run VRRP?
> >>
> >> One of the benefits of EVPN is “virtual-gateway-address” which
> advertises the gateway address from all IRBs with that configured - and
> they are all “active” rather than VRRP’s active/standby.
> >> If you don’t have a need for VRRP specifically, this might be a better
> solution for you.
> >>
> >> Incidentally, by default it uses the VRRP group 1 MAC, but it is not
> running VRRP at all. You can configure it to use a different MAC, if
> required (i.e. if you have other devices on that broadcast domain running
> VRRP group 1 perhaps).
> >>
> >> --
> >> Nathan Ward
> >>
> > _______________________________________________
> > juniper-nsp mailing list juniper-nsp@puck.nether.net
> > https://puck.nether.net/mailman/listinfo/juniper-nsp
> _______________________________________________
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
>
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: MX VRRP on VXLAN enviroment [ In reply to ]
Hi
Thanks for the tip, I'll set it up here.

Em seg., 19 de jul. de 2021 às 14:36, Nitzan Tzelniker via juniper-nsp
<juniper-nsp@puck.nether.net> escreveu:
>
> Take a look on this KB
> https://kb.juniper.net/InfoCenter/index?page=content&id=KB35676&cat=EX9208&actp=LIST
>
> The default duplicate-mac-detection settings are far to high
>
> Nitzan
>
> On Mon, Jul 19, 2021 at 4:50 PM Cathal Mooney via juniper-nsp <
> juniper-nsp@puck.nether.net> wrote:
>
> > In theory when the VRRP falls over, the promoted MX80 should send a
> > frame with the VRRP virtual MAC as source, causing it to be learnt on a
> > different switch. That switch should then send an EVPN type-2 UPDATE
> > for the MAC and the remaining switches will update their local MAC
> > tables with VXLAN-encap next-hop. Can you verify any of that? Like
> > does the MAC get learnt from the other MX80 properly? Does the EVPN
> > update get generated?
> >
> >
> > On 19/07/2021 13:56, Cristian Cardoso via juniper-nsp wrote:
> > > I had several problems using the virtual gateway via EVPN on the
> > > switches, even the function of being switches and not routers. In my
> > > scenario it is important to have a minimal firewall on the interfaces,
> > > and in the models I have here, this is not possible.
> > > My idea of using VRRP on the routers above the VXLAN switches, is just
> > > to let the switches do only the VXLAN and thus remove the functions of
> > > the layer 3 gateway from the qfx.
> > > Using a more "simple" Layer 3 scenario for routing.
> > >
> > > Em seg., 19 de jul. de 2021 às 09:41, Nathan Ward
> > > <juniper-nsp@daork.net> escreveu:
> > >> Hi,
> > >>
> > >>> On 20/07/2021, at 12:23 AM, Cristian Cardoso via juniper-nsp <
> > juniper-nsp@puck.nether.net> wrote:
> > >>>
> > >>> I have a scenario here where I use EVPN-VXLAN with qfx5120 switches
> > >>> and until then I was using the gateways on the switches, but as the
> > >>> switch does not have the possibility to use any kind of firewall on
> > >>> the irb interfaces, I had the idea to migrate the networks to two
> > >>> routers MX80.
> > >>> But I caught a problem with these routers, when using VRRP over VXLAN.
> > >>> I configured the two MX80 routers with VRRP in IPv4 and IPv6,
> > >>> sometimes IPv4 dies and the virtual IP stops responding, generating
> > >>> timeout in network accesses.
> > >>> Apparently it seems that the mac address of the virtual IP and the
> > >>> table of mac's of the VXLAN are lost, causing the problem.
> > >>> Does anyone happen to have a scenario like this and faced this problem?
> > >> I’m not sure exactly what is causing your problem, though there are
> > certainly a lot of curly edge cases in EVPN where this sort of thing can
> > happen.
> > >> Do you have a specific need to run VRRP?
> > >>
> > >> One of the benefits of EVPN is “virtual-gateway-address” which
> > advertises the gateway address from all IRBs with that configured - and
> > they are all “active” rather than VRRP’s active/standby.
> > >> If you don’t have a need for VRRP specifically, this might be a better
> > solution for you.
> > >>
> > >> Incidentally, by default it uses the VRRP group 1 MAC, but it is not
> > running VRRP at all. You can configure it to use a different MAC, if
> > required (i.e. if you have other devices on that broadcast domain running
> > VRRP group 1 perhaps).
> > >>
> > >> --
> > >> Nathan Ward
> > >>
> > > _______________________________________________
> > > juniper-nsp mailing list juniper-nsp@puck.nether.net
> > > https://puck.nether.net/mailman/listinfo/juniper-nsp
> > _______________________________________________
> > juniper-nsp mailing list juniper-nsp@puck.nether.net
> > https://puck.nether.net/mailman/listinfo/juniper-nsp
> >
> _______________________________________________
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: MX VRRP on VXLAN enviroment [ In reply to ]
Cristian Cardoso via juniper-nsp ????? 2021-07-19 14:15:
> Hi
> Thanks for the tip, I'll set it up here.
>

Are you trying to setup MX80 as end-host, without including it in EVPN?
If so, then you can extend EVPN to MX80 and run virtual-gateway from it.
No need for VRRP in this case.

Kind regards,
Andrey
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: MX VRRP on VXLAN enviroment [ In reply to ]
Hi Andrey

My idea was to keep only with VRRP to be something simpler for the
team to manage.
I set the config that Nitzan Tzelniker suggested and so far I haven't
seen more occurrences of arp suppression problems in VXLAN.
Of course, the last time this happened it took about 2 weeks to end up
happening.
I did several reboots between one MX80 and another and so far there
was no problem with the gateway dying for the network that is in VRRP.

Em sex., 23 de jul. de 2021 às 14:34, Andrey Kostin
<ankost@podolsk.ru> escreveu:
>
> Cristian Cardoso via juniper-nsp ????? 2021-07-19 14:15:
> > Hi
> > Thanks for the tip, I'll set it up here.
> >
>
> Are you trying to setup MX80 as end-host, without including it in EVPN?
> If so, then you can extend EVPN to MX80 and run virtual-gateway from it.
> No need for VRRP in this case.
>
> Kind regards,
> Andrey
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp