Mailing List Archive

EVPN/VXLAN on ASR9001 - BGP announcements not working
Hi,

I'm trying to make EVPN via VXLAN encapsulation work between two ASR9001
(with the goal of eventually making it work between ASR9001 and Arista
boxes, but right now I'm failing ASR9001 <-> ASR9001 already).

As warmup, I tried EVPN with MPLS encapsulation, and that is much better
documented and very straightforward to set up (basically, all the defaults
work - configure neighbours that have labels, ensure RDs match, attach
EVI <n> to bridge-domain <n>, done)... so, BGP works, IGP works,
bridge-group setup works.


With VXLAN, it "should" be similarily straightforward, but I'm overlooking
something. Namely, both sides create their l2vpn evpn BGP routes, and
the neighbours talk to each other, but neither side is willing to export
anything but Type 4 routes (ESI).

Tried with and without ESI config, without explicit RT config, with
lots of different RT combinations... no avail, routers are not announcing.


The two boxes are 195.30.3.251 and 195.30.3.252 (you'll see that in the
RDs below, and redacting BGP output never helps debugging).

This is what I have in BGP right now on 195.30.3.252:

RP/0/RSP0/CPU0:M52#sh bgp l2vpn evpn
...
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 195.30.3.251:0
*>i[4][0040.0000.0000.0034.3502][32][195.30.3.251]/128
195.30.3.251 100 0 i
Route Distinguisher: 195.30.3.252:0 (default for vrf ES:GLOBAL)
*> [1][195.30.3.252:1][0040.0000.0000.0052.0002][4294967295]/184
0.0.0.0 0 i
*>i[4][0040.0000.0000.0034.3502][32][195.30.3.251]/128
195.30.3.251 100 0 i
*> [4][0040.0000.0000.0052.0002][32][195.30.3.252]/128
0.0.0.0 0 i
Route Distinguisher: 195.30.3.252:2799 (default for vrf v2799)
*> [1][0040.0000.0000.0052.0002][0]/120
0.0.0.0 0 i
*> [2][0][48][3cfd.febd.7835][0]/104
0.0.0.0 0 i
*> [2][0][48][3cfd.febd.7835][32][10.27.99.2]/136
0.0.0.0 0 i
*> [2][0][48][a80c.0d56.503f][0]/104
0.0.0.0 0 i
*> [3][0][32][195.30.3.252]/80
0.0.0.0 0 i

P/0/RSP0/CPU0:M52#sh bgp l2vpn evpn su
Neighbor Spk AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down St/PfxRcd
195.30.3.251 0 5539 21 21 480 0 0 00:15:28 1

RP/0/RSP0/CPU0:M52#sh bgp l2vpn evpn neigh 195.30.3.251 advertised-routes
Sun Mar 29 11:37:43.281 MEDST
Network Next Hop From AS Path
Route Distinguisher: 195.30.3.252:0
[4][0040.0000.0000.0052.0002][32][195.30.3.252]/128
195.30.3.252 Local i

so both sides symmetrically announce the Type 4 ESI DR election route
(which isn't really relevant as all ESIs are single-homed right now),
but refuse to announce anything else.


The neighbour config is trivial:

interface Loopback30
description VXLAN/EVPN endpoint
ipv4 address 195.30.3.252 255.255.255.255

router bgp 5539
neighbor 195.30.3.251
remote-as 5539
description m34/evpn-vxlan-test
update-source Loopback30
address-family l2vpn evpn
encapsulation-type vxlan
!
!
!

(no matter of what policy statements or import/export statements I
added there had any positive effect)


So is the EVPN config:

RP/0/RSP0/CPU0:M52#sh run evpn
Sun Mar 29 11:40:23.694 MEDST
evpn
evi 2799
bgp
!
advertise-mac
bvi-mac
!
!
interface TenGigE0/0/2/2
ethernet-segment
identifier type 0 40.00.00.00.00.00.52.00.02
!
!
interface Bundle-Ether102
ethernet-segment
identifier type 0 40.00.00.00.00.00.52.00.01
!
!
source interface Loopback30
!


no explicit configured RDs or RTs (I did try about everything...), but
the default values seem to make sense, for example for the Type 3:

BGP routing table entry for [3][0][32][195.30.3.252]/80, Route Distinguisher: 195.30.3.252:2799
Versions:
Process bRIB/RIB SendTblVer
Speaker 480 480
Last Modified: Mar 29 11:23:56.823 for 00:18:44
Paths: (1 available, best #1)
Not advertised to any peer
Path #1: Received by speaker 0
Not advertised to any peer
Local
0.0.0.0 from 0.0.0.0 (193.149.44.20)
Origin IGP, localpref 100, valid, redistributed, best, group-best, import-candidate
Received Path ID 0, Local Path ID 1, version 480
Extended community: RT:5539:2799
PMSI: flags 0x00, type 6, label 24022, ID 0xc31e03fc


"Not advertised to any peer" << and *this*...


For reference, here's the bridge group and the NVE config, but those
are also trivial enough so I assume the issue is "in BGP", not somewhere
there:

l2vpn
bridge group vlandb
bridge-domain v2799
interface TenGigE0/0/2/2.2799
!
routed interface BVI2799
!
evi 2799
!
member vni 102799
!
!

Interface nve1
member vni 102799
host-reachability protocol bgp
!
overlay-encapsulation vxlan
source-interface Loopback30
!



Oh, finally, this is on 32bit IOS XR 6.5.3.sp2 - in case this matters...


So... any ideas or suggestions what to try next? Or "just give up on
this and go with EVPN/MPLS instead" (which would not help me with the
larger goal "connect a remote location that only has Arista gear")?

thanks,

gert

--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de
Re: EVPN/VXLAN on ASR9001 - now, L2FIB/VXLAN weirdness [ In reply to ]
Hi,

On Sun, Mar 29, 2020 at 11:52:03AM +0200, Gert Doering wrote:
> I'm trying to make EVPN via VXLAN encapsulation work between two ASR9001
> (with the goal of eventually making it work between ASR9001 and Arista
> boxes, but right now I'm failing ASR9001 <-> ASR9001 already).

So, spent some more hours on this, ignoring non-existant documentation,
and found a different way to configure EVPN-with-VXLAN.

Bridge-group like this:

bridge group vlandb
bridge-domain v2799
interface TenGigE0/0/2/2.2799
!
routed interface BVI2799
!
vni 102799

(so no "member vni ..." and no "evi ..." either)

EVPN like this:

evpn
vni 102799
bgp
rd 195.30.3.252:2799
!
advertise-mac
bvi-mac
!
!


VTEP like this:

interface nve1
member vni 102799
host-reachability protocol bgp
!
source-interface Loopback30
ingress-replication protocol bgp <<<< new knob, docs lacking
!


and BGP neighbour like this (without any frills):

router bgp 5539
neighbor 195.30.3.251
remote-as 5539
description m34/evpn-vxlan-test
update-source Loopback30
address-family l2vpn evpn
encapsulation-type vxlan
soft-reconfiguration inbound always
!
!
!


with that, I get proper BGP signalling, and I see my EVPN VLAN and
the associated MAC addresses in "show evpn evi", "show evpn mac":

RP/0/RSP0/CPU0:M52#show evpn evi
VPN-ID Encap Bridge Domain Type
---------- ------ ---------------------------- -------------------
65535 MPLS ES:GLOBAL Invalid
102799 VXLAN v2799 EVPN

RP/0/RSP0/CPU0:M52#show evpn evi inclusive-multicast
Sun Mar 29 17:30:51.532 MEDST

VPN-ID Encap EtherTag Originating IP
---------- ------ ---------- ----------------------------------------
102799 VXLAN 0 195.30.3.249
102799 VXLAN 0 195.30.3.251
102799 VXLAN 0 195.30.3.252

RP/0/RSP0/CPU0:M52#show evpn evi mac
VPN-ID Encap MAC address IP address Nexthop Label
---------- ------ -------------- ---------------------------------------- --------------------------------------- --------
65535 N/A a80c.0d56.503d :: Local 0
102799 VXLAN 0050.569c.338e :: 195.30.3.251 102799
102799 VXLAN 0050.569c.338e 10.27.99.10 195.30.3.251 102799
102799 VXLAN 3cfd.febd.7835 :: TenGigE0/0/2/2.2799 102799
102799 VXLAN 3cfd.febd.7835 10.27.99.2 TenGigE0/0/2/2.2799 102799

(... and more, everything I'd *expect* to be there)


and the VXLAN NVE VNI is "up":

RP/0/RSP0/CPU0:M52#sh nve vni
Sun Mar 29 17:30:27.100 MEDST
Interface VNI MCAST VNI State Mode
nve1 102799 N/A Up L2 Control


... so, generally speaking, this should be working now... alas, it
doesn't.


RP/0/RSP0/CPU0:M52#show l2vpn forw bridge-domain vlandb:v2799 mac loc 0/0/CPU0
Mac Address Type Learned from/Filtered on LC learned Resync Age/Last Change Mapped to
-------------- ------- --------------------------- ---------- ---------------------- --------------
3cfd.febd.7835 dynamic Te0/0/2/2.2799 N/A 29 Mar 17:31:04 N/A
9803.9b97.8f36 dynamic BD id: 0(nve1) N/A 29 Mar 17:06:34 195.30.3.249
a80c.0d56.503f routed BD id: 0 N/A N/A N/A


MAC addresses get not properly mapped to the NVE1 *unless* they are seen
from there first - so, the address above is something behind an Arista,
which happily does everything in a straightforward way. Arista sends
packet, NVE1 decapsulates, and does mac-learning-from-VXLAN. No other
EVPN MAC addresses show up in the L2 forwarding table...

Said address shows up in "show evpn evi mac" in a "funky" way too:

RP/0/RSP0/CPU0:M52#show evpn evi mac
102799 VXLAN 9803.9b97.8f36 :: Unknown(No Forwarder for XID) 102799
102799 VXLAN 9803.9b97.8f36 :: 195.30.3.249 102799
102799 VXLAN 9803.9b97.8f36 10.27.99.202 Unknown(No Forwarder for XID) 102799



So it seems that some sort of disconnect still happens between L2 FIB
and EVPN MAC table.

I'm out of ideas how to debug this, or what further knobs to twiddle...


So - what I'd appreciate most, right now, is a working sample config
for "ASR9000 to ASR9000, basic L2, with EVPN and VXLAN transport".

No fancy L2/L3 stuff, no fancy route-target importing/exporting, no
stitching MPLS<->VXLAN, just barebones and *working*...

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de
Re: EVPN/VXLAN on ASR9001 - MACs not installed [ In reply to ]
Hi,

On Sun, Mar 29, 2020 at 11:52:03AM +0200, Gert Doering wrote:
> I'm trying to make EVPN via VXLAN encapsulation work between two ASR9001
> (with the goal of eventually making it work between ASR9001 and Arista
> boxes, but right now I'm failing ASR9001 <-> ASR9001 already).

Just to keep you amused... it's the simple things that make a difference.

After experimenting around for many hours, with and without explicit
RDs and RTs, I'm back to a very basic config with "everything on auto",
with one significant difference:

interface nve1
member vni 102799
host-reachability protocol bgp
!
overlay-encapsulation vxlan
source-interface Loopback30
ingress-replication protocol bgp <<<< this!
!

so now I see BGP peers talking and exchanging type 1, 2, 3, 4 routes
(great!), and "show evpn evi" and "show evpn evi mac" confirms "yes,
we have MAC addresses and VXLAN transport, with the right VNI"

RP/0/RSP0/CPU0:M52#sh evpn evi mac
Tue Mar 31 13:24:28.939 MEDST

VPN-ID Encap MAC address IP address Nexthop Label
---------- ------ -------------- ---------------------------------------- --------------------------------------- --------
2799 MPLS 3cfd.febd.7835 :: TenGigE0/0/2/2.2799 24022
2799 MPLS 3cfd.febd.7835 10.27.99.2 TenGigE0/0/2/2.2799 24022
2799 MPLS a80c.0d56.503f :: BVI2799 24022
65535 N/A a80c.0d56.503d :: Local 0
102799 VXLAN 0050.569c.338e :: 195.30.3.251 102799
102799 VXLAN 0050.569c.338e 10.27.99.10 195.30.3.251 102799
102799 VXLAN 00c1.6465.920f :: 195.30.3.251 102799
102799 VXLAN 9803.9b97.8f36 :: 195.30.3.249 102799


... but now the next major puzzlement is hitting me - only MAC addresses
that point to singlehomed ES (ESI 0) get installed:

RP/0/RSP0/CPU0:M52#show l2vpn forw bridge-domain vlandb:v2799 mac loc 0/0/CPU0
..
Mac Address Type Learned from/Filtered on LC learned Resync Age/Last Change Mapped to
-------------- ------- --------------------------- ---------- ---------------------- --------------
3cfd.febd.7835 dynamic Te0/0/2/2.2799 N/A 31 Mar 12:33:28 N/A
9803.9b97.8f36 EVPN BD id: 0(nve1) N/A N/A 195.30.3.249
a80c.0d56.503f routed BD id: 0 N/A N/A N/A

though the other addresses *should* be fine, according to "show evpn":

RP/0/RSP0/CPU0:M52#sh evpn evi mac 00c1.6465.920f det
Tue Mar 31 13:26:11.520 MEDST

VPN-ID Encap MAC address IP address Nexthop Label
---------- ------ -------------- ---------------------------------------- --------------------------------------- --------
102799 VXLAN 00c1.6465.920f :: 195.30.3.251 102799
Ethernet Tag : 0
Multi-paths Resolved : True
Multi-paths Internal label : 24031
Local Static : No
Remote Static : Yes
Local Ethernet Segment : N/A
Remote Ethernet Segment : 0034.0000.0000.0000.00ff
Local Sequence Number : N/A
Remote Sequence Number : 0
Local Encapsulation : N/A
Remote Encapsulation : VXLAN


I see "Multi-paths Resolved : True" as indication that it knows which
router is forwarder for 0034.0000.0000.0000.00ff and the route should
be eligible for installation.


Those hosts that have MACs that are in the L2FIB can talk to each other,
but only if I setup static ARP entries - flooding (broadcast) from
"local attachment circuit" to "vtep" still does not work.

So, next question :-)

- should I be seeing peers sending IMET routes in "show nve peers"
(output is empty)

- how to figure out why it's not flooding?

(Type 3 routes are there for :102799 and look reasonable)

- how to figure out why it's not installing non-0 EVI routes?

documentation is out there, but there's way too many knobs... :-/

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de
Re: EVPN/VXLAN on ASR9001 - BGP announcements not working [ In reply to ]
Hi,

remember me...? Fighting with this "simple task" since quite a few weeks...

On Sun, Mar 29, 2020 at 11:52:03AM +0200, Gert Doering wrote:
> I'm trying to make EVPN via VXLAN encapsulation work between two ASR9001
> (with the goal of eventually making it work between ASR9001 and Arista
> boxes, but right now I'm failing ASR9001 <-> ASR9001 already).

... so, whatever I did, it wouldn't work. Unicast packets actually do
work (provided the B end is not using ESIs, in that case I can see them
in the "show evpn evi mac" table, but never in the hardware table), but
most prominently, flooding never worked in the

local AC --> VXLAN peers

direction (while everything in BGP and EVPN and etc. was displaying the
expected results).

So, long story short, TAC time. XR TAC engineer (very competent) did
not bother much with looking at my RDs and ESIs and what not, but looked
at NPU packet drops instead - and lo and behold, packet drops in the
"RSV_DROP_IPM4_ING_RTE_DROP" category, which could be identified as
"yes, these are the ARP packets that are not flooded".


After some more research, this is what came back today...

"... Typhoon supports VXLAN EVPN features that were introduced up to
6.2.2. Features that were introduced after 6.3.1 are not supported.
Ingress-replication bgp is not supported in Typhoon LCs"

Which is slightly annoying, given that this seems to be not documented
anywhere AND that the ASR9001 is still not EOLed (as far as my google
fu can find)...

But anyway. In case someone stumbles across this thread in the archives -
it's not implemented and not expected to work.

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de
Re: EVPN/VXLAN on ASR9001 - BGP announcements not working [ In reply to ]
Hi,

> After some more research, this is what came back today...
>
> "... Typhoon supports VXLAN EVPN features that were introduced up to
> 6.2.2. Features that were introduced after 6.3.1 are not supported.
> Ingress-replication bgp is not supported in Typhoon LCs"
>
> Which is slightly annoying, given that this seems to be not documented
> anywhere AND that the ASR9001 is still not EOLed (as far as my google
> fu can find)...
>
> But anyway. In case someone stumbles across this thread in the archives -
> it's not implemented and not expected to work.

Oh, the joy of ASR9k… I'm so glad we have chosen another platform.

Cheers,
Sander
Re: EVPN/VXLAN on ASR9001 - BGP announcements not working [ In reply to ]
Gert Doering wrote on 29/04/2020 16:21:
> But anyway. In case someone stumbles across this thread in the archives -
> it's not implemented and not expected to work.

now that this is documented, it's a feature, right?

Nick
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: EVPN/VXLAN on ASR9001 - BGP announcements not working [ In reply to ]
On Wed, 29 Apr 2020 at 18:36, Sander Steffann <sander@steffann.nl> wrote:

> Oh, the joy of ASR9k… I'm so glad we have chosen another platform.

I think this is a bit unfair, of course vendors drop support to older
platforms. And this whole problem with tomahawk and lightspeed
transition isn't exactly new news, this has been known for 5years or
so. It's a balance between being an early adopter and being supported
for many years.

I think for Gert committing on such an old platform was matter of
density, no need for 9901 density, and I think this is where vendors
need to wake up. Less dense and lower speed interface doesn't mean we
can get away with TH1 or such ASIC, we still need full-blown NPU,
large FIB, large RIB, large buffers. Vendors should release 1GE, 10GE,
100GE, 400GE optimised front-plates with latest gen NPU when ever new
NPU happens, so what if you have 1% of NPU capacity in the
front-plate, that's good problem to have, you can maybe cost optimise
bit, by accepting from fab NPU which has lot of cores broken, or book
less off-chip memory for delay buffer (less front-plate, less off-chip
memory).

Just because the new NPU can do 40x400GE, doesn't mean we don't still
have applications for 1GE optimised devices with the same features and
functions as the latest and greatest chip.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: EVPN/VXLAN on ASR9001 - BGP announcements not working [ In reply to ]
Hi,

On Wed, Apr 29, 2020 at 05:11:46PM +0100, Nick Hilliard wrote:
> Gert Doering wrote on 29/04/2020 16:21:
> > But anyway. In case someone stumbles across this thread in the archives -
> > it's not implemented and not expected to work.
>
> now that this is documented, it's a feature, right?

This is all just to save power consumption in the NPU!

gert

--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de
Re: EVPN/VXLAN on ASR9001 - BGP announcements not working [ In reply to ]
Hi,

>> Oh, the joy of ASR9k… I'm so glad we have chosen another platform.
>
> I think this is a bit unfair, of course vendors drop support to older
> platforms.

Oh, I'm not commenting on something not being supported on older hardware. That's totally understandable. What does annoy me is the lack of documentation, the CLI accepting commands that aren't implemented without warning and stuff like that. That combined with the whole mess with "your PSU and fan tray are not compatible with the 64-bit OS a few years ago" makes me avoid the platform.

Cheers,
Sander
Re: EVPN/VXLAN on ASR9001 - BGP announcements not working [ In reply to ]
Hi,

On Wed, Apr 29, 2020 at 07:16:29PM +0300, Saku Ytti wrote:
> I think for Gert committing on such an old platform was matter of
> density, no need for 9901 density,

Right. 9001 is officially still supported, and while I could use a
few more 10GE ports, it does what is needed in these POPs - namely,
transport about ~6-8 Gbit/s (!!!) with a decent number of 10 Gbit/s
ports (so we can go over 1 Gbit on each of them, even if the total
throughput is not all that much), full BGP table, proper incoming
anti-DDoS rate limiting (the usual reflective UDP crap).

We're a small shop - our BGP edge is all 10Gbit/s, but due to
"we want it distributed so no box has all the links" the total
traffic in the box is much much lower than it could do - so the
extra bang of the 9901 and the accompanying price tag makes it not
very interesting.

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de
Re: EVPN/VXLAN on ASR9001 - BGP announcements not working [ In reply to ]
On 29/Apr/20 18:16, Saku Ytti wrote:

> Just because the new NPU can do 40x400GE, doesn't mean we don't still
> have applications for 1GE optimised devices with the same features and
> functions as the latest and greatest chip.

This!

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: EVPN/VXLAN on ASR9001 - BGP announcements not working [ In reply to ]
> Gert Doering
> Sent: Wednesday, April 29, 2020 4:22 PM
>
> remember me...? Fighting with this "simple task" since quite a few
weeks...
>
> On Sun, Mar 29, 2020 at 11:52:03AM +0200, Gert Doering wrote:
> > I'm trying to make EVPN via VXLAN encapsulation work between two
> > ASR9001 (with the goal of eventually making it work between ASR9001
> > and Arista boxes, but right now I'm failing ASR9001 <-> ASR9001
already).
>
Would it be feasible to go the EVPN over MPLS route? Arista might support
some basic MPLS forwarding (most Data-Centre switches do).

Never understood this VXLAN nonsense in DC universe (not the comics), in SP
space this was a solved problem and is a well-trodden path since ever -PWs,
VPLS now EVPN (even with traffic engineering possibilities -you know the
mice around elephant flows...)
MPLS to DCs is my answer.

adam

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: EVPN/VXLAN on ASR9001 - BGP announcements not working [ In reply to ]
adamv0025@netconsultings.com wrote on 30/04/2020 13:14:
> Never understood this VXLAN nonsense in DC universe (not the comics), in SP
> space this was a solved problem and is a well-trodden path since ever -PWs,

everything has its use. PWs have major problems loadbalancing over
multiple links. Also the complexity of mpls + associated control plane
is a drawback in situations. Sometimes vxlan provides the required
functionality. Why use a more complicated protocol when a simpler one
is a 100% match for your requirements?

Nick
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: EVPN/VXLAN on ASR9001 - BGP announcements not working [ In reply to ]
Hi,

On Thu, Apr 30, 2020 at 01:14:12PM +0100, adamv0025@netconsultings.com wrote:
> > On Sun, Mar 29, 2020 at 11:52:03AM +0200, Gert Doering wrote:
> > > I'm trying to make EVPN via VXLAN encapsulation work between two
> > > ASR9001 (with the goal of eventually making it work between ASR9001
> > > and Arista boxes, but right now I'm failing ASR9001 <-> ASR9001
> > > already).
>
> Would it be feasible to go the EVPN over MPLS route? Arista might support
> some basic MPLS forwarding (most Data-Centre switches do).

Hah!

As in: the Trident 2+/Trident 3 boxes *could* do MPLS P, but Arista says
that their MPLS capabilities are too limited so they do not provide any
MPLS support for these boxes. Not sure if it actually could do enough
actual encap/decap for the MPLS PE bits required for EVPN/MPLS - but if
it can't even do P, no good asking.

The Jericho boxes can do "all of this", but given the price difference,
we run the "customer edge" on Trident boxes - no need for full tables
there, rarely a need for extra large buffers or detailed QoS ("these
links are never full").

OTOH, Trident 2+ does 802.1q<->VXLAN perfectly well, and T3 even does
"routed into VXLAN" without performance impact, so this is where the
journey is headed to. Whether we like it or not.


Now, as a side note: of course ASR9k does VXLAN with multicast underlay
and "MAC learning from the data plane" just fine, but *this* is not
supported on the Arista side. Of course.

Arista could statically configured VXLAN VTEP neighbours, but *that* is
not supported on the ASR9k side. Of course.

So, EVPN/VXLAN was the promise "hey, we have an IETF standard here, and
both vendors claim to support it". And of course some other BUs inside
Cisco actually support this. Yeah. (#include <rants/standard/cisco-bu>)


> Never understood this VXLAN nonsense in DC universe (not the comics), in SP
> space this was a solved problem and is a well-trodden path since ever -PWs,
> VPLS now EVPN (even with traffic engineering possibilities -you know the
> mice around elephant flows...)
> MPLS to DCs is my answer.

Unfortunately, vendors still think MPLS is something for people with
deep pockets... (like, Juniper charging tons of extra money for the
MPLS license for their QFX5k gear...)

I'm totally fine with MPLS... we've run that for like 10+ years with
lots of cludges around our 6500/sup720s... (like, looped ports from
the box to itself to be able to EoMPLS-away a single VLAN which is
also routed on the box itself)

In the end, I do not really care. The box config and the linkage
between "boxes in the same distributed VLAN" is coming from the
provisioning magic box, I just need to figure out which of the parts
actually work.

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de
Re: EVPN/VXLAN on ASR9001 - BGP announcements not working [ In reply to ]
hi,

On Thu, Apr 30, 2020 at 07:13:32PM +0100, Nick Hilliard wrote:
> everything has its use. PWs have major problems loadbalancing over
> multiple links. Also the complexity of mpls + associated control plane
> is a drawback in situations. Sometimes vxlan provides the required
> functionality. Why use a more complicated protocol when a simpler one
> is a 100% match for your requirements?

Which actually makes me wonder why I'm not ditching EVPN completely
and going back to VXLAN with configured VTEPs on the Aristas - the
system knows where it's peers are, and it MUCH MUCH simpler to figure
out all the moving pieces...

*sigh*

(The idea behind using EVPN was twofold - one: it would interop with
A9K. Ditch that. Two: if I add <node N> into the same VLAN/VXLAN
mesh, I only need configure *that* node, and it will BGP-signal to
all the other nodes "hey, I'm part of it!". OTOH, most affected
VLANs will only ever see 2 or 3 clusters, and the provisioning knows
which devices are part of it... meh)

((Now, if all these ugly hypervisors would actually speak EVPN/whatever,
in a reasonably interoperable way, I could just leave the mess to them,
and provide a default gateway in some place(s), not bothering with all
this layer2 thinking at all. But that's not going to happen either))

Can I retire yet?

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de
Re: EVPN/VXLAN on ASR9001 - BGP announcements not working [ In reply to ]
On 4/29/20 12:23 PM, Sander Steffann wrote:
> That combined with the whole mess with "your PSU and fan tray are not compatible with the 64-bit OS a few years ago" makes me avoid the platform.

We got bit by this, not just by the 64-bit thing, but when Cisco
deprecated and stopped supporting the whole original chassis in favor of
the V2 chassis. Like fork trucking a whole 1/2 rack DC powered chassis
is easy to do or something you want to deal with. Big peeve when
everything we were running in the chassis was still well within the
power budget of what we had wired to the PEM's... yet, they wanted us to
replace all of it to get is back under support. NEVER again.

--
inoc.net!rblayzor
XMPP: rblayzor.AT.inoc.net
PGP: https://pgp.inoc.net/rblayzor/
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: EVPN/VXLAN on ASR9001 - BGP announcements not working [ In reply to ]
> From: Nick Hilliard <nick@foobar.org>
> Sent: Thursday, April 30, 2020 7:14 PM
>
> adamv0025@netconsultings.com wrote on 30/04/2020 13:14:
> > Never understood this VXLAN nonsense in DC universe (not the comics),
> > in SP space this was a solved problem and is a well-trodden path since
> > ever -PWs,
>
> everything has its use. PWs have major problems loadbalancing over multiple
> links.
>
Yes back in the days, but one can use flow labels now,

> Also the complexity of mpls + associated control plane is a drawback in
> situations.
>
Well if you compare EVPN VXLAN -there's not that much of a difference compared to EVPN MPLS, oh and with VPN labels you get micro-segmentation for free and not as an afterthought.

> Sometimes vxlan provides the required functionality. Why use a
> more complicated protocol when a simpler one is a 100% match for your
> requirements?
>
Futureproofing.
DC-folks# This STP sucks, let's MC-LAG/VSS everything, ok that sucks let's do TRILL et, al., that sucked let's do VXLAN, wait, how do we do CP-based mac learning? Let's do EVPN VXLAN, Oh has anyone reserved VXLAN header field that can be used for micro-segmentation? Tumbleweed ...
SP-folks# no way we'll have STP to core, let's sue VPLS, that sucked let's use EVPN/PBB-EVPN......

adam


_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: EVPN/VXLAN on ASR9001 - BGP announcements not working [ In reply to ]
>[...]
> DC-folks# This STP sucks, let's MC-LAG/VSS everything, ok that sucks let's
do
> TRILL et, al., that sucked let's do VXLAN, wait, how do we do CP-based mac

> learning? Let's do EVPN VXLAN, Oh has anyone reserved VXLAN header field
> that can be used for micro-segmentation? Tumbleweed ...
> SP-folks# no way we'll have STP to core, let's sue VPLS, that sucked let's
use
> EVPN/PBB-EVPN......
>
> adam

Today everything must go over https (like dns, ...)
but do not forget to use XML to over bloat everything
and use at least TLS Rev. 9.11 .

Will be punted into the dollar-note big cards (Hollerith) (80 Characters
wide)
Fragmentation will be managed by punting a "C" in Column 5 .

So we will soon see MPLS over HTTPS with fancy XML-Schemes-
Network-devices will be CHROMOS-Browser-Devices,
Don't think of performance or saleable implementation in hardware,
When that would be ready, the standard has been obsoleted and replaced by

Just my 0.01$

Juergen


_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: EVPN/VXLAN on ASR9001 - BGP announcements not working [ In reply to ]
cnsp@marenda.net wrote on 04/05/2020 10:11:
> So we will soon see MPLS over HTTPS with fancy XML-Schemes-
> Network-devices will be CHROMOS-Browser-Devices,
> Don't think of performance or saleable implementation in hardware,
> When that would be ready, the standard has been obsoleted and replaced by

as long as you can encapsulate all that mess in json, I'm good.

Nick

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: EVPN/VXLAN on ASR9001 - BGP announcements not working [ In reply to ]
On Mon, 4 May 2020 at 12:15, <cnsp@marenda.net> wrote:

> Just my 0.01$

Can I get a refund?

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: EVPN/VXLAN on ASR9001 - BGP announcements not working [ In reply to ]
On 4/May/20 11:11, cnsp@marenda.net wrote:

> Today everything must go over https (like dns, ...)
> but do not forget to use XML to over bloat everything
> and use at least TLS Rev. 9.11 .
>
> Will be punted into the dollar-note big cards (Hollerith) (80 Characters
> wide)
> Fragmentation will be managed by punting a "C" in Column 5 .
>
> So we will soon see MPLS over HTTPS with fancy XML-Schemes-
> Network-devices will be CHROMOS-Browser-Devices,
> Don't think of performance or saleable implementation in hardware,
> When that would be ready, the standard has been obsoleted and replaced by

Over course, all going over BGP :-).

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: EVPN/VXLAN on ASR9001 - BGP announcements not working [ In reply to ]
On 4/May/20 11:11, cnsp@marenda.net wrote:

> Today everything must go over https (like dns, ...) but do not forget
> to use XML to over bloat everything and use at least TLS Rev. 9.11 .
> Will be punted into the dollar-note big cards (Hollerith) (80 Characters
> wide) Fragmentation will be managed by punting a "C" in Column 5 .
> So we will soon see MPLS over HTTPS with fancy XML-Schemes-
> Network-devices will be CHROMOS-Browser-Devices, Don't think of
> performance or saleable implementation in hardware, When that would be
> ready, the standard has been obsoleted and replaced by

Of course, all going over BGP :-).

Mark.

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: EVPN/VXLAN on ASR9001 - BGP announcements not working [ In reply to ]
On 4/May/20 11:17, Nick Hilliard wrote:

>  
>
> as long as you can encapsulate all that mess in json, I'm good.

Well, I'll leave that up to my SDN to wrap into my Segment Routing and
move it into my SD-WAN over my 5G :-).

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: EVPN/VXLAN on ASR9001 - BGP announcements not working [ In reply to ]
On 4/May/20 11:23, Saku Ytti wrote:

>
> Can I get a refund?

Can I mail it to your WFH :-)?

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: EVPN/VXLAN on ASR9001 - BGP announcements not working [ In reply to ]
Hi,

On Mon, May 04, 2020 at 11:36:09AM +0200, Mark Tinka wrote:
> Of course, all going over BGP :-).

This is *not* funny.

The work that IETF/IDR is doing these days is truly and deeply scaring
me. "Hey, we have this nice reliable protocol to transport information
from here to there. Why not add tons of different other things we want
to be transported from routers to collectors, or between routers, to
BGP, because, you know, we can?".

Trying to understand what a BGP speaker is doing is difficult enough
today. Or, for that matter, get an implementation right.

gert

--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de

1 2  View All