Mailing List Archive

On Sat, 18 Apr 2020 at 13:47, Antonio Prado via cisco-nsp
<cisco-nsp@puck.nether.net> wrote:

> If not, can you elaborate on the reasons?

I read this question as you think carrying the information in iBGP is
the norm, I view it as an exception. I'm not sure why you'd want to do
that, so I'm curious to hear what is your use-case for needing it.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Right Saku - the filtering is best to be done on the ASBRs facing eBGP.

However in some topologies you may not have all paths on all ASBRs and
there you need to validate on all BGP speakers (or at least RRs). If you do
have all external paths on all ASBRs - case solved - leave IBGP alone.

Using BGP predefined ext communities is one way to enable origin validation
on all your routers. Then if you do you may want to enable or disable
invalid paths to be best path eligible. By default they would not be part
of best path.

If you like to only deprefer them I am marking them with local pref and do
not need to touch any of the IBGP routers.

I guess this is a bit bigger discussion what are you really using origin
validation for.

Thx,
R.



On Sat, Apr 18, 2020 at 1:03 PM Saku Ytti <saku@ytti.fi> wrote:

> On Sat, 18 Apr 2020 at 13:47, Antonio Prado via cisco-nsp
> <cisco-nsp@puck.nether.net> wrote:
>
> > If not, can you elaborate on the reasons?
>
> I read this question as you think carrying the information in iBGP is
> the norm, I view it as an exception. I'm not sure why you'd want to do
> that, so I'm curious to hear what is your use-case for needing it.
>
> --
> ++ytti
> _______________________________________________
> cisco-nsp mailing list cisco-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Hi,

On Sat, Apr 18, 2020 at 01:20:58PM +0200, Robert Raszuk wrote:
> Using BGP predefined ext communities is one way to enable origin validation
> on all your routers. Then if you do you may want to enable or disable
> invalid paths to be best path eligible. By default they would not be part
> of best path.
>
> If you like to only deprefer them I am marking them with local pref and do
> not need to touch any of the IBGP routers.

For "more specific announcement" attacks, local-pref'ing the RPKI invalids
to "ineligible" isn't going to work, as there are no valid alternates.

Ceterum censeo, the only reasonable approach to RPKI OV seems to be
"drop invalids" or "do not bother" (except for a certain phase in
between where you want to monitor first to see if it has any noticeable
effect on production traffic).

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Hi,

On Sat, Apr 18, 2020 at 12:41:43PM +0000, Ben Maddison wrote:
> Feel free to tell your Cisco SE if you think that's dumb.

Indeed, that's dumb... and worse, nonintuitively dangerous.

gert

--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de

On Sat, Apr 18, 2020 at 03:23:33PM +0200, Gert Doering wrote:
> On Sat, Apr 18, 2020 at 12:41:43PM +0000, Ben Maddison wrote:
> > Feel free to tell your Cisco SE if you think that's dumb.
>
> Indeed, that's dumb... and worse, nonintuitively dangerous.

And this comes on top of XE's lack of RFC 8212 compliance! The default
settings on those Cisco IOS XE boxes really seem to set their owners up
for failure.

These devices - without explicit manual workarounds - will leak full BGP
tables, loop traffic around, drop it on the floor, and attempt to take
the rest of the Internet down with them, all in one go! thisisfine.jpg

I wish IOS XE was more like Cisco IOS XR in this regard: XR provides
clever visual clues if no policies are attached to an EBGP neighbor, and
by default XR won't import or export BGP routes on EBGP sessions. This
is a much safer approach to internet routing, probably has prevented a
good many incidents. XR also doesn't require fiddling with communities
to get RPKI OV going.

Kind regards,

Job
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Hi,

> On XE and Classic:
> 1. you can only preform validation on eBGP-received routes;
> 2. any iBGP-received route will get marked "Valid" unless it has a 8097
> extcomm to the contrary; and
> 2. bestpath selection will prefer "Valid" to "Unknown", at the first-
> step in the selection process.
>
> Thus, without 8097 extcomms to mark validation status, you get a
> forwarding loop for every prefix that a) you learn at two-or-more ASBRs
> and b) has no covering ROA.
> That's the majority of the DFZ table for any multihomed AS.

Thanks for the warning!
Sander

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Hello,

On Sat, 18 Apr 2020 at 14:44, Ben Maddison via cisco-nsp
<cisco-nsp@puck.nether.net> wrote:
> Going back to the OP's question, though: we (AS37271) use 8097.
> Not because I think that it's a particularly sensible design (I don't),
> but because we have IOS-XE bgp-speakers, and you can't do ROV on XE or
> Classic without it. At least, if you want routing to work ;-)

And this is why the conversation with the OP started in the first
place (not on this list).

I'm not deploying 8097 because it serves no purpose, other than
working around Cisco IOS stupidities and I'm not going to deploy the
former only to workaround the latter, because it introduces
unnecessary variables.

More about this issue here:
https://www.mail-archive.com/nanog@nanog.org/msg104776.html

Code with CSCvc84848 fixed will hopefully ship this summer, until then
I'm not touching RPKI on IOS(-XE) devices.


As per the 8097 implementation, it looks like both Juniper and Cisco botched it:

https://www.nog.bt/wp-content/uploads/2019/06/rpki_deployment_in_tashicell.pdf

Money quote:
> Both cisco & juniper doesn't follow rfc 8097


- lukas
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Hi,

On Sat, Apr 18, 2020 at 02:13:07PM +0000, Ben Maddison wrote:
> I meant "dumb" as in "I painted the life-saving emergency stop button
> green and mounted it on a green wall behind a locked fire-proof door in
> a different building, which was dumb"

Very dumb indeed :-) - in a very british tradition of fine understatement.

I do not very much like IOS XE anyway, and my classic IOS boxes that do
speak BGP are too old for RPKI support ("no, you can't have new control
plane features, if we think your hardware is old") and will retire in
the next few months anyway.

Yes, you Sup720 and Sup2Ts, this is you. Out!

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de

Hi Gert,


On Sat, 18 Apr 2020 at 16:24, Gert Doering <gert@greenie.muc.de> wrote:
>
> Hi,
>
> On Sat, Apr 18, 2020 at 02:13:07PM +0000, Ben Maddison wrote:
> > I meant "dumb" as in "I painted the life-saving emergency stop button
> > green and mounted it on a green wall behind a locked fire-proof door in
> > a different building, which was dumb"
>
> Very dumb indeed :-) - in a very british tradition of fine understatement.
>
> I do not very much like IOS XE anyway, and my classic IOS boxes that do
> speak BGP are too old for RPKI support ("no, you can't have new control
> plane features, if we think your hardware is old") and will retire in
> the next few months anyway.
>
> Yes, you Sup720 and Sup2Ts, this is you. Out!

You should really upgrade to 7600 then (but 7200 or ME3600 will work
too, slightly different platforms of course) ;)

FWIW: Mark is trying to get the fix as far back as 7200 series:

https://mailman.nanog.org/pipermail/nanog/2020-March/107009.html



-lukas
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Hi,

On Sat, Apr 18, 2020 at 05:24:27PM +0200, Lukas Tribus wrote:
> On Sat, 18 Apr 2020 at 16:24, Gert Doering <gert@greenie.muc.de> wrote:
> > On Sat, Apr 18, 2020 at 02:13:07PM +0000, Ben Maddison wrote:
> > > I meant "dumb" as in "I painted the life-saving emergency stop button
> > > green and mounted it on a green wall behind a locked fire-proof door in
> > > a different building, which was dumb"
> >
> > Very dumb indeed :-) - in a very british tradition of fine understatement.
> >
> > I do not very much like IOS XE anyway, and my classic IOS boxes that do
> > speak BGP are too old for RPKI support ("no, you can't have new control
> > plane features, if we think your hardware is old") and will retire in
> > the next few months anyway.
> >
> > Yes, you Sup720 and Sup2Ts, this is you. Out!
>
> You should really upgrade to 7600 then (but 7200 or ME3600 will work
> too, slightly different platforms of course) ;)

7600, really? This was the first Cisco BU to try the "how much can
we annoy customers before they go and buy elsewhere" stunt. And no.

And no, the ME3600-now-ASR920 is not exactly what I'd call "trustworthy
routers". Nice metro/mpls switches.

> FWIW: Mark is trying to get the fix as far back as 7200 series:
>
> https://mailman.nanog.org/pipermail/nanog/2020-March/107009.html

I didn't know that 7200s actually had any sort of RPKI support, given
that they are all long EOL and much more interesting bugfixes have
never come (like the L2TP crash bugs affecting LNSes).

But anyway, my last few 7301s that do "close to full table" BGP have
no eBGP sessions anymore, and will go into retirement soon as well.


As for "what do you use instead?" - for the edge stuff with no full
tables, 6500s are being replaced by Arista 7050 (T2+/T3), "full BGP"
stuff is being replaced by ASR9k, and testing Arista 7280R now.

We really like Arista. Yes, they can not do everything Cisco can,
but the stuff that they *do* has been really rock-solid for us
(and the "oh, that does not work" bits are not as surprising as with
Cisco... ran into "no, you cannot have STP for a VLAN with mixed
tagged untagged ports on IOS XR" yesterday, much to my dismay).

gert

--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de

Hi Ben,

On XE and Classic:
> 1. you can only preform validation on eBGP-received routes;
> 2. any iBGP-received route will get marked "Valid" unless it has a 8097
> extcomm to the contrary; and
> 2. bestpath selection will prefer "Valid" to "Unknown", at the first-
> step in the selection process.
>

Yes that is exactly the default dumb behaviour. And frankly these days I am
not sure who to even talk in Cisco with about XE BGP :)

Thus, without 8097 extcomms to mark validation status, you get a
> forwarding loop for every prefix that a) you learn at two-or-more ASBRs
> and b) has no covering ROA.
> That's the majority of the DFZ table for any multihomed AS.
>

Well that one I do not think is going to be always the case. It may be if
your ASBRs are also RRs or you enable best external or add-paths.

In the described case I just tested this and one ASBR will use local EBGP
path as best and the other one IBGP learned which pretends to be valid. So
there is no forwarding loop. If it would always be one perhaps cisco
regression testing would fail :)

In the mean time adding the knob "announce rpki state" is the way to go.

Thx,
R.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> Gert Doering
> Sent: Saturday, April 18, 2020 4:58 PM
>
> We really like Arista. Yes, they can not do everything Cisco can, but the
stuff
> that they *do* has been really rock-solid for us (and the "oh, that does
not
> work" bits are not as surprising as with Cisco... ran into "no, you cannot
have
> STP for a VLAN with mixed tagged untagged ports on IOS XR" yesterday,
> much to my dismay).
>
They actually do have STP on XR? Why?
Yes ASR9k was meant as L2 switch initially, but then they changed their mind
and turned it into a router, i.e. LAG. MC-LAG, BUM rate limiters and
split-horizon groups in bridge-domains I'd consider as the right tools (or
REP if pushed hard), but STP? I guess you found yourself between a rock and
a hard place in your design.

adam


_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> Robert Raszuk
> Sent: Saturday, April 18, 2020 5:03 PM
>
> Hi Ben,
>
> On XE and Classic:
> > 1. you can only preform validation on eBGP-received routes; 2. any
> > iBGP-received route will get marked "Valid" unless it has a 8097
> > extcomm to the contrary; and 2. bestpath selection will prefer "Valid"
> > to "Unknown", at the first- step in the selection process.
> >
>
> Yes that is exactly the default dumb behaviour. And frankly these days I
am
> not sure who to even talk in Cisco with about XE BGP :)
>
My view is that XE is falling out of the "ISP frame" with all support
directed toward XR and the XR7 going forward....

adam

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Hi,

On Sat, Apr 18, 2020 at 08:34:57PM +0100, adamv0025@netconsultings.com wrote:
> > We really like Arista. Yes, they can not do everything Cisco can, but the
> stuff
> > that they *do* has been really rock-solid for us (and the "oh, that does
> not
> > work" bits are not as surprising as with Cisco... ran into "no, you cannot
> have
> > STP for a VLAN with mixed tagged untagged ports on IOS XR" yesterday,
> > much to my dismay).
>
> They actually do have STP on XR? Why?

I'd say because their EVPN stuff wasn't finished yet :-)


> Yes ASR9k was meant as L2 switch initially, but then they changed their mind
> and turned it into a router, i.e. LAG. MC-LAG, BUM rate limiters and
> split-horizon groups in bridge-domains I'd consider as the right tools (or
> REP if pushed hard), but STP? I guess you found yourself between a rock and
> a hard place in your design.

Small POP, two ASR9001 for routed connectivity, redundant paths to
two switches that have 1GE customer connections. Customer has two
firewalls, that speak their sort of cluster redundancy protocol
("VRRP-ish", with a virtual MAC that moves around).

Customers are not to be trusted, so STP on the customer ports is a must.

Switch hardware / SFP+ can break, so the switches can lose their
interconnect - in which case, split brain, unless you do bridge-domain
"ring" through the two ASRs. And then you have a ring, and want STP
to break flooding.

(But indeed, for that topology, split-horizon groups might actually
be a better answer. Need to revisit our deployments to see if I
can generalize our tooling to avoid STP in simple topologies like
this one... in other places, we've managed to totally go away from
STP already except as "protect customer ports against short-circuiting",
doing everything with MLAGs on the Aristas. MC-LAG on the ASR9001
eats too many ports...)

Thanks for giving me something to think about :-)

gert

--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de

On 18/Apr/20 12:45, Antonio Prado via cisco-nsp wrote:
> Hello,
>
> is there anyone who is using in production "RPKI extended-community" to
> carry the validation state inside an autonomous system (RFC8097)?
>
> If yes, how large is your AS?
>
> If not, can you elaborate on the reasons?

As part of the BCP's we taught and discussed during the last APRICOT
meeting in Melbourne, I advise against using BGP communities to convey
RPKI state.

One of the most elegant things about RPKI is that every router in your
network can make RPKI-based decisions independently of any other router.
That means you could have thousands of nodes each maintaining the same
RPKI state, without ever speaking to each other.

When you choose to convey RPKI state in BGP communities, you create a
dependence between routers which degrades your resiliency. If you have
multiple vendors in your network, you open yourself up to issues when
you upgrade or downgrade code that breaks things.

As we discovered in Melbourne, earlier versions of Junos break the
well-known RPKI BGP communities. Imagine the havoc this could cause on
your network if you assumed one vendor was doing the right thing. and
they aren't.

Don't use BGP communities to convey RPKI state. You don't need to.
Servers scale better than router control planes. A server handling RTR
sessions for thousands of routers is far better than trying to get your
entire network to exchange RPKI BGP communities cohesively.

For the Melbourne number, see here:

    https://2020.apricot.net/program/schedule/#/day/7/rpki-deployment-1

Mark.

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 18/Apr/20 13:20, Robert Raszuk wrote:
> Right Saku - the filtering is best to be done on the ASBRs facing eBGP.
>
> However in some topologies you may not have all paths on all ASBRs and
> there you need to validate on all BGP speakers (or at least RRs). If you do
> have all external paths on all ASBRs - case solved - leave IBGP alone.

OV only needs to be done on eBGP sessions.

Clean eBGP sessions = clean iBGP sessions, automatically.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 18/Apr/20 14:03, Gert Doering wrote:

>
> Ceterum censeo, the only reasonable approach to RPKI OV seems to be
> "drop invalids" or "do not bother" (except for a certain phase in
> between where you want to monitor first to see if it has any noticeable
> effect on production traffic).

This.

Drop Invalids, accept everything else.

You don't need to explicitly accept Valid or NotFound.

Mark.

On 18/Apr/20 14:44, Ben Maddison via cisco-nsp wrote:
> Going back to the OP's question, though: we (AS37271) use 8097.
> Not because I think that it's a particularly sensible design (I don't),
> but because we have IOS-XE bgp-speakers, and you can't do ROV on XE or
> Classic without it. At least, if you want routing to work ;-)

Yes, Ben's situation is just about the only reason it would be worth
considering conveying RPKI state in BGP communities.

On our end, we just disabled RPKI on IOS XE platforms. Fair point, it's
only about 4 running BGP with customers. Going through the hassle with
Cisco to fix this rather than work magic on them was easier for us,
since the majority of our transit edge is on Junos :-).

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 18/Apr/20 16:05, Job Snijders wrote:

> And this comes on top of XE's lack of RFC 8212 compliance! The default
> settings on those Cisco IOS XE boxes really seem to set their owners up
> for failure.
>
> These devices - without explicit manual workarounds - will leak full BGP
> tables, loop traffic around, drop it on the floor, and attempt to take
> the rest of the Internet down with them, all in one go! thisisfine.jpg
>
> I wish IOS XE was more like Cisco IOS XR in this regard: XR provides
> clever visual clues if no policies are attached to an EBGP neighbor, and
> by default XR won't import or export BGP routes on EBGP sessions. This
> is a much safer approach to internet routing, probably has prevented a
> good many incidents. XR also doesn't require fiddling with communities
> to get RPKI OV going.

It's like they are two totally different companies, isn't it :-).

Who'da thunk it...

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 18/Apr/20 16:23, Lukas Tribus wrote:

>
> More about this issue here:
> https://www.mail-archive.com/nanog@nanog.org/msg104776.html
>
> Code with CSCvc84848 fixed will hopefully ship this summer, until then
> I'm not touching RPKI on IOS(-XE) devices.

Time to check in with them.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/