Mailing List Archive

RPKI extended-community RFC8097
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
On Sat, 18 Apr 2020 at 13:47, Antonio Prado via cisco-nsp
<cisco-nsp@puck.nether.net> wrote:

> If not, can you elaborate on the reasons?

I read this question as you think carrying the information in iBGP is
the norm, I view it as an exception. I'm not sure why you'd want to do
that, so I'm curious to hear what is your use-case for needing it.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
Right Saku - the filtering is best to be done on the ASBRs facing eBGP.

However in some topologies you may not have all paths on all ASBRs and
there you need to validate on all BGP speakers (or at least RRs). If you do
have all external paths on all ASBRs - case solved - leave IBGP alone.

Using BGP predefined ext communities is one way to enable origin validation
on all your routers. Then if you do you may want to enable or disable
invalid paths to be best path eligible. By default they would not be part
of best path.

If you like to only deprefer them I am marking them with local pref and do
not need to touch any of the IBGP routers.

I guess this is a bit bigger discussion what are you really using origin
validation for.

Thx,
R.



On Sat, Apr 18, 2020 at 1:03 PM Saku Ytti <saku@ytti.fi> wrote:

> On Sat, 18 Apr 2020 at 13:47, Antonio Prado via cisco-nsp
> <cisco-nsp@puck.nether.net> wrote:
>
> > If not, can you elaborate on the reasons?
>
> I read this question as you think carrying the information in iBGP is
> the norm, I view it as an exception. I'm not sure why you'd want to do
> that, so I'm curious to hear what is your use-case for needing it.
>
> --
> ++ytti
> _______________________________________________
> cisco-nsp mailing list cisco-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
Hi,

On Sat, Apr 18, 2020 at 01:20:58PM +0200, Robert Raszuk wrote:
> Using BGP predefined ext communities is one way to enable origin validation
> on all your routers. Then if you do you may want to enable or disable
> invalid paths to be best path eligible. By default they would not be part
> of best path.
>
> If you like to only deprefer them I am marking them with local pref and do
> not need to touch any of the IBGP routers.

For "more specific announcement" attacks, local-pref'ing the RPKI invalids
to "ineligible" isn't going to work, as there are no valid alternates.

Ceterum censeo, the only reasonable approach to RPKI OV seems to be
"drop invalids" or "do not bother" (except for a certain phase in
between where you want to monitor first to see if it has any noticeable
effect on production traffic).

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de
Re: RPKI extended-community RFC8097 [ In reply to ]
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
Hi,

On Sat, Apr 18, 2020 at 12:41:43PM +0000, Ben Maddison wrote:
> Feel free to tell your Cisco SE if you think that's dumb.

Indeed, that's dumb... and worse, nonintuitively dangerous.

gert

--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de
Re: RPKI extended-community RFC8097 [ In reply to ]
On Sat, Apr 18, 2020 at 03:23:33PM +0200, Gert Doering wrote:
> On Sat, Apr 18, 2020 at 12:41:43PM +0000, Ben Maddison wrote:
> > Feel free to tell your Cisco SE if you think that's dumb.
>
> Indeed, that's dumb... and worse, nonintuitively dangerous.

And this comes on top of XE's lack of RFC 8212 compliance! The default
settings on those Cisco IOS XE boxes really seem to set their owners up
for failure.

These devices - without explicit manual workarounds - will leak full BGP
tables, loop traffic around, drop it on the floor, and attempt to take
the rest of the Internet down with them, all in one go! thisisfine.jpg

I wish IOS XE was more like Cisco IOS XR in this regard: XR provides
clever visual clues if no policies are attached to an EBGP neighbor, and
by default XR won't import or export BGP routes on EBGP sessions. This
is a much safer approach to internet routing, probably has prevented a
good many incidents. XR also doesn't require fiddling with communities
to get RPKI OV going.

Kind regards,

Job
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
Hi,

> On XE and Classic:
> 1. you can only preform validation on eBGP-received routes;
> 2. any iBGP-received route will get marked "Valid" unless it has a 8097
> extcomm to the contrary; and
> 2. bestpath selection will prefer "Valid" to "Unknown", at the first-
> step in the selection process.
>
> Thus, without 8097 extcomms to mark validation status, you get a
> forwarding loop for every prefix that a) you learn at two-or-more ASBRs
> and b) has no covering ROA.
> That's the majority of the DFZ table for any multihomed AS.

Thanks for the warning!
Sander
Re: RPKI extended-community RFC8097 [ In reply to ]
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
Hello,

On Sat, 18 Apr 2020 at 14:44, Ben Maddison via cisco-nsp
<cisco-nsp@puck.nether.net> wrote:
> Going back to the OP's question, though: we (AS37271) use 8097.
> Not because I think that it's a particularly sensible design (I don't),
> but because we have IOS-XE bgp-speakers, and you can't do ROV on XE or
> Classic without it. At least, if you want routing to work ;-)

And this is why the conversation with the OP started in the first
place (not on this list).

I'm not deploying 8097 because it serves no purpose, other than
working around Cisco IOS stupidities and I'm not going to deploy the
former only to workaround the latter, because it introduces
unnecessary variables.

More about this issue here:
https://www.mail-archive.com/nanog@nanog.org/msg104776.html

Code with CSCvc84848 fixed will hopefully ship this summer, until then
I'm not touching RPKI on IOS(-XE) devices.


As per the 8097 implementation, it looks like both Juniper and Cisco botched it:

https://www.nog.bt/wp-content/uploads/2019/06/rpki_deployment_in_tashicell.pdf

Money quote:
> Both cisco & juniper doesn't follow rfc 8097


- lukas
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
Hi,

On Sat, Apr 18, 2020 at 02:13:07PM +0000, Ben Maddison wrote:
> I meant "dumb" as in "I painted the life-saving emergency stop button
> green and mounted it on a green wall behind a locked fire-proof door in
> a different building, which was dumb"

Very dumb indeed :-) - in a very british tradition of fine understatement.

I do not very much like IOS XE anyway, and my classic IOS boxes that do
speak BGP are too old for RPKI support ("no, you can't have new control
plane features, if we think your hardware is old") and will retire in
the next few months anyway.

Yes, you Sup720 and Sup2Ts, this is you. Out!

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de
Re: RPKI extended-community RFC8097 [ In reply to ]
Hi Gert,


On Sat, 18 Apr 2020 at 16:24, Gert Doering <gert@greenie.muc.de> wrote:
>
> Hi,
>
> On Sat, Apr 18, 2020 at 02:13:07PM +0000, Ben Maddison wrote:
> > I meant "dumb" as in "I painted the life-saving emergency stop button
> > green and mounted it on a green wall behind a locked fire-proof door in
> > a different building, which was dumb"
>
> Very dumb indeed :-) - in a very british tradition of fine understatement.
>
> I do not very much like IOS XE anyway, and my classic IOS boxes that do
> speak BGP are too old for RPKI support ("no, you can't have new control
> plane features, if we think your hardware is old") and will retire in
> the next few months anyway.
>
> Yes, you Sup720 and Sup2Ts, this is you. Out!

You should really upgrade to 7600 then (but 7200 or ME3600 will work
too, slightly different platforms of course) ;)

FWIW: Mark is trying to get the fix as far back as 7200 series:

https://mailman.nanog.org/pipermail/nanog/2020-March/107009.html



-lukas
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
Hi,

On Sat, Apr 18, 2020 at 05:24:27PM +0200, Lukas Tribus wrote:
> On Sat, 18 Apr 2020 at 16:24, Gert Doering <gert@greenie.muc.de> wrote:
> > On Sat, Apr 18, 2020 at 02:13:07PM +0000, Ben Maddison wrote:
> > > I meant "dumb" as in "I painted the life-saving emergency stop button
> > > green and mounted it on a green wall behind a locked fire-proof door in
> > > a different building, which was dumb"
> >
> > Very dumb indeed :-) - in a very british tradition of fine understatement.
> >
> > I do not very much like IOS XE anyway, and my classic IOS boxes that do
> > speak BGP are too old for RPKI support ("no, you can't have new control
> > plane features, if we think your hardware is old") and will retire in
> > the next few months anyway.
> >
> > Yes, you Sup720 and Sup2Ts, this is you. Out!
>
> You should really upgrade to 7600 then (but 7200 or ME3600 will work
> too, slightly different platforms of course) ;)

7600, really? This was the first Cisco BU to try the "how much can
we annoy customers before they go and buy elsewhere" stunt. And no.

And no, the ME3600-now-ASR920 is not exactly what I'd call "trustworthy
routers". Nice metro/mpls switches.

> FWIW: Mark is trying to get the fix as far back as 7200 series:
>
> https://mailman.nanog.org/pipermail/nanog/2020-March/107009.html

I didn't know that 7200s actually had any sort of RPKI support, given
that they are all long EOL and much more interesting bugfixes have
never come (like the L2TP crash bugs affecting LNSes).

But anyway, my last few 7301s that do "close to full table" BGP have
no eBGP sessions anymore, and will go into retirement soon as well.


As for "what do you use instead?" - for the edge stuff with no full
tables, 6500s are being replaced by Arista 7050 (T2+/T3), "full BGP"
stuff is being replaced by ASR9k, and testing Arista 7280R now.

We really like Arista. Yes, they can not do everything Cisco can,
but the stuff that they *do* has been really rock-solid for us
(and the "oh, that does not work" bits are not as surprising as with
Cisco... ran into "no, you cannot have STP for a VLAN with mixed
tagged untagged ports on IOS XR" yesterday, much to my dismay).

gert

--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de
Re: RPKI extended-community RFC8097 [ In reply to ]
Hi Ben,

On XE and Classic:
> 1. you can only preform validation on eBGP-received routes;
> 2. any iBGP-received route will get marked "Valid" unless it has a 8097
> extcomm to the contrary; and
> 2. bestpath selection will prefer "Valid" to "Unknown", at the first-
> step in the selection process.
>

Yes that is exactly the default dumb behaviour. And frankly these days I am
not sure who to even talk in Cisco with about XE BGP :)

Thus, without 8097 extcomms to mark validation status, you get a
> forwarding loop for every prefix that a) you learn at two-or-more ASBRs
> and b) has no covering ROA.
> That's the majority of the DFZ table for any multihomed AS.
>

Well that one I do not think is going to be always the case. It may be if
your ASBRs are also RRs or you enable best external or add-paths.

In the described case I just tested this and one ASBR will use local EBGP
path as best and the other one IBGP learned which pretends to be valid. So
there is no forwarding loop. If it would always be one perhaps cisco
regression testing would fail :)

In the mean time adding the knob "announce rpki state" is the way to go.

Thx,
R.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
> Gert Doering
> Sent: Saturday, April 18, 2020 4:58 PM
>
> We really like Arista. Yes, they can not do everything Cisco can, but the
stuff
> that they *do* has been really rock-solid for us (and the "oh, that does
not
> work" bits are not as surprising as with Cisco... ran into "no, you cannot
have
> STP for a VLAN with mixed tagged untagged ports on IOS XR" yesterday,
> much to my dismay).
>
They actually do have STP on XR? Why?
Yes ASR9k was meant as L2 switch initially, but then they changed their mind
and turned it into a router, i.e. LAG. MC-LAG, BUM rate limiters and
split-horizon groups in bridge-domains I'd consider as the right tools (or
REP if pushed hard), but STP? I guess you found yourself between a rock and
a hard place in your design.

adam


_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
> Robert Raszuk
> Sent: Saturday, April 18, 2020 5:03 PM
>
> Hi Ben,
>
> On XE and Classic:
> > 1. you can only preform validation on eBGP-received routes; 2. any
> > iBGP-received route will get marked "Valid" unless it has a 8097
> > extcomm to the contrary; and 2. bestpath selection will prefer "Valid"
> > to "Unknown", at the first- step in the selection process.
> >
>
> Yes that is exactly the default dumb behaviour. And frankly these days I
am
> not sure who to even talk in Cisco with about XE BGP :)
>
My view is that XE is falling out of the "ISP frame" with all support
directed toward XR and the XR7 going forward....

adam

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: STP on ASR9k [ In reply to ]
Hi,

On Sat, Apr 18, 2020 at 08:34:57PM +0100, adamv0025@netconsultings.com wrote:
> > We really like Arista. Yes, they can not do everything Cisco can, but the
> stuff
> > that they *do* has been really rock-solid for us (and the "oh, that does
> not
> > work" bits are not as surprising as with Cisco... ran into "no, you cannot
> have
> > STP for a VLAN with mixed tagged untagged ports on IOS XR" yesterday,
> > much to my dismay).
>
> They actually do have STP on XR? Why?

I'd say because their EVPN stuff wasn't finished yet :-)


> Yes ASR9k was meant as L2 switch initially, but then they changed their mind
> and turned it into a router, i.e. LAG. MC-LAG, BUM rate limiters and
> split-horizon groups in bridge-domains I'd consider as the right tools (or
> REP if pushed hard), but STP? I guess you found yourself between a rock and
> a hard place in your design.

Small POP, two ASR9001 for routed connectivity, redundant paths to
two switches that have 1GE customer connections. Customer has two
firewalls, that speak their sort of cluster redundancy protocol
("VRRP-ish", with a virtual MAC that moves around).

Customers are not to be trusted, so STP on the customer ports is a must.

Switch hardware / SFP+ can break, so the switches can lose their
interconnect - in which case, split brain, unless you do bridge-domain
"ring" through the two ASRs. And then you have a ring, and want STP
to break flooding.

(But indeed, for that topology, split-horizon groups might actually
be a better answer. Need to revisit our deployments to see if I
can generalize our tooling to avoid STP in simple topologies like
this one... in other places, we've managed to totally go away from
STP already except as "protect customer ports against short-circuiting",
doing everything with MLAGs on the Aristas. MC-LAG on the ASR9001
eats too many ports...)

Thanks for giving me something to think about :-)

gert

--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de
Re: RPKI extended-community RFC8097 [ In reply to ]
On 18/Apr/20 12:45, Antonio Prado via cisco-nsp wrote:
> Hello,
>
> is there anyone who is using in production "RPKI extended-community" to
> carry the validation state inside an autonomous system (RFC8097)?
>
> If yes, how large is your AS?
>
> If not, can you elaborate on the reasons?

As part of the BCP's we taught and discussed during the last APRICOT
meeting in Melbourne, I advise against using BGP communities to convey
RPKI state.

One of the most elegant things about RPKI is that every router in your
network can make RPKI-based decisions independently of any other router.
That means you could have thousands of nodes each maintaining the same
RPKI state, without ever speaking to each other.

When you choose to convey RPKI state in BGP communities, you create a
dependence between routers which degrades your resiliency. If you have
multiple vendors in your network, you open yourself up to issues when
you upgrade or downgrade code that breaks things.

As we discovered in Melbourne, earlier versions of Junos break the
well-known RPKI BGP communities. Imagine the havoc this could cause on
your network if you assumed one vendor was doing the right thing. and
they aren't.

Don't use BGP communities to convey RPKI state. You don't need to.
Servers scale better than router control planes. A server handling RTR
sessions for thousands of routers is far better than trying to get your
entire network to exchange RPKI BGP communities cohesively.

For the Melbourne number, see here:

    https://2020.apricot.net/program/schedule/#/day/7/rpki-deployment-1

Mark.

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
On 18/Apr/20 13:20, Robert Raszuk wrote:
> Right Saku - the filtering is best to be done on the ASBRs facing eBGP.
>
> However in some topologies you may not have all paths on all ASBRs and
> there you need to validate on all BGP speakers (or at least RRs). If you do
> have all external paths on all ASBRs - case solved - leave IBGP alone.

OV only needs to be done on eBGP sessions.

Clean eBGP sessions = clean iBGP sessions, automatically.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
On 18/Apr/20 14:03, Gert Doering wrote:

>
> Ceterum censeo, the only reasonable approach to RPKI OV seems to be
> "drop invalids" or "do not bother" (except for a certain phase in
> between where you want to monitor first to see if it has any noticeable
> effect on production traffic).

This.

Drop Invalids, accept everything else.

You don't need to explicitly accept Valid or NotFound.

Mark.
Re: RPKI extended-community RFC8097 [ In reply to ]
On 18/Apr/20 14:44, Ben Maddison via cisco-nsp wrote:
> Going back to the OP's question, though: we (AS37271) use 8097.
> Not because I think that it's a particularly sensible design (I don't),
> but because we have IOS-XE bgp-speakers, and you can't do ROV on XE or
> Classic without it. At least, if you want routing to work ;-)

Yes, Ben's situation is just about the only reason it would be worth
considering conveying RPKI state in BGP communities.

On our end, we just disabled RPKI on IOS XE platforms. Fair point, it's
only about 4 running BGP with customers. Going through the hassle with
Cisco to fix this rather than work magic on them was easier for us,
since the majority of our transit edge is on Junos :-).

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
On 18/Apr/20 16:05, Job Snijders wrote:

> And this comes on top of XE's lack of RFC 8212 compliance! The default
> settings on those Cisco IOS XE boxes really seem to set their owners up
> for failure.
>
> These devices - without explicit manual workarounds - will leak full BGP
> tables, loop traffic around, drop it on the floor, and attempt to take
> the rest of the Internet down with them, all in one go! thisisfine.jpg
>
> I wish IOS XE was more like Cisco IOS XR in this regard: XR provides
> clever visual clues if no policies are attached to an EBGP neighbor, and
> by default XR won't import or export BGP routes on EBGP sessions. This
> is a much safer approach to internet routing, probably has prevented a
> good many incidents. XR also doesn't require fiddling with communities
> to get RPKI OV going.

It's like they are two totally different companies, isn't it :-).

Who'da thunk it...

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
On 18/Apr/20 16:23, Lukas Tribus wrote:

>
> More about this issue here:
> https://www.mail-archive.com/nanog@nanog.org/msg104776.html
>
> Code with CSCvc84848 fixed will hopefully ship this summer, until then
> I'm not touching RPKI on IOS(-XE) devices.

Time to check in with them.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
On 18/Apr/20 16:24, Gert Doering wrote:

>
> Very dumb indeed :-) - in a very british tradition of fine understatement.
>
> I do not very much like IOS XE anyway, and my classic IOS boxes that do
> speak BGP are too old for RPKI support ("no, you can't have new control
> plane features, if we think your hardware is old") and will retire in
> the next few months anyway.

Well, if I'm honest, both IOS XR, for me, is too verbose in the Metro
and both it and Junos are too bloated as an RR.

CSR1000v is plain IOS XE, which we use extensively for an RR.

ASR920's run IOS XE, for the Metro. However, I can stomach Junos there too.

Happy with those.

Mark.
Re: RPKI extended-community RFC8097 [ In reply to ]
On 18/Apr/20 17:57, Gert Doering wrote:

> I didn't know that 7200s actually had any sort of RPKI support, given
> that they are all long EOL and much more interesting bugfixes have
> never come (like the L2TP crash bugs affecting LNSes).

They've had RPKI support since 2013/2014. It's where we tested first,
back then... them and the ME3600X and ASR1000.

Sadly, aside from being less crashy, the code has pretty much been the
same since then, both in IOS and IOS XE land.

We have a ton of 7201's and NPE-G2's we use as looking glass routers
(moving to CSR1000v), that support RPKI for the public:

    http://as37100.net/?lookingglass

Mark.
Re: RPKI extended-community RFC8097 [ In reply to ]
On 18/Apr/20 21:39, Ben Maddison via cisco-nsp wrote:
> Mark says he's having more luck down that same avenue. We'll see...

I like to give space, otherwise the girl runs away, fast.

But exactly a month is not unreasonable, so I'm picking up the phone to
say "Hey, remember me?", if my parents remembered to pay the bill :-).

Mark.

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
On 18/Apr/20 21:41, adamv0025@netconsultings.com wrote:

> My view is that XE is falling out of the "ISP frame" with all support
> directed toward XR and the XR7 going forward....

I still think there is room for IOS XE.

If they drop CSR1000v, I'm moving to Arista for my RR :-).

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
On Sat, Apr 18, 2020, at 21:41, adamv0025@netconsultings.com wrote:
> My view is that XE is falling out of the "ISP frame" with all support
> directed toward XR and the XR7 going forward....

Do they plan some LNS implementation on XR ?
... beacause in some parts of the world LNS is still a necessity.

But hey, if they don't, there's still Juniper and Nokia (and Huawei where acceptable).

--
R.-A. Feurdean
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
On Sun, Apr 19, 2020, at 03:00, Mark Tinka wrote:
> Yes, Ben's situation is just about the only reason it would be worth
> considering conveying RPKI state in BGP communities.

Another one would be route-servers in an IXP : drop invalids, and tag differently Valid and NotFound.
But that scenario usually does NOT involve Cisco gear; it's usually BIRD. And RFC8097 doesn't exactly match that scenario either.

--
R.-A. Feurdean
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
Radu-Adrian FEURDEAN wrote on 21/04/2020 21:28:
> Another one would be route-servers in an IXP : drop invalids, and tag
> differently Valid and NotFound.
> But that scenario usually does NOT involve Cisco gear; it's usually
> BIRD. And RFC8097 doesn't exactly match that scenario either.

RFC8097 is ibgp only. There are compelling reasons not to do this with
ebgp.

Nick
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
On Tue, Apr 21, 2020, at 22:42, Nick Hilliard wrote:
> > BIRD. And RFC8097 doesn't exactly match that scenario either.
>
> RFC8097 is ibgp only. There are compelling reasons not to do this with ebgp.

That's why it "doesn't exactly match the scenario". And why IXPs that signal validation state, do it their own way (LINX and France-IX comes to my mind).
However, did you note that after the "MUST drop the extcomm", the next phrase says: "However, it SHOULD be possible to configure an implementation to send or *accept* the community when warranted" ?

--
R.-A. Feurdean
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
On Tue, Apr 21, 2020, at 23:34, Radu-Adrian FEURDEAN wrote:
> On Tue, Apr 21, 2020, at 22:42, Nick Hilliard wrote:
> > > BIRD. And RFC8097 doesn't exactly match that scenario either.
> >
> > RFC8097 is ibgp only. There are compelling reasons not to do this with ebgp.
>
> That's why it "doesn't exactly match the scenario". And why IXPs that
> signal validation state, do it their own way (LINX and France-IX comes
> to my mind).
> However, did you note that after the "MUST drop the extcomm", the next
> phrase says: "However, it SHOULD be possible to configure an
> implementation to send or *accept* the community when warranted" ?

Any operator can attach any BGP Community to signal things like "valid", "not-found", "orange", but the two operators need to agree on it between their ASNs and assume both sides will delete & set such communities in the right places. If you want to signal something, pick a normal or a large community (within your own 'namespace') and tell your peers that's the one you are using for a specific purpose.

However, I don't think you can really signal the validation state across administrative boundaries. The trust is not transitive, especially over most-likely unsecured BGP transport. There is no mechanism in BGP to verify if the peer can be trusted to set the right communities, operational parameters about the peer's validation process are not visible through BGP.

Kind regards,

Job
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
On Tue, Apr 21, 2020, at 23:53, Job Snijders wrote:

> a normal or a large community (within your own 'namespace') and tell
> your peers that's the one you are using for a specific purpose.

This is what LINX and France-IX do, this also works on IBGP, and this is why RFC8097 has a very low (close to zero) value.

> However, I don't think you can really signal the validation state
> across administrative boundaries. The trust is not transitive,
> especially over most-likely unsecured BGP transport. There is no
> mechanism in BGP to verify if the peer can be trusted to set the right
> communities, operational parameters about the peer's validation process
> are not visible through BGP.

Take it like "RPKI As A Service". People ready to take/use pretty much everything "aaS" (whether it makes sense or not) are not difficult to find. You have several kinds of "security as a service", including "managed security", so RPKIaaS isn't much worse than that.

--
R.-A. Feurdean
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
On 22/Apr/20 01:15, Radu-Adrian FEURDEAN wrote:

> Take it like "RPKI As A Service". People ready to take/use pretty much everything "aaS" (whether it makes sense or not) are not difficult to find. You have several kinds of "security as a service", including "managed security", so RPKIaaS isn't much worse than that.

Oh dear, let's not open that can of bats :-).

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
Radu-Adrian FEURDEAN wrote on 22/04/2020 00:15:
> Take it like "RPKI As A Service". People ready to take/use pretty
> much everything "aaS" (whether it makes sense or not) are not
> difficult to find. You have several kinds of "security as a service",
> including "managed security", so RPKIaaS isn't much worse than that.
really, for ebgp it is much worse than that. There was some discussion
about this on the IETF sidrops mailing list a couple of years ago which
unearthed several fundamental difficulties with the ebgp equivalent,
none of which were (or can be) resolved.

Nick
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
Hello Mark, hello Jakob,


On Sun, 19 Apr 2020 at 03:03, Mark Tinka <mark.tinka@seacom.mu> wrote:
> On 18/Apr/20 16:23, Lukas Tribus wrote:
>
> >
> > More about this issue here:
> > https://www.mail-archive.com/nanog@nanog.org/msg104776.html
> >
> > Code with CSCvc84848 fixed will hopefully ship this summer, until then
> > I'm not touching RPKI on IOS(-XE) devices.
>
> Time to check in with them.

So CSCvc84848 actually shipped (in 16.9.6, 16.12.4 and 17.3.2).

The change appears to be that IBGP routes are now tagged as "NotFound"
as opposed to "Valid". This addresses one specific case, doesn't
tackle the root cause and introduces new problems.

It is still intervening in the best path selection, even if
"prefix-validate allow-invalid" is set, despite the documentation
claiming otherwise (see "Use of the Validation State in BGP Best Path
Determination" [1]). Only "prefix-validate disable" really disables
best path intervention madness (because it basically switches it off).

Here's an example of the post-CSCvc84848 behavior:

An AS has both a customer and a peer/transit, but not on the same
router; also the customer routers are RPKI Valid.

Router1 connects to the customer, accepts the RPKI valid routes from
the customers and makes them best-path (and announces them in IBGP).
Router2 connects to your peer/transit, does not consider the customers
routes from Router1, because IBGP routes are NotFound, while the same
routes from peer/transit (EBGP) are actually Valid, so the latter
become best-path.

Before CSCvc84848 the issue was with prefixes not covered by ROA's
(IBGP is always Valid, which trump's EBGP NotFound routes, ignoring
best-path selection).
Now with CSCvc84848 applied, a similar issue exists, just with RPKI
Valid routes instead (IBGP is always NotFound, which is trumped by
EBGP Valid routes, ignoring best-path selection).

I'm assuming this will surprise folks when they bring their networks
to post-CSCvc84848 code and suddenly they don't announce their
customers' valid prefixes to transits anymore. It would be helpful if
the release notes would actually mention CSCvc84848.


The only way to avoid messing with the best-path is to disable
prefix-validation completely. RPKI Invalids can still be discarded in
route-maps, although this disables all the outputs from the BGP table
(like whether a route is valid or notfound, which we may still want to
know).

router bgp ...
bgp rpki server tcp [...]
address-family ipv4
bgp bestpath prefix-validate disable
[...]
route-map RM_EBGP_IN deny 10
match rpki invalid
route-map RM_EBGP_IN permit 20
[...]


Vpnv[46] support and RTR via SSH is still not there.


cheers,
lukas

[1] https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_bgp/configuration/xe-17/irg-xe-17-book/bgp-origin-as-validation.html
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
On 11/28/20 02:32, Ben Maddison via cisco-nsp wrote:
> At this stage, I don't really care if they fix this anymore.
> We're now firmly pointed at the "no more XE in the field" target, and
> this is only one (admittedly large) reason why.

This; for us too.

Simply moving on.

In relation to this particular issue, we've been chasing this since
2014. I mean, seriously...

Mark.

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
Hi Ben,


On Sat, 28 Nov 2020 at 01:32, Ben Maddison <benm@workonline.africa> wrote:
> > router bgp ...
> > bgp rpki server tcp [...]
> > address-family ipv4
> > bgp bestpath prefix-validate disable
> > [...]
> > route-map RM_EBGP_IN deny 10
> > match rpki invalid
> > route-map RM_EBGP_IN permit 20
> > [...]
> >
> Does the route-map 'match' still work here? Which release?
> I remember trying this workaround before our initial rollout of ROV and
> nothing matched that statement when 'prefix-validate disable' was
> configured. I forget the exact release, but that would have been
> 16.9.3-ish.

It works for me in both recent (Amsterdam, 17.03.02) and older (Fuji,
16.09.02) code.

I did not try matching NotFound or Valid, or setting different
locpref's, just denying invalid routes.


> > Vpnv[46] support and RTR via SSH is still not there.
> >
> Hahaha, don't hold your breath. Source interface selection isn't even
> available.

With SSH support we would get source interface selection for free :(

CLI helptext actually mentions SSH username and password and a
"local-port" option, but it's undocumented and unclear how it is
supposed to work...

LAB1(config-router)#$bgp rpki server tcp 1.2.3.4 port 3232 ref 600
password secret ?
username SSH Username
<cr> <cr>
LAB1(config-router)#$bgp rpki server tcp 1.2.3.4 port 3232 ref 600
password secret username user1 password secret2 ?
local-port SSH Local Port
LAB1(config-router)#$


It's probably a leftover from someone trying to get SSH support in.
Unsure why SSH support would be combined with TCP-MD5 support on the
socket (which is what the first password argument is about).


cheers,
lukas
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
On 12/18/20 08:58, Jakob Heitz (jheitz) wrote:
> Hi Lukas, Mark, Ben,
>
> The default bestpath prefix-validate behavior treats invalid routes
> as unfeasible and prefers valid routes over not-found.
>
> The default bestpath prefix-validate behavior cannot be used unless
> all paths of a net have the correct RPKI validity. That can only
> happen if all EBGP sessions into an AS validate their incoming
> routes and apply the RFC8097 extended community.
> If these conditions are not satisfied, then you cannot use the
> bestpath prefix-validate behavior and you must use
> route-maps to process the RPKI validity, like this:
>
> router bgp ...
> bgp rpki server tcp [...]
> address-family ipv4
> bgp bestpath prefix-validate disable
> [...]
> route-map RM_EBGP_IN deny 10
> match rpki invalid
> [...]
>
> I have a proposal to improve the bestpath prefix-validate behavior
> to better match how most operators use it. By a new configuration,
> I would treat valid and not-found with the same preference. Invalid
> would continue to be unfeasible. Then, a received IBGP route without
> the RFC8097 community will be fine.
>
> Thoughts?

What I've been asking Cisco to do since 2014 is to prevent IOS XE from
applying policy by default. This is broken and is in direct violation of
the RFC.

All RPKI policy must only be applied by the operator.

The router has no business using RPKI state as part of its best path
calculation process, unless specifically told to do so by the operator.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
Hi,

On Fri, Dec 18, 2020 at 10:25:30AM +0200, Ben Maddison via cisco-nsp wrote:
> The router should not *act* on validation status unless told to by the
> operator at all.

This!

> I would suggest that the 'bgp bestpath prefix-validate ...' commands be
> deprecated altogether, and be replaced with a single per-afi/safi
> command that simply enables rov-checking (i.e. records the status in the
> RIB, but takes no policy action).
> Everything else can be done in a route-map.

And this!

gert

--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de
Re: RPKI extended-community RFC8097 [ In reply to ]
On 12/18/20 10:25, Ben Maddison wrote:

> Getting from where you are today, to what I'm advocating obviously means
> changing existing behaviors, which can be an unwelcome surprise.

Earlier this year, I did ask Jeff (and Jakob) to help fix the IOS XE
RPKI brokenness with a myriad of suggestions.

There hasn't been much progress (specifically, feedback) on that since
then. I was promised some test code with the recommended fixes, but the
year is almost out, and I can understand that with the pandemic,
priorities may have shifted.

At any rate, these are the suggestions I made to Jeff and Jakob back in
March. Not sure if they are still relevant anymore, as we turned off
RPKI on all our IOS XE boxes, and haven't bothered to keep up with how
it has developed on that platform since:

*****

*  When an RTR session is established with the validator, the router
should be able to accept and store all validation information.
However, it SHOULD NOT automatically use this information to make
routing decisions. So this would be the first thing to fix so we
have a good working base.

* Have a single command that is configurable within BGP to
automatically drop Invalid routes. This is useful if an operator
wants to use RPKI mainly to drop Invalid routes, and has no need to
evaluate other RPKI states. With this command, it negates the need
to build a route-map to achieve the same thing. A sample
configuration for this would be:

        router bgp 1234
            address-family ipv4
                bgp bestpath prefix drop-invalids
            address-family ipv6
                bgp bestpath prefix drop-invalids

* Remove the "bgp bestpath prefix-validate allow-invalid" command, as
with this fix, the operator can decide to do this via a route-map if
they want to. It also takes the gun - that they could use to easily
shoot their foot - out of their hands.

* Remove the "bgp bestpath prefix-validate disable" command, as with
this fix, validation does not occur by default unless the operator
enables this via a route-map, or via the "bgp bestpath prefix
drop-invalids" command.

* All other methods to enable validation should be done via a
route-map only. When a route-map with RPKI match and set conditions
is applied to a BGP session, the router will consult the validation
database as populated by the RTR session, and then apply the policy
on the router accordingly, as dictated by the route-map
configuration. The route-map should be able to apply this policy
both on the inbound and outbound directions of a BGP session.

* An example of what a route-map should be able to do is as per below:

          route-map BGP-POLICY permit|deny 10
            match rpki valid|not-found|invalid
            set rpki valid|not-found|invalid

* So as per the route-map example above, matching an RPKI state ONLY
tells the router to look for any routes in the validation database
that correspond to the match condition defined in the route-map.
This DOES NOT tell the router to do anything with those routes from
a routing perspective. This is CRITICAL. It does, however, allow the
router to export or drop those routes toward an eBGP or iBGP
neighbor if applied in the outbound direction.

* ONLY when the operator uses a "set" condition in the route-map does
the router then apply the corresponding policy when used in the
inbound direction from an eBGP or iBGP neighbor. If the operator
does not use the "set" condition in the route-map, but applies it on
an eBGP or iBGP session in the inbound direction, the router WILL
NOT apply any policy related to RPKI.

* It's VERY CRITICAL to note that the router SHOULD NOT make any
INHERENT DECISIONS on routing policy based on the "match" or "set"
conditions in the route-map. The decision on what to do SHOULD ONLY
be based on whether the route-map is a "permit" or "deny" route-map.
In other words, the router is unaware about the significance of RPKI
state, i.e., Valid, Invalid, NotFound. To the router, those RPKI
states are just arbitrary values with no real meaning. The router
SHOULD ONLY make the decision under one of two circumstances:

            - The route-map's "permit" or "deny" action.
            - The "bgp bestpath prefix drop-invalids" command.

* If an operator tries to configure both the "bgp bestpath prefix
drop-invalids" command and the route-map version of dropping
Invalids with a "deny" action, the router SHOULD prefer the
route-map version over the global "bgp bestpath prefix
drop-invalids" command. In fact, it would be helpful if the router
can throw back an error if it recognizes that the operator has
configured (or has tried to configure) both of these options at the
same time. That way, the operator knows that one of the two has
already been configured, and that the router SHOULD tell the
operator which one will be used (and which one will be ignored) if
both are present in the running configuration.

* Validation state SHOULD ONLY be applied to a route if the operator
has defined "match" and "set" conditions in a route-map in the
inbound direction of the BGP session. If this has not been done, the
router should mark all routes in the RIB/FIB as "RPKI State:
Unverified".

* The router SHOULD be able to allow operators to view a summary of
Valid, Invalid and Not-Found records in the validation database.
This is important in case an operator wants to quickly see what the
validation database thinks is the RPKI state of a route vs. what the
validator said it is. Examples of these commands would be:

            show bgp ipv4 unicast rpki table valid|notfound|invalid
            show bgp ipv6 unicast rpki table valid|notfound|invalid
            show ip bgp rpki table valid|notfound|invalid

Also, as discussed in a previous e-mail, please provide a knob that
allows the operator to configure how long the router will hold all
validation information during a failure of the RTR session. It would be
good if the timeout can be configured to or beyond an hour.

*****

Would welcome any comments/feedback/suggestions from the community on
the suggestions above.

Jakob, would also appreciate if you can let us know how far you and the
team have come with the above suggestions, if at all. Thanks.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
On Fri, 18 Dec 2020 at 22:07, Jakob Heitz (jheitz) via cisco-nsp
<cisco-nsp@puck.nether.net> wrote:

> Testing the RPKI validity in route-map causes BGP REFRESH messages.
> Lots of them.

I think the community largely got blindsided by this, I suspect
marketability of the whole solution would have been a lot poorer if
this argument was thrown around at standardisation. However, that ship
has sailed, we can implement new cheaper methods, but the damage is
done and it will be there long after we've retired.

I know I got blindsided, and it was so obvious, but not a problem I
was aware until a customer complained about excessive refresh. It
would be funny to analyse how much more wattage is drawn because of
this globally. how many early control-plane upgrades. Is it
immaterial or material? I don't know. But it does seem to put some
customers control-planes over the edge.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
Jakob,

It has been a while, but IIRC the original idea for the validation was that
regardless if this is done by configuration enabling pre-best path
eligibility or in route map no path will be dropped. At no point in the BGP
design discussions there was a plan to automatically do any of this. So
your REFRESH story or soft-in alternative sound like the original plan has
somehow changed.

See even if you validate in route map you may just mark it not-eligible or
set higher local pref for VALID etc .... I am not sure how anyone could
come with the idea to just drop there.

So IMHO there is nothing wrong with specification.

It is suboptimal implementation or configuration which needs to be fixed.
It beats me why it is taking so long ...

Thx,
R.

On Fri, Dec 18, 2020 at 9:04 PM Jakob Heitz (jheitz) via cisco-nsp <
cisco-nsp@puck.nether.net> wrote:

>
>
>
> ---------- Forwarded message ----------
> From: "Jakob Heitz (jheitz)" <jheitz@cisco.com>
> To: Ben Maddison <benm@workonline.africa>
> Cc: Lukas Tribus <lukas@ltri.eu>, Mark Tinka <mark.tinka@seacom.mu>,
> Cisco-nsp <cisco-nsp@puck.nether.net>
> Bcc:
> Date: Fri, 18 Dec 2020 20:00:02 +0000
> Subject: RE: [c-nsp] RPKI extended-community RFC8097
> There is an issue with route-maps.
>
> Testing the RPKI validity in route-map causes BGP REFRESH messages.
> Lots of them.
> soft-reconfig helps, but that causes risk of memory exhaustion and does
> not fix the internal CPU usage of evaluating the needed route-maps.
> (soft-reconfig saves a copy of all pre-policy incoming routes).
> Using the validity in the bestpath computation does not cause REFRESHes.
>
> Consider if validity is used in a route-map and the router drops a route.
> When a ROA update comes from the validator, then the route-map needs
> to be re-evaluated to determine if the route can now be used and what
> sets need to be made to it.
> For all routes, not just one.
> Because when a route is dropped, all information about which route was
> dropped is lost.
> The REFRESH must go to any router that has dropped at least one incoming
> route and that tests RPKI validity in its route-map.
>
> We have some work going on in XR to reduce the impact of the REFRESH
> messages and to reduce the risk of memory exhaustion when using
> soft-reconfig.
>
> When validity is used in bestpath computation, invalids are not actually
> dropped. They are made unfeasible. It's effectively dropped, but can
> come back from the dead after a ROA update from the validator.
> Importantly, the route-map has already run on it and does not need to run
> again when the validity changes. Thus no REFRESH.
>
> Regards,
> Jakob.
>
> -----Original Message-----
> From: Ben Maddison <benm@workonline.africa>
> Sent: Friday, December 18, 2020 12:26 AM
> To: Jakob Heitz (jheitz) <jheitz@cisco.com>
> Cc: Lukas Tribus <lukas@ltri.eu>; Mark Tinka <mark.tinka@seacom.mu>;
> Cisco-nsp <cisco-nsp@puck.nether.net>
> Subject: Re: [c-nsp] RPKI extended-community RFC8097
>
> Hi Jakob,
>
> On 12/18, Jakob Heitz (jheitz) wrote:
> > Hi Lukas, Mark, Ben,
> >
> > The default bestpath prefix-validate behavior treats invalid routes
> > as unfeasible and prefers valid routes over not-found.
> >
> > The default bestpath prefix-validate behavior cannot be used unless
> > all paths of a net have the correct RPKI validity. That can only
> > happen if all EBGP sessions into an AS validate their incoming
> > routes and apply the RFC8097 extended community.
>
> And, iff all ASBRs have a consistent set of VRPs from the validation
> caches, which is a very fragile assumption, and the root of most of the
> impact we've seen from this.
>
> > If these conditions are not satisfied, then you cannot use the
> > bestpath prefix-validate behavior and you must use
> > route-maps to process the RPKI validity, like this:
> >
> > router bgp ...
> > bgp rpki server tcp [...]
> > address-family ipv4
> > bgp bestpath prefix-validate disable
> > [...]
> > route-map RM_EBGP_IN deny 10
> > match rpki invalid
> > [...]
> >
> > I have a proposal to improve the bestpath prefix-validate behavior
> > to better match how most operators use it. By a new configuration,
> > I would treat valid and not-found with the same preference. Invalid
> > would continue to be unfeasible. Then, a received IBGP route without
> > the RFC8097 community will be fine.
> >
> > Thoughts?
> >
> That certainly sounds like an improvement (a large one), but it doesn't go
> far enough
> for me.
>
> The router should not *act* on validation status unless told to by the
> operator at all.
> I would suggest that the 'bgp bestpath prefix-validate ...' commands be
> deprecated altogether, and be replaced with a single per-afi/safi
> command that simply enables rov-checking (i.e. records the status in the
> RIB, but takes no policy action).
> Everything else can be done in a route-map.
>
> I have heard the argument from other vendors that "operators want a
> single command to apply, without touching routing policy".
> I think this is false. At least in 2020, you would be very hard pressed
> to find an operator doing ROV, but scared of writing a non-trivial
> routing policy.
>
> Getting from where you are today, to what I'm advocating obviously means
> changing existing behaviors, which can be an unwelcome surprise.
>
> I would suggest the following strategy:
> 1. Introduce a new address-family mode command (something like 'bgp
> rpki-validate origin' to provide syntax space for aspa and others in
> future).
> Make it a no-op if 'bgp bestpath prefix-validate' is enabled, otherwise
> make it enable rov state checking as above.
> 2. Start issuing a warning on the CLI, in logs, etc, when 'bgp bestpath
> prefix-validate'
> is used.
> 3. After some time make 'bgp bestpath prefix-validate' a no-op hidden
> command.
> 4. After some more time, error if 'bgp bestpath prefix-validate' is
> used.
>
> Hope that helps.
>
> Cheers,
>
> Ben
>
>
>
> ---------- Forwarded message ----------
> From: "Jakob Heitz (jheitz) via cisco-nsp" <cisco-nsp@puck.nether.net>
> To: Ben Maddison <benm@workonline.africa>
> Cc: Mark Tinka <mark.tinka@seacom.mu>, Cisco-nsp <
> cisco-nsp@puck.nether.net>
> Bcc:
> Date: Fri, 18 Dec 2020 20:00:02 +0000
> Subject: Re: [c-nsp] RPKI extended-community RFC8097
> _______________________________________________
> cisco-nsp mailing list cisco-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
On 12/19/20 10:45, Saku Ytti wrote:

> I think the community largely got blindsided by this, I suspect
> marketability of the whole solution would have been a lot poorer if
> this argument was thrown around at standardisation. However, that ship
> has sailed, we can implement new cheaper methods, but the damage is
> done and it will be there long after we've retired.
>
> I know I got blindsided, and it was so obvious, but not a problem I
> was aware until a customer complained about excessive refresh. It
> would be funny to analyse how much more wattage is drawn because of
> this globally. how many early control-plane upgrades. Is it
> immaterial or material? I don't know. But it does seem to put some
> customers control-planes over the edge.

We suffered with this a great deal when we used ASR9001's for peering. A
bunch of peers complained about it, to the extent that they had to drop
sessions.

Problem was fixed by moving to the MX204. Not elegant, and an
unnecessary spend.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
On 12/19/20 11:13, Robert Raszuk wrote:

> Jakob,
>
> It has been a while, but IIRC the original idea for the validation was
> that regardless if this is done by configuration enabling pre-best
> path eligibility or in route map no path will be dropped. At no point
> in the BGP design discussions there was a plan to automatically do any
> of this. So your REFRESH story or soft-in alternative sound like the
> original plan has somehow changed.
>
> See even if you validate in route map you may just mark it
> not-eligible or set higher local pref for VALID etc .... I am not sure
> how anyone could come with the idea to just drop there.
>
> So IMHO there is nothing wrong with specification.
>
> It is suboptimal implementation or configuration which needs to be
> fixed. It beats me why it is taking so long ...

Absolutely!

The spec. is clear on nodes only performing validation at the behest of
the operator, and never automatically or inherently.

This is a Cisco-specific issue, and either a mis-interpretation of the
RFC, or a workaround to the impact of the spec. on their implementation.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
Hi,

On Sat, Dec 19, 2020 at 10:13:36AM +0100, Robert Raszuk wrote:
> See even if you validate in route map you may just mark it not-eligible or
> set higher local pref for VALID etc .... I am not sure how anyone could
> come with the idea to just drop there.

In the face of invalid more-specifics, the "local-pref" thing just plain
doesn't work.

So "ineligible or drop for INVALID" is the only option.

We do "drop in route-map", on ASR 9k, and this thread has me thinking if
this was a good idea. As far as I know, no way to set "ineligible" from
a route-map. Is there?

So back to the drawing board.

gert

--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de
Re: RPKI extended-community RFC8097 [ In reply to ]
> As far as I know, no way to set "ineligible" from a route-map. Is there?

A workaround could be to set unreachable next hop instead of dropping :)
That automatically disables such path from best path comparison yet it
keeps in BGP.

But as said implementation could make it easier with a knob.

The question to ask if you want to advertise INVALID paths around ? Even if
not best path once you enable add-paths it may be advertised.

Thx,
R.


On Sat, Dec 19, 2020 at 10:47 AM Gert Doering <gert@greenie.muc.de> wrote:

> Hi,
>
> On Sat, Dec 19, 2020 at 10:13:36AM +0100, Robert Raszuk wrote:
> > See even if you validate in route map you may just mark it not-eligible
> or
> > set higher local pref for VALID etc .... I am not sure how anyone could
> > come with the idea to just drop there.
>
> In the face of invalid more-specifics, the "local-pref" thing just plain
> doesn't work.
>
> So "ineligible or drop for INVALID" is the only option.
>
> We do "drop in route-map", on ASR 9k, and this thread has me thinking if
> this was a good idea. As far as I know, no way to set "ineligible" from
> a route-map. Is there?
>
> So back to the drawing board.
>
> gert
>
> --
> "If was one thing all people took for granted, was conviction that if you
> feed honest figures into a computer, honest figures come out. Never
> doubted
> it myself till I met a computer with a sense of humor."
> Robert A. Heinlein, The Moon is a Harsh
> Mistress
>
> Gert Doering - Munich, Germany
> gert@greenie.muc.de
>
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
Hi,

On Sat, Dec 19, 2020 at 11:02:16AM +0100, Robert Raszuk wrote:
> > As far as I know, no way to set "ineligible" from a route-map. Is there?
>
> A workaround could be to set unreachable next hop instead of dropping :)
> That automatically disables such path from best path comparison yet it
> keeps in BGP.

Interesting trick. Need to test.

> But as said implementation could make it easier with a knob.
>
> The question to ask if you want to advertise INVALID paths around ? Even if
> not best path once you enable add-paths it may be advertised.

We do not use add-paths today, but if we did, I wouldn't want INVALIDs
advertised anywhere.

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de
Re: RPKI extended-community RFC8097 [ In reply to ]
On Sat, 19 Dec 2020 at 10:40, Gert Doering <gert@greenie.muc.de> wrote:
>
> Hi,
>
> On Sat, Dec 19, 2020 at 10:13:36AM +0100, Robert Raszuk wrote:
> > See even if you validate in route map you may just mark it not-eligible or
> > set higher local pref for VALID etc .... I am not sure how anyone could
> > come with the idea to just drop there.
>
> In the face of invalid more-specifics, the "local-pref" thing just plain
> doesn't work.
>
> So "ineligible or drop for INVALID" is the only option.
>
> We do "drop in route-map", on ASR 9k, and this thread has me thinking if
> this was a good idea. As far as I know, no way to set "ineligible" from
> a route-map. Is there?

soft-reconfig inbound always amounts to 100 MB of memory consumption
for a v4 + v6 full feed as of last week on 32-bit XR. I can live with
100MB of memory consumption per full feed, so I'm doing soft-reconfig
inbound always everywhere. This also helps with troubleshooting.


lukas
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
On Sat, 19 Dec 2020 at 13:45, Lukas Tribus <lukas@ltri.eu> wrote:

> soft-reconfig inbound always amounts to 100 MB of memory consumption
> for a v4 + v6 full feed as of last week on 32-bit XR. I can live with
> 100MB of memory consumption per full feed, so I'm doing soft-reconfig
> inbound always everywhere. This also helps with troubleshooting.

It is also DRAM exhaustion attack vector. But of course routers are
extremely fragile and anyone motivated can easily bring them down by
plethora of methods. Even with max-prefix it may be, as max-prefix may
mean before or after policy count, depending on platform and
configuration toggle.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
Hello Jakob,


On Fri, 18 Dec 2020 at 07:58, Jakob Heitz (jheitz) <jheitz@cisco.com> wrote:
>
> Hi Lukas, Mark, Ben,
>
> The default bestpath prefix-validate behavior treats invalid routes
> as unfeasible and prefers valid routes over not-found.
>
> The default bestpath prefix-validate behavior cannot be used unless
> all paths of a net have the correct RPKI validity. That can only
> happen if all EBGP sessions into an AS validate their incoming
> routes and apply the RFC8097 extended community.

First of all let me say that I appreciate your effort to improve the
situation very much, thank you for this.

The problem is that this is not a "default". If this would be a
default, we could just disable it. However we can only disable it by
disabling validation, which is incorrect according to the
documentation (see below).

I understand that this works just fine in a greenfield deployment
(just deploy the network with everything enabled), but I never
understood how this implementation should work in existing networks.

An existing network would need to enable validation on all routers
SIMULTANEOUSLY. If this is a small shop, let's say 5 routers, maybe
they could bypass change management procedures, get both engineers in
a room, each engineer enables RPKI validation on 2 routers as fast as
possible, and maybe they could get Janice from Accounting to help on
the fifth terminal for Router nr 5 ("just hit the Enter button,
Janice") on the terminal of the fifth router. The deployment would be
quasi-simultaneously and routing anomalies would be limited to a short
period of time.

But you cannot make a configuration change atomically across an entire
backbone, and also even if this would be technically possible, nobody
would ever approve this change because the risk is just too great.

"We will shutdown our entire fully redundant and geographically
diverse AS for 2 hours on Saturday as we deploy RPKI ROV" <-- that's
not how SP's deploy new features.

Network operators need to be able to roll this out over a period of
time and rollback the configuration on individual nodes, based on the
operational requirements.

I understand that there are configuration knobs that impact best path
selection that will have such effects, like the bgp best-path med
knobs. That doesn't mean RPKI ROV has to be.



> If these conditions are not satisfied, then you cannot use the
> bestpath prefix-validate behavior and you must use
> route-maps to process the RPKI validity, like this:
>
> router bgp ...
> bgp rpki server tcp [...]
> address-family ipv4
> bgp bestpath prefix-validate disable
> [...]
> route-map RM_EBGP_IN deny 10
> match rpki invalid
> [...]

Which also disables any visibility into the RPKI status in the routing
table. At least we can drop RPKI invalids this way, although it does
come with the complete lack of visibility. Also there is the issue you
mention in your subsequent email (router refresh, CPU load, memory
exhaustion with soft-reconfig inbound always).



> I have a proposal to improve the bestpath prefix-validate behavior
> to better match how most operators use it. By a new configuration,
> I would treat valid and not-found with the same preference. Invalid
> would continue to be unfeasible. Then, a received IBGP route without
> the RFC8097 community will be fine.


All those years the documentation already claims that this should work
by just allowing invalids [1]:

> - You can completely disable the validation of prefixes [...]
> - You can allow an invalid prefix to be used as the BGP best path [...]
> [...]
> During BGP best path selection, the default behavior, ***if neither of the
> above options is configured***, is that the system will prefer prefixes in
> the following order:

It's just not actually implemented this way.

If you want to introduce a new configuration knob, please just do it
EXACTLY like IOS-XR: if the new code path is used, there needs to be
NO best path intervention, NONE whatsoever, not even for INVALIDs.

XR only modifies the bestpath when enabling the (afaik non-default) option:

bgp bestpath origin-as use validity behavior

It will not change anything based on the RPKI validation status
without this option enabled, not even for invalids.


Let's not introduce yet another bandaid change for one single specific
routing anomalie, that still doesn't fully disable best path
intervention. Let's get it right this time.



Thank you!

Lukas


[1] https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_bgp/configuration/xe-17/irg-xe-17-book/bgp-origin-as-validation.html
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
> Robert Raszuk
> Sent: Saturday, December 19, 2020 10:02 AM
>
> > As far as I know, no way to set "ineligible" from a route-map. Is
there?
>
> A workaround could be to set unreachable next hop instead of dropping :)
> That automatically disables such path from best path comparison yet it
keeps
> in BGP.
>
> But as said implementation could make it easier with a knob.
>
Yeah I was thinking along the same lines, keep it in BGP for sure, just not
use it for actual data routing, (if that's the desired local policy).
(was thinking filtering invalids on RIB->FIB level, BGP table map/selective
route download -and limit the churn between RIB and FIB on a local box
rather than BGP between boxes).
-is there an option to filter invalids at the table map/selective route
download attach point?

> The question to ask if you want to advertise INVALID paths around ? Even
if
> not best path once you enable add-paths it may be advertised.
>
Well I'd say yes?
-to leave BGP in a role of a messenger, then let each individual box/AS
decide locally what to do (with the message).

adam


_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
> Saku Ytti
> Sent: Saturday, December 19, 2020 12:57 PM
>
> On Sat, 19 Dec 2020 at 13:45, Lukas Tribus <lukas@ltri.eu> wrote:
>
> > soft-reconfig inbound always amounts to 100 MB of memory consumption
> > for a v4 + v6 full feed as of last week on 32-bit XR. I can live with
> > 100MB of memory consumption per full feed, so I'm doing soft-reconfig
> > inbound always everywhere. This also helps with troubleshooting.
>
> It is also DRAM exhaustion attack vector. But of course routers are
extremely
> fragile and anyone motivated can easily bring them down by plethora of
> methods. Even with max-prefix it may be, as max-prefix may mean before or
> after policy count, depending on platform and configuration toggle.
>
Good point, also all the potential attribute filtering (in XR) would it be
applied prior to accepting the route into soft-reconfig version of the
table?
I guess the enhance bgp error correction would kick in prior to letting the
malformed update (i.e. at the update process time).

adam


_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
> was thinking filtering invalids on RIB->FIB level,

Globally this would not work as in global RIB only best path is installed
(unless you run multipath).

Even for bRIB it would be the same.

The selection of eligible paths must happen prior to best path selection
for a given net.

Thx,
R.


On Mon, Dec 21, 2020 at 5:03 PM <adamv0025@netconsultings.com> wrote:

> > Robert Raszuk
> > Sent: Saturday, December 19, 2020 10:02 AM
> >
> > > As far as I know, no way to set "ineligible" from a route-map. Is
> there?
> >
> > A workaround could be to set unreachable next hop instead of dropping :)
> > That automatically disables such path from best path comparison yet it
> keeps
> > in BGP.
> >
> > But as said implementation could make it easier with a knob.
> >
> Yeah I was thinking along the same lines, keep it in BGP for sure, just not
> use it for actual data routing, (if that's the desired local policy).
> (was thinking filtering invalids on RIB->FIB level, BGP table map/selective
> route download -and limit the churn between RIB and FIB on a local box
> rather than BGP between boxes).
> -is there an option to filter invalids at the table map/selective route
> download attach point?
>
> > The question to ask if you want to advertise INVALID paths around ? Even
> if
> > not best path once you enable add-paths it may be advertised.
> >
> Well I'd say yes?
> -to leave BGP in a role of a messenger, then let each individual box/AS
> decide locally what to do (with the message).
>
> adam
>
>
>
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: RPKI extended-community RFC8097 [ In reply to ]
On Mon, 21 Dec 2020 at 18:07, <adamv0025@netconsultings.com> wrote:

> Good point, also all the potential attribute filtering (in XR) would it be
> applied prior to accepting the route into soft-reconfig version of the
> table?

IOS-XR is only post-policy. So whatever you reject does not contribute
towards the limit, allowing DRAM exhaustion attack.
SROS is only pre-policy. So if someone leaks bad prefixes you reject
in policy, it's still going to be flap, potentially BGP reset attack.
JunOS supports pre and post.

Both are needed as they protect from different issues.
--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/