Mailing List Archive

RE: Newbie x Cisco IOS-XR x ROV: BCP to not harassing peer(s)
To address the risk of somebody exhausting your memory by dumping a ton of routes on you,
we added two new options to "soft-reconfiguration inbound" in IOS-XR.

RPKI-dropped-only
Saves a copy of only the routes dropped by an RPKI validation-state test in neighbor-in route-policy.

RPKI-tested-only
Saves a copy of only the routes tested in an RPKI validation-state test in neighbor-in route-policy.

This was released in 7.3.1 in Feb 2021.

The bug CSCwb17937 was fixed in 7.5.2, just released. Fixed a few other things in 7.5.2 also.
Tomoya, apologies that you had a terrible time with it.


Regards,
Jakob.

-----Original Message-----
Date: Wed, 11 May 2022 14:31:28 -0700
From: Randy Bush <randy@psg.com>
To: Pirawat WATANAPONGSE via NANOG <nanog@nanog.org>
Subject: Re: Newbie x Cisco IOS-XR x ROV: BCP to not harassing peer(s)
and upstream(s)
Message-ID: <m2lev7kb7z.wl-randy@psg.com>
Content-Type: text/plain; charset=US-ASCII

> Is setting 'Soft Reconfiguration' enough for me to keep ROV running?

yes, should be.

> If not, is there any other solution?

yes. jakob says he has implemented
https://datatracker.ietf.org/doc/draft-ietf-sidrops-rov-no-rr/, though i
do not known in what xr image(s)

randy
Re: Newbie x Cisco IOS-XR x ROV: BCP to not harassing peer(s) [ In reply to ]
On 5/12/22 23:40, Jakob Heitz (jheitz) via NANOG wrote:

> To address the risk of somebody exhausting your memory by dumping a ton of routes on you,
> we added two new options to "soft-reconfiguration inbound" in IOS-XR.
>
> RPKI-dropped-only
> Saves a copy of only the routes dropped by an RPKI validation-state test in neighbor-in route-policy.
>
> RPKI-tested-only
> Saves a copy of only the routes tested in an RPKI validation-state test in neighbor-in route-policy.
>
> This was released in 7.3.1 in Feb 2021.
>
> The bug CSCwb17937 was fixed in 7.5.2, just released. Fixed a few other things in 7.5.2 also.
> Tomoya, apologies that you had a terrible time with it.

Awesome!

Mark.
Re: Newbie x Cisco IOS-XR x ROV: BCP to not harassing peer(s) [ In reply to ]
On Fri, 13 May 2022 at 00:44, Jakob Heitz (jheitz) via NANOG
<nanog@nanog.org> wrote:

> RPKI-dropped-only
> Saves a copy of only the routes dropped by an RPKI validation-state test in neighbor-in route-policy.
>
> RPKI-tested-only
> Saves a copy of only the routes tested in an RPKI validation-state test in neighbor-in route-policy.

What does this mean? If any term refers to validation-state, the route
gets stored?

Eg.

if validation-state is valid then
pass
else
drop


a) Would 'RPKI-dropped-only' store everything or nothing?
b) Would 'RPKI-tested-only' store everything?

--
++ytti
RE: Newbie x Cisco IOS-XR x ROV: BCP to not harassing peer(s) [ In reply to ]
'RPKI-tested-only' will store all routes that encounter a 'validation-state' test
in the inbound route policy. In that case, when an RPKI server updates a VRP to the
router, it can re-run the inbound policy from the stored route and not require a
refresh request to be sent.

This option saves memory if you use a coarse filter in the route-policy before
the validation test. For example, you use a peer-locking filter to drop peer
routes from your customers before they hit the validation-state test. Then
a massive route leak won't chew up soft-reconfiguration memory.

If a validation-state test drops a route and that route is not stored by
soft-reconfiguration, then when the RPKI server updates any VRP, the router
needs to send a route-refresh request.

'RPKI-dropped-only' causes the dropped routes to be stored. This will prevent
the unnecessary route-refreshes described above. It does not prevent all
route-refreshes, but uses significantly less memory than 'RPKI-tested-only'

Regards,
Jakob.

-----Original Message-----
From: Saku Ytti <saku@ytti.fi>
Sent: Friday, May 13, 2022 12:36 AM
To: Jakob Heitz (jheitz) <jheitz@cisco.com>
Cc: nanog@nanog.org
Subject: Re: Newbie x Cisco IOS-XR x ROV: BCP to not harassing peer(s)

On Fri, 13 May 2022 at 00:44, Jakob Heitz (jheitz) via NANOG
<nanog@nanog.org> wrote:

> RPKI-dropped-only
> Saves a copy of only the routes dropped by an RPKI validation-state test in neighbor-in route-policy.
>
> RPKI-tested-only
> Saves a copy of only the routes tested in an RPKI validation-state test in neighbor-in route-policy.

What does this mean? If any term refers to validation-state, the route
gets stored?

Eg.

if validation-state is valid then
pass
else
drop


a) Would 'RPKI-dropped-only' store everything or nothing?
b) Would 'RPKI-tested-only' store everything?

--
++ytti
Re: Newbie x Cisco IOS-XR x ROV: BCP to not harassing peer(s) [ In reply to ]
On 5/13/22 23:16, Jakob Heitz (jheitz) via NANOG wrote:


> 'RPKI-tested-only' will store all routes that encounter a 'validation-state' test
> in the inbound route policy. In that case, when an RPKI server updates a VRP to the
> router, it can re-run the inbound policy from the stored route and not require a
> refresh request to be sent.
>
> This option saves memory if you use a coarse filter in the route-policy before
> the validation test. For example, you use a peer-locking filter to drop peer
> routes from your customers before they hit the validation-state test. Then
> a massive route leak won't chew up soft-reconfiguration memory.
>
> If a validation-state test drops a route and that route is not stored by
> soft-reconfiguration, then when the RPKI server updates any VRP, the router
> needs to send a route-refresh request.
>
> 'RPKI-dropped-only' causes the dropped routes to be stored. This will prevent
> the unnecessary route-refreshes described above. It does not prevent all
> route-refreshes, but uses significantly less memory than 'RPKI-tested-only'

Jakob, thank you and your team for quickly implementing this. It is most
appreciated.

I hope someone from the IOS XE team is working on it, too :-).

Mark.
Re: Newbie x Cisco IOS-XR x ROV: BCP to not harassing peer(s) [ In reply to ]
On Sat, 14 May 2022 at 00:17, Jakob Heitz (jheitz) <jheitz@cisco.com> wrote:

Hey Jakob,

> 'RPKI-tested-only' will store all routes that encounter a 'validation-state' test
> in the inbound route policy. In that case, when an RPKI server updates a VRP to the
> router, it can re-run the inbound policy from the stored route and not require a
> refresh request to be sent.

> 'RPKI-dropped-only' causes the dropped routes to be stored. This will prevent
> the unnecessary route-refreshes described above. It does not prevent all
> route-refreshes, but uses significantly less memory than 'RPKI-tested-only'

I'm sorry, but I am unable to reason what these answers mean in
context of question that was:


---
if validation-state is valid then
pass
else
drop
----

I am assuming

a) RPKI-tested-only would be same normal, and keep every single route
b) RPKI-dropped-only would not keep anything (but it also might keep
everything and be same as a))

That is, in this specific scenario, as far as I understand, there is
no effect on the optimisations.



Just to clarify why this type of policy may not be insane. IOS-XR has
a 300k prefix limit for prefix-set, this limit is regularly hit by low
quality as-set. By low quality I mean almost all as-set expand to
unnecessarily large prefix-set, because as-set tend to be 'add only',
there is no incentive to remove, so they just grow over time and do
not represent in a meaningful manner the set of prefixes neighbours
might advertise.
And if we abstract what we the operators are actually doing, no one is
doing prefix filtering, what everyone does is build AS-tree, by
starting recursion from some as-set. So this AS-tree is the source of
truth, no prefixes at all, prefixes are almost incentidental. After we
have this AS-tree, we flatten it and for each element we ask for a
route object with that origin. And then send this list to routers.

Understanding what we actually do here, offers a mechanism for config
size reduction as well as a standardized way to programmatically
deploy those prefix-lists, by (ab)using RTR for this. We can fill all
gaps from IRR data that RPKI data leaves us, then send a complete set
of DFZ origins to routers, this allows us to accept only valid
prefixes. Further the as-graph we created and flattened we can
implement per-neighbour, which is trivial size compared to prefix-set
size.

Now compiling those AS path filters are regexp may not be so cheap,
but some NOS offer cheap way to implement such AS filtering at scale:
https://www.juniper.net/documentation/us/en/software/junos/routing-policy/topics/topic-map/Improve-as-path-lookup.html

If we do this 100% complete RTR and AS-set filter per neighbor, then
we actually have better routing security than we have in the most
canonical way, because we are enforcing origin:prefix relation, which
we are not enforcing when we dump larger and low quality prefix-sets
to routers. This makes us much less vulnerable to the low quality
as-set both in operational manner by not inflating config sizes and
cause commits to fail and by improving routing security.


>
> Regards,
> Jakob.
>
> -----Original Message-----
> From: Saku Ytti <saku@ytti.fi>
> Sent: Friday, May 13, 2022 12:36 AM
> To: Jakob Heitz (jheitz) <jheitz@cisco.com>
> Cc: nanog@nanog.org
> Subject: Re: Newbie x Cisco IOS-XR x ROV: BCP to not harassing peer(s)
>
> On Fri, 13 May 2022 at 00:44, Jakob Heitz (jheitz) via NANOG
> <nanog@nanog.org> wrote:
>
> > RPKI-dropped-only
> > Saves a copy of only the routes dropped by an RPKI validation-state test in neighbor-in route-policy.
> >
> > RPKI-tested-only
> > Saves a copy of only the routes tested in an RPKI validation-state test in neighbor-in route-policy.
>
> What does this mean? If any term refers to validation-state, the route
> gets stored?
>
> Eg.
>
> if validation-state is valid then
> pass
> else
> drop
>
>
> a) Would 'RPKI-dropped-only' store everything or nothing?
> b) Would 'RPKI-tested-only' store everything?
>
> --
> ++ytti



--
++ytti
Re: Newbie x Cisco IOS-XR x ROV: BCP to not harassing peer(s) [ In reply to ]
On 14/05/2022 00:16, Jakob Heitz (jheitz) via NANOG wrote:


> 'RPKI-dropped-only' causes the dropped routes to be stored. This will prevent
> the unnecessary route-refreshes described above. It does not prevent all
> route-refreshes, but uses significantly less memory than 'RPKI-tested-only'
>
> Regards,
> Jakob.

In the end, the reason for all this RPKI-thingy is to prevent route
spoofing by malicious actors. It sure would be nice if someone from the
top 20: https://asrank.caida.org/ would be able to have an auto-updated
site that showed all RPKI dropped from their end.

This would complement https://bgpstream.crosswork.cisco.com/ for those
of us who want to know who is trying to hijack our routes at the core.

Regards,
Hank
Re: Newbie x Cisco IOS-XR x ROV: BCP to not harassing peer(s) [ In reply to ]
Hank Nussbacher wrote on 14/05/2022 19:15:
> In the end, the reason for all this RPKI-thingy is to prevent route
> spoofing by malicious actors.

a malicious actor will spoof the origin AS. The aim of RPKI to help
stop mis-origination of prefixes, and the root cause of most of this is
accidental.

Nick
Re: Newbie x Cisco IOS-XR x ROV: BCP to not harassing peer(s) [ In reply to ]
> In the end, the reason for all this RPKI-thingy is to prevent route
> spoofing by malicious actors.

sigh. for my quarterly posting of the same many year old text

To be clear, as people keep calling BGP security 'RPKI',

RPKI

The RPKI is an X.509 based hierarchy [RFC 6481] which is congruent
with the internet IP address allocation administration, the IANA,
RIRs, ISPs, ... It is just a database, but is the substrate on
which the next two mechanisms are based. It is currently deployed
in all five administrative regions.

RPKI-based Origin Validation (ROV)

RPKI-based Origin Validation [RFC 6811] uses some of the RPKI data
to allow a router to verify that the autonomous system originating
an IP address prefix is in fact authorized to do so. This is not
crypto checked so can be violated. But it should prevent the vast
majority of accidental 'hijackings' on the internet today, e.g. the
famous Pakistani accidental announcement of YouTube's address space.
RPKI-based origin validation is in shipping code from AlcaLu, Cisco,
Juniper, and possibly others.

BGPsec

RPKI-based Path Validation, AKA BGPsec, a future technology still
being designed [draft-ietf-sidr-bgpsec-overview], uses the full
crypto information of the RPKI to make up for the embarrassing
mistake that, like much of the internet BGP was designed with no
thought to securing the BGP protocol itself from being
gamed/violated. It allows a receiver of a BGP announcement to
cryptographically validate that the autonomous systems through which
the announcement passed were indeed those which the sender/forwarder
at each hop intended.

Sorry to drone on, but these three really need to be differentiated.
RE: Newbie x Cisco IOS-XR x ROV: BCP to not harassing peer(s) [ In reply to ]
Saku,

You have two questions. I'll address the second one first.

Beginning in IOS-XR 7.3.1, there is a new O(log n) scalable way to test for autonomous system numbers (ASN) in route-policy. ASNs can be grouped into an as-set as follows:

as-set foo
64496,
64497
end-set
!
route-policy bar
if not as-path originates-from foo then
drop
endif
pass
end-policy

The first question:
If you use several tests in your route-policy and put the validation-state
test last, then any route that gets dropped before the validation-state
test is reached will not be saved with
"soft-reconfiguration inbound RPKI-tested-only".
For example:

route-policy bar
if not as-path originates-from foo then
drop
endif
if validation-state is invalid then
drop
endif
pass
end-policy

Regards,
Jakob.

-----Original Message-----
From: Saku Ytti <saku@ytti.fi>
Sent: Saturday, May 14, 2022 12:09 AM
To: Jakob Heitz (jheitz) <jheitz@cisco.com>
Cc: nanog@nanog.org
Subject: Re: Newbie x Cisco IOS-XR x ROV: BCP to not harassing peer(s)

On Sat, 14 May 2022 at 00:17, Jakob Heitz (jheitz) <jheitz@cisco.com> wrote:

Hey Jakob,

> 'RPKI-tested-only' will store all routes that encounter a 'validation-state' test
> in the inbound route policy. In that case, when an RPKI server updates a VRP to the
> router, it can re-run the inbound policy from the stored route and not require a
> refresh request to be sent.

> 'RPKI-dropped-only' causes the dropped routes to be stored. This will prevent
> the unnecessary route-refreshes described above. It does not prevent all
> route-refreshes, but uses significantly less memory than 'RPKI-tested-only'

I'm sorry, but I am unable to reason what these answers mean in
context of question that was:


---
if validation-state is valid then
pass
else
drop
----

I am assuming

a) RPKI-tested-only would be same normal, and keep every single route
b) RPKI-dropped-only would not keep anything (but it also might keep
everything and be same as a))

That is, in this specific scenario, as far as I understand, there is
no effect on the optimisations.



Just to clarify why this type of policy may not be insane. IOS-XR has
a 300k prefix limit for prefix-set, this limit is regularly hit by low
quality as-set. By low quality I mean almost all as-set expand to
unnecessarily large prefix-set, because as-set tend to be 'add only',
there is no incentive to remove, so they just grow over time and do
not represent in a meaningful manner the set of prefixes neighbours
might advertise.
And if we abstract what we the operators are actually doing, no one is
doing prefix filtering, what everyone does is build AS-tree, by
starting recursion from some as-set. So this AS-tree is the source of
truth, no prefixes at all, prefixes are almost incentidental. After we
have this AS-tree, we flatten it and for each element we ask for a
route object with that origin. And then send this list to routers.

Understanding what we actually do here, offers a mechanism for config
size reduction as well as a standardized way to programmatically
deploy those prefix-lists, by (ab)using RTR for this. We can fill all
gaps from IRR data that RPKI data leaves us, then send a complete set
of DFZ origins to routers, this allows us to accept only valid
prefixes. Further the as-graph we created and flattened we can
implement per-neighbour, which is trivial size compared to prefix-set
size.

Now compiling those AS path filters are regexp may not be so cheap,
but some NOS offer cheap way to implement such AS filtering at scale:
https://www.juniper.net/documentation/us/en/software/junos/routing-policy/topics/topic-map/Improve-as-path-lookup.html

If we do this 100% complete RTR and AS-set filter per neighbor, then
we actually have better routing security than we have in the most
canonical way, because we are enforcing origin:prefix relation, which
we are not enforcing when we dump larger and low quality prefix-sets
to routers. This makes us much less vulnerable to the low quality
as-set both in operational manner by not inflating config sizes and
cause commits to fail and by improving routing security.


>
> Regards,
> Jakob.
>
> -----Original Message-----
> From: Saku Ytti <saku@ytti.fi>
> Sent: Friday, May 13, 2022 12:36 AM
> To: Jakob Heitz (jheitz) <jheitz@cisco.com>
> Cc: nanog@nanog.org
> Subject: Re: Newbie x Cisco IOS-XR x ROV: BCP to not harassing peer(s)
>
> On Fri, 13 May 2022 at 00:44, Jakob Heitz (jheitz) via NANOG
> <nanog@nanog.org> wrote:
>
> > RPKI-dropped-only
> > Saves a copy of only the routes dropped by an RPKI validation-state test in neighbor-in route-policy.
> >
> > RPKI-tested-only
> > Saves a copy of only the routes tested in an RPKI validation-state test in neighbor-in route-policy.
>
> What does this mean? If any term refers to validation-state, the route
> gets stored?
>
> Eg.
>
> if validation-state is valid then
> pass
> else
> drop
>
>
> a) Would 'RPKI-dropped-only' store everything or nothing?
> b) Would 'RPKI-tested-only' store everything?
>
> --
> ++ytti



--
++ytti
Re: Newbie x Cisco IOS-XR x ROV: BCP to not harassing peer(s) [ In reply to ]
15.05.22 00:19, Nick Hilliard ????:
> a malicious actor will spoof the origin AS.  The aim of RPKI to help
> stop mis-origination of prefixes, and the root cause of most of this is
> accidental.

To make a working hijack of the routed prefix (for sniffing traffic,
DDoS or something similar), you have to announce a more specific
prefix(es). It can be denied by RPKI.

If you signed RPKI prefix is still unannounced - yes, somebody can
hijack it by forging the origin ASN - that's quite easy.
Re: Newbie x Cisco IOS-XR x ROV: BCP to not harassing peer(s) [ In reply to ]
On Tue, 24 May 2022 at 11:23, Max Tulyev <maxtul@netassist.ua> wrote:

> To make a working hijack of the routed prefix (for sniffing traffic,
> DDoS or something similar), you have to announce a more specific
> prefix(es). It can be denied by RPKI.
>
> If you signed RPKI prefix is still unannounced - yes, somebody can
> hijack it by forging the origin ASN - that's quite easy.

This axiomatically assumes first come, first serve, which is obviously
not complete understanding of BGP best path algorithm.

--
++ytti
RE: Newbie x Cisco IOS-XR x ROV: BCP to not harassing peer(s) [ In reply to ]
This attack will work very well until the victim starts advertising
its prefix. The victim may not notice the fake advertisement because the fake
advertisement will not reach the victim AS due to AS-path loop checking.

So potential victims must advertise all prefixes that they register in
RPKI or subscribe to an Internet monitoring service to detect the
fake advertisements.

And don't forget maxlen. You must advertise in BGP every prefix
covered by maxlen.

Regards,
Jakob.

-----Original Message-----
From: Saku Ytti <saku@ytti.fi>

On Tue, 24 May 2022 at 11:23, Max Tulyev <maxtul@netassist.ua> wrote:

> To make a working hijack of the routed prefix (for sniffing traffic,
> DDoS or something similar), you have to announce a more specific
> prefix(es). It can be denied by RPKI.
>
> If you signed RPKI prefix is still unannounced - yes, somebody can
> hijack it by forging the origin ASN - that's quite easy.

This axiomatically assumes first come, first serve, which is obviously
not complete understanding of BGP best path algorithm.

--
++ytti
Re: Newbie x Cisco IOS-XR x ROV: BCP to not harassing peer(s) [ In reply to ]
> On 25 May 2022, at 5:45 am, Jakob Heitz (jheitz) via NANOG <nanog@nanog.org> wrote:
>
> This attack will work very well until the victim starts advertising
> its prefix. The victim may not notice the fake advertisement because the fake
> advertisement will not reach the victim AS due to AS-path loop checking.


Often the best forms of attack are ones that are scoped in locality. Advertising the
same prefix from a different location in BGP may create a localised preference to follow the
synthesised route which is not visible everywhere. Sometimes this is exactly what the
attacker wants to achieve.

Geoff