Mailing List Archive

rib-groups && VPN reflection
Hello all.

I figure this topic is a fundamental and probably frequently asked/answered
although it's new problem space for me. I thought I'd consult the font of
knowledge here to seek any advice.

Environment: MX, JUNOS 15.1F6
Headline requirement: Leak EBGP routes from global inet.0 into a VPN (in
order to implement off-ramp/on-ramp for DDoS protection traffic
conditioning).

Experience:
The challenge is quite simple on the surface. Use a rib-group directive on
the EBGP peer to group together inet.0 and the VRF.inet.0 together as
import-rib/Adj-Rib-In for the peer. Indeed this works as you would expect,
and received routes appear in both inet.0 and VRF.inet.0

But the problem is that if rpd is also configured with any of:
- IBGP reflection for inetvpn family
- EBGP for inetvpn
- advertise-from-main-vpn-table,

then any leaked routes, while being present in the VRF, do not get
advertised internally to other PE VPN routing tables.

The cause seems to be that these features cause the mechanics of
advertising VPN routes internally to change. These features bring in a
requirement for rpd to retain VPN routes in their "native" inet-vpn form,
rather than simply consult the origin routing-instsances and synthesise on
demand so that the interaction with reflection clients or EBGP peers can be
handled.

So when these features are enabled, rpd opportunistically switches to a
mode where it goes to the trouble of cloning the instance-based vanilla
routes as inetvpn within bgp.l3vpn.0 or equiv.

Indeed battle-scarred Juniper engineers are probably familiar with this
document that offers counsel for how to maintain uptime in the face of this
optimisation gear-shift:
https://www.juniper.net/documentation/en_US/junos/topics/example/bgp-vpn-session-flap-prevention.html

I can understand and appreciate this, even if I might not like it.

But the abstraction seems to be incomplete. The method of copying routes to
bgp.l3vpn.0 is similar if not identical, under-the-hood, to the initial
rib-group operation I am performing at route source to leak the original
inet.0 route and this route, as seen in the VRF.inet.0 table, becomes a
Secondary route.

As such, it apparently isn't candidate for further cloning/copying into
bgp.l3vpn.0, and as a consequence the "leaked" route doesn't actually make
it into the VPN tables of other PEs.

The document suggests a workaround of maintaining the original route in
inet.0, but sadly for my use case, the whole premise of the leak operation
is to ultimately result in a global table inet.0 redirect elsewhere, so
depending on inet.0 route selection is a bit fragile for this.

My question to others is, is this a well-known man-trap that I am naively
unaware of? Is it simply the case that best practice to get reflection off
of production VRF-hosting PEs is actually mandatory here, or are others
surprised by this apparent feature clash? Can I reasonably expect it to be
addressed further down the software road? Or is there another, perhaps
better, way of achieving my objective?

Any thoughts appreciated.

-- Adam.
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: rib-groups && VPN reflection [ In reply to ]
Hi,

On 18.04.2019 10:13, Adam Chappell wrote:
> But the abstraction seems to be incomplete. The method of copying routes to
> bgp.l3vpn.0 is similar if not identical, under-the-hood, to the initial
> rib-group operation I am performing at route source to leak the original
> inet.0 route and this route, as seen in the VRF.inet.0 table, becomes a
> Secondary route.
>
> As such, it apparently isn't candidate for further cloning/copying into
> bgp.l3vpn.0, and as a consequence the "leaked" route doesn't actually make
> it into the VPN tables of other PEs.

Yes, L3VPN under the hood is more or less rib-groups in disguise. There is actually a command i forgot which shows you the internal rib-groups it uses to do the L3VPN magic.

> My question to others is, is this a well-known man-trap that I am naively
> unaware of? Is it simply the case that best practice to get reflection off
> of production VRF-hosting PEs is actually mandatory here, or are others
> surprised by this apparent feature clash? Can I reasonably expect it to be
> addressed further down the software road? Or is there another, perhaps
> better, way of achieving my objective?

This behavior is probably deeply rooted in the architecture, so i would not expect it to change.

I faced the same issue when building a DDoS Mitigation ON/OFF Ramp setup.

My workaround was to bring up an lt-interface and run a routing protocol between VRF and inet.0 announcing all the routes you need.
As i did not want to the actual traffic to forward over that lt interfaces (and stealing BW from the PFE) i created a policy to change the next-hop to a specific dummy next-hop IP.

That dummy-next-hop IP used next-table XYZ and pointed directly into the table i wanted. Once the routes are learned and resolved the Forwarding table points directly into the other VRF/Table and i could not see any problems in terms of performance or similiar with this.

The setup is running in production for a couple of years now. It is a bit ugly and violates the "4am Rule" where any on Call Engineer woken at 4am should immediately understand what is going on, but well it is what it is ;)

--
Kind Regards
Tobias Heister
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: rib-groups && VPN reflection [ In reply to ]
On Thu, 18 Apr 2019 at 14:25, Tobias Heister <lists@tobias-heister.de>
wrote:

> Hi,
>
> On 18.04.2019 10:13, Adam Chappell wrote:
> > But the abstraction seems to be incomplete. The method of copying routes
> to
> > bgp.l3vpn.0 is similar if not identical, under-the-hood, to the initial
> > rib-group operation I am performing at route source to leak the original
> > inet.0 route and this route, as seen in the VRF.inet.0 table, becomes a
> > Secondary route.
> >
> > As such, it apparently isn't candidate for further cloning/copying into
> > bgp.l3vpn.0, and as a consequence the "leaked" route doesn't actually
> make
> > it into the VPN tables of other PEs.
>
> Yes, L3VPN under the hood is more or less rib-groups in disguise. There is
> actually a command i forgot which shows you the internal rib-groups it uses
> to do the L3VPN magic.
>
> > My question to others is, is this a well-known man-trap that I am naively
> > unaware of? Is it simply the case that best practice to get reflection
> off
> > of production VRF-hosting PEs is actually mandatory here, or are others
> > surprised by this apparent feature clash? Can I reasonably expect it to
> be
> > addressed further down the software road? Or is there another, perhaps
> > better, way of achieving my objective?
>
> This behavior is probably deeply rooted in the architecture, so i would
> not expect it to change.
>
> I faced the same issue when building a DDoS Mitigation ON/OFF Ramp setup.
>
> My workaround was to bring up an lt-interface and run a routing protocol
> between VRF and inet.0 announcing all the routes you need.
> As i did not want to the actual traffic to forward over that lt interfaces
> (and stealing BW from the PFE) i created a policy to change the next-hop to
> a specific dummy next-hop IP.
>
> That dummy-next-hop IP used next-table XYZ and pointed directly into the
> table i wanted. Once the routes are learned and resolved the Forwarding
> table points directly into the other VRF/Table and i could not see any
> problems in terms of performance or similiar with this.
>

FWIW, I've also built a quite similar solution for this use case.

Best regards,
Johannes
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: rib-groups && VPN reflection [ In reply to ]
reminds me of the time we also tested it...

On April 18, 2019 17:35:45 Johannes Resch <jr@xor.at> wrote:

> On Thu, 18 Apr 2019 at 14:25, Tobias Heister <lists@tobias-heister.de>
> wrote:
>
>> Hi,
>>
>> On 18.04.2019 10:13, Adam Chappell wrote:
>>> But the abstraction seems to be incomplete. The method of copying routes
>> to
>>> bgp.l3vpn.0 is similar if not identical, under-the-hood, to the initial
>>> rib-group operation I am performing at route source to leak the original
>>> inet.0 route and this route, as seen in the VRF.inet.0 table, becomes a
>>> Secondary route.
>>>
>>> As such, it apparently isn't candidate for further cloning/copying into
>>> bgp.l3vpn.0, and as a consequence the "leaked" route doesn't actually
>> make
>>> it into the VPN tables of other PEs.
>>
>> Yes, L3VPN under the hood is more or less rib-groups in disguise. There is
>> actually a command i forgot which shows you the internal rib-groups it uses
>> to do the L3VPN magic.
>>
>>> My question to others is, is this a well-known man-trap that I am naively
>>> unaware of? Is it simply the case that best practice to get reflection
>> off
>>> of production VRF-hosting PEs is actually mandatory here, or are others
>>> surprised by this apparent feature clash? Can I reasonably expect it to
>> be
>>> addressed further down the software road? Or is there another, perhaps
>>> better, way of achieving my objective?
>>
>> This behavior is probably deeply rooted in the architecture, so i would
>> not expect it to change.
>>
>> I faced the same issue when building a DDoS Mitigation ON/OFF Ramp setup.
>>
>> My workaround was to bring up an lt-interface and run a routing protocol
>> between VRF and inet.0 announcing all the routes you need.
>> As i did not want to the actual traffic to forward over that lt interfaces
>> (and stealing BW from the PFE) i created a policy to change the next-hop to
>> a specific dummy next-hop IP.
>>
>> That dummy-next-hop IP used next-table XYZ and pointed directly into the
>> table i wanted. Once the routes are learned and resolved the Forwarding
>> table points directly into the other VRF/Table and i could not see any
>> problems in terms of performance or similiar with this.
>
> FWIW, I've also built a quite similar solution for this use case.
>
> Best regards,
> Johannes
> _______________________________________________
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp



_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: rib-groups && VPN reflection [ In reply to ]
> Adam Chappell
> Sent: Thursday, April 18, 2019 9:13 AM
>
> Hello all.
>
> I figure this topic is a fundamental and probably frequently
asked/answered
> although it's new problem space for me. I thought I'd consult the font of
> knowledge here to seek any advice.
>
> Environment: MX, JUNOS 15.1F6
> Headline requirement: Leak EBGP routes from global inet.0 into a VPN (in
> order to implement off-ramp/on-ramp for DDoS protection traffic
> conditioning).
>
> Experience:
> The challenge is quite simple on the surface. Use a rib-group directive on
the
> EBGP peer to group together inet.0 and the VRF.inet.0 together as import-
> rib/Adj-Rib-In for the peer. Indeed this works as you would expect, and
> received routes appear in both inet.0 and VRF.inet.0
>
> But the problem is that if rpd is also configured with any of:
> - IBGP reflection for inetvpn family
> - EBGP for inetvpn
> - advertise-from-main-vpn-table,
>
> then any leaked routes, while being present in the VRF, do not get
> advertised internally to other PE VPN routing tables.
>
> The cause seems to be that these features cause the mechanics of
> advertising VPN routes internally to change. These features bring in a
> requirement for rpd to retain VPN routes in their "native" inet-vpn form,
> rather than simply consult the origin routing-instsances and synthesise on
> demand so that the interaction with reflection clients or EBGP peers can
be
> handled.
>
> So when these features are enabled, rpd opportunistically switches to a
> mode where it goes to the trouble of cloning the instance-based vanilla
> routes as inetvpn within bgp.l3vpn.0 or equiv.
>
> Indeed battle-scarred Juniper engineers are probably familiar with this
> document that offers counsel for how to maintain uptime in the face of
this
> optimisation gear-shift:
> https://www.juniper.net/documentation/en_US/junos/topics/example/bg
> p-vpn-session-flap-prevention.html
>
> I can understand and appreciate this, even if I might not like it.
>
> But the abstraction seems to be incomplete. The method of copying routes
> to
> bgp.l3vpn.0 is similar if not identical, under-the-hood, to the initial
rib-group
> operation I am performing at route source to leak the original
> inet.0 route and this route, as seen in the VRF.inet.0 table, becomes a
> Secondary route.
>
> As such, it apparently isn't candidate for further cloning/copying into
> bgp.l3vpn.0, and as a consequence the "leaked" route doesn't actually make
> it into the VPN tables of other PEs.
>
> The document suggests a workaround of maintaining the original route in
> inet.0, but sadly for my use case, the whole premise of the leak operation
is
> to ultimately result in a global table inet.0 redirect elsewhere, so
depending
> on inet.0 route selection is a bit fragile for this.
>
> My question to others is, is this a well-known man-trap that I am naively
> unaware of? Is it simply the case that best practice to get reflection
off of
> production VRF-hosting PEs is actually mandatory here, or are others
> surprised by this apparent feature clash? Can I reasonably expect it to
be
> addressed further down the software road? Or is there another, perhaps
> better, way of achieving my objective?
>
> Any thoughts appreciated.
>
I'm not sure I understand your objective, so just to confirm.
Is your objective to leak route from routing table A to routing table B
while being able to advertise the leaked route from table B to other PEs
(where the route is expected to land in table B).
If that's the case then this is not allowed as it can form routing loops.
Instead one is expected to set the RTs on export from table A so that other
PEs can import these in the desired table.
This is where the use of inet.0 is troublesome -so in that case one is
expected to do the route leaking on all remote ends.
But you can see the pattern here, advertising is done from the originating
table and then the "leaking" is supposed to be done on/by the remote end.

adam



_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: rib-groups && VPN reflection [ In reply to ]
I think you've got it clear, Adam. Take routes that are originally destined
for inet.0, create a rib-group in order to leak them into a secondary
routing instance, and then expect normal L3VPN behaviour for those
route-instance routes. (The idea is to drop the routes into an overlay
topology, which includes a multi-homed network element, with an interface
and BGP relationship to inet.0 in order to influence it and "override"
based on original route properties.).

Yes, of course, loops are possible, but this is perfectly possible, and
policed by all the usual routing protocol mechanisms, IFF the route table
advertisement mode is non-reflector mode (meaning that routes are exported
directly from the VRFs and not from the bgp.l3vpn.0 holding table).

It's this change in functionality and behaviour based on other features
that disappoints me most here. KB32423 describes the situation, I've since
found, and it advises me to off the reflection. Off-box reflection, for
VPN, is probably the best idea anyway, but it's a shame that it isn't
clearer in the feature documentation that this road has serious pitfalls.

I do understand your point on using RTs to determine remote PE table
destination. But there's no easy way to take undecorated inet.0 routes and
annotate with RTs, is there? Even if there was, I have to assume that the
original route in inet.0 can/will be displaced. That was actually the very
nice thing about rib-groups: the two copies of the route do not share any
route selection fate.

Appreciate the feedback from all. Haven't quite given up yet.

-- Adam.

On Sun, 21 Apr 2019 at 11:22, <adamv0025@netconsultings.com> wrote:

>
> I'm not sure I understand your objective, so just to confirm.
> Is your objective to leak route from routing table A to routing table B
> while being able to advertise the leaked route from table B to other PEs
> (where the route is expected to land in table B).
> If that's the case then this is not allowed as it can form routing loops.
> Instead one is expected to set the RTs on export from table A so that other
> PEs can import these in the desired table.
> This is where the use of inet.0 is troublesome -so in that case one is
> expected to do the route leaking on all remote ends.
> But you can see the pattern here, advertising is done from the originating
> table and then the "leaking" is supposed to be done on/by the remote end.
>
> adam
>
>
>
>

--

-- Adam.
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp