Mailing List Archive

08.10.2020 20:00, Drew Weaver ?????:
> Hello,
>
> I have two sets of core routers due to a transition period from one set to the other.
>
> I have noticed that when there is a connectivity disruption between the two sets of core routers and one upstream peering/edge router:
>
> Oct 7 12:01:14 EDT: %OSPF-5-ADJCHG: Process 1, Nbr <removed> on TenGigabitEthernet2/1 from FULL to DOWN, Neighbor Down: BFD node down
>
> <Two+ minutes of null routing traffic for no reason>
>
> Oct 7 12:03:29 EDT: %BGP-5-ADJCHANGE: neighbor <removed> Down BGP Notification sent
>
> What I expect to happen is:
>
> The route to the peering edge router's loopback interface is withdrawn when OSPF/OSPFv3 closes.
> The core router will close the BGP session when the route to the dead peering edge router is withdrawn and will begin using one of the 5 other copies of the same route that it has.
>
> Things I have implemented to avoid this:
>
> The peering edge router and the core routers peer with IP addresses that are only learnable via OSPF and aren't available in any other protocol. [It's not part of our IP space]
>
> I guess I just need a sanity check regarding whether my assumption that it shouldn't be null routing traffic for 2+ minutes if one of our peering edge routers gets hit by a meteor is correct since we have 5 peering edge routers.

This may depend on BGP synchronization that could be disabled by default. Did you enable it?


_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

08.10.2020 20:00, Drew Weaver wrote:

> I have two sets of core routers due to a transition period from one set to the other.
>
> I have noticed that when there is a connectivity disruption between the two sets of core routers and one upstream peering/edge router:
>
> Oct 7 12:01:14 EDT: %OSPF-5-ADJCHG: Process 1, Nbr <removed> on TenGigabitEthernet2/1 from FULL to DOWN, Neighbor Down: BFD node down
>
> <Two+ minutes of null routing traffic for no reason>
>
> Oct 7 12:03:29 EDT: %BGP-5-ADJCHANGE: neighbor <removed> Down BGP Notification sent
>
> What I expect to happen is:
>
> The route to the peering edge router's loopback interface is withdrawn when OSPF/OSPFv3 closes.
> The core router will close the BGP session when the route to the dead peering edge router is withdrawn and will begin using one of the 5 other copies of the same route that it has.
>
> Things I have implemented to avoid this:
>
> The peering edge router and the core routers peer with IP addresses that are only learnable via OSPF and aren't available in any other protocol. [It's not part of our IP space]
>
> I guess I just need a sanity check regarding whether my assumption that it shouldn't be null routing traffic for 2+ minutes if one of our peering edge routers gets hit by a meteor is correct since we have 5 peering edge routers.

This may depend on BGP synchronization that could be disabled by default. Did you enable it?


_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

I wonder if bgp neighboring isn't timing out quickly enough for your
satisfaction and holding routes for a few minutes


-----Original Message-----
From: cisco-nsp <cisco-nsp-bounces@puck.nether.net> On Behalf Of Drew Weaver
Sent: Thursday, October 8, 2020 8:01 AM
To: 'cisco-nsp@puck.nether.net' <cisco-nsp@puck.nether.net>
Subject: [c-nsp] Sanity check OSPF/BGP

Hello,

I have two sets of core routers due to a transition period from one set to
the other.

I have noticed that when there is a connectivity disruption between the two
sets of core routers and one upstream peering/edge router:

Oct 7 12:01:14 EDT: %OSPF-5-ADJCHG: Process 1, Nbr <removed> on
TenGigabitEthernet2/1 from FULL to DOWN, Neighbor Down: BFD node down

<Two+ minutes of null routing traffic for no reason>

Oct 7 12:03:29 EDT: %BGP-5-ADJCHANGE: neighbor <removed> Down BGP
Notification sent

What I expect to happen is:

The route to the peering edge router's loopback interface is
withdrawn when OSPF/OSPFv3 closes.
The core router will close the BGP session when the route to
the dead peering edge router is withdrawn and will begin using one of the 5
other copies of the same route that it has.

Things I have implemented to avoid this:

The peering edge router and the core routers peer with IP
addresses that are only learnable via OSPF and aren't available in any other
protocol. [It's not part of our IP space]

I guess I just need a sanity check regarding whether my assumption that it
shouldn't be null routing traffic for 2+ minutes if one of our peering edge
routers gets hit by a meteor is correct since we have 5 peering edge
routers.

Thanks in advance friends,
-Drew



_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

You didn't specify the platform or code version it is running. Would help
with platform specifics


On Thu, Oct 8, 2020 at 11:47 AM <aaron1@gvtc.com> wrote:

> I wonder if bgp neighboring isn't timing out quickly enough for your
> satisfaction and holding routes for a few minutes
>
>
> -----Original Message-----
> From: cisco-nsp <cisco-nsp-bounces@puck.nether.net> On Behalf Of Drew
> Weaver
> Sent: Thursday, October 8, 2020 8:01 AM
> To: 'cisco-nsp@puck.nether.net' <cisco-nsp@puck.nether.net>
> Subject: [c-nsp] Sanity check OSPF/BGP
>
> Hello,
>
> I have two sets of core routers due to a transition period from one set to
> the other.
>
> I have noticed that when there is a connectivity disruption between the two
> sets of core routers and one upstream peering/edge router:
>
> Oct 7 12:01:14 EDT: %OSPF-5-ADJCHG: Process 1, Nbr <removed> on
> TenGigabitEthernet2/1 from FULL to DOWN, Neighbor Down: BFD node down
>
> <Two+ minutes of null routing traffic for no reason>
>
> Oct 7 12:03:29 EDT: %BGP-5-ADJCHANGE: neighbor <removed> Down BGP
> Notification sent
>
> What I expect to happen is:
>
> The route to the peering edge router's loopback interface is
> withdrawn when OSPF/OSPFv3 closes.
> The core router will close the BGP session when the route to
> the dead peering edge router is withdrawn and will begin using one of the 5
> other copies of the same route that it has.
>
> Things I have implemented to avoid this:
>
> The peering edge router and the core routers peer with IP
> addresses that are only learnable via OSPF and aren't available in any
> other
> protocol. [It's not part of our IP space]
>
> I guess I just need a sanity check regarding whether my assumption that it
> shouldn't be null routing traffic for 2+ minutes if one of our peering edge
> routers gets hit by a meteor is correct since we have 5 peering edge
> routers.
>
> Thanks in advance friends,
> -Drew
>
>
>
> _______________________________________________
> cisco-nsp mailing list cisco-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
> _______________________________________________
> cisco-nsp mailing list cisco-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Previously, when we had this issue it was determined that it was because there was still a route to the BGP neighbor in the routing table (because the neighbor IP was part of our IP announcements and it would wait until the hold time to expire) but we got around that particular issue by using RFC1918 IPs for the neighbors and BGP Next hop address tracking took care of the rest (made it much faster, like 1-2 seconds) but it seems like in our current architecture with the new core it’s operating differently.

It’s almost like the new core routers are continuing to be seen as a path to this neighbor by the old core routers and vice versa even though OSPF went down on both of them at the same exact time.

It’s a mystery for sure.

From: Aaron <dudepron@gmail.com>
Sent: Thursday, October 8, 2020 11:52 AM
To: Aaron <aaron1@gvtc.com>
Cc: Drew Weaver <drew.weaver@thenap.com>; cisco-nsp <cisco-nsp@puck.nether.net>
Subject: Re: [c-nsp] Sanity check OSPF/BGP

You didn't specify the platform or code version it is running. Would help with platform specifics


On Thu, Oct 8, 2020 at 11:47 AM <aaron1@gvtc.com<mailto:aaron1@gvtc.com>> wrote:
I wonder if bgp neighboring isn't timing out quickly enough for your
satisfaction and holding routes for a few minutes


-----Original Message-----
From: cisco-nsp <cisco-nsp-bounces@puck.nether.net<mailto:cisco-nsp-bounces@puck.nether.net>> On Behalf Of Drew Weaver
Sent: Thursday, October 8, 2020 8:01 AM
To: 'cisco-nsp@puck.nether.net<mailto:cisco-nsp@puck.nether.net>' <cisco-nsp@puck.nether.net<mailto:cisco-nsp@puck.nether.net>>
Subject: [c-nsp] Sanity check OSPF/BGP

Hello,

I have two sets of core routers due to a transition period from one set to
the other.

I have noticed that when there is a connectivity disruption between the two
sets of core routers and one upstream peering/edge router:

Oct 7 12:01:14 EDT: %OSPF-5-ADJCHG: Process 1, Nbr <removed> on
TenGigabitEthernet2/1 from FULL to DOWN, Neighbor Down: BFD node down

<Two+ minutes of null routing traffic for no reason>

Oct 7 12:03:29 EDT: %BGP-5-ADJCHANGE: neighbor <removed> Down BGP
Notification sent

What I expect to happen is:

The route to the peering edge router's loopback interface is
withdrawn when OSPF/OSPFv3 closes.
The core router will close the BGP session when the route to
the dead peering edge router is withdrawn and will begin using one of the 5
other copies of the same route that it has.

Things I have implemented to avoid this:

The peering edge router and the core routers peer with IP
addresses that are only learnable via OSPF and aren't available in any other
protocol. [It's not part of our IP space]

I guess I just need a sanity check regarding whether my assumption that it
shouldn't be null routing traffic for 2+ minutes if one of our peering edge
routers gets hit by a meteor is correct since we have 5 peering edge
routers.

Thanks in advance friends,
-Drew



_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net<mailto:cisco-nsp@puck.nether.net>
https://puck.nether.net/mailman/listinfo/cisco-nsp<https://urldefense.proofpoint.com/v2/url?u=https-3A__puck.nether.net_mailman_listinfo_cisco-2Dnsp&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=tarlS9_N9KOgbzThRUtBamnS96bi7lxKn6B4IhBwQGE&s=dEDvP0xBa0A44BIbRgqX1ZXJIWYtAST8C81l0z6ami0&e=>
archive at http://puck.nether.net/pipermail/cisco-nsp/<https://urldefense.proofpoint.com/v2/url?u=http-3A__puck.nether.net_pipermail_cisco-2Dnsp_&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=tarlS9_N9KOgbzThRUtBamnS96bi7lxKn6B4IhBwQGE&s=xxFbEjuX0EZUFK8EZRtPJdxtXaBxHgMkrklDwV0cY9A&e=>

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net<mailto:cisco-nsp@puck.nether.net>
https://puck.nether.net/mailman/listinfo/cisco-nsp<https://urldefense.proofpoint.com/v2/url?u=https-3A__puck.nether.net_mailman_listinfo_cisco-2Dnsp&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=tarlS9_N9KOgbzThRUtBamnS96bi7lxKn6B4IhBwQGE&s=dEDvP0xBa0A44BIbRgqX1ZXJIWYtAST8C81l0z6ami0&e=>
archive at http://puck.nether.net/pipermail/cisco-nsp/<https://urldefense.proofpoint.com/v2/url?u=http-3A__puck.nether.net_pipermail_cisco-2Dnsp_&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=tarlS9_N9KOgbzThRUtBamnS96bi7lxKn6B4IhBwQGE&s=xxFbEjuX0EZUFK8EZRtPJdxtXaBxHgMkrklDwV0cY9A&e=>
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Thursday, 8 October, 2020 17:08, "Drew Weaver" <drew.weaver@thenap.com> said:

> Previously, when we had this issue it was determined that it was because there was
> still a route to the BGP neighbor in the routing table (because the neighbor IP
> was part of our IP announcements and it would wait until the hold time to expire)
> but we got around that particular issue by using RFC1918 IPs for the neighbors and
> BGP Next hop address tracking took care of the rest (made it much faster, like 1-2
> seconds) but it seems like in our current architecture with the new core
> it’s operating differently.
>
> It’s almost like the new core routers are continuing to be seen as a path to
> this neighbor by the old core routers and vice versa even though OSPF went down on
> both of them at the same exact time.
>
> It’s a mystery for sure.

Back to basics - what does "show ip route <remote-loopback>" give you during that two-minute window?

Is there a default route somewhere in the network that could be flagging that remote loopback as still reachable until the BGP timers expire?

Regards,
Tim.


_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Those are both good questions. I didn't catch it when it occurred but I will remove traffic from the PE router and then simulate the failure.

Thanks,
-Drew


-----Original Message-----
From: cisco-nsp <cisco-nsp-bounces@puck.nether.net> On Behalf Of tim@pelican.org
Sent: Thursday, October 8, 2020 12:27 PM
To: 'cisco-nsp@puck.nether.net' <cisco-nsp@puck.nether.net>
Subject: Re: [c-nsp] Sanity check OSPF/BGP

On Thursday, 8 October, 2020 17:08, "Drew Weaver" <drew.weaver@thenap.com> said:

> Previously, when we had this issue it was determined that it was
> because there was still a route to the BGP neighbor in the routing
> table (because the neighbor IP was part of our IP announcements and it
> would wait until the hold time to expire) but we got around that
> particular issue by using RFC1918 IPs for the neighbors and BGP Next
> hop address tracking took care of the rest (made it much faster, like
> 1-2
> seconds) but it seems like in our current architecture with the new
> core it’s operating differently.
>
> It’s almost like the new core routers are continuing to be seen as a
> path to this neighbor by the old core routers and vice versa even
> though OSPF went down on both of them at the same exact time.
>
> It’s a mystery for sure.

Back to basics - what does "show ip route <remote-loopback>" give you during that two-minute window?

Is there a default route somewhere in the network that could be flagging that remote loopback as still reachable until the BGP timers expire?

Regards,
Tim.


_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net https://urldefense.proofpoint.com/v2/url?u=https-3A__puck.nether.net_mailman_listinfo_cisco-2Dnsp&d=DwIGaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=_cSub4IuCJHW5XB79UH_LPiDCfSxIPffvNUrwnxvRvY&s=Kl9GNS56uGL5HQZUmIUfahKk_TD4gtJS5tDOWHYouq4&e=
archive at https://urldefense.proofpoint.com/v2/url?u=http-3A__puck.nether.net_pipermail_cisco-2Dnsp_&d=DwIGaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=OPufM5oSy-PFpzfoijO_w76wskMALE1o4LtA3tMGmuw&m=_cSub4IuCJHW5XB79UH_LPiDCfSxIPffvNUrwnxvRvY&s=015GrnASWjArHeY-MuBADAzBLm9LzWlzvETEmquqtcw&e=
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

> Drew Weaver
> Sent: Thursday, October 8, 2020 2:01 PM
> What I expect to happen is:
>
> The route to the peering edge router's loopback interface is
> withdrawn when OSPF/OSPFv3 closes.
> The core router will close the BGP session when the route to
the dead
> peering edge router is withdrawn and will begin using one of the 5 other
> copies of the same route that it has.
>

Number of things come to mind since you provided no details regarding the
setup

Case A)
If all 5 peering points are not advertising best-external prefixes -then
there's only a single path for each of the 700K prefixes in the entire AS
via one of the 5 peering points.
-in case one peering point fails all prefixes it offered a best path for
will be withdrawn from all BGP speakers in the AS at OSPF convergence
speeds, but then the remaining 4 peering points needs to realize they now
have the overall best path for a given prefix and start advertising it to
all BGP speakers in the AS -tedious process that converges at "BGP-speed".

Case B)
If all 5 peering points are advertising best-external prefixes and all BGP
speakers in the AS already have all 5 paths available in RIB, but none of
the BGP speakers has hierarchical FIB so there's a direct correlation
between a prefix and it's NH,
-in case one peering point fails all prefixes it offered a best path for
will be withdrawn from all BGP speakers in the AS at OSPF convergence
speeds, but now each BGP speaker will need to painstakingly update its FIB
on a prefix by prefix bases for each of the each of the 700K prefixes.

Case C)
If all 5 peering points are advertising best-external prefixes and all BGP
speakers in the AS already have all 5 paths available in RIB, and all BGP
speakers not even have hierarchical FIBs but also PIC-CORE enabled where a
backup path for each prefix is programmed in FIB,
-in case one peering point fails all prefixes it offered a best path for
will be withdrawn from all BGP speakers in the AS at OSPF convergence speeds
and each BGP speaker will then just need to change 5 HN pointers to point to
remaining 4 peering points in FIB.


Note,
The above assumes full mesh between all BGP speakers or otherwise assumes
the RR infrastructure emulates full-mesh with regards to prefix distribution
to all BGP speakers in the AS via one of the several available mechanisms.


Adam



_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Hello,

For clarity's sake:

Each of the 5 edge/peering routers have the full routing table installed and they are totally unaware of one another as far as BGP is concerned

Downstream from there, the view from any of the four core routers is this:

Neighbor V MsgRcvd MsgSent InQ OutQ Up/Down State PfxRcd PfxAcc
192.168.222.25 4 45082982 171835 0 0 6d05h Estab 812608 812608
192.168.222.26 4 56573846 173130 0 0 18d04h Estab 812623 812623
192.168.222.27 4 45082982 171835 0 0 6d02h Estab 812609 812609
192.168.222.28 4 56573846 173130 0 0 118d04h Estab 812625 812625
192.168.222.29 4 45082982 171835 0 0 7d02h Estab 812607 812607

The issue I was running into and asking about was regarding the delay between when OSPF closes (next-hop is no longer reachable) and when the next-hop that is no longer reachable stops being used as a route to a destination.

Not only is the next hop unreachable once OSPF closes, there isn't even a route to that next hop anymore.

So the reason I asked the question was to validate my thinking that if there is no longer a route to the next-hop than the router shouldn't be waiting for the hold timer to expire prior to selecting a different path.

But I still need to validate a few things.

Thanks,
-Drew


-----Original Message-----
From: adamv0025@netconsultings.com <adamv0025@netconsultings.com>
Sent: Monday, October 12, 2020 10:40 AM
To: Drew Weaver <drew.weaver@thenap.com>; cisco-nsp@puck.nether.net
Subject: RE: [c-nsp] Sanity check OSPF/BGP

> Drew Weaver
> Sent: Thursday, October 8, 2020 2:01 PM What I expect to happen is:
>
> The route to the peering edge router's loopback
> interface is withdrawn when OSPF/OSPFv3 closes.
> The core router will close the BGP session when the
> route to
the dead
> peering edge router is withdrawn and will begin using one of the 5
> other copies of the same route that it has.
>

Number of things come to mind since you provided no details regarding the setup

Case A)
If all 5 peering points are not advertising best-external prefixes -then there's only a single path for each of the 700K prefixes in the entire AS via one of the 5 peering points.
-in case one peering point fails all prefixes it offered a best path for will be withdrawn from all BGP speakers in the AS at OSPF convergence speeds, but then the remaining 4 peering points needs to realize they now have the overall best path for a given prefix and start advertising it to all BGP speakers in the AS -tedious process that converges at "BGP-speed".

Case B)
If all 5 peering points are advertising best-external prefixes and all BGP speakers in the AS already have all 5 paths available in RIB, but none of the BGP speakers has hierarchical FIB so there's a direct correlation between a prefix and it's NH, -in case one peering point fails all prefixes it offered a best path for will be withdrawn from all BGP speakers in the AS at OSPF convergence speeds, but now each BGP speaker will need to painstakingly update its FIB on a prefix by prefix bases for each of the each of the 700K prefixes.

Case C)
If all 5 peering points are advertising best-external prefixes and all BGP speakers in the AS already have all 5 paths available in RIB, and all BGP speakers not even have hierarchical FIBs but also PIC-CORE enabled where a backup path for each prefix is programmed in FIB, -in case one peering point fails all prefixes it offered a best path for will be withdrawn from all BGP speakers in the AS at OSPF convergence speeds and each BGP speaker will then just need to change 5 HN pointers to point to remaining 4 peering points in FIB.


Note,
The above assumes full mesh between all BGP speakers or otherwise assumes the RR infrastructure emulates full-mesh with regards to prefix distribution to all BGP speakers in the AS via one of the several available mechanisms.


Adam



_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

I've typically seen this happen when there is a covering route causing the next-hop tracking to not work correctly. It happens quite a bit when there is a local null0 route covering the NH. You can use a route-map to specify only OSPF routes are used under the specific AFI.

You can also look at the state of a next-hop using "show bgp nexthops"

Thanks,
Phil

?On 10/13/20, 2:58 PM, "cisco-nsp on behalf of Drew Weaver" <cisco-nsp-bounces@puck.nether.net on behalf of drew.weaver@thenap.com> wrote:

Hello,

For clarity's sake:

Each of the 5 edge/peering routers have the full routing table installed and they are totally unaware of one another as far as BGP is concerned

Downstream from there, the view from any of the four core routers is this:

Neighbor V MsgRcvd MsgSent InQ OutQ Up/Down State PfxRcd PfxAcc
192.168.222.25 4 45082982 171835 0 0 6d05h Estab 812608 812608
192.168.222.26 4 56573846 173130 0 0 18d04h Estab 812623 812623
192.168.222.27 4 45082982 171835 0 0 6d02h Estab 812609 812609
192.168.222.28 4 56573846 173130 0 0 118d04h Estab 812625 812625
192.168.222.29 4 45082982 171835 0 0 7d02h Estab 812607 812607

The issue I was running into and asking about was regarding the delay between when OSPF closes (next-hop is no longer reachable) and when the next-hop that is no longer reachable stops being used as a route to a destination.

Not only is the next hop unreachable once OSPF closes, there isn't even a route to that next hop anymore.

So the reason I asked the question was to validate my thinking that if there is no longer a route to the next-hop than the router shouldn't be waiting for the hold timer to expire prior to selecting a different path.

But I still need to validate a few things.

Thanks,
-Drew


-----Original Message-----
From: adamv0025@netconsultings.com <adamv0025@netconsultings.com>
Sent: Monday, October 12, 2020 10:40 AM
To: Drew Weaver <drew.weaver@thenap.com>; cisco-nsp@puck.nether.net
Subject: RE: [c-nsp] Sanity check OSPF/BGP

> Drew Weaver
> Sent: Thursday, October 8, 2020 2:01 PM What I expect to happen is:
>
> The route to the peering edge router's loopback
> interface is withdrawn when OSPF/OSPFv3 closes.
> The core router will close the BGP session when the
> route to
the dead
> peering edge router is withdrawn and will begin using one of the 5
> other copies of the same route that it has.
>

Number of things come to mind since you provided no details regarding the setup

Case A)
If all 5 peering points are not advertising best-external prefixes -then there's only a single path for each of the 700K prefixes in the entire AS via one of the 5 peering points.
-in case one peering point fails all prefixes it offered a best path for will be withdrawn from all BGP speakers in the AS at OSPF convergence speeds, but then the remaining 4 peering points needs to realize they now have the overall best path for a given prefix and start advertising it to all BGP speakers in the AS -tedious process that converges at "BGP-speed".

Case B)
If all 5 peering points are advertising best-external prefixes and all BGP speakers in the AS already have all 5 paths available in RIB, but none of the BGP speakers has hierarchical FIB so there's a direct correlation between a prefix and it's NH, -in case one peering point fails all prefixes it offered a best path for will be withdrawn from all BGP speakers in the AS at OSPF convergence speeds, but now each BGP speaker will need to painstakingly update its FIB on a prefix by prefix bases for each of the each of the 700K prefixes.

Case C)
If all 5 peering points are advertising best-external prefixes and all BGP speakers in the AS already have all 5 paths available in RIB, and all BGP speakers not even have hierarchical FIBs but also PIC-CORE enabled where a backup path for each prefix is programmed in FIB, -in case one peering point fails all prefixes it offered a best path for will be withdrawn from all BGP speakers in the AS at OSPF convergence speeds and each BGP speaker will then just need to change 5 HN pointers to point to remaining 4 peering points in FIB.


Note,
The above assumes full mesh between all BGP speakers or otherwise assumes the RR infrastructure emulates full-mesh with regards to prefix distribution to all BGP speakers in the AS via one of the several available mechanisms.


Adam



_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/


_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/