Mailing List Archive: BGP router process using way more memory on one system

BGP router process using way more memory on one system

May 24, 2020, 11:20 AM

Post #1 of 3 (644 views)

Hello,

We have two routers that have a mirrored configuration. Peers, BGP configuration, everything. Exactly the same [except for IP addresses]

One of the routers BGP router process is holding 617576024. The other is holding 577596716.

The one that is holding more appears to be suffering from an out of memory condition.

I am planning on rebooting it but before I do is there any known way of freeing up enough memory to allow basic virtual exec processes to execute?

I've tried basic things like shutting down BGP peers, etc but even though the total memory that BGP says it's using goes down.. it still won't free up the memory.

Thanks in advance.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: BGP router process using way more memory on one system [ In reply to ]

nick at foobar

May 25, 2020, 12:50 AM

Post #2 of 3 (640 views)

Permalink

Drew Weaver wrote on 24/05/2020 19:20:
> We have two routers that have a mirrored configuration. Peers, BGP
> configuration, everything. Exactly the same [except for IP
> addresses]
>
> One of the routers BGP router process is holding 617576024. The other
> is holding 577596716.
>
> The one that is holding more appears to be suffering from an out of
> memory condition.

There were a couple of releases where the ipv4_rib process had a
persistent memory leak. Try this:

Router# admin process restart ipv4_rib

This is non service affecting - restarting the process temporarily stops
FIB reprogramming, then does a full RIB reload from all RIB sources,
then does a FIB check across the device. I.e. it's safer to do this than
to hobble along with OOM errors.

Nick
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: BGP router process using way more memory on one system [ In reply to ]

drew.weaver at thenap

May 28, 2020, 5:12 AM

Post #3 of 3 (628 views)

Permalink

I'll leave this here incase it helps anyone but I was able to get it to respond to a few simple validation commands by just clearing a BGP session.

Thanks,
-Drew

-----Original Message-----
From: Nick Hilliard <nick@foobar.org>
Sent: Monday, May 25, 2020 3:51 AM
To: Drew Weaver <drew.weaver@thenap.com>
Cc: 'cisco-nsp@puck.nether.net' <cisco-nsp@puck.nether.net>
Subject: Re: [c-nsp] BGP router process using way more memory on one system

Drew Weaver wrote on 24/05/2020 19:20:
> We have two routers that have a mirrored configuration. Peers, BGP
> configuration, everything. Exactly the same [except for IP addresses]
>
> One of the routers BGP router process is holding 617576024. The other
> is holding 577596716.
>
> The one that is holding more appears to be suffering from an out of
> memory condition.

There were a couple of releases where the ipv4_rib process had a persistent memory leak. Try this:

Router# admin process restart ipv4_rib

This is non service affecting - restarting the process temporarily stops FIB reprogramming, then does a full RIB reload from all RIB sources, then does a FIB check across the device. I.e. it's safer to do this than to hobble along with OOM errors.

Nick
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/