Mailing List Archive

Entries disappear and re appear.
Hello everyone,

We have an issue whereby rancid is pulling configs and showing that an entry has disappeared and then the next config check the entry is back. This is happening on a Cisco ASA and also on a Mikrotik.

For the Mikrotik, it shows that the console port name changed to nothing and then the next time shows that it came back as serial0. The name is not actually changing.

For the ASA, the entry !BootFlash: BOOT variable = disk0:/asa917-19-k8.bin will show in one config and the next time it shows !BootFlash: BOOT variable =

The boot variable is not actually changing, so I have to wonder if this is a bug.

We are running rancid version 3.8.

Has anyone run into these issues and is there a fix for them?

Best,

-Troy
_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Entries disappear and re appear. [ In reply to ]
Thu, Apr 11, 2019 at 09:31:05AM -0700, Troy Beisigl:
> Hello everyone,
>
> We have an issue whereby rancid is pulling configs and showing that an entry has disappeared and then the next config check the entry is back. This is happening on a Cisco ASA and also on a Mikrotik.
>
> For the Mikrotik, it shows that the console port name changed to nothing and then the next time shows that it came back as serial0. The name is not actually changing.
>
> For the ASA, the entry !BootFlash: BOOT variable = disk0:/asa917-19-k8.bin will show in one config and the next time it shows !BootFlash: BOOT variable =
>
> The boot variable is not actually changing, so I have to wonder if this is a bug.
>
> We are running rancid version 3.8.
>
> Has anyone run into these issues and is there a fix for them?

the ASA is probably a bug in the ASA. I've seen it on IOSXE.

no idea about the microtik; possibly rancid isnt able to disable the
pager and thats causing some problem?

I'd manually collect the raw output repeatedly in an attempt to catch it
occuring. See FAQ S3Q2.

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Entries disappear and re appear. [ In reply to ]
> We have an issue whereby rancid is pulling configs and showing that an entry has disappeared and then the next config check the entry is back. This is happening on a Cisco ASA and also on a Mikrotik.

We're still running 3.6.1, and haven't seen any such issues with any of our ASA's (None are running version 9 though).

Does it only happen when rancid is run from cron? Or can you reproduce when running rancid manually?

> For the Mikrotik, it shows that the console port name changed to nothing and then the next time shows that it came back as serial0.

Overall, our experience with Mikrotiks have been pretty poor.
They frequently drop their entire config from the output, or selectively random blocks of config (Interfaces, GPS, firmware and version stats), but we just summed that up as another of Mikrotik's long line of humble quirks.


-----Original Message-----
From: Rancid-discuss [mailto:rancid-discuss-bounces@shrubbery.net] On Behalf Of Troy Beisigl
Sent: Thursday, April 11, 2019 9:31 AM
To: rancid-discuss@shrubbery.net
Subject: [rancid] Entries disappear and re appear.

Hello everyone,

We have an issue whereby rancid is pulling configs and showing that an entry has disappeared and then the next config check the entry is back. This is happening on a Cisco ASA and also on a Mikrotik.

For the Mikrotik, it shows that the console port name changed to nothing and then the next time shows that it came back as serial0. The name is not actually changing.

For the ASA, the entry !BootFlash: BOOT variable = disk0:/asa917-19-k8.bin will show in one config and the next time it shows !BootFlash: BOOT variable =

The boot variable is not actually changing, so I have to wonder if this is a bug.

We are running rancid version 3.8.

Has anyone run into these issues and is there a fix for them?

Best,

-Troy
_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Entries disappear and re appear. [ In reply to ]
Thu, Apr 11, 2019 at 09:51:20AM -0700, Emille Blanc:
> > For the Mikrotik, it shows that the console port name changed to nothing and then the next time shows that it came back as serial0.
>
> Overall, our experience with Mikrotiks have been pretty poor.
> They frequently drop their entire config from the output, or selectively random blocks of config (Interfaces, GPS, firmware and version stats), but we just summed that up as another of Mikrotik's long line of humble quirks.
>

while i've heard similar comments and have none of these boxes myself, its
possible that there is a problem with the rancid mikrotik support. if
someone wants to give me remote access to one, i can verify/improve the
rancid support for it.

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Entries disappear and re appear. [ In reply to ]
On 11/04/2019 19:54, heasley wrote:
> Thu, Apr 11, 2019 at 09:51:20AM -0700, Emille Blanc:
>>
>> Overall, our experience with Mikrotiks have been pretty poor.
>> They frequently drop their entire config from the output, or selectively random blocks of config (Interfaces, GPS, firmware and version stats), but we just summed that up as another of Mikrotik's long line of humble quirks.
>>
I've never had any problems of this nature with Mikrotik devices,
although I only have about 5 of them on RANCID.

> while i've heard similar comments and have none of these boxes myself, its
> possible that there is a problem with the rancid mikrotik support. if
> someone wants to give me remote access to one, i can verify/improve the
> rancid support for it.

You can spin up a virtual one for free. It will be limited to 1Mbps but
that is adequate for testing the RANCID module, I'm sure.

https://wiki.mikrotik.com/wiki/Manual:CHR

alexd

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Entries disappear and re appear. [ In reply to ]
Thanks. We are seeing it happen randomly across 17 RB3011UiAS-RM. It only happens every day or 2 and only happens to 1 or 2 units each time. The version of Rancid we are running was a fresh install on a new machine about 30 days ago, so wondering if it is something not setup correctly or if it is a bug with the plugins.

Best,

-Troy

> On Apr 12, 2019, at 1:39 AM, Alex DEKKER <rancid@ale.cx> wrote:
>
> On 11/04/2019 19:54, heasley wrote:
>> Thu, Apr 11, 2019 at 09:51:20AM -0700, Emille Blanc:
>>>
>>> Overall, our experience with Mikrotiks have been pretty poor.
>>> They frequently drop their entire config from the output, or selectively random blocks of config (Interfaces, GPS, firmware and version stats), but we just summed that up as another of Mikrotik's long line of humble quirks.
>>>
> I've never had any problems of this nature with Mikrotik devices, although I only have about 5 of them on RANCID.
>
>> while i've heard similar comments and have none of these boxes myself, its
>> possible that there is a problem with the rancid mikrotik support. if
>> someone wants to give me remote access to one, i can verify/improve the
>> rancid support for it.
>
> You can spin up a virtual one for free. It will be limited to 1Mbps but that is adequate for testing the RANCID module, I'm sure.
>
> https://wiki.mikrotik.com/wiki/Manual:CHR
>
> alexd
>
> _______________________________________________
> Rancid-discuss mailing list
> Rancid-discuss@shrubbery.net
> http://www.shrubbery.net/mailman/listinfo/rancid-discuss

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Entries disappear and re appear. [ In reply to ]
A strange thing has happened after removing the ASA host from rancid and adding it under the name it should have been identified as, the issue has stopped. It has been 24 hours with 12 rancid runs and no issues on the ASA. So I am not sure how changing the name of the device would impact that, but it would see that this might be resolved for now.

Still the random issue with the Mikrotiks though.

-Troy

> On Apr 11, 2019, at 9:47 AM, heasley <heas@shrubbery.net> wrote:
>
> Thu, Apr 11, 2019 at 09:31:05AM -0700, Troy Beisigl:
>> Hello everyone,
>>
>> We have an issue whereby rancid is pulling configs and showing that an entry has disappeared and then the next config check the entry is back. This is happening on a Cisco ASA and also on a Mikrotik.
>>
>> For the Mikrotik, it shows that the console port name changed to nothing and then the next time shows that it came back as serial0. The name is not actually changing.
>>
>> For the ASA, the entry !BootFlash: BOOT variable = disk0:/asa917-19-k8.bin will show in one config and the next time it shows !BootFlash: BOOT variable =
>>
>> The boot variable is not actually changing, so I have to wonder if this is a bug.
>>
>> We are running rancid version 3.8.
>>
>> Has anyone run into these issues and is there a fix for them?
>
> the ASA is probably a bug in the ASA. I've seen it on IOSXE.
>
> no idea about the microtik; possibly rancid isnt able to disable the
> pager and thats causing some problem?
>
> I'd manually collect the raw output repeatedly in an attempt to catch it
> occuring. See FAQ S3Q2.

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Entries disappear and re appear. [ In reply to ]
are you using dns, ptr, or hosts file?


>On Fri, 2019-04-12 at 08:40 -0700, Troy Beisigl wrote:
> A strange thing has happened after removing the ASA host from rancid
> and adding it under the name it should have been identified as, the
> issue has stopped. It has been 24 hours with 12 rancid runs and no
> issues on the ASA. So I am not sure how changing the name of the
> device would impact that, but it would see that this might be
> resolved for now.
>
> Still the random issue with the Mikrotiks though.
>
> -Troy
>
> > On Apr 11, 2019, at 9:47 AM, heasley <heas@shrubbery.net> wrote:
> >
> > Thu, Apr 11, 2019 at 09:31:05AM -0700, Troy Beisigl:
> > > Hello everyone,
> > >
> > > We have an issue whereby rancid is pulling configs and showing
> > > that an entry has disappeared and then the next config check the
> > > entry is back. This is happening on a Cisco ASA and also on a
> > > Mikrotik.
> > >
> > > For the Mikrotik, it shows that the console port name changed to
> > > nothing and then the next time shows that it came back as
> > > serial0. The name is not actually changing.
> > >
> > > For the ASA, the entry !BootFlash: BOOT variable = disk0:/asa917-
> > > 19-k8.bin will show in one config and the next time it shows
> > > !BootFlash: BOOT variable =
> > >
> > > The boot variable is not actually changing, so I have to wonder
> > > if this is a bug.
> > >
> > > We are running rancid version 3.8.
> > >
> > > Has anyone run into these issues and is there a fix for them?
> >
> > the ASA is probably a bug in the ASA. I've seen it on IOSXE.
> >
> > no idea about the microtik; possibly rancid isnt able to disable
> > the
> > pager and thats causing some problem?
> >
> > I'd manually collect the raw output repeatedly in an attempt to
> > catch it
> > occuring. See FAQ S3Q2.
>
> _______________________________________________
> Rancid-discuss mailing list
> Rancid-discuss@shrubbery.net
> http://www.shrubbery.net/mailman/listinfo/rancid-discuss


_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Entries disappear and re appear. [ In reply to ]
Fri, Apr 12, 2019 at 08:35:07AM -0700, Troy Beisigl:
> Thanks. We are seeing it happen randomly across 17 RB3011UiAS-RM. It only happens every day or 2 and only happens to 1 or 2 units each time. The version of Rancid we are running was a fresh install on a new machine about 30 days ago, so wondering if it is something not setup correctly or if it is a bug with the plugins.
>
> Best,
>
> -Troy

Discussing this with another user (who can reveal themselves if they wish)
that had experienced this, it was discovered their devices suffer from memory
shortages. When malloc failures occur, parts of the configuration may
disappear and reappear. Furthermore, it seems that there is no error or
indication that the missing configuration was due to such a failure vs. a
real configuration change, and therefore no way for rancid to recognize it.

Can you check if you are having malloc failures?

> > On Apr 12, 2019, at 1:39 AM, Alex DEKKER <rancid@ale.cx> wrote:
> >
> > On 11/04/2019 19:54, heasley wrote:
> >> Thu, Apr 11, 2019 at 09:51:20AM -0700, Emille Blanc:
> >>>
> >>> Overall, our experience with Mikrotiks have been pretty poor.
> >>> They frequently drop their entire config from the output, or selectively random blocks of config (Interfaces, GPS, firmware and version stats), but we just summed that up as another of Mikrotik's long line of humble quirks.
> >>>
> > I've never had any problems of this nature with Mikrotik devices, although I only have about 5 of them on RANCID.
> >
> >> while i've heard similar comments and have none of these boxes myself, its
> >> possible that there is a problem with the rancid mikrotik support. if
> >> someone wants to give me remote access to one, i can verify/improve the
> >> rancid support for it.
> >
> > You can spin up a virtual one for free. It will be limited to 1Mbps but that is adequate for testing the RANCID module, I'm sure.
> >
> > https://wiki.mikrotik.com/wiki/Manual:CHR
> >
> > alexd
> >
> > _______________________________________________
> > Rancid-discuss mailing list
> > Rancid-discuss@shrubbery.net
> > http://www.shrubbery.net/mailman/listinfo/rancid-discuss
>
> _______________________________________________
> Rancid-discuss mailing list
> Rancid-discuss@shrubbery.net
> http://www.shrubbery.net/mailman/listinfo/rancid-discuss

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Entries disappear and re appear. [ In reply to ]
Thanks. I don’t believe there were any memory issues but I will check the logs and see what they reveal.

Best,

-Troy

> On Apr 17, 2019, at 3:00 PM, heasley <heas@shrubbery.net> wrote:
>
> Fri, Apr 12, 2019 at 08:35:07AM -0700, Troy Beisigl:
>> Thanks. We are seeing it happen randomly across 17 RB3011UiAS-RM. It only happens every day or 2 and only happens to 1 or 2 units each time. The version of Rancid we are running was a fresh install on a new machine about 30 days ago, so wondering if it is something not setup correctly or if it is a bug with the plugins.
>>
>> Best,
>>
>> -Troy
>
> Discussing this with another user (who can reveal themselves if they wish)
> that had experienced this, it was discovered their devices suffer from memory
> shortages. When malloc failures occur, parts of the configuration may
> disappear and reappear. Furthermore, it seems that there is no error or
> indication that the missing configuration was due to such a failure vs. a
> real configuration change, and therefore no way for rancid to recognize it.
>
> Can you check if you are having malloc failures?
>
>>> On Apr 12, 2019, at 1:39 AM, Alex DEKKER <rancid@ale.cx> wrote:
>>>
>>> On 11/04/2019 19:54, heasley wrote:
>>>> Thu, Apr 11, 2019 at 09:51:20AM -0700, Emille Blanc:
>>>>>
>>>>> Overall, our experience with Mikrotiks have been pretty poor.
>>>>> They frequently drop their entire config from the output, or selectively random blocks of config (Interfaces, GPS, firmware and version stats), but we just summed that up as another of Mikrotik's long line of humble quirks.
>>>>>
>>> I've never had any problems of this nature with Mikrotik devices, although I only have about 5 of them on RANCID.
>>>
>>>> while i've heard similar comments and have none of these boxes myself, its
>>>> possible that there is a problem with the rancid mikrotik support. if
>>>> someone wants to give me remote access to one, i can verify/improve the
>>>> rancid support for it.
>>>
>>> You can spin up a virtual one for free. It will be limited to 1Mbps but that is adequate for testing the RANCID module, I'm sure.
>>>
>>> https://wiki.mikrotik.com/wiki/Manual:CHR
>>>
>>> alexd
>>>
>>> _______________________________________________
>>> Rancid-discuss mailing list
>>> Rancid-discuss@shrubbery.net
>>> http://www.shrubbery.net/mailman/listinfo/rancid-discuss
>>
>> _______________________________________________
>> Rancid-discuss mailing list
>> Rancid-discuss@shrubbery.net
>> http://www.shrubbery.net/mailman/listinfo/rancid-discuss

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss