Mailing List Archive

Server crashed and RANCID somehow restored a two year old version
Hello,

I have been running RANCID for several years on several different
networks.

I run RANCID on Solaris on an U5. The box crashed last night and when
it
came back up RANCID somehow restored a copy of the very old (2+ yrs old)
configs/directories/etc.

I am wondering how to go about getting the configs restored from two
days
ago?

Thanks,

-andrew
Server crashed and RANCID somehow restored a two year old version [ In reply to ]
Mon, Dec 29, 2003 at 09:56:34AM -0500, Elliott, Andrew:
> Hello,
>
> I have been running RANCID for several years on several different
> networks.
>
> I run RANCID on Solaris on an U5. The box crashed last night and when
> it
> came back up RANCID somehow restored a copy of the very old (2+ yrs old)
> configs/directories/etc.
>

rancid just uses CVS. The only way it could have restored 2-yr old configs
would be if someone restored from tape, a very nasty log-filesystem bug, or
possibly (w/o having looked at the cvs source) an incorrect system clock
might do it.

It could also be that someone has purposely done it. You can check for
other revisions (ie: newer than the one you're seeing) with cvs log; eg:

cd /usr/local/rancid/groupfoo/configs
cvs log router.foo.net | less

If the CVS revisions have multiple dots ('.') in them, you've ended-up with
a branch some how (again not something rancid does). You can return to the
head with:

cd /usr/local/rancid
cvs update -AdP

> I am wondering how to go about getting the configs restored from two
> days ago?

To get configs from specific times/revs/etc, see the -p, -r, and -D options
of cvs checkout. Also see the cvs FAQ noted in the rancid FAQ.
Server crashed and RANCID somehow restored a two year old version [ In reply to ]
> I run RANCID on Solaris on an U5. The box crashed last night and
> when it came back up RANCID somehow restored a copy of the very
> old (2+ yrs old) configs/directories/etc.

not bleedin' likely. on the target system, you forgot to
save/force the configs to nvram so that, when it rebooted,
it got the last saved config, which was 2+ years old.

> I am wondering how to go about getting the configs restored from
> two days ago?

that depends a lot on the target box's command language, of course.
but rancid has the configs in cvs, so just check out the one you
want. or, if you like the most recent, you will find it in

../rancid/<netname>/configs/<device>

and then stick it up the target.

randy
Server crashed and RANCID somehow restored a two year old version [ In reply to ]
Mon, Dec 29, 2003 at 07:49:29AM -0800, Randy Bush:
> that depends a lot on the target box's command language, of course.
> but rancid has the configs in cvs, so just check out the one you
> want. or, if you like the most recent, you will find it in
>
> ../rancid/<netname>/configs/<device>
>
> and then stick it up the target.

if/when you do this, be sure to replace the '<removed>' text in rancid
configs with the proper password/etc.
Server crashed and RANCID somehow restored a two year old version [ In reply to ]
> It could also be that someone has purposely done it. You can
> check for
> other revisions (ie: newer than the one you're seeing) with
> cvs log; eg:
>
> cd /usr/local/rancid/groupfoo/configs
> cvs log router.foo.net | less

10:05am rancid at bosshogg:/routers/Backbone/Cisco/configs> cvs log
chr1.nyc-ny | less

RCS file: /tftpboot/routers/Backbone/CVS/Cisco/configs/chr1.nyc-ny,v
Working file: chr1.nyc-ny
head: 1.23
branch:
locks: strict
access list:
symbolic names:
keyword substitution: o
total revisions: 23; selected revisions: 23
description:
----------------------------
revision 1.23
date: 2003/12/29 06:14:16; author: rancid; state: Exp; lines: +1377
-566
updates
----------------------------
revision 1.22
date: 2002/05/17 05:06:56; author: rancid; state: Exp; lines: +117 -8
updates
----------------------------
Server crashed and RANCID somehow restored a two year old version [ In reply to ]
Mon, Dec 29, 2003 at 11:18:30AM -0500, Elliott, Andrew:
>
> > It could also be that someone has purposely done it. You can
> > check for
> > other revisions (ie: newer than the one you're seeing) with
> > cvs log; eg:
> >
> > cd /usr/local/rancid/groupfoo/configs
> > cvs log router.foo.net | less
>
> 10:05am rancid at bosshogg:/routers/Backbone/Cisco/configs> cvs log
> chr1.nyc-ny | less
>
> RCS file: /tftpboot/routers/Backbone/CVS/Cisco/configs/chr1.nyc-ny,v
> Working file: chr1.nyc-ny
> head: 1.23
> branch:
> locks: strict
> access list:
> symbolic names:
> keyword substitution: o
> total revisions: 23; selected revisions: 23
> description:
> ----------------------------
> revision 1.23
> date: 2003/12/29 06:14:16; author: rancid; state: Exp; lines: +1377
> -566
> updates
> ----------------------------
> revision 1.22
> date: 2002/05/17 05:06:56; author: rancid; state: Exp; lines: +117 -8
> updates
> ----------------------------

hmm. Given the revs are serial, there are only 23, and the jump in dates;
I'd guess that either rancid had trouble committing updates (see the log
files) and the errors were ignored, the CVS file was damaged in the crash
(check the ",v" RCS file and compare to your backups), or someone mucked
with the cvs file.

Note: I do not recommend manually mucking with the respository (ie:
/tftpboot/routers/backbone/CVS in this case) in any fashion.
Server crashed and RANCID somehow restored a two year old version [ In reply to ]
> hmm. Given the revs are serial, there are only 23, and the
> jump in dates;
> I'd guess that either rancid had trouble committing updates
> (see the log
> files) and the errors were ignored, the CVS file was damaged
> in the crash
> (check the ",v" RCS file and compare to your backups), or
> someone mucked
> with the cvs file.

This is one of the few routers that are still in existance from March
2002. 99% of the routers in the configs dir are no longer in operation.
Also, other than Cisco, we also used to have Bay, Redback, etc, and we
no longer even had those Vendors directories anymore... also several of
the newer directories are missing (IE: Catalyst, Juniper, Unisphere)...


I am starting to think I need to just blow it away and start over since
I obviously don't know enough about CVS... all I have ever known about
CVS was how to create new entries, delete old ones, and update and
commit.

Thanks for all the help,

-andrew
Server crashed and RANCID somehow restored a two year old version [ In reply to ]
Thanks for all the help. Found the problem to be a corrupt vfstab, so
the "real" filesystem wasn't mounted properly on /tftpboot... the old
/tftpboot was still showing up in the / filesystem.

Sorry for the confusion.

-andrew