Mailing List Archive

Reporting subsets of Rancid data.
We have 2 major campuses, and we've always reported our config diffs and pretty much everything else to all members of our small network team. All the switch configs are co-located in the same directory, etc. But now, one campus is complaining that they don't want to see all the config diffs from the other because it's difficult to know if they have data in the config diff report. I was asked if it was possible to split the report into 2, one for each campus. The IP addresses are such that it would be possible to identify them easily. But rancid just seems to be focused on reporting what is in the directory. I'm not sure I'd want to go to great effort to make this kind of thing happen, just to have it break every time I update Rancid. Our boss is keen on network knowing everything on either campus (we back one another up to a high level of degree). Is it easy to carve up the reporting based on IP ranges and provide different email addresses for each set of reports without impacting the future upgrading process? I just want to be able to say I investigated it, but I think the boss would be against it anyway.

Thanks.
Chris
Re: Reporting subsets of Rancid data. [ In reply to ]
If you created an additional set of groups, you could do a group per campus and send the reports/diffs for each group or groups to different e-mail addresses based on the entries in your /etc/aliases file. That's 100% supported and wouldn't change during upgrades. People who wanted to see all of the reports/diffs would be in all of the group aliases and those who didn't would only be in a subset.


Something along the lines of

campus1_switches: boss_person, campus1_people
campus2_switches: boss_person, campus2_people, campus1_people

-- Dan
On Apr 8, 2021, 12:11 PM -0400, Chris Davis <Chris.Davis@principia.edu>, wrote:
> We have 2 major campuses, and we've always reported our config diffs and pretty much everything else to all members of our small network team.  All the switch configs are co-located in the same directory, etc.  But now, one campus is complaining that they don't want to see all the config diffs from the other because it's difficult to know if they have data in the config diff report.  I was asked if it was possible to split the report into 2, one for each campus.  The IP addresses are such that it would be possible to identify them easily.  But rancid just seems to be focused on reporting what is in the directory.  I'm not sure I'd want to go to great effort to make this kind of thing happen, just to have it break every time I update Rancid.  Our boss is keen on network knowing everything on either campus (we back one another up to a high level of degree).  Is it easy to carve up the reporting based on IP ranges and provide different email addresses for each set of reports without impacting the future upgrading process?   I just want to be able to say I investigated it, but I think the boss would be against it anyway.
>
> Thanks.
> Chris
> _______________________________________________
> Rancid-discuss mailing list
> Rancid-discuss@www.shrubbery.net
> https://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Reporting subsets of Rancid data. [ In reply to ]
This is totally supported. It’s simply a matter of arranging your device
groups and email aliases in a way that meets your teams requirements.

Speaking of which, how are people maintaining their mailing lists for
Rancid these days? Is majordomo still best of breed here? I looked at it a
while ago but my rancid servers can’t receive email, only send. So I don’t
think that’s going to work.

Part of me still wants to me email subscriptions more of a self-service
thing and it’s nice to be able to do that without editing the aliases file.


On Fri, 9 Apr 2021 at 4:34 AM, Dan Anderson <dan.w.anderson@gmail.com>
wrote:

> If you created an additional set of groups, you could do a group per
> campus and send the reports/diffs for each group or groups to different
> e-mail addresses based on the entries in your /etc/aliases file. That's
> 100% supported and wouldn't change during upgrades. People who wanted to
> see all of the reports/diffs would be in all of the group aliases and those
> who didn't would only be in a subset.
>
>
> Something along the lines of
>
> campus1_switches: boss_person, campus1_people
> campus2_switches: boss_person, campus2_people, campus1_people
>
> -- Dan
> On Apr 8, 2021, 12:11 PM -0400, Chris Davis <Chris.Davis@principia.edu>,
> wrote:
>
> We have 2 major campuses, and we've always reported our config diffs and
> pretty much everything else to all members of our small network team. All
> the switch configs are co-located in the same directory, etc. But now, one
> campus is complaining that they don't want to see all the config diffs from
> the other because it's difficult to know if they have data in the config
> diff report. I was asked if it was possible to split the report into 2,
> one for each campus. The IP addresses are such that it would be possible
> to identify them easily. But rancid just seems to be focused on reporting
> what is in the directory. I'm not sure I'd want to go to great effort to
> make this kind of thing happen, just to have it break every time I update
> Rancid. Our boss is keen on network knowing everything on either campus
> (we back one another up to a high level of degree). Is it easy to carve up
> the reporting based on IP ranges and provide different email addresses for
> each set of reports without impacting the future upgrading process? I
> just want to be able to say I investigated it, but I think the boss would
> be against it anyway.
>
> Thanks.
> Chris
> _______________________________________________
> Rancid-discuss mailing list
> Rancid-discuss@www.shrubbery.net
> https://www.shrubbery.net/mailman/listinfo/rancid-discuss
>
> _______________________________________________
> Rancid-discuss mailing list
> Rancid-discuss@www.shrubbery.net
> https://www.shrubbery.net/mailman/listinfo/rancid-discuss
>
Re: Reporting subsets of Rancid data. [ In reply to ]
We went a different route, we don’t e-mail from RANCID. In fact, I don’t think that server is setup to send e-mail. Instead, we syslog the log files from RANCID, and send them to our SIEM (Splunk). People can create their own alerts, we don’t have to maintain distribution lists. This does not get us the diffs, it just notes that a device was updated. If someone want to see the diff, they use our web front end to the version control system.

We do this by wrapping RANCID with a shell script that runs RANCID, then parses the log files. The script is below, if anyone is interested.

Skye Hagen
Network Engineer
University of Idaho

#!/usr/bin/sh

# This shell script is the main script for running RANCID. This puts
# the whole package together.
#
# It runs RANCID for all groups, and sends the RANCID logs to syslog.

# Function to send a RANCID log to syslog

LogIt() {

# This routine will take a single RANCID log file, condense it, and
# send it to syslog.
#
# Parameters
# $1 - The name of the RANCID log file to process

# Build temp files
ERR=$(mktemp)
ADD=$(mktemp)
UPD=$(mktemp)
LOG=$(mktemp)

# Get the name of the rancid group
GRP=$(expr match $1 '.*\/\([a-z]*\)\.')

# Get new devices
grep "Added " $1 > $ADD

# Get updated devices
grep "Checking in " $1 > $UPD

# Get and reduce errors to a single line per device
grep "clogin error" $1 | sort | uniq -c > $ERR

# Compute some statistics on added, updated and errors
ADDCNT=$(wc -l < $ADD)
UPDCNT=$(wc -l < $UPD)
ERRCNT=$(wc -l < $ERR)

# Create a file of the lines to send to syslog
grep "starting:" $1 > $LOG
cat $ADD >> $LOG
cat $UPD >> $LOG
cat $ERR >> $LOG
echo "Added=$ADDCNT Updated=$UPDCNT Errors=$ERRCNT" >> $LOG
grep "ending:" $1 >> $LOG

# Send the file to syslog
logger -s -f $LOG -p local0.info -t "rancid-run Group=$GRP "

# Clean up temp files
rm $LOG $UPD $ADD $ERR
}

# ===== Main routine

# Run default ENVFILE to get the LOGDIR.
ENVFILE="/rancid/etc/rancid.conf"
. $ENVFILE

# Test user and test/set a lock file
LOCKFILE="/rancid/locks/processing"
USER="rancid"

WHOAMI=$(whoami)
if [ $WHOAMI != $USER ]
then
echo "This routine must be run as user $USER."
exit
fi

if [ -e $LOCKFILE ]
then
echo "Lock file $LOCKFILE exists."
exit
fi
touch $LOCKFILE

# Run RANCID
echo "Running rancid-run"
/rancid/bin/rancid-run

# Because RANCID does not syslog directly, we will need to convert
# the RANCID logs to syslog events. And, we don't know the name of
# the log files. But, we do know the directory that the log files are
# stored in. We also know that they will be newer than the date/time
# on our lock file. So, we use 'find' to find all log files in the
# LOGDIR newer than the lock file, and process them one at a time.
LOGS=$(find $LOGDIR -type f -newer $LOCKFILE)
for FILE in $LOGS
do
LogIt $FILE
done

# Now, remove the lock file
rm $LOCKFILE


From: Rancid-discuss <rancid-discuss-bounces@www.shrubbery.net> On Behalf Of Daniel Kerse
Sent: Thursday, April 8, 2021 10:08 PM
To: Dan Anderson <dan.w.anderson@gmail.com>
Cc: rancid-discuss@www.shrubbery.net
Subject: Re: [rancid] Reporting subsets of Rancid data.

This is totally supported. It’s simply a matter of arranging your device groups and email aliases in a way that meets your teams requirements.

Speaking of which, how are people maintaining their mailing lists for Rancid these days? Is majordomo still best of breed here? I looked at it a while ago but my rancid servers can’t receive email, only send. So I don’t think that’s going to work.

Part of me still wants to me email subscriptions more of a self-service thing and it’s nice to be able to do that without editing the aliases file.


On Fri, 9 Apr 2021 at 4:34 AM, Dan Anderson <dan.w.anderson@gmail.com<mailto:dan.w.anderson@gmail.com>> wrote:
If you created an additional set of groups, you could do a group per campus and send the reports/diffs for each group or groups to different e-mail addresses based on the entries in your /etc/aliases file. That's 100% supported and wouldn't change during upgrades. People who wanted to see all of the reports/diffs would be in all of the group aliases and those who didn't would only be in a subset.


Something along the lines of

campus1_switches: boss_person, campus1_people
campus2_switches: boss_person, campus2_people, campus1_people

-- Dan
On Apr 8, 2021, 12:11 PM -0400, Chris Davis <Chris.Davis@principia.edu<mailto:Chris.Davis@principia.edu>>, wrote:

We have 2 major campuses, and we've always reported our config diffs and pretty much everything else to all members of our small network team. All the switch configs are co-located in the same directory, etc. But now, one campus is complaining that they don't want to see all the config diffs from the other because it's difficult to know if they have data in the config diff report. I was asked if it was possible to split the report into 2, one for each campus. The IP addresses are such that it would be possible to identify them easily. But rancid just seems to be focused on reporting what is in the directory. I'm not sure I'd want to go to great effort to make this kind of thing happen, just to have it break every time I update Rancid. Our boss is keen on network knowing everything on either campus (we back one another up to a high level of degree). Is it easy to carve up the reporting based on IP ranges and provide different email addresses for each set of reports without impacting the future upgrading process? I just want to be able to say I investigated it, but I think the boss would be against it anyway.

Thanks.
Chris
_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@www.shrubbery.net<mailto:Rancid-discuss@www.shrubbery.net>
https://www.shrubbery.net/mailman/listinfo/rancid-discuss<https://urldefense.com/v3/__https:/www.shrubbery.net/mailman/listinfo/rancid-discuss__;!!JYXjzlvb!2xbLRtS5UnSohUmB64cMNdUA6sgfou3pTaRRAgSg4uzPL8Hmq-FH8Iiq6rWifqc$>
_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@www.shrubbery.net<mailto:Rancid-discuss@www.shrubbery.net>
https://www.shrubbery.net/mailman/listinfo/rancid-discuss<https://urldefense.com/v3/__https:/www.shrubbery.net/mailman/listinfo/rancid-discuss__;!!JYXjzlvb!2xbLRtS5UnSohUmB64cMNdUA6sgfou3pTaRRAgSg4uzPL8Hmq-FH8Iiq6rWifqc$>
Re: Reporting subsets of Rancid data. [ In reply to ]
Thu, Apr 08, 2021 at 12:34:21PM -0400, Dan Anderson:
> If you created an additional set of groups, you could do a group per campus and send the reports/diffs for each group or groups to different e-mail addresses based on the entries in your /etc/aliases file. That's 100% supported and wouldn't change during upgrades. People who wanted to see all of the reports/diffs would be in all of the group aliases and those who didn't would only be in a subset.
>
>
> Something along the lines of
>
> campus1_switches: boss_person, campus1_people
> campus2_switches: boss_person, campus2_people, campus1_people
>
> -- Dan

Another option is parse the diff with a rancid.conf:DIFFSCRIPT script; awk
out the diffs by IP or whatever and email them to <group>-a and <group>-b
AND send them to stdout which will go to rancid-<group> (ie: the cumulative).


Yet another option is to add a rancid.conf knob that allows the admin to
create their own script to perform the diff AND email them. Leaving the
default and the rancid-<group>-admin mail as the current process.

the script would have to accept some arguments to know where to send
the diffs, and so forth.

Is that of general interest?

> On Apr 8, 2021, 12:11 PM -0400, Chris Davis <Chris.Davis@principia.edu>, wrote:
> > We have 2 major campuses, and we've always reported our config diffs and pretty much everything else to all members of our small network team.? All the switch configs are co-located in the same directory, etc.? But now, one campus is complaining that they don't want to see all the config diffs from the other because it's difficult to know if they have data in the config diff report.? I was asked if it was possible to split the report into 2, one for each campus.? The IP addresses are such that it would be possible to identify them easily.? But rancid just seems to be focused on reporting what is in the directory.? I'm not sure I'd want to go to great effort to make this kind of thing happen, just to have it break every time I update Rancid.? Our boss is keen on network knowing everything on either campus (we back one another up to a high level of degree).? Is it easy to carve up the reporting based on IP ranges and provide different email addresses for each set of reports without impacting the future upgrading process? ? I just want to be able to say I investigated it, but I think the boss would be against it anyway.
> >
> > Thanks.
> > Chris
> > _______________________________________________
> > Rancid-discuss mailing list
> > Rancid-discuss@www.shrubbery.net
> > https://www.shrubbery.net/mailman/listinfo/rancid-discuss

> _______________________________________________
> Rancid-discuss mailing list
> Rancid-discuss@www.shrubbery.net
> https://www.shrubbery.net/mailman/listinfo/rancid-discuss

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@www.shrubbery.net
https://www.shrubbery.net/mailman/listinfo/rancid-discuss