Mailing List Archive

%commands vs. @commands
On Wed, Jun 08, 2005 at 06:37:51AM +0100, Jee Kay wrote:
> On a side note, what is the difference between the %commands and
> @commands list in rancid? Which one does it actually use?

Both are used. The %commands hash is an association between commands run
on the device and the subroutine used to process the output of that command:

# Main
%commands=(
'admin show version' => "ShowVersion",
'show version' => "ShowVersion",
'show redundancy secondary' => "ShowRedundancy",
[...]

But RANCID also needs to send the commands to the device in a certain
order. Unfortunately, there's no concept of order in a hash, you can ask
for a list of every item in the hash with the keys() function, but it
doesn't promise to you that it will return them in any particular order.
Hence the comment below that precedes the declaration of @commands:

# keys() doesnt return things in the order entered and the order of the
# cmds is important (show version first and write term last). pita
@commands=(
"admin show version",
"show version",
"show redundancy secondary",
[...]

This redundant construction has been grating on me when working on RANCID
code, so I've spent a few minutes thinking about how to fix it. One way
to fix this would be to use an array of anonymous hashes. Since it starts
out as an array, you keep the order of the list, and the individual entries
give you the association between command name and subroutine name:

@commandtable= (
{'admin show version' => "ShowVersion"},
{'show version' => "ShowVersion"},
{'show redundancy secondary' => "ShowRedundancy"},
[...]
);

And then, to keep code changes to a minimum, build @commands and %commands
automatically from @commandtable. Here we don't care about the order of
things returned by keys() or values(), because each has has only one entry:

my $i= 0;
foreach my $cmdref (@commandtable)
{
my @tmp= keys(%{$cmdref});
@commands[$i]= $tmp[0];
@tmp= values(%{$cmdref});
$commands{$commands[$i]}= $tmp[0];
$i++;
};

And the rest of the RANCID code remains unchanged, since @commands and
%commands now exist just as they did before.

Any thoughts from the RANCID maintainers about this? I'll be happy to
test this out and submit patches. The fragments above work as expected
when I tested them in isolation.

> I'd like to
> prune out all the commands I know my switches/routers don't support
> (or in the case of write term, will always support). Do I need to
> add/remove any new commands to both lists?

Yes, you would need to edit both lists. Or use the code above so that
there's only one list.

On the larger issue of pruning out commands, note my previous (unanswered)
query to the list about running both "show running-config" and "write term".
RANCID's philosophy seems to be "send all commands, let RANCID sort 'em out
afterwards". This makes sense because the xxxrancid programs don't talk
directly to the router, the xxxlogin program does that and produces an
output file that is presented to xxxrancid for parsing. RANCID happily
ignores commands that aren't supported on the device. And if one day you
upgrade IOS and one of those commands is supported, then all the better,
you get more data in your repository. In almost all cases, there's barely
any penalty for sending the unrecognized commands, so why bother pruning
them? My query about "show running-config" / "write term" was due to
a router here that takes 30-45 seconds to dump its config - which to my
mind was a penalty worth trying to program around. Or maybe not -
remember, RANCID connects to multiple devices in parallel, so unless there
are dozens of devices at your site that are slow to dump their config,
the RANCID won't take that much longer to finish.

I'm a firm believer in letting the computer do extra work so that the
programmer doesn't have to. Unless there's really a lot of damage caused
by the extra commands, I'd rather leave them in that figure out how to
code them out.

-- Ed
%commands vs. @commands [ In reply to ]
Wed, Jun 08, 2005 at 11:23:20AM -0400, Ed Ravin:

> And then, to keep code changes to a minimum, build @commands and %commands
> automatically from @commandtable. Here we don't care about the order of

> Any thoughts from the RANCID maintainers about this? I'll be happy to
> test this out and submit patches. The fragments above work as expected
> when I tested them in isolation.

I've never considered it much of a burden, but that change would be ok
by me.

> > I'd like to
> > prune out all the commands I know my switches/routers don't support
> > (or in the case of write term, will always support). Do I need to
> > add/remove any new commands to both lists?
>
> Yes, you would need to edit both lists. Or use the code above so that
> there's only one list.
>
> On the larger issue of pruning out commands, note my previous (unanswered)
> query to the list about running both "show running-config" and "write term".
> RANCID's philosophy seems to be "send all commands, let RANCID sort 'em out
> afterwards". This makes sense because the xxxrancid programs don't talk
> directly to the router, the xxxlogin program does that and produces an
> output file that is presented to xxxrancid for parsing. RANCID happily
> ignores commands that aren't supported on the device. And if one day you
> upgrade IOS and one of those commands is supported, then all the better,
> you get more data in your repository. In almost all cases, there's barely
> any penalty for sending the unrecognized commands, so why bother pruning
> them? My query about "show running-config" / "write term" was due to
> a router here that takes 30-45 seconds to dump its config - which to my
> mind was a penalty worth trying to program around. Or maybe not -
> remember, RANCID connects to multiple devices in parallel, so unless there
> are dozens of devices at your site that are slow to dump their config,
> the RANCID won't take that much longer to finish.

I had hoped that andrew would reply; he remembers much more history than
i do.

but, we're not trying to drop support for older devices; there are folks
who use really old boxes (eg ags) in places where it is difficult to get
equipment and then there are versions of IOS that actually run for a year
w/o crashing so why upgrade them (eg 11.1 on 2500) if there is no compelling
reason.

that aside, given that rancid does it's thing in the background, I see no
reason to remove the compatibility. I'd rather work on the configurable-
rancid idea, where one can define the commands they want rancid to run, or
skip. This way, possibly one could use a *login script (clogin -s) which
has the intelligence to skip unnecessary commands. but, thats TBD.
%commands vs. @commands [ In reply to ]
On Wed, Jun 08, 2005 at 11:23:20AM -0400, Ed Ravin wrote:
> On the larger issue of pruning out commands, note my previous (unanswered)
> query to the list about running both "show running-config" and "write term".
> RANCID's philosophy seems to be "send all commands, let RANCID sort 'em out
> afterwards".

Yup; we try to support a large number of routers running many
versions of code with a small number of programs. E.g.: all junipers
are supported by jlogin/jrancid, even if this means that some routers
are sent commands that they don't support and if we are sending
commands that only work in older code. Another e.g.: clogin is
used for ciscos & a number of non-cisco routers.

It pains me to have some ciscos supported by rancid/clogin and other
ciscos by cat5rancid/clogin. At least both types use clogin.

The ideal would be to have one login program and one rancid program
& put the smarts into the program to figure out what its talking
to.

As it is, we currently have 12 *login programs and 23 *rancid
programs; most of which share a fair amount of code. Trying to
keep them in sync & trying to make sure that changes that works on
one doesn't blow up another is a pain.

I'd much rather send 100 useless commands to a router than expand
the number of *login & *rancid programs we have, trying to taylor
each one to a particular type of router running a particular version
of code.

I think Heas' comment of working on the configurable rancid is the
way to go - fewer commands for us humans to remember; more smarts
in the code.
--asp
%commands vs. @commands [ In reply to ]
On Wed, Jun 08, 2005 at 12:39:30PM -0400, Andrew Partan wrote:

> As it is, we currently have 12 *login programs and 23 *rancid
> programs; most of which share a fair amount of code. Trying to
> keep them in sync & trying to make sure that changes that works on
> one doesn't blow up another is a pain.

And I just added another *login program, and will release hp4000m.rancid
and hp4000m.clogin shortly. :-(

> I think Heas' comment of working on the configurable rancid is the
> way to go - fewer commands for us humans to remember; more smarts
> in the code.

What would such a configurable rancid look like? Here's one vision:

We have one *rancid program. It has a series of device-specific
"plug-ins", implemented as Perl modules. The modules are separated into
generic code (like ProcessHistory) and vendor-specific or device-specific
code (like processing config output). Maybe you can get fancy with classes
and have device classes that get extended when needed. There's a (single)
table somewhere that has a list of commands and subroutines that get run for
the command output of a particular device. There will probably need to be
another table or a master subroutine for each device that handles things like
what final prompt you look for, device nuances, or any other weird stuff like
filtering HP Procurve output.

Something similar should be done to *login, but as my Expect knowledge is
very limited (I can program my way out of a paper bag at this point, but
not much else), I can't say how to do it. If there's a way to use common
library code with Expect, you could move a lot of the code into libraries
so that there's only one copy of it - the cloginrc parsing stuff, a generic
login routine that will work for most devices, etc. Even if we keep the
multiple *login programs, at least there will be only one copy of most of
the code.