Mailing List Archive

Assistance creating a perl module to use with ZPE Systems NodegridOS Devices
Hi Heas, all,

I'm struggling with an issue in trying to get some outputs from our ZPE estate of Out of Band devices using RANCID. What I have tried to do thus far is as follows:

1. Set credentials for my devices in .cloginrc
2. Cloned both clogin and rancid files in /var/lib/rancid/bin to zpelogin and zperancid respectively (just for separation)
- Here's the diff of the files:
rancid@rancid-staging:~/bin$ diff clogin zpelogin
1052,1072c1052,1072
< if { $do_command || $do_script } {
< if { [string compare "extreme" "$platform"] } {
< # If the prompt is (enable), then we are on a switch and the
< # command is "set length 0"; otherwise its "terminal length 0".
< if [regexp -- ".*> .*enable" "$prompt"] {
< send "set length 0\r"
< expect -re $prompt {}
< send "set width 132\r"
< expect -re $prompt {}
< send "set logging session disable\r"
< } else {
< send "terminal length 0\r"
< expect -re $prompt {}
< send "terminal width 132\r"
< }
< expect -re $prompt {}
< } else {
< send "disable clipaging\r"
< expect -re $prompt {}
< }
< }
---
> # if { $do_command || $do_script } {
> # if { [string compare "extreme" "$platform"] } {
> # # If the prompt is (enable), then we are on a switch and the
> # # command is "set length 0"; otherwise its "terminal length 0".
> # if [regexp -- ".*> .*enable" "$prompt"] {
> # send "set length 0\r"
> # expect -re $prompt {}
> # send "set width 132\r"
> # expect -re $prompt {}
> # send "set logging session disable\r"
> # } else {
> # send "terminal length 0\r"
> # expect -re $prompt {}
> # send "terminal width 132\r"
> # }
> # expect -re $prompt {}
> # } else {
> # send "disable clipaging\r"
> # expect -re $prompt {}
> # }
> # }

^^^ so essentially I've removed the cisco specific terminal length setting, etc

rancid@rancid-staging:~/bin$ diff rancid zperancid
140a141,143
> # I don't care if it sees "exit"
> $clean_run = 1;
>

^^^ It was recommended to me several weeks back to make this change but I can't for the life of me remember why.

3. edited /etc/rancid/rancid.types.[conf?/base] to include the following lines

zpe;script;zperancid -t zpe
zpe;login;zpelogin
zpe;module;eos
zpe;inloop;eos::inloop
zpe;command;eos::ShowSoftware;show system/about
zpe;command;eos::ShowConfig;export_settings

^^^ the above trying the eos.pm module

When I run some debug against a host I receive the following:

rancid -d -t zpe <redacted>
loadtype: device type zpe
loadtype: found device type zpe in /etc/rancid/rancid.types.base
executing zpelogin -t 90 -c"show system/about;export_settings;show_settings" <redacted>
PROMPT MATCH: \[heanet@<redacted> /\]#
HIT COMMAND:[heanet@<redacted> /]# show system/about
In ShowSoftware: [heanet@<redacted> /]# show system/about
<redacted>: missed cmd(s): export_settings,show_settings
<redacted>: missed cmd(s): export_settings,show_settings
<redacted>: End of run not found
<redacted>: End of run not found

My reading of the above is that the module in question can properly understand the output received from the "show system/about" command and therefore stops and doesn't run the subsequent command. I would like to either get this sorted using pre-existing modules (probably not possible) or create a new perl module specific to zpe (happy to share with the community for others to use if needed).

For completeness, he's a snip of the outputs received in each of the commands I'm looking to run:

[heanet@<redacted> /]# show system/about/
system: Nodegrid Gate SR
licenses: 25
software: vX.X.X (Sep 9 2021 - 09:39:02)
cpu: Intel(R) Atom(TM) CPU C3538 @ 2.10GHz
cpu_cores: 4
bogomips_per_core: 4200.00
serial_number: XXX
uptime: 2 days, 23 hours, 49 minutes
model: GateSR
part_number: GSR-T8-BASE
bios_version: 90816T00
psu: 2
revision tag: Initial Config 2021-01-18 11:03:08.917159
bios sed compatible: no
ssd sed compatible: yes

[heanet@<redacted> /]# export_settings
<SNIP>
/settings/auditing/destinations/email email_port=25
/settings/auditing/destinations/email password=********
/settings/auditing/destinations/email confirm_password=********
/settings/auditing/destinations/email start_tls=yes


This is something that has been bugging me for a while so any help would be much appreciated!

Regards,

--
Mick O’Donovan
Senior Network Engineer

HEAnet CLG
Ireland's National Education and Research Network

1st Floor | 5 George's Dock | IFSC | Dublin D01 X8N7 | Ireland
+353 1 6609040 | mick.odonovan@heanet.ie | www.heanet.ie <http://www.heanet.ie>
Registered in Ireland, No. 275301 | CRA No. 20036270





_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@www.shrubbery.net
https://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Assistance creating a perl module to use with ZPE Systems NodegridOS Devices [ In reply to ]
Fri, Oct 15, 2021 at 02:59:01PM +0000, Mick O'Donovan:
> rancid@rancid-staging:~/bin$ diff rancid zperancid
> 140a141,143
> > # I don't care if it sees "exit"
> > $clean_run = 1;
> >
>
> ^^^ It was recommended to me several weeks back to make this change but I can't for the life of me remember why.

IMO, it is part of the conversation and is further confirmation that was
normal. Various devices have had cli bugs over time. Does not hurt to
remove it for debugging.

> rancid -d -t zpe <redacted>
> loadtype: device type zpe
> loadtype: found device type zpe in /etc/rancid/rancid.types.base
> executing zpelogin -t 90 -c"show system/about;export_settings;show_settings" <redacted>
> PROMPT MATCH: \[heanet@<redacted> /\]#

I'm curious about the redacted bit; if it contains regex atoms that were
not escaped by the login or perl script, that could be the problem.

> HIT COMMAND:[heanet@<redacted> /]# show system/about
> In ShowSoftware: [heanet@<redacted> /]# show system/about
> <redacted>: missed cmd(s): export_settings,show_settings
> <redacted>: missed cmd(s): export_settings,show_settings
> <redacted>: End of run not found
> <redacted>: End of run not found
>
> My reading of the above is that the module in question can properly understand the output received from the "show system/about" command and therefore stops and doesn't run the subsequent command. I would like to either get this sorted using pre-existing modules (probably not possible) or create a new perl module specific to zpe (happy to share with the community for others to use if needed).
>

I would first verify that the login script is running successfully.

eval `rancid -C -t zpe` >& raw

Look at the raw file to ensure that it has collected the complete output,
not prematurely sent commands, etc - ie: it looks normal. My guess is
that it is not matching the prompts promptly and sending command prematurely.
Look for them

[heanet@<redacted> /]# show system/about/
[heanet@<redacted> /]# export_settings
[heanet@<redacted> /]# show_settings
[heanet@<redacted> /]# exit

if the regex match in the perl module is not correct, it might not be
matching the prompt and therefore ShowSoftware() might be eating too
much of the input - ie: the entire input.

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@www.shrubbery.net
https://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Assistance creating a perl module to use with ZPE Systems NodegridOS Devices [ In reply to ]
?On 15/10/2021, 17:40, "heasley" <heas@shrubbery.net> wrote:

IMO, it is part of the conversation and is further confirmation that was
normal. Various devices have had cli bugs over time. Does not hurt to
remove it for debugging.


By this do you mean to leave the original file alone or my version is fine? Sorry excuse my ignorance here.

> rancid -d -t zpe <redacted>
> loadtype: device type zpe
> loadtype: found device type zpe in /etc/rancid/rancid.types.base
> executing zpelogin -t 90 -c"show system/about;export_settings;show_settings" <redacted>
> PROMPT MATCH: \[heanet@<redacted> /\]#

I'm curious about the redacted bit; if it contains regex atoms that were
not escaped by the login or perl script, that could be the problem.

My apologies, this was a bit overly paranoid on my part. The <redacted> is simply the test host FQDN or hostname.

> HIT COMMAND:[heanet@<redacted> /]# show system/about
> In ShowSoftware: [heanet@<redacted> /]# show system/about
> <redacted>: missed cmd(s): export_settings,show_settings
> <redacted>: missed cmd(s): export_settings,show_settings
> <redacted>: End of run not found
> <redacted>: End of run not found
>
> My reading of the above is that the module in question can properly understand the output received from the "show system/about" command and therefore stops and doesn't run the subsequent command. I would like to either get this sorted using pre-existing modules (probably not possible) or create a new perl module specific to zpe (happy to share with the community for others to use if needed).
>

I would first verify that the login script is running successfully.

eval `rancid -C -t zpe` >& raw

Look at the raw file to ensure that it has collected the complete output,
not prematurely sent commands, etc - ie: it looks normal. My guess is
that it is not matching the prompts promptly and sending command prematurely.
Look for them

[heanet@<redacted> /]# show system/about/
[heanet@<redacted> /]# export_settings
[heanet@<redacted> /]# show_settings
[heanet@<redacted> /]# exit

if the regex match in the perl module is not correct, it might not be
matching the prompt and therefore ShowSoftware() might be eating too
much of the input - ie: the entire input.


Something must be up alright, here's the contents of the raw file that was created...

===
Error: Usage: /usr/bin/zpelogin [-dhSV] [-m|M] [-autoenable] [-noenable] [-c command] [-Evar=x] [-e enable-password] [-f cloginrc-file] [-p user-password] [-r passphrase] [-s script-file] [-t timeout] [-u username] [-v vty-password] [-w enable-username] [-x command-file] [-y ssh_cypher_type] router [router...]
===

Any ideas?

Mick


_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@www.shrubbery.net
https://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Assistance creating a perl module to use with ZPE Systems NodegridOS Devices [ In reply to ]
Fri, Oct 15, 2021 at 07:10:45PM +0000, Mick O'Donovan:
> IMO, it is part of the conversation and is further confirmation that was
> normal. Various devices have had cli bugs over time. Does not hurt to
> remove it for debugging.
>
> By this do you mean to leave the original file alone or my version is fine? Sorry excuse my ignorance here.

Leave the modification for testing, but to be pedantic, try to restore the
check once everything else is working.

> Something must be up alright, here's the contents of the raw file that was created...
>
> ===
> Error: Usage: /usr/bin/zpelogin [-dhSV] [-m|M] [-autoenable] [-noenable] [-c command] [-Evar=x] [-e enable-password] [-f cloginrc-file] [-p user-password] [-r passphrase] [-s script-file] [-t timeout] [-u username] [-v vty-password] [-w enable-username] [-x command-file] [-y ssh_cypher_type] router [router...]
> ===
>
> Any ideas?

my mistake; the hostname is missing, whatever it is:

eval `rancid -C -t zpe <hostname>` >& raw

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@www.shrubbery.net
https://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Assistance creating a perl module to use with ZPE Systems NodegridOS Devices [ In reply to ]
?On 15/10/2021, 20:16, "heasley" <heas@shrubbery.net> wrote:

CAUTION[External]: This email originated from outside of the organisation. Do not click on links or open the attachments unless you recognise the sender and know the content is safe.


Fri, Oct 15, 2021 at 07:10:45PM +0000, Mick O'Donovan:
> IMO, it is part of the conversation and is further confirmation that was
> normal. Various devices have had cli bugs over time. Does not hurt to
> remove it for debugging.
>
> By this do you mean to leave the original file alone or my version is fine? Sorry excuse my ignorance here.

Leave the modification for testing, but to be pedantic, try to restore the
check once everything else is working.

> Something must be up alright, here's the contents of the raw file that was created...
>
> ===
> Error: Usage: /usr/bin/zpelogin [-dhSV] [-m|M] [-autoenable] [-noenable] [-c command] [-Evar=x] [-e enable-password] [-f cloginrc-file] [-p user-password] [-r passphrase] [-s script-file] [-t timeout] [-u username] [-v vty-password] [-w enable-username] [-x command-file] [-y ssh_cypher_type] router [router...]
> ===
>
> Any ideas?

my mistake; the hostname is missing, whatever it is:

eval `rancid -C -t zpe <hostname>` >& raw

The raw file is flawless!

This is most frustrating ?

The raw file shows:

1. the login
2. command 1 being run
3. the command 1 output (in full and as expected)
4. command 2 being run (I've reduced it to just two commands - "show system/about" and "export_settings")
5. the command 2 output (in full and as expected)
6. the exit being issued and successfully exiting

Mick


_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@www.shrubbery.net
https://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Assistance creating a perl module to use with ZPE Systems NodegridOS Devices [ In reply to ]
I'm looking to start backing up some ZPE console servers, so, of course, I
checked to see if rancid could do it, which led me to to this thread.

The CentOS 7 system I'm testing this on has an older version of rancid
(rancid-3.12-1.el7.x86_64) which caused a bit of confusion, as I don't
have the eos module...but I was still able to reproduce what you did, down
to having rancid run zpelogin, run the two commands, receive all the
expected output, exit, save the entire raw session to a file, and then
complain.

executing zpelogin -t 90 -c"show system/about;export_settings"
devicename
PROMPT MATCH: \[user@hostname /\]#
HIT COMMAND:[user@hostname /]# show system/about
In ShowVersion: [user@hostname /]# show system/about
devicename: End of run not found
devicename: found_end is false
!sed pba version: none

Did you (or anyone else) ever get further than this with ZPE backups?

On Fri, 15 Oct 2021, Mick O'Donovan wrote:

>
> ?On 15/10/2021, 20:16, "heasley" <heas@shrubbery.net> wrote:
>
> CAUTION[External]: This email originated from outside of the organisation. Do not click on links or open the attachments unless you recognise the sender and know the content is safe.
>
>
> Fri, Oct 15, 2021 at 07:10:45PM +0000, Mick O'Donovan:
> > IMO, it is part of the conversation and is further confirmation that was
> > normal. Various devices have had cli bugs over time. Does not hurt to
> > remove it for debugging.
> >
> > By this do you mean to leave the original file alone or my version is fine? Sorry excuse my ignorance here.
>
> Leave the modification for testing, but to be pedantic, try to restore the
> check once everything else is working.
>
> > Something must be up alright, here's the contents of the raw file that was created...
> >
> > ===
> > Error: Usage: /usr/bin/zpelogin [-dhSV] [-m|M] [-autoenable] [-noenable] [-c command] [-Evar=x] [-e enable-password] [-f cloginrc-file] [-p user-password] [-r passphrase] [-s script-file] [-t timeout] [-u username] [-v vty-password] [-w enable-username] [-x command-file] [-y ssh_cypher_type] router [router...]
> > ===
> >
> > Any ideas?
>
> my mistake; the hostname is missing, whatever it is:
>
> eval `rancid -C -t zpe <hostname>` >& raw
>
> The raw file is flawless!
>
> This is most frustrating ?
>
> The raw file shows:
>
> 1. the login
> 2. command 1 being run
> 3. the command 1 output (in full and as expected)
> 4. command 2 being run (I've reduced it to just two commands - "show system/about" and "export_settings")
> 5. the command 2 output (in full and as expected)
> 6. the exit being issued and successfully exiting
>
> Mick
>
>
> _______________________________________________
> Rancid-discuss mailing list
> Rancid-discuss@www.shrubbery.net
> https://www.shrubbery.net/mailman/listinfo/rancid-discuss
>

----------------------------------------------------------------------
Jon Lewis, MCP :) | I route
StackPath, Sr. Neteng | therefore you are
_________ http://www.lewis.org/~jlewis/pgp for PGP public key_________
Re: Assistance creating a perl module to use with ZPE Systems NodegridOS Devices [ In reply to ]
On Tue, Feb 15, 2022 at 01:35:07PM -0500, Jon Lewis wrote:
> The CentOS 7 system I'm testing this on has an older version of rancid
> (rancid-3.12-1.el7.x86_64) which caused a bit of confusion, as I don't

rancid-3.13-3.el7 is in testing now if you want to update and provide testing feedback:

https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2022-076fdd31f3

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@www.shrubbery.net
https://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Assistance creating a perl module to use with ZPE Systems NodegridOS Devices [ In reply to ]
Tue, Feb 15, 2022 at 01:35:07PM -0500, Jon Lewis:
> I'm looking to start backing up some ZPE console servers, so, of course, I
> checked to see if rancid could do it, which led me to to this thread.
>
> The CentOS 7 system I'm testing this on has an older version of rancid
> (rancid-3.12-1.el7.x86_64) which caused a bit of confusion, as I don't
> have the eos module...but I was still able to reproduce what you did, down
> to having rancid run zpelogin, run the two commands, receive all the
> expected output, exit, save the entire raw session to a file, and then
> complain.
>
> executing zpelogin -t 90 -c"show system/about;export_settings"
> devicename
> PROMPT MATCH: \[user@hostname /\]#
> HIT COMMAND:[user@hostname /]# show system/about
> In ShowVersion: [user@hostname /]# show system/about
> devicename: End of run not found
> devicename: found_end is false
> !sed pba version: none
>
> Did you (or anyone else) ever get further than this with ZPE backups?

If one of you shares the raw file and the code, i can try to help resolve
that failure.

> On Fri, 15 Oct 2021, Mick O'Donovan wrote:
>
> >
> > ?On 15/10/2021, 20:16, "heasley" <heas@shrubbery.net> wrote:
> >
> > Fri, Oct 15, 2021 at 07:10:45PM +0000, Mick O'Donovan:
> > > IMO, it is part of the conversation and is further confirmation that was
> > > normal. Various devices have had cli bugs over time. Does not hurt to
> > > remove it for debugging.
> > >
> > > By this do you mean to leave the original file alone or my version is fine? Sorry excuse my ignorance here.
> >
> > Leave the modification for testing, but to be pedantic, try to restore the
> > check once everything else is working.
> >
> > > Something must be up alright, here's the contents of the raw file that was created...
> > >
> > > ===
> > > Error: Usage: /usr/bin/zpelogin [-dhSV] [-m|M] [-autoenable] [-noenable] [-c command] [-Evar=x] [-e enable-password] [-f cloginrc-file] [-p user-password] [-r passphrase] [-s script-file] [-t timeout] [-u username] [-v vty-password] [-w enable-username] [-x command-file] [-y ssh_cypher_type] router [router...]
> > > ===
> > >
> > > Any ideas?
> >
> > my mistake; the hostname is missing, whatever it is:
> >
> > eval `rancid -C -t zpe <hostname>` >& raw
> >
> > The raw file is flawless!
> >
> > This is most frustrating ?
> >
> > The raw file shows:
> >
> > 1. the login
> > 2. command 1 being run
> > 3. the command 1 output (in full and as expected)
> > 4. command 2 being run (I've reduced it to just two commands - "show system/about" and "export_settings")
> > 5. the command 2 output (in full and as expected)
> > 6. the exit being issued and successfully exiting
> >
> > Mick

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@www.shrubbery.net
https://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Assistance creating a perl module to use with ZPE Systems NodegridOS Devices [ In reply to ]
Hi Heas, Jon, community,


I’m overdue sharing this module content and for this I apologise.



I have this working for several months now for ZPE devices using the following configuration.



--- File /etc/rancid/rancid.types.base or /etc/rancid/rancid.types.conf ---



# This section to the next hash governs ZPE devices running Nodegrid OS

zpe;script;zperancid -t zpe

zpe;login;zpelogin

zpe;module;zpe

zpe;inloop;zpe::inloop

zpe;command;zpe::ShowVersion;show system/about

zpe;command;zpe::ShowConfig;export_settings

#



--- File /usr/share/perl5/rancid/zpe.pm (module) in perl5 module directory ---

Take a copy of the IOS module (for example – ios.pm) and strip this down to remove the sections which are not relevant to you. For example, as you can see in my rancid types above, I’m only currently interested, in my use case, in the “ShowVersion” and “ShowConfig” *sub* sections.



Then you simply edit the existing inloop part of the module as follows



# main loop of input of device output

sub inloop {

    my($INPUT, $OUTPUT) = @_;

    my($cmd, $rval);



TOP: while(<$INPUT>) {

                tr/\015//d;

                if ((/\>\s?logout$/) || $found_end) {

                    $clean_run=1;

                    last;

                }

                if (/^Error:/) {

                    print STDOUT ("$host complogin error: $_");

                    print STDERR ("$host complogin error: $_") if ($debug);

                    $clean_run=0;

                    last;

                }

               

        while (/[>#][\s\a]*($cmds_regexp)\s*$/) {

                    $cmd = $1;

                    if (!defined($prompt)) {

                                $prompt = ($_ =~ /^([^#>]+[#>])/)[0];

                                $prompt =~ s/([][}{)(\\])/\\$1/g;

                                print STDERR ("PROMPT MATCH: $prompt\n") if ($debug);

                    }

                    print STDERR ("HIT COMMAND:$_") if ($debug);

                    if (! defined($commands{$cmd})) {

                                print STDERR "$host: found unexpected command - \"$cmd\"\n";

                                $clean_run = 0;

                                last TOP;

                    }

                    if (! defined(&{$commands{$cmd}})) {

                                printf(STDERR "$host: undefined function - \"%s\"\n",

                                       $commands{$cmd});

                                $clean_run = 0;

                                last TOP;

                    }

                    $rval = &{$commands{$cmd}}($INPUT, $OUTPUT, $cmd);

                    delete($commands{$cmd});

                    if ($rval == -1) {

                                $clean_run = 0;

                                last TOP;

                    }

                }

    }

}



The root of the fix to get this working is the *while* line and the *prompt* lines and the associated regex



For completeness, I separated out the bin files and duplicated the clogin to zpelogin and rancid to zperancid aswell – but this may not be necessary for you.



HTH,



Mick





From: heasley <heas@shrubbery.net>
Date: Thursday, 17 February 2022 at 04:45
To: Jon Lewis <jlewis@lewis.org>
Cc: Mick O'Donovan <mick.odonovan@heanet.ie>, rancid-discuss@www.shrubbery.net <rancid-discuss@sea.shrubbery.net>
Subject: Re: [rancid] Assistance creating a perl module to use with ZPE Systems NodegridOS Devices


If one of you shares the raw file and the code, i can try to help resolve
that failure.