Mailing List Archive

clogin commenting script commands following multiple blanks lines
So here's an odd thing I just ran across. Running clogin with a script with multiple blank lines has some very unexpected behaviour. It looks like it's turning \n\n into \n;, with the net effect of commenting out any command that follows two blank lines. Reproducible on ubuntu 12.04 and OSX 10.13 with stock 3.8 source. Works as expected in 2.3.8 ubuntu packages.

The culprit is definitely in the "# handle escaped ;s in commands, and ;; and ^;" section of clogin (rolling that block back to what was in 2.3.8 fixes it), but trying to grok that in expect language to provide a real fix makes my head hurt, so I'll leave this as a bug report. Examples below.

thanks,
-e

3.8 output:
erikm@status:~/src/rancid-3.8$ cat ~/test1
show ip int br | inc 701


show ip int br | inc 701
erikm@status:~/src/rancid-3.8$ ~/rancidtest/bin/clogin -x ~/test1 csw1.xxx
spawn ssh -c aes192-ctr -x -l erikm csw1.xxx
...
csw1.xxx#terminal width 132
csw1.xxx#show ip int br | inc 701
Vlan701 10.254.248.1 YES manual up up
csw1.xxx#
csw1.xxx#;show ip int br | inc 701
csw1.xxx#
csw1.xxx#exit
Connection to csw1.xxx closed.
erikm@status:~/src/rancid-3.8$ cat ~/test2
show ip int br | inc 701

show ip int br | inc 1blanks


show ip int br | inc 2blanks



show ip int br | inc 3blanks




show ip int br | inc 4blanks





show ip int br | inc 5blanks
erikm@status:~/src/rancid-3.8$ ~/rancidtest/bin/clogin -x ~/test2 csw1.xxx
...
csw1.xxx#show ip int br | inc 701
Vlan701 10.254.248.1 YES manual up up
csw1.xxx#
csw1.xxx#show ip int br | inc 1blanks
csw1.xxx#
csw1.xxx#;show ip int br | inc 2blanks
csw1.xxx#
csw1.xxx#;
csw1.xxx#show ip int br | inc 3blanks
csw1.xxx#
csw1.xxx#;
csw1.xxx#
csw1.xxx#show ip int br | inc 4blanks
csw1.xxx#
csw1.xxx#;
csw1.xxx#
csw1.xxx#;show ip int br | inc 5blanks
csw1.xxx#
csw1.xxx#exit
Connection to csw1.xxx closed by remote host.



2.3.8 output:
erikm@status:~/src/rancid-3.8$ clogin -x ~/test2 csw1.xxx
...
csw1.xxx#show ip int br | inc 701
Vlan701 10.254.248.1 YES manual up up
csw1.xxx#
csw1.xxx#show ip int br | inc 1blanks
csw1.xxx#
csw1.xxx#
csw1.xxx#show ip int br | inc 2blanks
csw1.xxx#
csw1.xxx#
csw1.xxx#
csw1.xxx#show ip int br | inc 3blanks
csw1.xxx#
csw1.xxx#
csw1.xxx#
csw1.xxx#
csw1.xxx#show ip int br | inc 4blanks
csw1.xxx#
csw1.xxx#
csw1.xxx#
csw1.xxx#
csw1.xxx#
csw1.xxx#show ip int br | inc 5blanks
csw1.xxx#
csw1.xxx#exit
Connection to csw1.xxx closed.


_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: clogin commenting script commands following multiple blanks lines [ In reply to ]
Sat, Sep 29, 2018 at 03:19:00PM -0700, Erik Muller:
> So here's an odd thing I just ran across. Running clogin with a script with multiple blank lines has some very unexpected behaviour. It looks like it's turning \n\n into \n;, with the net effect of commenting out any command that follows two blank lines. Reproducible on ubuntu 12.04 and OSX 10.13 with stock 3.8 source. Works as expected in 2.3.8 ubuntu packages.
>
> The culprit is definitely in the "# handle escaped ;s in commands, and ;; and ^;" section of clogin (rolling that block back to what was in 2.3.8 fixes it), but trying to grok that in expect language to provide a real fix makes my head hurt, so I'll leave this as a bug report. Examples below.
>
> thanks,
> -e

I believe that patch addresses this bug.

Index: bin/clogin.in
===================================================================
--- bin/clogin.in (revision 3909)
+++ bin/clogin.in (working copy)
@@ -76,12 +76,12 @@
}

# handle escaped ;s in commands, and ;; and ^;
- regsub -all {([^\\]);;} $command "\\1;\u002;" esccommand
- regsub {^;} $esccommand "\u002;" command
- set sep "\\1\u001"
- regsub -all {([^\\])\;} $command "$sep" esccommand
- set sep "\u001"
- set commands [split $esccommand $sep]
+ regsub -all {([^\\]);} $command "\\1\u002;" esccommand
+ regsub -all {([^\\]);;} $esccommand "\\1;\u002;" command
+ regsub {^;} $command "\u002;" esccommand
+ regsub -all {[\\];} $esccommand ";" command
+ set sep "\u002;"
+ set commands [split $command $sep]
set num_commands [llength $commands]
set rshfail 0
for {set i 0} {$i < $num_commands && !$rshfail} { incr i} {
@@ -373,12 +373,12 @@
log_user 0

# handle escaped ;s in commands, and ;; and ^;
- regsub -all {([^\\]);;} $command "\\1;\u002;" esccommand
- regsub {^;} $esccommand "\u002;" command
- set sep "\\1\u001"
- regsub -all {([^\\])\;} $command "$sep" esccommand
- set sep "\u001"
- set commands [split $esccommand $sep]
+ regsub -all {([^\\]);} $command "\\1\u002;" esccommand
+ regsub -all {([^\\]);;} $esccommand "\\1;\u002;" command
+ regsub {^;} $command "\u002;" esccommand
+ regsub -all {[\\];} $esccommand ";" command
+ set sep "\u002;"
+ set commands [split $command $sep]
set num_commands [llength $commands]
# the pager can not be turned off on the PIX, so we have to look
# for the "More" prompt. the extreme is equally obnoxious in pre-12.3 XOS,

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: clogin commenting script commands following multiple blanks lines [ In reply to ]
On 10/24/18 13:28 , heasley wrote:
> Sat, Sep 29, 2018 at 03:19:00PM -0700, Erik Muller:
>> So here's an odd thing I just ran across. Running clogin with a script with multiple blank lines has some very unexpected behaviour. It looks like it's turning \n\n into \n;, with the net effect of commenting out any command that follows two blank lines. Reproducible on ubuntu 12.04 and OSX 10.13 with stock 3.8 source. Works as expected in 2.3.8 ubuntu packages.
>>
>> The culprit is definitely in the "# handle escaped ;s in commands, and ;; and ^;" section of clogin (rolling that block back to what was in 2.3.8 fixes it), but trying to grok that in expect language to provide a real fix makes my head hurt, so I'll leave this as a bug report. Examples below.
>>
>> thanks,
>> -e
>
> I believe that patch addresses this bug.


That definitely fixes the blanks-may-comment-the-next-line issue, though it looks like it also breaks handling of escaped ;s as well.

before:
$ clogin-3.8 -c 'sh ip bgp nei | inc 1\; ; show ver | inc ^Model Num' fl1-as01
...
fl1-as01#sh ip bgp nei | inc 1;
Route to peer address reachability Up: 1; Down: 0
Connections established 1; dropped 0


with patch:
$ clogin -c 'sh ip bgp nei | inc 1\; ; show ver | inc ^Model Num' fl1-as01
...
fl1-as01#sh ip bgp nei | inc 1
BGP neighbor is x.x.x.x, remote AS 4200000000, external link



And it seems to eat leading semicolon comments as well:

$ cat ~/clogin-blanks-test2
show bridge
; just a comment
$ clogin -x ~/clogin-blanks-test2 fl1-as01.polaris.corp
...
fl1-as01#show bridge
fl1-as01#
fl1-as01#
fl1-as01#
fl1-as01# just a comment
^
% Invalid input detected at '^' marker.



_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: clogin commenting script commands following multiple blanks lines [ In reply to ]
Wed, Oct 24, 2018 at 04:43:41PM -0400, Erik Muller:
> On 10/24/18 13:28 , heasley wrote:
> > Sat, Sep 29, 2018 at 03:19:00PM -0700, Erik Muller:
> >> So here's an odd thing I just ran across. Running clogin with a script with multiple blank lines has some very unexpected behaviour. It looks like it's turning \n\n into \n;, with the net effect of commenting out any command that follows two blank lines. Reproducible on ubuntu 12.04 and OSX 10.13 with stock 3.8 source. Works as expected in 2.3.8 ubuntu packages.
> >>
> >> The culprit is definitely in the "# handle escaped ;s in commands, and ;; and ^;" section of clogin (rolling that block back to what was in 2.3.8 fixes it), but trying to grok that in expect language to provide a real fix makes my head hurt, so I'll leave this as a bug report. Examples below.
> >>
> >> thanks,
> >> -e
> >
> > I believe that patch addresses this bug.
>
>
> That definitely fixes the blanks-may-comment-the-next-line issue, though it looks like it also breaks handling of escaped ;s as well.
>
> before:
> $ clogin-3.8 -c 'sh ip bgp nei | inc 1\; ; show ver | inc ^Model Num' fl1-as01
> ...
> fl1-as01#sh ip bgp nei | inc 1;
> Route to peer address reachability Up: 1; Down: 0
> Connections established 1; dropped 0
>
>
> with patch:
> $ clogin -c 'sh ip bgp nei | inc 1\; ; show ver | inc ^Model Num' fl1-as01
> ...
> fl1-as01#sh ip bgp nei | inc 1
> BGP neighbor is x.x.x.x, remote AS 4200000000, external link
>
>
>
> And it seems to eat leading semicolon comments as well:
>
> $ cat ~/clogin-blanks-test2
> show bridge
> ; just a comment
> $ clogin -x ~/clogin-blanks-test2 fl1-as01.polaris.corp
> ...
> fl1-as01#show bridge
> fl1-as01#
> fl1-as01#
> fl1-as01#
> fl1-as01# just a comment
> ^
> % Invalid input detected at '^' marker.
>

I missed that in your example, but noticed it in testing...after I sent that
patch. This is my final patch, i think. I havent committed it yet, as I
want to review it once more.

Index: bin/clogin.in
===================================================================
--- bin/clogin.in (revision 3909)
+++ bin/clogin.in (working copy)
@@ -76,11 +76,12 @@
}

# handle escaped ;s in commands, and ;; and ^;
- regsub -all {([^\\]);;} $command "\\1;\u002;" esccommand
- regsub {^;} $esccommand "\u002;" command
- set sep "\\1\u001"
- regsub -all {([^\\])\;} $command "$sep" esccommand
- set sep "\u001"
+ regsub -all {([^\\]);} $command "\\1\u002;" esccommand
+ regsub -all {([^\\]);;} $esccommand "\\1;\u002;" command
+ regsub {^;} $command "\u002;" esccommand
+ regsub -all {[\\];} $esccommand ";" command
+ regsub -all {\u002;} $command "\u002" esccommand
+ set sep "\u002;"
set commands [split $esccommand $sep]
set num_commands [llength $commands]
set rshfail 0
@@ -373,11 +374,12 @@
log_user 0

# handle escaped ;s in commands, and ;; and ^;
- regsub -all {([^\\]);;} $command "\\1;\u002;" esccommand
- regsub {^;} $esccommand "\u002;" command
- set sep "\\1\u001"
- regsub -all {([^\\])\;} $command "$sep" esccommand
- set sep "\u001"
+ regsub -all {([^\\]);} $command "\\1\u002;" esccommand
+ regsub -all {([^\\]);;} $esccommand "\\1;\u002;" command
+ regsub {^;} $command "\u002;" esccommand
+ regsub -all {[\\];} $esccommand ";" command
+ regsub -all {\u002;} $command "\u002" esccommand
+ set sep "\u002"
set commands [split $esccommand $sep]
set num_commands [llength $commands]
# the pager can not be turned off on the PIX, so we have to look
@@ -384,11 +386,7 @@
# for the "More" prompt. the extreme is equally obnoxious in pre-12.3 XOS,
# with a global switch in the config.
for {set i 0} {$i < $num_commands} { incr i} {
- if { [lindex $commands $i] == "\u002" } {
- send -- "\r"
- } else {
- send -- "[subst -nocommands [lindex $commands $i]]\r"
- }
+ send -h -- "[subst -nocommands [lindex $commands $i]]\r"
expect {
-re "^\b+" { exp_continue }
-re "^\[^\n\r *]*$reprompt" { send_user -- "$expect_out(buffer)"

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: clogin commenting script commands following multiple blanks lines [ In reply to ]
Wed, Oct 24, 2018 at 09:19:31PM +0000, heasley:
> I missed that in your example, but noticed it in testing...after I sent that
> patch. This is my final patch, i think. I havent committed it yet, as I
> want to review it once more.

ok; committed. Either the alpha tarball or the svn repo. Welcome testers,
esp for palo alto, of which I have none.

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: clogin commenting script commands following multiple blanks lines [ In reply to ]
On 10/24/18 18:32 , heasley wrote:
> Wed, Oct 24, 2018 at 09:19:31PM +0000, heasley:
>> I missed that in your example, but noticed it in testing...after I sent that
>> patch. This is my final patch, i think. I havent committed it yet, as I
>> want to review it once more.
>
> ok; committed. Either the alpha tarball or the svn repo. Welcome testers,
> esp for palo alto, of which I have none.

That one was really close, but still didn't pass through ^; from command files properly.
After a bit more tweaking, the following change seems to get it to handle all my test cases correctly.

erikm@vpn41:~/ports-dev [15:21 - 1497]$ diff -Naur /opt/local/libexec/rancid/clogin-3.99.99.bak /opt/local/libexec/rancid/clogin-3.99.99
--- /opt/local/libexec/rancid/clogin-3.99.99.bak 2018-10-24 19:46:30.000000000 -0400
+++ /opt/local/libexec/rancid/clogin-3.99.99 2018-10-25 15:21:18.000000000 -0400
@@ -253,7 +253,8 @@
}
set cmd_text [read $cmd_fd]
close $cmd_fd
- set command [join [split $cmd_text \n] \;]
+ regsub -all {;} $cmd_text "\\;" cmd_text
+ set command [join [split $cmd_text \n] \u002;]
set do_command 1
# 'ssh -c' cypher type
} -y* {
@@ -444,13 +445,8 @@
continue;
}

- # handle escaped ;s in commands, and ;; and ^;
- regsub -all {([^\\]);} $command "\\1\u002;" esccommand
- regsub -all {([^\\]);;} $esccommand "\\1;\u002;" command
- regsub {^;} $command "\u002;" esccommand
- regsub -all {[\\];} $esccommand ";" command
- regsub -all {\u002;} $command "\u002" esccommand
- set sep "\u002;"
+ set esccommand [escapecommandlist $command]
+ set sep "\u002"
set commands [split $esccommand $sep]
set num_commands [llength $commands]
set rshfail 0
@@ -724,6 +720,21 @@
return 0
}

+# handle escaped ;s in commands, and ;; and ^;
+proc escapecommandlist {command} {
+ # \; should be passed through as a ;
+ # ^; should be treated as a comment (when coming from a command file)
+ # ;; represents a literal ; before a subsequent command (?)
+ # other ;s are separators between items in a sequence of commands
+ # note this is processed as one big multiline text blob, so ^ anchors may
+ # not work as expected
+ regsub -all {([^\\\u002]);} $command "\\1\u002;" esccommand
+ regsub -all {([^\\\u00a\u00d\u002]);;} $esccommand "\\1;\u002;" command
+ regsub -all {\u002;} $command "\u002" esccommand
+ regsub -all {[\\];} $esccommand ";" command
+ return $command
+}
+
# Run commands given on the command line.
proc run_commands { prompt command } {
global do_interact do_saveconfig in_proc platform
@@ -742,12 +753,7 @@
# this is the only way i see to get rid of more prompts in o/p..grrrrr
log_user 0

- # handle escaped ;s in commands, and ;; and ^;
- regsub -all {([^\\]);} $command "\\1\u002;" esccommand
- regsub -all {([^\\]);;} $esccommand "\\1;\u002;" command
- regsub {^;} $command "\u002;" esccommand
- regsub -all {[\\];} $esccommand ";" command
- regsub -all {\u002;} $command "\u002" esccommand
+ set esccommand [escapecommandlist $command]
set sep "\u002"
set commands [split $esccommand $sep]
set num_commands [llength $commands]


_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: clogin commenting script commands following multiple blanks lines [ In reply to ]
> Am 25.10.2018 um 12:53 schrieb Erik Muller <erikm@buh.org>:
>
>> On 10/24/18 18:32 , heasley wrote:
>> Wed, Oct 24, 2018 at 09:19:31PM +0000, heasley:
>>> I missed that in your example, but noticed it in testing...after I sent that
>>> patch. This is my final patch, i think. I havent committed it yet, as I
>>> want to review it once more.
>>
>> ok; committed. Either the alpha tarball or the svn repo. Welcome testers,
>> esp for palo alto, of which I have none.
>
> That one was really close, but still didn't pass through ^; from command files properly.
> After a bit more tweaking, the following change seems to get it to handle all my test cases correctly.
>

You have to escape the ; in the file (\;). The An argument could be made either way, i leN toward not altering the behavior of -x.

> erikm@vpn41:~/ports-dev [15:21 - 1497]$ diff -Naur /opt/local/libexec/rancid/clogin-3.99.99.bak /opt/local/libexec/rancid/clogin-3.99.99
> --- /opt/local/libexec/rancid/clogin-3.99.99.bak 2018-10-24 19:46:30.000000000 -0400
> +++ /opt/local/libexec/rancid/clogin-3.99.99 2018-10-25 15:21:18.000000000 -0400
> @@ -253,7 +253,8 @@
> }
> set cmd_text [read $cmd_fd]
> close $cmd_fd
> - set command [join [split $cmd_text \n] \;]
> + regsub -all {;} $cmd_text "\\;" cmd_text
> + set command [join [split $cmd_text \n] \u002;]
> set do_command 1
> # 'ssh -c' cypher type
> } -y* {
> @@ -444,13 +445,8 @@
> continue;
> }
>
> - # handle escaped ;s in commands, and ;; and ^;
> - regsub -all {([^\\]);} $command "\\1\u002;" esccommand
> - regsub -all {([^\\]);;} $esccommand "\\1;\u002;" command
> - regsub {^;} $command "\u002;" esccommand
> - regsub -all {[\\];} $esccommand ";" command
> - regsub -all {\u002;} $command "\u002" esccommand
> - set sep "\u002;"
> + set esccommand [escapecommandlist $command]
> + set sep "\u002"
> set commands [split $esccommand $sep]
> set num_commands [llength $commands]
> set rshfail 0
> @@ -724,6 +720,21 @@
> return 0
> }
>
> +# handle escaped ;s in commands, and ;; and ^;
> +proc escapecommandlist {command} {
> + # \; should be passed through as a ;
> + # ^; should be treated as a comment (when coming from a command file)
> + # ;; represents a literal ; before a subsequent command (?)
> + # other ;s are separators between items in a sequence of commands
> + # note this is processed as one big multiline text blob, so ^ anchors may
> + # not work as expected
> + regsub -all {([^\\\u002]);} $command "\\1\u002;" esccommand
> + regsub -all {([^\\\u00a\u00d\u002]);;} $esccommand "\\1;\u002;" command
> + regsub -all {\u002;} $command "\u002" esccommand
> + regsub -all {[\\];} $esccommand ";" command
> + return $command
> +}
> +
> # Run commands given on the command line.
> proc run_commands { prompt command } {
> global do_interact do_saveconfig in_proc platform
> @@ -742,12 +753,7 @@
> # this is the only way i see to get rid of more prompts in o/p..grrrrr
> log_user 0
>
> - # handle escaped ;s in commands, and ;; and ^;
> - regsub -all {([^\\]);} $command "\\1\u002;" esccommand
> - regsub -all {([^\\]);;} $esccommand "\\1;\u002;" command
> - regsub {^;} $command "\u002;" esccommand
> - regsub -all {[\\];} $esccommand ";" command
> - regsub -all {\u002;} $command "\u002" esccommand
> + set esccommand [escapecommandlist $command]
> set sep "\u002"
> set commands [split $esccommand $sep]
> set num_commands [llength $commands]
>

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: clogin commenting script commands following multiple blanks lines [ In reply to ]
On 10/25/18 16:02 , Heasley wrote:
>
>
>> Am 25.10.2018 um 12:53 schrieb Erik Muller <erikm@buh.org>:
>>
>>> On 10/24/18 18:32 , heasley wrote:
>>> Wed, Oct 24, 2018 at 09:19:31PM +0000, heasley:
>>>> I missed that in your example, but noticed it in testing...after I sent that
>>>> patch. This is my final patch, i think. I havent committed it yet, as I
>>>> want to review it once more.
>>>
>>> ok; committed. Either the alpha tarball or the svn repo. Welcome testers,
>>> esp for palo alto, of which I have none.
>>
>> That one was really close, but still didn't pass through ^; from command files properly.
>> After a bit more tweaking, the following change seems to get it to handle all my test cases correctly.
>>
>
> You have to escape the ; in the file (\;). The An argument could be made either way, i leN toward not altering the behavior of -x.

I can live with that. Looks like ^; was equally broken in 2.x as well, and
really it should be !comments instead since ~ios 10 anyway ;)

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss