Mailing List Archive

[BUG?] sftp is echoing back prompt and commands in batch mode
Short description: All comands sent to server in batch mode are being
echoed back along with prompt.
Software Version: "OpenSSH_7.9p1, OpenSSL 1.1.1 11 Sep 2018" (as
printed by ssh -V)
Server and client systems: ArchLinux x86_64 fully updated as of 2018-11-02.

## How to reproduce

Provided that you have proper SSH key authentication in place, from
command line run:
echo "dir /var" | sftp -b - 0

## Expected output

/var/backups /var/cache /var/lib /var/local /var/lock
/var/log /var/mail /var/opt /var/puppet /var/run
/var/spool /var/tmp /var/www

## Actual output

sftp> dir /var
/var/backups /var/cache /var/lib /var/local /var/lock
/var/log /var/mail /var/opt /var/puppet /var/run
/var/spool /var/tmp /var/www

Please note the first line is the server prompt followed by the command echo.

## Workaround

Expunge the first line from the client output
echo "dir /var" | sftp -b - 0 | tail -n +2

## Notes
Batch mode is meant for script automation, not human operations, so
echoing the prompt and the commands doesn't make any sense and creates
problems while processing the output.
A possible solution that's also backward compatible with all scripts
using batch mode is to allow for a special prefix character to
commands to avoid any echoing, just like the "-" is used to avoid a
single failing command to terminate the client. Something like "@" or
"#" would be ok IMHO.

--
Vincenzo Romano - NotOrAnd.IT
Information Technologies
--
NON QVIETIS MARIBVS NAVTA PERITVS
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: [BUG?] sftp is echoing back prompt and commands in batch mode [ In reply to ]
If you truly intend this as a bug report, you should file it at
bugzilla.mindrot.org.

On Fri, Nov 02, 2018 at 12:25:22 +0100, Vincenzo Romano wrote:
> Short description: All comands sent to server in batch mode are being
> echoed back along with prompt.
> Software Version: "OpenSSH_7.9p1, OpenSSL 1.1.1 11 Sep 2018" (as
> printed by ssh -V)
> Server and client systems: ArchLinux x86_64 fully updated as of 2018-11-02.
>
> ## How to reproduce
>
> Provided that you have proper SSH key authentication in place, from
> command line run:
> echo "dir /var" | sftp -b - 0
>
> ## Expected output
>
> /var/backups /var/cache /var/lib /var/local /var/lock
> /var/log /var/mail /var/opt /var/puppet /var/run
> /var/spool /var/tmp /var/www
>
> ## Actual output
>
> sftp> dir /var
> /var/backups /var/cache /var/lib /var/local /var/lock
> /var/log /var/mail /var/opt /var/puppet /var/run
> /var/spool /var/tmp /var/www
>
> Please note the first line is the server prompt followed by the command echo.
>
> ## Workaround
>
> Expunge the first line from the client output
> echo "dir /var" | sftp -b - 0 | tail -n +2
>
> ## Notes
> Batch mode is meant for script automation, not human operations, so
> echoing the prompt and the commands doesn't make any sense and creates
> problems while processing the output.
> A possible solution that's also backward compatible with all scripts
> using batch mode is to allow for a special prefix character to
> commands to avoid any echoing, just like the "-" is used to avoid a
> single failing command to terminate the client. Something like "@" or
> "#" would be ok IMHO.
>
> --
> Vincenzo Romano - NotOrAnd.IT
> Information Technologies
> --
> NON QVIETIS MARIBVS NAVTA PERITVS
> _______________________________________________
> openssh-unix-dev mailing list
> openssh-unix-dev@mindrot.org
> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev

--
Iain Morgan
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: [BUG?] sftp is echoing back prompt and commands in batch mode [ In reply to ]
Thanks, Iain.
I am willing to hear from other users whether anyone else sees this as a
bug before filing it.


--
Vincenzo Romano

Il giorno ven 2 nov 2018, 20:03 Iain Morgan <imorgan@nas.nasa.gov> ha
scritto:

> If you truly intend this as a bug report, you should file it at
> bugzilla.mindrot.org.
>
> On Fri, Nov 02, 2018 at 12:25:22 +0100, Vincenzo Romano wrote:
> > Short description: All comands sent to server in batch mode are being
> > echoed back along with prompt.
> > Software Version: "OpenSSH_7.9p1, OpenSSL 1.1.1 11 Sep 2018" (as
> > printed by ssh -V)
> > Server and client systems: ArchLinux x86_64 fully updated as of
> 2018-11-02.
> >
> > ## How to reproduce
> >
> > Provided that you have proper SSH key authentication in place, from
> > command line run:
> > echo "dir /var" | sftp -b - 0
> >
> > ## Expected output
> >
> > /var/backups /var/cache /var/lib /var/local /var/lock
> > /var/log /var/mail /var/opt /var/puppet /var/run
> > /var/spool /var/tmp /var/www
> >
> > ## Actual output
> >
> > sftp> dir /var
> > /var/backups /var/cache /var/lib /var/local /var/lock
> > /var/log /var/mail /var/opt /var/puppet /var/run
> > /var/spool /var/tmp /var/www
> >
> > Please note the first line is the server prompt followed by the command
> echo.
> >
> > ## Workaround
> >
> > Expunge the first line from the client output
> > echo "dir /var" | sftp -b - 0 | tail -n +2
> >
> > ## Notes
> > Batch mode is meant for script automation, not human operations, so
> > echoing the prompt and the commands doesn't make any sense and creates
> > problems while processing the output.
> > A possible solution that's also backward compatible with all scripts
> > using batch mode is to allow for a special prefix character to
> > commands to avoid any echoing, just like the "-" is used to avoid a
> > single failing command to terminate the client. Something like "@" or
> > "#" would be ok IMHO.
> >
> > --
> > Vincenzo Romano - NotOrAnd.IT
> > Information Technologies
> > --
> > NON QVIETIS MARIBVS NAVTA PERITVS
> > _______________________________________________
> > openssh-unix-dev mailing list
> > openssh-unix-dev@mindrot.org
> > https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
>
> --
> Iain Morgan
>

Il giorno ven 2 nov 2018, 20:03 Iain Morgan <imorgan@nas.nasa.gov> ha
scritto:

> If you truly intend this as a bug report, you should file it at
> bugzilla.mindrot.org.
>
> On Fri, Nov 02, 2018 at 12:25:22 +0100, Vincenzo Romano wrote:
> > Short description: All comands sent to server in batch mode are being
> > echoed back along with prompt.
> > Software Version: "OpenSSH_7.9p1, OpenSSL 1.1.1 11 Sep 2018" (as
> > printed by ssh -V)
> > Server and client systems: ArchLinux x86_64 fully updated as of
> 2018-11-02.
> >
> > ## How to reproduce
> >
> > Provided that you have proper SSH key authentication in place, from
> > command line run:
> > echo "dir /var" | sftp -b - 0
> >
> > ## Expected output
> >
> > /var/backups /var/cache /var/lib /var/local /var/lock
> > /var/log /var/mail /var/opt /var/puppet /var/run
> > /var/spool /var/tmp /var/www
> >
> > ## Actual output
> >
> > sftp> dir /var
> > /var/backups /var/cache /var/lib /var/local /var/lock
> > /var/log /var/mail /var/opt /var/puppet /var/run
> > /var/spool /var/tmp /var/www
> >
> > Please note the first line is the server prompt followed by the command
> echo.
> >
> > ## Workaround
> >
> > Expunge the first line from the client output
> > echo "dir /var" | sftp -b - 0 | tail -n +2
> >
> > ## Notes
> > Batch mode is meant for script automation, not human operations, so
> > echoing the prompt and the commands doesn't make any sense and creates
> > problems while processing the output.
> > A possible solution that's also backward compatible with all scripts
> > using batch mode is to allow for a special prefix character to
> > commands to avoid any echoing, just like the "-" is used to avoid a
> > single failing command to terminate the client. Something like "@" or
> > "#" would be ok IMHO.
> >
> > --
> > Vincenzo Romano - NotOrAnd.IT
> > Information Technologies
> > --
> > NON QVIETIS MARIBVS NAVTA PERITVS
> > _______________________________________________
> > openssh-unix-dev mailing list
> > openssh-unix-dev@mindrot.org
> > https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
>
> --
> Iain Morgan
>
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: [BUG?] sftp is echoing back prompt and commands in batch mode [ In reply to ]
I don't see it as a bug. As if I'm writing a batch script I want to see
the echo of the command and the output so if there is a failure I know
where the failure is.

I could see adding a mode not to echo back the command or prompt. But
it isn't a bug to me.

Ben



Vincenzo Romano wrote:
> Thanks, Iain.
> I am willing to hear from other users whether anyone else sees this as a
> bug before filing it.
>
>
> --
> Vincenzo Romano
>
> Il giorno ven 2 nov 2018, 20:03 Iain Morgan<imorgan@nas.nasa.gov> ha
> scritto:
>
>> If you truly intend this as a bug report, you should file it at
>> bugzilla.mindrot.org.
>>
>> On Fri, Nov 02, 2018 at 12:25:22 +0100, Vincenzo Romano wrote:
>>> Short description: All comands sent to server in batch mode are being
>>> echoed back along with prompt.
>>> Software Version: "OpenSSH_7.9p1, OpenSSL 1.1.1 11 Sep 2018" (as
>>> printed by ssh -V)
>>> Server and client systems: ArchLinux x86_64 fully updated as of
>> 2018-11-02.
>>> ## How to reproduce
>>>
>>> Provided that you have proper SSH key authentication in place, from
>>> command line run:
>>> echo "dir /var" | sftp -b - 0
>>>
>>> ## Expected output
>>>
>>> /var/backups /var/cache /var/lib /var/local /var/lock
>>> /var/log /var/mail /var/opt /var/puppet /var/run
>>> /var/spool /var/tmp /var/www
>>>
>>> ## Actual output
>>>
>>> sftp> dir /var
>>> /var/backups /var/cache /var/lib /var/local /var/lock
>>> /var/log /var/mail /var/opt /var/puppet /var/run
>>> /var/spool /var/tmp /var/www
>>>
>>> Please note the first line is the server prompt followed by the command
>> echo.
>>> ## Workaround
>>>
>>> Expunge the first line from the client output
>>> echo "dir /var" | sftp -b - 0 | tail -n +2
>>>
>>> ## Notes
>>> Batch mode is meant for script automation, not human operations, so
>>> echoing the prompt and the commands doesn't make any sense and creates
>>> problems while processing the output.
>>> A possible solution that's also backward compatible with all scripts
>>> using batch mode is to allow for a special prefix character to
>>> commands to avoid any echoing, just like the "-" is used to avoid a
>>> single failing command to terminate the client. Something like "@" or
>>> "#" would be ok IMHO.
>>>
>>> --
>>> Vincenzo Romano - NotOrAnd.IT
>>> Information Technologies
>>> --
>>> NON QVIETIS MARIBVS NAVTA PERITVS
>>> _______________________________________________
>>> openssh-unix-dev mailing list
>>> openssh-unix-dev@mindrot.org
>>> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
>> --
>> Iain Morgan
>>
>
> Il giorno ven 2 nov 2018, 20:03 Iain Morgan<imorgan@nas.nasa.gov> ha
> scritto:
>
>> If you truly intend this as a bug report, you should file it at
>> bugzilla.mindrot.org.
>>
>> On Fri, Nov 02, 2018 at 12:25:22 +0100, Vincenzo Romano wrote:
>>> Short description: All comands sent to server in batch mode are being
>>> echoed back along with prompt.
>>> Software Version: "OpenSSH_7.9p1, OpenSSL 1.1.1 11 Sep 2018" (as
>>> printed by ssh -V)
>>> Server and client systems: ArchLinux x86_64 fully updated as of
>> 2018-11-02.
>>> ## How to reproduce
>>>
>>> Provided that you have proper SSH key authentication in place, from
>>> command line run:
>>> echo "dir /var" | sftp -b - 0
>>>
>>> ## Expected output
>>>
>>> /var/backups /var/cache /var/lib /var/local /var/lock
>>> /var/log /var/mail /var/opt /var/puppet /var/run
>>> /var/spool /var/tmp /var/www
>>>
>>> ## Actual output
>>>
>>> sftp> dir /var
>>> /var/backups /var/cache /var/lib /var/local /var/lock
>>> /var/log /var/mail /var/opt /var/puppet /var/run
>>> /var/spool /var/tmp /var/www
>>>
>>> Please note the first line is the server prompt followed by the command
>> echo.
>>> ## Workaround
>>>
>>> Expunge the first line from the client output
>>> echo "dir /var" | sftp -b - 0 | tail -n +2
>>>
>>> ## Notes
>>> Batch mode is meant for script automation, not human operations, so
>>> echoing the prompt and the commands doesn't make any sense and creates
>>> problems while processing the output.
>>> A possible solution that's also backward compatible with all scripts
>>> using batch mode is to allow for a special prefix character to
>>> commands to avoid any echoing, just like the "-" is used to avoid a
>>> single failing command to terminate the client. Something like "@" or
>>> "#" would be ok IMHO.
>>>
>>> --
>>> Vincenzo Romano - NotOrAnd.IT
>>> Information Technologies
>>> --
>>> NON QVIETIS MARIBVS NAVTA PERITVS
>>> _______________________________________________
>>> openssh-unix-dev mailing list
>>> openssh-unix-dev@mindrot.org
>>> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
>> --
>> Iain Morgan
>>
> _______________________________________________
> openssh-unix-dev mailing list
> openssh-unix-dev@mindrot.org
> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: [BUG?] sftp is echoing back prompt and commands in batch mode [ In reply to ]
Il giorno dom 4 nov 2018 alle ore 01:45 Ben Lindstrom
<mouring@offwriting.org> ha scritto:
>
> I don't see it as a bug.

Yes, this is why I put a question mark in the subject.

> As if I'm writing a batch script I want to see the echo of the command and the output so if there is a failure I know where the failure is.

I see: you have a single batch file with no conditionals.
I have multiple dynamic 1-command batches with conditionals.
That makes the big difference.

> I could see adding a mode not to echo back the command or prompt.

Me too.

> But it isn't a bug to me.

This makes sense.

> Ben
>
>
>
> Vincenzo Romano wrote:
>
> Thanks, Iain.
> I am willing to hear from other users whether anyone else sees this as a
> bug before filing it.
>
>
> --
> Vincenzo Romano
>
> Il giorno ven 2 nov 2018, 20:03 Iain Morgan <imorgan@nas.nasa.gov> ha
> scritto:
>
> If you truly intend this as a bug report, you should file it at
> bugzilla.mindrot.org.
>
> On Fri, Nov 02, 2018 at 12:25:22 +0100, Vincenzo Romano wrote:
>
> Short description: All comands sent to server in batch mode are being
> echoed back along with prompt.
> Software Version: "OpenSSH_7.9p1, OpenSSL 1.1.1 11 Sep 2018" (as
> printed by ssh -V)
> Server and client systems: ArchLinux x86_64 fully updated as of
>
> 2018-11-02.
>
> ## How to reproduce
>
> Provided that you have proper SSH key authentication in place, from
> command line run:
> echo "dir /var" | sftp -b - 0
>
> ## Expected output
>
> /var/backups /var/cache /var/lib /var/local /var/lock
> /var/log /var/mail /var/opt /var/puppet /var/run
> /var/spool /var/tmp /var/www
>
> ## Actual output
>
> sftp> dir /var
> /var/backups /var/cache /var/lib /var/local /var/lock
> /var/log /var/mail /var/opt /var/puppet /var/run
> /var/spool /var/tmp /var/www
>
> Please note the first line is the server prompt followed by the command
>
> echo.
>
> ## Workaround
>
> Expunge the first line from the client output
> echo "dir /var" | sftp -b - 0 | tail -n +2
>
> ## Notes
> Batch mode is meant for script automation, not human operations, so
> echoing the prompt and the commands doesn't make any sense and creates
> problems while processing the output.
> A possible solution that's also backward compatible with all scripts
> using batch mode is to allow for a special prefix character to
> commands to avoid any echoing, just like the "-" is used to avoid a
> single failing command to terminate the client. Something like "@" or
> "#" would be ok IMHO.
>
> --
> Vincenzo Romano - NotOrAnd.IT
> Information Technologies
> --
> NON QVIETIS MARIBVS NAVTA PERITVS
> _______________________________________________
> openssh-unix-dev mailing list
> openssh-unix-dev@mindrot.org
> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
>
> --
> Iain Morgan
>
> Il giorno ven 2 nov 2018, 20:03 Iain Morgan <imorgan@nas.nasa.gov> ha
> scritto:
>
> If you truly intend this as a bug report, you should file it at
> bugzilla.mindrot.org.
>
> On Fri, Nov 02, 2018 at 12:25:22 +0100, Vincenzo Romano wrote:
>
> Short description: All comands sent to server in batch mode are being
> echoed back along with prompt.
> Software Version: "OpenSSH_7.9p1, OpenSSL 1.1.1 11 Sep 2018" (as
> printed by ssh -V)
> Server and client systems: ArchLinux x86_64 fully updated as of
>
> 2018-11-02.
>
> ## How to reproduce
>
> Provided that you have proper SSH key authentication in place, from
> command line run:
> echo "dir /var" | sftp -b - 0
>
> ## Expected output
>
> /var/backups /var/cache /var/lib /var/local /var/lock
> /var/log /var/mail /var/opt /var/puppet /var/run
> /var/spool /var/tmp /var/www
>
> ## Actual output
>
> sftp> dir /var
> /var/backups /var/cache /var/lib /var/local /var/lock
> /var/log /var/mail /var/opt /var/puppet /var/run
> /var/spool /var/tmp /var/www
>
> Please note the first line is the server prompt followed by the command
>
> echo.
>
> ## Workaround
>
> Expunge the first line from the client output
> echo "dir /var" | sftp -b - 0 | tail -n +2
>
> ## Notes
> Batch mode is meant for script automation, not human operations, so
> echoing the prompt and the commands doesn't make any sense and creates
> problems while processing the output.
> A possible solution that's also backward compatible with all scripts
> using batch mode is to allow for a special prefix character to
> commands to avoid any echoing, just like the "-" is used to avoid a
> single failing command to terminate the client. Something like "@" or
> "#" would be ok IMHO.
>
> --
> Vincenzo Romano - NotOrAnd.IT
> Information Technologies
> --
> NON QVIETIS MARIBVS NAVTA PERITVS
> _______________________________________________
> openssh-unix-dev mailing list
> openssh-unix-dev@mindrot.org
> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
>
> --
> Iain Morgan
>
> _______________________________________________
> openssh-unix-dev mailing list
> openssh-unix-dev@mindrot.org
> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
>
>


--
Vincenzo Romano - NotOrAnd.IT
Information Technologies
--
NON QVIETIS MARIBVS NAVTA PERITVS
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: [BUG?] sftp is echoing back prompt and commands in batch mode [ In reply to ]
Vincenzo Romano wrote:
> [..]
>> As if I'm writing a batch script I want to see the echo of the command and the output so if there is a failure I know where the failure is.
>
> I see: you have a single batch file with no conditionals.
> I have multiple dynamic 1-command batches with conditionals.
> That makes the big difference.

A simple conditional may be useful within sftp. But I don't believe
upstream would accept a full-fledge scripting language as that would be
better served with a python script or a real language.

But a few cases it could be useful like:

cd /path
failed:
mkdir /path
cd /path
:deliaf
put foo

(Shitty language design constructions, but I'm just throwing out an idea
for the wider community and for Darren/Damien to thumb up or down.)


Ben
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: [BUG?] sftp is echoing back prompt and commands in batch mode [ In reply to ]
Il giorno dom 4 nov 2018, 16:01 Ben Lindstrom <mouring@offwriting.org> ha
scritto:

>
>
> Vincenzo Romano wrote:
>
> [..]
>
> As if I'm writing a batch script I want to see the echo of the command and the output so if there is a failure I know where the failure is.
>
> I see: you have a single batch file with no conditionals.
> I have multiple dynamic 1-command batches with conditionals.
> That makes the big difference.
>
>
> A simple conditional may be useful within sftp. But I don't believe
> upstream would accept a full-fledge scripting language as that would be
> better served with a python script or a real language.
>
> But a few cases it could be useful like:
>
> cd /path
> failed:
> mkdir /path
> cd /path
> :deliaf
> put foo
>
> (Shitty language design constructions, but I'm just throwing out an idea
> for the wider community and for Darren/Damien to thumb up or down.)
>
>
> Ben
>

I am not aiming at a scripting language #inside# SFTP. I have the shell for
that!

>
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: [BUG?] sftp is echoing back prompt and commands in batch mode [ In reply to ]
Il giorno dom 4 nov 2018 alle ore 11:04 Vincenzo Romano
<vincenzo.romano@notorand.it> ha scritto:
>
> Il giorno dom 4 nov 2018 alle ore 01:45 Ben Lindstrom
> <mouring@offwriting.org> ha scritto:
> >
> > I don't see it as a bug.
>
> Yes, this is why I put a question mark in the subject.
>
> > As if I'm writing a batch script I want to see the echo of the command and the output so if there is a failure I know where the failure is.
>
> I see: you have a single batch file with no conditionals.
> I have multiple dynamic 1-command batches with conditionals.
> That makes the big difference.
>
> > I could see adding a mode not to echo back the command or prompt.
>
> Me too.
>
> > But it isn't a bug to me.
>
> This makes sense.

For those who are still interested into this topic:

https://bugzilla.mindrot.org/show_bug.cgi?id=2926

--
Vincenzo Romano - NotOrAnd.IT
Information Technologies
--
NON QVIETIS MARIBVS NAVTA PERITVS
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev