Mailing List Archive

sftp client: upload from pipe
Hi,

the sftp client from openssh package can't upload data from local pipe
to remote file. For example, such a command fails:

$ cat file | sftp -b <(echo 'put /dev/stdin /directory/filename') -i ~/.ssh/key user@remote.host
sftp> put /dev/stdin /directory/filename
/dev/stdin is not a regular file

What is a purpose for such a behaviour and limitation? As experiment,
I removed following piece of code which is responsible for this check
(sftp-client.c, do_upload() function):

if (!S_ISREG(sb.st_mode)) {
error("%s is not a regular file", local_path);
close(local_fd);
return(-1);
}

and nothing bad happened. It was still possible to upload regular files,
but additionaly there was an opportunity for upload data piped from other
command (what in my opinion is very useful feature) and even directly
from terminal. Would it be possible to remove this restriction from sftp
client?

Regards.
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: sftp client: upload from pipe [ In reply to ]
On Fri, 21 Jun 2019, Adam Osuchowski wrote:

> Hi,
>
> the sftp client from openssh package can't upload data from local pipe
> to remote file. For example, such a command fails:
>
> $ cat file | sftp -b <(echo 'put /dev/stdin /directory/filename') -i ~/.ssh/key user@remote.host
> sftp> put /dev/stdin /directory/filename
> /dev/stdin is not a regular file
>
> What is a purpose for such a behaviour and limitation? As experiment,
> I removed following piece of code which is responsible for this check
> (sftp-client.c, do_upload() function):
>
> if (!S_ISREG(sb.st_mode)) {
> error("%s is not a regular file", local_path);
> close(local_fd);
> return(-1);
> }
>
> and nothing bad happened. It was still possible to upload regular files,
> but additionaly there was an opportunity for upload data piped from other
> command (what in my opinion is very useful feature) and even directly
> from terminal. Would it be possible to remove this restriction from sftp
> client?

IIRC it's there because reads on named pipes can hang if nothing is
attached to them and because reading from devices can have side-effects
(classic example being /dev/mt)

I'm not opposed to relaxing this restriction, but I think that we'd need to
preserve the behaviour I just mentioned.

-d
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: sftp client: upload from pipe [ In reply to ]
Damien Miller wrote:
> IIRC it's there because reads on named pipes can hang if nothing is
> attached to them

And what is the problem with that? It should break nothing if this named
pipe was passed to sftp explicitly (I'm not talking about recursively
uploading of directory that contains named pipes). Besides, there is
O_NONBLOCK if it would be very necessary.

> and because reading from devices can have side-effects (classic example
> being /dev/mt)

As above, what is the problem with that? Nobody tells anybody to use
sftp with such devices. Sometimes, It could even be desirable if e.g.
somebody would like to backup content of device on remote host via sftp.
Again as above, I'm not talking about recursively uploading of directory
that contains devices; only devices that are explicitly specified in
command. If it would be a big problem, it's ok for me that sftp could
allow uploading regular files and pipes/fifos only.
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: sftp client: upload from pipe [ In reply to ]
On Sat, 22 Jun 2019, Adam Osuchowski wrote:

> Damien Miller wrote:
> > IIRC it's there because reads on named pipes can hang if nothing is
> > attached to them
>
> And what is the problem with that?

That it can hang peoples transfers unexpectedly.

> It should break nothing if this named
> pipe was passed to sftp explicitly (I'm not talking about recursively
> uploading of directory that contains named pipes). Besides, there is
> O_NONBLOCK if it would be very necessary.

O_NONBLOCK is not a solution. There's no way to discern between "this FIFO
has nothing connected" and "this FIFO just needs a moment to start".

> > and because reading from devices can have side-effects (classic example
> > being /dev/mt)
>
> As above, what is the problem with that?

Same as above.

> Nobody tells anybody to use
> sftp with such devices. Sometimes, It could even be desirable if e.g.
> somebody would like to backup content of device on remote host via sftp.
> Again as above, I'm not talking about recursively uploading of directory

It's not just recursion that causes surprises, but wildcards. Say
"put /tmp/foo*" and one of the entries is a FIFO or device.

> that contains devices; only devices that are explicitly specified in
> command. If it would be a big problem, it's ok for me that sftp could
> allow uploading regular files and pipes/fifos only.

IMO the best user experience is to refuse to handle them. I'm happy to
leave sftp with some corner-case limitiations in return for it being a
generally more reliable tool.

-d
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: sftp client: upload from pipe [ In reply to ]
Damien Miller wrote:
> IMO the best user experience is to refuse to handle them. I'm happy to
> leave sftp with some corner-case limitiations in return for it being a
> generally more reliable tool.

And can you agree to and make an exception for pipe attached to stdin?
Because, in fact, it's a case I'm concerned with. It could be handled
with special put command syntax, e.g. with dash:

sftp> put - /directory/file

so that there was no need to cope with all pipes/fifos cases.
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev