Mailing List Archive

Deprecation of scp protocol and improving sftp client
Hello all,

I believe we all can agree that scp is ugly protocol carried for ages
only for its simplicity of its usage and really no dependencies as it
is installed together with every ssh client. But as we have seen
recently, its simplicity and flexibility comes with security issues
[1], it does not have great performance and there is really no
development in there.

Over the years, we still keep recommending people to use sftp instead,
but its api is not that flexible and simple to be usable as a drop-in
replacement in scripts nor for the occasional ad-hoc transfers of few
files from one server to another.

Before I start hacking, I would like to hear some opinions from others,
whether this is something planned, welcomed or whether there are some
good reasons to keep scp alive.

I have in my mind three things/steps that would make it possible:

* Update sftp client to be drop-in replacement for scp
(and/or)
* Change scp to use sftp internally

* Modify sshd to use some compatibility "scpd" to support old clients

and some time later

* Remove scp or replace it with a symlink


[1] http://www.openssh.com/txt/release-8.0

Any ideas/comments/suggestions?


Best regards,
--
Jakub Jelen
Senior Software Engineer
Security Technologies
Red Hat, Inc.

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Tue, 2020-06-16 at 18:41 +0200, Jakub Jelen wrote:
> * Change scp to use sftp internally
Sounds like the best to me... given that scp is used in countless of
scripts, etc..


> Any ideas/comments/suggestions?
Probably unrelated, but what I'd be waiting for years is: XATTRs/ACLs
support in SFTP (thus one could eventually get support for them in
things like sshfs)


Cheer,
Chris.

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On 2020/06/16 18:41, Jakub Jelen wrote:
> Hello all,
>
> I believe we all can agree that scp is ugly protocol carried for ages
> only for its simplicity of its usage and really no dependencies as it
> is installed together with every ssh client. But as we have seen
> recently, its simplicity and flexibility comes with security issues
> [1], it does not have great performance and there is really no
> development in there.
>
> Over the years, we still keep recommending people to use sftp instead,
> but its api is not that flexible and simple to be usable as a drop-in
> replacement in scripts nor for the occasional ad-hoc transfers of few
> files from one server to another.

I've tried to switch to sftp several times, the thing that always
stops me is not being able to copy from local->remote directly
on the command line.

> Before I start hacking, I would like to hear some opinions from others,
> whether this is something planned, welcomed or whether there are some
> good reasons to keep scp alive.
>
> I have in my mind three things/steps that would make it possible:
>
> * Update sftp client to be drop-in replacement for scp
> (and/or)

This would seem a good starting point. It would allow using
"alias scp=sftp" for interactive shells (muscle memory / easier
typing) and makes it possible to convert scripts across one by one
from scp to sftp without unexpectedly breaking anything.

> * Change scp to use sftp internally

Then you either have no fallback for scp with very old servers, or a
mess of either/or code to cope with both protocols. And it would seem
more complex to change the protocol code than the UI code.

>
> * Modify sshd to use some compatibility "scpd" to support old clients
>
> and some time later
>
> * Remove scp or replace it with a symlink
>
>
> [1] http://www.openssh.com/txt/release-8.0
>
> Any ideas/comments/suggestions?
>
>
> Best regards,
> --
> Jakub Jelen
> Senior Software Engineer
> Security Technologies
> Red Hat, Inc.
>
> _______________________________________________
> openssh-unix-dev mailing list
> openssh-unix-dev@mindrot.org
> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
scp may be an ugly protocol, but it works, works nicely from a command line, and is quite convenient. FTP (and, presumably, sftp) is not nearly as convenient.

Why do you think your recommendation to "use sftp instead" keeps falling on the deaf ear? Usability, perhaps?

Perhaps it's time to stop preaching to people about what they should use, but instead - if you really want a change - design, implement, and release a new tool that would combine the usability of scp with the security you want to see?
SFTP did not seem to make that cut. Next option?



?On 6/16/20, 13:14, "openssh-unix-dev on behalf of Stuart Henderson" <openssh-unix-dev-bounces+uri=ll.mit.edu@mindrot.org on behalf of stu@spacehopper.org> wrote:

On 2020/06/16 18:41, Jakub Jelen wrote:
> Hello all,
>
> I believe we all can agree that scp is ugly protocol carried for ages
> only for its simplicity of its usage and really no dependencies as it
> is installed together with every ssh client. But as we have seen
> recently, its simplicity and flexibility comes with security issues
> [1], it does not have great performance and there is really no
> development in there.
>
> Over the years, we still keep recommending people to use sftp instead,
> but its api is not that flexible and simple to be usable as a drop-in
> replacement in scripts nor for the occasional ad-hoc transfers of few
> files from one server to another.

I've tried to switch to sftp several times, the thing that always
stops me is not being able to copy from local->remote directly
on the command line.

> Before I start hacking, I would like to hear some opinions from others,
> whether this is something planned, welcomed or whether there are some
> good reasons to keep scp alive.
>
> I have in my mind three things/steps that would make it possible:
>
> * Update sftp client to be drop-in replacement for scp
> (and/or)

This would seem a good starting point. It would allow using
"alias scp=sftp" for interactive shells (muscle memory / easier
typing) and makes it possible to convert scripts across one by one
from scp to sftp without unexpectedly breaking anything.

> * Change scp to use sftp internally

Then you either have no fallback for scp with very old servers, or a
mess of either/or code to cope with both protocols. And it would seem
more complex to change the protocol code than the UI code.

>
> * Modify sshd to use some compatibility "scpd" to support old clients
>
> and some time later
>
> * Remove scp or replace it with a symlink
>
>
> [1] http://www.openssh.com/txt/release-8.0
>
> Any ideas/comments/suggestions?
>
>
> Best regards,
> --
> Jakub Jelen
> Senior Software Engineer
> Security Technologies
> Red Hat, Inc.
>
> _______________________________________________
> openssh-unix-dev mailing list
> openssh-unix-dev@mindrot.org
> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
Jakub Jelen wrote:
> Any ideas/comments/suggestions?

I doubt that upstream would drop the scp utility and protocol easily.
There is no point in breaking compatibility, so the scp utility would
stay at the very least for the benefit of remote scp clients.

That said, I think it would be worthwhile to implement or at the very
least research the possibility of an SFTP client with scp semantics.

You'd have to look closely at the protocols and scp man page, and it
could well turn out that the SFTP protocol doesn't currently offer
everything needed to provide full scp compatibility.

I'd be interested to hear what you find, if you go ahead.


//Peter
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
Agreed with what Uri said on convenience. Being able to use "scp" with the
same syntax as "cp" is wired into a lot of peoples hands. sftp doesn't
allow for the same easy syntax. Whatever is done to replace it should allow
for that same easy syntax if you want to see adoption with a minimum of
kicking & screaming.

On Tue, Jun 16, 2020 at 10:38 AM Peter Stuge <peter@stuge.se> wrote:

> Jakub Jelen wrote:
> > Any ideas/comments/suggestions?
>
> I doubt that upstream would drop the scp utility and protocol easily.
> There is no point in breaking compatibility, so the scp utility would
> stay at the very least for the benefit of remote scp clients.
>
> That said, I think it would be worthwhile to implement or at the very
> least research the possibility of an SFTP client with scp semantics.
>
> You'd have to look closely at the protocols and scp man page, and it
> could well turn out that the SFTP protocol doesn't currently offer
> everything needed to provide full scp compatibility.
>
> I'd be interested to hear what you find, if you go ahead.
>
>
> //Peter
> _______________________________________________
> openssh-unix-dev mailing list
> openssh-unix-dev@mindrot.org
> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
>
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
Dear Uri,

On Tue, Jun 16, 2020 at 05:20:35PM +0000, Blumenthal, Uri - 0553 - MITLL wrote:
> scp may be an ugly protocol, but it works, works nicely from a command
> line, and is quite convenient. FTP (and, presumably, sftp) is not
> nearly as convenient.
>
> Why do you think your recommendation to "use sftp instead" keeps
> falling on the deaf ear? Usability, perhaps?

Yes - usability!

> Perhaps it's time to stop preaching to people about what they should
> use, but instead - if you really want a change - design, implement,
> and release a new tool that would combine the usability of scp with
> the security you want to see? SFTP did not seem to make that cut.
> Next option?

I think Jakub Jelen is proposing exactly that: somehow keep the scp UI
we all know and love, and figure out a path to migrate the user base to
the sftp protocol under the hood.

You ask for a next option, and Jakub's email is exactly that! He wants
to build the next option. I'm excited to see where this will bring us.

Kind regards,

Job
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On 2020/06/16 17:37, Peter Stuge wrote:
> Jakub Jelen wrote:
> > Any ideas/comments/suggestions?
>
> I doubt that upstream would drop the scp utility and protocol easily.
> There is no point in breaking compatibility

OpenSSH does break compatibility where there's good reason to.
SSH 1, <1024 bit keys.

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
I don’t think the majority of users even know what’s under the hood of scp, let alone care.

So, whatever change one can make that is transparent to the users *and* doesn’t negatively impact performance, is bound to be a winner.

Somehow, I doubt sftp can provide that, but I’d be happy to be proven wrong here.

Regards,
Uri

> On Jun 16, 2020, at 14:02, Stuart Henderson <stu@spacehopper.org> wrote:
>
> ?On 2020/06/16 17:37, Peter Stuge wrote:
>> Jakub Jelen wrote:
>>> Any ideas/comments/suggestions?
>>
>> I doubt that upstream would drop the scp utility and protocol easily.
>> There is no point in breaking compatibility
>
> OpenSSH does break compatibility where there's good reason to.
> SSH 1, <1024 bit keys.
>
> _______________________________________________
> openssh-unix-dev mailing list
> openssh-unix-dev@mindrot.org
> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
Hi

On Tue, Jun 16, 2020 at 05:51:07PM +0000, Job Snijders wrote:
> You ask for a next option, and Jakub's email is exactly that! He wants
> to build the next option. I'm excited to see where this will bring us.

+1

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Tue, Jun 16, 2020 at 06:41:09PM +0200, Jakub Jelen wrote:
> * Change scp to use sftp internally

As an existence proof, pscp (from PuTTY) does exactly this; it tries the
sftp protocol and falls back to the scp protocol if that fails, and it
has -scp or -sftp options to force one or the other. I've long meant to
get round to putting something similar for OpenSSH, but never got far
enough to really be worth mentioning. (Of course it would still need to
retain scp "source" and "sink" modes if invoked with -f or -t, to retain
compatibility, since those are used on the server when an scp client
connects; but there's no particular obstacle to that.)

> * Modify sshd to use some compatibility "scpd" to support old clients

This should be unnecessary. When an scp client connects to an scp
server, it passes the -f (source) or -t (sink) flag as appropriate;
neither is part of the documented user-facing interface to scp. I don't
see any particular reason why scp (the program) couldn't continue to
speak the scp protocol when invoked with -f/-t, but speak the sftp
protocol when invoked in the normal way.

--
Colin Watson (he/him) [cjwatson@debian.org]
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Tue, 2020-06-16 at 19:47 +0100, Colin Watson wrote:
> sftp protocol and falls back to the scp protocol if that fails, and
> it
> has -scp or -sftp options to force one or the other.

I think fallback should be something that doesn't happen automatically,
at least at some point.

I mean the whole reason of getting rid of scp is mostly the security
issues, isn't it?


I wonder if an "new" scp could bring in some new features like --
interactive (not overwrite files without askin)... or if that would
break too much

Cheers,
Chris.


_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Tuesday, 16 June 2020 20:47:24 CEST Colin Watson wrote:
> On Tue, Jun 16, 2020 at 06:41:09PM +0200, Jakub Jelen wrote:
> > * Change scp to use sftp internally
>
> As an existence proof, pscp (from PuTTY) does exactly this; it tries the
> sftp protocol and falls back to the scp protocol if that fails, and it
> has -scp or -sftp options to force one or the other. I've long meant to
> get round to putting something similar for OpenSSH, but never got far
> enough to really be worth mentioning. (Of course it would still need to
> retain scp "source" and "sink" modes if invoked with -f or -t, to retain
> compatibility, since those are used on the server when an scp client
> connects; but there's no particular obstacle to that.)
>
> > * Modify sshd to use some compatibility "scpd" to support old clients
>
> This should be unnecessary. When an scp client connects to an scp
> server, it passes the -f (source) or -t (sink) flag as appropriate;
> neither is part of the documented user-facing interface to scp. I don't
> see any particular reason why scp (the program) couldn't continue to
> speak the scp protocol when invoked with -f/-t, but speak the sftp
> protocol when invoked in the normal way.

It is necessary. If you replace the protocol of the scp command with sftp then
there are still scp commands out there speaking the scp protocol. You need to
keep the server side supported.


Andreas

--
Andreas Schneider asn@cryptomilk.org
GPG-ID: 8DFF53E18F2ABC8D8F3C92237EE0FC4DCC014E3D


_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Tuesday, 16 June 2020 18:41:09 CEST Jakub Jelen wrote:
> Hello all,

Hi Jakub,

> I have in my mind three things/steps that would make it possible:
>
> * Update sftp client to be drop-in replacement for scp
> (and/or)
> * Change scp to use sftp internally

I would suggest to go this route and let the `scp` tool use the sftp protocol
by default. Keep the `sftp` command as is.

> * Modify sshd to use some compatibility "scpd" to support old clients

Yes, and add an sshd option to disable it. This way we can flip the option to
to disabled by default at one point.


Andreas


--
Andreas Schneider asn@cryptomilk.org
GPG-ID: 8DFF53E18F2ABC8D8F3C92237EE0FC4DCC014E3D


_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Tuesday, 16 June 2020 20:47:24 CEST Colin Watson wrote:
> On Tue, Jun 16, 2020 at 06:41:09PM +0200, Jakub Jelen wrote:
> > * Change scp to use sftp internally
>
> As an existence proof, pscp (from PuTTY) does exactly this; it tries the
> sftp protocol and falls back to the scp protocol if that fails, and it
> has -scp or -sftp options to force one or the other. I've long meant to
> get round to putting something similar for OpenSSH, but never got far
> enough to really be worth mentioning. (Of course it would still need to
> retain scp "source" and "sink" modes if invoked with -f or -t, to retain
> compatibility, since those are used on the server when an scp client
> connects; but there's no particular obstacle to that.)

You can easily detect the sink mode on connect and redirect to a scpd server
implementation. There is no need to make it more complex than it is.

We have sftp server implementation around for long enough that the `scp`
command can use the protocol.

Also if you have a scpd. You can reject the scp protocol completely by a
config option.

> > * Modify sshd to use some compatibility "scpd" to support old clients
>
> This should be unnecessary. When an scp client connects to an scp
> server, it passes the -f (source) or -t (sink) flag as appropriate;
> neither is part of the documented user-facing interface to scp. I don't
> see any particular reason why scp (the program) couldn't continue to
> speak the scp protocol when invoked with -f/-t, but speak the sftp
> protocol when invoked in the normal way.

The scp command should only handle the client side, for the server you should
have a server only implementation which could be disabled. Some people are not
interested in the scp protocol if sftp can do the job. One security hole less
:-)


Andreas

--
Andreas Schneider asn@cryptomilk.org
GPG-ID: 8DFF53E18F2ABC8D8F3C92237EE0FC4DCC014E3D


_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Mon, Jun 22, 2020 at 10:47:35AM +0200, Andreas Schneider wrote:
> On Tuesday, 16 June 2020 20:47:24 CEST Colin Watson wrote:
> > On Tue, Jun 16, 2020 at 06:41:09PM +0200, Jakub Jelen wrote:
> > > * Modify sshd to use some compatibility "scpd" to support old clients
> >
> > This should be unnecessary. When an scp client connects to an scp
> > server, it passes the -f (source) or -t (sink) flag as appropriate;
> > neither is part of the documented user-facing interface to scp. I don't
> > see any particular reason why scp (the program) couldn't continue to
> > speak the scp protocol when invoked with -f/-t, but speak the sftp
> > protocol when invoked in the normal way.
>
> It is necessary. If you replace the protocol of the scp command with sftp then
> there are still scp commands out there speaking the scp protocol. You need to
> keep the server side supported.

I think you must have misread what I wrote. Let me try to clarify: the
way to keep the server side supported is simply to continue handling the
-f and -t flags appropriately in scp, which can be independent of
whether scp speaks sftp when invoked as an ordinary client. It isn't
necessary to modify sshd to use a different "scpd" executable.

--
Colin Watson (he/him) [cjwatson@debian.org]
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
I had something in mind like this for years, but with slightly different steps:
My naive approach would be to keep the scp user interface and switch
to the sftp protocol internally. We could add a -M [scp|sftp] option
to scp and select the internal protocol. Later we switch the default
from scp to sftp.
No need to change sshd or write scpd.

-m

Am Di., 16. Juni 2020 um 18:48 Uhr schrieb Jakub Jelen <jjelen@redhat.com>:
>
> Hello all,
>
> I believe we all can agree that scp is ugly protocol carried for ages
> only for its simplicity of its usage and really no dependencies as it
> is installed together with every ssh client. But as we have seen
> recently, its simplicity and flexibility comes with security issues
> [1], it does not have great performance and there is really no
> development in there.
>
> Over the years, we still keep recommending people to use sftp instead,
> but its api is not that flexible and simple to be usable as a drop-in
> replacement in scripts nor for the occasional ad-hoc transfers of few
> files from one server to another.
>
> Before I start hacking, I would like to hear some opinions from others,
> whether this is something planned, welcomed or whether there are some
> good reasons to keep scp alive.
>
> I have in my mind three things/steps that would make it possible:
>
> * Update sftp client to be drop-in replacement for scp
> (and/or)
> * Change scp to use sftp internally
>
> * Modify sshd to use some compatibility "scpd" to support old clients
>
> and some time later
>
> * Remove scp or replace it with a symlink
>
>
> [1] http://www.openssh.com/txt/release-8.0
>
> Any ideas/comments/suggestions?
>
>
> Best regards,
> --
> Jakub Jelen
> Senior Software Engineer
> Security Technologies
> Red Hat, Inc.
>
> _______________________________________________
> openssh-unix-dev mailing list
> openssh-unix-dev@mindrot.org
> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Tue, 2020-06-23 at 08:06 +0200, Markus Friedl wrote:
> I had something in mind like this for years, but with slightly
> different steps:
> My naive approach would be to keep the scp user interface and switch
> to the sftp protocol internally. We could add a -M [scp|sftp] option
> to scp and select the internal protocol. Later we switch the default
> from scp to sftp.
> No need to change sshd or write scpd.

Thank you all for comments and initial feedback. It looks like this is
something that many people already had in minds, but for some reasons,
it never happened.

I tried to put together something that now works and passes the scp
testsuite (with both scp and sftp modes):

https://github.com/openssh/openssh-portable/pull/194

What does not work is the extended remote-to-remote through local,
which would require some more low-level protocol tweaks.

Most of the code is taken and adapted from the sftp.c . There are still
a few TODOs, but lets take it as a first iteration/proof of concept.

Considering the automatic fallback to scp, the way how it works now
(executing another ssh in subprocess) it would require going through
the whole key exchange and authentication again, spawning a new ssh
process, which sounds very cumbersome.

Much nicer solution would be using ssh_api to handle commands/subsystem
requests inside a single ssh session, but it would be much more code
and changes that I wanted to avoid in the first iteration (if the
ssh_api is already usable for something like this -- I would have to
check).

So again, comments, suggestions and feedback welcomed. I am not sure if
there is some other mailing list to get more attention from other
OpenBSD developers or this one is fine.

Thanks,
Jakub

> -m
>
> Am Di., 16. Juni 2020 um 18:48 Uhr schrieb Jakub Jelen <
> jjelen@redhat.com>:
> > Hello all,
> >
> > I believe we all can agree that scp is ugly protocol carried for
> > ages
> > only for its simplicity of its usage and really no dependencies as
> > it
> > is installed together with every ssh client. But as we have seen
> > recently, its simplicity and flexibility comes with security issues
> > [1], it does not have great performance and there is really no
> > development in there.
> >
> > Over the years, we still keep recommending people to use sftp
> > instead,
> > but its api is not that flexible and simple to be usable as a drop-
> > in
> > replacement in scripts nor for the occasional ad-hoc transfers of
> > few
> > files from one server to another.
> >
> > Before I start hacking, I would like to hear some opinions from
> > others,
> > whether this is something planned, welcomed or whether there are
> > some
> > good reasons to keep scp alive.
> >
> > I have in my mind three things/steps that would make it possible:
> >
> > * Update sftp client to be drop-in replacement for scp
> > (and/or)
> > * Change scp to use sftp internally
> >
> > * Modify sshd to use some compatibility "scpd" to support old
> > clients
> >
> > and some time later
> >
> > * Remove scp or replace it with a symlink
> >
> >
> > [1] http://www.openssh.com/txt/release-8.0
> >
> > Any ideas/comments/suggestions?
> >
> >
> > Best regards,
> > --
> > Jakub Jelen
> > Senior Software Engineer
> > Security Technologies
> > Red Hat, Inc.
> >
> > _______________________________________________
> > openssh-unix-dev mailing list
> > openssh-unix-dev@mindrot.org
> > https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
> _______________________________________________
> openssh-unix-dev mailing list
> openssh-unix-dev@mindrot.org
> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
>
--
Jakub Jelen
Senior Software Engineer
Security Technologies
Red Hat, Inc.

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
I have had this in my .bashrc for years:

alias scp='rsync -avzP'

maybe rsync is a better replacement for scp than sftp would be?


On Wed, Jul 15, 2020 at 5:07 AM Jakub Jelen <jjelen@redhat.com> wrote:

> On Tue, 2020-06-23 at 08:06 +0200, Markus Friedl wrote:
> > I had something in mind like this for years, but with slightly
> > different steps:
> > My naive approach would be to keep the scp user interface and switch
> > to the sftp protocol internally. We could add a -M [scp|sftp] option
> > to scp and select the internal protocol. Later we switch the default
> > from scp to sftp.
> > No need to change sshd or write scpd.
>
> Thank you all for comments and initial feedback. It looks like this is
> something that many people already had in minds, but for some reasons,
> it never happened.
>
> I tried to put together something that now works and passes the scp
> testsuite (with both scp and sftp modes):
>
> https://github.com/openssh/openssh-portable/pull/194
>
> What does not work is the extended remote-to-remote through local,
> which would require some more low-level protocol tweaks.
>
> Most of the code is taken and adapted from the sftp.c . There are still
> a few TODOs, but lets take it as a first iteration/proof of concept.
>
> Considering the automatic fallback to scp, the way how it works now
> (executing another ssh in subprocess) it would require going through
> the whole key exchange and authentication again, spawning a new ssh
> process, which sounds very cumbersome.
>
> Much nicer solution would be using ssh_api to handle commands/subsystem
> requests inside a single ssh session, but it would be much more code
> and changes that I wanted to avoid in the first iteration (if the
> ssh_api is already usable for something like this -- I would have to
> check).
>
> So again, comments, suggestions and feedback welcomed. I am not sure if
> there is some other mailing list to get more attention from other
> OpenBSD developers or this one is fine.
>
> Thanks,
> Jakub
>
> > -m
> >
> > Am Di., 16. Juni 2020 um 18:48 Uhr schrieb Jakub Jelen <
> > jjelen@redhat.com>:
> > > Hello all,
> > >
> > > I believe we all can agree that scp is ugly protocol carried for
> > > ages
> > > only for its simplicity of its usage and really no dependencies as
> > > it
> > > is installed together with every ssh client. But as we have seen
> > > recently, its simplicity and flexibility comes with security issues
> > > [1], it does not have great performance and there is really no
> > > development in there.
> > >
> > > Over the years, we still keep recommending people to use sftp
> > > instead,
> > > but its api is not that flexible and simple to be usable as a drop-
> > > in
> > > replacement in scripts nor for the occasional ad-hoc transfers of
> > > few
> > > files from one server to another.
> > >
> > > Before I start hacking, I would like to hear some opinions from
> > > others,
> > > whether this is something planned, welcomed or whether there are
> > > some
> > > good reasons to keep scp alive.
> > >
> > > I have in my mind three things/steps that would make it possible:
> > >
> > > * Update sftp client to be drop-in replacement for scp
> > > (and/or)
> > > * Change scp to use sftp internally
> > >
> > > * Modify sshd to use some compatibility "scpd" to support old
> > > clients
> > >
> > > and some time later
> > >
> > > * Remove scp or replace it with a symlink
> > >
> > >
> > > [1] http://www.openssh.com/txt/release-8.0
> > >
> > > Any ideas/comments/suggestions?
> > >
> > >
> > > Best regards,
> > > --
> > > Jakub Jelen
> > > Senior Software Engineer
> > > Security Technologies
> > > Red Hat, Inc.
> > >
> > > _______________________________________________
> > > openssh-unix-dev mailing list
> > > openssh-unix-dev@mindrot.org
> > > https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
> > _______________________________________________
> > openssh-unix-dev mailing list
> > openssh-unix-dev@mindrot.org
> > https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
> >
> --
> Jakub Jelen
> Senior Software Engineer
> Security Technologies
> Red Hat, Inc.
>
> _______________________________________________
> openssh-unix-dev mailing list
> openssh-unix-dev@mindrot.org
> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
>
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Wed, 15 Jul 2020, Red Cricket wrote:

> I have had this in my .bashrc for years:
>
> alias scp='rsync -avzP'

Similar, though I named it rcp because nobody has the real rcp installed
any more, but sometimes I need scp to connect to systems that lack rsync.

https://evolvis.org/plugins/scmgit/cgi-bin/gitweb.cgi?p=shellsnippets/shellsnippets.git;a=blob;f=mksh/rcp;hb=HEAD

> maybe rsync is a better replacement for scp than sftp would be?

It could be, were it not under a restrictive licence…


This doesn’t preclude people from making SSH’s builtin transfers
better, though.

bye,
//mirabilos
--
«MyISAM tables -will- get corrupted eventually. This is a fact of life. »
“mysql is about as much database as ms access” – “MSSQL at least descends
from a database” “it's a rebranded SyBase” “MySQL however was born from a
flatfile and went downhill from there” – “at least jetDB doesn’t claim to
be a database” (#nosec) ??? Please let MySQL and MariaDB finally die!
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
I wanted to bring this up again due to:
https://github.com/cpandya2909/CVE-2020-15778/. This showcases a clear
issue with scp which it sounds like cannot be fixed without breaking scp.
This seems like it would lend some impetus to doing _something_, even if it
breaks scp or necessitates using something new.

Cheers,

Ethan

On Wed, Jul 15, 2020 at 7:47 AM Thorsten Glaser <t.glaser@tarent.de> wrote:

> On Wed, 15 Jul 2020, Red Cricket wrote:
>
> > I have had this in my .bashrc for years:
> >
> > alias scp='rsync -avzP'
>
> Similar, though I named it rcp because nobody has the real rcp installed
> any more, but sometimes I need scp to connect to systems that lack rsync.
>
>
> https://evolvis.org/plugins/scmgit/cgi-bin/gitweb.cgi?p=shellsnippets/shellsnippets.git;a=blob;f=mksh/rcp;hb=HEAD
>
> > maybe rsync is a better replacement for scp than sftp would be?
>
> It could be, were it not under a restrictive licence…
>
>
> This doesn’t preclude people from making SSH’s builtin transfers
> better, though.
>
> bye,
> //mirabilos
> --
> «MyISAM tables -will- get corrupted eventually. This is a fact of life. »
> “mysql is about as much database as ms access” – “MSSQL at least descends
> from a database” “it's a rebranded SyBase” “MySQL however was born from a
> flatfile and went downhill from there” – “at least jetDB doesn’t claim to
> be a database” (#nosec) ??? Please let MySQL and MariaDB finally die!
> _______________________________________________
> openssh-unix-dev mailing list
> openssh-unix-dev@mindrot.org
> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
>
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
Why can the local and remote paths be sanitized?

Regards,
Uri

> On Jul 31, 2020, at 19:57, Ethan Rahn <ethan.rahn@gmail.com> wrote:
>
> ?I wanted to bring this up again due to:
> https://github.com/cpandya2909/CVE-2020-15778/. This showcases a clear
> issue with scp which it sounds like cannot be fixed without breaking scp.
> This seems like it would lend some impetus to doing _something_, even if it
> breaks scp or necessitates using something new.
>
> Cheers,
>
> Ethan
>
>> On Wed, Jul 15, 2020 at 7:47 AM Thorsten Glaser <t.glaser@tarent.de> wrote:
>>
>>> On Wed, 15 Jul 2020, Red Cricket wrote:
>>>
>>> I have had this in my .bashrc for years:
>>>
>>> alias scp='rsync -avzP'
>>
>> Similar, though I named it rcp because nobody has the real rcp installed
>> any more, but sometimes I need scp to connect to systems that lack rsync.
>>
>>
>> https://evolvis.org/plugins/scmgit/cgi-bin/gitweb.cgi?p=shellsnippets/shellsnippets.git;a=blob;f=mksh/rcp;hb=HEAD
>>
>>> maybe rsync is a better replacement for scp than sftp would be?
>>
>> It could be, were it not under a restrictive licence…
>>
>>
>> This doesn’t preclude people from making SSH’s builtin transfers
>> better, though.
>>
>> bye,
>> //mirabilos
>> --
>> «MyISAM tables -will- get corrupted eventually. This is a fact of life. »
>> “mysql is about as much database as ms access” – “MSSQL at least descends
>> from a database” “it's a rebranded SyBase” “MySQL however was born from a
>> flatfile and went downhill from there” – “at least jetDB doesn’t claim to
>> be a database” (#nosec) ??? Please let MySQL and MariaDB finally die!
>> _______________________________________________
>> openssh-unix-dev mailing list
>> openssh-unix-dev@mindrot.org
>> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
>>
> _______________________________________________
> openssh-unix-dev mailing list
> openssh-unix-dev@mindrot.org
> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Fri, Jul 31, 2020 at 04:29:13PM -0700, Ethan Rahn <ethan.rahn@gmail.com> wrote:

> I wanted to bring this up again due to:
> https://github.com/cpandya2909/CVE-2020-15778/. This showcases a clear
> issue with scp which it sounds like cannot be fixed without breaking scp.
> This seems like it would lend some impetus to doing _something_, even if it
> breaks scp or necessitates using something new.
>
> Cheers,
> Ethan

Surely, executing the scp -t command without using the
shell would fix this without breaking any legitimate
usage. And it would be much easier and more effective
than sanitising the path. Paths can contain almost any
byte.

Mind you, it wouldn't stop the legitimate user from
just logging in and performing the same actions manually.
But it would help in cases where users can scp but not ssh
to a host.

cheers,
raf

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Mon, Aug 03, 2020 at 09:48:30AM +1000, raf <ssh@raf.org> wrote:

> On Fri, Jul 31, 2020 at 04:29:13PM -0700, Ethan Rahn <ethan.rahn@gmail.com> wrote:
>
> > I wanted to bring this up again due to:
> > https://github.com/cpandya2909/CVE-2020-15778/. This showcases a clear
> > issue with scp which it sounds like cannot be fixed without breaking scp.
> > This seems like it would lend some impetus to doing _something_, even if it
> > breaks scp or necessitates using something new.
> >
> > Cheers,
> > Ethan
>
> Surely, executing the scp -t command without using the
> shell would fix this without breaking any legitimate
> usage. And it would be much easier and more effective
> than sanitising the path. Paths can contain almost any
> byte.
>
> Mind you, it wouldn't stop the legitimate user from
> just logging in and performing the same actions manually.
> But it would help in cases where users can scp but not ssh
> to a host.

And another problem is that probably, the ssh server just sees
this as a command to run, like any other, and so of course it
executes it via the user's shell. Unless it makes some distinction
between scp connections and everything else. It probably doesn't.

cheers,
raf

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Sat, 2020-08-01 at 00:17 +0000, Blumenthal, Uri - 0553 - MITLL
wrote:
> Why can the local and remote paths be sanitized?

Because remote path is *expected* to be expanded by remote shell before
executing remote scp. If you sanitize it in any way, you will break
existing use cases.

> Regards,
> Uri
>
> > On Jul 31, 2020, at 19:57, Ethan Rahn <ethan.rahn@gmail.com> wrote:
> >
> > ?I wanted to bring this up again due to:
> > https://github.com/cpandya2909/CVE-2020-15778/. This showcases a
> > clear
> > issue with scp which it sounds like cannot be fixed without
> > breaking scp.
> > This seems like it would lend some impetus to doing _something_,
> > even if it
> > breaks scp or necessitates using something new.
> >
> > Cheers,
> >
> > Ethan
> >
> > > On Wed, Jul 15, 2020 at 7:47 AM Thorsten Glaser <
> > > t.glaser@tarent.de> wrote:
> > >
> > > > On Wed, 15 Jul 2020, Red Cricket wrote:
> > > >
> > > > I have had this in my .bashrc for years:
> > > >
> > > > alias scp='rsync -avzP'
> > >
> > > Similar, though I named it rcp because nobody has the real rcp
> > > installed
> > > any more, but sometimes I need scp to connect to systems that
> > > lack rsync.
> > >
> > >
> > > https://evolvis.org/plugins/scmgit/cgi-bin/gitweb.cgi?p=shellsnippets/shellsnippets.git;a=blob;f=mksh/rcp;hb=HEAD
> > >
> > > > maybe rsync is a better replacement for scp than sftp would be?
> > >
> > > It could be, were it not under a restrictive licence…
> > >
> > >
> > > This doesn’t preclude people from making SSH’s builtin transfers
> > > better, though.
> > >
> > > bye,
> > > //mirabilos
> > > --
> > > «MyISAM tables -will- get corrupted eventually. This is a fact of
> > > life. »
> > > “mysql is about as much database as ms access” – “MSSQL at least
> > > descends
> > > from a database” “it's a rebranded SyBase” “MySQL however was
> > > born from a
> > > flatfile and went downhill from there” – “at least jetDB doesn’t
> > > claim to
> > > be a database” (#nosec) ??? Please let MySQL and MariaDB
> > > finally die!
> > > _______________________________________________
> > > openssh-unix-dev mailing list
> > > openssh-unix-dev@mindrot.org
> > > https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
> > >
> > _______________________________________________
> > openssh-unix-dev mailing list
> > openssh-unix-dev@mindrot.org
> > https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
--
Jakub Jelen
Senior Software Engineer
Security Technologies
Red Hat, Inc.

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
I conjecture that only few of the existing use cases rely on remote expansion.

In any case (no pun intended), IMHO it would be better to break a few of the current use cases but leave the majority functional - than kill scp for all.

Regards,
Uri

> On Aug 3, 2020, at 02:50, Jakub Jelen <jjelen@redhat.com> wrote:
>
> ?On Sat, 2020-08-01 at 00:17 +0000, Blumenthal, Uri - 0553 - MITLL
> wrote:
>> Why can the local and remote paths be sanitized?
>
> Because remote path is *expected* to be expanded by remote shell before
> executing remote scp. If you sanitize it in any way, you will break
> existing use cases.
>
>> Regards,
>> Uri
>>
>>>> On Jul 31, 2020, at 19:57, Ethan Rahn <ethan.rahn@gmail.com> wrote:
>>>
>>> ?I wanted to bring this up again due to:
>>> https://github.com/cpandya2909/CVE-2020-15778/. This showcases a
>>> clear
>>> issue with scp which it sounds like cannot be fixed without
>>> breaking scp.
>>> This seems like it would lend some impetus to doing _something_,
>>> even if it
>>> breaks scp or necessitates using something new.
>>>
>>> Cheers,
>>>
>>> Ethan
>>>
>>>> On Wed, Jul 15, 2020 at 7:47 AM Thorsten Glaser <
>>>> t.glaser@tarent.de> wrote:
>>>>
>>>>> On Wed, 15 Jul 2020, Red Cricket wrote:
>>>>>
>>>>> I have had this in my .bashrc for years:
>>>>>
>>>>> alias scp='rsync -avzP'
>>>>
>>>> Similar, though I named it rcp because nobody has the real rcp
>>>> installed
>>>> any more, but sometimes I need scp to connect to systems that
>>>> lack rsync.
>>>>
>>>>
>>>> https://evolvis.org/plugins/scmgit/cgi-bin/gitweb.cgi?p=shellsnippets/shellsnippets.git;a=blob;f=mksh/rcp;hb=HEAD
>>>>
>>>>> maybe rsync is a better replacement for scp than sftp would be?
>>>>
>>>> It could be, were it not under a restrictive licence…
>>>>
>>>>
>>>> This doesn’t preclude people from making SSH’s builtin transfers
>>>> better, though.
>>>>
>>>> bye,
>>>> //mirabilos
>>>> --
>>>> «MyISAM tables -will- get corrupted eventually. This is a fact of
>>>> life. »
>>>> “mysql is about as much database as ms access” – “MSSQL at least
>>>> descends
>>>> from a database” “it's a rebranded SyBase” “MySQL however was
>>>> born from a
>>>> flatfile and went downhill from there” – “at least jetDB doesn’t
>>>> claim to
>>>> be a database” (#nosec) ??? Please let MySQL and MariaDB
>>>> finally die!
>>>> _______________________________________________
>>>> openssh-unix-dev mailing list
>>>> openssh-unix-dev@mindrot.org
>>>> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
>>>>
>>> _______________________________________________
>>> openssh-unix-dev mailing list
>>> openssh-unix-dev@mindrot.org
>>> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
> --
> Jakub Jelen
> Senior Software Engineer
> Security Technologies
> Red Hat, Inc.
>
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
Blumenthal, Uri - 0553 - MITLL wrote:
> I conjecture that only few of the existing use cases rely on remote expansion.

For local-to-remote operation your conjecture may be correct,
for remote-to-local not so much.

It's a fairly key feature that scp globs on the remote, without roundtrip,
and the SFTP protocol doesn't offer a method to do so.


//Peter
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Mon, Aug 3, 2020 at 9:51 AM Blumenthal, Uri - 0553 - MITLL
<uri@ll.mit.edu> wrote:
>
> I conjecture that only few of the existing use cases rely on remote expansion.
>
> In any case (no pun intended), IMHO it would be better to break a few of the current use cases but leave the majority functional - than kill scp for all.
>
> Regards,
> Uri

Especially with "WinSCP" users still out there.
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Mon, 3 Aug 2020, Blumenthal, Uri - 0553 - MITLL wrote:

> I conjecture that only few of the existing use cases rely on remote expansion.

No, this is used all the time.

scp remotehost:foo\* .

(Unless rsync is available, but sadly that’s ? GPLv3 and ? not
universally installed.)

bye,
//mirabilos
--
tarent solutions GmbH
Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/
Tel: +49 228 54881-393 • Fax: +49 228 54881-235
HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941
Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
I hear you - but it seems that the choice is between (a) limiting "scp" functionality to address the security vulnerability, and (b) killing "scp" altogether.

I'd much prefer (a), even if it means I lose "scp remotehost:foo\* .".

Especially, since (almost always) I have equal privileges on both local and remote hosts, so in that case I just originate that "scp" from that remote. ;-)

TNX


?On 8/3/20, 11:09, "Thorsten Glaser" <t.glaser@tarent.de> wrote:

On Mon, 3 Aug 2020, Blumenthal, Uri - 0553 - MITLL wrote:

> I conjecture that only few of the existing use cases rely on remote expansion.

No, this is used all the time.

scp remotehost:foo\* .

(Unless rsync is available, but sadly that’s ? GPLv3 and ? not
universally installed.)

bye,
//mirabilos
--
tarent solutions GmbH
Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/
Tel: +49 228 54881-393 • Fax: +49 228 54881-235
HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941
Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Mon, 3 Aug 2020, Blumenthal, Uri - 0553 - MITLL wrote:

> I'd much prefer (a), even if it means I lose "scp remotehost:foo\* .".

That would be the same as killing scp…

> Especially, since (almost always) I have equal privileges on both
> local and remote hosts, so in that case I just originate that "scp"
> from that remote. ;-)

There’s privilegues, and there’s network (NAT gateways or
firewalls in between)…

bye,
//mirabilos
--
tarent solutions GmbH
Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/
Tel: +49 228 54881-393 • Fax: +49 228 54881-235
HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941
Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On 8/3/20, 13:18, "Thorsten Glaser" <t.glaser@tarent.de> wrote:
On Mon, 3 Aug 2020, Blumenthal, Uri - 0553 - MITLL wrote:

>> I'd much prefer (a), even if it means I lose "scp remotehost:foo\* .".
>
> That would be the same as killing scp…

Definitely not for me - and I'm pretty sure there are others in the same boat/position. So, again - the choice is between "killing scp" for some, and killing it for everybody. And I'd much prefer that we don't enforce "misery spreading" to cover everybody,

>> Especially, since (almost always) I have equal privileges on both
>> local and remote hosts, so in that case I just originate that "scp"
>> from that remote. ;-)
>
> There’s privileges, and there’s network (NAT gateways or
> firewalls in between)…

True. That's a good point. My use case doesn't include/involve crossing firewalls. I think there's enough users on either side of this issue (those who need "scp" mostly/only within the cluster/domain, and those who use "scp" across NAT and/or firewall(s)). I'd say that there are greater security risks for the "firewall-crossing" users, so they should worry about potential vulnerabilities/exploits more.
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
Thorsten Glaser kirjoitti maanantai 3. elokuuta 2020:
> On Mon, 3 Aug 2020, Blumenthal, Uri - 0553 - MITLL wrote:
>
> > I conjecture that only few of the existing use cases rely on remote expansion.
>
> No, this is used all the time.
>
> scp remotehost:foo\* .
>
> (Unless rsync is available, but sadly that’s ? GPLv3 and ? not
> universally installed.)

And even if rsync *would* be there it is not really suited to fetching files without recursing into subdirectories... Hence case for scp.

- juice -


--
Sent from my SFOS/XperiaX
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
The fact that scp has been carried for ages is part of its value
proposition. It’s going to work. Much as I am fond of my dd/cat/tar/smoke
signal hacks, scp is the standard.

A bit of a tangent: There is a security hole in scp, ssh, TLS, basically
everything running over IP. As transfer sizes have increased significantly,
the identity of a certain population of files being moved is increasingly
obvious due to byte count, compressed or otherwise.

No, you don’t get to pad every transfer out to the maximum possible size,
or delay transfers until you can aggregate payloads. Yes, somebody thought
of that.

There are limits.

On Tue, Jun 16, 2020 at 9:47 AM Jakub Jelen <jjelen@redhat.com> wrote:

> Hello all,
>
> I believe we all can agree that scp is ugly protocol carried for ages
> only for its simplicity of its usage and really no dependencies as it
> is installed together with every ssh client. But as we have seen
> recently, its simplicity and flexibility comes with security issues
> [1], it does not have great performance and there is really no
> development in there.
>
> Over the years, we still keep recommending people to use sftp instead,
> but its api is not that flexible and simple to be usable as a drop-in
> replacement in scripts nor for the occasional ad-hoc transfers of few
> files from one server to another.
>
> Before I start hacking, I would like to hear some opinions from others,
> whether this is something planned, welcomed or whether there are some
> good reasons to keep scp alive.
>
> I have in my mind three things/steps that would make it possible:
>
> * Update sftp client to be drop-in replacement for scp
> (and/or)
> * Change scp to use sftp internally
>
> * Modify sshd to use some compatibility "scpd" to support old clients
>
> and some time later
>
> * Remove scp or replace it with a symlink
>
>
> [1] http://www.openssh.com/txt/release-8.0
>
> Any ideas/comments/suggestions?
>
>
> Best regards,
> --
> Jakub Jelen
> Senior Software Engineer
> Security Technologies
> Red Hat, Inc.
>
> _______________________________________________
> openssh-unix-dev mailing list
> openssh-unix-dev@mindrot.org
> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
>
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Mon, 2020-08-03 at 19:17 +0200, Thorsten Glaser wrote:
> That would be the same as killing scp…

Better that... than having an inherently insecure scp... or at least
make it absolutely clear and rename it to i[nsecure]scp.

If the core functionality of a program (which is here probably the
"secure") is no longer given, then it's IMO better to rather cause
breakage (at least for old clients), than to keep going.


Cheers,
Chris.

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On 8/3/20, 14:40, "openssh-unix-dev on behalf of Christoph Anton Mitterer" <openssh-unix-dev-bounces+uri=ll.mit.edu@mindrot.org on behalf of calestyo@scientia.net> wrote:
>> That would be the same as killing scp…
>
> Better that... than having an inherently insecure scp...

Not for me! Security provided by "scp" satisfies *my* use case.

> or at least
> make it absolutely clear and rename it to i[nsecure]scp.

Couldn't care less. There's a saying "You may call me 'pot' - just don't stick me into the oven"

> If the core functionality of a program (which is here probably the
> "secure") is no longer given, then it's IMO better to rather cause
> breakage (at least for old clients), than to keep going.

Again. For me that remote path expansion is not the "core". The "core" is the ability to do "cp" from one host within my security domain to another.
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Mon, 2020-08-03 at 18:44 +0000, Blumenthal, Uri - 0553 - MITLL
wrote:
> Again. For me that remote path expansion is not the "core". The
> "core" is the ability to do "cp" from one host within my security
> domain to another.

Well I guess mandatory should what most people want/can reasonably
expect/and are promised by a given program,... which is here most
likely secure copying over any line from/to an remote host (i.e.
including a scenario with rogue clients and/or servers).

I.e. it should not focus on what just certain people (who e.g. get
their security by other means) need... or even those who don't do their
homework, by e.g. not upgrading to recent versions/etc..



Cheers.

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
It seems to me that the "exploit" of

scp /sourcefile remoteserver:'`touch /tmp/exploit.sh`/targetfile'

can be simplified to

ssh remoteservertouch /tmp/exploit.sh

Or are we talking about using ssh in conjunction with some third-party
tool like "rssh", which claims to be able to grant scp access without
shell access?  If ssh itself has never claimed that was possible, then
maybe rssh should stop claiming that it is.

FWIW, I consider scp as a convenient shortcut for "ssh cat". Indeed, I
sometimes find myself transferring files which are multiple hops away
exactly like that:

ssh foo ssh bar cat baz >baz

If I want to transfer files to or from untrusted machines, or to offer
file transfer access without shell access, then that is what sftp is for.

Regards,

Brian.

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Mon, Aug 03, 2020 at 05:06:35PM +0000, "Blumenthal, Uri - 0553 - MITLL" <uri@ll.mit.edu> wrote:

> I hear you - but it seems that the choice is between (a) limiting
> "scp" functionality to address the security vulnerability, and (b)
> killing "scp" altogether.
>
> I'd much prefer (a), even if it means I lose "scp remotehost:foo\* .".
>
> Especially, since (almost always) I have equal privileges on both
> local and remote hosts, so in that case I just originate that "scp"
> from that remote. ;-)
>
> TNX

If you have equal privileges on both hosts, this isn't
a vulnerability. It's only a vulnerability in cases
where you have scp access to the remote host but you
are not supposed to have general ssh access (i.e. shell
access).

In such cases, this vulnerability can be mitigated by
the use of an ssh-specific command whitelisting control
such as:

github.com/raforg/sshdo (auto learn/unlearn policy, exact cmds, no regex)
github.com/daethnir/authprogs (manual policy, supports regex)

Disclaimer: I made sshdo so I'm biased. But if you
really think you need regex support and don't mind the
extra effort and the risk, authprogs will solve the
problem too. But I'd recommend reading the sshdo FAQ
before choosing.

cheers,
raf

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Mon, Aug 03, 2020 at 08:34:04PM +0200, Christoph Anton Mitterer <calestyo@scientia.net> wrote:

> On Mon, 2020-08-03 at 19:17 +0200, Thorsten Glaser wrote:
> > That would be the same as killing scp…
>
> Better that... than having an inherently insecure scp... or at least
> make it absolutely clear and rename it to i[nsecure]scp.

But it's not inherently insecure. For most cases, or at
least for the default case, where the users of scp are
also allowed to use ssh, this is not a vulnerability.
It only becomes insecure when general ssh access is not
allowed but scp access is.

> If the core functionality of a program (which is here probably the
> "secure") is no longer given, then it's IMO better to rather cause
> breakage (at least for old clients), than to keep going.

The core functionality is the encrypted transfer of
files. That is still there.

> Cheers,
> Chris.

cheers,
raf

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
Thank you - I wasn’t aware. Will check sshdo out.

Regards,
Uri

> On Aug 3, 2020, at 19:21, raf <ssh@raf.org> wrote:
>
> ?On Mon, Aug 03, 2020 at 05:06:35PM +0000, "Blumenthal, Uri - 0553 - MITLL" <uri@ll.mit.edu> wrote:
>
>> I hear you - but it seems that the choice is between (a) limiting
>> "scp" functionality to address the security vulnerability, and (b)
>> killing "scp" altogether.
>>
>> I'd much prefer (a), even if it means I lose "scp remotehost:foo\* .".
>>
>> Especially, since (almost always) I have equal privileges on both
>> local and remote hosts, so in that case I just originate that "scp"
>> from that remote. ;-)
>>
>> TNX
>
> If you have equal privileges on both hosts, this isn't
> a vulnerability. It's only a vulnerability in cases
> where you have scp access to the remote host but you
> are not supposed to have general ssh access (i.e. shell
> access).
>
> In such cases, this vulnerability can be mitigated by
> the use of an ssh-specific command whitelisting control
> such as:
>
> github.com/raforg/sshdo (auto learn/unlearn policy, exact cmds, no regex)
> github.com/daethnir/authprogs (manual policy, supports regex)
>
> Disclaimer: I made sshdo so I'm biased. But if you
> really think you need regex support and don't mind the
> extra effort and the risk, authprogs will solve the
> problem too. But I'd recommend reading the sshdo FAQ
> before choosing.
>
> cheers,
> raf
>
> _______________________________________________
> openssh-unix-dev mailing list
> openssh-unix-dev@mindrot.org
> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Tue, 4 Aug 2020, raf wrote:

> In such cases, this vulnerability can be mitigated by
> the use of an ssh-specific command whitelisting control
> such as:

Probably just as easy: give the user a restricted shell
(/bin/rmksh) as shell and set their PATH etc. suitably,
to not include any other commands.

bye,
//mirabilos
PS: Full disclosure: I’m the mksh developer
--
«MyISAM tables -will- get corrupted eventually. This is a fact of life. »
“mysql is about as much database as ms access” – “MSSQL at least descends
from a database” “it's a rebranded SyBase” “MySQL however was born from a
flatfile and went downhill from there” – “at least jetDB doesn’t claim to
be a database” (#nosec) ??? Please let MySQL and MariaDB finally die!
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
The fundamental problem is that the command line syntax of sftp is not
great. There's no fundamental reason that can't be fixed, without even
changing the existing sftp protocol. But as long as scp (and rsync)
exist the motivation to actually make the sftp command line tool nicer
is so low that even though people have talked about it for 20 years, it
hasn't happened yet. (And even though it would require orders of
magnitude less effort than has been expended complaining about sftp &
scp over the last 20 years...)

I don't see an obvious way forward until someone decides to actually
make the sftp client nicer to use. (No need to even talk about it: just
show up with a finished version that does what scp does!)
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
I don't understand the concern. I just tried

sftp remotehost:foo\* .

...and it did exactly what one would expect. It successfully retrieved all
files starting with foo.

It sure looks like globbing Just Works with sftp, so replacing the
underlying implementation of scp to use sftp should also work.

--Greg

On Mon, Aug 03, 2020 at 05:08:28PM +0200, Thorsten Glaser wrote:
> On Mon, 3 Aug 2020, Blumenthal, Uri - 0553 - MITLL wrote:
>
> > I conjecture that only few of the existing use cases rely on remote expansion.
>
> No, this is used all the time.
>
> scp remotehost:foo\* .
>
> (Unless rsync is available, but sadly that’s ? GPLv3 and ? not
> universally installed.)
>
> bye,
> //mirabilos
> --
> tarent solutions GmbH
> Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/
> Tel: +49 228 54881-393 • Fax: +49 228 54881-235
> HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941
> Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg
> _______________________________________________
> openssh-unix-dev mailing list
> openssh-unix-dev@mindrot.org
> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Tue, Aug 04, 2020 at 01:29:52AM +0200, Thorsten Glaser <t.glaser@tarent.de> wrote:

> On Tue, 4 Aug 2020, raf wrote:
>
> > In such cases, this vulnerability can be mitigated by
> > the use of an ssh-specific command whitelisting control
> > such as:
>
> Probably just as easy: give the user a restricted shell
> (/bin/rmksh) as shell and set their PATH etc. suitably,
> to not include any other commands.
>
> bye,
> //mirabilos
> PS: Full disclosure: I’m the mksh developer

I've thought of a valid use for this kind of behaviour
that someone might actually be relying on. :-)

scp sourcefile remoteserver:'`[ -d /a/b/c ] || mkdir -p /a/b/c`/a/b/c/targetfile'

(i.e. ensure that the destination directory exists before writing the file to it)

cheers,
raf

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
It seems that there are a few camps here:

* The scp power users - this camp believes that scp supporting backtick
notation is fine and that running arbitrary commands is a perfectly fine
thing to do.
* The restricted shell users - this camp believes that scp supporting
backtick may not be the best, and there are various restricted shells which
can prevent this. Power users may belong to this camp.
* The novice users - this camp is surprised to find that scp can be used to
run commands. Once they understand that the server runs "scp -t" it makes a
little more sense.

The problem that I see here is that this is not going to be obvious to
novice users. If you read the man pages ( https://man.openbsd.org/scp.1 ) I
don't see anything that suggests one could use backticks nor run shell
commands. If the solution to this is that the openssh team includes this as
a note in the man pages and posts under their security page that they are
clarifying that behavior I think that would be fine. Where this is going to
cause pain is if there are novice users who want to have a fileserver ( or
an account ) which disallows ssh access, but allows scp to send/receive
files. Those users are likely going to be bit by this.

I understand that the openssh team is not interested in making changes to
scp, but would a clarification on this being intentional behavior be
possible? Then the novice users could account for this in their restricted
shell setups.

Cheers,

Ethan

On Tue, Aug 4, 2020 at 3:41 PM raf <ssh@raf.org> wrote:

> On Tue, Aug 04, 2020 at 01:29:52AM +0200, Thorsten Glaser <
> t.glaser@tarent.de> wrote:
>
> > On Tue, 4 Aug 2020, raf wrote:
> >
> > > In such cases, this vulnerability can be mitigated by
> > > the use of an ssh-specific command whitelisting control
> > > such as:
> >
> > Probably just as easy: give the user a restricted shell
> > (/bin/rmksh) as shell and set their PATH etc. suitably,
> > to not include any other commands.
> >
> > bye,
> > //mirabilos
> > PS: Full disclosure: I’m the mksh developer
>
> I've thought of a valid use for this kind of behaviour
> that someone might actually be relying on. :-)
>
> scp sourcefile remoteserver:'`[ -d /a/b/c ] || mkdir -p
> /a/b/c`/a/b/c/targetfile'
>
> (i.e. ensure that the destination directory exists before writing the file
> to it)
>
> cheers,
> raf
>
> _______________________________________________
> openssh-unix-dev mailing list
> openssh-unix-dev@mindrot.org
> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
>
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Tue, Aug 04, 2020 at 10:23:57PM -0700, Ethan Rahn <ethan.rahn@gmail.com> wrote:

> On Tue, Aug 4, 2020 at 3:41 PM raf <ssh@raf.org> wrote:
>
> > On Tue, Aug 04, 2020 at 01:29:52AM +0200, Thorsten Glaser <
> > t.glaser@tarent.de> wrote:
> >
> > > On Tue, 4 Aug 2020, raf wrote:
> > >
> > > > In such cases, this vulnerability can be mitigated by
> > > > the use of an ssh-specific command whitelisting control
> > > > such as:
> > >
> > > Probably just as easy: give the user a restricted shell
> > > (/bin/rmksh) as shell and set their PATH etc. suitably,
> > > to not include any other commands.
> > >
> > > bye,
> > > //mirabilos
> > > PS: Full disclosure: I’m the mksh developer
> >
> > I've thought of a valid use for this kind of behaviour
> > that someone might actually be relying on. :-)
> >
> > scp sourcefile remoteserver:'`[ -d /a/b/c ] || mkdir -p
> > /a/b/c`/a/b/c/targetfile'
> >
> > (i.e. ensure that the destination directory exists before writing the file
> > to it)
> >
> > cheers,
> > raf
> >
>
> It seems that there are a few camps here:

I'm not sure that they are all necessarily different camps.

> * The scp power users - this camp believes that scp supporting backtick
> notation is fine and that running arbitrary commands is a perfectly fine
> thing to do.
> * The restricted shell users - this camp believes that scp supporting
> backtick may not be the best, and there are various restricted shells which
> can prevent this. Power users may belong to this camp.

Command whitelisting and/or restricted shells can allow
authorised uses of backtick shenanigans while
disallowing unauthorised uses of backtick shenanigans.
I think these are the same camp. Some users are allowed
to run arbitrary code. Other aren't. They can coexist.

> * The novice users - this camp is surprised to find that scp can be used to
> run commands. Once they understand that the server runs "scp -t" it makes a
> little more sense.

I really do think that the novice/default user is
someone who also has interactive ssh access and so
there's no problem. Whatever they can execute via scp
with backtick shenanigans, they could more easily
execute with ssh itself.

The problem is when, for example, you only have
scp/sftp access to a remote server, such as your bank,
and you use WinSCP to transfer transaction files to
them to be actioned (people do this where I work), and
the bank hasn't properly protected themselves from this
"vulnerability". I really hope all banks do take this
vulnerability into account (e.g. by just supporting
sftp). It matters a lot for them. But it's an issue for
the bank / remote server, not an issue for the user who
doesn't and shouldn't need to know anything about this
(in the banking case).

It's not novice users that need to think about this.
Most novice/default users have general ssh access and
they are just using it to use scp.

> The problem that I see here is that this is not going to be obvious to
> novice users. If you read the man pages ( https://man.openbsd.org/scp.1 ) I
> don't see anything that suggests one could use backticks nor run shell
> commands. If the solution to this is that the openssh team includes this as
> a note in the man pages and posts under their security page that they are
> clarifying that behavior I think that would be fine. Where this is going to
> cause pain is if there are novice users who want to have a fileserver ( or
> an account ) which disallows ssh access, but allows scp to send/receive
> files. Those users are likely going to be bit by this.

I don't think they will, because, if they want to
disallow general purpose ssh access (i.e. shell
access), then they must discover and implement forced
commands. I think that's the standard way to prevent
general shell access via ssh (at least that's what my
government's cyberspooks recommend). Once they are
administering an ssh server and implementing forced
commands, they are no longer a novice user, surely?

But yes, more documentation is always a good idea
in my opinion. The question is where it should go.

You also have to remember that novice users are not
going to try to compromise their remote server using
this technique. They're novice users, not malefactors.
They are not going to try this, and even if they do,
they are only doing things that are are allowed to do.

> I understand that the openssh team is not interested in making changes to
> scp, but would a clarification on this being intentional behavior be
> possible? Then the novice users could account for this in their restricted
> shell setups.

That sounds sensible. I think something should be done
to make the world realise that this CVE isn't important
in most cases. But its writeup seems OK in that
respect. I think it's a shame that there's a CVE out
there that makes scp look bad, when it's really fine in
most cases, and in cases where it's not fine, there are
solutions to mitigate it. Perhaps the scp manpage
could reference the CVE and list a few tools that can
mitigate the problem, or just reference the CVE and
mention forced commands.

But such documentation might not even belong in the scp
manpage itself. Perhaps it belongs whereever people
searching for CVEs will come across it.

Although I agree that it would be cute to put the
"mkdir -p" example in the scp manpage, and reference
the CVE, and refer the reader to the forced command
information in the sshd_config manpage.

I'd be happy to supply an scp manpage patch if there's
any interest in using it. Just let me know what I
should include. But maybe it doesn't belong there.

> Cheers,
> Ethan

cheers,
raf

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On 2020/08/05 16:17, raf wrote:
> The problem is when, for example, you only have
> scp/sftp access to a remote server, such as your bank,
> and you use WinSCP to transfer transaction files to
> them to be actioned (people do this where I work), and
> the bank hasn't properly protected themselves from this
> "vulnerability". I really hope all banks do take this
> vulnerability into account (e.g. by just supporting
> sftp). It matters a lot for them. But it's an issue for
> the bank / remote server, not an issue for the user who
> doesn't and shouldn't need to know anything about this
> (in the banking case).

It matters for the user too. They need to know whether to use an sftp
or an scp client, and if it's sftp then some things they may want to do
(copying a file *to* a remote server) need a complicated method if using
openssh's sftp client (echo "put foo" | sftp -f - hostname).

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Wed, 2020-08-05 at 08:33 +0100, Stuart Henderson wrote:
> On 2020/08/05 16:17, raf wrote:
> > The problem is when, for example, you only have
> > scp/sftp access to a remote server, such as your bank,
> > and you use WinSCP to transfer transaction files to
> > them to be actioned (people do this where I work), and
> > the bank hasn't properly protected themselves from this
> > "vulnerability". I really hope all banks do take this
> > vulnerability into account (e.g. by just supporting
> > sftp). It matters a lot for them. But it's an issue for
> > the bank / remote server, not an issue for the user who
> > doesn't and shouldn't need to know anything about this
> > (in the banking case).
>
> It matters for the user too. They need to know whether to use an sftp
> or an scp client, and if it's sftp then some things they may want to
> do
> (copying a file *to* a remote server) need a complicated method if
> using
> openssh's sftp client (echo "put foo" | sftp -f - hostname).

At this moment, downloading files using sftp works the same as with
scp:

sftp localhost:/tmp/scp.c /tmp/tmp

Extending sftp to work the same way for uploading files to avoid the
above mess should be also pretty easy and would cover the most common
use cases.

Getting complete feature-parity with scp would be another feat though.

Regards,
--
Jakub Jelen
Senior Software Engineer
Security Technologies
Red Hat, Inc.

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Tue, Aug 04, 2020 at 10:23:57PM -0700, Ethan Rahn <ethan.rahn@gmail.com> wrote:

> I understand that the openssh team is not interested in making changes to
> scp, but would a clarification on this being intentional behavior be
> possible? Then the novice users could account for this in their restricted
> shell setups.
>
> Cheers,
> Ethan

I think the key message worth documenting is the fact
that scp is as powerful as ssh and that to think that
it can only transfer files is a mistake that can lead
to security issues. If users need to be limited to
transferring files only, then their access should be
restricted by the server to sftp only.

Just a thought.

cheers,
raf

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On Wed, Aug 05, 2020 at 11:03:41AM +0200, Jakub Jelen wrote:
> At this moment, downloading files using sftp works the same as with
> scp:
>
> sftp localhost:/tmp/scp.c /tmp/tmp
>
> Extending sftp to work the same way for uploading files to avoid the
> above mess should be also pretty easy and would cover the most common
> use cases.

yes, but in 20 years nobody has gotten around to it. :)

> Getting complete feature-parity with scp would be another feat though.

I don't think many people want *complete* feature parity, and that's
probably impossible to do without reintroducing the same security issues
as scp. What would make sftp a viable replacement is simply supporting
sending and receiving files with optional permission preservation and
recursion (99.9% of what scp is used for). sftp is most of the way
there, but if someone can't send a file with a simple sftp invocation
they'll just keep using scp.

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Deprecation of scp protocol and improving sftp client [ In reply to ]
On 8/5/20 1:23 AM, Ethan Rahn wrote:
> It seems that there are a few camps here:
>
> * The scp power users - this camp believes that scp supporting backtick
> notation is fine and that running arbitrary commands is a perfectly fine
> thing to do.
> * The restricted shell users - this camp believes that scp supporting
> backtick may not be the best, and there are various restricted shells which
> can prevent this. Power users may belong to this camp.
> * The novice users - this camp is surprised to find that scp can be used to
> run commands. Once they understand that the server runs "scp -t" it makes a
> little more sense.


Sorry to come into this late but there is a very large camp that simply
doesn't care. They use scp because they have to in order to transfer
files due to requirements placed on them by admins. They aren't
concerned about security nearly as much as they just want to get their
files from A to B so they can do their work. For these people scp is the
default because that's what all the instructions and examples are based
on. It's a big part of the reason why I developed hpn-ssh. We couldn't
get the users to change their behaviour and they kept complaining about
slow transfers.

In short - for a whole lot of users scp is just a component of their
workflow. They don't really think about it unless it's causing problems.

So I'm all for getting rid of scp as long as you can get sftp to work in
exactly the same way. Then you just get replace scp with a symlink to
sftp. Which is far easier said than done.
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev