Mailing List Archive

SCP in SFTP mode
Hello,

When we use scp in sftp mode to copy folder to the machines with the "old"
version of ssh, we get the following error:
======
$ scp -vvv -r test/ touser@machine:/home/touser
...
debug1: Sending subsystem: sftp
...
debug2: subsystem request accepted on channel 0
debug2: Remote version: 3
debug2: Server supports extension "posix-rename@openssh.com" revision 1
debug2: Server supports extension "statvfs@openssh.com" revision 2
debug2: Server supports extension "fstatvfs@openssh.com" revision 2
debug2: Server supports extension "hardlink@openssh.com" revision 1
debug2: Server supports extension "fsync@openssh.com" revision 1
debug2: Server supports extension "lsetstat@openssh.com" revision 1
debug3: Sent message fd 6 T:17 I:1
debug3: Recevied stat reply T:105 I:1 F:0x000f M:40700
debug3: source_sftp: copying local test to remote /home/touser/test
debug3: Sent message fd 6 T:16 I:2
Couldn't canonicalize: No such file or directory


*Unable to canonicalize path "/home/touser/test"failed to upload directory
test to /home/touser/test*debug2: channel 0: read<=0 rfd 4 len 0
debug2: channel 0: read failed
debug2: chan_shutdown_read: channel 0: (i0 o0 sock -1 wfd 4 efd 6 [write])
debug2: channel 0: input open -> drain
debug2: channel 0: ibuf empty
debug2: channel 0: send eof
========

If I understand correctly, the failure is caused by the lack of the "
expand-path@openssh.com"
extension to sftp protocol in old releases and (lack of) processing it on
the server side.

For the recent version self-compatibility it was fixed in
https://github.com/openssh/openssh-portable/commit/ac7c9ec894ed0825d04ef69c55babb49bab1d32e

Is a similar fix feasible to reach better backward compatibility?

Many thanks in advance!

--
Dmitry Belyavskiy
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: SCP in SFTP mode [ In reply to ]
On Thu, 27 Jan 2022, Dmitry Belyavskiy wrote:

> Hello,
>
> When we use scp in sftp mode to copy folder to the machines with the "old"
> version of ssh, we get the following error:
> ======
> $ scp -vvv -r test/ touser@machine:/home/touser

What version of OpenSSH are you using? There are a bunch of fixes in git
HEAD after 8.8.

> If I understand correctly, the failure is caused by the lack of the "
> expand-path@openssh.com"
> extension to sftp protocol in old releases and (lack of) processing it on
> the server side.

No, that extension is only invoked when a ~-prefixed path is used.

AFAIK this was juat a behaviour difference: scp in sftp mode wasn't
creating the destination directory if it didn't exist, whereas legacy
scp would do so. It was fixed in the commit you identify below.

> For the recent version self-compatibility it was fixed in
> https://github.com/openssh/openssh-portable/commit/ac7c9ec894ed0825d04ef69c55babb49bab1d32e

-d
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: SCP in SFTP mode [ In reply to ]
Dear Damien,

On Fri, Jan 28, 2022 at 1:40 AM Damien Miller <djm@mindrot.org> wrote:

> On Thu, 27 Jan 2022, Dmitry Belyavskiy wrote:
>
> > Hello,
> >
> > When we use scp in sftp mode to copy folder to the machines with the
> "old"
> > version of ssh, we get the following error:
> > ======
> > $ scp -vvv -r test/ touser@machine:/home/touser
>
> What version of OpenSSH are you using? There are a bunch of fixes in git
> HEAD after 8.8.
>
> > If I understand correctly, the failure is caused by the lack of the "
> > expand-path@openssh.com"
> > extension to sftp protocol in old releases and (lack of) processing it on
> > the server side.
>
> No, that extension is only invoked when a ~-prefixed path is used.
>
> AFAIK this was juat a behaviour difference: scp in sftp mode wasn't
> creating the destination directory if it didn't exist, whereas legacy
> scp would do so. It was fixed in the commit you identify below.
>
> > For the recent version self-compatibility it was fixed in
> >
> https://github.com/openssh/openssh-portable/commit/ac7c9ec894ed0825d04ef69c55babb49bab1d32e
>
> -d
>
> I've just retested with the recent master

$ ./scp -S `pwd`/ssh -vvv -r -s `pwd`/openbsd-compat/ user@machine:

debug2: subsystem request accepted on channel 0
./scp: debug2: Remote version: 3
./scp: debug2: Server supports extension "posix-rename@openssh.com"
revision 1
./scp: debug2: Server supports extension "statvfs@openssh.com" revision 2
./scp: debug2: Server supports extension "fstatvfs@openssh.com" revision 2
./scp: debug2: Server supports extension "hardlink@openssh.com" revision 1
./scp: debug2: Server supports extension "fsync@openssh.com" revision 1
./scp: debug2: Server supports extension "lsetstat@openssh.com" revision 1
./scp: debug2: Sending SSH2_FXP_STAT "/root"
./scp: debug3: Sent message fd 6 T:17 I:1
./scp: debug3: Received stat reply T:105 I:1 F:0x000f M:40550
./scp: debug3: source_sftp: copying local
/home/dbelyavs/work/upstream/openssh-portable/openbsd-compat to remote
/root/openbsd-compat
./scp: debug2: Sending SSH2_FXP_REALPATH "/root/openbsd-compat"
./scp: debug3: Sent message fd 6 T:16 I:2
./scp: realpath /root/openbsd-compat: No such file
./scp: upload "/root/openbsd-compat": path canonicalization failed
./scp: failed to upload directory
/home/dbelyavs/work/upstream/openssh-portable/openbsd-compat to
/root/openbsd-compat

The failure occurs even when I specify absolute paths on the
source/destination machine.

The destination machine has OpenSSH_8.0p1 with patches.

--
Dmitry Belyavskiy
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: SCP in SFTP mode [ In reply to ]
Dear Damien,

Sorry for being repetitive - but should we consider it to be a known
limitation?

Many thanks in advance!

On Fri, Jan 28, 2022 at 10:53 AM Dmitry Belyavskiy <dbelyavs@redhat.com>
wrote:

> Dear Damien,
>
> On Fri, Jan 28, 2022 at 1:40 AM Damien Miller <djm@mindrot.org> wrote:
>
>> On Thu, 27 Jan 2022, Dmitry Belyavskiy wrote:
>>
>> > Hello,
>> >
>> > When we use scp in sftp mode to copy folder to the machines with the
>> "old"
>> > version of ssh, we get the following error:
>> > ======
>> > $ scp -vvv -r test/ touser@machine:/home/touser
>>
>> What version of OpenSSH are you using? There are a bunch of fixes in git
>> HEAD after 8.8.
>>
>> > If I understand correctly, the failure is caused by the lack of the "
>> > expand-path@openssh.com"
>> > extension to sftp protocol in old releases and (lack of) processing it
>> on
>> > the server side.
>>
>> No, that extension is only invoked when a ~-prefixed path is used.
>>
>> AFAIK this was juat a behaviour difference: scp in sftp mode wasn't
>> creating the destination directory if it didn't exist, whereas legacy
>> scp would do so. It was fixed in the commit you identify below.
>>
>> > For the recent version self-compatibility it was fixed in
>> >
>> https://github.com/openssh/openssh-portable/commit/ac7c9ec894ed0825d04ef69c55babb49bab1d32e
>>
>> -d
>>
>> I've just retested with the recent master
>
> $ ./scp -S `pwd`/ssh -vvv -r -s `pwd`/openbsd-compat/ user@machine:
>
> debug2: subsystem request accepted on channel 0
> ./scp: debug2: Remote version: 3
> ./scp: debug2: Server supports extension "posix-rename@openssh.com"
> revision 1
> ./scp: debug2: Server supports extension "statvfs@openssh.com" revision 2
> ./scp: debug2: Server supports extension "fstatvfs@openssh.com" revision 2
> ./scp: debug2: Server supports extension "hardlink@openssh.com" revision 1
> ./scp: debug2: Server supports extension "fsync@openssh.com" revision 1
> ./scp: debug2: Server supports extension "lsetstat@openssh.com" revision 1
> ./scp: debug2: Sending SSH2_FXP_STAT "/root"
> ./scp: debug3: Sent message fd 6 T:17 I:1
> ./scp: debug3: Received stat reply T:105 I:1 F:0x000f M:40550
> ./scp: debug3: source_sftp: copying local
> /home/dbelyavs/work/upstream/openssh-portable/openbsd-compat to remote
> /root/openbsd-compat
> ./scp: debug2: Sending SSH2_FXP_REALPATH "/root/openbsd-compat"
> ./scp: debug3: Sent message fd 6 T:16 I:2
> ./scp: realpath /root/openbsd-compat: No such file
> ./scp: upload "/root/openbsd-compat": path canonicalization failed
> ./scp: failed to upload directory
> /home/dbelyavs/work/upstream/openssh-portable/openbsd-compat to
> /root/openbsd-compat
>
> The failure occurs even when I specify absolute paths on the
> source/destination machine.
>
> The destination machine has OpenSSH_8.0p1 with patches.
>
> --
> Dmitry Belyavskiy
>


--
Dmitry Belyavskiy
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: SCP in SFTP mode [ In reply to ]
I haven't had time to investigate this yet

On Wed, 2 Feb 2022, Dmitry Belyavskiy wrote:

> Dear Damien,
> Sorry for being repetitive - but should we consider it to be a known
> limitation?
>
> Many thanks in advance!
>
> On Fri, Jan 28, 2022 at 10:53 AM Dmitry Belyavskiy <dbelyavs@redhat.com>
> wrote:
> Dear Damien,
>
> On Fri, Jan 28, 2022 at 1:40 AM Damien Miller <djm@mindrot.org> wrote:
> On Thu, 27 Jan 2022, Dmitry Belyavskiy wrote:
>
> > Hello,
> >
> > When we use scp in sftp mode to copy folder to the
> machines with the "old"
> > version of ssh, we get the following error:
> > ======
> > $ scp -vvv -r test/ touser@machine:/home/touser
>
> What version of OpenSSH are you using? There are a bunch
> of fixes in git
> HEAD after 8.8.
>
> > If I understand correctly, the failure is caused by the
> lack of the "
> > expand-path@openssh.com"
> > extension to sftp protocol in old releases and (lack of)
> processing it on
> > the server side.
>
> No, that extension is only invoked when a ~-prefixed path
> is used.
>
> AFAIK this was juat a behaviour difference: scp in sftp
> mode wasn't
> creating the destination directory if it didn't exist,
> whereas legacy
> scp would do so. It was fixed in the commit you identify
> below.
>
> > For the recent version self-compatibility it was fixed
> in
> >https://github.com/openssh/openssh-portable/commit/ac7c9ec894ed0825d04ef69c
> 55babb49bab1d32e
>
> -d
>
> I've just retested with the recent master
> $ ./scp -S `pwd`/ssh -vvv -r -s `pwd`/openbsd-compat/ user@machine:
>
> debug2: subsystem request accepted on channel 0
> ./scp: debug2: Remote version: 3
> ./scp: debug2: Server supports extension "posix-rename@openssh.com"
> revision 1
> ./scp: debug2: Server supports extension "statvfs@openssh.com"
> revision 2
> ./scp: debug2: Server supports extension "fstatvfs@openssh.com"
> revision 2
> ./scp: debug2: Server supports extension "hardlink@openssh.com"
> revision 1
> ./scp: debug2: Server supports extension "fsync@openssh.com" revision
> 1
> ./scp: debug2: Server supports extension "lsetstat@openssh.com"
> revision 1
> ./scp: debug2: Sending SSH2_FXP_STAT "/root"
> ./scp: debug3: Sent message fd 6 T:17 I:1
> ./scp: debug3: Received stat reply T:105 I:1 F:0x000f M:40550
> ./scp: debug3: source_sftp: copying local
> /home/dbelyavs/work/upstream/openssh-portable/openbsd-compat to remote
> /root/openbsd-compat
> ./scp: debug2: Sending SSH2_FXP_REALPATH "/root/openbsd-compat"
> ./scp: debug3: Sent message fd 6 T:16 I:2
> ./scp: realpath /root/openbsd-compat: No such file
> ./scp: upload "/root/openbsd-compat": path canonicalization failed
> ./scp: failed to upload directory
> /home/dbelyavs/work/upstream/openssh-portable/openbsd-compat to
> /root/openbsd-compat
>
> The failure occurs even when I specify absolute paths on the
> source/destination machine.
>
> The destination machine has OpenSSH_8.0p1 with patches.
>
> --
> Dmitry Belyavskiy
>
>
>
> --
> Dmitry Belyavskiy
>
>
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: SCP in SFTP mode [ In reply to ]
Dear Damien,

Here is a proof-of-concept patch fixing this issue for me. If it could be
polished, it would be great.

Many thanks in advance!

On Wed, Feb 2, 2022 at 10:57 PM Damien Miller <djm@mindrot.org> wrote:

> I haven't had time to investigate this yet
>
> On Wed, 2 Feb 2022, Dmitry Belyavskiy wrote:
>
> > Dear Damien,
> > Sorry for being repetitive - but should we consider it to be a known
> > limitation?
> >
> > Many thanks in advance!
> >
> > On Fri, Jan 28, 2022 at 10:53 AM Dmitry Belyavskiy <dbelyavs@redhat.com>
> > wrote:
> > Dear Damien,
> >
> > On Fri, Jan 28, 2022 at 1:40 AM Damien Miller <djm@mindrot.org> wrote:
> > On Thu, 27 Jan 2022, Dmitry Belyavskiy wrote:
> >
> > > Hello,
> > >
> > > When we use scp in sftp mode to copy folder to the
> > machines with the "old"
> > > version of ssh, we get the following error:
> > > ======
> > > $ scp -vvv -r test/ touser@machine:/home/touser
> >
> > What version of OpenSSH are you using? There are a bunch
> > of fixes in git
> > HEAD after 8.8.
> >
> > > If I understand correctly, the failure is caused by the
> > lack of the "
> > > expand-path@openssh.com"
> > > extension to sftp protocol in old releases and (lack of)
> > processing it on
> > > the server side.
> >
> > No, that extension is only invoked when a ~-prefixed path
> > is used.
> >
> > AFAIK this was juat a behaviour difference: scp in sftp
> > mode wasn't
> > creating the destination directory if it didn't exist,
> > whereas legacy
> > scp would do so. It was fixed in the commit you identify
> > below.
> >
> > > For the recent version self-compatibility it was fixed
> > in
> > >
> https://github.com/openssh/openssh-portable/commit/ac7c9ec894ed0825d04ef69c
> > 55babb49bab1d32e
> >
> > -d
> >
> > I've just retested with the recent master
> > $ ./scp -S `pwd`/ssh -vvv -r -s `pwd`/openbsd-compat/ user@machine:
> >
> > debug2: subsystem request accepted on channel 0
> > ./scp: debug2: Remote version: 3
> > ./scp: debug2: Server supports extension "posix-rename@openssh.com"
> > revision 1
> > ./scp: debug2: Server supports extension "statvfs@openssh.com"
> > revision 2
> > ./scp: debug2: Server supports extension "fstatvfs@openssh.com"
> > revision 2
> > ./scp: debug2: Server supports extension "hardlink@openssh.com"
> > revision 1
> > ./scp: debug2: Server supports extension "fsync@openssh.com" revision
> > 1
> > ./scp: debug2: Server supports extension "lsetstat@openssh.com"
> > revision 1
> > ./scp: debug2: Sending SSH2_FXP_STAT "/root"
> > ./scp: debug3: Sent message fd 6 T:17 I:1
> > ./scp: debug3: Received stat reply T:105 I:1 F:0x000f M:40550
> > ./scp: debug3: source_sftp: copying local
> > /home/dbelyavs/work/upstream/openssh-portable/openbsd-compat to remote
> > /root/openbsd-compat
> > ./scp: debug2: Sending SSH2_FXP_REALPATH "/root/openbsd-compat"
> > ./scp: debug3: Sent message fd 6 T:16 I:2
> > ./scp: realpath /root/openbsd-compat: No such file
> > ./scp: upload "/root/openbsd-compat": path canonicalization failed
> > ./scp: failed to upload directory
> > /home/dbelyavs/work/upstream/openssh-portable/openbsd-compat to
> > /root/openbsd-compat
> >
> > The failure occurs even when I specify absolute paths on the
> > source/destination machine.
> >
> > The destination machine has OpenSSH_8.0p1 with patches.
> >
> > --
> > Dmitry Belyavskiy
> >
> >
> >
> > --
> > Dmitry Belyavskiy
> >
> >



--
Dmitry Belyavskiy
Re: SCP in SFTP mode [ In reply to ]
On February 3, 2022 3:03:25 PM GMT+03:00, Dmitry Belyavskiy <dbelyavs@redhat.com> wrote:
>Dear Damien,
>
>Here is a proof-of-concept patch fixing this issue for me. If it could be
>polished, it would be great.
>
Didn't you forget to free both error message string and ssh buffer in the error path after do_mkdir?

Sincerely Yours, Dan.
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: SCP in SFTP mode [ In reply to ]
On Fri, Feb 4, 2022 at 7:29 AM Dan Yefihmov <dan@lightwave.net.ru> wrote:

> On February 3, 2022 3:03:25 PM GMT+03:00, Dmitry Belyavskiy <
> dbelyavs@redhat.com> wrote:
> >Dear Damien,
> >
> >Here is a proof-of-concept patch fixing this issue for me. If it could be
> >polished, it would be great.
> >
> Didn't you forget to free both error message string and ssh buffer in the
> error path after do_mkdir?
>

I did, and there are definitely more problems related to this patch. I'll
try to elaborate a better version.

--
Dmitry Belyavskiy
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: SCP in SFTP mode [ In reply to ]
Hello,

After a deeper investigation I think I can explain what happens.
Special thanks to Jakub Jelen who did the server side investigation.

Red Hat Enterprise Linux 8 series is based on OpenSSH 8.0, which is rather
old.

Soon after the OpenSSH 8.0 was published, the upstream introduced a
server-side fix for sftp, which changes the use of POSIX realpath (on
LInux) to the OpenBSD one with various quirks:
https://github.com/openssh/openssh-portable/commit/569b650f93b561c09c655f83f128e1dfffe74101

If the OpenBSD realpath is in use on the server, the current master
successfully copies directories using scp in SFTP mode. Otherwise problems
occur.

As RHEL 8 series will definitely be in wide use for several more years, I
kindly ask the OpenSSH upstream to consider this patch, which is trying to
workaround the server side on the client side:
https://github.com/openssh/openssh-portable/pull/299
It's a slightly tidied up version of the POC patch I've sent to this ML
before.

Many thanks in advance!


On Fri, Feb 4, 2022 at 11:15 AM Dmitry Belyavskiy <dbelyavs@redhat.com>
wrote:

>
>
> On Fri, Feb 4, 2022 at 7:29 AM Dan Yefihmov <dan@lightwave.net.ru> wrote:
>
>> On February 3, 2022 3:03:25 PM GMT+03:00, Dmitry Belyavskiy <
>> dbelyavs@redhat.com> wrote:
>> >Dear Damien,
>> >
>> >Here is a proof-of-concept patch fixing this issue for me. If it could be
>> >polished, it would be great.
>> >
>> Didn't you forget to free both error message string and ssh buffer in the
>> error path after do_mkdir?
>>
>
> I did, and there are definitely more problems related to this patch. I'll
> try to elaborate a better version.
>
> --
> Dmitry Belyavskiy
>


--
Dmitry Belyavskiy
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: SCP in SFTP mode [ In reply to ]
Dmitry Belyavskiy wrote:
> As RHEL 8 series will definitely be in wide use for several more years, I
> kindly ask the OpenSSH upstream to consider this patch, which is trying to
> workaround the server side on the client side:

I find it rude and quite outrageous to push a burden of compatibility
with a paid-for commercial product onto an open source project.

It's asking for the wrong solution in the wrong place.

If RHEL doesn't work as you like then you should obviously engage RH
to get them to qualify the software you want their product to have.

Now, if you can make a case explicitly for how OpenSSH git can improve
compatibility with older releases of upstream OpenSSH then that could
be a different story, and if you do so smartly it might also benefit
your commercial systems.


//Peter
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: SCP in SFTP mode [ In reply to ]
On Sun, Feb 6, 2022 at 11:05 AM Peter Stuge <peter@stuge.se> wrote:
>
> Dmitry Belyavskiy wrote:
> > As RHEL 8 series will definitely be in wide use for several more years, I
> > kindly ask the OpenSSH upstream to consider this patch, which is trying to
> > workaround the server side on the client side:
>
> I find it rude and quite outrageous to push a burden of compatibility
> with a paid-for commercial product onto an open source project.

He asked, nicely. He actually wrote the patch. And compatibility to a
quite popular and long lived distribution is a feature, not a bug.

I've myself tried to update OpenSSH for long-lived distributions like
RHEL. It can be a painful process, especially because OpenSSH is used
to manage other servers and can take out entire networks if
misapplied. And yes, been there, done that when someone demanded an
update without very thorough regression testing. Man, oh man, was I
glad I kept the active SSH connection open and was I glad that
restarting the daemon did not kill my active session on those!!!!
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: SCP in SFTP mode [ In reply to ]
On Fri, Feb 4, 2022 at 10:36 AM Dmitry Belyavskiy <dbelyavs@redhat.com> wrote:

> Red Hat Enterprise Linux 8 series is based on OpenSSH 8.0, which is
> rather old.
>
> Soon after the OpenSSH 8.0 was published, the upstream introduced a
> server-side fix for sftp, which changes the use of POSIX realpath
> (on Linux) to the OpenBSD one with various quirks:
>
> https://github.com/openssh/openssh-portable/commit/569b650f93b561c09c655f83f128e1dfffe74101
>
> If the OpenBSD realpath is in use on the server, the current master
> successfully copies directories using scp in SFTP mode. Otherwise
> problems occur.
>
> As RHEL 8 series will definitely be in wide use for several more
> years, I kindly ask the OpenSSH upstream to consider this patch,
> which is trying to workaround the server side on the client side:
>
> https://github.com/openssh/openssh-portable/pull/299
>
> It's a slightly tidied up version of the POC patch I've sent to this
> ML before.

Something else to consider: RHEL8 is still in Full Support phase:

https://access.redhat.com/support/policy/updates/errata/

While Red Hat almost never updates software packages to new versions
within the same release of RHEL, Red Hat will frequently apply
patches, so long as:

1. The version of RHEL for which you are requested the patch is still
under the Full Support phase.

2. The patch has already been accepted by upstream; that is, you are
requesting a backport of a feature / bugfix that was officially
added to the package after that version of RHEL shipped.

Both of these are true for RHEL8: RHEL8 is under Full Support through
2024, and the patch in question is part of the official OpenSSH
portable distribution. The patch is also small and reasonably
self-contained.

While it would be nice if later OpenSSH sftp client versions could
work around OpenSSH sftp server versions that still use the POSIX
realpath() function instead of the OpenBSD sftp_realpath() function,
fixing the RHEL8 sftp server would be nice, too.

What I normally do in cases like this is open a polite and thorough
Bugzilla bug to request the feature/patch, then open a Red Hat support
case and refer to the Bugzilla bug. I’ve had pretty good success over
the years getting fixes into RHEL with this approach.
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: SCP in SFTP mode [ In reply to ]
On Sun, Feb 6, 2022 at 11:06 AM Peter Stuge <peter@stuge.se> wrote:
> I find it rude and quite outrageous to push a burden of
> compatibility with a paid-for commercial product onto an open source
> project.

The OP is asking for official OpenSSH portable 8.8 (circa 2021-09-26)
sftp clients to behave better when talking to official OpenSSH 8.0
(circa 2019-04-17) portable sftp servers. Red Hat has nothing to do
with the request, except for the fact that a RHEL8 system is where the
OP is encountering the OpenSSH 8.0 sftp server.

This request is neither rude nor outrageous. The OP asked politely,
and attempted to do most of the work.

> It's asking for the wrong solution in the wrong place.

It is not. Not everyone has the ability to upgrade every host to the
latest version of OpenSSH client/server. Both backwards and forwards
compatibility are desirable things.

> Now, if you can make a case explicitly for how OpenSSH git can
> improve compatibility with older releases of upstream OpenSSH then
> that could be a different story, and if you do so smartly it might
> also benefit your commercial systems.

That is, quite literally, *exactly* the case the OP made.
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: SCP in SFTP mode [ In reply to ]
On Thu, 3 Feb 2022, Dmitry Belyavskiy wrote:

> Dear Damien,
>
> Here is a proof-of-concept patch fixing this issue for me. If it could be
> polished, it would be great.


> diff --git a/sftp-client.c b/sftp-client.c
> index c7565755..98f986d6 100644
> --- a/sftp-client.c
> +++ b/sftp-client.c
> @@ -1028,11 +1028,42 @@ do_realpath_expand(struct sftp_conn *conn, const char *path, int expand)
...
> + if (status != SSH2_FX_NO_SUCH_FILE) {
> + error("%s %s: %s", expand ? "expand" : "realpath",
> + path, *errmsg == '\0' ? fx2txt(status) : errmsg);
> + free(errmsg);
> + sshbuf_free(msg);
> + return NULL;
> + } else {
> + if((r = do_mkdir(conn, path, &a, 0)) != 0) {
> + return NULL;
> + }

Sorry, but no way. This will attempt a directory creation any time a
realpath operations fails. Any "cd" in the sftp client will have the
chance to create a directory. scp without -r could create directories,
etc.

This server-side bug is difficult to fix in the client. Basically
you'd need to reimplement realpath() on the client side using remote
stat/readlink operations. It's possible, but not trivial, and a fair bit
of compat code for us to carry around in perpetuity.

I strongly recommend getting your employer to backport the existing
sftp-server fixes (569b650f93b and 53a6ebf1445).

-d
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: SCP in SFTP mode [ In reply to ]
Dear Damien,
Thanks for your review!

On Sun, Feb 6, 2022 at 11:37 PM Damien Miller <djm@mindrot.org> wrote:

> On Thu, 3 Feb 2022, Dmitry Belyavskiy wrote:
>
> > Dear Damien,
> >
> > Here is a proof-of-concept patch fixing this issue for me. If it could be
> > polished, it would be great.
>
>
> > diff --git a/sftp-client.c b/sftp-client.c
> > index c7565755..98f986d6 100644
> > --- a/sftp-client.c
> > +++ b/sftp-client.c
> > @@ -1028,11 +1028,42 @@ do_realpath_expand(struct sftp_conn *conn, const
> char *path, int expand)
> ...
> > + if (status != SSH2_FX_NO_SUCH_FILE) {
> > + error("%s %s: %s", expand ? "expand" : "realpath",
> > + path, *errmsg == '\0' ? fx2txt(status) :
> errmsg);
> > + free(errmsg);
> > + sshbuf_free(msg);
> > + return NULL;
> > + } else {
> > + if((r = do_mkdir(conn, path, &a, 0)) != 0) {
> > + return NULL;
> > + }
>
> Sorry, but no way. This will attempt a directory creation any time a
> realpath operations fails. Any "cd" in the sftp client will have the
> chance to create a directory. scp without -r could create directories,
> etc.
>

The updated version of the patch (
https://github.com/openssh/openssh-portable/pull/299) looks like fixing
these issues:
I explicitly pass the flag allowing creating the directory only in case
when scp -r is in use.


> This server-side bug is difficult to fix in the client. Basically
> you'd need to reimplement realpath() on the client side using remote
> stat/readlink operations. It's possible, but not trivial, and a fair bit
> of compat code for us to carry around in perpetuity.
>

I agree, this is not feasible.


> I strongly recommend getting your employer to backport the existing
> sftp-server fixes (569b650f93b and 53a6ebf1445).
>

Sure, these fixes are worth backporting though deploying them may take a
long time.

--
Dmitry Belyavskiy
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev