Mailing List Archive

Connection hang, can't stop SSH
Using OpenSSH_8.3p1 I had an open (working) connection to some other
box; after a bit of inactivity, some device in the middle seems to have
forgotten about the TCP connection (NAT) and broke it.

I've got an EscapeChar defined, though; so first I tried to send a BREAK
and, when that didn't help (TCP already gone, packets get lost!), I
tried (just out of curiosity) a Rekey.

Now I can see my TCP side hanging with about 4KB in the socket buffers -
but the SSH process (that previously reacted fine upon my EscapeChar)
now is completely dead.

- EscapeChar doesn't work
- Ctrl-Z doesn't work
- Ctrl-C doesn't work


strace shows ssh hanging in
pselect6(8, [3], [], NULL, NULL, NULL...


So yeah, my fault for trying a rekey on a broken connection... but I'd
have thought that my EscapeChar stays available, so that I can easily
get my terminal back!
(Yeah, I can just kill ssh from another. But what if it was my last
terminal after breaking the sshd setup?!?!)


Is it easy to get EscapeChar working all of the time?


Thanks!

Ph.
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Connection hang, can't stop SSH [ In reply to ]
On Wed, 14 Oct 2020, Philipp Marek wrote:

> Using OpenSSH_8.3p1 I had an open (working) connection to some other
> box; after a bit of inactivity, some device in the middle seems to have
> forgotten about the TCP connection (NAT) and broke it.
>
> I've got an EscapeChar defined, though; so first I tried to send a BREAK
> and, when that didn't help (TCP already gone, packets get lost!), I
> tried (just out of curiosity) a Rekey.
>
> Now I can see my TCP side hanging with about 4KB in the socket buffers -
> but the SSH process (that previously reacted fine upon my EscapeChar)
> now is completely dead.
>
> - EscapeChar doesn't work
> - Ctrl-Z doesn't work
> - Ctrl-C doesn't work
>
>
> strace shows ssh hanging in
> pselect6(8, [3], [], NULL, NULL, NULL...
>
>
> So yeah, my fault for trying a rekey on a broken connection... but I'd
> have thought that my EscapeChar stays available, so that I can easily
> get my terminal back!
> (Yeah, I can just kill ssh from another. But what if it was my last
> terminal after breaking the sshd setup?!?!)
>
>
> Is it easy to get EscapeChar working all of the time?

no, not easy unfortunately. While key exchange is in progress, the input
channels are not polled. Fixing this would involve some fiddly plumbing
as the IO handling is fairly intimately tied to the channel state
machine, and the channel state machine drives all sorts of events that
will try to send packets to the server, and we cannot emit packets while
in rekeying.

Attached is a stab at it, completely untested :)

-d
Re: Connection hang, can't stop SSH [ In reply to ]
Hi Damien,

thanks for the quick answer!

>> Is it easy to get EscapeChar working all of the time?
>
> no, not easy unfortunately. While key exchange is in progress, the
> input
> channels are not polled. Fixing this would involve some fiddly plumbing
> as the IO handling is fairly intimately tied to the channel state
> machine, and the channel state machine drives all sorts of events that
> will try to send packets to the server, and we cannot emit packets
> while
> in rekeying.
>
> Attached is a stab at it, completely untested :)

Hey, great! Any chance to get that upstream as well?


Thanks!

Ph.

_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev
Re: Connection hang, can't stop SSH [ In reply to ]
On Thu, 15 Oct 2020, Philipp Marek wrote:

> Hi Damien,
>
> thanks for the quick answer!
>
> > > Is it easy to get EscapeChar working all of the time?
> >
> > no, not easy unfortunately. While key exchange is in progress, the input
> > channels are not polled. Fixing this would involve some fiddly plumbing
> > as the IO handling is fairly intimately tied to the channel state
> > machine, and the channel state machine drives all sorts of events that
> > will try to send packets to the server, and we cannot emit packets while
> > in rekeying.
> >
> > Attached is a stab at it, completely untested :)
>
> Hey, great! Any chance to get that upstream as well?

It will need some testing first :/

-d
_______________________________________________
openssh-unix-dev mailing list
openssh-unix-dev@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev