Mailing List Archive: [PATCH] Add a Maximum Idle Time (1.2.2)

[PATCH] Add a Maximum Idle Time (1.2.2)

jacob at velius

Mar 3, 2000, 1:33 PM

Post #1 of 16 (3380 views)

The attached patch adds an option (off by default to preserve current
behavior) to set a timeout on the select() statement that waits for input
in clientloop.c. This fixes a timeout issue for me (explained below) and
probably also fixes the timeouts mentioned in last month's thread "Idle
time out". The patch is also available by http from:
http://www.chaos2.org/~jacob/code/patch-openssh-1.2.2-trans_inter

I am ssh-ing from a machine on my home network to one on the
internet. This goes out over a Linux ip_masquerade firewall. When I
wrote the attached patch, I thought it was the firewall that was killing
the connection by timing out on the redirected port due to lack of
traffic. But after reading some similar posts on this list, I think there
might be problems even if a firewall isn't involved. Also note that in
the tcpdump below, I did have KeepAlive turned on (both server and client)
and yet I don't see any traffic being generated due to this, which seems
to render KeepAlive pretty useless...

When ssh dies on me (when no max idle time is set) it gives me the
error below:

"
velius:~% Read from remote host velius.chaos2.org: Connection reset by peer
Connection to velius.chaos2.org closed.
jacob:~#
"

From the tcpdump below, we see that the firewall has assigned a new
ip_masq port. This shows all the packets; specifically, none are
generated in the interim.

"
00:59:19.987703 velius.chaos2.org.ssh > c392100-a.crvlls1.or.home.com.64579: P 1:21(20) ack 20 win 32120
<nop,nop,timestamp 46926353 47417028> (DF)
00:59:19.998389 c392100-a.crvlls1.or.home.com.64579 > velius.chaos2.org.ssh: . ack 21 win 32120
<nop,nop,timestamp 47417072 46926353> (DF) [tos 0x10]
... time passes here but no traffic to velius ...
01:20:37.477884 c392100-a.crvlls1.or.home.com.64687 > velius.chaos2.org.ssh: P 2954940853:2954940873(20) ack
2970631452 win 32120 <nop,nop,timestamp 47544804 46926353> (DF) [tos 0x10]
01:20:37.583097 velius.chaos2.org.ssh > c392100-a.crvlls1.or.home.com.64687: R 2970631452:2970631452(0) win 0
[tos 0x10]
"

The attached patch allows the user to put a TransmitInterlude option
in their ssh_config file that gives how many seconds are allowed to pass
without generating traffic. A value of 300 completely solves the timeouts
for me and I haven't observed any stability issues.

Please cc me with comments as I am not subscribed to the list.

Jacob Lundberg
jacob@chaos2.org

--

"Heh. You mean this is Stef's source code?"
-User Friendly

Re: [PATCH] Add a Maximum Idle Time (1.2.2) [ In reply to ]

Mar 4, 2000, 2:24 PM

Post #2 of 16 (3365 views)

Hi, I have applied this patch, and it works pretty well. Well, I agree
with you about the KeepAlive thing, it seems not functioning, at least on
Linux machine.
I just glaced at your patch, seems you send a "NOP" every TransInterclude
time. While this solves the problem while using the openssh client, but
people who use different ssh client may be unlucky. Maybe the best choice
is to implement it on server side??
Anyway, my problem is gone and thank you very much!
On Fri, 3 Mar 2000, Jacob Luna Lundberg wrote:

>
> The attached patch adds an option (off by default to preserve current
> behavior) to set a timeout on the select() statement that waits for input
> in clientloop.c. This fixes a timeout issue for me (explained below) and
> probably also fixes the timeouts mentioned in last month's thread "Idle
> time out". The patch is also available by http from:
> http://www.chaos2.org/~jacob/code/patch-openssh-1.2.2-trans_inter
>
> I am ssh-ing from a machine on my home network to one on the
> internet. This goes out over a Linux ip_masquerade firewall. When I
> wrote the attached patch, I thought it was the firewall that was killing
> the connection by timing out on the redirected port due to lack of
> traffic. But after reading some similar posts on this list, I think there
> might be problems even if a firewall isn't involved. Also note that in
> the tcpdump below, I did have KeepAlive turned on (both server and client)
> and yet I don't see any traffic being generated due to this, which seems
> to render KeepAlive pretty useless...
>
> When ssh dies on me (when no max idle time is set) it gives me the
> error below:
>
> "
> velius:~% Read from remote host velius.chaos2.org: Connection reset by peer
> Connection to velius.chaos2.org closed.
> jacob:~#
> "
>
> From the tcpdump below, we see that the firewall has assigned a new
> ip_masq port. This shows all the packets; specifically, none are
> generated in the interim.
>
> "
> 00:59:19.987703 velius.chaos2.org.ssh > c392100-a.crvlls1.or.home.com.64579: P 1:21(20) ack 20 win 32120
> <nop,nop,timestamp 46926353 47417028> (DF)
> 00:59:19.998389 c392100-a.crvlls1.or.home.com.64579 > velius.chaos2.org.ssh: . ack 21 win 32120
> <nop,nop,timestamp 47417072 46926353> (DF) [tos 0x10]
> ... time passes here but no traffic to velius ...
> 01:20:37.477884 c392100-a.crvlls1.or.home.com.64687 > velius.chaos2.org.ssh: P 2954940853:2954940873(20) ack
> 2970631452 win 32120 <nop,nop,timestamp 47544804 46926353> (DF) [tos 0x10]
> 01:20:37.583097 velius.chaos2.org.ssh > c392100-a.crvlls1.or.home.com.64687: R 2970631452:2970631452(0) win 0
> [tos 0x10]
> "
>
> The attached patch allows the user to put a TransmitInterlude option
> in their ssh_config file that gives how many seconds are allowed to pass
> without generating traffic. A value of 300 completely solves the timeouts
> for me and I haven't observed any stability issues.
>
> Please cc me with comments as I am not subscribed to the list.
>
> Jacob Lundberg
> jacob@chaos2.org
>
> --
>
> "Heh. You mean this is Stef's source code?"
> -User Friendly
>

-- "My grandpa told me to remember two things in life. Look out
for Number One, and remember your number" - Orville Cogswell
-- http://members.xoom.com/_XOOM/dizhao/index.html

Re: [PATCH] Add a Maximum Idle Time (1.2.2) [ In reply to ]

Mar 4, 2000, 2:44 PM

Post #3 of 16 (3373 views)

On Fri, 3 Mar 2000, Jacob Luna Lundberg wrote:

> The attached patch allows the user to put a TransmitInterlude
> option in their ssh_config file that gives how many seconds are
> allowed to pass without generating traffic. A value of 300
> completely solves the timeouts for me and I haven't observed any
> stability issues.

I would first rather get to the bottom of figuring out why keepalives
aren't working.

Is "KeepAlive yes" set for both client and server? Is
/proc/sys/net/ipv4/tcp_keepalive_time set to less than the masquerading
timeouts?

-d

--
| "Bombay is 250ms from New York in the new world order" - Alan Cox
| Damien Miller - http://www.mindrot.org/
| Email: djm@mindrot.org (home) -or- djm@ibs.com.au (work)

Re: [PATCH] Add a Maximum Idle Time (1.2.2) [ In reply to ]

jacob at velius

Mar 4, 2000, 3:12 PM

Post #4 of 16 (3365 views)

On Sun, 5 Mar 2000, Damien Miller wrote:

> I would first rather get to the bottom of figuring out why keepalives
> aren't working.

Which brings a question to mind. I haven't really programmed with
keepalives before. I presume they're a field in the tcp frames (as
opposed to a periodic empty frame)?

> Is "KeepAlive yes" set for both client and server?

Yes. To no avail. Both are Linux 2.2.14 boxen, but I have been
seeing this problem since I switched to 2.1.x (client with server still
2.0.x). I still see it with the not-so-open ssh suite as well, both 1.x
and 2.x. I tried kernel 2.3.42 and was still seeing it there too.

> Is /proc/sys/net/ipv4/tcp_keepalive_time set

It is set to 7200.

> to less than the masquerading timeouts?

I checked after reading the recent list entries and actually I see
the exact same behavior when I run ssh out from the firewall (thus
bypassing the ip_masq). So while the ip_masq is an issue, it is
orthogonal to the problem and could be resolved (as you say) by setting
the keepalives to less than the ip_masq timeout. For some reason,
keepalives aren't sufficient to keep some connections alive right now.

On a side note, Di Zhao asked if I should have implemented a server
version of the patch as well. I rather felt that (from what I've seen)
the problem is a bit too infrequent for that (let the users turn it on if
they discover they need it)... But I suppose it does leave people using
different clients out in the cold. Any preference there?

-Jacob

--

"Heh. You mean this is Stef's source code?"
-User Friendly

Re: [PATCH] Add a Maximum Idle Time (1.2.2) [ In reply to ]

jacob at velius

Mar 5, 2000, 12:55 AM

Post #5 of 16 (3367 views)

> I would first rather get to the bottom of figuring out why keepalives
> aren't working.

Ok. I've played around some and now understand keepalives a bit
better. So that 7200 setting would be two hours, which is rather long.
And in fact it turns out that setting it to 300 solves the problem for me.
But it is of note that 7200 is the _default_ value. And also I'm still
not sure why a setting of 7200 (both server and client) would break
things. (Also finally I see some keepalive packets going by so now I
understand much better what they are.)

> /proc/sys/net/ipv4/tcp_keepalive_time set to less than the masquerading timeouts?

The question remains here: what if you can't get your sysadmin to go
tweaking with the kernel default keepalive of 7200 seconds? Do we just
say to such a person, "too bad!" or do we let them send packets on their
own to keep the connection alive?

Unless I misunderstand, keepalive default is set here:
/usr/src/linux/include/net/tcp.h line 264 (Linux 2.2.14)
#define TCP_KEEPALIVE_TIME (120*60*HZ) /* two hours */

I know it could be contrued as bloat, which is why my patch didn't
include a commandline option. I think the option itself is useful for the
purpose of empowering the user (in a non-security threatening way ;).

-Jacob

--

"Heh. You mean this is Stef's source code?"
-User Friendly

Re: [PATCH] Add a Maximum Idle Time (1.2.2) [ In reply to ]

openssh-unix-dev.mindrot.org at marc-haber

Mar 5, 2000, 3:08 AM

Post #6 of 16 (3368 views)

On Fri, 3 Mar 2000 13:33:40 -0800 (PST), you wrote:
> When ssh dies on me (when no max idle time is set) it gives me the
>error below:
>
>"
>velius:~% Read from remote host velius.chaos2.org: Connection reset by peer
>Connection to velius.chaos2.org closed.
>jacob:~#
>"

I see the very same problem in my setup. Do I have to apply that patch
to both client and server?

Greetings
Marc

--
-------------------------------------- !! No courtesy copies, please !! -----
Marc Haber | " Questions are the | Mailadresse im Header
Karlsruhe, Germany | Beginning of Wisdom " | Fon: *49 721 966 32 15
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fax: *49 721 966 31 29

Re: [PATCH] Add a Maximum Idle Time (1.2.2) [ In reply to ]

jacob at velius

Mar 5, 2000, 12:57 PM

Post #7 of 16 (3366 views)

> I see the very same problem in my setup. Do I have to apply that patch
> to both client and server?

It only patches the client, so you can get away with just putting it
on your client (which won't help people ssh-ing _from_ the server). If
you're running Linux on the client and can become root, you could also try
(with KeepAlive yes set on both server and client):
/bin/echo "300\c" > /proc/sys/net/ipv4/tcp_keepalive_time

> !! No courtesy copies, please !!

Fair enough, but please do cc me; I don't usually read the list.

-Jacob

--

"Heh. You mean this is Stef's source code?"
-User Friendly

Re: [PATCH] Add a Maximum Idle Time (1.2.2) [ In reply to ]

markus.friedl at informatik

Mar 5, 2000, 11:50 PM

Post #8 of 16 (3373 views)

the patch looks reasonable, but SSH_MSG_NONE type packets
must not travel over the wire. this violates the protocol spec.

On Fri, Mar 03, 2000 at 01:33:40PM -0800, Jacob Luna Lundberg wrote:
> + packet_start(SSH_MSG_NONE);
> + packet_send();

SSH_MSG_IGNORE should be used, e.g.:

packet_start(SSH_MSG_IGNORE);
packet_put_string("bla", 3);
packet_send();

-markus

Re: [PATCH] Add a Maximum Idle Time (1.2.2) [ In reply to ]

jacob at velius

Mar 6, 2000, 12:38 AM

Post #9 of 16 (3368 views)

On Mon, 6 Mar 2000, Markus Friedl wrote:
> the patch looks reasonable, but SSH_MSG_NONE type packets
> must not travel over the wire. this violates the protocol spec.

Ok. Corrected patch attached. :) As before, it's also here:
http://www.chaos2.org/~jacob/code/patch-openssh-1.2.2-trans_inter-r1

> SSH_MSG_IGNORE should be used, e.g.:
> packet_start(SSH_MSG_IGNORE);
> packet_put_string("bla", 3);
> packet_send();

I wondered if I needed to stuff them with something. Open sshd
didn't seem to mind if they were empty, but closed sshd terminated the
connection. Thanks for the showing me how...

-Jacob

--

"Heh. You mean this is Stef's source code?"
-User Friendly

Re: [PATCH] Add a Maximum Idle Time (1.2.2) [ In reply to ]

openssh-unix-dev.mindrot.org at marc-haber

Mar 6, 2000, 12:45 AM

Post #10 of 16 (3367 views)

On Sun, 5 Mar 2000 12:57:42 -0800 (PST), you wrote:
>> I see the very same problem in my setup. Do I have to apply that patch
>> to both client and server?
>
> It only patches the client, so you can get away with just putting it
>on your client (which won't help people ssh-ing _from_ the server).

Then my problem is somewhere else since my client is TeraTerm ssh on a
Windows box.

>> !! No courtesy copies, please !!
>
> Fair enough, but please do cc me; I don't usually read the list.

Done.

Greetings
Marc

--
-------------------------------------- !! No courtesy copies, please !! -----
Marc Haber | " Questions are the | Mailadresse im Header
Karlsruhe, Germany | Beginning of Wisdom " | Fon: *49 721 966 32 15
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fax: *49 721 966 31 29

Re: [PATCH] Add a Maximum Idle Time (1.2.2) [ In reply to ]

jacob at velius

Mar 6, 2000, 1:08 AM

Post #11 of 16 (3368 views)

Marc Haber <openssh-unix-dev.mindrot.org@marc-haber.de> wrote:
> Then my problem is somewhere else since my client is TeraTerm ssh on a
> Windows box.

I presume that the server is a Linux box? If you can get root on the
box that houses the server, try changing _its_ keepalive timeout to 300
seconds. That might do it for you.

The serverloop itself is a bit more complex conceptually than the
clientloop and already implements a timeout in order to collate small
transmissions into larger ones. The attached patch should be categorized
under "Garish Malformed Hackish Patchlets" and is not really implemented
right but could give a general idea of where you'd have to work to
implement ignore packets in the serverloop.

-Jacob

--

"Heh. You mean this is Stef's source code?"
-User Friendly

Re: [PATCH] Add a Maximum Idle Time (1.2.2) [ In reply to ]

nebulous at owlnet

Mar 6, 2000, 9:07 AM

Post #12 of 16 (3366 views)

> the patch looks reasonable, but SSH_MSG_NONE type packets
> must not travel over the wire. this violates the protocol spec.

> SSH_MSG_IGNORE should be used, e.g.:
>
> packet_start(SSH_MSG_IGNORE);
> packet_put_string("bla", 3);
> packet_send();
>
> -markus

One worry... I don't know very much about the protocol specs for ssh, but
I know a -little- about crypto. Would this allow an attacker to mount a
known-text attack? I.e. given that an attacker could now look for
regularly-spaced (probably default-spaced in most cases) packets, and
assume that they contained the string "bla", could the attacker then
have a toehold on finding out the key?

I recommend sending some random characters instead of a constant string
like "bla".

-Sean Lisse

Re: [PATCH] Add a Maximum Idle Time (1.2.2) [ In reply to ]

jacob at velius

Mar 7, 2000, 2:54 AM

Post #13 of 16 (3369 views)

Sean Aaron Lisse <nebulous@owlnet.rice.edu> wrote:
> I recommend sending some random characters instead of a constant string
> like "bla".

Done. I used random strings with a maximum length of 256 chars.
Sound good to everybody? And the patch is also at:
http://www.chaos2.org/~jacob/code/patch-openssh-1.2.2-trans_inter-r2

-Jacob

--

"Heh. You mean this is Stef's source code?"
-User Friendly

Re: [PATCH] Add a Maximum Idle Time (1.2.2) [ In reply to ]

openssh-unix-dev.mindrot.org at marc-haber

Mar 7, 2000, 6:02 AM

Post #14 of 16 (3363 views)

On Mon, 6 Mar 2000 01:08:59 -0800 (PST), you wrote:
>Marc Haber <openssh-unix-dev.mindrot.org@marc-haber.de> wrote:
>> Then my problem is somewhere else since my client is TeraTerm ssh on a
>> Windows box.
>
> I presume that the server is a Linux box?

Right.

>If you can get root on the
>box that houses the server, try changing _its_ keepalive timeout to 300
>seconds. That might do it for you.

Which config setting is that? sshd_config does only seem to have
keepalive = yes.

And I still don't understand why this problem only shows when the TCP
connection is NATed.

Greetings
Marc

--
-------------------------------------- !! No courtesy copies, please !! -----
Marc Haber | " Questions are the | Mailadresse im Header
Karlsruhe, Germany | Beginning of Wisdom " | Fon: *49 721 966 32 15
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fax: *49 721 966 31 29

Re: [PATCH] Add a Maximum Idle Time (1.2.2) [ In reply to ]

jacob at velius

Mar 7, 2000, 1:12 PM

Post #15 of 16 (3370 views)

Marc Haber <openssh-unix-dev.mindrot.org@marc-haber.de> wrote:
> >If you can get root on the box that houses the server, try changing
> >_its_ keepalive timeout to 300 seconds. That might do it for you.
>
> Which config setting is that? sshd_config does only seem to have
> keepalive =3D yes.

/proc/sys/net/ipv4/tcp_keepalive_time
Which defaults to 7200 on recent Linux kernel. So try:
/bin/echo "300\c" > /proc/sys/net/ipv4/tcp_keepalive_time
Which will give you a 5 min timeout instead of 2 hours...

> And I still don't understand why this problem only shows when the TCP
> connection is NATed.

A good question. The ssh connection seems to expect the keepalives more
frequently than every two hours. So without keepalives (or other packets)
happening however frequently it is ssh wants them to, it terminates the
connection (or perhaps linux reclaims the socket, I don't know).

-Jacob

--

"Heh. You mean this is Stef's source code?"
-User Friendly

Re: [PATCH] Add a Maximum Idle Time (1.2.2) [ In reply to ]

jacob at velius

Mar 7, 2000, 1:56 PM

Post #16 of 16 (3365 views)

Updated my last patch to 1.2.2p1.
http://www.chaos2.org/~jacob/code/patch-openssh-1.2.2p1-trans_inter-r2

I presume ;) that you guys will let me know if there is a preferred
way to generate a random string rather than using the function that just
moved into random.c...

-Jacob

--

"Heh. You mean this is Stef's source code?"
-User Friendly