Mailing List Archive: suspend/resume in trouble...

suspend/resume in trouble...

Mar 23, 2006, 2:06 AM

Post #1 of 10 (2488 views)

I love it when I learn something, but I hate when it is a new bug in UNIX.

It seems that once you have listen(2)'ed a socket, there is no way to
unlisten(2) again.

I might intuitively be expected that listen(2) with a queue length
of zero would make it reject all future connection attempts, but
the semantics for that case is to accept the one connection and
reject all others while that connection exists.

So to implement suspend/resume we will have to close and recreate
the (remote) listener socket(s) which means that doing so moves
from the mgt to the cache process and that restarting the cache
process without a break in service is not possible any more. (Unless
we resort to passing fd's over unix domain sockets, but I'm firmly
against that on religious reasons)

I've put the "suspend/resume" facility on the "needs further study"
for now.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

suspend/resume in trouble... [ In reply to ]

Mar 23, 2006, 2:06 AM

Post #2 of 10 (2469 views)

I love it when I learn something, but I hate when it is a new bug in UNIX.

It seems that once you have listen(2)'ed a socket, there is no way to
unlisten(2) again.

I might intuitively be expected that listen(2) with a queue length
of zero would make it reject all future connection attempts, but
the semantics for that case is to accept the one connection and
reject all others while that connection exists.

So to implement suspend/resume we will have to close and recreate
the (remote) listener socket(s) which means that doing so moves
from the mgt to the cache process and that restarting the cache
process without a break in service is not possible any more. (Unless
we resort to passing fd's over unix domain sockets, but I'm firmly
against that on religious reasons)

I've put the "suspend/resume" facility on the "needs further study"
for now.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

suspend/resume in trouble... [ In reply to ]

Mar 23, 2006, 4:39 AM

Post #3 of 10 (2459 views)

"Poul-Henning Kamp" <phk at phk.freebsd.dk> writes:
> I love it when I learn something, but I hate when it is a new bug in
> UNIX.
>
> It seems that once you have listen(2)'ed a socket, there is no way
> to unlisten(2) again.
>
> I might intuitively be expected that listen(2) with a queue length
> of zero would make it reject all future connection attempts, but the
> semantics for that case is to accept the one connection and reject
> all others while that connection exists.

Well, we can easily fix that in FreeBSD.

Strictly speaking, POSIX allows listen(0) to work as we want it to:

A backlog argument of 0 may allow the socket to accept connections,
in which case the length of the listen queue may be set to an
implementation-defined minimum value.

(note "may")

However, there's a significant chance this may break existing
applications which expect listen(0) to be equivalent to
listen(SOME_IMPLEMENTATION_DEFINED_NON_ZERO_VALUE). It's also very
hard to test for in a configure script. Therefore, it's probably
simpler to add an unlisten() syscall.

I'm fairly certain I can come up with a patch for Linux as well, but
it will take time to get it accepted.

On systems which lack unlisten(), a suspended server can simply reply
to any incoming connections with 503 Service Unavailable, without even
bothering to look at the request.

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no

suspend/resume in trouble... [ In reply to ]

Mar 23, 2006, 4:39 AM

Post #4 of 10 (2472 views)

"Poul-Henning Kamp" <phk at phk.freebsd.dk> writes:
> I love it when I learn something, but I hate when it is a new bug in
> UNIX.
>
> It seems that once you have listen(2)'ed a socket, there is no way
> to unlisten(2) again.
>
> I might intuitively be expected that listen(2) with a queue length
> of zero would make it reject all future connection attempts, but the
> semantics for that case is to accept the one connection and reject
> all others while that connection exists.

Well, we can easily fix that in FreeBSD.

Strictly speaking, POSIX allows listen(0) to work as we want it to:

A backlog argument of 0 may allow the socket to accept connections,
in which case the length of the listen queue may be set to an
implementation-defined minimum value.

(note "may")

However, there's a significant chance this may break existing
applications which expect listen(0) to be equivalent to
listen(SOME_IMPLEMENTATION_DEFINED_NON_ZERO_VALUE). It's also very
hard to test for in a configure script. Therefore, it's probably
simpler to add an unlisten() syscall.

I'm fairly certain I can come up with a patch for Linux as well, but
it will take time to get it accepted.

On systems which lack unlisten(), a suspended server can simply reply
to any incoming connections with 503 Service Unavailable, without even
bothering to look at the request.

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no

suspend/resume in trouble... [ In reply to ]

Mar 23, 2006, 4:51 AM

Post #5 of 10 (2472 views)

In message <ujrirq5xud9.fsf at cat.linpro.no>, Dag-Erling =?iso-8859-1?Q?Sm=F8rgra
v?= writes:

>However, there's a significant chance this may break existing
>applications which expect listen(0) to be equivalent to
>listen(SOME_IMPLEMENTATION_DEFINED_NON_ZERO_VALUE). It's also very
>hard to test for in a configure script. Therefore, it's probably
>simpler to add an unlisten() syscall.

I would probably go for listen(-1) instead. Adding a syscall
is a major pita, and if you do unlisten(2) on a socket, does
that mean you can connect(2) it then ?

listen(-1) should be trivial to test for in autoconf.

>On systems which lack unlisten(), a suspended server can simply reply
>to any incoming connections with 503 Service Unavailable, without even
>bothering to look at the request.

But that sort of defeats the entire purpose behind suspend/resume:
to let the load-balancer front-end discover that we're not answering.

I would say:

if system offers listen(-1):
suspend / resume works as advertised.
else
suspend is not implemented
resume works only once.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

suspend/resume in trouble... [ In reply to ]

Mar 23, 2006, 4:51 AM

Post #6 of 10 (2458 views)

In message <ujrirq5xud9.fsf at cat.linpro.no>, Dag-Erling =?iso-8859-1?Q?Sm=F8rgra
v?= writes:

>However, there's a significant chance this may break existing
>applications which expect listen(0) to be equivalent to
>listen(SOME_IMPLEMENTATION_DEFINED_NON_ZERO_VALUE). It's also very
>hard to test for in a configure script. Therefore, it's probably
>simpler to add an unlisten() syscall.

I would probably go for listen(-1) instead. Adding a syscall
is a major pita, and if you do unlisten(2) on a socket, does
that mean you can connect(2) it then ?

listen(-1) should be trivial to test for in autoconf.

>On systems which lack unlisten(), a suspended server can simply reply
>to any incoming connections with 503 Service Unavailable, without even
>bothering to look at the request.

But that sort of defeats the entire purpose behind suspend/resume:
to let the load-balancer front-end discover that we're not answering.

I would say:

if system offers listen(-1):
suspend / resume works as advertised.
else
suspend is not implemented
resume works only once.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

suspend/resume in trouble... [ In reply to ]

Mar 23, 2006, 5:23 AM

Post #7 of 10 (2469 views)

"Poul-Henning Kamp" <phk at phk.freebsd.dk> writes:
> I would probably go for listen(-1) instead.

No, POSIX requires listen(-1) to be equivalent to listen(0).

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no

suspend/resume in trouble... [ In reply to ]

Mar 23, 2006, 5:23 AM

Post #8 of 10 (2459 views)

"Poul-Henning Kamp" <phk at phk.freebsd.dk> writes:
> I would probably go for listen(-1) instead.

No, POSIX requires listen(-1) to be equivalent to listen(0).

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no

suspend/resume in trouble... [ In reply to ]

Mar 23, 2006, 5:34 AM

Post #9 of 10 (2469 views)

In message <ujrek0txscf.fsf at cat.linpro.no>, Dag-Erling =?iso-8859-1?Q?Sm=F8rgra
v?= writes:
>"Poul-Henning Kamp" <phk at phk.freebsd.dk> writes:
>> I would probably go for listen(-1) instead.
>
>No, POSIX requires listen(-1) to be equivalent to listen(0).

But it also allows listen(0) to have the behaviour we seek (implicit
in the "may allow the socket to accept connections", so I'm sure I could
wiggle myself out of that :-)

Either way, lets park this issue on the to-do list for now.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

suspend/resume in trouble... [ In reply to ]

Mar 23, 2006, 5:34 AM

Post #10 of 10 (2470 views)

In message <ujrek0txscf.fsf at cat.linpro.no>, Dag-Erling =?iso-8859-1?Q?Sm=F8rgra
v?= writes:
>"Poul-Henning Kamp" <phk at phk.freebsd.dk> writes:
>> I would probably go for listen(-1) instead.
>
>No, POSIX requires listen(-1) to be equivalent to listen(0).

But it also allows listen(0) to have the behaviour we seek (implicit
in the "may allow the socket to accept connections", so I'm sure I could
wiggle myself out of that :-)

Either way, lets park this issue on the to-do list for now.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.