Mailing List Archive: [mod_backhand-users] socket:: No buffer space available

[mod_backhand-users] socket:: No buffer space available

Nov 3, 2000, 8:06 AM

Post #1 of 3 (451 views)

Hi all.

I use mod_backhand for testing purpose.
apache 1.3.14, mod-rewrite 1.10, FreeBSD 4.1.x

Primary server accpet connection, mod_backhand work for <Dicrectory
/images/*>
As loading on the primary server always BIG, mod_backhand always redirect
request (over internal proxy) to secondary "image server".
I undestand, that may be mod_rewrite can serve much better this
configurations, but I want to test mod_backhand under heavy load.
First (primary) server work well all time, but secondary "image server"
after some time (approximately several hours) begin write to apache
error.log diagnostic:
connect:: No such file or directory

This string occured from 100 to 1000(0?) time, and after this in error.log
appears string :
socket:: No buffer space available

And after 1-10 hour error.log looks so:

socket:: No buffer space available
connect:: No such file or directory
socket:: No buffer space available
socket:: No buffer space available
connect:: No such file or directory
socket:: No buffer space available

And death final !

All network services blocked, but if You login over console, error.log
has only:
socket:: No buffer space available
socket:: No buffer space available
socket:: No buffer space available
socket:: No buffer space available
socket:: No buffer space available

Some kind of socket leak, imho.

Comment , pls.

b.r.
Kozin Maxim

[mod_backhand-users] socket:: No buffer space available [ In reply to ]

theos at cnds

Nov 6, 2000, 3:10 PM

Post #2 of 3 (428 views)

Permalink

I tried to replicate the problem, but I've only got one FreeBSD machine :-(.
So, no luck. I can see this happenning however. I am pretty sure thatt here
is not a file descriptor leak. I am pretty darn aggressive about closing file
descriptors.

The no buffer space available sounds like you are running our of mbufs. Since
I am not a BSD guru, I don't really know how to fix this, but I am absolutely
sure that there is a way (probably with sysctl).

After running our of mbufs, the connects are probably failing because the
processes (Child <-> backhand parent) cannot talk over their UNIX domain
sockets that live in the UnixSocketDir. This my guess as to what is going on.

The following could be blatant display of my ignorance of BSD, but you might
want to look at:

kern.maxfiles: 1064
kern.ipc.maxsockbuf: 262144
kern.ipc.nmbclusters: 1024
kern.ipc.mbuf_wait: 32
kern.ipc.nmbufs: 4096
kern.ipc.maxsockets: 1064

Those are the sysctl values I would imagine are pertinent on my system and I
would assume they are *WAY* to low! For perspectiv eon the amount of resource
usage mod_backhand will use, the following holds:
In addition to all of the fd's Apache will use normally, mod_backhand will
use up to 1000+(2 * # of children) more file descriptors. These are used for
the interprocess communication and the session pooling.

Maxim Kozin wrote:
> I use mod_backhand for testing purpose.
> apache 1.3.14, mod-rewrite 1.10, FreeBSD 4.1.x
>
> connect:: No such file or directory
> socket:: No buffer space available
> And death final !
>
> Some kind of socket leak, imho.

--
Theo Schlossnagle
1024D/A8EBCF8F/13BD 8C08 6BE2 629A 527E 2DC2 72C2 AD05 A8EB CF8F
2047R/33131B65/71 F7 95 64 49 76 5D BA 3D 90 B9 9F BE 27 24 E7

[mod_backhand-users] socket:: No buffer space available [ In reply to ]

madmax at express

Nov 10, 2000, 9:07 AM

Post #3 of 3 (426 views)

Permalink

On Mon, 6 Nov 2000, Theo E. Schlossnagle wrote:

> The no buffer space available sounds like you are running our of mbufs. Since
> I am not a BSD guru, I don't really know how to fix this, but I am absolutely
> sure that there is a way (probably with sysctl).
>
> After running our of mbufs, the connects are probably failing because the
> processes (Child <-> backhand parent) cannot talk over their UNIX domain
> sockets that live in the UnixSocketDir. This my guess as to what is going on.
>
> The following could be blatant display of my ignorance of BSD, but you might
> want to look at:
>
> kern.maxfiles: 1064
> kern.ipc.maxsockbuf: 262144
> kern.ipc.nmbclusters: 1024
> kern.ipc.mbuf_wait: 32
> kern.ipc.nmbufs: 4096
> kern.ipc.maxsockets: 1064

Hello.

On both mashines kernel parametrs about identical.
On first host "No buffer space available" never occured, while on second -
after 5-10 hours apahce+mod_nackhand work.
Moreover, on second host running much less process which can use mbufs -
exim, pop and httpd.

b.r.
Kozin Maxim