Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. mod_backhand>
  3. users
mod_backhandunresponsive, then failure to requests
lister at primetime
Feb 20, 2005, 4:20 PM
Post #1 of 6 (15246 views)
Permalink
Configuration :
10.200.1.111 and 10.200.1.112
OS : 4.11-STABLE FreeBSD
packages (all source compiled from ports) :
apache+mod_ssl-1.3.33+2.8.22
mod_backhand-1.2.2
mod_bandwidth-2.0.5
mod_dav-1.0.3_2
mod_php4-4.3.10_2,1

Node 1 httpd.conf :
<IfModule mod_backhand.c>
UnixSocketDir /var/backhand/backhand
MulticastStats 10.200.1.111 10.200.1.255:4445,1
AcceptStats 10.200.1.0/24
</IfModule>

<Location "/backhand/">
SetHandler backhand-handler
</Location>

<Directory /usr/local/www/data/www.lauraandamber.com>
Backhand removeSelf
Backhand byBusyChildren
</Directory>

Node 2 httpd/conf :
<IfModule mod_backhand.c>
UnixSocketDir /var/backhand/backhand
MulticastStats 10.200.1.112 10.200.1.255:4445,1
AcceptStats 10.200.1.0/24
</IfModule>

<Location "/backhand/">
SetHandler backhand-handler
</Location>
Both with and with out this :
<Directory /usr/local/www/data/www.lauraandamber.com>
Backhand addSelf
</Directory>

I (re)start apache and see
[Sun Feb 20 17:10:17 2005] [notice] SIGUSR1 received. Doing graceful
restart
[Sun Feb 20 17:10:17 2005] [notice] mod_backhand -- UnixSocketDir set to
/var/backhand/backhand
[Sun Feb 20 17:10:17 2005] [notice] mod_backhand -- Broadcast
10.200.1.255:4445 added
[Sun Feb 20 17:10:17 2005] [notice] mod_backhand -- Multicast accept
10.200.1.0/24
[Sun Feb 20 17:10:17 2005] [notice] backhand_init(4859) spawning
moderator (PID 4910)
[Sun Feb 20 17:10:17 2005] [notice] mod_backhand moderator ready to go
[Sun Feb 20 17:10:18 2005] [notice] Apache/1.3.33 (Unix) PHP/4.3.10
mod_backhand/1.2.2 DAV/1.0.3 configured -- resuming normal operations
[Sun Feb 20 17:10:18 2005] [notice] Accept mutex: flock (Default: flock)

After a bunch of refreshes normal (I see the pages in each system's
access log) to the page :
http://bookpc1/www.lauraandamber.com/
It stops responding, and I see (something like) this :

> netstat
tcp4 0 0 10.200.1.111.4179 10.200.1.112.http SYN_SENT
tcp4 0 0 10.200.1.111.3412 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.1875 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.4199 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.2039 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.3527 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.4922 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.4064 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.3785 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.4986 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.3011 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.1702 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.2660 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.3068 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.4948 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.2797 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.3074 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.4851 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.4449 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.3046 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.4777 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.http 10.200.1.90.4496 TIME_WAIT
tcp4 0 0 10.200.1.111.4098 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.3982 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.1582 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.1488 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.1329 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.http 10.200.1.90.1974 TIME_WAIT
tcp4 0 0 10.200.1.111.4357 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.2882 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.2489 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.3332 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.4193 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.3413 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.3676 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.http 10.200.1.90.1715 TIME_WAIT
tcp4 0 0 10.200.1.111.http 10.200.1.90.1416 TIME_WAIT
tcp4 0 0 10.200.1.111.4907 10.200.1.112.http TIME_WAIT
tcp4 0 0 10.200.1.111.3817 10.200.1.112.http TIME_WAIT

I see nothing in the back end's (bookpc2, .112) logs, but in the front
end's logs 9bookpc1, .111) I start to (slowly) see these :
[Sun Feb 20 16:29:55 2005] [warn] (4)Interrupted system call: connect()
timed out

Eventually, I see this :
[Sun Feb 20 14:18:09 2005] [error] [client 10.200.1.90] Request exceeded
the limit of 20 internal redirects due to probable configuration error.
Use 'LimitInternalRecursion' to increase the limit if necessary. Use
'LogLevel debug' to get a backtrace.

And / or this :
[Sun Feb 20 16:53:12 2005] [error] mod_backhand: child 4734 hung talking
to moderator. Die!
And
[Sun Feb 20 17:06:41 2005] [error] (61)Connection refused: Child 4874
failed to establish umbilical to moderator!
And
[Sun Feb 20 17:08:09 2005] [notice] child pid 4860 exit signal Broken
pipe (13)
And
message format error

This list / package appears to be have relatively low volume of
chatter. This could either mean 'really stable' or 'dead project'.
I asked on #apache, and people said it was 'in use, but not being
developed (for)'.
I would love to use this to load balance for some XML-RPC scripts
I am developing, but it has to berock solid for this. If this is not the
tool, is there something that people are using now?

TIA


_______________________________________________
backhand-users mailing list
backhand-users@lists.backhand.org
http://lists.backhand.org/mailman/listinfo/backhand-users
Re: mod_backhandunresponsive, then failure to requests [ In reply to ]
jesus at omniti
Feb 20, 2005, 6:21 PM
Post #2 of 6 (14574 views)
Permalink
Turn of connection pooling and see if that helps your situation.

BackhandConnectionPools off

On Feb 20, 2005, at 8:16 PM, Lister wrote:
> I see nothing in the back end's (bookpc2, .112) logs, but in the front
> end's logs 9bookpc1, .111) I start to (slowly) see these :
> [Sun Feb 20 16:29:55 2005] [warn] (4)Interrupted system call:
> connect() timed out
>
> Eventually, I see this :
> [Sun Feb 20 14:18:09 2005] [error] [client 10.200.1.90] Request
> exceeded the limit of 20 internal redirects due to probable
> configuration error. Use 'LimitInternalRecursion' to increase the
> limit if necessary. Use 'LogLevel debug' to get a backtrace.

// Theo Schlossnagle
// Principal Engineer -- http://www.omniti.com/~jesus/
// OmniTI Computer Consulting, Inc. -- http://www.omniti.com/
// Ecelerity: fastest MTA on Earth


_______________________________________________
backhand-users mailing list
backhand-users@lists.backhand.org
http://lists.backhand.org/mailman/listinfo/backhand-users
Re: mod_backhandunresponsive, then failure to requests [ In reply to ]
lister at primetime
Feb 20, 2005, 8:20 PM
Post #3 of 6 (14583 views)
Permalink
Theo Schlossnagle wrote:

> Turn of connection pooling and see if that helps your situation.
>
> BackhandConnectionPools off

No change.

I also tried bumping up the kernel params :
kern.ipc.nmbclusters=16384
kern.ipc.maxsockets=16384
I think it may have pushed the envelope, but didn't fix anything.

>
> On Feb 20, 2005, at 8:16 PM, Lister wrote:
>
>> I see nothing in the back end's (bookpc2, .112) logs, but in the front
>> end's logs 9bookpc1, .111) I start to (slowly) see these :
>> [Sun Feb 20 16:29:55 2005] [warn] (4)Interrupted system call:
>> connect() timed out
>>
>> Eventually, I see this :
>> [Sun Feb 20 14:18:09 2005] [error] [client 10.200.1.90] Request
>> exceeded the limit of 20 internal redirects due to probable
>> configuration error. Use 'LimitInternalRecursion' to increase the
>> limit if necessary. Use 'LogLevel debug' to get a backtrace.
>
>
> // Theo Schlossnagle
> // Principal Engineer -- http://www.omniti.com/~jesus/
> // OmniTI Computer Consulting, Inc. -- http://www.omniti.com/
> // Ecelerity: fastest MTA on Earth
>
>
> _______________________________________________
> backhand-users mailing list
> backhand-users@lists.backhand.org
> http://lists.backhand.org/mailman/listinfo/backhand-users



_______________________________________________
backhand-users mailing list
backhand-users@lists.backhand.org
http://lists.backhand.org/mailman/listinfo/backhand-users
Re: mod_backhandunresponsive, then failure to requests [ In reply to ]
lister at primetime
Feb 21, 2005, 6:20 AM
Post #4 of 6 (14562 views)
Permalink
Theo Schlossnagle wrote:

> Turn of connection pooling and see if that helps your situation.
>
> BackhandConnectionPools off


No, wait, I had it in the <Directory> </Directory> not I put it
in the <IfModule mod_backhand.c> </IfModule> and it seems
better. Thanks.
Now it just runs really slow, on a 10 mbit lan, 3 minutes for a
109 kB.

_______________________________________________
backhand-users mailing list
backhand-users@lists.backhand.org
http://lists.backhand.org/mailman/listinfo/backhand-users
Re: mod_backhandunresponsive, then failure to requests [ In reply to ]
jesus at omniti
Feb 21, 2005, 6:23 AM
Post #5 of 6 (14580 views)
Permalink
Lister wrote:

> Theo Schlossnagle wrote:
>
>> Turn of connection pooling and see if that helps your situation.
>>
>> BackhandConnectionPools off
>
>
>
> No, wait, I had it in the <Directory> </Directory> not I put it
> in the <IfModule mod_backhand.c> </IfModule> and it seems
> better. Thanks.
> Now it just runs really slow, on a 10 mbit lan, 3 minutes for a
> 109 kB.

Sounds like a networking problem to me. Once mod_backhand does the
proxying, it is dirt simple:

while(read from backend) { write to client }.

--
// Theo Schlossnagle
// Principal Engineer -- http://www.omniti.com/~jesus/
// Postal Engine -- http://www.postalengine.com/
// Ecelerity: fastest MTA on Earth


_______________________________________________
backhand-users mailing list
backhand-users@lists.backhand.org
http://lists.backhand.org/mailman/listinfo/backhand-users
Re: mod_backhandunresponsive, then failure to requests [ In reply to ]
lister at primetime
Feb 21, 2005, 8:20 AM
Post #6 of 6 (14589 views)
Permalink
Theo Schlossnagle wrote:

> Lister wrote:
>
>> Theo Schlossnagle wrote:
>>
>>> Turn of connection pooling and see if that helps your situation.
>>>
>>> BackhandConnectionPools off
>>
>>
>>
>>
>> No, wait, I had it in the <Directory> </Directory> not I put it
>> in the <IfModule mod_backhand.c> </IfModule> and it seems
>> better. Thanks.
>> Now it just runs really slow, on a 10 mbit lan, 3 minutes for a
>> 109 kB.
>
>
> Sounds like a networking problem to me. Once mod_backhand does the
> proxying, it is dirt simple:
>
> while(read from backend) { write to client }.
>
Exactly right. Thanks.

_______________________________________________
backhand-users mailing list
backhand-users@lists.backhand.org
http://lists.backhand.org/mailman/listinfo/backhand-users

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal