Mailing List Archive

About load balancing for HTML
Dear Sir,

Hello !

I have been using mod_backhand with apache 1.3.20 on Solaris9 x86. In
many case , our purpose has been attained. But when I checked the
simple HTML contents , it did not do the load balancing for HTML
contents.

Why ?

I have set the directive as folllows.

Backhand byAge
Backhand byLoad
Backhand bySession FSESSIONID=

We have tryed to get debugging log. Log is as follows.

---------------------------------------------------------------------------------------
[Wed Mar 24 00:08:41 2004] [error] mod_backhand: Check 6 -- headers(6)
[Wed Mar 24 00:08:41 2004] [error] mod_backhand: Check 7 (Chunked) (Keep-Alive)
[Wed Mar 24 00:08:41 2004] [error] mod_backhand: Check 8
[Wed Mar 24 00:08:41 2004] [error] mod_backhand: (29897) Sending back fd(12).
[Wed Mar 24 00:08:41 2004] [error] mod_backhand: Request tally 1666 [33]
[Wed Mar 24 00:08:41 2004] [notice] (Translate <=) = { /test1.html, (null), (null) }
[Wed Mar 24 00:08:41 2004] [notice] (Translate =>) = { /test1.html, (null), (null) }
[Wed Mar 24 00:08:41 2004] [notice] (Postread =>) = { DECLINED }
[Wed Mar 24 00:08:41 2004] [notice] (Translate <=) = { /test1.html, (null), (null) }
[Wed Mar 24 00:08:41 2004] [notice] (Translate =>) = { /test1.html, (null), (null) }
[Wed Mar 24 00:08:41 2004] [notice] (Type <=) = { cgi-script, /cgi-bin/test.cgi, (null) }
[Wed Mar 24 00:08:41 2004] [notice] Func executed for (null) [byAge(NULL)] (2 -> 2)
[Wed Mar 24 00:08:41 2004] [notice] New server list: [ 0 1 ]
[Wed Mar 24 00:08:41 2004] [notice] Func executed for (null) [byRandom(NULL)] (2 -> 2)
[Wed Mar 24 00:08:41 2004] [notice] New server list: [ 1 0 ]
[Wed Mar 24 00:08:41 2004] [notice] Func executed for (null) [byLogWindow(NULL)] (2 -> 1)
[Wed Mar 24 00:08:41 2004] [notice] New server list: [ 1 ]
[Wed Mar 24 00:08:41 2004] [notice] Func executed for (null) [byLoad(NULL)] (1 -> 1)
[Wed Mar 24 00:08:41 2004] [notice] New server list: [ 1 ]
[Wed Mar 24 00:08:41 2004] [notice] All funcs executed -> web.flight.co.jp
[Wed Mar 24 00:08:41 2004] [notice] (Type =>) = *{ backhand-redirection, backhand:/cgi-
bin/test.cgi, backhand/redirect }
[Wed Mar 24 00:08:41 2004] [notice] (Postread =>) = { Backhanding to 1 }
[Wed Mar 24 00:08:41 2004] [notice] (Postread =>) = { backhand:/cgi-bin/test.cgi, backhand-
redirection, (null) }
[Wed Mar 24 00:08:41 2004] [notice] (Translate <=) = { backhand:/cgi-bin/test.cgi, backhand-
redirection, (null) }
[Wed Mar 24 00:08:41 2004] [notice] (Type <=) = { backhand-redirection, /cgi-bin/test.cgi, (null)
}
[Wed Mar 24 00:08:41 2004] [notice] (Type =>) = *{ backhand-redirection, /cgi-bin/test.cgi,
(null) }
[Wed Mar 24 00:08:41 2004] [error] mod_backhand: Received fd(12).
[Wed Mar 24 00:08:41 2004] [error] mod_backhand: Check 1
[Wed Mar 24 00:08:41 2004] [error] mod_backhand: Check 2
[Wed Mar 24 00:08:41 2004] [error] mod_backhand: Check 3 --
---------------------------------------------------------------------------------------

I have two worrisome points.

One is error messages.
Another is the 'DECLINED' message.

When HTML contents come , mod_backhand seems to give up balancing.
And when cgi contents come , mod_backhand seems to try to do balancing.

The log which was recorded with only HTML contents is as follows.
---------------------------------------------------------------------------------------
[Wed Mar 24 00:11:29 2004] [notice] (Translate <=) = { /test1.html, (null), (null) }
[Wed Mar 24 00:11:29 2004] [notice] (Translate =>) = { /test1.html, (null), (null) }
[Wed Mar 24 00:11:29 2004] [notice] (Postread =>) = { DECLINED }
[Wed Mar 24 00:11:29 2004] [notice] (Translate <=) = { /test1.html, (null), (null) }
[Wed Mar 24 00:11:29 2004] [notice] (Translate =>) = { /test1.html, (null), (null) }
[Wed Mar 24 00:11:29 2004] [notice] (Translate <=) = { /test1.html, (null), (null) }
[Wed Mar 24 00:11:29 2004] [notice] (Translate =>) = { /test1.html, (null), (null) }
[Wed Mar 24 00:11:29 2004] [notice] (Postread =>) = { DECLINED }
[Wed Mar 24 00:11:29 2004] [notice] (Translate <=) = { /test1.html, (null), (null) }
[Wed Mar 24 00:11:29 2004] [notice] (Translate =>) = { /test1.html, (null), (null) }
[Wed Mar 24 00:11:29 2004] [notice] (Translate <=) = { /test1.html, (null), (null) }
[Wed Mar 24 00:11:29 2004] [notice] (Translate =>) = { /test1.html, (null), (null) }
---------------------------------------------------------------------------------------

Why mod_backhand gives up to do load balancing ? We have given heavy
enough load to one server.

Directive is same about HTML and cgi.

## for HTML ##
<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>
<Directory "/opt/FLIGHTweb/web_service/web_root/htdocs">
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Order allow,deny
Allow from all
Backhand byAge
Backhand byLoad
Backhand bySession FSESSIONID=
</Directory>

## for cgi ##
<IfModule mod_alias.c>
Alias /icons/ "/opt/FLIGHTapps/httpd/icons/"
<Directory "/opt/FLIGHTapps/httpd/icons">
Options Indexes MultiViews
AllowOverride None
Order allow,deny
Allow from all
</Directory>
ScriptAlias /cgi-bin/ "/opt/FLIGHTweb/web_service/web_root/cgi-bin/"
<Directory "/opt/FLIGHTweb/web_service/web_root/cgi-bin">
AllowOverride None
Options None
Order allow,deny
Allow from all
Backhand byAge
Backhand byLoad
Backhand bySession FSESSIONID=
</Directory>
</IfModule>


Best Regards,
Keiichiro Katayama

-------------------------------------------------------------
// Moved to the new office ! //
Keiichiro Katayama
< FLIGHT SYSTEM CONSULTING Inc. >
E-mail : katayama@flight.co.jp
Home Page : http://www.flight.co.jp/
Addr : 3F Ebisu MF BLdg. 4-6-1 , Ebisu , Shibuya-ku , Tokyo JAPAN 150-0013
TEL 81-3-3440-6100 / FAX 81-3-5791-2241
-------------------------------------------------------------
About load balancing for HTML [ In reply to ]
Can you send your _whole_ httpd.conf file? The parts you posted look
good, perhaps something else is odd.

> When HTML contents come , mod_backhand seems to give up balancing.
> And when cgi contents come , mod_backhand seems to try to do balancing.
>
> Why mod_backhand gives up to do load balancing ? We have given heavy
> enough load to one server.

// Theo Schlossnagle
// Principal Engineer -- http://www.omniti.com/~jesus/
// Postal Engine -- http://www.postalengine.com/
// Ecelerity: fastest MTA on Earth
About load balancing for HTML [ In reply to ]
Dear Theo,

Thank you for the quick response.

Theo Schlossnagle worte it! :
>Can you send your _whole_ httpd.conf file? The parts you posted look
>good, perhaps something else is odd.

I'm sorry to write long e-mail for conf file , but I write the conf as
follows.

----------------------------------------------------------------------
ServerType standalone
ServerRoot "/opt/FLIGHTapps/httpd"
PidFile /opt/FLIGHTapps/httpd/logs/httpd.pid
ScoreBoardFile /opt/FLIGHTapps/httpd/logs/httpd.scoreboard
Timeout 300
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 15
MinSpareServers 5
MaxSpareServers 10
StartServers 5
MaxClients 150
MaxRequestsPerChild 0
User flight
Group staff
LoadModule backhand_module libexec/mod_backhand.so
LoadModule vhost_alias_module libexec/mod_vhost_alias.so
LoadModule status_module libexec/mod_status.so
LoadModule info_module libexec/mod_info.so
LoadModule includes_module libexec/mod_include.so
LoadModule rewrite_module libexec/mod_rewrite.so
ClearModuleList
AddModule mod_backhand.c
AddModule mod_vhost_alias.c
AddModule mod_env.c
AddModule mod_log_config.c
AddModule mod_mime.c
AddModule mod_negotiation.c
AddModule mod_status.c
AddModule mod_info.c
AddModule mod_include.c
#AddModule mod_autoindex.c
AddModule mod_dir.c
AddModule mod_cgi.c
AddModule mod_asis.c
AddModule mod_imap.c
AddModule mod_actions.c
AddModule mod_userdir.c
AddModule mod_alias.c
AddModule mod_rewrite.c
AddModule mod_access.c
AddModule mod_auth.c
AddModule mod_so.c
AddModule mod_setenvif.c
Port 80
ServerAdmin cluster@flight.co.jp
ServerName web.flight.co.jp
DocumentRoot "/opt/FLIGHTweb/web_service/web_root/htdocs"
<Directory />
Options FollowSymLinks
AllowOverride None
Backhand byLoad
</Directory>
<Directory "/opt/FLIGHTweb/web_service/web_root/htdocs">
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Order allow,deny
Allow from all
Backhand byAge
Backhand byLoad
Backhand bySession FSESSIONID=
</Directory>
<Directory "/opt/FLIGHTweb/web_service/web_root/htdocs">
AllowOverride None
Options None
Order allow,deny
Allow from all
Backhand byAge
Backhand byRandom
Backhand byLogWindow
Backhand byLoad
</Directory>
<IfModule mod_userdir.c>
UserDir public_html
</IfModule>
<IfModule mod_dir.c>
DirectoryIndex index.html index.htm Index.html Index.htm
</IfModule>
AccessFileName .htaccess
<Files ~ "^\.ht">
Order allow,deny
Deny from all
</Files>
UseCanonicalName On
<IfModule mod_mime.c>
TypesConfig /opt/FLIGHTapps/httpd/conf/mime.types
</IfModule>
DefaultType text/plain
<IfModule mod_mime_magic.c>
MIMEMagicFile /opt/FLIGHTapps/httpd/conf/magic
</IfModule>
HostnameLookups Off
<IfModule mod_backhand.c>
UnixSocketDir /opt/FLIGHTapps/httpd/backhand
MulticastStats 172.16.200.255:4445
AcceptStats 172.16.200.0/24
BackhandLogLevel +netall
BackhandLogLevel +mbcsall
BackhandLogLevel +dcsnall
<Location "/LoadBalanceMonitor/">
SetHandler backhand-handler
</Location>
</IfModule>
ErrorLog "|/opt/FLIGHTapps/bin/flogrotate /opt/FLIGHTweb/web_service/logs/error_log 10M 5"
LogLevel warn
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent
CustomLog "|/opt/FLIGHTapps/bin/flogrotate /opt/FLIGHTweb/web_service/logs/access_log 10M 5" common
ServerSignature On
<IfModule mod_alias.c>
Alias /icons/ "/opt/FLIGHTapps/httpd/icons/"
<Directory "/opt/FLIGHTapps/httpd/icons">
Options Indexes MultiViews
AllowOverride None
Order allow,deny
Allow from all
</Directory>
ScriptAlias /cgi-bin/ "/opt/FLIGHTweb/web_service/web_root/cgi-bin/"
<Directory "/opt/FLIGHTweb/web_service/web_root/cgi-bin">
AllowOverride None
Options None
Order allow,deny
Allow from all
Backhand byAge
Backhand byLoad
Backhand bySession FSESSIONID=
</Directory>
<Directory "/opt/FLIGHTweb/web_service/web_root/cgi-bin">
AllowOverride None
Options None
Order allow,deny
Allow from all
Backhand byAge
Backhand byRandom
Backhand byLogWindow
Backhand byLoad
</Directory>
</IfModule>
<IfModule mod_autoindex.c>
IndexOptions FancyIndexing
AddIconByEncoding (CMP,/icons/compressed.gif) x-compress x-gzip
AddIconByType (TXT,/icons/text.gif) text/*
AddIconByType (IMG,/icons/image2.gif) image/*
AddIconByType (SND,/icons/sound2.gif) audio/*
AddIconByType (VID,/icons/movie.gif) video/*
AddIcon /icons/binary.gif .bin .exe
AddIcon /icons/binhex.gif .hqx
AddIcon /icons/tar.gif .tar
AddIcon /icons/world2.gif .wrl .wrl.gz .vrml .vrm .iv
AddIcon /icons/compressed.gif .Z .z .tgz .gz .zip
AddIcon /icons/a.gif .ps .ai .eps
AddIcon /icons/layout.gif .html .shtml .htm .pdf
AddIcon /icons/text.gif .txt
AddIcon /icons/c.gif .c
AddIcon /icons/p.gif .pl .py
AddIcon /icons/f.gif .for
AddIcon /icons/dvi.gif .dvi
AddIcon /icons/uuencoded.gif .uu
AddIcon /icons/script.gif .conf .sh .shar .csh .ksh .tcl
AddIcon /icons/tex.gif .tex
AddIcon /icons/bomb.gif core
AddIcon /icons/back.gif ..
AddIcon /icons/hand.right.gif README
AddIcon /icons/folder.gif ^^DIRECTORY^^
AddIcon /icons/blank.gif ^^BLANKICON^^
DefaultIcon /icons/unknown.gif
ReadmeName README
HeaderName HEADER
IndexIgnore .??* *~ *# HEADER* README* RCS CVS *,v *,t
</IfModule>
<IfModule mod_mime.c>
AddEncoding x-compress Z
AddEncoding x-gzip gz tgz
AddLanguage da .dk
AddLanguage nl .nl
AddLanguage en .en
AddLanguage et .ee
AddLanguage fr .fr
AddLanguage de .de
AddLanguage el .el
AddLanguage he .he
AddCharset ISO-8859-8 .iso8859-8
AddLanguage it .it
AddLanguage ja .ja
AddCharset ISO-2022-JP .jis
AddLanguage kr .kr
AddCharset ISO-2022-KR .iso-kr
AddLanguage no .no
AddLanguage pl .po
AddCharset ISO-8859-2 .iso-pl
AddLanguage pt .pt
AddLanguage pt-br .pt-br
AddLanguage ltz .lu
AddLanguage ca .ca
AddLanguage es .es
AddLanguage sv .se
AddLanguage cz .cz
AddLanguage ru .ru
AddLanguage zh-tw .tw
AddLanguage tw .tw
AddCharset Big5 .Big5 .big5
AddCharset WINDOWS-1251 .cp-1251
AddCharset CP866 .cp866
AddCharset ISO-8859-5 .iso-ru
AddCharset KOI8-R .koi8-r
AddCharset UCS-2 .ucs2
AddCharset UCS-4 .ucs4
AddCharset UTF-8 .utf8
<IfModule mod_negotiation.c>
LanguagePriority en da nl et fr de el it ja kr no pl pt pt-br ru ltz ca es sv tw
</IfModule>
AddType application/x-tar .tgz
AddType text/html .shtml .cgi
AddHandler server-parsed .shtml .cgi
</IfModule>
<IfModule mod_setenvif.c>
BrowserMatch "Mozilla/2" nokeepalive
BrowserMatch "MSIE 4\.0b2;" nokeepalive downgrade-1.0 force-response-1.0
BrowserMatch "RealPlayer 4\.0" force-response-1.0
BrowserMatch "Java/1\.0" force-response-1.0
BrowserMatch "JDK/1\.0" force-response-1.0
</IfModule>
<Location /server-status>
SetHandler server-status
Order deny,allow
Deny from all
Allow from .your_domain.com
</Location>
<Location /server-info>
SetHandler server-info
Order deny,allow
Deny from all
Allow from .your_domain.com
</Location>

include /opt/FLIGHTapps/catalina/conf/mod_jk.conf
----------------------------------------------------------------------

-------------------------------------------------------------
// Moved to the new office ! //
Keiichiro Katayama
< FLIGHT SYSTEM CONSULTING Inc. >
E-mail : katayama@flight.co.jp
Home Page : http://www.flight.co.jp/
Addr : 3F Ebisu MF BLdg. 4-6-1 , Ebisu , Shibuya-ku , Tokyo JAPAN 150-0013
TEL 81-3-3440-6100 / FAX 81-3-5791-2241
-------------------------------------------------------------
About load balancing for HTML [ In reply to ]
On Apr 6, 2004, at 10:50 PM, Keiichiro Katayama wrote:
> <Directory "/opt/FLIGHTweb/web_service/web_root/htdocs">
> Options Indexes FollowSymLinks MultiViews
> AllowOverride None
> Order allow,deny
> Allow from all
> Backhand byAge
> Backhand byLoad
> Backhand bySession FSESSIONID=
> </Directory>
> <Directory "/opt/FLIGHTweb/web_service/web_root/htdocs">
> AllowOverride None
> Options None
> Order allow,deny
> Allow from all
> Backhand byAge
> Backhand byRandom
> Backhand byLogWindow
> Backhand byLoad
> </Directory>

You have two Directory stanzas with the same dir name... what are you
trying to accomplish? This could trigger a problem in the way that
mod_backhand will try to merge the declarations. The above is
confusing though.

> ScriptAlias /cgi-bin/
> "/opt/FLIGHTweb/web_service/web_root/cgi-bin/"
> <Directory "/opt/FLIGHTweb/web_service/web_root/cgi-bin">
> AllowOverride None
> Options None
> Order allow,deny
> Allow from all
> Backhand byAge
> Backhand byLoad
> Backhand bySession FSESSIONID=
> </Directory>
> <Directory "/opt/FLIGHTweb/web_service/web_root/cgi-bin">
> AllowOverride None
> Options None
> Order allow,deny
> Allow from all
> Backhand byAge
> Backhand byRandom
> Backhand byLogWindow
> Backhand byLoad
> </Directory>

Again a double directory clause.

> include /opt/FLIGHTapps/catalina/conf/mod_jk.conf

Hmm... what's in there? If you are using rewrite rules, unexpected
things can happen sometimes.

After removing the dupped <Directory> clauses in your conf, if it still
doesn't work, try moding the AddModule mod_backhand.c at the _end_ of
the AddModule list or somewhere in the middle. I know that many people
experience module ordering pains when trying to get mod_backhand and
mod_gzip working together -- perhaps you are seeing a similar effect.

// Theo Schlossnagle
// Principal Engineer -- http://www.omniti.com/~jesus/
// Postal Engine -- http://www.postalengine.com/
// Ecelerity: fastest MTA on Earth
About load balancing for HTML [ In reply to ]
Dear Theo,

Thank you again!

Theo Schlossnagle worte it! :
>You have two Directory stanzas with the same dir name... what are you
>trying to accomplish? This could trigger a problem in the way that
>mod_backhand will try to merge the declarations. The above is
>confusing though.
>Again a double directory clause.

Thank you for pointing out! I'll try test after the modification of
these faults

>> include /opt/FLIGHTapps/catalina/conf/mod_jk.conf
>
>Hmm... what's in there? If you are using rewrite rules, unexpected
>things can happen sometimes.

This directive is the forwarding module for servlet. We set some URL
which is wanted to use servlet on the mod_jk.conf file , so when
apache receives the request for that URL , mod_jk forwards the request
to the servlet engine 'Tomcat'.

>After removing the dupped <Directory> clauses in your conf, if it still
>doesn't work, try moding the AddModule mod_backhand.c at the _end_ of
>the AddModule list or somewhere in the middle. I know that many people
>experience module ordering pains when trying to get mod_backhand and
>mod_gzip working together -- perhaps you are seeing a similar effect.

Thank you for your advice. I'll re-report you after our trial.


Best Regards,
Keiichiro Katayama

-------------------------------------------------------------
// Moved to the new office ! //
Keiichiro Katayama
< FLIGHT SYSTEM CONSULTING Inc. >
E-mail : katayama@flight.co.jp
Home Page : http://www.flight.co.jp/
Addr : 3F Ebisu MF BLdg. 4-6-1 , Ebisu , Shibuya-ku , Tokyo JAPAN 150-0013
TEL 81-3-3440-6100 / FAX 81-3-5791-2241
-------------------------------------------------------------
About load balancing for HTML [ In reply to ]
Dear Theo,

I tested without mod_jk and mod_rewrite. These two modules intercept
the HTML , so I suspected these modules. But situation did not
change. CGI can balance easily as usual , but HTML can not balance.

What situation will cause the conclusion 'DECLINED' for HTML ?
---------------------------------------------------------------------------------------
[Wed Mar 24 00:08:41 2004] [notice] (Translate <=) = { /test1.html, (null), (null) }
[Wed Mar 24 00:08:41 2004] [notice] (Translate =>) = { /test1.html, (null), (null) }
[Wed Mar 24 00:08:41 2004] [notice] (Postread =>) = { DECLINED }
---------------------------------------------------------------------------------------


-------------------------------------------------------------
// Moved to the new office ! //
Keiichiro Katayama
< FLIGHT SYSTEM CONSULTING Inc. >
E-mail : katayama@flight.co.jp
Home Page : http://www.flight.co.jp/
Addr : 3F Ebisu MF BLdg. 4-6-1 , Ebisu , Shibuya-ku , Tokyo JAPAN 150-0013
TEL 81-3-3440-6100 / FAX 81-3-5791-2241
-------------------------------------------------------------
About load balancing for HTML [ In reply to ]
Keiichiro Katayama wrote:

>Dear Theo,
>
>I tested without mod_jk and mod_rewrite. These two modules intercept
>the HTML , so I suspected these modules. But situation did not
>change. CGI can balance easily as usual , but HTML can not balance.
>
>What situation will cause the conclusion 'DECLINED' for HTML ?
>
>
It happens when mod_backhand looks at the request and decides that it
cannot balance it. As in it doesn't the URI for the request doesn't
trigger any of the Backhand directives.

mod_proxy internally changes the uri, so I assume that mod_jk may do
something similar.

Your mod_jk setup, does it use Location, Files or Directory? Can you
put the Backhand declarations in there?

--
// Theo Schlossnagle
// Principal Engineer -- http://www.omniti.com/~jesus/
// Postal Engine -- http://www.postalengine.com/
// Ecelerity: fastest MTA on Earth
About load balancing for HTML [ In reply to ]
Dear Theo,

Thank you for your repeated quick answers.

Theo Schlossnagle worte it! :
>It happens when mod_backhand looks at the request and decides that it
>cannot balance it. As in it doesn't the URI for the request doesn't
>trigger any of the Backhand directives.
>mod_proxy internally changes the uri, so I assume that mod_jk may do
>something similar.
>Your mod_jk setup, does it use Location, Files or Directory? Can you
>put the Backhand declarations in there?

Now we excepted mod_jk , but situation did not change.
So mod_jk is not criminal , I guess.

We have set the modules as follows. It seems look correct.

--------------------------
LoadModule
mod_backhand
mod_vhost_alias
mod_status
mod_info
mod_include
ClearModuleList
AddModule
mod_backhand
mod_vhost_alias
mod_env
mod_log_config
mod__mime
mod_negotiation
mod_status
mod_info
mod_autoindex
mod_dir
mod_cgi
mod_asis
mod_imap
mod_actions
mod_userdir
mod_alias
mod_auth
mod_so
mod_setenvif
--------------------------

-------------------------------------------------------------
// Moved to the new office ! //
Keiichiro Katayama
< FLIGHT SYSTEM CONSULTING Inc. >
E-mail : katayama@flight.co.jp
Home Page : http://www.flight.co.jp/
Addr : 3F Ebisu MF BLdg. 4-6-1 , Ebisu , Shibuya-ku , Tokyo JAPAN 150-0013
TEL 81-3-3440-6100 / FAX 81-3-5791-2241
-------------------------------------------------------------