Sean Chittenden wrote:
> 1) For the resource collection/broadcast process, can't you give it
> a higher priority and make this artifact go away?
Yes. I have done this on some mod_perl driven sites and I know it has
been done by others on mod_php driven sites where the load on the
webservers is very high. The PID of the resource collector is printed
into the error_log, so you can get it from there and renice your process
to -19 or something like that (that is what I do).
I guess if you were feeling spiffy, you could add the one line of code
required for mod_backhand to do this itself (right before it drops
privs) in broadcast_my_stats in back_util.c. I may add this as an
option later, after I experiment with portaility issues. Of course, it
should only do this if you specify:
BackhandDaemonPriority <integer>
in the httpd.conf (which doesn't exist yet -- just an idea).
> 2) This brings up a great point of discussion that I've long
> wondered about. Proxying vs. backhanding everything. Let's say that the
> pages that you're generating are over 8KB in size (the kernel's network
> socket buffer size) and that my Apache processes are about 35MB in size.
> That being the case, I would think that you would want to:
>
> * use backhand on everything
> * make everything an app server
> * increase the network buffer size to something insane like 100KB
> and hope that your pages/images are smaller than 100KB
> * bank on the fact that after the Apache process writes to the
> socket (which should be able to buffer the entire page), that the apache
> process will be able to disconnect from the socket and go about serving more
> incoming requests
I would definitely try the above over the below. I know it is standard
practice to have a row of proxies and and row of apps (sometimes running
on the same machine)... But, that doesn't make it right, or the best
solution -- just practice. You will need to turn off keep alives to do
that right; most big sites running Apache do this anyway.
mod_backhand can greatly benefit from keep-alives turned on. So, there
is a patch on the website (under performance tips) that will allow
mod_backhand to override your KeepAlives off directive, while
connections coming from the outside world will respect it. It is
explained a little more there.
> Or, is the recommended solution to:
>
> * use a row of proxy servers
> [ ... snip ... ]
> * a row of app servers that serve/create the content
>
> Right now I'm doing the second option, but am toying with the idea
> presented in the first. Does anyone have any experience with the first or
> know of any side effects of increasing the buffer size to something close to
> 100KB?
There should be no side effects... Under BSD you will use a lot of mbufs
and under Linux it (the kernel) will eat a lot of memory -- but, then
again, that is what you want.
--
Theo Schlossnagle
1024D/A8EBCF8F/13BD 8C08 6BE2 629A 527E 2DC2 72C2 AD05 A8EB CF8F
2047R/33131B65/71 F7 95 64 49 76 5D BA 3D 90 B9 9F BE 27 24 E7
> 1) For the resource collection/broadcast process, can't you give it
> a higher priority and make this artifact go away?
Yes. I have done this on some mod_perl driven sites and I know it has
been done by others on mod_php driven sites where the load on the
webservers is very high. The PID of the resource collector is printed
into the error_log, so you can get it from there and renice your process
to -19 or something like that (that is what I do).
I guess if you were feeling spiffy, you could add the one line of code
required for mod_backhand to do this itself (right before it drops
privs) in broadcast_my_stats in back_util.c. I may add this as an
option later, after I experiment with portaility issues. Of course, it
should only do this if you specify:
BackhandDaemonPriority <integer>
in the httpd.conf (which doesn't exist yet -- just an idea).
> 2) This brings up a great point of discussion that I've long
> wondered about. Proxying vs. backhanding everything. Let's say that the
> pages that you're generating are over 8KB in size (the kernel's network
> socket buffer size) and that my Apache processes are about 35MB in size.
> That being the case, I would think that you would want to:
>
> * use backhand on everything
> * make everything an app server
> * increase the network buffer size to something insane like 100KB
> and hope that your pages/images are smaller than 100KB
> * bank on the fact that after the Apache process writes to the
> socket (which should be able to buffer the entire page), that the apache
> process will be able to disconnect from the socket and go about serving more
> incoming requests
I would definitely try the above over the below. I know it is standard
practice to have a row of proxies and and row of apps (sometimes running
on the same machine)... But, that doesn't make it right, or the best
solution -- just practice. You will need to turn off keep alives to do
that right; most big sites running Apache do this anyway.
mod_backhand can greatly benefit from keep-alives turned on. So, there
is a patch on the website (under performance tips) that will allow
mod_backhand to override your KeepAlives off directive, while
connections coming from the outside world will respect it. It is
explained a little more there.
> Or, is the recommended solution to:
>
> * use a row of proxy servers
> [ ... snip ... ]
> * a row of app servers that serve/create the content
>
> Right now I'm doing the second option, but am toying with the idea
> presented in the first. Does anyone have any experience with the first or
> know of any side effects of increasing the buffer size to something close to
> 100KB?
There should be no side effects... Under BSD you will use a lot of mbufs
and under Linux it (the kernel) will eat a lot of memory -- but, then
again, that is what you want.
--
Theo Schlossnagle
1024D/A8EBCF8F/13BD 8C08 6BE2 629A 527E 2DC2 72C2 AD05 A8EB CF8F
2047R/33131B65/71 F7 95 64 49 76 5D BA 3D 90 B9 9F BE 27 24 E7