Mailing List Archive

On Sunday, September 29, 2002, at 02:04 , Rama K. McIntosh wrote:
> Is there a supported way to get backhand to evenly
> distribute requests, sending one request to each server
> in sucession?

That is called round-robin. You should be able to to write a candidacy
function to do that. However, this is generally a bad bad idea.

If a machine begins to slow down (due to a variety of reasonable
events). It will be hit with transactions at the same rate, yet be
servicing them more slowly. This will provide an overall decreased
turnaround time for requests across the service and can potential
overload the machine and thrash it.

It you are concerned about the number of concurrent requests on each
machine, byBusyChildren it your best bet.

> I have been trying to get backhand to balance the load
> for a heavy application server on 4-5
> unltrasparc 10's.
>
> The applicaton is very heavy and each server can
> only handle 3-4 max simutanious requests.

While I am sure that there are applications that are this heavy, but
this seems like a very serious bottleneck. I would profile and due a
thorough performance analysis of the application before spending a great
deal of time architecting your load balancing system to meet such a
sensitive request allocation requirement.

> However, during my simulated load test
> backhand inevitably hands 6-9 requests
> to one server while other servers are
> practically idle.
>
> In different runs I have tried byRandom, byLoad,
> byBusyChildren and byCPU (not all together of
> course). I have also tried having a dedicated
> backhand proxy (removeSelf) as well as
> distruting it accross all 5 machines. I am
> only backhanding the heavy requests, not
> light image requests.
>
> I know I have enough machines because I wrote
> a perl .cgi that psudo evenly distributed the requests
> and I get good results with 4 machines and better results
> with 5 machines.
>
> Any help would be much appreciated.
>
> Thanks,
> -Rama
>
>
> _______________________________________________
> backhand-users mailing list
> backhand-users@lists.backhand.org
> http://lists.backhand.org/mailman/listinfo/backhand-users
>
>
Theo Schlossnagle
Principal Consultant
OmniTI Computer Consulting, Inc. -- http://www.omniti.com/
Phone: +1 410 872 4910 x201 Fax: +1 410 872 4911
1024D/82844984/95FD 30F1 489E 4613 F22E 491A 7E88 364C 8284 4984
2047R/33131B65/71 F7 95 64 49 76 5D BA 3D 90 B9 9F BE 27 24 E7

---------------------------------------------------------
-- Presenting at ApacheCon --
-- November 18th, 2002 --
-- Las Vegas, Nevada --
-- Backhand: understanding and building HA/LB clusters --
-- http://apachecon.com/2002/US/ --
-- --
-- Learn all there is to know about high availability --
-- internet systems and load balancing techniques --
-- focusing on applications driven by the Apache web --
-- server! --
---------------------------------------------------------

Theo,

Thanks for your response. I have one question though.

It seems as though I am getting 6 requests at a time to
one server. Is backhand makeing its descision for
every request, or is it making it's decision and then
sending several requests?

Thanks,
-Rama


Theo Schlossnagle wrote:

>
> On Sunday, September 29, 2002, at 02:04 , Rama K. McIntosh wrote:
>
>> Is there a supported way to get backhand to evenly
>> distribute requests, sending one request to each server
>> in sucession?
>
>
> That is called round-robin. You should be able to to write a
> candidacy function to do that. However, this is generally a bad bad idea.
>
> If a machine begins to slow down (due to a variety of reasonable
> events). It will be hit with transactions at the same rate, yet be
> servicing them more slowly. This will provide an overall decreased
> turnaround time for requests across the service and can potential
> overload the machine and thrash it.
>
> It you are concerned about the number of concurrent requests on each
> machine, byBusyChildren it your best bet.
>
>> I have been trying to get backhand to balance the load
>> for a heavy application server on 4-5
>> unltrasparc 10's.
>>
>> The applicaton is very heavy and each server can
>> only handle 3-4 max simutanious requests.
>
>
> While I am sure that there are applications that are this heavy, but
> this seems like a very serious bottleneck. I would profile and due a
> thorough performance analysis of the application before spending a
> great deal of time architecting your load balancing system to meet
> such a sensitive request allocation requirement.
>
>> However, during my simulated load test
>> backhand inevitably hands 6-9 requests
>> to one server while other servers are
>> practically idle.
>>
>> In different runs I have tried byRandom, byLoad,
>> byBusyChildren and byCPU (not all together of
>> course). I have also tried having a dedicated
>> backhand proxy (removeSelf) as well as
>> distruting it accross all 5 machines. I am
>> only backhanding the heavy requests, not
>> light image requests.
>>
>> I know I have enough machines because I wrote
>> a perl .cgi that psudo evenly distributed the requests
>> and I get good results with 4 machines and better results
>> with 5 machines.
>>
>> Any help would be much appreciated.
>>
>> Thanks,
>> -Rama
>>
>>
>> _______________________________________________
>> backhand-users mailing list
>> backhand-users@lists.backhand.org
>> http://lists.backhand.org/mailman/listinfo/backhand-users
>>
>>
> Theo Schlossnagle
> Principal Consultant
> OmniTI Computer Consulting, Inc. -- http://www.omniti.com/
> Phone: +1 410 872 4910 x201 Fax: +1 410 872 4911
> 1024D/82844984/95FD 30F1 489E 4613 F22E 491A 7E88 364C 8284 4984
> 2047R/33131B65/71 F7 95 64 49 76 5D BA 3D 90 B9 9F BE 27 24 E7
>
> ---------------------------------------------------------
> -- Presenting at ApacheCon --
> -- November 18th, 2002 --
> -- Las Vegas, Nevada --
> -- Backhand: understanding and building HA/LB clusters --
> -- http://apachecon.com/2002/US/ --
> -- --
> -- Learn all there is to know about high availability --
> -- internet systems and load balancing techniques --
> -- focusing on applications driven by the Apache web --
> -- server! --
> ---------------------------------------------------------
>
>
> _______________________________________________
> backhand-users mailing list
> backhand-users@lists.backhand.org
> http://lists.backhand.org/mailman/listinfo/backhand-users
>

mod_backhand makes a decision for each request. However, it's
information is only update once per second. So if you have no
randomness in your decision, then it will most likely make the same
decision until the cluster resource information is updated.

Rama K. McIntosh wrote:

> It seems as though I am getting 6 requests at a time to
> one server. Is backhand makeing its descision for
> every request, or is it making it's decision and then
> sending several requests?



--
Theo Schlossnagle
Principal Consultant
OmniTI Computer Consulting, Inc. -- http://www.omniti.com/
Phone: +1 410 872 4910 x201 Fax: +1 410 872 4911
1024D/82844984/95FD 30F1 489E 4613 F22E 491A 7E88 364C 8284 4984
2047R/33131B65/71 F7 95 64 49 76 5D BA 3D 90 B9 9F BE 27 24 E7

-----------------------------------------------------------
-- Presenting at ApacheCon --
-- November 18th, 2002 --
-- Las Vegas, Nevada --
-- Backhand: understanding and building HA/LB clusters --
-- http://apachecon.com/2002/US/ --
-- --
-- Learn all there is to know about high availability --
-- internet systems and load balancing techniques --
-- focusing on applications driven by the Apache web --
-- server! --
-----------------------------------------------------------

Hello Backhand List,

I have a resolution to my "very heavy" load
distribution problem and I thought I would
share it with the list.

None of the built in candacy functions work for
me because my process is so heavy that by the
time statistics where updated, a machine was
overloaded.

To fix the problem I wrote my own version of
byBusyChildren that updated the statistics
for the "winning" server (decremented the number
of available servers). This way the number
of available servers is updaded instantly,
instead of waiting for statistics to be
reported back. NOTE: This is a hack that
does not play well with other candidacy functions.
In order for it to play well with other cadidacy
functions, the statistics would have to be
updated in the core backhand software,
not a candidate function. However I
did not want to modify the core software
as this would make upgrading difficult.

Anyway, this worked to evenly distribute
my heavy requests. This works better
than a round robin candidacy function
as the statistics will eventually be updated
with real data from the backhanded server,
thus letting backhand do it's job.

Hope this helps someone else out there.

Thanks,
-Rama


Rama K. McIntosh wrote:

> Theo,
>
> Thanks for your response. I have one question though.
>
> It seems as though I am getting 6 requests at a time to
> one server. Is backhand makeing its descision for
> every request, or is it making it's decision and then
> sending several requests?
>
> Thanks,
> -Rama
>
>
> Theo Schlossnagle wrote:
>
>>
>> On Sunday, September 29, 2002, at 02:04 , Rama K. McIntosh wrote:
>>
>>> Is there a supported way to get backhand to evenly
>>> distribute requests, sending one request to each server
>>> in sucession?
>>
>>
>>
>> That is called round-robin. You should be able to to write a
>> candidacy function to do that. However, this is generally a bad bad
>> idea.
>>
>> If a machine begins to slow down (due to a variety of reasonable
>> events). It will be hit with transactions at the same rate, yet be
>> servicing them more slowly. This will provide an overall decreased
>> turnaround time for requests across the service and can potential
>> overload the machine and thrash it.
>>
>> It you are concerned about the number of concurrent requests on each
>> machine, byBusyChildren it your best bet.
>>
>>> I have been trying to get backhand to balance the load
>>> for a heavy application server on 4-5
>>> unltrasparc 10's.
>>>
>>> The applicaton is very heavy and each server can
>>> only handle 3-4 max simutanious requests.
>>
>>
>>
>> While I am sure that there are applications that are this heavy, but
>> this seems like a very serious bottleneck. I would profile and due a
>> thorough performance analysis of the application before spending a
>> great deal of time architecting your load balancing system to meet
>> such a sensitive request allocation requirement.
>>
>>> However, during my simulated load test
>>> backhand inevitably hands 6-9 requests
>>> to one server while other servers are
>>> practically idle.
>>>
>>> In different runs I have tried byRandom, byLoad,
>>> byBusyChildren and byCPU (not all together of
>>> course). I have also tried having a dedicated
>>> backhand proxy (removeSelf) as well as
>>> distruting it accross all 5 machines. I am
>>> only backhanding the heavy requests, not
>>> light image requests.
>>>
>>> I know I have enough machines because I wrote
>>> a perl .cgi that psudo evenly distributed the requests
>>> and I get good results with 4 machines and better results
>>> with 5 machines.
>>>
>>> Any help would be much appreciated.
>>>
>>> Thanks,
>>> -Rama
>>>
>>>
>>> _______________________________________________
>>> backhand-users mailing list
>>> backhand-users@lists.backhand.org
>>> http://lists.backhand.org/mailman/listinfo/backhand-users
>>>
>>>
>> Theo Schlossnagle
>> Principal Consultant
>> OmniTI Computer Consulting, Inc. -- http://www.omniti.com/
>> Phone: +1 410 872 4910 x201 Fax: +1 410 872 4911
>> 1024D/82844984/95FD 30F1 489E 4613 F22E 491A 7E88 364C 8284 4984
>> 2047R/33131B65/71 F7 95 64 49 76 5D BA 3D 90 B9 9F BE 27 24 E7
>>
>> ---------------------------------------------------------
>> -- Presenting at ApacheCon --
>> -- November 18th, 2002 --
>> -- Las Vegas, Nevada --
>> -- Backhand: understanding and building HA/LB clusters --
>> -- http://apachecon.com/2002/US/ --
>> -- --
>> -- Learn all there is to know about high availability --
>> -- internet systems and load balancing techniques --
>> -- focusing on applications driven by the Apache web --
>> -- server! --
>> ---------------------------------------------------------
>>
>>
>> _______________________________________________
>> backhand-users mailing list
>> backhand-users@lists.backhand.org
>> http://lists.backhand.org/mailman/listinfo/backhand-users
>>
>
>
>
>
> _______________________________________________
> backhand-users mailing list
> backhand-users@lists.backhand.org
> http://lists.backhand.org/mailman/listinfo/backhand-users
>

On Sunday, October 6, 2002, at 02:43 , Rama K. McIntosh wrote:
> To fix the problem I wrote my own version of
> byBusyChildren that updated the statistics
> for the "winning" server (decremented the number
> of available servers). This way the number
> of available servers is updaded instantly,
> instead of waiting for statistics to be
> reported back. NOTE: This is a hack that
> does not play well with other candidacy functions.
> In order for it to play well with other cadidacy
> functions, the statistics would have to be
> updated in the core backhand software,
> not a candidate function. However I
> did not want to modify the core software
> as this would make upgrading difficult.

Backhand addPrediction does the same thing with Load instead of
available servers. It is a hack, but in the right environment it can
work like a charm. The real problem comes when you have 2 or more
front-tier machines. They are unaware of the other machines
decrementing the available server count. -- Still is a decent
approximation.

It almost justifies adding that to "addPrediction". This way, you just
add Backhand addPrediction as your last candidacy function and it will
hack the stats. It is not inherently damaging to modify these stats as
they will be reset fresh every second by the moderator.

I will think about this some more.

--
Theo Schlossnagle
Principal Consultant
OmniTI Computer Consulting, Inc. -- http://www.omniti.com/
Phone: +1 410 872 4910 x201 Fax: +1 410 872 4911
1024D/82844984/95FD 30F1 489E 4613 F22E 491A 7E88 364C 8284 4984
2047R/33131B65/71 F7 95 64 49 76 5D BA 3D 90 B9 9F BE 27 24 E7

---------------------------------------------------------
-- Presenting at ApacheCon --
-- November 18th, 2002 --
-- Las Vegas, Nevada --
-- Backhand: understanding and building HA/LB clusters --
-- http://apachecon.com/2002/US/ --
-- --
-- Learn all there is to know about high availability --
-- internet systems and load balancing techniques --
-- focusing on applications driven by the Apache web --
-- server! --
---------------------------------------------------------
>>>
>>>
>>> _______________________________________________
>>> backhand-users mailing list
>>> backhand-users@lists.backhand.org
>>> http://lists.backhand.org/mailman/listinfo/backhand-users
>>>
>>
>>
>>
>>
>> _______________________________________________
>> backhand-users mailing list
>> backhand-users@lists.backhand.org
>> http://lists.backhand.org/mailman/listinfo/backhand-users
>>
>
>
>
>
> _______________________________________________
> backhand-users mailing list
> backhand-users@lists.backhand.org
> http://lists.backhand.org/mailman/listinfo/backhand-users

Thanks! I now see how I can
use the built in byBusyChildren
and call a candidacy function at
the end to update stats.

This is much better than what I
did as it works with other
candidacy functions and requires
no modification of the core
backhand software.

Thanks again,
-Rama

>
> Backhand addPrediction does the same thing with Load instead of
> available servers. It is a hack, but in the right environment it can
> work like a charm. The real problem comes when you have 2 or more
> front-tier machines. They are unaware of the other machines
> decrementing the available server count. -- Still is a decent
> approximation.
>
> It almost justifies adding that to "addPrediction". This way, you
> just add Backhand addPrediction as your last candidacy function and it
> will hack the stats. It is not inherently damaging to modify these
> stats as they will be reset fresh every second by the moderator.
>
> I will think about this some more.