Mailing List Archive

Will it be possible to implement custom Load Balancing algorithms in
VCL? There are numerous algorithms that can be used to distribute load like
Round Robin, Weighted Round Robin, Least Connections, Fastest Response,
Most Bandwidth, etc.

Each of these algorithms can be made sticky i.e. only balance NEW
requests and send existing ones to the same backend as they were
previously sent to (to reuse caches, etc).

In order to support these VCL will need access to various statistics like
* Number of active requests per server
* Number of requests per second per server
* Avg response time per server
* Bandwidth per server

Are these stats available in the VCL environment?

Thanks

shiraz
skanga at gmail.com

Shiraz Kanga <skanga at gmail.com> writes:
> In order to support these VCL will need access to various statistics like
> * Number of active requests per server
> * Number of requests per second per server
> * Avg response time per server
> * Bandwidth per server
>
> Are these stats available in the VCL environment?

Not presently, and our current take on backend load balancing is that
it is best done in C, probably with pluggable algorithms (or directors
as they are called in the code).

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no

Shiraz Kanga <skanga at gmail.com> writes:
> In order to support these VCL will need access to various statistics like
> * Number of active requests per server
> * Number of requests per second per server
> * Avg response time per server
> * Bandwidth per server
>
> Are these stats available in the VCL environment?

Not presently, and our current take on backend load balancing is that
it is best done in C, probably with pluggable algorithms (or directors
as they are called in the code).

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no

Dag-Erling Sm?rgrav wrote:
> Shiraz Kanga <skanga at gmail.com> writes:
>
>> In order to support these VCL will need access to various statistics like
>> * Number of active requests per server
>> * Number of requests per second per server
>> * Avg response time per server
>> * Bandwidth per server
>>
>> Are these stats available in the VCL environment?
>>
>
> Not presently, and our current take on backend load balancing is that
> it is best done in C, probably with pluggable algorithms (or directors
> as they are called in the code).
>
> DES
>
Thanks for your reply. Would you mind explaining why load balancing is
"best done in C" especially since VCL should have the same performance
as C as it goes through the C compiler.

shiraz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://projects.linpro.no/pipermail/varnish-dev/attachments/20080303/30dec5df/attachment.htm

Dag-Erling Sm?rgrav wrote:
> Shiraz Kanga <skanga at gmail.com> writes:
>
>> In order to support these VCL will need access to various statistics like
>> * Number of active requests per server
>> * Number of requests per second per server
>> * Avg response time per server
>> * Bandwidth per server
>>
>> Are these stats available in the VCL environment?
>>
>
> Not presently, and our current take on backend load balancing is that
> it is best done in C, probably with pluggable algorithms (or directors
> as they are called in the code).
>
> DES
>
Thanks for your reply. Would you mind explaining why load balancing is
"best done in C" especially since VCL should have the same performance
as C as it goes through the C compiler.

shiraz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://projects.linpro.no/pipermail/varnish-dev/attachments/20080303/30dec5df/attachment-0001.htm

Shiraz Kanga <skanga at gmail.com> writes:
> Thanks for your reply. Would you mind explaining why load balancing is
> "best done in C" especially since VCL should have the same performance
> as C as it goes through the C compiler.

I don't see why I have to explain our decision. This is the way
things are. If it doesn't work for you, tell us what you're trying to
do (not how you're trying to do it) and we'll think about it.

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no

Shiraz Kanga <skanga at gmail.com> writes:
> Thanks for your reply. Would you mind explaining why load balancing is
> "best done in C" especially since VCL should have the same performance
> as C as it goes through the C compiler.

I don't see why I have to explain our decision. This is the way
things are. If it doesn't work for you, tell us what you're trying to
do (not how you're trying to do it) and we'll think about it.

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no

In message <47BB56F8.4080902 at gmail.com>, Shiraz Kanga writes:
>
>Will it be possible to implement custom Load Balancing algorithms in
>VCL? There are numerous algorithms that can be used to distribute load like
>Round Robin, Weighted Round Robin, Least Connections, Fastest Response,
>Most Bandwidth, etc.

The plan is to implement such "directors" in C (we already have "random")
and I'm not convinced that doing it in VCL makes much sense, but can
be persuaded otherwise by good arguments.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

In message <47BB56F8.4080902 at gmail.com>, Shiraz Kanga writes:
>
>Will it be possible to implement custom Load Balancing algorithms in
>VCL? There are numerous algorithms that can be used to distribute load like
>Round Robin, Weighted Round Robin, Least Connections, Fastest Response,
>Most Bandwidth, etc.

The plan is to implement such "directors" in C (we already have "random")
and I'm not convinced that doing it in VCL makes much sense, but can
be persuaded otherwise by good arguments.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

On Mar 4, 2008, at 6:34 AM, Poul-Henning Kamp wrote:

> In message <47BB56F8.4080902 at gmail.com>, Shiraz Kanga writes:
>>
>> Will it be possible to implement custom Load Balancing algorithms in
>> VCL? There are numerous algorithms that can be used to distribute
>> load like
>> Round Robin, Weighted Round Robin, Least Connections, Fastest
>> Response,
>> Most Bandwidth, etc.
>
> The plan is to implement such "directors" in C (we already have
> "random")
> and I'm not convinced that doing it in VCL makes much sense, but can
> be persuaded otherwise by good arguments.


Load balancing algorithms can be complicated over larger sets of
backends (if that is the desired use case). mod_backhand was written
specifically as a platform to test different algorithms in the lab.
What I learned from that is that if you have a good (and very simple)
API to implement new algorithms, then it is easy enough to test. For
example, we found interesting and successful approaches using cost-
benefit-based algorithms and randomized-log2-window algorithms (much
better than simple random). As a note, simple random performs really
well in the lab (better than least-connections and fastest-response)
and in practice, while it doesn't do as well, it still does better
than most alternatives -- so good first algorithm selection.

On the other hand, VCL compiles to C. It seems you have an excellent
platform to expose that algorithm composition up the stack and make it
even more accessible without compromising efficiency. At the end of
the day, it would only be a convenience thing. If not in VCL, I hope
that it could be at least dlopened, so algorithms can be tested post-
install.

Best regards,

Theo

--
Theo Schlossnagle
Esoteric Curio -- http://lethargy.org/
OmniTI Computer Consulting, Inc. -- http://omniti.com/

On Mar 4, 2008, at 6:34 AM, Poul-Henning Kamp wrote:

> In message <47BB56F8.4080902 at gmail.com>, Shiraz Kanga writes:
>>
>> Will it be possible to implement custom Load Balancing algorithms in
>> VCL? There are numerous algorithms that can be used to distribute
>> load like
>> Round Robin, Weighted Round Robin, Least Connections, Fastest
>> Response,
>> Most Bandwidth, etc.
>
> The plan is to implement such "directors" in C (we already have
> "random")
> and I'm not convinced that doing it in VCL makes much sense, but can
> be persuaded otherwise by good arguments.


Load balancing algorithms can be complicated over larger sets of
backends (if that is the desired use case). mod_backhand was written
specifically as a platform to test different algorithms in the lab.
What I learned from that is that if you have a good (and very simple)
API to implement new algorithms, then it is easy enough to test. For
example, we found interesting and successful approaches using cost-
benefit-based algorithms and randomized-log2-window algorithms (much
better than simple random). As a note, simple random performs really
well in the lab (better than least-connections and fastest-response)
and in practice, while it doesn't do as well, it still does better
than most alternatives -- so good first algorithm selection.

On the other hand, VCL compiles to C. It seems you have an excellent
platform to expose that algorithm composition up the stack and make it
even more accessible without compromising efficiency. At the end of
the day, it would only be a convenience thing. If not in VCL, I hope
that it could be at least dlopened, so algorithms can be tested post-
install.

Best regards,

Theo

--
Theo Schlossnagle
Esoteric Curio -- http://lethargy.org/
OmniTI Computer Consulting, Inc. -- http://omniti.com/