Mailing List Archive

In message <20070705181517.GA5791 at falcon>, Christoph writes:
>Hi,
>
>are there any plans to include support for multiple backends in varnish?

You can already use multiple backens in Varnish.

>It would be great to have either fault-tolerance and/or simple
>load-balancing.

Yes, this is indeed on our plans, I keep trying to get somebody to
tell me how it should work, but with very little luck :-)

Just saying "fault-tolerance" or "load-balancing" isn't saying much
you see, we need to formulate strategies and algorithms...

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

Poul-Henning Kamp wrote:

> Just saying "fault-tolerance" or "load-balancing" isn't saying much
> you see, we need to formulate strategies and algorithms...

What's wrong with the usual list:

- round-robin
- by load (measure response time from last x requests, send to the faster)
- by content (request .pngs from one server, .jpgs from the other)
- by address (request /dir1/* from one server, /dir2/* from the other)
- weighted (60% from one server, 40% from the other)

and the freaks:
- by hashing the incoming IP (so each IP gets bound to one server,
useful for cookies)
- by size (learn object sizes, get smaller objects from one server,
larger from the other)
- by cookie (learn cookies, hash/bin them to servers)

? :)

(this post is a half-joke, I agree there are so many options that it's
hard to pick the best ones :) )

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 249 bytes
Desc: OpenPGP digital signature
Url : http://projects.linpro.no/pipermail/varnish-misc/attachments/20070706/69b05934/attachment.pgp

On 7/6/07, Poul-Henning Kamp <phk at phk.freebsd.dk> wrote:
> In message <20070705181517.GA5791 at falcon>, Christoph writes:
> >Hi,
> >
> >are there any plans to include support for multiple backends in varnish?
>
> You can already use multiple backens in Varnish.
>
> >It would be great to have either fault-tolerance and/or simple
> >load-balancing.
>
> Yes, this is indeed on our plans, I keep trying to get somebody to
> tell me how it should work, but with very little luck :-)
>
> Just saying "fault-tolerance" or "load-balancing" isn't saying much
> you see, we need to formulate strategies and algorithms...

for fault-tolerance the proposal from anton stonor could be used.
Extend it with the backend-name

># First, we setup decide how to "sniff" that a backend is down
>#
># options_ping: Send a HTTP OPTIONS (Perlbal does that)
># timeout: If the backend does not answer within x seconds, it is
># probably down
># icp: Abuse the protocol. (Squid + Zope does that)
>backend.default.down_protocol = options_ping | timeout | icp

># What is the timeout limit?
>backend.default.timeout = 30

backend main {
set backend.host = "IP";
set backend.port = "80";
}
backend backup {
set backend.host = "IP";
set backend.port = "80";
}



then we could use

sub vcl_recv {
set req.backend = main
if (backend.main.down) {
set req.backend = backup
}
}



for loadbalancing we could either extend the backend definition

backend main {
set backend.1.host = "IP";
set backend.1.port = "80";
set backend.1.weight = 1;
set backend.2.host = "IP";
set backend.2.port = "80";
set backend.2.weight = 1;
}

or introduce a new balancer (i don't know if we could just nest backends)

balancer mainbl {
set balancer.strategy = "roundrobin";
# set balancer.strategy = "random";
backend one {
set backend.host = IP;
set backend.port = 80;
set backend.weight = 1;
}
backend two {
set backend.host = IP;
set backend.port = 80;
set backend.weight = 1;
}
}

and use it

sub vcl_recv {
set req.backend = mainbl
}



Greetings
Christoph

"Poul-Henning Kamp" <phk at phk.freebsd.dk> writes:

> Yes, this is indeed on our plans, I keep trying to get somebody to
> tell me how it should work, but with very little luck :-)
>
> Just saying "fault-tolerance" or "load-balancing" isn't saying much
> you see, we need to formulate strategies and algorithms...

Common load-balancing strategies are:

* Round robin. (easy to implement)

* Weighted, i.e. this host can handle twice as much as that host.
(twice as much what? Bytes? Requests?) Some black boxes can do a
TCP connect or a HTTP GET to the backend to fetch a number
representing load from the web server, used to weigh backends.

* Weighted based on measured response time from backend. (Send
requests to the fastest responding node, one should perhaps base
this on 2xx responses only)

* Hash on client IP, Host: header, substring of session cookie, url,
or phase of the moon to select backend. Some application servers
generate session cookies where the first n bytes represent the ID of
a backend server.

What to do with requests that are waiting for a failing backend?

* Return a 5xx message

* Try again on another backend? (Could have undesirable results, and
not be possible for all request types. For instance, it's hard to
replay a piped request)

How often should one retry a failed backend?
> Never attribute to malice what can adequately be explained by incompetence.

Any sufficiently advanced incompetence is indistinguishable from malice.

--
Stig Sandbeck Mathisen, Linpro

In message <468E1E78.3050202 at fer.hr>, Ivan Voras writes:

>
>> Just saying "fault-tolerance" or "load-balancing" isn't saying much
>> you see, we need to formulate strategies and algorithms...
>
>What's wrong with the usual list:

>[...]

>(this post is a half-joke, I agree there are so many options that it's=20
>hard to pick the best ones :) )

There is your point right there: You won't have "all of the above"
any time soon. What I'm looking for is input in which 20% of the work
gives 80% of the benefit.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

----- Poul-Henning Kamp <phk at phk.freebsd.dk> wrote:
> There is your point right there: You won't have "all of the above"
> any time soon. What I'm looking for is input in which 20% of the
> work gives 80% of the benefit.

A huge step would be to have even a basic facility for multiple backends, like round-robin or weighted round-robin (which should theoretically be the easiest models to implement as they require little in the way of monitoring of the backends?).

Having such an option would enable people to setup simple fault-tolerance and would increase the usability of Varnish a lot.
More advanced models of dividing the load on the backends are (very) nice to have features imho, the big leap is any functionality at all.

Of course this is all purely my humble opinion.

Regards
--
Denis Braekhus - Teknisk Ansvarlig ABC Startsiden AS
http://www.startsiden.no

On Tue, 10 Jul 2007 13:52:14 +0200, Denis Br?khus
<denis at startsiden.no> wrote:

> ----- Poul-Henning Kamp <phk at phk.freebsd.dk>
> wrote:
>> There is your point right there: You won't have "all of the above"
>> any time soon. What I'm looking for is input in which 20% of the
>> work gives 80% of the benefit.
>
> A huge step would be to have even a basic facility for multiple
> backends, like round-robin or weighted round-robin (which should
> theoretically be the easiest models to implement as they require little
> in the way of monitoring of the backends?).
>
> Having such an option would enable people to setup simple
> fault-tolerance and would increase the usability of Varnish a lot.
> More advanced models of dividing the load on the backends are (very)
> nice to have features imho, the big leap is any functionality at all.
>
> Of course this is all purely my humble opinion.
>
> Regards


Hi!

Why don't you use a dedicated load balancer for this? SOmething like
HAProxy.

Regards,
Florian Schulze

----- Florian Schulze <florian.schulze at gmx.net> wrote:
> > A huge step would be to have even a basic facility for multiple
> > backends, like round-robin or weighted round-robin (which should
> > theoretically be the easiest models to implement as they require
> > little in the way of monitoring of the backends?).
> > Having such an option would enable people to setup simple
> > fault-tolerance and would increase the usability of Varnish a lot.
> > More advanced models of dividing the load on the backends are (very)
> > nice to have features imho, the big leap is any functionality at
> > all.
> > Of course this is all purely my humble opinion.
> Why don't you use a dedicated load balancer for this? SOmething like
> HAProxy.

Of course there are quite a few alternatives, both software and hardware, open source and proprietary. However Varnish is currently positioned to be placed as near the end user as possible, and having the option of not implementing a separate layer in the design for this kind of functionality would be excellent, both reducing complexity and possible points of failure / bugs.

As previously discussed, other reverse proxies (like nginx and Perlbal) implement such functionality in some form or another (Perlbal being the most advanced of the two it seems).

I would very much like to have simple loadbalancing built into Varnish instead of having to rely on an additional 3rd party solution in all setups.

Regards
--
Denis Braekhus - Teknisk Ansvarlig ABC Startsiden AS
http://www.startsiden.no