Mailing List Archive

"Releaselog | RLSLOG.net" <rlslog at gmail.com> writes:
> I have a dedicated server running Fedora 6 and hosting a Wordpress
> based website with ~80 000 daily unique visitors, already using
> Wp-Cache plugin and eAccelerator. The load was increasing lately a
> lot because of growing number of visitors so I decided to try Varnish
> as it looks way more powerful than Squid and other alternatives.
> Installation went fine, all is running now, but the server load
> *increased* instead of decreasing. I didn't change any configuration,
> all is set to default (also because of lack of the documentation), but
> as I said, the server load increased at least twice. Do you have any
> idea what could be wrong and what should I check.

The load average is not a reliable indicator of the amount of work your
server is doing. Depending on OS, pthread implementation and scheduler
strategy, you could have a very high load average on a nearly idle
machine.

Looking at your site, it seems that Varnish is working fine and caching
what it should. There isn't much more I can say without additional
information (version, command-line parameters, VCL code etc.)

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no

In message <000301c7ce9f$b4af0780$c600a8c0 at r00t>, "Releaselog | RLSLOG.net" wri
tes:


Hi Martin,

You need to tell us what you mean by "load increased", which parameter
are you looking at ?

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

By load increased, I mean the load which is returned by server after
"uptime" command. It was previously at values about 1-3 out of the peak and
now it stays well above 6 most of the time, getting to something as high as
20 or 40 in the peak time (evenings). Overally, I think it's MySQL database
which is overloading, but there aren't many things in my mind which could
help the situation apart from buying next (expensive) server if you say
Varnish works fine at my box...

Martin

----- Original Message -----
From: "Poul-Henning Kamp" <phk@phk.freebsd.dk>
To: "Releaselog | RLSLOG.net" <rlslog at gmail.com>
Cc: <varnish-misc at projects.linpro.no>
Sent: Wednesday, July 25, 2007 12:42 PM
Subject: Re: varnish increased the load


> In message <000301c7ce9f$b4af0780$c600a8c0 at r00t>, "Releaselog |
> RLSLOG.net" wri
> tes:
>
>
> Hi Martin,
>
> You need to tell us what you mean by "load increased", which parameter
> are you looking at ?
>
> --
> Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG | TCP/IP since RFC 956
> FreeBSD committer | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by
> incompetence.

"Releaselog | RLSLOG.net" <rlslog at gmail.com> writes:
> By load increased, I mean the load which is returned by server after
> "uptime" command. It was previously at values about 1-3 out of the
> peak and now it stays well above 6 most of the time, getting to
> something as high as 20 or 40 in the peak time (evenings). Overally, I
> think it's MySQL database which is overloading, but there aren't many
> things in my mind which could help the situation apart from buying
> next (expensive) server if you say Varnish works fine at my box...

You provide very little information, and you don't seem to have made any
attempt to troubleshoot the problem yourself. All I can say is that
from the looks of the HTTP headers I get from your site, Varnish seems
to cache most of it for 120 seconds. I have no idea what you used
before Varnish, nor how it was configured, so I can't tell if that's an
improvement.

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no

In message <001601c7cece$ad1c5b10$c600a8c0 at r00t>, "Releaselog | RLSLOG.net" wri
tes:

>By load increased, I mean the load which is returned by server after
>"uptime" command. It was previously at values about 1-3 out of the peak and
>now it stays well above 6 most of the time, getting to something as high as
>20 or 40 in the peak time (evenings). Overally, I think it's MySQL database
>which is overloading, but there aren't many things in my mind which could
>help the situation apart from buying next (expensive) server if you say
>Varnish works fine at my box...

It's kind of like a patient saying "Doctor it hurts", doctors really
hate that, at the bare minimum they'd like to know _where_ it hurts
and what you did before it started hurting :-)

Here are some suggestions:

What is your hit-rate in varnish (use varnishstat to see the stats)

Are you happy with that number or should it be higher ?

If it should be higher, you need to use varnishlog to find out why
varnish doesn't cache things (...as long...) as you expect.

A likely culprit is the calculation of cache-TTL, by default Varnish
uses 120 seconds if nothing else can be deduced from the headers
from the backend server. (The 120 is a parameter you can set
while running).

If your backend doesn't send headers which varnish can use to
calculate the TTL, you can use VCL to change it for specific
content, for instance:

vcl_fetch {
if (req.url ~ ".png") {
obj.ttl = 1h;
}
}

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

Releaselog | RLSLOG.net wrote:
> By load increased, I mean the load which is returned by server after
> "uptime" command. It was previously at values about 1-3 out of the peak and
> now it stays well above 6 most of the time, getting to something as high as
> 20 or 40 in the peak time (evenings). Overally, I think it's MySQL database
> which is overloading, but there aren't many things in my mind which could
> help the situation apart from buying next (expensive) server if you say
> Varnish works fine at my box...

Keep in mind you are adding processes to the system, which will mostly
be running - this will probably increase your load. Grossly simplified,
the load average can be thought of as the number of processes *waiting*
for execution. Not how much work your CPUs are actually doing.

Important question: Are your "customers" happy? ;)

Other than that, first of all, move the SQL server to a separate box
with nice I/O and memory. MySQL can really KILL a webserver if it is on
the same box as apache.

--
Kenneth R?rvik, IT HiO
Tlf 22 45 20 83
Kenneth.Rorvik at hio.no