Mailing List Archive

eculp at encontacto.net wrote:
> I have a couple of questions:
>
> 1. Is the above the best way to cache dynamically generated CMS pages?
>

I'm trying to do something similar and it seems to be the recommended
way to cache pages even when cookies are present.

> 2. I am running it without vhosts. I have tried and cannot get vhosts
> to work. The only thing that I've found that might work would be
> the vcl routines for multiple vhost backends from the vcl man page:
>

I'll let other people more experienced answer that one.

> 3. I'm also running eaccelerator for php. Is that a good idea?
>

eAccelerator, or any php cache (like xcache or apc) is really a side
issue from Varnish. I've used all of those before with not much trouble.
I would recommend its usage.

-- james

Small varnish 1.1 test with openrealty and joomla. [ In reply to ]

Jul 30, 2007, 11:09 PM

Post #3 of 15 (1846 views)

eculp at encontacto.net writes:
> sub vcl_fetch {
> if (obj.http.Set-Cookie) {
> insert;
> }
> }

You should strip the Set-Cookie header before inserting the object into
the cache:

sub vcl_fetch {
if (obj.http.Set-Cookie) {
remove obj.http.Set-Cookie;
}
}

(the "insert" is implicit)

> 1. Is the above the best way to cache dynamically generated CMS
> pages?

It depends on the CMS, really :)

> 2. I am running it without vhosts. I have tried and cannot get vhosts
> to work. The only thing that I've found that might work would be
> the vcl routines for multiple vhost backends from the vcl man page:
>
> The following example shows how to support multiple sites
> running on separate backends in the same Varnish instance,
> by selecting backends based on the request URL.
>
> but that seems to be overkill. What is the recommended way to
> handle apache virtual hosts?

The example you mention illustrates how to cache multiple virtual hosts
served by *separate* backends. If all your virtual hosts are on the
same backend, you shouldn't need to do anything.

> 3. I'm also running eaccelerator for php. Is that a good idea?

I'm not familiar with eAccelerator. If it's a byte-code cache, it will
help speed up cache misses. If it only caches finished pages, it's
probably redundant but not harmful.

> 4. Is there additional documentation on initial set up? The manuals
> seem to be pretty good, they have gotten me this far.

I keep intending to write more documentation, but like everyone else's
my days only have 24 hours :)

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no

Small varnish 1.1 test with openrealty and joomla. [ In reply to ]

Jul 31, 2007, 3:47 AM

Post #4 of 15 (1852 views)

Quoting Dag-Erling Sm?rgrav <des at linpro.no>:

> eculp at encontacto.net writes:
>> sub vcl_fetch {
>> if (obj.http.Set-Cookie) {
>> insert;
>> }
>> }
>
> You should strip the Set-Cookie header before inserting the object into
> the cache:
>
> sub vcl_fetch {
> if (obj.http.Set-Cookie) {
> remove obj.http.Set-Cookie;
> }
> }

Substituted the above for the "insert" Thanks.

> (the "insert" is implicit)
>
>> 1. Is the above the best way to cache dynamically generated CMS
>> pages?
>
> It depends on the CMS, really :)

That is what I was afraid of. ;)

>> 2. I am running it without vhosts. I have tried and cannot get vhosts
>> to work. The only thing that I've found that might work would be
>> the vcl routines for multiple vhost backends from the vcl man page:
>>
>> The following example shows how to support multiple sites
>> running on separate backends in the same Varnish instance,
>> by selecting backends based on the request URL.
>>
>> but that seems to be overkill. What is the recommended way to
>> handle apache virtual hosts?
>
> The example you mention illustrates how to cache multiple virtual hosts
> served by *separate* backends. If all your virtual hosts are on the
> same backend, you shouldn't need to do anything.

That is what I thought so it must be something in my apache vhost
configuration. It really doesn't make sense yet, to me.

At least I now know that it does or should work. That is the
important thing for me for now.

>> 3. I'm also running eaccelerator for php. Is that a good idea?
>
> I'm not familiar with eAccelerator. If it's a byte-code cache, it will
> help speed up cache misses. If it only caches finished pages, it's
> probably redundant but not harmful.
>
>> 4. Is there additional documentation on initial set up? The manuals
>> seem to be pretty good, they have gotten me this far.
>
> I keep intending to write more documentation, but like everyone else's
> my days only have 24 hours :)

If I can get it going, the doc's that exist are pretty good. I think
that a solid varnish is more important that the doc's for now.

Thanks for all your help and your work on varnish.

ed

> DES
> --
> Dag-Erling Sm?rgrav
> Senior Software Developer
> Linpro AS - www.linpro.no
>

Small varnish 1.1 test with openrealty and joomla. [ In reply to ]

Jul 31, 2007, 4:30 AM

Post #5 of 15 (1849 views)

Quoting James <james at nyi.net>:

> eculp at encontacto.net wrote:
>> I have a couple of questions:
>>
>> 1. Is the above the best way to cache dynamically generated CMS pages?
>>
>
> I'm trying to do something similar and it seems to be the
> recommended way to cache pages even when cookies are present.
>
>> 2. I am running it without vhosts. I have tried and cannot get vhosts
>> to work. The only thing that I've found that might work would be
>> the vcl routines for multiple vhost backends from the vcl man page:
>>
>
> I'll let other people more experienced answer that one.

If you do happen to get Apache Vhosts running I would certainly
appreciate your feedback on how you did it. In my case, the only way
it seems to work is with the example in the man vcl page:

The following example shows how to support multiple sites running on sep-
arate backends in the same Varnish instance, by selecting backends based
on the request URL.

and then directing them to the vhost url which is actually the same
backend. Of course I may somehow be shooting myself in the foot by
doing this, I'm just begining to test.

The default configuration for varnish has:

# backend default {
# set backend.host = "127.0.0.1";
# set backend.port = "8080";
# }

And as far as I can tell kills the the apache vhost directive by
substituting local host for the vdomain What I tried was to set the
default backend to

set backend.host = req.http.host;

but the sintax is incorrect as I expected. I need to scratch arround
to figure out a bit about vcl.

Comments?

>
>> 3. I'm also running eaccelerator for php. Is that a good idea?
>>
>
> eAccelerator, or any php cache (like xcache or apc) is really a side
> issue from Varnish. I've used all of those before with not much
> trouble. I would recommend its usage.

It works, fine, so I see no reason to remove it for now, either. I
use apc on other servers too. I'm trying to decide which I like best.

Thanks again and have a great day,

ed

Small varnish 1.1 test with openrealty and joomla. [ In reply to ]

Jul 31, 2007, 9:09 AM

Post #6 of 15 (1847 views)

eculp at encontacto.net writes:
> Quoting Dag-Erling Sm?rgrav <des at linpro.no>:
> > The example you mention illustrates how to cache multiple virtual hosts
> > served by *separate* backends. If all your virtual hosts are on the
> > same backend, you shouldn't need to do anything.
> That is what I thought so it must be something in my apache vhost
> configuration. It really doesn't make sense yet, to me.

Is Varnish passing the correct Host: header to Apache?

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no

Small varnish 1.1 test with openrealty and joomla. [ In reply to ]

Jul 31, 2007, 10:16 AM

Post #7 of 15 (1850 views)

Quoting Dag-Erling Sm?rgrav <des at linpro.no>:

> eculp at encontacto.net writes:
>> Quoting Dag-Erling Sm?rgrav <des at linpro.no>:
>> > The example you mention illustrates how to cache multiple virtual hosts
>> > served by *separate* backends. If all your virtual hosts are on the
>> > same backend, you shouldn't need to do anything.
>> That is what I thought so it must be something in my apache vhost
>> configuration. It really doesn't make sense yet, to me.
>
> Is Varnish passing the correct Host: header to Apache?

AFAIK, it is passing what I am/was telling it in default.vcl. The
localhost trumps the orginal named base vhost for apache and that is
what my configuration file was asking for. So it is correct but not
what I'm trying to do;)

backend default {
set backend.host = "127.0.0.1";
set backend.port = "8080";
}

What I am trying to do is to get it to pass something like:

set backend.host = req.http.host;

Which, from my limited perspective, would be a solution for this vhost
issue although there are probably others and hopefully better and/or
simpler options.

Thanks again, DES, for spending time on this somewhat trivial issue
that I'm having with my learning curve.

ed

>
> DES
> --
> Dag-Erling Sm?rgrav
> Senior Software Developer
> Linpro AS - www.linpro.no
>

Small varnish 1.1 test with openrealty and joomla. [ In reply to ]

Jul 31, 2007, 1:12 PM

Post #8 of 15 (1847 views)

eculp at encontacto.net writes:
> Quoting Dag-Erling Sm?rgrav <des at linpro.no>:
> > Is Varnish passing the correct Host: header to Apache?
> AFAIK, it is passing what I am/was telling it in default.vcl.

I'm not asking you to guess or speculate; I'm asking you to check your
logs. Varnish does *not* use backend.host for the Host: header, it uses
whatever it got from the client.

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no

Small varnish 1.1 test with openrealty and joomla. [ In reply to ]

ric at digitalmarbles

Jul 31, 2007, 1:20 PM

Post #9 of 15 (1855 views)

On Jul 31, 2007, at 10:16 AM, eculp at encontacto.net wrote:

> Quoting Dag-Erling Sm?rgrav <des at linpro.no>:
>
>> eculp at encontacto.net writes:
>>> Quoting Dag-Erling Sm?rgrav <des at linpro.no>:
>>>> The example you mention illustrates how to cache multiple
>>>> virtual hosts
>>>> served by *separate* backends. If all your virtual hosts are on
>>>> the
>>>> same backend, you shouldn't need to do anything.
>>> That is what I thought so it must be something in my apache vhost
>>> configuration. It really doesn't make sense yet, to me.
>>
>> Is Varnish passing the correct Host: header to Apache?
>
> AFAIK, it is passing what I am/was telling it in default.vcl. The
> localhost trumps the orginal named base vhost for apache and that is
> what my configuration file was asking for. So it is correct but not
> what I'm trying to do;)
>
> backend default {
> set backend.host = "127.0.0.1";
> set backend.port = "8080";
> }
>
> What I am trying to do is to get it to pass something like:
>
> set backend.host = req.http.host;
>
> Which, from my limited perspective, would be a solution for this vhost
> issue although there are probably others and hopefully better and/or
> simpler options.
>
> Thanks again, DES, for spending time on this somewhat trivial issue
> that I'm having with my learning curve.

The following doesn't work?

vcl_recv {
set backend.host = req.http.Host;
}

or if it's still missing the Host header, does the following work:

vcl_miss {
set bereq.http.Host = req.http.Host;
}

and ditto for vcl_pipe and vcl_pass

Ric

Small varnish 1.1 test with openrealty and joomla. [ In reply to ]

ric at digitalmarbles

Jul 31, 2007, 1:23 PM

Post #10 of 15 (1849 views)

On Jul 31, 2007, at 1:20 PM, Ricardo Newbery wrote:

>
> The following doesn't work?
>
> vcl_recv {
> set backend.host = req.http.Host;
> }
>
> or if it's still missing the Host header, does the following work:
>
> vcl_miss {
> set bereq.http.Host = req.http.Host;
> }
>
> and ditto for vcl_pipe and vcl_pass
>
> Ric

Scratch that. DES says that Host already comes from the request.

Ric

Small varnish 1.1 test with openrealty and joomla. [ In reply to ]

Jul 31, 2007, 3:19 PM

Post #11 of 15 (1870 views)

Quoting Dag-Erling Sm?rgrav <des at linpro.no>:

> eculp at encontacto.net writes:
>> Quoting Dag-Erling Sm?rgrav <des at linpro.no>:
>> > Is Varnish passing the correct Host: header to Apache?
>> AFAIK, it is passing what I am/was telling it in default.vcl.
>
> I'm not asking you to guess or speculate; I'm asking you to check your
> logs. Varnish does *not* use backend.host for the Host: header, it uses
> whatever it got from the client.

This is from the apache log:

127.0.0.1 - - [31/Jul/2007:16:33:22 -0500] "GET /favicon.ico HTTP/1.1"
404 209 "http://nuevo.ecomania.info/" "Opera/9.22 (X11; Linux i686; U;
en)"

and displays the index in the root directory under which are all the
virtual hosts such as nuevo.encomania.info.

The varnishlog -r /var/log/varnish.log is attached (4k) and again my
interpretation is the same.

If I change the default

backend default {
set backend.host = "127.0.0.1";
set backend.port = "8080";
}

to

backend nuevo {
set backend.host = "nuevo.ecomania.info";
set backend.port = "8080";
}

with all else the same, it works as expected because
http://nuevo.ecomania.info is configured as an apache vhost. It also
works for this one site if I DO NOT configure virtual hosts. Maybe
I'm barking up the wrong tree again. If so, I apologize. There is
still a lot that i don't understand.

Thanks again for your help,

ed
.
>
> DES
> --
> Dag-Erling Sm?rgrav
> Senior Software Developer
> Linpro AS - www.linpro.no
>

-------------- next part --------------
0 CLI Rd ping
0 CLI Wr 0 200 PONG 1185919372
0 CLI Rd ping
0 CLI Wr 0 200 PONG 1185919375
15 SessionOpen c 189.129.12.64 59655
15 Debug c "/#nuevo.ecomania.info#"
15 ReqStart c 189.129.12.64 59655 1115880254
15 RxRequest c GET
15 RxURL c /
15 RxProtocol c HTTP/1.1
15 RxHeader c User-Agent: Opera/9.22 (X11; Linux i686; U; en)
15 RxHeader c Host: nuevo.ecomania.info
15 RxHeader c Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1
15 RxHeader c Accept-Language: es,en;q=0.9
15 RxHeader c Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1
15 RxHeader c Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0
15 RxHeader c Cookie: __utma=233483905.1461977794.1185903227.1185903227.1185903227.1; __utmc=233483905; __utmz=233483905.1185903227.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none); __utma=108653974.1335907909.1185903226.1185903226.1185903226.1; __utmc=108653974; __ut
15 RxHeader c Cookie2: $Version=1
15 RxHeader c Cache-Control: no-cache
15 RxHeader c Connection: Keep-Alive, TE
15 RxHeader c TE: deflate, gzip, chunked, identity, trailers
15 VCL_call c recv
15 VCL_return c lookup
15 VCL_call c hash
15 VCL_return c hash
15 Hit c 1115880252
15 VCL_call c hit
15 VCL_return c deliver
15 Length c 2187
15 VCL_call c deliver
15 VCL_return c deliver
15 TxProtocol c HTTP/1.1
15 TxStatus c 200
15 TxResponse c OK
15 TxHeader c Date: Tue, 31 Jul 2007 22:01:00 GMT
15 TxHeader c Server: Apache/2.2.4 (FreeBSD) mod_ssl/2.2.4 OpenSSL/0.9.8e DAV/2 PHP/5.2.3 with Suhosin-Patch
15 TxHeader c Content-Type: text/html
15 TxHeader c Content-Length: 2187
15 TxHeader c X-Varnish: 1115880254 1115880252
15 TxHeader c Age: 118
15 TxHeader c Via: 1.1 varnish
15 TxHeader c Connection: keep-alive
15 ReqEnd c 1115880254 1185919378.468985558 1185919378.469070435 0.084838629 0.000037909 0.000046968
0 StatAddr 189.129.12.64 0 1776 4 7 0 0 4 2021 9375
15 Debug c "/favicon.ico#nuevo.ecomania.info#"
15 ReqStart c 189.129.12.64 59655 1115880255
15 RxRequest c GET
15 RxURL c /favicon.ico
15 RxProtocol c HTTP/1.1
15 RxHeader c User-Agent: Opera/9.22 (X11; Linux i686; U; en)
15 RxHeader c Host: nuevo.ecomania.info
15 RxHeader c Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1
15 RxHeader c Accept-Language: es,en;q=0.9
15 RxHeader c Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1
15 RxHeader c Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0
15 RxHeader c Referer: http://nuevo.ecomania.info/
15 RxHeader c Cache-Control: no-cache
15 RxHeader c Connection: Keep-Alive, TE
15 RxHeader c TE: deflate, gzip, chunked, identity, trailers
15 VCL_call c recv
15 VCL_return c lookup
15 VCL_call c hash
15 VCL_return c hash
15 Hit c 1115880253
15 VCL_call c hit
15 VCL_return c deliver
15 Length c 209
15 VCL_call c deliver
15 VCL_return c deliver
15 TxProtocol c HTTP/1.1
15 TxStatus c 404
15 TxResponse c Not Found
15 TxHeader c Date: Tue, 31 Jul 2007 22:01:00 GMT
15 TxHeader c Server: Apache/2.2.4 (FreeBSD) mod_ssl/2.2.4 OpenSSL/0.9.8e DAV/2 PHP/5.2.3 with Suhosin-Patch
15 TxHeader c Content-Type: text/html; charset=iso-8859-1
15 TxHeader c Content-Length: 209
15 TxHeader c X-Varnish: 1115880255 1115880253
15 TxHeader c Age: 118
15 TxHeader c Via: 1.1 varnish
15 TxHeader c Connection: keep-alive
15 ReqEnd c 1115880255 1185919378.646611929 1185919378.646687508 0.177541494 0.000034094 0.000041485
0 StatAddr 189.129.12.64 0 1776 4 8 0 0 4 2332 9584
0 CLI Rd ping
0 CLI Wr 0 200 PONG 1185919378
0 CLI Rd ping
0 CLI Wr 0 200 PONG 1185919381
15 SessionClose c timeout
15 StatSess c 189.129.12.64 59655 nan 1 2 0 0 0 596 2396
0 CLI Rd ping
0 CLI Wr 0 200 PONG 1185919384
0 CLI Rd ping
0 CLI Wr 0 200 PONG 1185919387
0 CLI Rd ping
0 CLI Wr 0 200 PONG 1185919390

Small varnish 1.1 test with openrealty and joomla. [ In reply to ]

Jul 31, 2007, 10:16 PM

Post #12 of 15 (1852 views)

eculp at encontacto.net writes:
> This is from the apache log:
>
> 127.0.0.1 - - [31/Jul/2007:16:33:22 -0500] "GET /favicon.ico HTTP/1.1"
> 404 209 "http://nuevo.ecomania.info/" "Opera/9.22 (X11; Linux i686; U;
> en)"

This shows that Apache gets the correct Host: header from Varnish.

> The varnishlog -r /var/log/varnish.log is attached (4k) and again my
> interpretation is the same.

The log you attached shows a cache hit and is therefore useless.

> If I change the default
>
> backend default {
> set backend.host = "127.0.0.1";
> set backend.port = "8080";
> }
>
> to
>
> backend nuevo {
> set backend.host = "nuevo.ecomania.info";
> set backend.port = "8080";
> }
>
> with all else the same, it works as expected because
> http://nuevo.ecomania.info is configured as an apache vhost.

No. The value of backend.host is *not* used for the Host: header.
However, "nuevo.ecomania.info" and "127.0.0.1" are *not* equivalent. My
conclusion is that Apache is misconfigured; you probably have
"NameVirtualHost 189.129.5.82" instead of "NameVirtualHost *", or your
VirtualHost entry starts with "<VirtualHost nuevo.ecomania.info>"
instead of "<VirtualHost *>".

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no

Small varnish 1.1 test with openrealty and joomla. [ In reply to ]

Aug 1, 2007, 4:50 AM

Post #13 of 15 (1855 views)