Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. Varnish>
  3. Misc
Varnish not respecting pass to backend for specified http hosts
rsreese at gmail
Mar 31, 2020, 2:12 PM
Post #1 of 5 (1047 views)
Permalink
Running Varnish 6.0.6 in a Docker container for several Wordpress sites. I
have several domains that I would like to pass to the backend verse having
them cached, but the configuration is not behaving as intended. Looking to
understand why I am unable to specify which sites I would like to pass. If
I do something like:

if ((req.http.host ~ "(domain.com) || (dev.domain.com)")) {
set req.backend_hint = default;
} else {
return (pass);
}

then every hostname's content is cached where I would expect only the two
specified domains to cache, and everything not defined, to pass. Also, if I
do not specify the configuration, all sites are cached (as expected). If I
use something like:

if ((req.http.host ~ "(domain.com) || (dev.domain.com)")) {
return (pass);
}

then no sites are cached where I would expect everything to cache except
for the two domains. What might be causing this behavior? I looked at the
requests with varnishlog, the undefined domains are indeed being fetched
from the backend verse being cached:

- VCL_call RECV
- VCL_acl NO_MATCH forbidden
- VCL_return pass
- VCL_call HASH
- VCL_return lookup
- VCL_call PASS
- VCL_return fetch

Varnish configuration is attached.

Attached Files:

Re: Varnish not respecting pass to backend for specified http hosts [ In reply to ]
guillaume at varnish-software
Mar 31, 2020, 2:56 PM
Post #2 of 5 (1046 views)
Permalink
Hi,

I think there is a bit of confusion regarding what "return(pass)" does.

Basically, when a request comes in, you need to answer two questions:
- do I want to cache it?
- if I need data from the backend, well, what is my backend?

"return(pass)" answers the first question with a big fat "no", and this is
what you see in your log ("VCL_call PASS" and "VCL_return pass"), the
request will be fetched backend, but won't be stored in cache.

You then need to decide where to fetch the data from, but in your vcl, you
only have one backend, so everything comes from the same backend, what you
want is to create a second backend, and do

if ((req.http.host ~ "(domain.com) || (dev.domain.com)")) {
set req.backend_hint = default;
} else {
set req.backend_hint = other;
return (pass);
}


Hopefully that will clarify things a bit.

Out of curiosity, are you using the official varnish image (
https://hub.docker.com/_/varnish) or something else?

Cheers,

--
Guillaume Quintard


On Tue, Mar 31, 2020 at 2:13 PM Stephen Reese <rsreese@gmail.com> wrote:

> Running Varnish 6.0.6 in a Docker container for several Wordpress sites. I
> have several domains that I would like to pass to the backend verse having
> them cached, but the configuration is not behaving as intended. Looking to
> understand why I am unable to specify which sites I would like to pass. If
> I do something like:
>
> if ((req.http.host ~ "(domain.com) || (dev.domain.com)")) {
> set req.backend_hint = default;
> } else {
> return (pass);
> }
>
> then every hostname's content is cached where I would expect only the two
> specified domains to cache, and everything not defined, to pass. Also, if I
> do not specify the configuration, all sites are cached (as expected). If I
> use something like:
>
> if ((req.http.host ~ "(domain.com) || (dev.domain.com)")) {
> return (pass);
> }
>
> then no sites are cached where I would expect everything to cache except
> for the two domains. What might be causing this behavior? I looked at the
> requests with varnishlog, the undefined domains are indeed being fetched
> from the backend verse being cached:
>
> - VCL_call RECV
> - VCL_acl NO_MATCH forbidden
> - VCL_return pass
> - VCL_call HASH
> - VCL_return lookup
> - VCL_call PASS
> - VCL_return fetch
>
> Varnish configuration is attached.
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
Re: Varnish not respecting pass to backend for specified http hosts [ In reply to ]
dridi at varni
Mar 31, 2020, 3:06 PM
Post #3 of 5 (1046 views)
Permalink
On Tue, Mar 31, 2020 at 9:58 PM Guillaume Quintard
<guillaume@varnish-software.com> wrote:
>
> Hi,
>
> I think there is a bit of confusion regarding what "return(pass)" does.
>
> Basically, when a request comes in, you need to answer two questions:
> - do I want to cache it?
> - if I need data from the backend, well, what is my backend?
>
> "return(pass)" answers the first question with a big fat "no", and this is what you see in your log ("VCL_call PASS" and "VCL_return pass"), the request will be fetched backend, but won't be stored in cache.
>
> You then need to decide where to fetch the data from, but in your vcl, you only have one backend, so everything comes from the same backend, what you want is to create a second backend, and do
>
> if ((req.http.host ~ "(domain.com) || (dev.domain.com)")) {

No matter how you look at it this if statement is broken.

You can either go for this:

> if (req.http.host ~ "^(dev\.)?domain\.com")

Or you can do this:

> if (req.http.host == "domain.com" || req.http.host == "dev.domain.com")

The ~ operator in this context matches regular expressions, and your
regex doesn't make sense.

Dridi

> set req.backend_hint = default;
> } else {
> set req.backend_hint = other;
> return (pass);
> }
>
>
> Hopefully that will clarify things a bit.
>
> Out of curiosity, are you using the official varnish image (https://hub.docker.com/_/varnish) or something else?
>
> Cheers,
>
> --
> Guillaume Quintard
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish not respecting pass to backend for specified http hosts [ In reply to ]
rsreese at gmail
Mar 31, 2020, 3:53 PM
Post #4 of 5 (1046 views)
Permalink
On Tue, Mar 31, 2020 at 5:56 PM Guillaume Quintard
<guillaume@varnish-software.com> wrote:

> Basically, when a request comes in, you need to answer two questions:
> - do I want to cache it?
> - if I need data from the backend, well, what is my backend?
>
> "return(pass)" answers the first question with a big fat "no", and this is what you see in your log ("VCL_call PASS" and "VCL_return pass"), the request will be fetched backend, but won't be stored in cache.
>
> You then need to decide where to fetch the data from, but in your vcl, you only have one backend, so everything comes from the same backend, what you want is to create a second backend, and do

This cleared things up, the following seems to work in that it will
not cache requests for the two domains but appears to cache everything
I would like to:

if ( req.http.host == "domain.com" || req.http.host ==
"dev.domain.com") {
return (pass);
} else {
set req.backend_hint = default;
}

Or since I only have one backend:

if ( req.http.host == "domain.com" || req.http.host ==
"dev.domain.com") {
return (pass);
}

> Out of curiosity, are you using the official varnish image (https://hub.docker.com/_/varnish) or something else?

I am using the official Varnish image from the aforementioned link,
works great! Nginx -> Varnish -> Nginx -> PHP-FPM. With fairly heavy
Wordpress theme and plugins, site will sees ~1-3k ms DOMContentLoaded
without Varnish and ~300ms with!
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish not respecting pass to backend for specified http hosts [ In reply to ]
rsreese at gmail
Mar 31, 2020, 3:54 PM
Post #5 of 5 (1046 views)
Permalink
On Tue, Mar 31, 2020 at 6:06 PM Dridi Boukelmoune <dridi@varni.sh> wrote:
> > if ((req.http.host ~ "(domain.com) || (dev.domain.com)")) {
>
> No matter how you look at it this if statement is broken.
>
> You can either go for this:
>
> > if (req.http.host ~ "^(dev\.)?domain\.com")
>
> Or you can do this:
>
> > if (req.http.host == "domain.com" || req.http.host == "dev.domain.com")
>
> The ~ operator in this context matches regular expressions, and your
> regex doesn't make sense.

Thanks for clearing this up!
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal