Mailing List Archive

TTL <=0 ?
Hello all, varnish tells me that the TTL is smaller or equal to 0, but
looking at the response headers that is not the case as the BerespHeader
has  Expires: Wed, 27 Sep 2023 12:23:11 GMT which is in 2023!

This is the part of the log.

-   Begin          bereq 8203147 pass
-   VCL_use        boot
-   Timestamp      Start: 1664281391.802802 0.000000 0.000000
-   BereqMethod    GET
-   BereqURL /wp-content/plugins/canvas/assets/fonts/canvas-icons.woff
-   BereqProtocol  HTTP/1.1
-   BereqHeader    Host: www.site.nl
-   BereqHeader    X-Forwarded-Proto: https
-   BereqHeader    Origin: https://www.site.nl
-   BereqHeader    Accept: */*
-   BereqHeader    User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 15_6_1
like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/19G82
[FBAN/FBIOS;FBDV/iPhone12,1;FBMD/iPhone;FBSN/iOS;FBSV/15.6.1;FBSS/2;FBID/phone;FBLC/nl_NL;FBOP/5]
-   BereqHeader    Accept-Language: nl-NL,nl;q=0.9
-   BereqHeader    Referer: https://www.site.nl/url
-   BereqHeader    Accept-Encoding: gzip, deflate, br
-   BereqHeader    X-forwarded-for: 123.123.123.1
-   BereqHeader    Cookie: _sharedid=redacted; cto_bundle=redacted
-   BereqHeader    X-Varnish-Nocache-Recv: Got request cookie
(_sharedid=redacted; cto_bundle=redacted
-   BereqHeader    X-Varnish: 8203148
-   VCL_call       BACKEND_FETCH
-   VCL_return     fetch
-   BackendOpen    47 default 127.0.0.1 8080 127.0.0.1 48076
-   BackendStart   127.0.0.1 8080
-   Timestamp      Bereq: 1664281391.802852 0.000050 0.000050
-   Timestamp      Beresp: 1664281391.802989 0.000186 0.000137
-   BerespProtocol HTTP/1.1
-   BerespStatus   200
-   BerespReason   OK
-   BerespHeader   Server: nginx
-   BerespHeader   Date: Tue, 27 Sep 2022 12:23:11 GMT
-   BerespHeader   Content-Type: application/font-woff
-   BerespHeader   Content-Length: 4316
-   BerespHeader   Last-Modified: Tue, 06 Sep 2022 23:45:48 GMT
-   BerespHeader   Connection: keep-alive
-   BerespHeader   ETag: "6317dbac-10dc"
-   BerespHeader   Expires: Wed, 27 Sep 2023 12:23:11 GMT
-   BerespHeader   Cache-Control: max-age=31536000
-   BerespHeader   Accept-Ranges: bytes
-   VCL_call       BACKEND_RESPONSE
-   BerespHeader   X-Req-Host: www.site.nl
-   BerespHeader   X-Req-URL:
/wp-content/plugins/canvas/assets/fonts/canvas-icons.woff
-   BerespHeader   X-Req-URL-Base:
/wp-content/plugins/canvas/assets/fonts/canvas-icons.woff
-   BerespHeader   X-Varnish-Nocache-Beresp: TTL <= 0s
-   TTL            VCL 120 0 0 1664281392 uncacheable
-   TTL            VCL 120 0 0 1664281392 uncacheable
-   BerespHeader   X-Varnish-TTL: 120.000
-   BerespHeader   X-Varnish-Backend: default
-   VCL_return     deliver

The config from the vcl backend_responce is as follows, this sets the
X-Varnish-Nocache-Beresp header.

sub vcl_backend_response {
    if (beresp.ttl <= 0s || beresp.http.Set-Cookie ||
beresp.http.Surrogate-control ~ "no-store" ||
      (!beresp.http.Surrogate-Control && beresp.http.Cache-Control ~
"no-cache|no-store|private") ||
      beresp.http.Vary == "*") {

        # DEBUG INFORMATION - DOES NOT INFLUENCE CACHING #
        if (beresp.ttl <= 0s) {
            if (beresp.http.X-Varnish-Nocache-Beresp) {
                set beresp.http.X-Varnish-Nocache-Beresp =
beresp.http.X-Varnish-Nocache-Beresp + ", TTL <= 0s";
            }
            else {
                set beresp.http.X-Varnish-Nocache-Beresp = "TTL <= 0s";
            }
        }

        if(beresp.http.Set-Cookie) {
            if (beresp.http.X-Varnish-Nocache-Beresp) {
                set beresp.http.X-Varnish-Nocache-Beresp =
beresp.http.X-Varnish-Nocache-Beresp + ", got response cookie (" +
beresp.http.Set-Cookie + ")" ;
            }
            else {
                set beresp.http.X-Varnish-Nocache-Beresp = "Response
cookie (" + beresp.http.Set-Cookie + ")";
            }
        }

        if (beresp.http.Surrogate-control ~ "no-store" ||
(!beresp.http.Surrogate-Control && beresp.http.Cache-Control ~
"no-cache|no-store|private")) {
            if (beresp.http.X-Varnish-Nocache-Beresp) {
                set beresp.http.X-Varnish-Nocache-Beresp =
beresp.http.X-Varnish-Nocache-Beresp + ", got no-cache|no-store|private";
            }
            else {
                set beresp.http.X-Varnish-Nocache-Beresp = "Got
no-cache|no-store|private";
            }
        }
        # DEBUG INFORMATION - DOES NOT INFLUENCE CACHING #

        /*
        * Mark as "Hit-For-Pass" for the next 2 minutes
        */
        set beresp.ttl = 120s;
        set beresp.uncacheable = true;
    }

Am i right that the TTL is <=0 because it sends a cookie to the backend?
Thank you for your time!



_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: TTL <=0 ? [ In reply to ]
On 9/27/22 15:07, Johan Hendriks wrote:
> Hello all, varnish tells me that the TTL is smaller or equal to 0, but
> looking at the response headers that is not the case as the BerespHeader
> has  Expires: Wed, 27 Sep 2023 12:23:11 GMT which is in 2023!
>
> -   Begin          bereq 8203147 pass

The request was set to pass on the client side; that sets
bereq.uncacheable=true, which is passed along to the backend side as
beresp.uncacheable=true.

The Expires response header (and also Cache-Control in your example)
might at least tell browser caches that they can cache the response. But
Varnish won't cache it.

[...]
> -   BereqHeader    Cookie: _sharedid=redacted; cto_bundle=redacted
[...]

> Am i right that the TTL is <=0 because it sends a cookie to the backend?

If you haven't changed this part of builtin.vcl, then yes:

sub vcl_req_cookie {
if (req.http.Cookie) {
# Risky to cache by default.
return (pass);
}
}

If a request/response has a property such as a Cookie header, and a
number of other things that suggest that the response may be
personalized, then it can't take the chance of caching it by default.
That can be one of the worst mistakes you can make with a caching proxy.

So if you need to be able to cache despite the presence of cookies, as
do many sites these days, you need to write rules for that in VCL.
Default VCL has to make the safest choice.


Best,
Geoff
--
** * * UPLEX - Nils Goroll Systemoptimierung

Scheffelstraße 32
22301 Hamburg

Tel +49 40 2880 5731
Mob +49 176 636 90917
Fax +49 40 42949753

http://uplex.de
Re: TTL <=0 ? [ In reply to ]
One minor addition to Geoff's excellent answer: you may want to try this
VCL: https://docs.varnish-software.com/tutorials/hit-miss-logging/
Specially at the beginning it helps knowing what happened to the request.

--
Guillaume Quintard


On Tue, Sep 27, 2022 at 6:33 AM Geoff Simmons <geoff@uplex.de> wrote:

> On 9/27/22 15:07, Johan Hendriks wrote:
> > Hello all, varnish tells me that the TTL is smaller or equal to 0, but
> > looking at the response headers that is not the case as the BerespHeader
> > has Expires: Wed, 27 Sep 2023 12:23:11 GMT which is in 2023!
> >
> > - Begin bereq 8203147 pass
>
> The request was set to pass on the client side; that sets
> bereq.uncacheable=true, which is passed along to the backend side as
> beresp.uncacheable=true.
>
> The Expires response header (and also Cache-Control in your example)
> might at least tell browser caches that they can cache the response. But
> Varnish won't cache it.
>
> [...]
> > - BereqHeader Cookie: _sharedid=redacted; cto_bundle=redacted
> [...]
>
> > Am i right that the TTL is <=0 because it sends a cookie to the backend?
>
> If you haven't changed this part of builtin.vcl, then yes:
>
> sub vcl_req_cookie {
> if (req.http.Cookie) {
> # Risky to cache by default.
> return (pass);
> }
> }
>
> If a request/response has a property such as a Cookie header, and a
> number of other things that suggest that the response may be
> personalized, then it can't take the chance of caching it by default.
> That can be one of the worst mistakes you can make with a caching proxy.
>
> So if you need to be able to cache despite the presence of cookies, as
> do many sites these days, you need to write rules for that in VCL.
> Default VCL has to make the safest choice.
>
>
> Best,
> Geoff
> --
> ** * * UPLEX - Nils Goroll Systemoptimierung
>
> Scheffelstraße 32
> 22301 Hamburg
>
> Tel +49 40 2880 5731
> Mob +49 176 636 90917
> Fax +49 40 42949753
>
> http://uplex.de
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
Re: TTL <=0 ? [ In reply to ]
On 27/09/2022 18:00, Guillaume Quintard wrote:
> One minor addition to Geoff's excellent answer: you may want to try
> this VCL: https://docs.varnish-software.com/tutorials/hit-miss-logging/
> Specially at the beginning it helps knowing what happened to the request.
>
> --
> Guillaume Quintard
>
>
> On Tue, Sep 27, 2022 at 6:33 AM Geoff Simmons <geoff@uplex.de> wrote:
>
> On 9/27/22 15:07, Johan Hendriks wrote:
> > Hello all, varnish tells me that the TTL is smaller or equal to
> 0, but
> > looking at the response headers that is not the case as the
> BerespHeader
> > has  Expires: Wed, 27 Sep 2023 12:23:11 GMT which is in 2023!
> >
> > -   Begin          bereq 8203147 pass
>
> The request was set to pass on the client side; that sets
> bereq.uncacheable=true, which is passed along to the backend side as
> beresp.uncacheable=true.
>
> The Expires response header (and also Cache-Control in your example)
> might at least tell browser caches that they can cache the
> response. But
> Varnish won't cache it.
>
> [...]
> > -   BereqHeader    Cookie: _sharedid=redacted; cto_bundle=redacted
> [...]
>
> > Am i right that the TTL is <=0 because it sends a cookie to the
> backend?
>
> If you haven't changed this part of builtin.vcl, then yes:
>
> sub vcl_req_cookie {
>         if (req.http.Cookie) {
>                 # Risky to cache by default.
>                 return (pass);
>         }
> }
>
> If a request/response has a property such as a Cookie header, and a
> number of other things that suggest that the response may be
> personalized, then it can't take the chance of caching it by default.
> That can be one of the worst mistakes you can make with a caching
> proxy.
>
> So if you need to be able to cache despite the presence of
> cookies, as
> do many sites these days, you need to write rules for that in VCL.
> Default VCL has to make the safest choice.
>
>
> Best,
> Geoff
> --
> ** * * UPLEX - Nils Goroll Systemoptimierung
>
> Scheffelstraße 32
> 22301 Hamburg
>
> Tel +49 40 2880 5731
> Mob +49 176 636 90917
> Fax +49 40 42949753
>
> http://uplex.de
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Thank you all, that explains a lot.
regards
Johan