Mailing List Archive

Hi,

You can't detect a softpurge, but you can tell Varnish to ignore grace:

```
sub vcl_recv {
if (req.http.no-grace) {
set req.grace = 0s;
}
}
```

the softpurge kill the ttl at the object level, and this kills the grace at
the request level, so Varnish will reach out to the backend.

But note that it will also do the same even without a prior softpurge, it
just needs an expired ttl.

--
Guillaume Quintard


On Thu, Sep 3, 2020 at 3:57 AM Batanun B <batanun@hotmail.com> wrote:

> Hi,
>
> We sometimes have a problem with the backend using its internal cache for
> a few seconds too long after something has been updated. We trigger a
> softpurge (xkey vmod) in varnish, but if someone requests the page again
> very soon after that, the data that Varnish gets from the backend might be
> old. In this case, we would like to be able to tell the backend, maybe
> using an extra header, that it should skip its internal caches and give us
> the updated content.
>
> But, I'm not sure how to archive this in Varnish. Is it possible to detect
> that the page requested has been softpurged earlier? If yes, is it also
> possible to see when that softpurge took place? Because we would only ever
> need to do this if the softpurge happened less than let's say 30 seconds
> ago.
>
> And the reason that the backend data might be old after an update is that
> what we call "the backend" (from a Varnish perspective) is actually a
> complex setup of services. And there, the update happens in one place, and
> our "backend" is actually a frontend server that sometimes don't get the
> information about the update quick enough. I think that the implementation
> of this system is a bit faulty, but it is what it is, and I would like to
> use the power of Varnish to handle this, if possible.
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>

Hi,

I'm not sure how that "no-grace" header would be set. The softpurge could theoretically impact hundred of URLs, and what we would like is that any requests for these URLs after the softpurge should include a special header when talking with the backend.

Skipping grace in general, and sending that special header to all requests to the backend, is not what we want.

But now I am thinking of an alternative, that might give us somewhat what we want while being much simpler and not needing to know if a softpurge has happened or not. Since we really only need to do this in a short time period after a softpurge, and the softpurge sets ttl to zero, then we can skip the "after a softpurge" requirement and simply check if the ttl recently expired. As far as I understand it, when the ttl has expired, the obj.ttl is a negative value indicating how many seconds since the ttl expired. So 15 seconds after the ttl, it would be -15s. Then we can have something like this in in vcl_hit:

```
if (obj.ttl > -15s) {
set req.http.X-my-backend-skip-cache = "true";
return (miss);
}
```

I can't check this right now, from the computer I am at. But it should work, right? Then the only "false positives" we will end up with are the requests that happen to come in within 15 seconds of the regular ttl expiring. But if we get the cache invalidation to work fully (including in the backend), then we should be able to increase the regular ttl higher than the current 5s, and then this false positive should happen much more rarely.


Guillaume Quintard <guillaume@varnish-software.com> wrote:
>
> Hi,
>
> You can't detect a softpurge, but you can tell Varnish to ignore grace:
>
> ```
> sub vcl_recv {
> ? ? if (req.http.no-grace) {
> ? ? ? ? set req.grace = 0s;
> ? ? }
> }
> ```
>
> the softpurge kill the ttl at the object level, and this kills the grace at the request level, so Varnish will reach out to the backend.
>
> But note that it will also do the same even without a prior softpurge, it just needs an expired ttl.
>
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Wait a minute, I only offered to reduce the grace depending on a header
because I thought you only wanted to do that for some privilege clients.

From what I understand now, you want to make sure that after a softpurge,
you never get the stale object. So, if I may: why use a softpurge at all?
Just remove the objects completely and be done with it.
--
Guillaume Quintard


On Thu, Sep 3, 2020 at 9:55 AM Batanun B <batanun@hotmail.com> wrote:

> Hi,
>
> I'm not sure how that "no-grace" header would be set. The softpurge could
> theoretically impact hundred of URLs, and what we would like is that any
> requests for these URLs after the softpurge should include a special header
> when talking with the backend.
>
> Skipping grace in general, and sending that special header to all requests
> to the backend, is not what we want.
>
> But now I am thinking of an alternative, that might give us somewhat what
> we want while being much simpler and not needing to know if a softpurge has
> happened or not. Since we really only need to do this in a short time
> period after a softpurge, and the softpurge sets ttl to zero, then we can
> skip the "after a softpurge" requirement and simply check if the ttl
> recently expired. As far as I understand it, when the ttl has expired, the
> obj.ttl is a negative value indicating how many seconds since the ttl
> expired. So 15 seconds after the ttl, it would be -15s. Then we can have
> something like this in in vcl_hit:
>
> ```
> if (obj.ttl > -15s) {
> set req.http.X-my-backend-skip-cache = "true";
> return (miss);
> }
> ```
>
> I can't check this right now, from the computer I am at. But it should
> work, right? Then the only "false positives" we will end up with are the
> requests that happen to come in within 15 seconds of the regular ttl
> expiring. But if we get the cache invalidation to work fully (including in
> the backend), then we should be able to increase the regular ttl higher
> than the current 5s, and then this false positive should happen much more
> rarely.
>
>
> Guillaume Quintard <guillaume@varnish-software.com> wrote:
> >
> > Hi,
> >
> > You can't detect a softpurge, but you can tell Varnish to ignore grace:
> >
> > ```
> > sub vcl_recv {
> > if (req.http.no-grace) {
> > set req.grace = 0s;
> > }
> > }
> > ```
> >
> > the softpurge kill the ttl at the object level, and this kills the grace
> at the request level, so Varnish will reach out to the backend.
> >
> > But note that it will also do the same even without a prior softpurge,
> it just needs an expired ttl.
> >
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>

On Thu, Sep 3, 2020 at 10:57 AM Batanun B <batanun@hotmail.com> wrote:
>
> Hi,
>
> We sometimes have a problem with the backend using its internal cache for a few seconds too long after something has been updated. We trigger a softpurge (xkey vmod) in varnish, but if someone requests the page again very soon after that, the data that Varnish gets from the backend might be old. In this case, we would like to be able to tell the backend, maybe using an extra header, that it should skip its internal caches and give us the updated content.

You can't really do that, but the closest I can think of would look like this:

sub vcl_backend_fetch {
if (bereq.is_bgfetch) {
# tell the backend to somehow not serve from cache
}
}

Since a soft purge artificially moves you inside the grace period, any
hit on those objects would trigger a background fetch.

Arguably, if you use Varnish to cache responses, you might as well
always tell your backend not to serve from cache. Because if a soft
purge moves you inside the grace period, there's no guarantee that the
next hit will happen before the object leaves the grace period. At
this point this will no longer trigger a background fetch...

> But, I'm not sure how to archive this in Varnish. Is it possible to detect that the page requested has been softpurged earlier? If yes, is it also possible to see when that softpurge took place? Because we would only ever need to do this if the softpurge happened less than let's say 30 seconds ago.
>
> And the reason that the backend data might be old after an update is that what we call "the backend" (from a Varnish perspective) is actually a complex setup of services. And there, the update happens in one place, and our "backend" is actually a frontend server that sometimes don't get the information about the update quick enough. I think that the implementation of this system is a bit faulty, but it is what it is, and I would like to use the power of Varnish to handle this, if possible.

Objects are immutable (modulus TTL) once they enter the cache so
there's nothing actionable to record that they were soft-purged.

Dridi
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

> From what I understand now, you want to make sure that after a softpurge,


> you never get the stale object. So, if I may: why use a softpurge at all?

> Just remove the objects completely and be done with it.

I think I was not explaining my use case properly. It is not the Varnish cache I was talking about. The backend has it's own cache, and it can sometimes serve stale data (if requested seconds after an update). So completely removing the object from the _Varnish_ cache does not help here. And in fact it would just do harm, since if then the backend goes down we don't have any stale data from Varnish cache to serve. With softpurge and a long grace, we always are ready for a backend failure (assuming the data is in the cache).

> Arguably, if you use Varnish to cache responses, you might as well
> always tell your backend not to serve from cache. Because if a soft
> purge moves you inside the grace period, there's no guarantee that the
> next hit will happen before the object leaves the grace period. At
> this point this will no longer trigger a background fetch...

Well, the caching in the backend is not on the same level as the Varnish cache. In Varnish, a single request results in a single object to cache. In the backend, a single request can result in hundreds or thousands of separate lookups (some involving separate http calls to other services), each cachable with their own unique key. And most of these objects are reused from the cache for other requests as well. And skipping that internal cache completely, letting the backend do all these lookups and sub-requests every single time a request comes in to the backend, that would be terrible for performance. So we really only want to skip this internal cache in special circumstances.

On Sat, Sep 12, 2020 at 9:56 PM Batanun B <batanun@hotmail.com> wrote:
>
> > Arguably, if you use Varnish to cache responses, you might as well
> > always tell your backend not to serve from cache. Because if a soft
> > purge moves you inside the grace period, there's no guarantee that the
> > next hit will happen before the object leaves the grace period. At
> > this point this will no longer trigger a background fetch...
>
> Well, the caching in the backend is not on the same level as the Varnish cache. In Varnish, a single request results in a single object to cache. In the backend, a single request can result in hundreds or thousands of separate lookups (some involving separate http calls to other services), each cachable with their own unique key. And most of these objects are reused from the cache for other requests as well. And skipping that internal cache completely, letting the backend do all these lookups and sub-requests every single time a request comes in to the backend, that would be terrible for performance. So we really only want to skip this internal cache in special circumstances.

In this case you might want to coordinate your tiers with
Last-Modified headers. Cached objects are immutable with Varnish,
except for the timing attributes so my previous answer is still
applicable: you can't record that an object was soft-purged by a VMOD.

The bereq.is_bgfetch variable will tell you whether you are in the
grace period but that's about it.

Dridi
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc