Mailing List Archive

Serve cached object without assembling ESI?
Hi,

I'm trying to understand whether it's feasible to build a CDN of Varnish instances, whose primary origin(backend) is a Varnish instance which processes an ESI-capable app.
So basically:
USA visitor -> Varnish USA -> Varnish primary -> THE APP with ESI blocks
RUS visitor -> Varnish RUS -> Varnish primary -> THE APP with ESI blocks
CAN -> Varnish primary -> THE APP with ESI blocks

The APP emits HTML with ESI tags for different blocks, and sure enough, those have different cache policies.
If a request arrives to an edge Varnish, I want it to talk to the primary Varnish (so as to cache on the edge against a primary cache), but this "Varnish primary" serves the content with ESI already assembled. Thus there is no telling, how long we should cache that full page HTML on the edges.

Q: Is there a magic way to instruct the primary Varnish to not do ESI processing on the content that was already cached ??
Meaning keep things as usual if request arrives directly to primary Varnish (ESI on), but receive non-assembled HTML (with esi tags inside) otherwise, so that edges can do assembling themselves, and thus cache things on the edge with proper TTLs for blocks.

Or I just have to resort to completely disabling do_esi on the primary server? Which I could do, but that must mean that the primary Varnish cannot serve visitors directly (= loosing the edge location with lowest latency to nearby users).
Re: Serve cached object without assembling ESI? [ In reply to ]
Hi Danila,

Check "man vcl", specifically beresp.do_esi and resp.do_esi.

From what I understand, you want to always enable the former, and only
enable the latter conditionally. For example, the edges could add a header
"varnish-edge-token" to their request to the primary varnish, and you could
have that VCL there:

sub vcl_deliver {
if (req.http.varnish-edge-token == "MYTOKEN") {
set resp.do_esi = false;
}
}

Should be fairly straightforward.

Cheers,

--
Guillaume Quintard


On Sun, Sep 15, 2019 at 12:19 AM Danila Vershinin <ciapnz@gmail.com> wrote:

> Hi,
>
> I'm trying to understand whether it's feasible to build a CDN of Varnish
> instances, whose primary origin(backend) is a Varnish instance which
> processes an ESI-capable app.
>
> So basically:
>
> USA visitor -> Varnish USA -> Varnish primary -> THE APP with ESI blocks
> RUS visitor -> Varnish RUS -> Varnish primary -> THE APP with ESI blocks
> CAN -> Varnish primary -> THE APP with ESI blocks
>
> The APP emits HTML with ESI tags for different blocks, and sure enough,
> those have different cache policies.
> If a request arrives to an edge Varnish, I want it to talk to the primary
> Varnish (so as to cache on the edge against a primary cache), but this
> "Varnish primary" serves the content with ESI already assembled. Thus there
> is no telling, how long we should cache that full page HTML on the edges.
>
> Q: Is there a magic way to instruct the primary Varnish to not do ESI
> processing on the content that was *already cached* ??
> Meaning keep things as usual if request arrives directly to primary
> Varnish (ESI on), but receive non-assembled HTML (with esi tags inside)
> otherwise, so that edges can do assembling themselves, and thus cache
> things on the edge with proper TTLs for blocks.
>
> Or I just have to resort to completely disabling do_esi on the primary
> server? Which I could do, but that must mean that the primary Varnish
> cannot serve visitors directly (= loosing the edge location with lowest
> latency to nearby users).
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
Re: Serve cached object without assembling ESI? [ In reply to ]
Thank you very much.

Sent from my iPhone

> On 15 Sep 2019, at 09:28, Guillaume Quintard <guillaume@varnish-software.com> wrote:
>
> Hi Danila,
>
> Check "man vcl", specifically beresp.do_esi and resp.do_esi.
>
> From what I understand, you want to always enable the former, and only enable the latter conditionally. For example, the edges could add a header "varnish-edge-token" to their request to the primary varnish, and you could have that VCL there:
>
> sub vcl_deliver {
> if (req.http.varnish-edge-token == "MYTOKEN") {
> set resp.do_esi = false;
> }
> }
>
> Should be fairly straightforward.
>
> Cheers,
>
> --
> Guillaume Quintard
>
>
>> On Sun, Sep 15, 2019 at 12:19 AM Danila Vershinin <ciapnz@gmail.com> wrote:
>> Hi,
>>
>> I'm trying to understand whether it's feasible to build a CDN of Varnish instances, whose primary origin(backend) is a Varnish instance which processes an ESI-capable app.
>>
>> So basically:
>>
>> USA visitor -> Varnish USA -> Varnish primary -> THE APP with ESI blocks
>> RUS visitor -> Varnish RUS -> Varnish primary -> THE APP with ESI blocks
>> CAN -> Varnish primary -> THE APP with ESI blocks
>>
>> The APP emits HTML with ESI tags for different blocks, and sure enough, those have different cache policies.
>> If a request arrives to an edge Varnish, I want it to talk to the primary Varnish (so as to cache on the edge against a primary cache), but this "Varnish primary" serves the content with ESI already assembled. Thus there is no telling, how long we should cache that full page HTML on the edges.
>>
>> Q: Is there a magic way to instruct the primary Varnish to not do ESI processing on the content that was already cached ??
>> Meaning keep things as usual if request arrives directly to primary Varnish (ESI on), but receive non-assembled HTML (with esi tags inside) otherwise, so that edges can do assembling themselves, and thus cache things on the edge with proper TTLs for blocks.
>>
>> Or I just have to resort to completely disabling do_esi on the primary server? Which I could do, but that must mean that the primary Varnish cannot serve visitors directly (= loosing the edge location with lowest latency to nearby users).
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc@varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc