Mailing List Archive

It can, you will have to play a bit with the retry/restart mechanics after
changing and cleaning bereq.url.

On Dec 1, 2016 23:49, "Star Dot" <stardothosting@gmail.com> wrote:

> Hello!
>
> I am dealing with Varnish 4 piping requests to a backend server. The
> backend server processes the request and redirects to Amazon S3 to serve
> the actual file.
>
> So in short :
>
> Request -> Varnish -> Nginx 302 Redirect -> Amazon S3 file
>
> Varnish is happily caching the 302 response only, but I'm curious if I can
> somehow follow the redirect and cache the destination file completely? This
> will alleviate load off the nginx server obviously.
>
> I've seen several topics dance around this issue but I'm curious if
> osmeone can point me in the right direction!
>
> Thanks
> --
>
>
> StackStar Managed Hosting Services : https://www.stackstar.com
> Shift8 Web Design in Toronto : https://www.shift8web.ca
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>

On Thu, Dec 01, 2016 at 05:23:19PM -0500, Star Dot wrote:

> Varnish is happily caching the 302 response only, but I'm curious if I can
> somehow follow the redirect and cache the destination file completely? This
> will alleviate load off the nginx server obviously.

Yes. This is what we do:

sub vcl_deliver {
if ((resp.status == 301) || (resp.status == 302)) {
set req.url = regsub(resp.http.Location,"^http://[^/]+(.*)","\1");
return(restart);
}
}

Handling of the host header (and all the possible backends) is left as an
exercise for the reader.

What this does is that it caches the redirect as well as the destination, thus
sort-of normalizing the request, while still caching the ultimate destination
only once.

--
Andreas

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

On Fri, Dec 2, 2016 at 3:51 AM, Andreas Plesner <apj@mutt.dk> wrote:

>
> Yes. This is what we do:
>
> sub vcl_deliver {
> if ((resp.status == 301) || (resp.status == 302)) {
> set req.url = regsub(resp.http.Location,"^http://[^/]+(.*)","\1");
> return(restart);
> }
> }
>
> Handling of the host header (and all the possible backends) is left as an
> exercise for the reader.
>
> What this does is that it caches the redirect as well as the destination,
> thus
> sort-of normalizing the request, while still caching the ultimate
> destination
> only once.
>
>

Tentatively your rules work for me. Is there any way for varnish to cache
the 302 redirect if its on a completely different host? In my example we
are hitting media.domain.com which hits an nginx server that redirects it
to a completely different URL (S3) that varnish is not caching. After
testing your rule, I see that its definitely skipping a step and providing
the destination URL (S3) to the browser. But that destination is not being
cached.

Is there a way to hide that hostname/url change completely so its
transparent to the end user?