Mailing List Archive: Varnish-cache as private CDN

Hi!

So, the big question is: do you own the content/domains that the users will
access?

If yes, there's absolutely no problem, route to Varnish, let it cache, and
you're done. There are certain vmods, like vmod_dynamic or vmod_reqwest
that will allow you to dynamically find a backend based on a hostname.

If you don't own the content, it isn't advisable to try and cache it, like,
at all.
Let's say for example you want to use varnish to cache content for
facebook.com and let's assume you can hijack DNS response to send your
users to Varnish instead of to the actual facebook servers.

If the request Varnish receives is HTTPS (encrypted), well, you're out of
luck because you won't have the certificates to pretend being facebook.com,
your users will realize it and bail out. The only way around it is to try
something like what Kazakhstan did a few years back [1], but I don't think
that would fly in Canada.
If you're thinking "wait, can't I just cache the response without
decrypting it?", nope, because the whole connection is encrypted, and
either you see everything (you have the certificate/key), or nothing (you
don't have them).
In that latter case, the best you can do is blindly redirect the connection
to the facebook server, but then you are just an HTTPS proxy, and caching
isn't relevant.

If we are talking about plaintext HTTP, and ignoring that your browser and
any website worth its salt (including facebook.com) will fight you very
hard and try to go encrypted, you have another issue: you need to know
what's cacheable, and that's a doozy.
There's no universal rule to what's cacheable, and whatever set of rules
you come up with, I'll bet I can find a website that'll break them.
And the price of failure is super high too: imagine you start sending the
same cached bank statement to everybody, people will sue you into the
ground.

So, all in all, meh, I wouldn't worry about it. And it's not just Varnish,
it's any caching solution: you just can't "cache the internet".

Sorry if that reads like a very long-winded way of saying "NO", but as I've
had to answer this question many times over the years, I thought I'd hammer
that point home once and for all :-)

[1]: https://en.wikipedia.org/wiki/Kazakhstan_man-in-the-middle_attack

--
Guillaume Quintard

On Mon, Nov 21, 2022 at 7:13 PM InfoVerse Inc. <info@infoverse.ca> wrote:

> Hello list,
>
> I am working on a design to use Varnish-Cache as a private CDN. The
> solution is for a small regional ISP in a remote region who wants to
> provide fast cached content to its users and minimize access to the
> Internet.
>
> Since this is an ISP, the users accessing the Internet can be routed to
> varnish cache servers, however, in the event of a "miss" the content should
> be fetched from the Internet. This is a different requirement than the
> traditional backend server.
>
> How can this be achieved with Varnish? I have done a bit of research on
> backends, directors but they all require a server or group of servers whose
> content can be cached.
>
> Is it possible to configure multiple Varnish storage servers as backends?
> The storage servers will fetch data from the Internet in case of a miss. Is
> this a workable solution?
>
> Looking forward to a solution.
>
> Thanks
> InfoVerse
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>