Mailing List Archive

Varnish Dynamic Page Caching & Cache Purging vs Nginx+Redis
Hello all,   I am quite new to Varnish.  I have been reading about both Varnish and Nginx+Redis for page caching, and I am particularly interested in dynamic caching and cache purging.  I have read in a number of posts that Varnish is "more flexible" in this regard, but without many additional details on this.  Can you please explain what features Varnish provides for dynamic page caching and cache purging, especially which are not available (or are more limited) in Nginx+Redis?  Please forgive me if my question is very basic/ignorant.  As I said, I am new to Varnish. Thanks and Best Regards,Sal
Re: Varnish Dynamic Page Caching & Cache Purging vs Nginx+Redis [ In reply to ]
Hi,

That's a very broad question, and so I'll keep the answer pretty high-level.

All in all, Varnish has a lot fewer internal rules than nginx and really
only cares about requests at an HTTP level. This means "dynamic" content
doesn't matter to Varnish, it's just requests/objects with specific
headers, querystrings, etc. As a result, I feel that Varnish is way better
equipped to functionally handle any kind of traffic.

Of course, because Varnish operates at a lower level, with fewer rules, it
needs an excellent configuration scheme, and that's probably what trips
people: the configuration language is actually a programming language that
allows you to dictate very precisely how each request is handled. Here an
article I wrote some weeks ago about this:
https://info.varnish-software.com/blog/finally-understanding-built-in-vcl

For purging, I won't mince my words: nginx is bad and you should stay away
from it, it's limited and impractical. Varnish on the other side is once
again very low-level and will force you to implement your own logic, but
the primitives are much more powerful. And, lucky you, here's a ready-made
VCL framework you can use:
https://github.com/varnish/toolbox/tree/master/vcls/invalidate

Hope this helps

Cheers,
--
Guillaume Quintard


On Tue, Jul 27, 2021 at 3:17 AM s s <mailme.s@yandex.com> wrote:

> Hello all,
> I am quite new to Varnish. I have been reading about both Varnish and
> Nginx+Redis for page caching, and I am particularly interested in dynamic
> caching and cache purging. I have read in a number of posts that Varnish
> is "more flexible" in this regard, but without many additional details on
> this. Can you please explain what features Varnish provides for dynamic
> page caching and cache purging, especially which are not available (or are
> more limited) in Nginx+Redis? Please forgive me if my question is very
> basic/ignorant. As I said, I am new to Varnish.
>
> Thanks and Best Regards,
> Sal
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
Re: Varnish Dynamic Page Caching & Cache Purging vs Nginx+Redis [ In reply to ]
Hello Guillaume,   Thanks very much for your prompt response, I really appreciate your help.  Just a follow-up please: The great thing that I like with Nginx is that it can be used with Redis as the cache storage backend.  This provides three critical/excellent features: 1) A single storage backend/pool that can be used/shared by all running Nginx instances. 2) Redis Cluster+Sentinel, which provide a storage backend with auto sharding + replication, for scalability (both storage capacity and read+write throughput) and high availability 3) In-memory + disk-based storage, so that I can have persistent back-ups of in-memory cache on disk and use that persistent on-disk data to restart in case of failure. Can Varnish do the same, either with Redis or with any other cache storage backend?  I tried to read a bit about the Malloc and File backends, but it is still not clear to me if the features I mentioned above are available, especially after seeing some of the answers on StackExchange.  If these features are not yet currently available, are they planned for future releases of Varnish? Thanks very much again for your guidance, much appreciated. Best Regards,Sal 27.07.2021, 18:42, "Guillaume Quintard" <guillaume@varnish-software.com>:
Hi, That's a very broad question, and so I'll keep the answer pretty high-level. All in all, Varnish has a lot fewer internal rules than nginx and really only cares about requests at an HTTP level. This means "dynamic" content doesn't matter to Varnish, it's just requests/objects with specific headers, querystrings, etc. As a result, I feel that Varnish is way better equipped to functionally handle any kind of traffic. Of course, because Varnish operates at a lower level, with fewer rules, it needs an excellent configuration scheme, and that's probably what trips people: the configuration language is actually a programming language that allows you to dictate very precisely how each request is handled. Here an article I wrote some weeks ago about this: https://info.varnish-software.com/blog/finally-understanding-built-in-vcl"]https://info.varnish-software.com/blog/finally-understanding-built-in-vcl For purging, I won't mince my words: nginx is bad and you should stay away from it, it's limited and impractical. Varnish on the other side is once again very low-level and will force you to implement your own logic, but the primitives are much more powerful. And, lucky you, here's a ready-made VCL framework you can use: https://github.com/varnish/toolbox/tree/master/vcls/invalidate"]https://github.com/varnish/toolbox/tree/master/vcls/invalidate Hope this helps Cheers,--Guillaume Quintard On Tue, Jul 27, 2021 at 3:17 AM s s <mailme.s@yandex.com> wrote:
Hello all,   I am quite new to Varnish.  I have been reading about both Varnish and Nginx+Redis for page caching, and I am particularly interested in dynamic caching and cache purging.  I have read in a number of posts that Varnish is "more flexible" in this regard, but without many additional details on this.  Can you please explain what features Varnish provides for dynamic page caching and cache purging, especially which are not available (or are more limited) in Nginx+Redis?  Please forgive me if my question is very basic/ignorant.  As I said, I am new to Varnish. Thanks and Best Regards,Sal_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc"]https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish Dynamic Page Caching & Cache Purging vs Nginx+Redis [ In reply to ]
Hello,

Varnish provides ESI (Edge Side Includes). By using ESI you can split your pages in static parts (cachable) and dynamic parts (not cacheable). Each ESI can be cached with different rules. In my opinion, it makes no sense to use redis as an HTTP cache. I see the task of redis mainly as an object cache in the backend.

tom

Am Dienstag, den 27.07.2021, 14:16 +0400 schrieb s s:
Hello all,
I am quite new to Varnish. I have been reading about both Varnish and Nginx+Redis for page caching, and I am particularly interested in dynamic caching and cache purging. I have read in a number of posts that Varnish is "more flexible" in this regard, but without many additional details on this. Can you please explain what features Varnish provides for dynamic page caching and cache purging, especially which are not available (or are more limited) in Nginx+Redis? Please forgive me if my question is very basic/ignorant. As I said, I am new to Varnish.

Thanks and Best Regards,
Sal

_______________________________________________

varnish-misc mailing list

<mailto:varnish-misc@varnish-cache.org>

varnish-misc@varnish-cache.org


<https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc>

https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish Dynamic Page Caching & Cache Purging vs Nginx+Redis [ In reply to ]
Hi Tom,   Thanks for the tip on ESI.  Indeed, Redis is more commonly used as an object cache.  There are, however 3 features provided by Redis that I hope are available with Varnish, namely: 1) A single shared cache storage backend that can be used by multiple Nginx/etc. servers in a cluster.2) Scalability (of storage capacity and of write+read throughput) and high availability via clustering with auto sharding & replication.3) Both in-memory and persistent on-disk storage.  This is useful, for example, in the event of a failure, after which the data persisted on-disk can be used to restart. Does Varnish provide the above capabilities (either on its own or in combination with other tools)?  If not, is it on the roadmap? Thanks again for your help,Sal 28.07.2021, 13:49, "Tom Anheyer | BerlinOnline" <tom.anheyer@berlinonline.de>:
Hello, Varnish provides ESI (Edge Side Includes). By using ESI you can split your pages in static parts (cachable) and dynamic parts (not cacheable). Each ESI can be cached with different rules. In my opinion, it makes no sense to use redis as an HTTP cache. I see the task of redis mainly as an object cache in the backend. tom Am Dienstag, den 27.07.2021, 14:16 +0400 schrieb s s:
Hello all,   I am quite new to Varnish.  I have been reading about both Varnish and Nginx+Redis for page caching, and I am particularly interested in dynamic caching and cache purging.  I have read in a number of posts that Varnish is "more flexible" in this regard, but without many additional details on this.  Can you please explain what features Varnish provides for dynamic page caching and cache purging, especially which are not available (or are more limited) in Nginx+Redis?  Please forgive me if my question is very basic/ignorant.  As I said, I am new to Varnish. Thanks and Best Regards,Sal_______________________________________________varnish-misc mailing listvarnish-misc@varnish-cache.org  https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc"]https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc  
  ,

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc"]https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish Dynamic Page Caching & Cache Purging vs Nginx+Redis [ In reply to ]
Hi,

in our case (berlin.de) we use mosltly the standard ttl of 2mins. So we have no need for a persistent cache storage.
There a configuration tricks to combine two or more varnishd instances to act as a cluster with a "shared" cache.
Maybe this helps: https://info.varnish-software.com/blog/creating-self-routing-varnish-cluster

tom

Am Mittwoch, den 28.07.2021, 16:15 +0400 schrieb s s:
Hi Tom,
Thanks for the tip on ESI. Indeed, Redis is more commonly used as an object cache. There are, however 3 features provided by Redis that I hope are available with Varnish, namely:

1) A single shared cache storage backend that can be used by multiple Nginx/etc. servers in a cluster.
2) Scalability (of storage capacity and of write+read throughput) and high availability via clustering with auto sharding & replication.
3) Both in-memory and persistent on-disk storage. This is useful, for example, in the event of a failure, after which the data persisted on-disk can be used to restart.

Does Varnish provide the above capabilities (either on its own or in combination with other tools)? If not, is it on the roadmap?

Thanks again for your help,
Sal

28.07.2021, 13:49, "Tom Anheyer | BerlinOnline" <tom.anheyer@berlinonline.de>:
Hello,

Varnish provides ESI (Edge Side Includes). By using ESI you can split your pages in static parts (cachable) and dynamic parts (not cacheable). Each ESI can be cached with different rules. In my opinion, it makes no sense to use redis as an HTTP cache. I see the task of redis mainly as an object cache in the backend.

tom

Am Dienstag, den 27.07.2021, 14:16 +0400 schrieb s s:
Hello all,
I am quite new to Varnish. I have been reading about both Varnish and Nginx+Redis for page caching, and I am particularly interested in dynamic caching and cache purging. I have read in a number of posts that Varnish is "more flexible" in this regard, but without many additional details on this. Can you please explain what features Varnish provides for dynamic page caching and cache purging, especially which are not available (or are more limited) in Nginx+Redis? Please forgive me if my question is very basic/ignorant. As I said, I am new to Varnish.

Thanks and Best Regards,
Sal

_______________________________________________

varnish-misc mailing list

<mailto:varnish-misc@varnish-cache.org>

varnish-misc@varnish-cache.org



<https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc>

https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc






,

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org>
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

--

Tom Anheyer
Senior Developer

BerlinOnline Stadtportal GmbH & Co. KG
Alte Jakobstraße 105
10969 Berlin
Germany

Tel.: +49 30 2327-5210
Fax: +49 30 2327-5596
E-Mail: tom.anheyer@berlinonline.de<mailto:tom.anheyer@berlinonline.de>

berlin.de | berlinonline.net

Amtsgericht Berlin-Charlottenburg, HRA 31951
Sitz der Gesellschaft: Berlin, Deutschland
USt-IdNr.: DE219483549

Persönlich haftender Gesellschafter:
BerlinOnline Stadtportalbeteiligungsgesellschaft mbH
Amtsgericht Berlin-Charlottenburg, HRB 79077
Sitz der Gesellschaft: Berlin, Deutschland

Geschäftsführung: Olf Dziadek