Hi Varnish Community,
I am a software developer at Arista Network in our Tools and Infrastructure
Group. We use Varnish(6.3.1) to cache our build objects such as RPMS and
images and Varnish is an integral part of our internal build system.
Some of these objects are of the order of ~500 MB, so a GET of multiple
such objects can put a significant load on the backend(OpenSwift). Since
there is an adequate temporal locality(within a day) between the POST of
the large object and the subsequent GET, we would like to cache the object
when the POST happens, so that the subsequent GET is guaranteed to find it
in the cache(unless evicted or expired).
As I understand, Varnish bypasses the cache for POST and passes it straight
to the backend. We would like to change this behavior of POST to also add
an entry for the object in the cache if the write to the backend was
successful.
A cursory google search for “Caching Post requests on Varnish” lead to me
this article
<https://docs.varnish-software.com/tutorials/caching-post-requests/#step-1-prepare-the-vcl-for-caching-post-requests>.
Though I do not think this is exactly what we need. The article talks about
avoiding a write to the backend if another POST request is made with the
same body. That is different from what we are trying to do.
I would appreciate any insights you can provide in to:
-
What changes do I need to make to the VCL to achieve our desired behavior
-
Comments, on whether this is a bad idea for some reason.
I have been following the VCL state machine from this website
<https://varnish-cache.org/docs/6.3/reference/states.html#reference-states>.
I appreciate the help.
-Sid
I am a software developer at Arista Network in our Tools and Infrastructure
Group. We use Varnish(6.3.1) to cache our build objects such as RPMS and
images and Varnish is an integral part of our internal build system.
Some of these objects are of the order of ~500 MB, so a GET of multiple
such objects can put a significant load on the backend(OpenSwift). Since
there is an adequate temporal locality(within a day) between the POST of
the large object and the subsequent GET, we would like to cache the object
when the POST happens, so that the subsequent GET is guaranteed to find it
in the cache(unless evicted or expired).
As I understand, Varnish bypasses the cache for POST and passes it straight
to the backend. We would like to change this behavior of POST to also add
an entry for the object in the cache if the write to the backend was
successful.
A cursory google search for “Caching Post requests on Varnish” lead to me
this article
<https://docs.varnish-software.com/tutorials/caching-post-requests/#step-1-prepare-the-vcl-for-caching-post-requests>.
Though I do not think this is exactly what we need. The article talks about
avoiding a write to the backend if another POST request is made with the
same body. That is different from what we are trying to do.
I would appreciate any insights you can provide in to:
-
What changes do I need to make to the VCL to achieve our desired behavior
-
Comments, on whether this is a bad idea for some reason.
I have been following the VCL state machine from this website
<https://varnish-cache.org/docs/6.3/reference/states.html#reference-states>.
I appreciate the help.
-Sid