Mailing List Archive

Hi Radu,

Try increasing 'nuke_limit' (default value is 50). Check out https://github.com/varnishcache/varnish-cache/issues/1764 <https://github.com/varnishcache/varnish-cache/issues/1764> for details.

Best,

--
Carlos Abalde

> On 9 Dec 2017, at 10:05, Radu Moisa <rmoisa@yahoo.com> wrote:
>
> Hi,
>
> I have an issue with varnish (v5.2.1) returning incomplete responses when the cache gets full and it starts nuking objects.
> The request that triggered the object nuke is returned incomplete (tested with curl) and the python requests library complains with "ChunkedEncodingError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))".
>
> Do you see anything wrong in the vlc file? Should there be a mandatory return statement in the vcl_recv function?
>
>
> vcl 4.0;
>
> backend pub1 {
> .host = "pub1.example.com";
> .port = "80";
> .probe = {
> .url = "/";
> .timeout = 5s;
> .interval = 10s;
> .window = 5;
> .threshold = 3;
> }
> .connect_timeout = 10s;
> .first_byte_timeout = 900s;
> .between_bytes_timeout = 900s;
> }
>
> backend pub2 {
> .host = "pub2.example.com";
> .port = "80";
> .probe = {
> .url = "/";
> .timeout = 5s;
> .interval = 10s;
> .window = 5;
> .threshold = 3;
> }
> .connect_timeout = 10s;
> .first_byte_timeout = 900s;
> .between_bytes_timeout = 900s;
> }
>
> # Enables use of "Cache-Control: no-cache"
> sub vcl_recv {
> if (req.http.Cache-Control ~ "no-cache") {
> set req.hash_always_miss = true;
> }
>
> set req.backend_hint = pub1;
>
> if (req.http.host == "pub1-cache.example.com") {
> set req.backend_hint = pub1;
> }
>
> if (req.http.host == "pub2-cache.example.com") {
> set req.backend_hint = pub2;
> }
> }
>
> # Use smaller TTL for non 200 backend response codes
> sub vcl_backend_response {
> if (beresp.status != 200){
> set beresp.ttl = 60s;
> }
> }
>
> Thanks a lot!
> Radu
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

> On 11 Dec 2017, at 07:51, Radu Moisa <rmoisa@yahoo.com> wrote:
>
> Hi!
>
> Thanks a lot for the hint!
>
> Just so that I understand it better, nuke_limit is the "Maximum number of objects we attempt to nuke in order to make space for a object body."
> If I set it to something like 9999999, varnish will throw out only the number of objects needed to make room for the new request, not the nuke_limit number of objects, right?

Yes, that's right. While trying to store an object in the cache, if not enough free space is available, Varnish will nuke up to 'nuke_limit' objects. This will happen incrementally, while the object is being fetched from the backend, stored in the cache, and eventually also being streamed to one or more clients. If the 'nuke_limit' is reached the object won't be cached and client responses will be closed (and therefore clients will end up with a truncated response).

Best,

--
Carlos Abalde

I have caching turned off at the moment because of this (not a big deal -- the cache hit rate would be very low regardless).

It's a bit awkward to work around and this is the only case I can think of where varnish would cause a request that would otherwise succeed to fail.

I'm planning to have multiple caches (small object cache + large object cache for example) but this would not be possible if the response used chunked transfer encoding.

Setting nuke limit very high would work with chunked transfers but also makes it possible for a single response to evict everything else in the cache.

james

From: varnish-misc <varnish-misc-bounces+jmathiesen=tripadvisor.com@varnish-cache.org> on behalf of Carlos Abalde <carlos.abalde@gmail.com>
Date: Monday, December 11, 2017 at 4:11 AM
To: Radu Moisa <rmoisa@yahoo.com>
Cc: varnish-misc <varnish-misc@varnish-cache.org>
Subject: Re: Varnish sending incomplete responses when nuking objects




On 11 Dec 2017, at 07:51, Radu Moisa <rmoisa@yahoo.com<mailto:rmoisa@yahoo.com>> wrote:

Hi!

Thanks a lot for the hint!

Just so that I understand it better, nuke_limit is the "Maximum number of objects we attempt to nuke in order to make space for a object body."
If I set it to something like 9999999, varnish will throw out only the number of objects needed to make room for the new request, not the nuke_limit number of objects, right?

Yes, that's right. While trying to store an object in the cache, if not enough free space is available, Varnish will nuke up to 'nuke_limit' objects. This will happen incrementally, while the object is being fetched from the backend, stored in the cache, and eventually also being streamed to one or more clients. If the 'nuke_limit' is reached the object won't be cached and client responses will be closed (and therefore clients will end up with a truncated response).

Best,

--
Carlos Abalde