Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. Varnish>
  3. Bugs
#1659: ban lurker doesn't purge all effected objects
varnish-bugs at varnish-cache
Jan 14, 2015, 5:47 AM
Post #1 of 3 (891 views)
Permalink
#1659: ban lurker doesn't purge all effected objects
-----------------------+----------------------
Reporter: varnish@… | Type: defect
Status: new | Priority: normal
Milestone: | Component: varnishd
Version: 3.0.6 | Severity: normal
Keywords: |
-----------------------+----------------------
Hi *,

couple months ago I had a purging problem, where it seems that not all
objects where purged by the ban lurker thread. Today this issue happens
again and I had the chance to capture some logs and requests.

Story:
Our customer purges varnish every night at 1pm CET, which he does with a
PURGE-Request which is handled by the following vcl:
{{{
if (req.url ~ "\*") {
ban("obj.http.X-Purge-URL ~ " + req.url + " && obj.http.X
-Purge-Host ~ " + req.http.host);
error 200 "Purge added";
}
}}}

This request results in the following ban entry:
{{{
1421192472.105130 0G obj.http.X-Purge-URL ~ /* && obj.http.X
-Purge-Host ~ domain.tld
}}}

I know, the regex "/*" is some kind of weird, but it shoul match every URL
since the URL starts with a "/".

But now we got the problem, that not all URLs are purged, following a
varnishlog trace for an object which had have purged last night:
{{{
420 SessionOpen c X.X.X.X 38584 X.X.X.X:80
420 ReqStart c X.X.X.X 38584 2713433276
420 RxRequest c HEAD
420 RxURL c /en-gb/printed-denim-skirt-with-leather-trim.html
420 RxProtocol c HTTP/1.1
420 RxHeader c User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu)
libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
420 RxHeader c Host: www.domain.tld
420 RxHeader c Accept: */*
420 RxHeader c X-DEBUG:XXXXX
420 VCL_call c recv lookup
420 VCL_call c hash
420 Hash c /en-gb/printed-denim-skirt-with-leather-trim.html
420 Hash c www.domain.tld
420 VCL_return c hash
420 Hit c 2700988225
420 VCL_call c hit deliver
420 VCL_call c deliver deliver
420 TxProtocol c HTTP/1.1
420 TxStatus c 200
420 TxResponse c OK
420 TxHeader c Server: Apache
420 TxHeader c P3p: CP="CAO PSA OUR"
420 TxHeader c X-Cache-Debug: 1
420 TxHeader c X-Cache-Tags: catalog_product_327804
420 TxHeader c Vary: Accept-Encoding
420 TxHeader c Content-Type: text/html; charset=UTF-8
420 TxHeader c X-Purge-URL: /en-gb/printed-denim-skirt-with-leather-
trim.html
420 TxHeader c X-Purge-Host: www.domain.tld
420 TxHeader c X-Orig-Query-String:
420 TxHeader c X-Striped-Query-String:
420 TxHeader c Cache-Control: no-store, no-cache, must-revalidate,
post-check=0, pre-check=0
420 TxHeader c Pragma: no-cache
420 TxHeader c Expires: Mon, 31 Mar 2008 10:00:00 GMT
420 TxHeader c Date: Wed, 14 Jan 2015 12:42:02 GMT
420 TxHeader c X-Varnish: 2713433276 2700988225
420 TxHeader c Age: 74851
420 TxHeader c Via: 1.1 varnish
420 TxHeader c Connection: keep-alive
420 TxHeader c X-Cache: HIT
420 TxHeader c X-Cache-Hits: 12
420 TxHeader c X-BE: dir_domain_tld_random
420 TxHeader c X-BE-Name:
420 TxHeader c X-Lifetime: 11548.540
420 Length c 0
420 ReqEnd c 2713433276 1421239322.510934830 1421239322.511025429
0.000041962 0.000051498 0.000039101
}}}

The indicator for myself is the Age header of the object.

Some background infos, normally the varnish cache holds around 300.000
objects when the nightly purging starts. It takes about 30 minutes to
purge all objects from the cache.

Give me a hint if you need some more information.

Best regards,
Martin

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1659>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1659: ban lurker doesn't purge all effected objects [ In reply to ]
varnish-bugs at varnish-cache
Jan 14, 2015, 6:15 AM
Post #2 of 3 (880 views)
Permalink
#1659: ban lurker doesn't purge all effected objects
-----------------------+--------------------
Reporter: varnish@… | Owner:
Type: defect | Status: new
Priority: normal | Milestone:
Component: varnishd | Version: 3.0.6
Severity: normal | Resolution:
Keywords: |
-----------------------+--------------------

Comment (by varnish@…):

As requested, following a trace of an object entering the cache:
{{{
587 SessionOpen c X.X.X.X 42360 X.X.X.X:80
587 ReqStart c X.X.X.X 42360 2714897186
587 RxRequest c HEAD
587 RxURL c /en-gb/printed-denim-skirt-with-leather-trim.html?foo
587 RxProtocol c HTTP/1.1
587 RxHeader c User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu)
libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
587 RxHeader c Host: www.domain.tld
587 RxHeader c Accept: */*
587 RxHeader c X-DEBUG:XXXXXX
587 VCL_call c recv lookup
587 VCL_call c hash
587 Hash c /en-gb/printed-denim-skirt-with-leather-trim.html?foo
587 Hash c www.domain.tld
587 VCL_return c hash
587 VCL_call c miss fetch
587 Backend c 647 dir_domain_tld_random be_domain_server
587 TTL c 2714897186 RFC 86400 -1 -1 1421244637 0 1421244631
375007920 86400
587 VCL_call c fetch
587 TTL c 2714897186 VCL 86406 300 -1 1421244632 -6
587 VCL_return c deliver
587 ObjProtocol c HTTP/1.1
587 ObjResponse c OK
587 ObjHeader c Date: Wed, 14 Jan 2015 14:10:31 GMT
587 ObjHeader c Server: Apache
587 ObjHeader c P3p: CP="CAO PSA OUR"
587 ObjHeader c X-Cache-Debug: 1
587 ObjHeader c X-Cache-Tags: catalog_product_327804
587 ObjHeader c Vary: Accept-Encoding
587 ObjHeader c Content-Encoding: gzip
587 ObjHeader c Content-Type: text/html; charset=UTF-8
587 ObjHeader c X-Purge-URL: /en-gb/printed-denim-skirt-with-leather-
trim.html?foo
587 ObjHeader c X-Purge-Host: www.domain.tld
587 ObjHeader c X-Orig-Query-String: /en-gb/printed-denim-skirt-with-
leather-trim.html?foo
587 ObjHeader c X-Striped-Query-String: /en-gb/printed-denim-skirt-
with-leather-trim.html?foo
587 ObjHeader c Cache-Control: no-store, no-cache, must-revalidate,
post-check=0, pre-check=0
587 ObjHeader c Pragma: no-cache
587 ObjHeader c Expires: Mon, 31 Mar 2008 10:00:00 GMT
587 Gzip c U F E 34291 176585 80 214240 274260
587 Gzip c G F E 176585 34257 80 273976 273986
587 VCL_call c deliver deliver
587 TxProtocol c HTTP/1.1
587 TxStatus c 200
587 TxResponse c OK
587 TxHeader c Server: Apache
587 TxHeader c P3p: CP="CAO PSA OUR"
587 TxHeader c X-Cache-Debug: 1
587 TxHeader c X-Cache-Tags: catalog_product_327804
587 TxHeader c Vary: Accept-Encoding
587 TxHeader c Content-Type: text/html; charset=UTF-8
587 TxHeader c X-Purge-URL: /en-gb/printed-denim-skirt-with-leather-
trim.html?foo
587 TxHeader c X-Purge-Host: www.domain.tld
587 TxHeader c X-Orig-Query-String: /en-gb/printed-denim-skirt-with-
leather-trim.html?foo
587 TxHeader c X-Striped-Query-String: /en-gb/printed-denim-skirt-
with-leather-trim.html?foo
587 TxHeader c Cache-Control: no-store, no-cache, must-revalidate,
post-check=0, pre-check=0
587 TxHeader c Pragma: no-cache
587 TxHeader c Expires: Mon, 31 Mar 2008 10:00:00 GMT
587 TxHeader c Date: Wed, 14 Jan 2015 14:10:37 GMT
587 TxHeader c X-Varnish: 2714897186
587 TxHeader c Age: 0
587 TxHeader c Via: 1.1 varnish
587 TxHeader c Connection: keep-alive
587 TxHeader c X-Cache: MISS
587 TxHeader c X-BE: dir_domain_tld_random
587 TxHeader c X-BE-Name: be_domain_server
587 TxHeader c X-Lifetime:
587 Length c 0
587 ReqEnd c 2714897186 1421244631.851499319 1421244637.513417721
0.000064611 5.661878824 0.000039577
}}}

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1659#comment:1>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1659: ban lurker doesn't purge all effected objects [ In reply to ]
varnish-bugs at varnish-cache
Nov 11, 2015, 1:48 AM
Post #3 of 3 (720 views)
Permalink
#1659: ban lurker doesn't purge all effected objects
-----------------------+-------------------------
Reporter: varnish@… | Owner:
Type: defect | Status: closed
Priority: normal | Milestone:
Component: varnishd | Version: 3.0.6
Severity: normal | Resolution: worksforme
Keywords: |
-----------------------+-------------------------
Changes (by phk):

* status: new => closed
* resolution: => worksforme


Comment:

I'm timing this ticket out.

We have done significant work on the ban-lurker since then and there is a
very good chance that this issue has been fixed by them.

Please open a new ticket if this still happens in 4.1.x

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1659#comment:2>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal