Mailing List Archive: troubleshooting Varnish purges vs bans

troubleshooting Varnish purges vs bans

miguel_3_gonzalez at yahoo

Aug 8, 2017, 1:14 PM

Post #1 of 11 (1814 views)

Hi all,

I have Wordpress sites server by an Apache Server (Cpanel) behind
Varnish.

I´m trying to troubleshoot why Varnish http purge Wordpress plugin is
not purging well the Varnish cache when the clear cache button is pressed.

However, when I run a ban manually from command line:

echo "ban req.http.host ~ $URL" | varnishadm -S /etc/varnish/secret

it seems to work clearing all content for that domain in the varnish cache.

This is what I get when I purge the varnish cache from the plugin:

varnishlog -g request -q 'ReqMethod eq "PURGE"'

- Begin req 43813385 rxreq
- Timestamp Start: 1502221040.168021 0.000000 0.000000
- Timestamp Req: 1502221040.168021 0.000000 0.000000
- ReqStart 178.33.117.62 34248
- ReqMethod PURGE
- ReqURL /.*
- ReqProtocol HTTP/1.1
- ReqHeader host: www.mydomain.com
- ReqHeader User-Agent: WordPress/4.8; https://www.mydomain.com
- ReqHeader Accept: */*
- ReqHeader Accept-Encoding: deflate, gzip
- ReqHeader Referer: http://www.mydomain.com/.*
- ReqHeader X-Purge-Method: regex
- ReqHeader Connection: close
- ReqHeader X-Forwarded-For: 178.33.117.62
- VCL_call RECV
- ReqURL /.*
- ReqURL /.*
- ReqURL /.*
- ReqHeader X-Port: 80
- ReqUnset X-Forwarded-For: 178.33.117.62
- ReqHeader X-Forwarded-For: 178.33.117.62
- ReqUnset host: www.mydomain.com
- ReqHeader Host: www.mydomain.com
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- VCL_acl MATCH purge "178.33.117.62"
- VCL_return purge
- ReqUnset Accept-Encoding: deflate, gzip
- ReqHeader Accept-Encoding: gzip
- VCL_call HASH
- VCL_return lookup
- VCL_call PURGE
- VCL_return synth
- Timestamp Process: 1502221040.168079 0.000058 0.000058
- RespHeader Date: Tue, 08 Aug 2017 19:37:20 GMT
- RespHeader Server: Varnish
- RespHeader X-Varnish: 43813386
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespReason Purged
- VCL_call SYNTH
- RespHeader Content-Type: text/html; charset=utf-8
- RespHeader Retry-After: 5
- VCL_return deliver
- RespHeader Content-Length: 243
- Storage malloc Transient
- RespHeader Accept-Ranges: bytes
- Debug "RES_MODE 2"
- RespHeader Connection: close
- Timestamp Resp: 1502221040.168152 0.000131 0.000072
- ReqAcct 223 0 223 216 243 459
- End

------ manual ban ---------------

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Re: troubleshooting Varnish purges vs bans [ In reply to ]

Aug 9, 2017, 2:54 AM

Post #2 of 11 (1809 views)

> - ReqMethod PURGE
> - ReqURL /.*

Hello Miguel,

A purge expects an exact match of an object hash, it doesn't work on
criteria like bans do. In order for a purge to succeed, you usually
need the exact Host header and URL in your purge request unless you
changed the default hash.

Cheers,
Dridi

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Re: troubleshooting Varnish purges vs bans [ In reply to ]

miguel_3_gonzalez at yahoo

Aug 9, 2017, 4:00 AM

Post #3 of 11 (1809 views)

On 08/09/17 11:54 AM, Dridi Boukelmoune wrote:
>> - ReqMethod PURGE
>> - ReqURL /.*
>
> Hello Miguel,
>
> A purge expects an exact match of an object hash, it doesn't work on
> criteria like bans do. In order for a purge to succeed, you usually
> need the exact Host header and URL in your purge request unless you
> changed the default hash.

This is not supposed to purge all objects for the whole site?

Regards,

Miguel

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Re: troubleshooting Varnish purges vs bans [ In reply to ]

Aug 9, 2017, 4:40 AM

Post #4 of 11 (1809 views)

On Wed, Aug 9, 2017 at 1:00 PM, Miguel González
<miguel_3_gonzalez@yahoo.es> wrote:
> On 08/09/17 11:54 AM, Dridi Boukelmoune wrote:
>>> - ReqMethod PURGE
>>> - ReqURL /.*
>>
>> Hello Miguel,
>>
>> A purge expects an exact match of an object hash, it doesn't work on
>> criteria like bans do. In order for a purge to succeed, you usually
>> need the exact Host header and URL in your purge request unless you
>> changed the default hash.
>
> This is not supposed to purge all objects for the whole site?

This would only purge "www.mydomain.com/.*" interpreted as a string,
not a regular expression.

Dridi

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Re: troubleshooting Varnish purges vs bans [ In reply to ]

miguel_3_gonzalez at yahoo

Aug 9, 2017, 8:50 AM

Post #5 of 11 (1809 views)

And what would be the right way to purge all the objects in the url? Or it can only be done with ban?

Regards,

Miguel

Dridi Boukelmoune <dridi@varni.sh> wrote:

>On Wed, Aug 9, 2017 at 1:00 PM, Miguel González
><miguel_3_gonzalez@yahoo.es> wrote:
>> On 08/09/17 11:54 AM, Dridi Boukelmoune wrote:
>>>> - ReqMethod PURGE
>>>> - ReqURL /.*
>>>
>>> Hello Miguel,
>>>
>>> A purge expects an exact match of an object hash, it doesn't work on
>>> criteria like bans do. In order for a purge to succeed, you usually
>>> need the exact Host header and URL in your purge request unless you
>>> changed the default hash.
>>
>> This is not supposed to purge all objects for the whole site?
>
>This would only purge "www.mydomain.com/.*" interpreted as a string,
>not a regular expression.
>
>Dridi
>
>_______________________________________________
>varnish-misc mailing list
>varnish-misc@varnish-cache.org
>https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Re: troubleshooting Varnish purges vs bans [ In reply to ]

Aug 10, 2017, 4:12 AM

Post #6 of 11 (1803 views)

On Wed, Aug 9, 2017 at 5:50 PM, Miguel Gonzalez
<miguel_3_gonzalez@yahoo.es> wrote:
> And what would be the right way to purge all the objects in the url? Or it can only be done with ban?

I'm not sure what you mean by "all the objects in the url", can you
please give me an example involving several objects?

Dridi

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Re: troubleshooting Varnish purges vs bans [ In reply to ]

miguel_3_gonzalez at yahoo

Aug 10, 2017, 10:15 AM

Post #7 of 11 (1793 views)

On 08/10/17 1:12 PM, Dridi Boukelmoune wrote:
> On Wed, Aug 9, 2017 at 5:50 PM, Miguel Gonzalez
> <miguel_3_gonzalez@yahoo.es> wrote:
>> And what would be the right way to purge all the objects in the url? Or it can only be done with ban?
>
> I'm not sure what you mean by "all the objects in the url", can you
> please give me an example involving several objects?
>
> Dridi
>

Hi,

The Wordpress plugin is supposed to clean the whole cache for a website
when clicking on a button saying "Clear cache".

You say from the varnishlog excerpt I sent that´s not the correct way
of doing so, or apparently what is logged by varnish is not a correct purge.

So how do I tell the developer (or do it myself) to correct the code so
the whole cache for a given website is purged?.

Regards,

Miguel

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Re: troubleshooting Varnish purges vs bans [ In reply to ]

Aug 11, 2017, 3:22 AM

Post #8 of 11 (1792 views)

> The Wordpress plugin is supposed to clean the whole cache for a website
> when clicking on a button saying "Clear cache".
>
> You say from the varnishlog excerpt I sent that´s not the correct way
> of doing so, or apparently what is logged by varnish is not a correct purge.
>
> So how do I tell the developer (or do it myself) to correct the code so
> the whole cache for a given website is purged?.

Use the ban() function in your VCL in this case.

Dridi

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Re: troubleshooting Varnish purges vs bans [ In reply to ]

miguel_3_gonzalez at yahoo

Aug 15, 2017, 4:07 AM

Post #9 of 11 (1781 views)

On 08/11/17 12:22 PM, Dridi Boukelmoune wrote:
>> The Wordpress plugin is supposed to clean the whole cache for a website
>> when clicking on a button saying "Clear cache".
>>
>> You say from the varnishlog excerpt I sent that´s not the correct way
>> of doing so, or apparently what is logged by varnish is not a correct purge.
>>
>> So how do I tell the developer (or do it myself) to correct the code so
>> the whole cache for a given website is purged?.
>
> Use the ban() function in your VCL in this case.
>
> Dridi
>

I currently have in default.vcl this:

# Allow purging from ACL
if (req.method == "PURGE") {
# If not allowed then a error 405 is returned
if (!client.ip ~ purge) {
return(synth(405, "This IP is not allowed to
send PURGE requests."));
}
# If allowed, do a cache_lookup -> vlc_hit() or vlc_miss()
return (purge);
}

What do I have to change to use BAN? If I run ban at command line they
seem to work.

Do I need to change the Wordpress plugin to send BAN requests?

Thanks!

Miguel

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Re: troubleshooting Varnish purges vs bans [ In reply to ]

miguel_3_gonzalez at yahoo

Aug 16, 2017, 3:13 AM

Post #10 of 11 (1780 views)

I have found out how to make to work varnish http purge plugin with
Varnish 4.x in this article (funny it´s on the Varnish blog)

https://info.varnish-software.com/blog/step-step-speed-wordpress-varnish-software?success=true

I have to say If I added the snippet mentioned:

if (req.http.X-Purge-Method == "regex") {

ban("req.url ~ " + req.url + " && req.http.host ~ " + req.http.host);

return (synth(200, "Banned."));

} else {

return (purge);

}

It didn´t work out of the box. I got the following VCL_error:

- VCL_Error ban(): Expected && between conditions, found "&&"

I had to change it to:

if (req.http.X-Purge-Method == "regex") {

ban("req.url ~ " + req.url + " && req.http.host ~ " + req.http.host);

return (synth(200, "Banned."));

} else {

return (purge);

}

So maybe for other users using Wordpress with Varnish it might be useful

Regards,

Miguel

On 08/15/17 1:07 PM, Miguel González wrote:
> On 08/11/17 12:22 PM, Dridi Boukelmoune wrote:
>>> The Wordpress plugin is supposed to clean the whole cache for a website
>>> when clicking on a button saying "Clear cache".
>>>
>>> You say from the varnishlog excerpt I sent that´s not the correct way
>>> of doing so, or apparently what is logged by varnish is not a correct purge.
>>>
>>> So how do I tell the developer (or do it myself) to correct the code so
>>> the whole cache for a given website is purged?.
>>
>> Use the ban() function in your VCL in this case.
>>
>> Dridi
>>
>
> I currently have in default.vcl this:
>
> # Allow purging from ACL
> if (req.method == "PURGE") {
> # If not allowed then a error 405 is returned
> if (!client.ip ~ purge) {
> return(synth(405, "This IP is not allowed to
> send PURGE requests."));
> }
> # If allowed, do a cache_lookup -> vlc_hit() or vlc_miss()
> return (purge);
> }
>
>
> What do I have to change to use BAN? If I run ban at command line they
> seem to work.
>
> Do I need to change the Wordpress plugin to send BAN requests?
>
> Thanks!
>
> Miguel
>

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Re: troubleshooting Varnish purges vs bans [ In reply to ]

Aug 21, 2017, 12:00 AM

Post #11 of 11 (1770 views)

On Wed, Aug 16, 2017 at 12:13 PM, Miguel González
<miguel_3_gonzalez@yahoo.es> wrote:
> I have found out how to make to work varnish http purge plugin with
> Varnish 4.x in this article (funny it´s on the Varnish blog)

Thanks for letting us know, and sorry for not replying earlier!

Dridi

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc