Mailing List Archive

Varnish doesnt purge
Hi,
please help me to solve this problem, this is the link that i explaine my question completely:
https://varnish-cache.org/lists/pipermail/varnish-misc/2019-December/026754.html
Re: Varnish doesnt purge [ In reply to ]
Yes it does, we have already proved it./

Each instance purges and will refetch the object for its respective
backend. I explained how to test: start with the origin, change the data
and check that you actually get the new object. Then add a varnish layer,
purge, check, repeat.

You are testing everything at once and are unable to isolate the issue
because you refuse to take things step by step, and spamming won't change
that.

I'm sorry but until you have tested each layer individually and identified
the one that fails, I won't be able to help.

--
Guillaume Quintard


On Wed, Dec 11, 2019 at 3:08 PM hamidreza hosseini <hrhosseini@hotmail.com>
wrote:

> Hi,
> please help me to solve this problem, this is the link that i explaine my
> question completely:
>
> https://varnish-cache.org/lists/pipermail/varnish-misc/2019-December/026754.html
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
Re: Varnish doesnt purge [ In reply to ]
I do all steps now again, this is the resault:
STEP ONE:
First of all i configure a varnish (with file backend) in front of swift/Nginx and i ask the url that exist in swift with curl,

varnish-file$curl -I http://192.168.200.12:8080/Naserfeiz.mp4 -H "newtrack: yes"

HTTP/1.1 200 OK
Server: nginx
Date: Thu, 12 Dec 2019 07:29:32 GMT
Content-Type: text/plain
Content-Length: 23521499
Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
ETag: "3a1794b0-166e8db"
X-Varnish: 2
Age: 0
Via: 1.1 varnish (Varnish/6.0)
Accept-Ranges: bytes
Connection: keep-alive


varnish cach that object and then i purge the url:

varnish-file$ curl -v -k -X PURGE http://192.168.200.12:8080/Naserfeiz.mp4 -o /dev/null -H "newtrack: yes"

* Trying 192.168.200.12...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.12 (192.168.200.12) port 8080 (#0)
> PURGE /Naserfeiz.mp4 HTTP/1.1
> Host: 192.168.200.12:8080
> User-Agent: curl/7.47.0
> Accept: */*
> newtrack: yes
>
< HTTP/1.1 200 Purged
< Date: Thu, 12 Dec 2019 07:29:16 GMT
< Server: Varnish
< X-Varnish: 32770
< Content-Type: text/html; charset=utf-8
< Retry-After: 5
< Content-Length: 240
< Accept-Ranges: bytes
< Connection: keep-alive
<
{ [240 bytes data]
100 240 100 240 0 0 124k 0 --:--:-- --:--:-- --:--:-- 234k
* Connection #0 to host 192.168.200.12 left intact

The object purged successfully and it didnt exist in cach anymore.

This is varnishlog resault:

varnish-file$ sudo varnishlog -d -g request
* << Request >> 2
- Begin req 1 rxreq
- Timestamp Start: 1576135743.442646 0.000000 0.000000
- Timestamp Req: 1576135743.442646 0.000000 0.000000
- ReqStart 192.168.200.12 5432 a0
- ReqMethod HEAD
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.12:8080
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.12
- VCL_call RECV
- VCL_return hash
- VCL_call HASH
- VCL_return lookup
- VCL_call MISS
- VCL_return fetch
- Link bereq 3 fetch
- Timestamp Fetch: 1576135743.444347 0.001701 0.001701
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespHeader Server: nginx
- RespHeader Date: Thu, 12 Dec 2019 07:29:32 GMT
- RespHeader Content-Type: text/plain
- RespHeader Content-Length: 23521499
- RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
- RespHeader ETag: "3a1794b0-166e8db"
- RespHeader X-Varnish: 2
- RespHeader Age: 0
- RespHeader Via: 1.1 varnish (Varnish/6.0)
- VCL_call DELIVER
- VCL_return deliver
- Timestamp Process: 1576135743.444382 0.001736 0.000036
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576135743.444474 0.001828 0.000092
- ReqAcct 112 0 112 295 0 295
- End
** << BeReq >> 3
-- Begin bereq 2 fetch
-- VCL_use boot
-- Timestamp Start: 1576135743.442807 0.000000 0.000000
-- BereqMethod HEAD
-- BereqURL /Naserfeiz.mp4
-- BereqProtocol HTTP/1.1
-- BereqHeader Host: 192.168.200.12:8080
-- BereqHeader User-Agent: curl/7.47.0
-- BereqHeader Accept: */*
-- BereqHeader newtrack: yes
-- BereqHeader X-Forwarded-For: 192.168.200.12
-- BereqMethod GET
-- BereqHeader Accept-Encoding: gzip
-- BereqHeader X-Varnish: 3
-- VCL_call BACKEND_FETCH
-- VCL_return fetch
-- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6846
-- BackendStart 192.168.200.11 8080
-- Timestamp Bereq: 1576135743.443586 0.000779 0.000779
-- Timestamp Beresp: 1576135743.444017 0.001210 0.000431
-- BerespProtocol HTTP/1.1
-- BerespStatus 200
-- BerespReason OK
-- BerespHeader Server: nginx
-- BerespHeader Date: Thu, 12 Dec 2019 07:29:32 GMT
-- BerespHeader Content-Type: text/plain
-- BerespHeader Content-Length: 23521499
-- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
-- BerespHeader Connection: keep-alive
-- BerespHeader ETag: "3a1794b0-166e8db"
-- BerespHeader Accept-Ranges: bytes
-- TTL RFC 120 10 0 1576135743 1576135743 1576135772 0 0 cacheable
-- VCL_call BACKEND_RESPONSE
-- TTL VCL 86400 10 0 1576135743 cacheable
-- VCL_return deliver
-- Storage file s0
-- Fetch_Body 3 length stream
-- BackendReuse 27 boot.swift_proxy_1
-- Timestamp BerespBody: 1576135743.747835 0.305028 0.303818
-- Length 23521499
-- BereqAcct 181 0 181 241 23521499 23521740
-- End

* << Request >> 32770
- Begin req 32769 rxreq
- Timestamp Start: 1576135756.680792 0.000000 0.000000
- Timestamp Req: 1576135756.680792 0.000000 0.000000
- ReqStart 192.168.200.12 5438 a0
- ReqMethod PURGE
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.12:8080
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.12
- VCL_call RECV
- VCL_return purge
- VCL_call HASH
- VCL_return lookup
- VCL_call PURGE
- VCL_return synth
- Timestamp Process: 1576135756.680982 0.000190 0.000190
- RespHeader Date: Thu, 12 Dec 2019 07:29:16 GMT
- RespHeader Server: Varnish
- RespHeader X-Varnish: 32770
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespReason Purged
- VCL_call SYNTH
- RespHeader Content-Type: text/html; charset=utf-8
- RespHeader Retry-After: 5
- VCL_return deliver
- RespHeader Content-Length: 240
- Storage malloc Transient
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576135756.681190 0.000399 0.000208
- ReqAcct 113 0 113 218 240 458
- End

* << Request >> 32772
- Begin req 32771 rxreq
- Timestamp Start: 1576135769.304679 0.000000 0.000000
- Timestamp Req: 1576135769.304679 0.000000 0.000000
- ReqStart 192.168.200.12 5440 a0
- ReqMethod HEAD
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.12:8080
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.12
- VCL_call RECV
- VCL_return hash
- VCL_call HASH
- VCL_return lookup
- VCL_call MISS
- VCL_return fetch
- Link bereq 32773 fetch
- Timestamp Fetch: 1576135769.306630 0.001951 0.001951
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespHeader Server: nginx
- RespHeader Date: Thu, 12 Dec 2019 07:29:58 GMT
- RespHeader Content-Type: text/plain
- RespHeader Content-Length: 23521499
- RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
- RespHeader ETag: "3a1794b0-166e8db"
- RespHeader X-Varnish: 32772
- RespHeader Age: 0
- RespHeader Via: 1.1 varnish (Varnish/6.0)
- VCL_call DELIVER
- VCL_return deliver
- Timestamp Process: 1576135769.306657 0.001978 0.000027
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576135769.306748 0.002069 0.000091
- ReqAcct 112 0 112 299 0 299
- End
** << BeReq >> 32773
-- Begin bereq 32772 fetch
-- VCL_use boot
-- Timestamp Start: 1576135769.304834 0.000000 0.000000
-- BereqMethod HEAD
-- BereqURL /Naserfeiz.mp4
-- BereqProtocol HTTP/1.1
-- BereqHeader Host: 192.168.200.12:8080
-- BereqHeader User-Agent: curl/7.47.0
-- BereqHeader Accept: */*
-- BereqHeader newtrack: yes
-- BereqHeader X-Forwarded-For: 192.168.200.12
-- BereqMethod GET
-- BereqHeader Accept-Encoding: gzip
-- BereqHeader X-Varnish: 32773
-- VCL_call BACKEND_FETCH
-- VCL_return fetch
-- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6846
-- BackendStart 192.168.200.11 8080
-- Timestamp Bereq: 1576135769.304993 0.000160 0.000160
-- Timestamp Beresp: 1576135769.306170 0.001336 0.001176
-- BerespProtocol HTTP/1.1
-- BerespStatus 200
-- BerespReason OK
-- BerespHeader Server: nginx
-- BerespHeader Date: Thu, 12 Dec 2019 07:29:58 GMT
-- BerespHeader Content-Type: text/plain
-- BerespHeader Content-Length: 23521499
-- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
-- BerespHeader Connection: keep-alive
-- BerespHeader ETag: "3a1794b0-166e8db"
-- BerespHeader Accept-Ranges: bytes
-- TTL RFC 120 10 0 1576135769 1576135769 1576135798 0 0 cacheable
-- VCL_call BACKEND_RESPONSE
-- TTL VCL 86400 10 0 1576135769 cacheable
-- VCL_return deliver
-- Storage file s0
-- Fetch_Body 3 length stream
-- BackendReuse 27 boot.swift_proxy_1
-- Timestamp BerespBody: 1576135769.645490 0.340657 0.339321
-- Length 23521499
-- BereqAcct 185 0 185 241 23521499 23521740
-- End


########

STEP TWO:
First i restart varnish file and varnishlog then:
I add another varnish (with ram backend) in fornt of the varnish file and then i ask the url again in varnish ram (because it is my frontend server and users can access media though this server):

varnish-ram$ curl -I http://192.168.200.13/Naserfeiz.mp4 -H "newtrack: yes"
HTTP/1.1 200 OK
Server: nginx
Date: Thu, 12 Dec 2019 07:33:05 GMT
Content-Type: text/plain
Content-Length: 23521499
Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
ETag: "3a1794b0-166e8db"
X-Varnish: 2
Via: 1.1 varnish (Varnish/6.0)
X-Varnish: 2
Age: 0
Via: 1.1 varnish-v4
Accept-Ranges: bytes
Connection: keep-alive


First varnish file cached the url and then Varnish ram fetched the object from varnish file and cached it , then i purged that url in varnish ram and it purged successfuly and it exactly removed from cached:

varnish-ram:~$ curl -v -k -X PURGE http://192.168.200.13/Naserfeiz.mp4 -o /dev/null -H "newtrack: yes"
* Trying 192.168.200.13...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.13 (192.168.200.13) port 80 (#0)
> PURGE /Naserfeiz.mp4 HTTP/1.1
> Host: 192.168.200.13
> User-Agent: curl/7.47.0
> Accept: */*
> newtrack: yes
>
< HTTP/1.1 200 Purged
< Date: Thu, 12 Dec 2019 07:33:46 GMT
< Server: Varnish
< X-Varnish: 32770
< Content-Type: text/html; charset=utf-8
< Retry-After: 5
< Content-Length: 240
< Accept-Ranges: bytes
< Connection: keep-alive
<
{ [240 bytes data]
100 240 100 240 0 0 73846 0 --:--:-- --:--:-- --:--:-- 117k
* Connection #0 to host 192.168.200.13 left intact


Then i purged the url in varnish file it shows "200 purged" but it still exist in varnish file!!!!!!:

varnish-file$ curl -v -k -X PURGE http://192.168.200.12:8080/Naserfeiz.mp4 -o /dev/null -H "newtrack: yes"
* Trying 192.168.200.12...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.12 (192.168.200.12) port 8080 (#0)
> PURGE /Naserfeiz.mp4 HTTP/1.1
> Host: 192.168.200.12:8080
> User-Agent: curl/7.47.0
> Accept: */*
> newtrack: yes
>
< HTTP/1.1 200 Purged
< Date: Thu, 12 Dec 2019 07:33:39 GMT
< Server: Varnish
< X-Varnish: 5
< Content-Type: text/html; charset=utf-8
< Retry-After: 5
< Content-Length: 236
< Accept-Ranges: bytes
< Connection: keep-alive
<
{ [236 bytes data]
100 236 100 236 0 0 110k 0 --:--:-- --:--:-- --:--:-- 230k
* Connection #0 to host 192.168.200.12 left intact


This is varnishlog resault in varnish file:

varnish-file:~$ sudo varnishlog -d -g request
* << Request >> 2
- Begin req 1 rxreq
- Timestamp Start: 1576135956.460768 0.000000 0.000000
- Timestamp Req: 1576135956.460768 0.000000 0.000000
- ReqStart 192.168.200.13 22300 a0
- ReqMethod GET
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.13
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.13
- ReqHeader Accept-Encoding: gzip
- ReqHeader X-Varnish: 3
- ReqUnset X-Forwarded-For: 192.168.200.13
- ReqHeader X-Forwarded-For: 192.168.200.13, 192.168.200.13
- VCL_call RECV
- VCL_return hash
- VCL_call HASH
- VCL_return lookup
- VCL_call MISS
- VCL_return fetch
- Link bereq 3 fetch
- Timestamp Fetch: 1576135956.463065 0.002297 0.002297
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespHeader Server: nginx
- RespHeader Date: Thu, 12 Dec 2019 07:33:05 GMT
- RespHeader Content-Type: text/plain
- RespHeader Content-Length: 23521499
- RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
- RespHeader ETag: "3a1794b0-166e8db"
- RespHeader X-Varnish: 2
- RespHeader Age: 0
- RespHeader Via: 1.1 varnish (Varnish/6.0)
- VCL_call DELIVER
- VCL_return deliver
- Timestamp Process: 1576135956.463114 0.002346 0.000049
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576135956.758920 0.298152 0.295806
- ReqAcct 176 0 176 295 23521499 23521794
- End
** << BeReq >> 3
-- Begin bereq 2 fetch
-- VCL_use boot
-- Timestamp Start: 1576135956.461099 0.000000 0.000000
-- BereqMethod GET
-- BereqURL /Naserfeiz.mp4
-- BereqProtocol HTTP/1.1
-- BereqHeader Host: 192.168.200.13
-- BereqHeader User-Agent: curl/7.47.0
-- BereqHeader Accept: */*
-- BereqHeader newtrack: yes
-- BereqHeader Accept-Encoding: gzip
-- BereqHeader X-Varnish: 3
-- BereqHeader X-Forwarded-For: 192.168.200.13, 192.168.200.13
-- BereqHeader X-Varnish: 3
-- VCL_call BACKEND_FETCH
-- VCL_return fetch
-- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6860
-- BackendStart 192.168.200.11 8080
-- Timestamp Bereq: 1576135956.461955 0.000856 0.000856
-- Timestamp Beresp: 1576135956.462607 0.001508 0.000652
-- BerespProtocol HTTP/1.1
-- BerespStatus 200
-- BerespReason OK
-- BerespHeader Server: nginx
-- BerespHeader Date: Thu, 12 Dec 2019 07:33:05 GMT
-- BerespHeader Content-Type: text/plain
-- BerespHeader Content-Length: 23521499
-- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
-- BerespHeader Connection: keep-alive
-- BerespHeader ETag: "3a1794b0-166e8db"
-- BerespHeader Accept-Ranges: bytes
-- TTL RFC 120 10 0 1576135956 1576135956 1576135985 0 0 cacheable
-- VCL_call BACKEND_RESPONSE
-- TTL VCL 86400 10 0 1576135956 cacheable
-- VCL_return deliver
-- Storage file s0
-- Fetch_Body 3 length stream
-- BackendReuse 27 boot.swift_proxy_1
-- Timestamp BerespBody: 1576135956.758956 0.297858 0.296349
-- Length 23521499
-- BereqAcct 206 0 206 241 23521499 23521740
-- End

* << Request >> 5
- Begin req 4 rxreq
- Timestamp Start: 1576136019.230666 0.000000 0.000000
- Timestamp Req: 1576136019.230666 0.000000 0.000000
- ReqStart 192.168.200.12 5452 a0
- ReqMethod PURGE
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.12:8080
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.12
- VCL_call RECV
- VCL_return purge
- VCL_call HASH
- VCL_return lookup
- VCL_call PURGE
- VCL_return synth
- Timestamp Process: 1576136019.230863 0.000198 0.000198
- RespHeader Date: Thu, 12 Dec 2019 07:33:39 GMT
- RespHeader Server: Varnish
- RespHeader X-Varnish: 5
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespReason Purged
- VCL_call SYNTH
- RespHeader Content-Type: text/html; charset=utf-8
- RespHeader Retry-After: 5
- VCL_return deliver
- RespHeader Content-Length: 236
- Storage malloc Transient
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576136019.231138 0.000473 0.000275
- ReqAcct 113 0 113 214 236 450
- End

########
PLEASE test this scenario for yourself and you can see that i do correct and the object wont remove from varnish-file
Please do this scenario for yourself

________________________________
From: Guillaume Quintard <guillaume@varnish-software.com>
Sent: Wednesday, December 11, 2019 9:44 AM
To: hamidreza hosseini <hrhosseini@hotmail.com>
Cc: varnish-misc@varnish-cache.org <varnish-misc@varnish-cache.org>
Subject: Re: Varnish doesnt purge

Yes it does, we have already proved it./

Each instance purges and will refetch the object for its respective backend. I explained how to test: start with the origin, change the data and check that you actually get the new object. Then add a varnish layer, purge, check, repeat.

You are testing everything at once and are unable to isolate the issue because you refuse to take things step by step, and spamming won't change that.

I'm sorry but until you have tested each layer individually and identified the one that fails, I won't be able to help.

--
Guillaume Quintard


On Wed, Dec 11, 2019 at 3:08 PM hamidreza hosseini <hrhosseini@hotmail.com<mailto:hrhosseini@hotmail.com>> wrote:
Hi,
please help me to solve this problem, this is the link that i explaine my question completely:
https://varnish-cache.org/lists/pipermail/varnish-misc/2019-December/026754.html

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org>
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish doesnt purge [ In reply to ]
Hi,
Please annwer my question, I sent you my test

________________________________
From: varnish-misc <varnish-misc-bounces+hrhosseini=hotmail.com@varnish-cache.org> on behalf of hamidreza hosseini <hrhosseini@hotmail.com>
Sent: Wednesday, December 11, 2019 11:49 PM
To: Guillaume Quintard <guillaume@varnish-software.com>
Cc: varnish-misc@varnish-cache.org <varnish-misc@varnish-cache.org>
Subject: Re: Varnish doesnt purge

I do all steps now again, this is the resault:
STEP ONE:
First of all i configure a varnish (with file backend) in front of swift/Nginx and i ask the url that exist in swift with curl,

varnish-file$curl -I http://192.168.200.12:8080/Naserfeiz.mp4 -H "newtrack: yes"

HTTP/1.1 200 OK
Server: nginx
Date: Thu, 12 Dec 2019 07:29:32 GMT
Content-Type: text/plain
Content-Length: 23521499
Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
ETag: "3a1794b0-166e8db"
X-Varnish: 2
Age: 0
Via: 1.1 varnish (Varnish/6.0)
Accept-Ranges: bytes
Connection: keep-alive


varnish cach that object and then i purge the url:

varnish-file$ curl -v -k -X PURGE http://192.168.200.12:8080/Naserfeiz.mp4 -o /dev/null -H "newtrack: yes"

* Trying 192.168.200.12...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.12 (192.168.200.12) port 8080 (#0)
> PURGE /Naserfeiz.mp4 HTTP/1.1
> Host: 192.168.200.12:8080
> User-Agent: curl/7.47.0
> Accept: */*
> newtrack: yes
>
< HTTP/1.1 200 Purged
< Date: Thu, 12 Dec 2019 07:29:16 GMT
< Server: Varnish
< X-Varnish: 32770
< Content-Type: text/html; charset=utf-8
< Retry-After: 5
< Content-Length: 240
< Accept-Ranges: bytes
< Connection: keep-alive
<
{ [240 bytes data]
100 240 100 240 0 0 124k 0 --:--:-- --:--:-- --:--:-- 234k
* Connection #0 to host 192.168.200.12 left intact

The object purged successfully and it didnt exist in cach anymore.

This is varnishlog resault:

varnish-file$ sudo varnishlog -d -g request
* << Request >> 2
- Begin req 1 rxreq
- Timestamp Start: 1576135743.442646 0.000000 0.000000
- Timestamp Req: 1576135743.442646 0.000000 0.000000
- ReqStart 192.168.200.12 5432 a0
- ReqMethod HEAD
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.12:8080
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.12
- VCL_call RECV
- VCL_return hash
- VCL_call HASH
- VCL_return lookup
- VCL_call MISS
- VCL_return fetch
- Link bereq 3 fetch
- Timestamp Fetch: 1576135743.444347 0.001701 0.001701
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespHeader Server: nginx
- RespHeader Date: Thu, 12 Dec 2019 07:29:32 GMT
- RespHeader Content-Type: text/plain
- RespHeader Content-Length: 23521499
- RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
- RespHeader ETag: "3a1794b0-166e8db"
- RespHeader X-Varnish: 2
- RespHeader Age: 0
- RespHeader Via: 1.1 varnish (Varnish/6.0)
- VCL_call DELIVER
- VCL_return deliver
- Timestamp Process: 1576135743.444382 0.001736 0.000036
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576135743.444474 0.001828 0.000092
- ReqAcct 112 0 112 295 0 295
- End
** << BeReq >> 3
-- Begin bereq 2 fetch
-- VCL_use boot
-- Timestamp Start: 1576135743.442807 0.000000 0.000000
-- BereqMethod HEAD
-- BereqURL /Naserfeiz.mp4
-- BereqProtocol HTTP/1.1
-- BereqHeader Host: 192.168.200.12:8080
-- BereqHeader User-Agent: curl/7.47.0
-- BereqHeader Accept: */*
-- BereqHeader newtrack: yes
-- BereqHeader X-Forwarded-For: 192.168.200.12
-- BereqMethod GET
-- BereqHeader Accept-Encoding: gzip
-- BereqHeader X-Varnish: 3
-- VCL_call BACKEND_FETCH
-- VCL_return fetch
-- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6846
-- BackendStart 192.168.200.11 8080
-- Timestamp Bereq: 1576135743.443586 0.000779 0.000779
-- Timestamp Beresp: 1576135743.444017 0.001210 0.000431
-- BerespProtocol HTTP/1.1
-- BerespStatus 200
-- BerespReason OK
-- BerespHeader Server: nginx
-- BerespHeader Date: Thu, 12 Dec 2019 07:29:32 GMT
-- BerespHeader Content-Type: text/plain
-- BerespHeader Content-Length: 23521499
-- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
-- BerespHeader Connection: keep-alive
-- BerespHeader ETag: "3a1794b0-166e8db"
-- BerespHeader Accept-Ranges: bytes
-- TTL RFC 120 10 0 1576135743 1576135743 1576135772 0 0 cacheable
-- VCL_call BACKEND_RESPONSE
-- TTL VCL 86400 10 0 1576135743 cacheable
-- VCL_return deliver
-- Storage file s0
-- Fetch_Body 3 length stream
-- BackendReuse 27 boot.swift_proxy_1
-- Timestamp BerespBody: 1576135743.747835 0.305028 0.303818
-- Length 23521499
-- BereqAcct 181 0 181 241 23521499 23521740
-- End

* << Request >> 32770
- Begin req 32769 rxreq
- Timestamp Start: 1576135756.680792 0.000000 0.000000
- Timestamp Req: 1576135756.680792 0.000000 0.000000
- ReqStart 192.168.200.12 5438 a0
- ReqMethod PURGE
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.12:8080
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.12
- VCL_call RECV
- VCL_return purge
- VCL_call HASH
- VCL_return lookup
- VCL_call PURGE
- VCL_return synth
- Timestamp Process: 1576135756.680982 0.000190 0.000190
- RespHeader Date: Thu, 12 Dec 2019 07:29:16 GMT
- RespHeader Server: Varnish
- RespHeader X-Varnish: 32770
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespReason Purged
- VCL_call SYNTH
- RespHeader Content-Type: text/html; charset=utf-8
- RespHeader Retry-After: 5
- VCL_return deliver
- RespHeader Content-Length: 240
- Storage malloc Transient
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576135756.681190 0.000399 0.000208
- ReqAcct 113 0 113 218 240 458
- End

* << Request >> 32772
- Begin req 32771 rxreq
- Timestamp Start: 1576135769.304679 0.000000 0.000000
- Timestamp Req: 1576135769.304679 0.000000 0.000000
- ReqStart 192.168.200.12 5440 a0
- ReqMethod HEAD
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.12:8080
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.12
- VCL_call RECV
- VCL_return hash
- VCL_call HASH
- VCL_return lookup
- VCL_call MISS
- VCL_return fetch
- Link bereq 32773 fetch
- Timestamp Fetch: 1576135769.306630 0.001951 0.001951
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespHeader Server: nginx
- RespHeader Date: Thu, 12 Dec 2019 07:29:58 GMT
- RespHeader Content-Type: text/plain
- RespHeader Content-Length: 23521499
- RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
- RespHeader ETag: "3a1794b0-166e8db"
- RespHeader X-Varnish: 32772
- RespHeader Age: 0
- RespHeader Via: 1.1 varnish (Varnish/6.0)
- VCL_call DELIVER
- VCL_return deliver
- Timestamp Process: 1576135769.306657 0.001978 0.000027
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576135769.306748 0.002069 0.000091
- ReqAcct 112 0 112 299 0 299
- End
** << BeReq >> 32773
-- Begin bereq 32772 fetch
-- VCL_use boot
-- Timestamp Start: 1576135769.304834 0.000000 0.000000
-- BereqMethod HEAD
-- BereqURL /Naserfeiz.mp4
-- BereqProtocol HTTP/1.1
-- BereqHeader Host: 192.168.200.12:8080
-- BereqHeader User-Agent: curl/7.47.0
-- BereqHeader Accept: */*
-- BereqHeader newtrack: yes
-- BereqHeader X-Forwarded-For: 192.168.200.12
-- BereqMethod GET
-- BereqHeader Accept-Encoding: gzip
-- BereqHeader X-Varnish: 32773
-- VCL_call BACKEND_FETCH
-- VCL_return fetch
-- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6846
-- BackendStart 192.168.200.11 8080
-- Timestamp Bereq: 1576135769.304993 0.000160 0.000160
-- Timestamp Beresp: 1576135769.306170 0.001336 0.001176
-- BerespProtocol HTTP/1.1
-- BerespStatus 200
-- BerespReason OK
-- BerespHeader Server: nginx
-- BerespHeader Date: Thu, 12 Dec 2019 07:29:58 GMT
-- BerespHeader Content-Type: text/plain
-- BerespHeader Content-Length: 23521499
-- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
-- BerespHeader Connection: keep-alive
-- BerespHeader ETag: "3a1794b0-166e8db"
-- BerespHeader Accept-Ranges: bytes
-- TTL RFC 120 10 0 1576135769 1576135769 1576135798 0 0 cacheable
-- VCL_call BACKEND_RESPONSE
-- TTL VCL 86400 10 0 1576135769 cacheable
-- VCL_return deliver
-- Storage file s0
-- Fetch_Body 3 length stream
-- BackendReuse 27 boot.swift_proxy_1
-- Timestamp BerespBody: 1576135769.645490 0.340657 0.339321
-- Length 23521499
-- BereqAcct 185 0 185 241 23521499 23521740
-- End


########

STEP TWO:
First i restart varnish file and varnishlog then:
I add another varnish (with ram backend) in fornt of the varnish file and then i ask the url again in varnish ram (because it is my frontend server and users can access media though this server):

varnish-ram$ curl -I http://192.168.200.13/Naserfeiz.mp4 -H "newtrack: yes"
HTTP/1.1 200 OK
Server: nginx
Date: Thu, 12 Dec 2019 07:33:05 GMT
Content-Type: text/plain
Content-Length: 23521499
Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
ETag: "3a1794b0-166e8db"
X-Varnish: 2
Via: 1.1 varnish (Varnish/6.0)
X-Varnish: 2
Age: 0
Via: 1.1 varnish-v4
Accept-Ranges: bytes
Connection: keep-alive


First varnish file cached the url and then Varnish ram fetched the object from varnish file and cached it , then i purged that url in varnish ram and it purged successfuly and it exactly removed from cached:

varnish-ram:~$ curl -v -k -X PURGE http://192.168.200.13/Naserfeiz.mp4 -o /dev/null -H "newtrack: yes"
* Trying 192.168.200.13...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.13 (192.168.200.13) port 80 (#0)
> PURGE /Naserfeiz.mp4 HTTP/1.1
> Host: 192.168.200.13
> User-Agent: curl/7.47.0
> Accept: */*
> newtrack: yes
>
< HTTP/1.1 200 Purged
< Date: Thu, 12 Dec 2019 07:33:46 GMT
< Server: Varnish
< X-Varnish: 32770
< Content-Type: text/html; charset=utf-8
< Retry-After: 5
< Content-Length: 240
< Accept-Ranges: bytes
< Connection: keep-alive
<
{ [240 bytes data]
100 240 100 240 0 0 73846 0 --:--:-- --:--:-- --:--:-- 117k
* Connection #0 to host 192.168.200.13 left intact


Then i purged the url in varnish file it shows "200 purged" but it still exist in varnish file!!!!!!:

varnish-file$ curl -v -k -X PURGE http://192.168.200.12:8080/Naserfeiz.mp4 -o /dev/null -H "newtrack: yes"
* Trying 192.168.200.12...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.12 (192.168.200.12) port 8080 (#0)
> PURGE /Naserfeiz.mp4 HTTP/1.1
> Host: 192.168.200.12:8080
> User-Agent: curl/7.47.0
> Accept: */*
> newtrack: yes
>
< HTTP/1.1 200 Purged
< Date: Thu, 12 Dec 2019 07:33:39 GMT
< Server: Varnish
< X-Varnish: 5
< Content-Type: text/html; charset=utf-8
< Retry-After: 5
< Content-Length: 236
< Accept-Ranges: bytes
< Connection: keep-alive
<
{ [236 bytes data]
100 236 100 236 0 0 110k 0 --:--:-- --:--:-- --:--:-- 230k
* Connection #0 to host 192.168.200.12 left intact


This is varnishlog resault in varnish file:

varnish-file:~$ sudo varnishlog -d -g request
* << Request >> 2
- Begin req 1 rxreq
- Timestamp Start: 1576135956.460768 0.000000 0.000000
- Timestamp Req: 1576135956.460768 0.000000 0.000000
- ReqStart 192.168.200.13 22300 a0
- ReqMethod GET
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.13
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.13
- ReqHeader Accept-Encoding: gzip
- ReqHeader X-Varnish: 3
- ReqUnset X-Forwarded-For: 192.168.200.13
- ReqHeader X-Forwarded-For: 192.168.200.13, 192.168.200.13
- VCL_call RECV
- VCL_return hash
- VCL_call HASH
- VCL_return lookup
- VCL_call MISS
- VCL_return fetch
- Link bereq 3 fetch
- Timestamp Fetch: 1576135956.463065 0.002297 0.002297
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespHeader Server: nginx
- RespHeader Date: Thu, 12 Dec 2019 07:33:05 GMT
- RespHeader Content-Type: text/plain
- RespHeader Content-Length: 23521499
- RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
- RespHeader ETag: "3a1794b0-166e8db"
- RespHeader X-Varnish: 2
- RespHeader Age: 0
- RespHeader Via: 1.1 varnish (Varnish/6.0)
- VCL_call DELIVER
- VCL_return deliver
- Timestamp Process: 1576135956.463114 0.002346 0.000049
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576135956.758920 0.298152 0.295806
- ReqAcct 176 0 176 295 23521499 23521794
- End
** << BeReq >> 3
-- Begin bereq 2 fetch
-- VCL_use boot
-- Timestamp Start: 1576135956.461099 0.000000 0.000000
-- BereqMethod GET
-- BereqURL /Naserfeiz.mp4
-- BereqProtocol HTTP/1.1
-- BereqHeader Host: 192.168.200.13
-- BereqHeader User-Agent: curl/7.47.0
-- BereqHeader Accept: */*
-- BereqHeader newtrack: yes
-- BereqHeader Accept-Encoding: gzip
-- BereqHeader X-Varnish: 3
-- BereqHeader X-Forwarded-For: 192.168.200.13, 192.168.200.13
-- BereqHeader X-Varnish: 3
-- VCL_call BACKEND_FETCH
-- VCL_return fetch
-- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6860
-- BackendStart 192.168.200.11 8080
-- Timestamp Bereq: 1576135956.461955 0.000856 0.000856
-- Timestamp Beresp: 1576135956.462607 0.001508 0.000652
-- BerespProtocol HTTP/1.1
-- BerespStatus 200
-- BerespReason OK
-- BerespHeader Server: nginx
-- BerespHeader Date: Thu, 12 Dec 2019 07:33:05 GMT
-- BerespHeader Content-Type: text/plain
-- BerespHeader Content-Length: 23521499
-- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
-- BerespHeader Connection: keep-alive
-- BerespHeader ETag: "3a1794b0-166e8db"
-- BerespHeader Accept-Ranges: bytes
-- TTL RFC 120 10 0 1576135956 1576135956 1576135985 0 0 cacheable
-- VCL_call BACKEND_RESPONSE
-- TTL VCL 86400 10 0 1576135956 cacheable
-- VCL_return deliver
-- Storage file s0
-- Fetch_Body 3 length stream
-- BackendReuse 27 boot.swift_proxy_1
-- Timestamp BerespBody: 1576135956.758956 0.297858 0.296349
-- Length 23521499
-- BereqAcct 206 0 206 241 23521499 23521740
-- End

* << Request >> 5
- Begin req 4 rxreq
- Timestamp Start: 1576136019.230666 0.000000 0.000000
- Timestamp Req: 1576136019.230666 0.000000 0.000000
- ReqStart 192.168.200.12 5452 a0
- ReqMethod PURGE
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.12:8080
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.12
- VCL_call RECV
- VCL_return purge
- VCL_call HASH
- VCL_return lookup
- VCL_call PURGE
- VCL_return synth
- Timestamp Process: 1576136019.230863 0.000198 0.000198
- RespHeader Date: Thu, 12 Dec 2019 07:33:39 GMT
- RespHeader Server: Varnish
- RespHeader X-Varnish: 5
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespReason Purged
- VCL_call SYNTH
- RespHeader Content-Type: text/html; charset=utf-8
- RespHeader Retry-After: 5
- VCL_return deliver
- RespHeader Content-Length: 236
- Storage malloc Transient
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576136019.231138 0.000473 0.000275
- ReqAcct 113 0 113 214 236 450
- End

########
PLEASE test this scenario for yourself and you can see that i do correct and the object wont remove from varnish-file
Please do this scenario for yourself

________________________________
From: Guillaume Quintard <guillaume@varnish-software.com>
Sent: Wednesday, December 11, 2019 9:44 AM
To: hamidreza hosseini <hrhosseini@hotmail.com>
Cc: varnish-misc@varnish-cache.org <varnish-misc@varnish-cache.org>
Subject: Re: Varnish doesnt purge

Yes it does, we have already proved it./

Each instance purges and will refetch the object for its respective backend. I explained how to test: start with the origin, change the data and check that you actually get the new object. Then add a varnish layer, purge, check, repeat.

You are testing everything at once and are unable to isolate the issue because you refuse to take things step by step, and spamming won't change that.

I'm sorry but until you have tested each layer individually and identified the one that fails, I won't be able to help.

--
Guillaume Quintard


On Wed, Dec 11, 2019 at 3:08 PM hamidreza hosseini <hrhosseini@hotmail.com<mailto:hrhosseini@hotmail.com>> wrote:
Hi,
please help me to solve this problem, this is the link that i explaine my question completely:
https://varnish-cache.org/lists/pipermail/varnish-misc/2019-December/026754.html

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org>
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish doesnt purge [ In reply to ]
I appreciate Guillaume’s diplomacy :)

Your issue (one of them at least) is that in your cascaded varnish setup
the hash key of the cached url, in both varnish storages, is the one you
use to originally access the asset. Your vcl hash routine might also be a
custom one.. that would be the next thing to look at if what I’m saying
here doesn’t fix it.

So, purge
http://192.168.200.13/Naserfeiz.mp4
and NOT
http://192.168.200.12:8080/Naserfeiz.mp4
when in the cascaded setup

Cheers

On Sat, 14 Dec 2019 at 09:16, hamidreza hosseini <hrhosseini@hotmail.com>
wrote:

> Hi,
> Please annwer my question, I sent you my test
>
> ------------------------------
> *From:* varnish-misc <varnish-misc-bounces+hrhosseini=
> hotmail.com@varnish-cache.org> on behalf of hamidreza hosseini <
> hrhosseini@hotmail.com>
> *Sent:* Wednesday, December 11, 2019 11:49 PM
> *To:* Guillaume Quintard <guillaume@varnish-software.com>
> *Cc:* varnish-misc@varnish-cache.org <varnish-misc@varnish-cache.org>
> *Subject:* Re: Varnish doesnt purge
>
> I do all steps now again, this is the resault:
> STEP ONE:
> First of all i configure a varnish (with file backend) in front of
> swift/Nginx and i ask the url that exist in swift with curl,
>
> varnish-file$curl -I http://192.168.200.12:8080/Naserfeiz.mp4 -H
> "newtrack: yes"
>
> HTTP/1.1 200 OK
> Server: nginx
> Date: Thu, 12 Dec 2019 07:29:32 GMT
> Content-Type: text/plain
> Content-Length: 23521499
> Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
> ETag: "3a1794b0-166e8db"
> X-Varnish: 2
> Age: 0
> Via: 1.1 varnish (Varnish/6.0)
> Accept-Ranges: bytes
> Connection: keep-alive
>
>
> varnish cach that object and then i purge the url:
>
> varnish-file$ curl -v -k -X PURGE http://192.168.200.12:8080/Naserfeiz.mp4
> -o /dev/null -H "newtrack: yes"
>
> * Trying 192.168.200.12...
> % Total % Received % Xferd Average Speed Time Time Time
> Current
> Dload Upload Total Spent Left
> Speed
> 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:--
> 0* Connected to 192.168.200.12 (192.168.200.12) port 8080 (#0)
> > PURGE /Naserfeiz.mp4 HTTP/1.1
> > Host: 192.168.200.12:8080
> > User-Agent: curl/7.47.0
> > Accept: */*
> > newtrack: yes
> >
> < HTTP/1.1 200 Purged
> < Date: Thu, 12 Dec 2019 07:29:16 GMT
> < Server: Varnish
> < X-Varnish: 32770
> < Content-Type: text/html; charset=utf-8
> < Retry-After: 5
> < Content-Length: 240
> < Accept-Ranges: bytes
> < Connection: keep-alive
> <
> { [240 bytes data]
> 100 240 100 240 0 0 124k 0 --:--:-- --:--:-- --:--:--
> 234k
> * Connection #0 to host 192.168.200.12 left intact
>
> The object purged successfully and it didnt exist in cach anymore.
>
> This is varnishlog resault:
>
> varnish-file$ sudo varnishlog -d -g request
> * << Request >> 2
> - Begin req 1 rxreq
> - Timestamp Start: 1576135743.442646 0.000000 0.000000
> - Timestamp Req: 1576135743.442646 0.000000 0.000000
> - ReqStart 192.168.200.12 5432 a0
> - ReqMethod HEAD
> - ReqURL /Naserfeiz.mp4
> - ReqProtocol HTTP/1.1
> - ReqHeader Host: 192.168.200.12:8080
> - ReqHeader User-Agent: curl/7.47.0
> - ReqHeader Accept: */*
> - ReqHeader newtrack: yes
> - ReqHeader X-Forwarded-For: 192.168.200.12
> - VCL_call RECV
> - VCL_return hash
> - VCL_call HASH
> - VCL_return lookup
> - VCL_call MISS
> - VCL_return fetch
> - Link bereq 3 fetch
> - Timestamp Fetch: 1576135743.444347 0.001701 0.001701
> - RespProtocol HTTP/1.1
> - RespStatus 200
> - RespReason OK
> - RespHeader Server: nginx
> - RespHeader Date: Thu, 12 Dec 2019 07:29:32 GMT
> - RespHeader Content-Type: text/plain
> - RespHeader Content-Length: 23521499
> - RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
> - RespHeader ETag: "3a1794b0-166e8db"
> - RespHeader X-Varnish: 2
> - RespHeader Age: 0
> - RespHeader Via: 1.1 varnish (Varnish/6.0)
> - VCL_call DELIVER
> - VCL_return deliver
> - Timestamp Process: 1576135743.444382 0.001736 0.000036
> - RespHeader Accept-Ranges: bytes
> - RespHeader Connection: keep-alive
> - Timestamp Resp: 1576135743.444474 0.001828 0.000092
> - ReqAcct 112 0 112 295 0 295
> - End
> ** << BeReq >> 3
> -- Begin bereq 2 fetch
> -- VCL_use boot
> -- Timestamp Start: 1576135743.442807 0.000000 0.000000
> -- BereqMethod HEAD
> -- BereqURL /Naserfeiz.mp4
> -- BereqProtocol HTTP/1.1
> -- BereqHeader Host: 192.168.200.12:8080
> -- BereqHeader User-Agent: curl/7.47.0
> -- BereqHeader Accept: */*
> -- BereqHeader newtrack: yes
> -- BereqHeader X-Forwarded-For: 192.168.200.12
> -- BereqMethod GET
> -- BereqHeader Accept-Encoding: gzip
> -- BereqHeader X-Varnish: 3
> -- VCL_call BACKEND_FETCH
> -- VCL_return fetch
> -- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080
> 192.168.200.12 6846
> -- BackendStart 192.168.200.11 8080
> -- Timestamp Bereq: 1576135743.443586 0.000779 0.000779
> -- Timestamp Beresp: 1576135743.444017 0.001210 0.000431
> -- BerespProtocol HTTP/1.1
> -- BerespStatus 200
> -- BerespReason OK
> -- BerespHeader Server: nginx
> -- BerespHeader Date: Thu, 12 Dec 2019 07:29:32 GMT
> -- BerespHeader Content-Type: text/plain
> -- BerespHeader Content-Length: 23521499
> -- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
> -- BerespHeader Connection: keep-alive
> -- BerespHeader ETag: "3a1794b0-166e8db"
> -- BerespHeader Accept-Ranges: bytes
> -- TTL RFC 120 10 0 1576135743 1576135743 1576135772 0 0
> cacheable
> -- VCL_call BACKEND_RESPONSE
> -- TTL VCL 86400 10 0 1576135743 cacheable
> -- VCL_return deliver
> -- Storage file s0
> -- Fetch_Body 3 length stream
> -- BackendReuse 27 boot.swift_proxy_1
> -- Timestamp BerespBody: 1576135743.747835 0.305028 0.303818
> -- Length 23521499
> -- BereqAcct 181 0 181 241 23521499 23521740
> -- End
>
> * << Request >> 32770
> - Begin req 32769 rxreq
> - Timestamp Start: 1576135756.680792 0.000000 0.000000
> - Timestamp Req: 1576135756.680792 0.000000 0.000000
> - ReqStart 192.168.200.12 5438 a0
> - ReqMethod PURGE
> - ReqURL /Naserfeiz.mp4
> - ReqProtocol HTTP/1.1
> - ReqHeader Host: 192.168.200.12:8080
> - ReqHeader User-Agent: curl/7.47.0
> - ReqHeader Accept: */*
> - ReqHeader newtrack: yes
> - ReqHeader X-Forwarded-For: 192.168.200.12
> - VCL_call RECV
> - VCL_return purge
> - VCL_call HASH
> - VCL_return lookup
> - VCL_call PURGE
> - VCL_return synth
> - Timestamp Process: 1576135756.680982 0.000190 0.000190
> - RespHeader Date: Thu, 12 Dec 2019 07:29:16 GMT
> - RespHeader Server: Varnish
> - RespHeader X-Varnish: 32770
> - RespProtocol HTTP/1.1
> - RespStatus 200
> - RespReason OK
> - RespReason Purged
> - VCL_call SYNTH
> - RespHeader Content-Type: text/html; charset=utf-8
> - RespHeader Retry-After: 5
> - VCL_return deliver
> - RespHeader Content-Length: 240
> - Storage malloc Transient
> - RespHeader Accept-Ranges: bytes
> - RespHeader Connection: keep-alive
> - Timestamp Resp: 1576135756.681190 0.000399 0.000208
> - ReqAcct 113 0 113 218 240 458
> - End
>
> * << Request >> 32772
> - Begin req 32771 rxreq
> - Timestamp Start: 1576135769.304679 0.000000 0.000000
> - Timestamp Req: 1576135769.304679 0.000000 0.000000
> - ReqStart 192.168.200.12 5440 a0
> - ReqMethod HEAD
> - ReqURL /Naserfeiz.mp4
> - ReqProtocol HTTP/1.1
> - ReqHeader Host: 192.168.200.12:8080
> - ReqHeader User-Agent: curl/7.47.0
> - ReqHeader Accept: */*
> - ReqHeader newtrack: yes
> - ReqHeader X-Forwarded-For: 192.168.200.12
> - VCL_call RECV
> - VCL_return hash
> - VCL_call HASH
> - VCL_return lookup
> - VCL_call MISS
> - VCL_return fetch
> - Link bereq 32773 fetch
> - Timestamp Fetch: 1576135769.306630 0.001951 0.001951
> - RespProtocol HTTP/1.1
> - RespStatus 200
> - RespReason OK
> - RespHeader Server: nginx
> - RespHeader Date: Thu, 12 Dec 2019 07:29:58 GMT
> - RespHeader Content-Type: text/plain
> - RespHeader Content-Length: 23521499
> - RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
> - RespHeader ETag: "3a1794b0-166e8db"
> - RespHeader X-Varnish: 32772
> - RespHeader Age: 0
> - RespHeader Via: 1.1 varnish (Varnish/6.0)
> - VCL_call DELIVER
> - VCL_return deliver
> - Timestamp Process: 1576135769.306657 0.001978 0.000027
> - RespHeader Accept-Ranges: bytes
> - RespHeader Connection: keep-alive
> - Timestamp Resp: 1576135769.306748 0.002069 0.000091
> - ReqAcct 112 0 112 299 0 299
> - End
> ** << BeReq >> 32773
> -- Begin bereq 32772 fetch
> -- VCL_use boot
> -- Timestamp Start: 1576135769.304834 0.000000 0.000000
> -- BereqMethod HEAD
> -- BereqURL /Naserfeiz.mp4
> -- BereqProtocol HTTP/1.1
> -- BereqHeader Host: 192.168.200.12:8080
> -- BereqHeader User-Agent: curl/7.47.0
> -- BereqHeader Accept: */*
> -- BereqHeader newtrack: yes
> -- BereqHeader X-Forwarded-For: 192.168.200.12
> -- BereqMethod GET
> -- BereqHeader Accept-Encoding: gzip
> -- BereqHeader X-Varnish: 32773
> -- VCL_call BACKEND_FETCH
> -- VCL_return fetch
> -- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080
> 192.168.200.12 6846
> -- BackendStart 192.168.200.11 8080
> -- Timestamp Bereq: 1576135769.304993 0.000160 0.000160
> -- Timestamp Beresp: 1576135769.306170 0.001336 0.001176
> -- BerespProtocol HTTP/1.1
> -- BerespStatus 200
> -- BerespReason OK
> -- BerespHeader Server: nginx
> -- BerespHeader Date: Thu, 12 Dec 2019 07:29:58 GMT
> -- BerespHeader Content-Type: text/plain
> -- BerespHeader Content-Length: 23521499
> -- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
> -- BerespHeader Connection: keep-alive
> -- BerespHeader ETag: "3a1794b0-166e8db"
> -- BerespHeader Accept-Ranges: bytes
> -- TTL RFC 120 10 0 1576135769 1576135769 1576135798 0 0
> cacheable
> -- VCL_call BACKEND_RESPONSE
> -- TTL VCL 86400 10 0 1576135769 cacheable
> -- VCL_return deliver
> -- Storage file s0
> -- Fetch_Body 3 length stream
> -- BackendReuse 27 boot.swift_proxy_1
> -- Timestamp BerespBody: 1576135769.645490 0.340657 0.339321
> -- Length 23521499
> -- BereqAcct 185 0 185 241 23521499 23521740
> -- End
>
>
> ########
>
> STEP TWO:
> First i restart varnish file and varnishlog then:
> I add another varnish (with ram backend) in fornt of the varnish file and
> then i ask the url again in varnish ram (because it is my frontend server
> and users can access media though this server):
>
> varnish-ram$ curl -I http://192.168.200.13/Naserfeiz.mp4 -H "newtrack:
> yes"
> HTTP/1.1 200 OK
> Server: nginx
> Date: Thu, 12 Dec 2019 07:33:05 GMT
> Content-Type: text/plain
> Content-Length: 23521499
> Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
> ETag: "3a1794b0-166e8db"
> X-Varnish: 2
> Via: 1.1 varnish (Varnish/6.0)
> X-Varnish: 2
> Age: 0
> Via: 1.1 varnish-v4
> Accept-Ranges: bytes
> Connection: keep-alive
>
>
> First varnish file cached the url and then Varnish ram fetched the object
> from varnish file and cached it , then i purged that url in varnish ram and
> it purged successfuly and it exactly removed from cached:
>
> varnish-ram:~$ curl -v -k -X PURGE http://192.168.200.13/Naserfeiz.mp4 -o
> /dev/null -H "newtrack: yes"
> * Trying 192.168.200.13...
> % Total % Received % Xferd Average Speed Time Time Time
> Current
> Dload Upload Total Spent Left
> Speed
> 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:--
> 0* Connected to 192.168.200.13 (192.168.200.13) port 80 (#0)
> > PURGE /Naserfeiz.mp4 HTTP/1.1
> > Host: 192.168.200.13
> > User-Agent: curl/7.47.0
> > Accept: */*
> > newtrack: yes
> >
> < HTTP/1.1 200 Purged
> < Date: Thu, 12 Dec 2019 07:33:46 GMT
> < Server: Varnish
> < X-Varnish: 32770
> < Content-Type: text/html; charset=utf-8
> < Retry-After: 5
> < Content-Length: 240
> < Accept-Ranges: bytes
> < Connection: keep-alive
> <
> { [240 bytes data]
> 100 240 100 240 0 0 73846 0 --:--:-- --:--:-- --:--:--
> 117k
> * Connection #0 to host 192.168.200.13 left intact
>
>
> Then i purged the url in varnish file it shows "200 purged" but it still
> exist in varnish file!!!!!!:
>
> varnish-file$ curl -v -k -X PURGE http://192.168.200.12:8080/Naserfeiz.mp4
> -o /dev/null -H "newtrack: yes"
> * Trying 192.168.200.12...
> % Total % Received % Xferd Average Speed Time Time Time
> Current
> Dload Upload Total Spent Left
> Speed
> 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:--
> 0* Connected to 192.168.200.12 (192.168.200.12) port 8080 (#0)
> > PURGE /Naserfeiz.mp4 HTTP/1.1
> > Host: 192.168.200.12:8080
> > User-Agent: curl/7.47.0
> > Accept: */*
> > newtrack: yes
> >
> < HTTP/1.1 200 Purged
> < Date: Thu, 12 Dec 2019 07:33:39 GMT
> < Server: Varnish
> < X-Varnish: 5
> < Content-Type: text/html; charset=utf-8
> < Retry-After: 5
> < Content-Length: 236
> < Accept-Ranges: bytes
> < Connection: keep-alive
> <
> { [236 bytes data]
> 100 236 100 236 0 0 110k 0 --:--:-- --:--:-- --:--:--
> 230k
> * Connection #0 to host 192.168.200.12 left intact
>
>
> This is varnishlog resault in varnish file:
>
> varnish-file:~$ sudo varnishlog -d -g request
> * << Request >> 2
> - Begin req 1 rxreq
> - Timestamp Start: 1576135956.460768 0.000000 0.000000
> - Timestamp Req: 1576135956.460768 0.000000 0.000000
> - ReqStart 192.168.200.13 22300 a0
> - ReqMethod GET
> - ReqURL /Naserfeiz.mp4
> - ReqProtocol HTTP/1.1
> - ReqHeader Host: 192.168.200.13
> - ReqHeader User-Agent: curl/7.47.0
> - ReqHeader Accept: */*
> - ReqHeader newtrack: yes
> - ReqHeader X-Forwarded-For: 192.168.200.13
> - ReqHeader Accept-Encoding: gzip
> - ReqHeader X-Varnish: 3
> - ReqUnset X-Forwarded-For: 192.168.200.13
> - ReqHeader X-Forwarded-For: 192.168.200.13, 192.168.200.13
> - VCL_call RECV
> - VCL_return hash
> - VCL_call HASH
> - VCL_return lookup
> - VCL_call MISS
> - VCL_return fetch
> - Link bereq 3 fetch
> - Timestamp Fetch: 1576135956.463065 0.002297 0.002297
> - RespProtocol HTTP/1.1
> - RespStatus 200
> - RespReason OK
> - RespHeader Server: nginx
> - RespHeader Date: Thu, 12 Dec 2019 07:33:05 GMT
> - RespHeader Content-Type: text/plain
> - RespHeader Content-Length: 23521499
> - RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
> - RespHeader ETag: "3a1794b0-166e8db"
> - RespHeader X-Varnish: 2
> - RespHeader Age: 0
> - RespHeader Via: 1.1 varnish (Varnish/6.0)
> - VCL_call DELIVER
> - VCL_return deliver
> - Timestamp Process: 1576135956.463114 0.002346 0.000049
> - RespHeader Accept-Ranges: bytes
> - RespHeader Connection: keep-alive
> - Timestamp Resp: 1576135956.758920 0.298152 0.295806
> - ReqAcct 176 0 176 295 23521499 23521794
> - End
> ** << BeReq >> 3
> -- Begin bereq 2 fetch
> -- VCL_use boot
> -- Timestamp Start: 1576135956.461099 0.000000 0.000000
> -- BereqMethod GET
> -- BereqURL /Naserfeiz.mp4
> -- BereqProtocol HTTP/1.1
> -- BereqHeader Host: 192.168.200.13
> -- BereqHeader User-Agent: curl/7.47.0
> -- BereqHeader Accept: */*
> -- BereqHeader newtrack: yes
> -- BereqHeader Accept-Encoding: gzip
> -- BereqHeader X-Varnish: 3
> -- BereqHeader X-Forwarded-For: 192.168.200.13, 192.168.200.13
> -- BereqHeader X-Varnish: 3
> -- VCL_call BACKEND_FETCH
> -- VCL_return fetch
> -- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080
> 192.168.200.12 6860
> -- BackendStart 192.168.200.11 8080
> -- Timestamp Bereq: 1576135956.461955 0.000856 0.000856
> -- Timestamp Beresp: 1576135956.462607 0.001508 0.000652
> -- BerespProtocol HTTP/1.1
> -- BerespStatus 200
> -- BerespReason OK
> -- BerespHeader Server: nginx
> -- BerespHeader Date: Thu, 12 Dec 2019 07:33:05 GMT
> -- BerespHeader Content-Type: text/plain
> -- BerespHeader Content-Length: 23521499
> -- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
> -- BerespHeader Connection: keep-alive
> -- BerespHeader ETag: "3a1794b0-166e8db"
> -- BerespHeader Accept-Ranges: bytes
> -- TTL RFC 120 10 0 1576135956 1576135956 1576135985 0 0
> cacheable
> -- VCL_call BACKEND_RESPONSE
> -- TTL VCL 86400 10 0 1576135956 cacheable
> -- VCL_return deliver
> -- Storage file s0
> -- Fetch_Body 3 length stream
> -- BackendReuse 27 boot.swift_proxy_1
> -- Timestamp BerespBody: 1576135956.758956 0.297858 0.296349
> -- Length 23521499
> -- BereqAcct 206 0 206 241 23521499 23521740
> -- End
>
> * << Request >> 5
> - Begin req 4 rxreq
> - Timestamp Start: 1576136019.230666 0.000000 0.000000
> - Timestamp Req: 1576136019.230666 0.000000 0.000000
> - ReqStart 192.168.200.12 5452 a0
> - ReqMethod PURGE
> - ReqURL /Naserfeiz.mp4
> - ReqProtocol HTTP/1.1
> - ReqHeader Host: 192.168.200.12:8080
> - ReqHeader User-Agent: curl/7.47.0
> - ReqHeader Accept: */*
> - ReqHeader newtrack: yes
> - ReqHeader X-Forwarded-For: 192.168.200.12
> - VCL_call RECV
> - VCL_return purge
> - VCL_call HASH
> - VCL_return lookup
> - VCL_call PURGE
> - VCL_return synth
> - Timestamp Process: 1576136019.230863 0.000198 0.000198
> - RespHeader Date: Thu, 12 Dec 2019 07:33:39 GMT
> - RespHeader Server: Varnish
> - RespHeader X-Varnish: 5
> - RespProtocol HTTP/1.1
> - RespStatus 200
> - RespReason OK
> - RespReason Purged
> - VCL_call SYNTH
> - RespHeader Content-Type: text/html; charset=utf-8
> - RespHeader Retry-After: 5
> - VCL_return deliver
> - RespHeader Content-Length: 236
> - Storage malloc Transient
> - RespHeader Accept-Ranges: bytes
> - RespHeader Connection: keep-alive
> - Timestamp Resp: 1576136019.231138 0.000473 0.000275
> - ReqAcct 113 0 113 214 236 450
> - End
>
> ########
> PLEASE test this scenario for yourself and you can see that i do correct
> and the object wont remove from varnish-file
> Please do this scenario for yourself
>
> ------------------------------
> *From:* Guillaume Quintard <guillaume@varnish-software.com>
> *Sent:* Wednesday, December 11, 2019 9:44 AM
> *To:* hamidreza hosseini <hrhosseini@hotmail.com>
> *Cc:* varnish-misc@varnish-cache.org <varnish-misc@varnish-cache.org>
> *Subject:* Re: Varnish doesnt purge
>
> Yes it does, we have already proved it./
>
> Each instance purges and will refetch the object for its respective
> backend. I explained how to test: start with the origin, change the data
> and check that you actually get the new object. Then add a varnish layer,
> purge, check, repeat.
>
> You are testing everything at once and are unable to isolate the issue
> because you refuse to take things step by step, and spamming won't change
> that.
>
> I'm sorry but until you have tested each layer individually and identified
> the one that fails, I won't be able to help.
>
> --
> Guillaume Quintard
>
>
> On Wed, Dec 11, 2019 at 3:08 PM hamidreza hosseini <hrhosseini@hotmail.com>
> wrote:
>
> Hi,
> please help me to solve this problem, this is the link that i explaine my
> question completely:
>
> https://varnish-cache.org/lists/pipermail/varnish-misc/2019-December/026754.html
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
--
Sent from my iPhone
Re: Varnish doesnt purge [ In reply to ]
I dont understand your message,

I cant solve this problem

Instead of saying "appericiate to your diplomacy!!!!!!!!!!" you can help me!

I dont have enough knoledge on Varnish,

I dont get your text about hash, i didnt use hash.

Best Regards


________________________________
From: Vlad Rusu <vlad.rusu@lola.tech>
Sent: Sunday, December 15, 2019 7:48:42 PM
To: hamidreza hosseini <hrhosseini@hotmail.com>
Cc: Guillaume Quintard <guillaume@varnish-software.com>; varnish-misc@varnish-cache.org <varnish-misc@varnish-cache.org>
Subject: Re: Varnish doesnt purge

I appreciate Guillaume?s diplomacy :)

Your issue (one of them at least) is that in your cascaded varnish setup the hash key of the cached url, in both varnish storages, is the one you use to originally access the asset. Your vcl hash routine might also be a custom one.. that would be the next thing to look at if what I?m saying here doesn?t fix it.

So, purge
http://192.168.200.13/Naserfeiz.mp4
and NOT
http://192.168.200.12:8080/Naserfeiz.mp4
when in the cascaded setup

Cheers

On Sat, 14 Dec 2019 at 09:16, hamidreza hosseini <hrhosseini@hotmail.com<mailto:hrhosseini@hotmail.com>> wrote:
Hi,
Please annwer my question, I sent you my test

________________________________
From: varnish-misc <varnish-misc-bounces+hrhosseini=hotmail.com@varnish-cache.org<mailto:hotmail.com@varnish-cache.org>> on behalf of hamidreza hosseini <hrhosseini@hotmail.com<mailto:hrhosseini@hotmail.com>>
Sent: Wednesday, December 11, 2019 11:49 PM
To: Guillaume Quintard <guillaume@varnish-software.com<mailto:guillaume@varnish-software.com>>
Cc: varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org> <varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org>>
Subject: Re: Varnish doesnt purge

I do all steps now again, this is the resault:
STEP ONE:
First of all i configure a varnish (with file backend) in front of swift/Nginx and i ask the url that exist in swift with curl,

varnish-file$curl -I http://192.168.200.12:8080/Naserfeiz.mp4 -H "newtrack: yes"

HTTP/1.1 200 OK
Server: nginx
Date: Thu, 12 Dec 2019 07:29:32 GMT
Content-Type: text/plain
Content-Length: 23521499
Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
ETag: "3a1794b0-166e8db"
X-Varnish: 2
Age: 0
Via: 1.1 varnish (Varnish/6.0)
Accept-Ranges: bytes
Connection: keep-alive


varnish cach that object and then i purge the url:

varnish-file$ curl -v -k -X PURGE http://192.168.200.12:8080/Naserfeiz.mp4 -o /dev/null -H "newtrack: yes"

* Trying 192.168.200.12...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.12 (192.168.200.12) port 8080 (#0)
> PURGE /Naserfeiz.mp4 HTTP/1.1
> Host: 192.168.200.12:8080<http://192.168.200.12:8080>
> User-Agent: curl/7.47.0
> Accept: */*
> newtrack: yes
>
< HTTP/1.1 200 Purged
< Date: Thu, 12 Dec 2019 07:29:16 GMT
< Server: Varnish
< X-Varnish: 32770
< Content-Type: text/html; charset=utf-8
< Retry-After: 5
< Content-Length: 240
< Accept-Ranges: bytes
< Connection: keep-alive
<
{ [240 bytes data]
100 240 100 240 0 0 124k 0 --:--:-- --:--:-- --:--:-- 234k
* Connection #0 to host 192.168.200.12 left intact

The object purged successfully and it didnt exist in cach anymore.

This is varnishlog resault:

varnish-file$ sudo varnishlog -d -g request
* << Request >> 2
- Begin req 1 rxreq
- Timestamp Start: 1576135743.442646 0.000000 0.000000
- Timestamp Req: 1576135743.442646 0.000000 0.000000
- ReqStart 192.168.200.12 5432 a0
- ReqMethod HEAD
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.12:8080<http://192.168.200.12:8080>
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.12
- VCL_call RECV
- VCL_return hash
- VCL_call HASH
- VCL_return lookup
- VCL_call MISS
- VCL_return fetch
- Link bereq 3 fetch
- Timestamp Fetch: 1576135743.444347 0.001701 0.001701
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespHeader Server: nginx
- RespHeader Date: Thu, 12 Dec 2019 07:29:32 GMT
- RespHeader Content-Type: text/plain
- RespHeader Content-Length: 23521499
- RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
- RespHeader ETag: "3a1794b0-166e8db"
- RespHeader X-Varnish: 2
- RespHeader Age: 0
- RespHeader Via: 1.1 varnish (Varnish/6.0)
- VCL_call DELIVER
- VCL_return deliver
- Timestamp Process: 1576135743.444382 0.001736 0.000036
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576135743.444474 0.001828 0.000092
- ReqAcct 112 0 112 295 0 295
- End
** << BeReq >> 3
-- Begin bereq 2 fetch
-- VCL_use boot
-- Timestamp Start: 1576135743.442807 0.000000 0.000000
-- BereqMethod HEAD
-- BereqURL /Naserfeiz.mp4
-- BereqProtocol HTTP/1.1
-- BereqHeader Host: 192.168.200.12:8080<http://192.168.200.12:8080>
-- BereqHeader User-Agent: curl/7.47.0
-- BereqHeader Accept: */*
-- BereqHeader newtrack: yes
-- BereqHeader X-Forwarded-For: 192.168.200.12
-- BereqMethod GET
-- BereqHeader Accept-Encoding: gzip
-- BereqHeader X-Varnish: 3
-- VCL_call BACKEND_FETCH
-- VCL_return fetch
-- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6846
-- BackendStart 192.168.200.11 8080
-- Timestamp Bereq: 1576135743.443586 0.000779 0.000779
-- Timestamp Beresp: 1576135743.444017 0.001210 0.000431
-- BerespProtocol HTTP/1.1
-- BerespStatus 200
-- BerespReason OK
-- BerespHeader Server: nginx
-- BerespHeader Date: Thu, 12 Dec 2019 07:29:32 GMT
-- BerespHeader Content-Type: text/plain
-- BerespHeader Content-Length: 23521499
-- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
-- BerespHeader Connection: keep-alive
-- BerespHeader ETag: "3a1794b0-166e8db"
-- BerespHeader Accept-Ranges: bytes
-- TTL RFC 120 10 0 1576135743 1576135743 1576135772 0 0 cacheable
-- VCL_call BACKEND_RESPONSE
-- TTL VCL 86400 10 0 1576135743 cacheable
-- VCL_return deliver
-- Storage file s0
-- Fetch_Body 3 length stream
-- BackendReuse 27 boot.swift_proxy_1
-- Timestamp BerespBody: 1576135743.747835 0.305028 0.303818
-- Length 23521499
-- BereqAcct 181 0 181 241 23521499 23521740
-- End

* << Request >> 32770
- Begin req 32769 rxreq
- Timestamp Start: 1576135756.680792 0.000000 0.000000
- Timestamp Req: 1576135756.680792 0.000000 0.000000
- ReqStart 192.168.200.12 5438 a0
- ReqMethod PURGE
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.12:8080<http://192.168.200.12:8080>
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.12
- VCL_call RECV
- VCL_return purge
- VCL_call HASH
- VCL_return lookup
- VCL_call PURGE
- VCL_return synth
- Timestamp Process: 1576135756.680982 0.000190 0.000190
- RespHeader Date: Thu, 12 Dec 2019 07:29:16 GMT
- RespHeader Server: Varnish
- RespHeader X-Varnish: 32770
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespReason Purged
- VCL_call SYNTH
- RespHeader Content-Type: text/html; charset=utf-8
- RespHeader Retry-After: 5
- VCL_return deliver
- RespHeader Content-Length: 240
- Storage malloc Transient
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576135756.681190 0.000399 0.000208
- ReqAcct 113 0 113 218 240 458
- End

* << Request >> 32772
- Begin req 32771 rxreq
- Timestamp Start: 1576135769.304679 0.000000 0.000000
- Timestamp Req: 1576135769.304679 0.000000 0.000000
- ReqStart 192.168.200.12 5440 a0
- ReqMethod HEAD
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.12:8080<http://192.168.200.12:8080>
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.12
- VCL_call RECV
- VCL_return hash
- VCL_call HASH
- VCL_return lookup
- VCL_call MISS
- VCL_return fetch
- Link bereq 32773 fetch
- Timestamp Fetch: 1576135769.306630 0.001951 0.001951
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespHeader Server: nginx
- RespHeader Date: Thu, 12 Dec 2019 07:29:58 GMT
- RespHeader Content-Type: text/plain
- RespHeader Content-Length: 23521499
- RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
- RespHeader ETag: "3a1794b0-166e8db"
- RespHeader X-Varnish: 32772
- RespHeader Age: 0
- RespHeader Via: 1.1 varnish (Varnish/6.0)
- VCL_call DELIVER
- VCL_return deliver
- Timestamp Process: 1576135769.306657 0.001978 0.000027
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576135769.306748 0.002069 0.000091
- ReqAcct 112 0 112 299 0 299
- End
** << BeReq >> 32773
-- Begin bereq 32772 fetch
-- VCL_use boot
-- Timestamp Start: 1576135769.304834 0.000000 0.000000
-- BereqMethod HEAD
-- BereqURL /Naserfeiz.mp4
-- BereqProtocol HTTP/1.1
-- BereqHeader Host: 192.168.200.12:8080<http://192.168.200.12:8080>
-- BereqHeader User-Agent: curl/7.47.0
-- BereqHeader Accept: */*
-- BereqHeader newtrack: yes
-- BereqHeader X-Forwarded-For: 192.168.200.12
-- BereqMethod GET
-- BereqHeader Accept-Encoding: gzip
-- BereqHeader X-Varnish: 32773
-- VCL_call BACKEND_FETCH
-- VCL_return fetch
-- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6846
-- BackendStart 192.168.200.11 8080
-- Timestamp Bereq: 1576135769.304993 0.000160 0.000160
-- Timestamp Beresp: 1576135769.306170 0.001336 0.001176
-- BerespProtocol HTTP/1.1
-- BerespStatus 200
-- BerespReason OK
-- BerespHeader Server: nginx
-- BerespHeader Date: Thu, 12 Dec 2019 07:29:58 GMT
-- BerespHeader Content-Type: text/plain
-- BerespHeader Content-Length: 23521499
-- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
-- BerespHeader Connection: keep-alive
-- BerespHeader ETag: "3a1794b0-166e8db"
-- BerespHeader Accept-Ranges: bytes
-- TTL RFC 120 10 0 1576135769 1576135769 1576135798 0 0 cacheable
-- VCL_call BACKEND_RESPONSE
-- TTL VCL 86400 10 0 1576135769 cacheable
-- VCL_return deliver
-- Storage file s0
-- Fetch_Body 3 length stream
-- BackendReuse 27 boot.swift_proxy_1
-- Timestamp BerespBody: 1576135769.645490 0.340657 0.339321
-- Length 23521499
-- BereqAcct 185 0 185 241 23521499 23521740
-- End


########

STEP TWO:
First i restart varnish file and varnishlog then:
I add another varnish (with ram backend) in fornt of the varnish file and then i ask the url again in varnish ram (because it is my frontend server and users can access media though this server):

varnish-ram$ curl -I http://192.168.200.13/Naserfeiz.mp4 -H "newtrack: yes"
HTTP/1.1 200 OK
Server: nginx
Date: Thu, 12 Dec 2019 07:33:05 GMT
Content-Type: text/plain
Content-Length: 23521499
Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
ETag: "3a1794b0-166e8db"
X-Varnish: 2
Via: 1.1 varnish (Varnish/6.0)
X-Varnish: 2
Age: 0
Via: 1.1 varnish-v4
Accept-Ranges: bytes
Connection: keep-alive


First varnish file cached the url and then Varnish ram fetched the object from varnish file and cached it , then i purged that url in varnish ram and it purged successfuly and it exactly removed from cached:

varnish-ram:~$ curl -v -k -X PURGE http://192.168.200.13/Naserfeiz.mp4 -o /dev/null -H "newtrack: yes"
* Trying 192.168.200.13...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.13 (192.168.200.13) port 80 (#0)
> PURGE /Naserfeiz.mp4 HTTP/1.1
> Host: 192.168.200.13
> User-Agent: curl/7.47.0
> Accept: */*
> newtrack: yes
>
< HTTP/1.1 200 Purged
< Date: Thu, 12 Dec 2019 07:33:46 GMT
< Server: Varnish
< X-Varnish: 32770
< Content-Type: text/html; charset=utf-8
< Retry-After: 5
< Content-Length: 240
< Accept-Ranges: bytes
< Connection: keep-alive
<
{ [240 bytes data]
100 240 100 240 0 0 73846 0 --:--:-- --:--:-- --:--:-- 117k
* Connection #0 to host 192.168.200.13 left intact


Then i purged the url in varnish file it shows "200 purged" but it still exist in varnish file!!!!!!:

varnish-file$ curl -v -k -X PURGE http://192.168.200.12:8080/Naserfeiz.mp4 -o /dev/null -H "newtrack: yes"
* Trying 192.168.200.12...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.12 (192.168.200.12) port 8080 (#0)
> PURGE /Naserfeiz.mp4 HTTP/1.1
> Host: 192.168.200.12:8080<http://192.168.200.12:8080>
> User-Agent: curl/7.47.0
> Accept: */*
> newtrack: yes
>
< HTTP/1.1 200 Purged
< Date: Thu, 12 Dec 2019 07:33:39 GMT
< Server: Varnish
< X-Varnish: 5
< Content-Type: text/html; charset=utf-8
< Retry-After: 5
< Content-Length: 236
< Accept-Ranges: bytes
< Connection: keep-alive
<
{ [236 bytes data]
100 236 100 236 0 0 110k 0 --:--:-- --:--:-- --:--:-- 230k
* Connection #0 to host 192.168.200.12 left intact


This is varnishlog resault in varnish file:

varnish-file:~$ sudo varnishlog -d -g request
* << Request >> 2
- Begin req 1 rxreq
- Timestamp Start: 1576135956.460768 0.000000 0.000000
- Timestamp Req: 1576135956.460768 0.000000 0.000000
- ReqStart 192.168.200.13 22300 a0
- ReqMethod GET
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.13
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.13
- ReqHeader Accept-Encoding: gzip
- ReqHeader X-Varnish: 3
- ReqUnset X-Forwarded-For: 192.168.200.13
- ReqHeader X-Forwarded-For: 192.168.200.13, 192.168.200.13
- VCL_call RECV
- VCL_return hash
- VCL_call HASH
- VCL_return lookup
- VCL_call MISS
- VCL_return fetch
- Link bereq 3 fetch
- Timestamp Fetch: 1576135956.463065 0.002297 0.002297
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespHeader Server: nginx
- RespHeader Date: Thu, 12 Dec 2019 07:33:05 GMT
- RespHeader Content-Type: text/plain
- RespHeader Content-Length: 23521499
- RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
- RespHeader ETag: "3a1794b0-166e8db"
- RespHeader X-Varnish: 2
- RespHeader Age: 0
- RespHeader Via: 1.1 varnish (Varnish/6.0)
- VCL_call DELIVER
- VCL_return deliver
- Timestamp Process: 1576135956.463114 0.002346 0.000049
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576135956.758920 0.298152 0.295806
- ReqAcct 176 0 176 295 23521499 23521794
- End
** << BeReq >> 3
-- Begin bereq 2 fetch
-- VCL_use boot
-- Timestamp Start: 1576135956.461099 0.000000 0.000000
-- BereqMethod GET
-- BereqURL /Naserfeiz.mp4
-- BereqProtocol HTTP/1.1
-- BereqHeader Host: 192.168.200.13
-- BereqHeader User-Agent: curl/7.47.0
-- BereqHeader Accept: */*
-- BereqHeader newtrack: yes
-- BereqHeader Accept-Encoding: gzip
-- BereqHeader X-Varnish: 3
-- BereqHeader X-Forwarded-For: 192.168.200.13, 192.168.200.13
-- BereqHeader X-Varnish: 3
-- VCL_call BACKEND_FETCH
-- VCL_return fetch
-- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6860
-- BackendStart 192.168.200.11 8080
-- Timestamp Bereq: 1576135956.461955 0.000856 0.000856
-- Timestamp Beresp: 1576135956.462607 0.001508 0.000652
-- BerespProtocol HTTP/1.1
-- BerespStatus 200
-- BerespReason OK
-- BerespHeader Server: nginx
-- BerespHeader Date: Thu, 12 Dec 2019 07:33:05 GMT
-- BerespHeader Content-Type: text/plain
-- BerespHeader Content-Length: 23521499
-- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
-- BerespHeader Connection: keep-alive
-- BerespHeader ETag: "3a1794b0-166e8db"
-- BerespHeader Accept-Ranges: bytes
-- TTL RFC 120 10 0 1576135956 1576135956 1576135985 0 0 cacheable
-- VCL_call BACKEND_RESPONSE
-- TTL VCL 86400 10 0 1576135956 cacheable
-- VCL_return deliver
-- Storage file s0
-- Fetch_Body 3 length stream
-- BackendReuse 27 boot.swift_proxy_1
-- Timestamp BerespBody: 1576135956.758956 0.297858 0.296349
-- Length 23521499
-- BereqAcct 206 0 206 241 23521499 23521740
-- End

* << Request >> 5
- Begin req 4 rxreq
- Timestamp Start: 1576136019.230666 0.000000 0.000000
- Timestamp Req: 1576136019.230666 0.000000 0.000000
- ReqStart 192.168.200.12 5452 a0
- ReqMethod PURGE
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.12:8080<http://192.168.200.12:8080>
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.12
- VCL_call RECV
- VCL_return purge
- VCL_call HASH
- VCL_return lookup
- VCL_call PURGE
- VCL_return synth
- Timestamp Process: 1576136019.230863 0.000198 0.000198
- RespHeader Date: Thu, 12 Dec 2019 07:33:39 GMT
- RespHeader Server: Varnish
- RespHeader X-Varnish: 5
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespReason Purged
- VCL_call SYNTH
- RespHeader Content-Type: text/html; charset=utf-8
- RespHeader Retry-After: 5
- VCL_return deliver
- RespHeader Content-Length: 236
- Storage malloc Transient
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576136019.231138 0.000473 0.000275
- ReqAcct 113 0 113 214 236 450
- End

########
PLEASE test this scenario for yourself and you can see that i do correct and the object wont remove from varnish-file
Please do this scenario for yourself

________________________________
From: Guillaume Quintard <guillaume@varnish-software.com<mailto:guillaume@varnish-software.com>>
Sent: Wednesday, December 11, 2019 9:44 AM
To: hamidreza hosseini <hrhosseini@hotmail.com<mailto:hrhosseini@hotmail.com>>
Cc: varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org> <varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org>>
Subject: Re: Varnish doesnt purge

Yes it does, we have already proved it./

Each instance purges and will refetch the object for its respective backend. I explained how to test: start with the origin, change the data and check that you actually get the new object. Then add a varnish layer, purge, check, repeat.

You are testing everything at once and are unable to isolate the issue because you refuse to take things step by step, and spamming won't change that.

I'm sorry but until you have tested each layer individually and identified the one that fails, I won't be able to help.

--
Guillaume Quintard


On Wed, Dec 11, 2019 at 3:08 PM hamidreza hosseini <hrhosseini@hotmail.com<mailto:hrhosseini@hotmail.com>> wrote:
Hi,
please help me to solve this problem, this is the link that i explaine my question completely:
https://varnish-cache.org/lists/pipermail/varnish-misc/2019-December/026754.html

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org>
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org>
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
--
Sent from my iPhone
Re: Varnish doesnt purge [ In reply to ]
Try this when purging

varnish ram: curl -v -k -X PURGE http://192.168.200.13/Naserfeiz.mp4 <http://192.168.200.13/Naserfeiz.mp4> -o /dev/null -H "newtrack: yes"

varnish file (2nd one): curl -v -k -X PURGE http://192.168.200.12:8080/Naserfeiz.mp4 <http://192.168.200.12:8080/Naserfeiz.mp4> -o /dev/null -H "newtrack: yes” -H "Host: 192.168.200.13 <http://192.168.200.13/Naserfeiz.mp4>”

Notice the difference? You to understand HOW content is stored in varnish’s storage in order to understand how to evict it - my comment about the hash routine

Thanks

> On 15 Dec 2019, at 21:31, hamidreza hosseini <hrhosseini@hotmail.com> wrote:
>
> I dont understand your message,
>
> I cant solve this problem
>
> Instead of saying "appericiate to your diplomacy!!!!!!!!!!" you can help me!
>
> I dont have enough knoledge on Varnish,
>
> I dont get your text about hash, i didnt use hash.
>
> Best Regards
>
>
> From: Vlad Rusu <vlad.rusu@lola.tech>
> Sent: Sunday, December 15, 2019 7:48:42 PM
> To: hamidreza hosseini <hrhosseini@hotmail.com>
> Cc: Guillaume Quintard <guillaume@varnish-software.com>; varnish-misc@varnish-cache.org <varnish-misc@varnish-cache.org>
> Subject: Re: Varnish doesnt purge
>
> I appreciate Guillaume’s diplomacy :)
>
> Your issue (one of them at least) is that in your cascaded varnish setup the hash key of the cached url, in both varnish storages, is the one you use to originally access the asset. Your vcl hash routine might also be a custom one.. that would be the next thing to look at if what I’m saying here doesn’t fix it.
>
> So, purge
> http://192.168.200.13/Naserfeiz.mp4 <http://192.168.200.13/Naserfeiz.mp4>
> and NOT
> http://192.168.200.12:8080/Naserfeiz.mp4 <http://192.168.200.12:8080/Naserfeiz.mp4>when in the cascaded setup
>
> Cheers
>
> On Sat, 14 Dec 2019 at 09:16, hamidreza hosseini <hrhosseini@hotmail.com <mailto:hrhosseini@hotmail.com>> wrote:
> Hi,
> Please annwer my question, I sent you my test
>
> From: varnish-misc <varnish-misc-bounces+hrhosseini=hotmail.com@varnish-cache.org <mailto:hotmail.com@varnish-cache.org>> on behalf of hamidreza hosseini <hrhosseini@hotmail.com <mailto:hrhosseini@hotmail.com>>
> Sent: Wednesday, December 11, 2019 11:49 PM
> To: Guillaume Quintard <guillaume@varnish-software.com <mailto:guillaume@varnish-software.com>>
> Cc: varnish-misc@varnish-cache.org <mailto:varnish-misc@varnish-cache.org> <varnish-misc@varnish-cache.org <mailto:varnish-misc@varnish-cache.org>>
> Subject: Re: Varnish doesnt purge
>
> I do all steps now again, this is the resault:
> STEP ONE:
> First of all i configure a varnish (with file backend) in front of swift/Nginx and i ask the url that exist in swift with curl,
>
> varnish-file$curl -I http://192.168.200.12:8080/Naserfeiz.mp4 <http://192.168.200.12:8080/Naserfeiz.mp4> -H "newtrack: yes"
>
> HTTP/1.1 200 OK
> Server: nginx
> Date: Thu, 12 Dec 2019 07:29:32 GMT
> Content-Type: text/plain
> Content-Length: 23521499
> Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
> ETag: "3a1794b0-166e8db"
> X-Varnish: 2
> Age: 0
> Via: 1.1 varnish (Varnish/6.0)
> Accept-Ranges: bytes
> Connection: keep-alive
>
>
> varnish cach that object and then i purge the url:
>
> varnish-file$ curl -v -k -X PURGE http://192.168.200.12:8080/Naserfeiz.mp4 <http://192.168.200.12:8080/Naserfeiz.mp4> -o /dev/null -H "newtrack: yes"
>
> * Trying 192.168.200.12...
> % Total % Received % Xferd Average Speed Time Time Time Current
> Dload Upload Total Spent Left Speed
> 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.12 (192.168.200.12) port 8080 (#0)
> > PURGE /Naserfeiz.mp4 HTTP/1.1
> > Host: 192.168.200.12:8080 <http://192.168.200.12:8080/>
> > User-Agent: curl/7.47.0
> > Accept: */*
> > newtrack: yes
> >
> < HTTP/1.1 200 Purged
> < Date: Thu, 12 Dec 2019 07:29:16 GMT
> < Server: Varnish
> < X-Varnish: 32770
> < Content-Type: text/html; charset=utf-8
> < Retry-After: 5
> < Content-Length: 240
> < Accept-Ranges: bytes
> < Connection: keep-alive
> <
> { [240 bytes data]
> 100 240 100 240 0 0 124k 0 --:--:-- --:--:-- --:--:-- 234k
> * Connection #0 to host 192.168.200.12 left intact
>
> The object purged successfully and it didnt exist in cach anymore.
>
> This is varnishlog resault:
>
> varnish-file$ sudo varnishlog -d -g request
> * << Request >> 2
> - Begin req 1 rxreq
> - Timestamp Start: 1576135743.442646 0.000000 0.000000
> - Timestamp Req: 1576135743.442646 0.000000 0.000000
> - ReqStart 192.168.200.12 5432 a0
> - ReqMethod HEAD
> - ReqURL /Naserfeiz.mp4
> - ReqProtocol HTTP/1.1
> - ReqHeader Host: 192.168.200.12:8080 <http://192.168.200.12:8080/>
> - ReqHeader User-Agent: curl/7.47.0
> - ReqHeader Accept: */*
> - ReqHeader newtrack: yes
> - ReqHeader X-Forwarded-For: 192.168.200.12
> - VCL_call RECV
> - VCL_return hash
> - VCL_call HASH
> - VCL_return lookup
> - VCL_call MISS
> - VCL_return fetch
> - Link bereq 3 fetch
> - Timestamp Fetch: 1576135743.444347 0.001701 0.001701
> - RespProtocol HTTP/1.1
> - RespStatus 200
> - RespReason OK
> - RespHeader Server: nginx
> - RespHeader Date: Thu, 12 Dec 2019 07:29:32 GMT
> - RespHeader Content-Type: text/plain
> - RespHeader Content-Length: 23521499
> - RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
> - RespHeader ETag: "3a1794b0-166e8db"
> - RespHeader X-Varnish: 2
> - RespHeader Age: 0
> - RespHeader Via: 1.1 varnish (Varnish/6.0)
> - VCL_call DELIVER
> - VCL_return deliver
> - Timestamp Process: 1576135743.444382 0.001736 0.000036
> - RespHeader Accept-Ranges: bytes
> - RespHeader Connection: keep-alive
> - Timestamp Resp: 1576135743.444474 0.001828 0.000092
> - ReqAcct 112 0 112 295 0 295
> - End
> ** << BeReq >> 3
> -- Begin bereq 2 fetch
> -- VCL_use boot
> -- Timestamp Start: 1576135743.442807 0.000000 0.000000
> -- BereqMethod HEAD
> -- BereqURL /Naserfeiz.mp4
> -- BereqProtocol HTTP/1.1
> -- BereqHeader Host: 192.168.200.12:8080 <http://192.168.200.12:8080/>
> -- BereqHeader User-Agent: curl/7.47.0
> -- BereqHeader Accept: */*
> -- BereqHeader newtrack: yes
> -- BereqHeader X-Forwarded-For: 192.168.200.12
> -- BereqMethod GET
> -- BereqHeader Accept-Encoding: gzip
> -- BereqHeader X-Varnish: 3
> -- VCL_call BACKEND_FETCH
> -- VCL_return fetch
> -- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6846
> -- BackendStart 192.168.200.11 8080
> -- Timestamp Bereq: 1576135743.443586 0.000779 0.000779
> -- Timestamp Beresp: 1576135743.444017 0.001210 0.000431
> -- BerespProtocol HTTP/1.1
> -- BerespStatus 200
> -- BerespReason OK
> -- BerespHeader Server: nginx
> -- BerespHeader Date: Thu, 12 Dec 2019 07:29:32 GMT
> -- BerespHeader Content-Type: text/plain
> -- BerespHeader Content-Length: 23521499
> -- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
> -- BerespHeader Connection: keep-alive
> -- BerespHeader ETag: "3a1794b0-166e8db"
> -- BerespHeader Accept-Ranges: bytes
> -- TTL RFC 120 10 0 1576135743 1576135743 1576135772 0 0 cacheable
> -- VCL_call BACKEND_RESPONSE
> -- TTL VCL 86400 10 0 1576135743 cacheable
> -- VCL_return deliver
> -- Storage file s0
> -- Fetch_Body 3 length stream
> -- BackendReuse 27 boot.swift_proxy_1
> -- Timestamp BerespBody: 1576135743.747835 0.305028 0.303818
> -- Length 23521499
> -- BereqAcct 181 0 181 241 23521499 23521740
> -- End
>
> * << Request >> 32770
> - Begin req 32769 rxreq
> - Timestamp Start: 1576135756.680792 0.000000 0.000000
> - Timestamp Req: 1576135756.680792 0.000000 0.000000
> - ReqStart 192.168.200.12 5438 a0
> - ReqMethod PURGE
> - ReqURL /Naserfeiz.mp4
> - ReqProtocol HTTP/1.1
> - ReqHeader Host: 192.168.200.12:8080 <http://192.168.200.12:8080/>
> - ReqHeader User-Agent: curl/7.47.0
> - ReqHeader Accept: */*
> - ReqHeader newtrack: yes
> - ReqHeader X-Forwarded-For: 192.168.200.12
> - VCL_call RECV
> - VCL_return purge
> - VCL_call HASH
> - VCL_return lookup
> - VCL_call PURGE
> - VCL_return synth
> - Timestamp Process: 1576135756.680982 0.000190 0.000190
> - RespHeader Date: Thu, 12 Dec 2019 07:29:16 GMT
> - RespHeader Server: Varnish
> - RespHeader X-Varnish: 32770
> - RespProtocol HTTP/1.1
> - RespStatus 200
> - RespReason OK
> - RespReason Purged
> - VCL_call SYNTH
> - RespHeader Content-Type: text/html; charset=utf-8
> - RespHeader Retry-After: 5
> - VCL_return deliver
> - RespHeader Content-Length: 240
> - Storage malloc Transient
> - RespHeader Accept-Ranges: bytes
> - RespHeader Connection: keep-alive
> - Timestamp Resp: 1576135756.681190 0.000399 0.000208
> - ReqAcct 113 0 113 218 240 458
> - End
>
> * << Request >> 32772
> - Begin req 32771 rxreq
> - Timestamp Start: 1576135769.304679 0.000000 0.000000
> - Timestamp Req: 1576135769.304679 0.000000 0.000000
> - ReqStart 192.168.200.12 5440 a0
> - ReqMethod HEAD
> - ReqURL /Naserfeiz.mp4
> - ReqProtocol HTTP/1.1
> - ReqHeader Host: 192.168.200.12:8080 <http://192.168.200.12:8080/>
> - ReqHeader User-Agent: curl/7.47.0
> - ReqHeader Accept: */*
> - ReqHeader newtrack: yes
> - ReqHeader X-Forwarded-For: 192.168.200.12
> - VCL_call RECV
> - VCL_return hash
> - VCL_call HASH
> - VCL_return lookup
> - VCL_call MISS
> - VCL_return fetch
> - Link bereq 32773 fetch
> - Timestamp Fetch: 1576135769.306630 0.001951 0.001951
> - RespProtocol HTTP/1.1
> - RespStatus 200
> - RespReason OK
> - RespHeader Server: nginx
> - RespHeader Date: Thu, 12 Dec 2019 07:29:58 GMT
> - RespHeader Content-Type: text/plain
> - RespHeader Content-Length: 23521499
> - RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
> - RespHeader ETag: "3a1794b0-166e8db"
> - RespHeader X-Varnish: 32772
> - RespHeader Age: 0
> - RespHeader Via: 1.1 varnish (Varnish/6.0)
> - VCL_call DELIVER
> - VCL_return deliver
> - Timestamp Process: 1576135769.306657 0.001978 0.000027
> - RespHeader Accept-Ranges: bytes
> - RespHeader Connection: keep-alive
> - Timestamp Resp: 1576135769.306748 0.002069 0.000091
> - ReqAcct 112 0 112 299 0 299
> - End
> ** << BeReq >> 32773
> -- Begin bereq 32772 fetch
> -- VCL_use boot
> -- Timestamp Start: 1576135769.304834 0.000000 0.000000
> -- BereqMethod HEAD
> -- BereqURL /Naserfeiz.mp4
> -- BereqProtocol HTTP/1.1
> -- BereqHeader Host: 192.168.200.12:8080 <http://192.168.200.12:8080/>
> -- BereqHeader User-Agent: curl/7.47.0
> -- BereqHeader Accept: */*
> -- BereqHeader newtrack: yes
> -- BereqHeader X-Forwarded-For: 192.168.200.12
> -- BereqMethod GET
> -- BereqHeader Accept-Encoding: gzip
> -- BereqHeader X-Varnish: 32773
> -- VCL_call BACKEND_FETCH
> -- VCL_return fetch
> -- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6846
> -- BackendStart 192.168.200.11 8080
> -- Timestamp Bereq: 1576135769.304993 0.000160 0.000160
> -- Timestamp Beresp: 1576135769.306170 0.001336 0.001176
> -- BerespProtocol HTTP/1.1
> -- BerespStatus 200
> -- BerespReason OK
> -- BerespHeader Server: nginx
> -- BerespHeader Date: Thu, 12 Dec 2019 07:29:58 GMT
> -- BerespHeader Content-Type: text/plain
> -- BerespHeader Content-Length: 23521499
> -- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
> -- BerespHeader Connection: keep-alive
> -- BerespHeader ETag: "3a1794b0-166e8db"
> -- BerespHeader Accept-Ranges: bytes
> -- TTL RFC 120 10 0 1576135769 1576135769 1576135798 0 0 cacheable
> -- VCL_call BACKEND_RESPONSE
> -- TTL VCL 86400 10 0 1576135769 cacheable
> -- VCL_return deliver
> -- Storage file s0
> -- Fetch_Body 3 length stream
> -- BackendReuse 27 boot.swift_proxy_1
> -- Timestamp BerespBody: 1576135769.645490 0.340657 0.339321
> -- Length 23521499
> -- BereqAcct 185 0 185 241 23521499 23521740
> -- End
>
>
> ########
>
> STEP TWO:
> First i restart varnish file and varnishlog then:
> I add another varnish (with ram backend) in fornt of the varnish file and then i ask the url again in varnish ram (because it is my frontend server and users can access media though this server):
>
> varnish-ram$ curl -I http://192.168.200.13/Naserfeiz.mp4 <http://192.168.200.13/Naserfeiz.mp4> -H "newtrack: yes"
> HTTP/1.1 200 OK
> Server: nginx
> Date: Thu, 12 Dec 2019 07:33:05 GMT
> Content-Type: text/plain
> Content-Length: 23521499
> Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
> ETag: "3a1794b0-166e8db"
> X-Varnish: 2
> Via: 1.1 varnish (Varnish/6.0)
> X-Varnish: 2
> Age: 0
> Via: 1.1 varnish-v4
> Accept-Ranges: bytes
> Connection: keep-alive
>
>
> First varnish file cached the url and then Varnish ram fetched the object from varnish file and cached it , then i purged that url in varnish ram and it purged successfuly and it exactly removed from cached:
>
> varnish-ram:~$ curl -v -k -X PURGE http://192.168.200.13/Naserfeiz.mp4 <http://192.168.200.13/Naserfeiz.mp4> -o /dev/null -H "newtrack: yes"
> * Trying 192.168.200.13...
> % Total % Received % Xferd Average Speed Time Time Time Current
> Dload Upload Total Spent Left Speed
> 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.13 (192.168.200.13) port 80 (#0)
> > PURGE /Naserfeiz.mp4 HTTP/1.1
> > Host: 192.168.200.13
> > User-Agent: curl/7.47.0
> > Accept: */*
> > newtrack: yes
> >
> < HTTP/1.1 200 Purged
> < Date: Thu, 12 Dec 2019 07:33:46 GMT
> < Server: Varnish
> < X-Varnish: 32770
> < Content-Type: text/html; charset=utf-8
> < Retry-After: 5
> < Content-Length: 240
> < Accept-Ranges: bytes
> < Connection: keep-alive
> <
> { [240 bytes data]
> 100 240 100 240 0 0 73846 0 --:--:-- --:--:-- --:--:-- 117k
> * Connection #0 to host 192.168.200.13 left intact
>
>
> Then i purged the url in varnish file it shows "200 purged" but it still exist in varnish file!!!!!!:
>
> varnish-file$ curl -v -k -X PURGE http://192.168.200.12:8080/Naserfeiz.mp4 <http://192.168.200.12:8080/Naserfeiz.mp4> -o /dev/null -H "newtrack: yes"
> * Trying 192.168.200.12...
> % Total % Received % Xferd Average Speed Time Time Time Current
> Dload Upload Total Spent Left Speed
> 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.12 (192.168.200.12) port 8080 (#0)
> > PURGE /Naserfeiz.mp4 HTTP/1.1
> > Host: 192.168.200.12:8080 <http://192.168.200.12:8080/>
> > User-Agent: curl/7.47.0
> > Accept: */*
> > newtrack: yes
> >
> < HTTP/1.1 200 Purged
> < Date: Thu, 12 Dec 2019 07:33:39 GMT
> < Server: Varnish
> < X-Varnish: 5
> < Content-Type: text/html; charset=utf-8
> < Retry-After: 5
> < Content-Length: 236
> < Accept-Ranges: bytes
> < Connection: keep-alive
> <
> { [236 bytes data]
> 100 236 100 236 0 0 110k 0 --:--:-- --:--:-- --:--:-- 230k
> * Connection #0 to host 192.168.200.12 left intact
>
>
> This is varnishlog resault in varnish file:
>
> varnish-file:~$ sudo varnishlog -d -g request
> * << Request >> 2
> - Begin req 1 rxreq
> - Timestamp Start: 1576135956.460768 0.000000 0.000000
> - Timestamp Req: 1576135956.460768 0.000000 0.000000
> - ReqStart 192.168.200.13 22300 a0
> - ReqMethod GET
> - ReqURL /Naserfeiz.mp4
> - ReqProtocol HTTP/1.1
> - ReqHeader Host: 192.168.200.13
> - ReqHeader User-Agent: curl/7.47.0
> - ReqHeader Accept: */*
> - ReqHeader newtrack: yes
> - ReqHeader X-Forwarded-For: 192.168.200.13
> - ReqHeader Accept-Encoding: gzip
> - ReqHeader X-Varnish: 3
> - ReqUnset X-Forwarded-For: 192.168.200.13
> - ReqHeader X-Forwarded-For: 192.168.200.13, 192.168.200.13
> - VCL_call RECV
> - VCL_return hash
> - VCL_call HASH
> - VCL_return lookup
> - VCL_call MISS
> - VCL_return fetch
> - Link bereq 3 fetch
> - Timestamp Fetch: 1576135956.463065 0.002297 0.002297
> - RespProtocol HTTP/1.1
> - RespStatus 200
> - RespReason OK
> - RespHeader Server: nginx
> - RespHeader Date: Thu, 12 Dec 2019 07:33:05 GMT
> - RespHeader Content-Type: text/plain
> - RespHeader Content-Length: 23521499
> - RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
> - RespHeader ETag: "3a1794b0-166e8db"
> - RespHeader X-Varnish: 2
> - RespHeader Age: 0
> - RespHeader Via: 1.1 varnish (Varnish/6.0)
> - VCL_call DELIVER
> - VCL_return deliver
> - Timestamp Process: 1576135956.463114 0.002346 0.000049
> - RespHeader Accept-Ranges: bytes
> - RespHeader Connection: keep-alive
> - Timestamp Resp: 1576135956.758920 0.298152 0.295806
> - ReqAcct 176 0 176 295 23521499 23521794
> - End
> ** << BeReq >> 3
> -- Begin bereq 2 fetch
> -- VCL_use boot
> -- Timestamp Start: 1576135956.461099 0.000000 0.000000
> -- BereqMethod GET
> -- BereqURL /Naserfeiz.mp4
> -- BereqProtocol HTTP/1.1
> -- BereqHeader Host: 192.168.200.13
> -- BereqHeader User-Agent: curl/7.47.0
> -- BereqHeader Accept: */*
> -- BereqHeader newtrack: yes
> -- BereqHeader Accept-Encoding: gzip
> -- BereqHeader X-Varnish: 3
> -- BereqHeader X-Forwarded-For: 192.168.200.13, 192.168.200.13
> -- BereqHeader X-Varnish: 3
> -- VCL_call BACKEND_FETCH
> -- VCL_return fetch
> -- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6860
> -- BackendStart 192.168.200.11 8080
> -- Timestamp Bereq: 1576135956.461955 0.000856 0.000856
> -- Timestamp Beresp: 1576135956.462607 0.001508 0.000652
> -- BerespProtocol HTTP/1.1
> -- BerespStatus 200
> -- BerespReason OK
> -- BerespHeader Server: nginx
> -- BerespHeader Date: Thu, 12 Dec 2019 07:33:05 GMT
> -- BerespHeader Content-Type: text/plain
> -- BerespHeader Content-Length: 23521499
> -- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
> -- BerespHeader Connection: keep-alive
> -- BerespHeader ETag: "3a1794b0-166e8db"
> -- BerespHeader Accept-Ranges: bytes
> -- TTL RFC 120 10 0 1576135956 1576135956 1576135985 0 0 cacheable
> -- VCL_call BACKEND_RESPONSE
> -- TTL VCL 86400 10 0 1576135956 cacheable
> -- VCL_return deliver
> -- Storage file s0
> -- Fetch_Body 3 length stream
> -- BackendReuse 27 boot.swift_proxy_1
> -- Timestamp BerespBody: 1576135956.758956 0.297858 0.296349
> -- Length 23521499
> -- BereqAcct 206 0 206 241 23521499 23521740
> -- End
>
> * << Request >> 5
> - Begin req 4 rxreq
> - Timestamp Start: 1576136019.230666 0.000000 0.000000
> - Timestamp Req: 1576136019.230666 0.000000 0.000000
> - ReqStart 192.168.200.12 5452 a0
> - ReqMethod PURGE
> - ReqURL /Naserfeiz.mp4
> - ReqProtocol HTTP/1.1
> - ReqHeader Host: 192.168.200.12:8080 <http://192.168.200.12:8080/>
> - ReqHeader User-Agent: curl/7.47.0
> - ReqHeader Accept: */*
> - ReqHeader newtrack: yes
> - ReqHeader X-Forwarded-For: 192.168.200.12
> - VCL_call RECV
> - VCL_return purge
> - VCL_call HASH
> - VCL_return lookup
> - VCL_call PURGE
> - VCL_return synth
> - Timestamp Process: 1576136019.230863 0.000198 0.000198
> - RespHeader Date: Thu, 12 Dec 2019 07:33:39 GMT
> - RespHeader Server: Varnish
> - RespHeader X-Varnish: 5
> - RespProtocol HTTP/1.1
> - RespStatus 200
> - RespReason OK
> - RespReason Purged
> - VCL_call SYNTH
> - RespHeader Content-Type: text/html; charset=utf-8
> - RespHeader Retry-After: 5
> - VCL_return deliver
> - RespHeader Content-Length: 236
> - Storage malloc Transient
> - RespHeader Accept-Ranges: bytes
> - RespHeader Connection: keep-alive
> - Timestamp Resp: 1576136019.231138 0.000473 0.000275
> - ReqAcct 113 0 113 214 236 450
> - End
>
> ########
> PLEASE test this scenario for yourself and you can see that i do correct and the object wont remove from varnish-file
> Please do this scenario for yourself
>
> From: Guillaume Quintard <guillaume@varnish-software.com <mailto:guillaume@varnish-software.com>>
> Sent: Wednesday, December 11, 2019 9:44 AM
> To: hamidreza hosseini <hrhosseini@hotmail.com <mailto:hrhosseini@hotmail.com>>
> Cc: varnish-misc@varnish-cache.org <mailto:varnish-misc@varnish-cache.org> <varnish-misc@varnish-cache.org <mailto:varnish-misc@varnish-cache.org>>
> Subject: Re: Varnish doesnt purge
>
> Yes it does, we have already proved it./
>
> Each instance purges and will refetch the object for its respective backend. I explained how to test: start with the origin, change the data and check that you actually get the new object. Then add a varnish layer, purge, check, repeat.
>
> You are testing everything at once and are unable to isolate the issue because you refuse to take things step by step, and spamming won't change that.
>
> I'm sorry but until you have tested each layer individually and identified the one that fails, I won't be able to help.
>
> --
> Guillaume Quintard
>
>
> On Wed, Dec 11, 2019 at 3:08 PM hamidreza hosseini <hrhosseini@hotmail.com <mailto:hrhosseini@hotmail.com>> wrote:
> Hi,
> please help me to solve this problem, this is the link that i explaine my question completely:
> https://varnish-cache.org/lists/pipermail/varnish-misc/2019-December/026754.html <https://varnish-cache.org/lists/pipermail/varnish-misc/2019-December/026754.html>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org <mailto:varnish-misc@varnish-cache.org>
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc <https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org <mailto:varnish-misc@varnish-cache.org>
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc <https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc>
> --
> Sent from my iPhone
Re: Varnish doesnt purge [ In reply to ]
Many Thanks for your answer
You mean that in production i should add additional header to my purge request because it stored this with the header content like the ip address of first varnish
OK, for purging i need wich header to add to my purge request and how can i get that what are that? And how they stored and how can i purge them

Best Regards

Get Outlook for Android<https://aka.ms/ghei36>

________________________________
From: Vlad Rusu <vlad.rusu@lola.tech>
Sent: Sunday, December 15, 2019 11:16:59 PM
To: hamidreza hosseini <hrhosseini@hotmail.com>
Cc: Guillaume Quintard <guillaume@varnish-software.com>; varnish-misc@varnish-cache.org <varnish-misc@varnish-cache.org>
Subject: Re: Varnish doesnt purge

Try this when purging

varnish ram: curl -v -k -X PURGE http://192.168.200.13/Naserfeiz.mp4 -o /dev/null -H "newtrack: yes"

varnish file (2nd one): curl -v -k -X PURGE http://192.168.200.12:8080/Naserfeiz.mp4 -o /dev/null -H "newtrack: yes? -H "Host: 192.168.200.13<http://192.168.200.13/Naserfeiz.mp4>?

Notice the difference? You to understand HOW content is stored in varnish?s storage in order to understand how to evict it - my comment about the hash routine

Thanks

On 15 Dec 2019, at 21:31, hamidreza hosseini <hrhosseini@hotmail.com<mailto:hrhosseini@hotmail.com>> wrote:

I dont understand your message,

I cant solve this problem

Instead of saying "appericiate to your diplomacy!!!!!!!!!!" you can help me!

I dont have enough knoledge on Varnish,

I dont get your text about hash, i didnt use hash.

Best Regards


________________________________
From: Vlad Rusu <vlad.rusu@lola.tech<mailto:vlad.rusu@lola.tech>>
Sent: Sunday, December 15, 2019 7:48:42 PM
To: hamidreza hosseini <hrhosseini@hotmail.com<mailto:hrhosseini@hotmail.com>>
Cc: Guillaume Quintard <guillaume@varnish-software.com<mailto:guillaume@varnish-software.com>>; varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org> <varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org>>
Subject: Re: Varnish doesnt purge

I appreciate Guillaume?s diplomacy :)

Your issue (one of them at least) is that in your cascaded varnish setup the hash key of the cached url, in both varnish storages, is the one you use to originally access the asset. Your vcl hash routine might also be a custom one.. that would be the next thing to look at if what I?m saying here doesn?t fix it.

So, purge
http://192.168.200.13/Naserfeiz.mp4
and NOT
http://192.168.200.12:8080/Naserfeiz.mp4
when in the cascaded setup

Cheers

On Sat, 14 Dec 2019 at 09:16, hamidreza hosseini <hrhosseini@hotmail.com<mailto:hrhosseini@hotmail.com>> wrote:
Hi,
Please annwer my question, I sent you my test

________________________________
From: varnish-misc <varnish-misc-bounces+hrhosseini=hotmail.com@varnish-cache.org<mailto:hotmail.com@varnish-cache.org>> on behalf of hamidreza hosseini <hrhosseini@hotmail.com<mailto:hrhosseini@hotmail.com>>
Sent: Wednesday, December 11, 2019 11:49 PM
To: Guillaume Quintard <guillaume@varnish-software.com<mailto:guillaume@varnish-software.com>>
Cc: varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org> <varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org>>
Subject: Re: Varnish doesnt purge

I do all steps now again, this is the resault:
STEP ONE:
First of all i configure a varnish (with file backend) in front of swift/Nginx and i ask the url that exist in swift with curl,

varnish-file$curl -I http://192.168.200.12:8080/Naserfeiz.mp4 -H "newtrack: yes"

HTTP/1.1 200 OK
Server: nginx
Date: Thu, 12 Dec 2019 07:29:32 GMT
Content-Type: text/plain
Content-Length: 23521499
Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
ETag: "3a1794b0-166e8db"
X-Varnish: 2
Age: 0
Via: 1.1 varnish (Varnish/6.0)
Accept-Ranges: bytes
Connection: keep-alive


varnish cach that object and then i purge the url:

varnish-file$ curl -v -k -X PURGE http://192.168.200.12:8080/Naserfeiz.mp4 -o /dev/null -H "newtrack: yes"

* Trying 192.168.200.12...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.12 (192.168.200.12) port 8080 (#0)
> PURGE /Naserfeiz.mp4 HTTP/1.1
> Host: 192.168.200.12:8080<http://192.168.200.12:8080/>
> User-Agent: curl/7.47.0
> Accept: */*
> newtrack: yes
>
< HTTP/1.1 200 Purged
< Date: Thu, 12 Dec 2019 07:29:16 GMT
< Server: Varnish
< X-Varnish: 32770
< Content-Type: text/html; charset=utf-8
< Retry-After: 5
< Content-Length: 240
< Accept-Ranges: bytes
< Connection: keep-alive
<
{ [240 bytes data]
100 240 100 240 0 0 124k 0 --:--:-- --:--:-- --:--:-- 234k
* Connection #0 to host 192.168.200.12 left intact

The object purged successfully and it didnt exist in cach anymore.

This is varnishlog resault:

varnish-file$ sudo varnishlog -d -g request
* << Request >> 2
- Begin req 1 rxreq
- Timestamp Start: 1576135743.442646 0.000000 0.000000
- Timestamp Req: 1576135743.442646 0.000000 0.000000
- ReqStart 192.168.200.12 5432 a0
- ReqMethod HEAD
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.12:8080<http://192.168.200.12:8080/>
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.12
- VCL_call RECV
- VCL_return hash
- VCL_call HASH
- VCL_return lookup
- VCL_call MISS
- VCL_return fetch
- Link bereq 3 fetch
- Timestamp Fetch: 1576135743.444347 0.001701 0.001701
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespHeader Server: nginx
- RespHeader Date: Thu, 12 Dec 2019 07:29:32 GMT
- RespHeader Content-Type: text/plain
- RespHeader Content-Length: 23521499
- RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
- RespHeader ETag: "3a1794b0-166e8db"
- RespHeader X-Varnish: 2
- RespHeader Age: 0
- RespHeader Via: 1.1 varnish (Varnish/6.0)
- VCL_call DELIVER
- VCL_return deliver
- Timestamp Process: 1576135743.444382 0.001736 0.000036
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576135743.444474 0.001828 0.000092
- ReqAcct 112 0 112 295 0 295
- End
** << BeReq >> 3
-- Begin bereq 2 fetch
-- VCL_use boot
-- Timestamp Start: 1576135743.442807 0.000000 0.000000
-- BereqMethod HEAD
-- BereqURL /Naserfeiz.mp4
-- BereqProtocol HTTP/1.1
-- BereqHeader Host: 192.168.200.12:8080<http://192.168.200.12:8080/>
-- BereqHeader User-Agent: curl/7.47.0
-- BereqHeader Accept: */*
-- BereqHeader newtrack: yes
-- BereqHeader X-Forwarded-For: 192.168.200.12
-- BereqMethod GET
-- BereqHeader Accept-Encoding: gzip
-- BereqHeader X-Varnish: 3
-- VCL_call BACKEND_FETCH
-- VCL_return fetch
-- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6846
-- BackendStart 192.168.200.11 8080
-- Timestamp Bereq: 1576135743.443586 0.000779 0.000779
-- Timestamp Beresp: 1576135743.444017 0.001210 0.000431
-- BerespProtocol HTTP/1.1
-- BerespStatus 200
-- BerespReason OK
-- BerespHeader Server: nginx
-- BerespHeader Date: Thu, 12 Dec 2019 07:29:32 GMT
-- BerespHeader Content-Type: text/plain
-- BerespHeader Content-Length: 23521499
-- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
-- BerespHeader Connection: keep-alive
-- BerespHeader ETag: "3a1794b0-166e8db"
-- BerespHeader Accept-Ranges: bytes
-- TTL RFC 120 10 0 1576135743 1576135743 1576135772 0 0 cacheable
-- VCL_call BACKEND_RESPONSE
-- TTL VCL 86400 10 0 1576135743 cacheable
-- VCL_return deliver
-- Storage file s0
-- Fetch_Body 3 length stream
-- BackendReuse 27 boot.swift_proxy_1
-- Timestamp BerespBody: 1576135743.747835 0.305028 0.303818
-- Length 23521499
-- BereqAcct 181 0 181 241 23521499 23521740
-- End

* << Request >> 32770
- Begin req 32769 rxreq
- Timestamp Start: 1576135756.680792 0.000000 0.000000
- Timestamp Req: 1576135756.680792 0.000000 0.000000
- ReqStart 192.168.200.12 5438 a0
- ReqMethod PURGE
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.12:8080<http://192.168.200.12:8080/>
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.12
- VCL_call RECV
- VCL_return purge
- VCL_call HASH
- VCL_return lookup
- VCL_call PURGE
- VCL_return synth
- Timestamp Process: 1576135756.680982 0.000190 0.000190
- RespHeader Date: Thu, 12 Dec 2019 07:29:16 GMT
- RespHeader Server: Varnish
- RespHeader X-Varnish: 32770
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespReason Purged
- VCL_call SYNTH
- RespHeader Content-Type: text/html; charset=utf-8
- RespHeader Retry-After: 5
- VCL_return deliver
- RespHeader Content-Length: 240
- Storage malloc Transient
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576135756.681190 0.000399 0.000208
- ReqAcct 113 0 113 218 240 458
- End

* << Request >> 32772
- Begin req 32771 rxreq
- Timestamp Start: 1576135769.304679 0.000000 0.000000
- Timestamp Req: 1576135769.304679 0.000000 0.000000
- ReqStart 192.168.200.12 5440 a0
- ReqMethod HEAD
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.12:8080<http://192.168.200.12:8080/>
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.12
- VCL_call RECV
- VCL_return hash
- VCL_call HASH
- VCL_return lookup
- VCL_call MISS
- VCL_return fetch
- Link bereq 32773 fetch
- Timestamp Fetch: 1576135769.306630 0.001951 0.001951
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespHeader Server: nginx
- RespHeader Date: Thu, 12 Dec 2019 07:29:58 GMT
- RespHeader Content-Type: text/plain
- RespHeader Content-Length: 23521499
- RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
- RespHeader ETag: "3a1794b0-166e8db"
- RespHeader X-Varnish: 32772
- RespHeader Age: 0
- RespHeader Via: 1.1 varnish (Varnish/6.0)
- VCL_call DELIVER
- VCL_return deliver
- Timestamp Process: 1576135769.306657 0.001978 0.000027
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576135769.306748 0.002069 0.000091
- ReqAcct 112 0 112 299 0 299
- End
** << BeReq >> 32773
-- Begin bereq 32772 fetch
-- VCL_use boot
-- Timestamp Start: 1576135769.304834 0.000000 0.000000
-- BereqMethod HEAD
-- BereqURL /Naserfeiz.mp4
-- BereqProtocol HTTP/1.1
-- BereqHeader Host: 192.168.200.12:8080<http://192.168.200.12:8080/>
-- BereqHeader User-Agent: curl/7.47.0
-- BereqHeader Accept: */*
-- BereqHeader newtrack: yes
-- BereqHeader X-Forwarded-For: 192.168.200.12
-- BereqMethod GET
-- BereqHeader Accept-Encoding: gzip
-- BereqHeader X-Varnish: 32773
-- VCL_call BACKEND_FETCH
-- VCL_return fetch
-- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6846
-- BackendStart 192.168.200.11 8080
-- Timestamp Bereq: 1576135769.304993 0.000160 0.000160
-- Timestamp Beresp: 1576135769.306170 0.001336 0.001176
-- BerespProtocol HTTP/1.1
-- BerespStatus 200
-- BerespReason OK
-- BerespHeader Server: nginx
-- BerespHeader Date: Thu, 12 Dec 2019 07:29:58 GMT
-- BerespHeader Content-Type: text/plain
-- BerespHeader Content-Length: 23521499
-- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
-- BerespHeader Connection: keep-alive
-- BerespHeader ETag: "3a1794b0-166e8db"
-- BerespHeader Accept-Ranges: bytes
-- TTL RFC 120 10 0 1576135769 1576135769 1576135798 0 0 cacheable
-- VCL_call BACKEND_RESPONSE
-- TTL VCL 86400 10 0 1576135769 cacheable
-- VCL_return deliver
-- Storage file s0
-- Fetch_Body 3 length stream
-- BackendReuse 27 boot.swift_proxy_1
-- Timestamp BerespBody: 1576135769.645490 0.340657 0.339321
-- Length 23521499
-- BereqAcct 185 0 185 241 23521499 23521740
-- End


########

STEP TWO:
First i restart varnish file and varnishlog then:
I add another varnish (with ram backend) in fornt of the varnish file and then i ask the url again in varnish ram (because it is my frontend server and users can access media though this server):

varnish-ram$ curl -I http://192.168.200.13/Naserfeiz.mp4 -H "newtrack: yes"
HTTP/1.1 200 OK
Server: nginx
Date: Thu, 12 Dec 2019 07:33:05 GMT
Content-Type: text/plain
Content-Length: 23521499
Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
ETag: "3a1794b0-166e8db"
X-Varnish: 2
Via: 1.1 varnish (Varnish/6.0)
X-Varnish: 2
Age: 0
Via: 1.1 varnish-v4
Accept-Ranges: bytes
Connection: keep-alive


First varnish file cached the url and then Varnish ram fetched the object from varnish file and cached it , then i purged that url in varnish ram and it purged successfuly and it exactly removed from cached:

varnish-ram:~$ curl -v -k -X PURGE http://192.168.200.13/Naserfeiz.mp4 -o /dev/null -H "newtrack: yes"
* Trying 192.168.200.13...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.13 (192.168.200.13) port 80 (#0)
> PURGE /Naserfeiz.mp4 HTTP/1.1
> Host: 192.168.200.13
> User-Agent: curl/7.47.0
> Accept: */*
> newtrack: yes
>
< HTTP/1.1 200 Purged
< Date: Thu, 12 Dec 2019 07:33:46 GMT
< Server: Varnish
< X-Varnish: 32770
< Content-Type: text/html; charset=utf-8
< Retry-After: 5
< Content-Length: 240
< Accept-Ranges: bytes
< Connection: keep-alive
<
{ [240 bytes data]
100 240 100 240 0 0 73846 0 --:--:-- --:--:-- --:--:-- 117k
* Connection #0 to host 192.168.200.13 left intact


Then i purged the url in varnish file it shows "200 purged" but it still exist in varnish file!!!!!!:

varnish-file$ curl -v -k -X PURGE http://192.168.200.12:8080/Naserfeiz.mp4 -o /dev/null -H "newtrack: yes"
* Trying 192.168.200.12...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.12 (192.168.200.12) port 8080 (#0)
> PURGE /Naserfeiz.mp4 HTTP/1.1
> Host: 192.168.200.12:8080<http://192.168.200.12:8080/>
> User-Agent: curl/7.47.0
> Accept: */*
> newtrack: yes
>
< HTTP/1.1 200 Purged
< Date: Thu, 12 Dec 2019 07:33:39 GMT
< Server: Varnish
< X-Varnish: 5
< Content-Type: text/html; charset=utf-8
< Retry-After: 5
< Content-Length: 236
< Accept-Ranges: bytes
< Connection: keep-alive
<
{ [236 bytes data]
100 236 100 236 0 0 110k 0 --:--:-- --:--:-- --:--:-- 230k
* Connection #0 to host 192.168.200.12 left intact


This is varnishlog resault in varnish file:

varnish-file:~$ sudo varnishlog -d -g request
* << Request >> 2
- Begin req 1 rxreq
- Timestamp Start: 1576135956.460768 0.000000 0.000000
- Timestamp Req: 1576135956.460768 0.000000 0.000000
- ReqStart 192.168.200.13 22300 a0
- ReqMethod GET
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.13
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.13
- ReqHeader Accept-Encoding: gzip
- ReqHeader X-Varnish: 3
- ReqUnset X-Forwarded-For: 192.168.200.13
- ReqHeader X-Forwarded-For: 192.168.200.13, 192.168.200.13
- VCL_call RECV
- VCL_return hash
- VCL_call HASH
- VCL_return lookup
- VCL_call MISS
- VCL_return fetch
- Link bereq 3 fetch
- Timestamp Fetch: 1576135956.463065 0.002297 0.002297
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespHeader Server: nginx
- RespHeader Date: Thu, 12 Dec 2019 07:33:05 GMT
- RespHeader Content-Type: text/plain
- RespHeader Content-Length: 23521499
- RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
- RespHeader ETag: "3a1794b0-166e8db"
- RespHeader X-Varnish: 2
- RespHeader Age: 0
- RespHeader Via: 1.1 varnish (Varnish/6.0)
- VCL_call DELIVER
- VCL_return deliver
- Timestamp Process: 1576135956.463114 0.002346 0.000049
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576135956.758920 0.298152 0.295806
- ReqAcct 176 0 176 295 23521499 23521794
- End
** << BeReq >> 3
-- Begin bereq 2 fetch
-- VCL_use boot
-- Timestamp Start: 1576135956.461099 0.000000 0.000000
-- BereqMethod GET
-- BereqURL /Naserfeiz.mp4
-- BereqProtocol HTTP/1.1
-- BereqHeader Host: 192.168.200.13
-- BereqHeader User-Agent: curl/7.47.0
-- BereqHeader Accept: */*
-- BereqHeader newtrack: yes
-- BereqHeader Accept-Encoding: gzip
-- BereqHeader X-Varnish: 3
-- BereqHeader X-Forwarded-For: 192.168.200.13, 192.168.200.13
-- BereqHeader X-Varnish: 3
-- VCL_call BACKEND_FETCH
-- VCL_return fetch
-- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6860
-- BackendStart 192.168.200.11 8080
-- Timestamp Bereq: 1576135956.461955 0.000856 0.000856
-- Timestamp Beresp: 1576135956.462607 0.001508 0.000652
-- BerespProtocol HTTP/1.1
-- BerespStatus 200
-- BerespReason OK
-- BerespHeader Server: nginx
-- BerespHeader Date: Thu, 12 Dec 2019 07:33:05 GMT
-- BerespHeader Content-Type: text/plain
-- BerespHeader Content-Length: 23521499
-- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
-- BerespHeader Connection: keep-alive
-- BerespHeader ETag: "3a1794b0-166e8db"
-- BerespHeader Accept-Ranges: bytes
-- TTL RFC 120 10 0 1576135956 1576135956 1576135985 0 0 cacheable
-- VCL_call BACKEND_RESPONSE
-- TTL VCL 86400 10 0 1576135956 cacheable
-- VCL_return deliver
-- Storage file s0
-- Fetch_Body 3 length stream
-- BackendReuse 27 boot.swift_proxy_1
-- Timestamp BerespBody: 1576135956.758956 0.297858 0.296349
-- Length 23521499
-- BereqAcct 206 0 206 241 23521499 23521740
-- End

* << Request >> 5
- Begin req 4 rxreq
- Timestamp Start: 1576136019.230666 0.000000 0.000000
- Timestamp Req: 1576136019.230666 0.000000 0.000000
- ReqStart 192.168.200.12 5452 a0
- ReqMethod PURGE
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.12:8080<http://192.168.200.12:8080/>
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.12
- VCL_call RECV
- VCL_return purge
- VCL_call HASH
- VCL_return lookup
- VCL_call PURGE
- VCL_return synth
- Timestamp Process: 1576136019.230863 0.000198 0.000198
- RespHeader Date: Thu, 12 Dec 2019 07:33:39 GMT
- RespHeader Server: Varnish
- RespHeader X-Varnish: 5
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespReason Purged
- VCL_call SYNTH
- RespHeader Content-Type: text/html; charset=utf-8
- RespHeader Retry-After: 5
- VCL_return deliver
- RespHeader Content-Length: 236
- Storage malloc Transient
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576136019.231138 0.000473 0.000275
- ReqAcct 113 0 113 214 236 450
- End

########
PLEASE test this scenario for yourself and you can see that i do correct and the object wont remove from varnish-file
Please do this scenario for yourself

________________________________
From: Guillaume Quintard <guillaume@varnish-software.com<mailto:guillaume@varnish-software.com>>
Sent: Wednesday, December 11, 2019 9:44 AM
To: hamidreza hosseini <hrhosseini@hotmail.com<mailto:hrhosseini@hotmail.com>>
Cc: varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org> <varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org>>
Subject: Re: Varnish doesnt purge

Yes it does, we have already proved it./

Each instance purges and will refetch the object for its respective backend. I explained how to test: start with the origin, change the data and check that you actually get the new object. Then add a varnish layer, purge, check, repeat.

You are testing everything at once and are unable to isolate the issue because you refuse to take things step by step, and spamming won't change that.

I'm sorry but until you have tested each layer individually and identified the one that fails, I won't be able to help.

--
Guillaume Quintard


On Wed, Dec 11, 2019 at 3:08 PM hamidreza hosseini <hrhosseini@hotmail.com<mailto:hrhosseini@hotmail.com>> wrote:
Hi,
please help me to solve this problem, this is the link that i explaine my question completely:
https://varnish-cache.org/lists/pipermail/varnish-misc/2019-December/026754.html

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org>
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org>
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
--
Sent from my iPhone
Re: Varnish doesnt purge [ In reply to ]
I think the following will explain this in great detail

http://varnish-cache.org/docs/6.3/users-guide/vcl-hashing.html <http://varnish-cache.org/docs/6.3/users-guide/vcl-hashing.html>
https://github.com/varnishcache/varnish-cache/blob/6.3/bin/varnishd/builtin.vcl#L87-L95 <https://github.com/varnishcache/varnish-cache/blob/6.3/bin/varnishd/builtin.vcl#L87-L95>

By default (your case) any cache entry will be uniquely identified by the following combination: host header + url

No matter how many varnish server you stack, unless you specifically alter the Host header in one of them, they will all store the object in their own storage using the same hash key

If you call for https://google.com/a <https://google.com/a> and you have stacked 2 varnishes, both of them will save the object as: google.com/a <http://google.com/a>

When you want to purge that object, you have to send the same data - which is why I gave you an example that altered the the Host header when sending the PURGE request to the 2nd varnish

I’m personally a fan of BANNING when using varnish-cache (non commercial). Read more about this below.
https://varnish-cache.org/docs/6.3/users-guide/purging.html <https://varnish-cache.org/docs/6.3/users-guide/purging.html> + https://book.varnish-software.com/4.0/chapters/Cache_Invalidation.html#lurker-friendly-bans <https://book.varnish-software.com/4.0/chapters/Cache_Invalidation.html#lurker-friendly-bans>

Also worth printing this one on your wall if you want to truly understand the journey of a request through varnish http://varnish-cache.org/docs/6.3/reference/states.html#reference-states <http://varnish-cache.org/docs/6.3/reference/states.html#reference-states>

Thanks,


Vlad Rusu
Cell: +40758066019

Lola Tech | lola.tech <https://lola.tech/>

> On 15 Dec 2019, at 21:54, hamidreza hosseini <hrhosseini@hotmail.com> wrote:
>
> Many Thanks for your answer
> You mean that in production i should add additional header to my purge request because it stored this with the header content like the ip address of first varnish
> OK, for purging i need wich header to add to my purge request and how can i get that what are that? And how they stored and how can i purge them
>
> Best Regards
>
> Get Outlook for Android <https://aka.ms/ghei36>
> From: Vlad Rusu <vlad.rusu@lola.tech>
> Sent: Sunday, December 15, 2019 11:16:59 PM
> To: hamidreza hosseini <hrhosseini@hotmail.com>
> Cc: Guillaume Quintard <guillaume@varnish-software.com>; varnish-misc@varnish-cache.org <varnish-misc@varnish-cache.org>
> Subject: Re: Varnish doesnt purge
>
> Try this when purging
>
> varnish ram: curl -v -k -X PURGE http://192.168.200.13/Naserfeiz.mp4 <http://192.168.200.13/Naserfeiz.mp4> -o /dev/null -H "newtrack: yes"
>
> varnish file (2nd one): curl -v -k -X PURGE http://192.168.200.12:8080/Naserfeiz.mp4 <http://192.168.200.12:8080/Naserfeiz.mp4> -o /dev/null -H "newtrack: yes” -H "Host: 192.168.200.13 <http://192.168.200.13/Naserfeiz.mp4>”
>
> Notice the difference? You to understand HOW content is stored in varnish’s storage in order to understand how to evict it - my comment about the hash routine
>
> Thanks
>
>> On 15 Dec 2019, at 21:31, hamidreza hosseini <hrhosseini@hotmail.com <mailto:hrhosseini@hotmail.com>> wrote:
>>
>> I dont understand your message,
>>
>> I cant solve this problem
>>
>> Instead of saying "appericiate to your diplomacy!!!!!!!!!!" you can help me!
>>
>> I dont have enough knoledge on Varnish,
>>
>> I dont get your text about hash, i didnt use hash.
>>
>> Best Regards
>>
>>
>> From: Vlad Rusu <vlad.rusu@lola.tech <mailto:vlad.rusu@lola.tech>>
>> Sent: Sunday, December 15, 2019 7:48:42 PM
>> To: hamidreza hosseini <hrhosseini@hotmail.com <mailto:hrhosseini@hotmail.com>>
>> Cc: Guillaume Quintard <guillaume@varnish-software.com <mailto:guillaume@varnish-software.com>>; varnish-misc@varnish-cache.org <mailto:varnish-misc@varnish-cache.org> <varnish-misc@varnish-cache.org <mailto:varnish-misc@varnish-cache.org>>
>> Subject: Re: Varnish doesnt purge
>>
>> I appreciate Guillaume’s diplomacy :)
>>
>> Your issue (one of them at least) is that in your cascaded varnish setup the hash key of the cached url, in both varnish storages, is the one you use to originally access the asset. Your vcl hash routine might also be a custom one.. that would be the next thing to look at if what I’m saying here doesn’t fix it.
>>
>> So, purge
>> http://192.168.200.13/Naserfeiz.mp4 <http://192.168.200.13/Naserfeiz.mp4>
>> and NOT
>> http://192.168.200.12:8080/Naserfeiz.mp4 <http://192.168.200.12:8080/Naserfeiz.mp4>when in the cascaded setup
>>
>> Cheers
>>
>> On Sat, 14 Dec 2019 at 09:16, hamidreza hosseini <hrhosseini@hotmail.com <mailto:hrhosseini@hotmail.com>> wrote:
>> Hi,
>> Please annwer my question, I sent you my test
>>
>> From: varnish-misc <varnish-misc-bounces+hrhosseini=hotmail.com@varnish-cache.org <mailto:hotmail.com@varnish-cache.org>> on behalf of hamidreza hosseini <hrhosseini@hotmail.com <mailto:hrhosseini@hotmail.com>>
>> Sent: Wednesday, December 11, 2019 11:49 PM
>> To: Guillaume Quintard <guillaume@varnish-software.com <mailto:guillaume@varnish-software.com>>
>> Cc: varnish-misc@varnish-cache.org <mailto:varnish-misc@varnish-cache.org> <varnish-misc@varnish-cache.org <mailto:varnish-misc@varnish-cache.org>>
>> Subject: Re: Varnish doesnt purge
>>
>> I do all steps now again, this is the resault:
>> STEP ONE:
>> First of all i configure a varnish (with file backend) in front of swift/Nginx and i ask the url that exist in swift with curl,
>>
>> varnish-file$curl -I http://192.168.200.12:8080/Naserfeiz.mp4 <http://192.168.200.12:8080/Naserfeiz.mp4> -H "newtrack: yes"
>>
>> HTTP/1.1 200 OK
>> Server: nginx
>> Date: Thu, 12 Dec 2019 07:29:32 GMT
>> Content-Type: text/plain
>> Content-Length: 23521499
>> Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
>> ETag: "3a1794b0-166e8db"
>> X-Varnish: 2
>> Age: 0
>> Via: 1.1 varnish (Varnish/6.0)
>> Accept-Ranges: bytes
>> Connection: keep-alive
>>
>>
>> varnish cach that object and then i purge the url:
>>
>> varnish-file$ curl -v -k -X PURGE http://192.168.200.12:8080/Naserfeiz.mp4 <http://192.168.200.12:8080/Naserfeiz.mp4> -o /dev/null -H "newtrack: yes"
>>
>> * Trying 192.168.200.12...
>> % Total % Received % Xferd Average Speed Time Time Time Current
>> Dload Upload Total Spent Left Speed
>> 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.12 (192.168.200.12) port 8080 (#0)
>> > PURGE /Naserfeiz.mp4 HTTP/1.1
>> > Host: 192.168.200.12:8080 <http://192.168.200.12:8080/>
>> > User-Agent: curl/7.47.0
>> > Accept: */*
>> > newtrack: yes
>> >
>> < HTTP/1.1 200 Purged
>> < Date: Thu, 12 Dec 2019 07:29:16 GMT
>> < Server: Varnish
>> < X-Varnish: 32770
>> < Content-Type: text/html; charset=utf-8
>> < Retry-After: 5
>> < Content-Length: 240
>> < Accept-Ranges: bytes
>> < Connection: keep-alive
>> <
>> { [240 bytes data]
>> 100 240 100 240 0 0 124k 0 --:--:-- --:--:-- --:--:-- 234k
>> * Connection #0 to host 192.168.200.12 left intact
>>
>> The object purged successfully and it didnt exist in cach anymore.
>>
>> This is varnishlog resault:
>>
>> varnish-file$ sudo varnishlog -d -g request
>> * << Request >> 2
>> - Begin req 1 rxreq
>> - Timestamp Start: 1576135743.442646 0.000000 0.000000
>> - Timestamp Req: 1576135743.442646 0.000000 0.000000
>> - ReqStart 192.168.200.12 5432 a0
>> - ReqMethod HEAD
>> - ReqURL /Naserfeiz.mp4
>> - ReqProtocol HTTP/1.1
>> - ReqHeader Host: 192.168.200.12:8080 <http://192.168.200.12:8080/>
>> - ReqHeader User-Agent: curl/7.47.0
>> - ReqHeader Accept: */*
>> - ReqHeader newtrack: yes
>> - ReqHeader X-Forwarded-For: 192.168.200.12
>> - VCL_call RECV
>> - VCL_return hash
>> - VCL_call HASH
>> - VCL_return lookup
>> - VCL_call MISS
>> - VCL_return fetch
>> - Link bereq 3 fetch
>> - Timestamp Fetch: 1576135743.444347 0.001701 0.001701
>> - RespProtocol HTTP/1.1
>> - RespStatus 200
>> - RespReason OK
>> - RespHeader Server: nginx
>> - RespHeader Date: Thu, 12 Dec 2019 07:29:32 GMT
>> - RespHeader Content-Type: text/plain
>> - RespHeader Content-Length: 23521499
>> - RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
>> - RespHeader ETag: "3a1794b0-166e8db"
>> - RespHeader X-Varnish: 2
>> - RespHeader Age: 0
>> - RespHeader Via: 1.1 varnish (Varnish/6.0)
>> - VCL_call DELIVER
>> - VCL_return deliver
>> - Timestamp Process: 1576135743.444382 0.001736 0.000036
>> - RespHeader Accept-Ranges: bytes
>> - RespHeader Connection: keep-alive
>> - Timestamp Resp: 1576135743.444474 0.001828 0.000092
>> - ReqAcct 112 0 112 295 0 295
>> - End
>> ** << BeReq >> 3
>> -- Begin bereq 2 fetch
>> -- VCL_use boot
>> -- Timestamp Start: 1576135743.442807 0.000000 0.000000
>> -- BereqMethod HEAD
>> -- BereqURL /Naserfeiz.mp4
>> -- BereqProtocol HTTP/1.1
>> -- BereqHeader Host: 192.168.200.12:8080 <http://192.168.200.12:8080/>
>> -- BereqHeader User-Agent: curl/7.47.0
>> -- BereqHeader Accept: */*
>> -- BereqHeader newtrack: yes
>> -- BereqHeader X-Forwarded-For: 192.168.200.12
>> -- BereqMethod GET
>> -- BereqHeader Accept-Encoding: gzip
>> -- BereqHeader X-Varnish: 3
>> -- VCL_call BACKEND_FETCH
>> -- VCL_return fetch
>> -- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6846
>> -- BackendStart 192.168.200.11 8080
>> -- Timestamp Bereq: 1576135743.443586 0.000779 0.000779
>> -- Timestamp Beresp: 1576135743.444017 0.001210 0.000431
>> -- BerespProtocol HTTP/1.1
>> -- BerespStatus 200
>> -- BerespReason OK
>> -- BerespHeader Server: nginx
>> -- BerespHeader Date: Thu, 12 Dec 2019 07:29:32 GMT
>> -- BerespHeader Content-Type: text/plain
>> -- BerespHeader Content-Length: 23521499
>> -- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
>> -- BerespHeader Connection: keep-alive
>> -- BerespHeader ETag: "3a1794b0-166e8db"
>> -- BerespHeader Accept-Ranges: bytes
>> -- TTL RFC 120 10 0 1576135743 1576135743 1576135772 0 0 cacheable
>> -- VCL_call BACKEND_RESPONSE
>> -- TTL VCL 86400 10 0 1576135743 cacheable
>> -- VCL_return deliver
>> -- Storage file s0
>> -- Fetch_Body 3 length stream
>> -- BackendReuse 27 boot.swift_proxy_1
>> -- Timestamp BerespBody: 1576135743.747835 0.305028 0.303818
>> -- Length 23521499
>> -- BereqAcct 181 0 181 241 23521499 23521740
>> -- End
>>
>> * << Request >> 32770
>> - Begin req 32769 rxreq
>> - Timestamp Start: 1576135756.680792 0.000000 0.000000
>> - Timestamp Req: 1576135756.680792 0.000000 0.000000
>> - ReqStart 192.168.200.12 5438 a0
>> - ReqMethod PURGE
>> - ReqURL /Naserfeiz.mp4
>> - ReqProtocol HTTP/1.1
>> - ReqHeader Host: 192.168.200.12:8080 <http://192.168.200.12:8080/>
>> - ReqHeader User-Agent: curl/7.47.0
>> - ReqHeader Accept: */*
>> - ReqHeader newtrack: yes
>> - ReqHeader X-Forwarded-For: 192.168.200.12
>> - VCL_call RECV
>> - VCL_return purge
>> - VCL_call HASH
>> - VCL_return lookup
>> - VCL_call PURGE
>> - VCL_return synth
>> - Timestamp Process: 1576135756.680982 0.000190 0.000190
>> - RespHeader Date: Thu, 12 Dec 2019 07:29:16 GMT
>> - RespHeader Server: Varnish
>> - RespHeader X-Varnish: 32770
>> - RespProtocol HTTP/1.1
>> - RespStatus 200
>> - RespReason OK
>> - RespReason Purged
>> - VCL_call SYNTH
>> - RespHeader Content-Type: text/html; charset=utf-8
>> - RespHeader Retry-After: 5
>> - VCL_return deliver
>> - RespHeader Content-Length: 240
>> - Storage malloc Transient
>> - RespHeader Accept-Ranges: bytes
>> - RespHeader Connection: keep-alive
>> - Timestamp Resp: 1576135756.681190 0.000399 0.000208
>> - ReqAcct 113 0 113 218 240 458
>> - End
>>
>> * << Request >> 32772
>> - Begin req 32771 rxreq
>> - Timestamp Start: 1576135769.304679 0.000000 0.000000
>> - Timestamp Req: 1576135769.304679 0.000000 0.000000
>> - ReqStart 192.168.200.12 5440 a0
>> - ReqMethod HEAD
>> - ReqURL /Naserfeiz.mp4
>> - ReqProtocol HTTP/1.1
>> - ReqHeader Host: 192.168.200.12:8080 <http://192.168.200.12:8080/>
>> - ReqHeader User-Agent: curl/7.47.0
>> - ReqHeader Accept: */*
>> - ReqHeader newtrack: yes
>> - ReqHeader X-Forwarded-For: 192.168.200.12
>> - VCL_call RECV
>> - VCL_return hash
>> - VCL_call HASH
>> - VCL_return lookup
>> - VCL_call MISS
>> - VCL_return fetch
>> - Link bereq 32773 fetch
>> - Timestamp Fetch: 1576135769.306630 0.001951 0.001951
>> - RespProtocol HTTP/1.1
>> - RespStatus 200
>> - RespReason OK
>> - RespHeader Server: nginx
>> - RespHeader Date: Thu, 12 Dec 2019 07:29:58 GMT
>> - RespHeader Content-Type: text/plain
>> - RespHeader Content-Length: 23521499
>> - RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
>> - RespHeader ETag: "3a1794b0-166e8db"
>> - RespHeader X-Varnish: 32772
>> - RespHeader Age: 0
>> - RespHeader Via: 1.1 varnish (Varnish/6.0)
>> - VCL_call DELIVER
>> - VCL_return deliver
>> - Timestamp Process: 1576135769.306657 0.001978 0.000027
>> - RespHeader Accept-Ranges: bytes
>> - RespHeader Connection: keep-alive
>> - Timestamp Resp: 1576135769.306748 0.002069 0.000091
>> - ReqAcct 112 0 112 299 0 299
>> - End
>> ** << BeReq >> 32773
>> -- Begin bereq 32772 fetch
>> -- VCL_use boot
>> -- Timestamp Start: 1576135769.304834 0.000000 0.000000
>> -- BereqMethod HEAD
>> -- BereqURL /Naserfeiz.mp4
>> -- BereqProtocol HTTP/1.1
>> -- BereqHeader Host: 192.168.200.12:8080 <http://192.168.200.12:8080/>
>> -- BereqHeader User-Agent: curl/7.47.0
>> -- BereqHeader Accept: */*
>> -- BereqHeader newtrack: yes
>> -- BereqHeader X-Forwarded-For: 192.168.200.12
>> -- BereqMethod GET
>> -- BereqHeader Accept-Encoding: gzip
>> -- BereqHeader X-Varnish: 32773
>> -- VCL_call BACKEND_FETCH
>> -- VCL_return fetch
>> -- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6846
>> -- BackendStart 192.168.200.11 8080
>> -- Timestamp Bereq: 1576135769.304993 0.000160 0.000160
>> -- Timestamp Beresp: 1576135769.306170 0.001336 0.001176
>> -- BerespProtocol HTTP/1.1
>> -- BerespStatus 200
>> -- BerespReason OK
>> -- BerespHeader Server: nginx
>> -- BerespHeader Date: Thu, 12 Dec 2019 07:29:58 GMT
>> -- BerespHeader Content-Type: text/plain
>> -- BerespHeader Content-Length: 23521499
>> -- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
>> -- BerespHeader Connection: keep-alive
>> -- BerespHeader ETag: "3a1794b0-166e8db"
>> -- BerespHeader Accept-Ranges: bytes
>> -- TTL RFC 120 10 0 1576135769 1576135769 1576135798 0 0 cacheable
>> -- VCL_call BACKEND_RESPONSE
>> -- TTL VCL 86400 10 0 1576135769 cacheable
>> -- VCL_return deliver
>> -- Storage file s0
>> -- Fetch_Body 3 length stream
>> -- BackendReuse 27 boot.swift_proxy_1
>> -- Timestamp BerespBody: 1576135769.645490 0.340657 0.339321
>> -- Length 23521499
>> -- BereqAcct 185 0 185 241 23521499 23521740
>> -- End
>>
>>
>> ########
>>
>> STEP TWO:
>> First i restart varnish file and varnishlog then:
>> I add another varnish (with ram backend) in fornt of the varnish file and then i ask the url again in varnish ram (because it is my frontend server and users can access media though this server):
>>
>> varnish-ram$ curl -I http://192.168.200.13/Naserfeiz.mp4 <http://192.168.200.13/Naserfeiz.mp4> -H "newtrack: yes"
>> HTTP/1.1 200 OK
>> Server: nginx
>> Date: Thu, 12 Dec 2019 07:33:05 GMT
>> Content-Type: text/plain
>> Content-Length: 23521499
>> Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
>> ETag: "3a1794b0-166e8db"
>> X-Varnish: 2
>> Via: 1.1 varnish (Varnish/6.0)
>> X-Varnish: 2
>> Age: 0
>> Via: 1.1 varnish-v4
>> Accept-Ranges: bytes
>> Connection: keep-alive
>>
>>
>> First varnish file cached the url and then Varnish ram fetched the object from varnish file and cached it , then i purged that url in varnish ram and it purged successfuly and it exactly removed from cached:
>>
>> varnish-ram:~$ curl -v -k -X PURGE http://192.168.200.13/Naserfeiz.mp4 <http://192.168.200.13/Naserfeiz.mp4> -o /dev/null -H "newtrack: yes"
>> * Trying 192.168.200.13...
>> % Total % Received % Xferd Average Speed Time Time Time Current
>> Dload Upload Total Spent Left Speed
>> 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.13 (192.168.200.13) port 80 (#0)
>> > PURGE /Naserfeiz.mp4 HTTP/1.1
>> > Host: 192.168.200.13
>> > User-Agent: curl/7.47.0
>> > Accept: */*
>> > newtrack: yes
>> >
>> < HTTP/1.1 200 Purged
>> < Date: Thu, 12 Dec 2019 07:33:46 GMT
>> < Server: Varnish
>> < X-Varnish: 32770
>> < Content-Type: text/html; charset=utf-8
>> < Retry-After: 5
>> < Content-Length: 240
>> < Accept-Ranges: bytes
>> < Connection: keep-alive
>> <
>> { [240 bytes data]
>> 100 240 100 240 0 0 73846 0 --:--:-- --:--:-- --:--:-- 117k
>> * Connection #0 to host 192.168.200.13 left intact
>>
>>
>> Then i purged the url in varnish file it shows "200 purged" but it still exist in varnish file!!!!!!:
>>
>> varnish-file$ curl -v -k -X PURGE http://192.168.200.12:8080/Naserfeiz.mp4 <http://192.168.200.12:8080/Naserfeiz.mp4> -o /dev/null -H "newtrack: yes"
>> * Trying 192.168.200.12...
>> % Total % Received % Xferd Average Speed Time Time Time Current
>> Dload Upload Total Spent Left Speed
>> 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.12 (192.168.200.12) port 8080 (#0)
>> > PURGE /Naserfeiz.mp4 HTTP/1.1
>> > Host: 192.168.200.12:8080 <http://192.168.200.12:8080/>
>> > User-Agent: curl/7.47.0
>> > Accept: */*
>> > newtrack: yes
>> >
>> < HTTP/1.1 200 Purged
>> < Date: Thu, 12 Dec 2019 07:33:39 GMT
>> < Server: Varnish
>> < X-Varnish: 5
>> < Content-Type: text/html; charset=utf-8
>> < Retry-After: 5
>> < Content-Length: 236
>> < Accept-Ranges: bytes
>> < Connection: keep-alive
>> <
>> { [236 bytes data]
>> 100 236 100 236 0 0 110k 0 --:--:-- --:--:-- --:--:-- 230k
>> * Connection #0 to host 192.168.200.12 left intact
>>
>>
>> This is varnishlog resault in varnish file:
>>
>> varnish-file:~$ sudo varnishlog -d -g request
>> * << Request >> 2
>> - Begin req 1 rxreq
>> - Timestamp Start: 1576135956.460768 0.000000 0.000000
>> - Timestamp Req: 1576135956.460768 0.000000 0.000000
>> - ReqStart 192.168.200.13 22300 a0
>> - ReqMethod GET
>> - ReqURL /Naserfeiz.mp4
>> - ReqProtocol HTTP/1.1
>> - ReqHeader Host: 192.168.200.13
>> - ReqHeader User-Agent: curl/7.47.0
>> - ReqHeader Accept: */*
>> - ReqHeader newtrack: yes
>> - ReqHeader X-Forwarded-For: 192.168.200.13
>> - ReqHeader Accept-Encoding: gzip
>> - ReqHeader X-Varnish: 3
>> - ReqUnset X-Forwarded-For: 192.168.200.13
>> - ReqHeader X-Forwarded-For: 192.168.200.13, 192.168.200.13
>> - VCL_call RECV
>> - VCL_return hash
>> - VCL_call HASH
>> - VCL_return lookup
>> - VCL_call MISS
>> - VCL_return fetch
>> - Link bereq 3 fetch
>> - Timestamp Fetch: 1576135956.463065 0.002297 0.002297
>> - RespProtocol HTTP/1.1
>> - RespStatus 200
>> - RespReason OK
>> - RespHeader Server: nginx
>> - RespHeader Date: Thu, 12 Dec 2019 07:33:05 GMT
>> - RespHeader Content-Type: text/plain
>> - RespHeader Content-Length: 23521499
>> - RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
>> - RespHeader ETag: "3a1794b0-166e8db"
>> - RespHeader X-Varnish: 2
>> - RespHeader Age: 0
>> - RespHeader Via: 1.1 varnish (Varnish/6.0)
>> - VCL_call DELIVER
>> - VCL_return deliver
>> - Timestamp Process: 1576135956.463114 0.002346 0.000049
>> - RespHeader Accept-Ranges: bytes
>> - RespHeader Connection: keep-alive
>> - Timestamp Resp: 1576135956.758920 0.298152 0.295806
>> - ReqAcct 176 0 176 295 23521499 23521794
>> - End
>> ** << BeReq >> 3
>> -- Begin bereq 2 fetch
>> -- VCL_use boot
>> -- Timestamp Start: 1576135956.461099 0.000000 0.000000
>> -- BereqMethod GET
>> -- BereqURL /Naserfeiz.mp4
>> -- BereqProtocol HTTP/1.1
>> -- BereqHeader Host: 192.168.200.13
>> -- BereqHeader User-Agent: curl/7.47.0
>> -- BereqHeader Accept: */*
>> -- BereqHeader newtrack: yes
>> -- BereqHeader Accept-Encoding: gzip
>> -- BereqHeader X-Varnish: 3
>> -- BereqHeader X-Forwarded-For: 192.168.200.13, 192.168.200.13
>> -- BereqHeader X-Varnish: 3
>> -- VCL_call BACKEND_FETCH
>> -- VCL_return fetch
>> -- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6860
>> -- BackendStart 192.168.200.11 8080
>> -- Timestamp Bereq: 1576135956.461955 0.000856 0.000856
>> -- Timestamp Beresp: 1576135956.462607 0.001508 0.000652
>> -- BerespProtocol HTTP/1.1
>> -- BerespStatus 200
>> -- BerespReason OK
>> -- BerespHeader Server: nginx
>> -- BerespHeader Date: Thu, 12 Dec 2019 07:33:05 GMT
>> -- BerespHeader Content-Type: text/plain
>> -- BerespHeader Content-Length: 23521499
>> -- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
>> -- BerespHeader Connection: keep-alive
>> -- BerespHeader ETag: "3a1794b0-166e8db"
>> -- BerespHeader Accept-Ranges: bytes
>> -- TTL RFC 120 10 0 1576135956 1576135956 1576135985 0 0 cacheable
>> -- VCL_call BACKEND_RESPONSE
>> -- TTL VCL 86400 10 0 1576135956 cacheable
>> -- VCL_return deliver
>> -- Storage file s0
>> -- Fetch_Body 3 length stream
>> -- BackendReuse 27 boot.swift_proxy_1
>> -- Timestamp BerespBody: 1576135956.758956 0.297858 0.296349
>> -- Length 23521499
>> -- BereqAcct 206 0 206 241 23521499 23521740
>> -- End
>>
>> * << Request >> 5
>> - Begin req 4 rxreq
>> - Timestamp Start: 1576136019.230666 0.000000 0.000000
>> - Timestamp Req: 1576136019.230666 0.000000 0.000000
>> - ReqStart 192.168.200.12 5452 a0
>> - ReqMethod PURGE
>> - ReqURL /Naserfeiz.mp4
>> - ReqProtocol HTTP/1.1
>> - ReqHeader Host: 192.168.200.12:8080 <http://192.168.200.12:8080/>
>> - ReqHeader User-Agent: curl/7.47.0
>> - ReqHeader Accept: */*
>> - ReqHeader newtrack: yes
>> - ReqHeader X-Forwarded-For: 192.168.200.12
>> - VCL_call RECV
>> - VCL_return purge
>> - VCL_call HASH
>> - VCL_return lookup
>> - VCL_call PURGE
>> - VCL_return synth
>> - Timestamp Process: 1576136019.230863 0.000198 0.000198
>> - RespHeader Date: Thu, 12 Dec 2019 07:33:39 GMT
>> - RespHeader Server: Varnish
>> - RespHeader X-Varnish: 5
>> - RespProtocol HTTP/1.1
>> - RespStatus 200
>> - RespReason OK
>> - RespReason Purged
>> - VCL_call SYNTH
>> - RespHeader Content-Type: text/html; charset=utf-8
>> - RespHeader Retry-After: 5
>> - VCL_return deliver
>> - RespHeader Content-Length: 236
>> - Storage malloc Transient
>> - RespHeader Accept-Ranges: bytes
>> - RespHeader Connection: keep-alive
>> - Timestamp Resp: 1576136019.231138 0.000473 0.000275
>> - ReqAcct 113 0 113 214 236 450
>> - End
>>
>> ########
>> PLEASE test this scenario for yourself and you can see that i do correct and the object wont remove from varnish-file
>> Please do this scenario for yourself
>>
>> From: Guillaume Quintard <guillaume@varnish-software.com <mailto:guillaume@varnish-software.com>>
>> Sent: Wednesday, December 11, 2019 9:44 AM
>> To: hamidreza hosseini <hrhosseini@hotmail.com <mailto:hrhosseini@hotmail.com>>
>> Cc: varnish-misc@varnish-cache.org <mailto:varnish-misc@varnish-cache.org> <varnish-misc@varnish-cache.org <mailto:varnish-misc@varnish-cache.org>>
>> Subject: Re: Varnish doesnt purge
>>
>> Yes it does, we have already proved it./
>>
>> Each instance purges and will refetch the object for its respective backend. I explained how to test: start with the origin, change the data and check that you actually get the new object. Then add a varnish layer, purge, check, repeat.
>>
>> You are testing everything at once and are unable to isolate the issue because you refuse to take things step by step, and spamming won't change that.
>>
>> I'm sorry but until you have tested each layer individually and identified the one that fails, I won't be able to help.
>>
>> --
>> Guillaume Quintard
>>
>>
>> On Wed, Dec 11, 2019 at 3:08 PM hamidreza hosseini <hrhosseini@hotmail.com <mailto:hrhosseini@hotmail.com>> wrote:
>> Hi,
>> please help me to solve this problem, this is the link that i explaine my question completely:
>> https://varnish-cache.org/lists/pipermail/varnish-misc/2019-December/026754.html <https://varnish-cache.org/lists/pipermail/varnish-misc/2019-December/026754.html>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc@varnish-cache.org <mailto:varnish-misc@varnish-cache.org>
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc <https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc@varnish-cache.org <mailto:varnish-misc@varnish-cache.org>
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc <https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc>
>> --
>> Sent from my iPhone
>
Re: Varnish doesnt purge [ In reply to ]
Many Thanks to you (Vlad Rusu), my problem solved.

Regards
________________________________
From: Vlad Rusu <vlad.rusu@lola.tech>
Sent: Sunday, December 15, 2019 12:09 PM
To: hamidreza hosseini <hrhosseini@hotmail.com>
Cc: Guillaume Quintard <guillaume@varnish-software.com>; varnish-misc@varnish-cache.org <varnish-misc@varnish-cache.org>
Subject: Re: Varnish doesnt purge

I think the following will explain this in great detail

http://varnish-cache.org/docs/6.3/users-guide/vcl-hashing.html
https://github.com/varnishcache/varnish-cache/blob/6.3/bin/varnishd/builtin.vcl#L87-L95

By default (your case) any cache entry will be uniquely identified by the following combination: host header + url

No matter how many varnish server you stack, unless you specifically alter the Host header in one of them, they will all store the object in their own storage using the same hash key

If you call for https://google.com/a and you have stacked 2 varnishes, both of them will save the object as: google.com/a<http://google.com/a>

When you want to purge that object, you have to send the same data - which is why I gave you an example that altered the the Host header when sending the PURGE request to the 2nd varnish

I?m personally a fan of BANNING when using varnish-cache (non commercial). Read more about this below.
https://varnish-cache.org/docs/6.3/users-guide/purging.html + https://book.varnish-software.com/4.0/chapters/Cache_Invalidation.html#lurker-friendly-bans

Also worth printing this one on your wall if you want to truly understand the journey of a request through varnish http://varnish-cache.org/docs/6.3/reference/states.html#reference-states

Thanks,

?
Vlad Rusu
Cell: +40758066019

Lola Tech | lola.tech<https://lola.tech>

On 15 Dec 2019, at 21:54, hamidreza hosseini <hrhosseini@hotmail.com<mailto:hrhosseini@hotmail.com>> wrote:

Many Thanks for your answer
You mean that in production i should add additional header to my purge request because it stored this with the header content like the ip address of first varnish
OK, for purging i need wich header to add to my purge request and how can i get that what are that? And how they stored and how can i purge them

Best Regards

Get Outlook for Android<https://aka.ms/ghei36>

________________________________
From: Vlad Rusu <vlad.rusu@lola.tech<mailto:vlad.rusu@lola.tech>>
Sent: Sunday, December 15, 2019 11:16:59 PM
To: hamidreza hosseini <hrhosseini@hotmail.com<mailto:hrhosseini@hotmail.com>>
Cc: Guillaume Quintard <guillaume@varnish-software.com<mailto:guillaume@varnish-software.com>>; varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org> <varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org>>
Subject: Re: Varnish doesnt purge

Try this when purging

varnish ram: curl -v -k -X PURGE http://192.168.200.13/Naserfeiz.mp4 -o /dev/null -H "newtrack: yes"

varnish file (2nd one): curl -v -k -X PURGE http://192.168.200.12:8080/Naserfeiz.mp4 -o /dev/null -H "newtrack: yes? -H "Host: 192.168.200.13<http://192.168.200.13/Naserfeiz.mp4>?

Notice the difference? You to understand HOW content is stored in varnish?s storage in order to understand how to evict it - my comment about the hash routine

Thanks

On 15 Dec 2019, at 21:31, hamidreza hosseini <hrhosseini@hotmail.com<mailto:hrhosseini@hotmail.com>> wrote:

I dont understand your message,

I cant solve this problem

Instead of saying "appericiate to your diplomacy!!!!!!!!!!" you can help me!

I dont have enough knoledge on Varnish,

I dont get your text about hash, i didnt use hash.

Best Regards


________________________________
From: Vlad Rusu <vlad.rusu@lola.tech<mailto:vlad.rusu@lola.tech>>
Sent: Sunday, December 15, 2019 7:48:42 PM
To: hamidreza hosseini <hrhosseini@hotmail.com<mailto:hrhosseini@hotmail.com>>
Cc: Guillaume Quintard <guillaume@varnish-software.com<mailto:guillaume@varnish-software.com>>; varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org> <varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org>>
Subject: Re: Varnish doesnt purge

I appreciate Guillaume?s diplomacy :)

Your issue (one of them at least) is that in your cascaded varnish setup the hash key of the cached url, in both varnish storages, is the one you use to originally access the asset. Your vcl hash routine might also be a custom one.. that would be the next thing to look at if what I?m saying here doesn?t fix it.

So, purge
http://192.168.200.13/Naserfeiz.mp4
and NOT
http://192.168.200.12:8080/Naserfeiz.mp4
when in the cascaded setup

Cheers

On Sat, 14 Dec 2019 at 09:16, hamidreza hosseini <hrhosseini@hotmail.com<mailto:hrhosseini@hotmail.com>> wrote:
Hi,
Please annwer my question, I sent you my test

________________________________
From: varnish-misc <varnish-misc-bounces+hrhosseini=hotmail.com@varnish-cache.org<mailto:hotmail.com@varnish-cache.org>> on behalf of hamidreza hosseini <hrhosseini@hotmail.com<mailto:hrhosseini@hotmail.com>>
Sent: Wednesday, December 11, 2019 11:49 PM
To: Guillaume Quintard <guillaume@varnish-software.com<mailto:guillaume@varnish-software.com>>
Cc: varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org> <varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org>>
Subject: Re: Varnish doesnt purge

I do all steps now again, this is the resault:
STEP ONE:
First of all i configure a varnish (with file backend) in front of swift/Nginx and i ask the url that exist in swift with curl,

varnish-file$curl -I http://192.168.200.12:8080/Naserfeiz.mp4 -H "newtrack: yes"

HTTP/1.1 200 OK
Server: nginx
Date: Thu, 12 Dec 2019 07:29:32 GMT
Content-Type: text/plain
Content-Length: 23521499
Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
ETag: "3a1794b0-166e8db"
X-Varnish: 2
Age: 0
Via: 1.1 varnish (Varnish/6.0)
Accept-Ranges: bytes
Connection: keep-alive


varnish cach that object and then i purge the url:

varnish-file$ curl -v -k -X PURGE http://192.168.200.12:8080/Naserfeiz.mp4 -o /dev/null -H "newtrack: yes"

* Trying 192.168.200.12...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.12 (192.168.200.12) port 8080 (#0)
> PURGE /Naserfeiz.mp4 HTTP/1.1
> Host: 192.168.200.12:8080<http://192.168.200.12:8080/>
> User-Agent: curl/7.47.0
> Accept: */*
> newtrack: yes
>
< HTTP/1.1 200 Purged
< Date: Thu, 12 Dec 2019 07:29:16 GMT
< Server: Varnish
< X-Varnish: 32770
< Content-Type: text/html; charset=utf-8
< Retry-After: 5
< Content-Length: 240
< Accept-Ranges: bytes
< Connection: keep-alive
<
{ [240 bytes data]
100 240 100 240 0 0 124k 0 --:--:-- --:--:-- --:--:-- 234k
* Connection #0 to host 192.168.200.12 left intact

The object purged successfully and it didnt exist in cach anymore.

This is varnishlog resault:

varnish-file$ sudo varnishlog -d -g request
* << Request >> 2
- Begin req 1 rxreq
- Timestamp Start: 1576135743.442646 0.000000 0.000000
- Timestamp Req: 1576135743.442646 0.000000 0.000000
- ReqStart 192.168.200.12 5432 a0
- ReqMethod HEAD
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.12:8080<http://192.168.200.12:8080/>
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.12
- VCL_call RECV
- VCL_return hash
- VCL_call HASH
- VCL_return lookup
- VCL_call MISS
- VCL_return fetch
- Link bereq 3 fetch
- Timestamp Fetch: 1576135743.444347 0.001701 0.001701
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespHeader Server: nginx
- RespHeader Date: Thu, 12 Dec 2019 07:29:32 GMT
- RespHeader Content-Type: text/plain
- RespHeader Content-Length: 23521499
- RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
- RespHeader ETag: "3a1794b0-166e8db"
- RespHeader X-Varnish: 2
- RespHeader Age: 0
- RespHeader Via: 1.1 varnish (Varnish/6.0)
- VCL_call DELIVER
- VCL_return deliver
- Timestamp Process: 1576135743.444382 0.001736 0.000036
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576135743.444474 0.001828 0.000092
- ReqAcct 112 0 112 295 0 295
- End
** << BeReq >> 3
-- Begin bereq 2 fetch
-- VCL_use boot
-- Timestamp Start: 1576135743.442807 0.000000 0.000000
-- BereqMethod HEAD
-- BereqURL /Naserfeiz.mp4
-- BereqProtocol HTTP/1.1
-- BereqHeader Host: 192.168.200.12:8080<http://192.168.200.12:8080/>
-- BereqHeader User-Agent: curl/7.47.0
-- BereqHeader Accept: */*
-- BereqHeader newtrack: yes
-- BereqHeader X-Forwarded-For: 192.168.200.12
-- BereqMethod GET
-- BereqHeader Accept-Encoding: gzip
-- BereqHeader X-Varnish: 3
-- VCL_call BACKEND_FETCH
-- VCL_return fetch
-- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6846
-- BackendStart 192.168.200.11 8080
-- Timestamp Bereq: 1576135743.443586 0.000779 0.000779
-- Timestamp Beresp: 1576135743.444017 0.001210 0.000431
-- BerespProtocol HTTP/1.1
-- BerespStatus 200
-- BerespReason OK
-- BerespHeader Server: nginx
-- BerespHeader Date: Thu, 12 Dec 2019 07:29:32 GMT
-- BerespHeader Content-Type: text/plain
-- BerespHeader Content-Length: 23521499
-- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
-- BerespHeader Connection: keep-alive
-- BerespHeader ETag: "3a1794b0-166e8db"
-- BerespHeader Accept-Ranges: bytes
-- TTL RFC 120 10 0 1576135743 1576135743 1576135772 0 0 cacheable
-- VCL_call BACKEND_RESPONSE
-- TTL VCL 86400 10 0 1576135743 cacheable
-- VCL_return deliver
-- Storage file s0
-- Fetch_Body 3 length stream
-- BackendReuse 27 boot.swift_proxy_1
-- Timestamp BerespBody: 1576135743.747835 0.305028 0.303818
-- Length 23521499
-- BereqAcct 181 0 181 241 23521499 23521740
-- End

* << Request >> 32770
- Begin req 32769 rxreq
- Timestamp Start: 1576135756.680792 0.000000 0.000000
- Timestamp Req: 1576135756.680792 0.000000 0.000000
- ReqStart 192.168.200.12 5438 a0
- ReqMethod PURGE
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.12:8080<http://192.168.200.12:8080/>
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.12
- VCL_call RECV
- VCL_return purge
- VCL_call HASH
- VCL_return lookup
- VCL_call PURGE
- VCL_return synth
- Timestamp Process: 1576135756.680982 0.000190 0.000190
- RespHeader Date: Thu, 12 Dec 2019 07:29:16 GMT
- RespHeader Server: Varnish
- RespHeader X-Varnish: 32770
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespReason Purged
- VCL_call SYNTH
- RespHeader Content-Type: text/html; charset=utf-8
- RespHeader Retry-After: 5
- VCL_return deliver
- RespHeader Content-Length: 240
- Storage malloc Transient
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576135756.681190 0.000399 0.000208
- ReqAcct 113 0 113 218 240 458
- End

* << Request >> 32772
- Begin req 32771 rxreq
- Timestamp Start: 1576135769.304679 0.000000 0.000000
- Timestamp Req: 1576135769.304679 0.000000 0.000000
- ReqStart 192.168.200.12 5440 a0
- ReqMethod HEAD
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.12:8080<http://192.168.200.12:8080/>
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.12
- VCL_call RECV
- VCL_return hash
- VCL_call HASH
- VCL_return lookup
- VCL_call MISS
- VCL_return fetch
- Link bereq 32773 fetch
- Timestamp Fetch: 1576135769.306630 0.001951 0.001951
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespHeader Server: nginx
- RespHeader Date: Thu, 12 Dec 2019 07:29:58 GMT
- RespHeader Content-Type: text/plain
- RespHeader Content-Length: 23521499
- RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
- RespHeader ETag: "3a1794b0-166e8db"
- RespHeader X-Varnish: 32772
- RespHeader Age: 0
- RespHeader Via: 1.1 varnish (Varnish/6.0)
- VCL_call DELIVER
- VCL_return deliver
- Timestamp Process: 1576135769.306657 0.001978 0.000027
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576135769.306748 0.002069 0.000091
- ReqAcct 112 0 112 299 0 299
- End
** << BeReq >> 32773
-- Begin bereq 32772 fetch
-- VCL_use boot
-- Timestamp Start: 1576135769.304834 0.000000 0.000000
-- BereqMethod HEAD
-- BereqURL /Naserfeiz.mp4
-- BereqProtocol HTTP/1.1
-- BereqHeader Host: 192.168.200.12:8080<http://192.168.200.12:8080/>
-- BereqHeader User-Agent: curl/7.47.0
-- BereqHeader Accept: */*
-- BereqHeader newtrack: yes
-- BereqHeader X-Forwarded-For: 192.168.200.12
-- BereqMethod GET
-- BereqHeader Accept-Encoding: gzip
-- BereqHeader X-Varnish: 32773
-- VCL_call BACKEND_FETCH
-- VCL_return fetch
-- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6846
-- BackendStart 192.168.200.11 8080
-- Timestamp Bereq: 1576135769.304993 0.000160 0.000160
-- Timestamp Beresp: 1576135769.306170 0.001336 0.001176
-- BerespProtocol HTTP/1.1
-- BerespStatus 200
-- BerespReason OK
-- BerespHeader Server: nginx
-- BerespHeader Date: Thu, 12 Dec 2019 07:29:58 GMT
-- BerespHeader Content-Type: text/plain
-- BerespHeader Content-Length: 23521499
-- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
-- BerespHeader Connection: keep-alive
-- BerespHeader ETag: "3a1794b0-166e8db"
-- BerespHeader Accept-Ranges: bytes
-- TTL RFC 120 10 0 1576135769 1576135769 1576135798 0 0 cacheable
-- VCL_call BACKEND_RESPONSE
-- TTL VCL 86400 10 0 1576135769 cacheable
-- VCL_return deliver
-- Storage file s0
-- Fetch_Body 3 length stream
-- BackendReuse 27 boot.swift_proxy_1
-- Timestamp BerespBody: 1576135769.645490 0.340657 0.339321
-- Length 23521499
-- BereqAcct 185 0 185 241 23521499 23521740
-- End


########

STEP TWO:
First i restart varnish file and varnishlog then:
I add another varnish (with ram backend) in fornt of the varnish file and then i ask the url again in varnish ram (because it is my frontend server and users can access media though this server):

varnish-ram$ curl -I http://192.168.200.13/Naserfeiz.mp4 -H "newtrack: yes"
HTTP/1.1 200 OK
Server: nginx
Date: Thu, 12 Dec 2019 07:33:05 GMT
Content-Type: text/plain
Content-Length: 23521499
Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
ETag: "3a1794b0-166e8db"
X-Varnish: 2
Via: 1.1 varnish (Varnish/6.0)
X-Varnish: 2
Age: 0
Via: 1.1 varnish-v4
Accept-Ranges: bytes
Connection: keep-alive


First varnish file cached the url and then Varnish ram fetched the object from varnish file and cached it , then i purged that url in varnish ram and it purged successfuly and it exactly removed from cached:

varnish-ram:~$ curl -v -k -X PURGE http://192.168.200.13/Naserfeiz.mp4 -o /dev/null -H "newtrack: yes"
* Trying 192.168.200.13...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.13 (192.168.200.13) port 80 (#0)
> PURGE /Naserfeiz.mp4 HTTP/1.1
> Host: 192.168.200.13
> User-Agent: curl/7.47.0
> Accept: */*
> newtrack: yes
>
< HTTP/1.1 200 Purged
< Date: Thu, 12 Dec 2019 07:33:46 GMT
< Server: Varnish
< X-Varnish: 32770
< Content-Type: text/html; charset=utf-8
< Retry-After: 5
< Content-Length: 240
< Accept-Ranges: bytes
< Connection: keep-alive
<
{ [240 bytes data]
100 240 100 240 0 0 73846 0 --:--:-- --:--:-- --:--:-- 117k
* Connection #0 to host 192.168.200.13 left intact


Then i purged the url in varnish file it shows "200 purged" but it still exist in varnish file!!!!!!:

varnish-file$ curl -v -k -X PURGE http://192.168.200.12:8080/Naserfeiz.mp4 -o /dev/null -H "newtrack: yes"
* Trying 192.168.200.12...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 192.168.200.12 (192.168.200.12) port 8080 (#0)
> PURGE /Naserfeiz.mp4 HTTP/1.1
> Host: 192.168.200.12:8080<http://192.168.200.12:8080/>
> User-Agent: curl/7.47.0
> Accept: */*
> newtrack: yes
>
< HTTP/1.1 200 Purged
< Date: Thu, 12 Dec 2019 07:33:39 GMT
< Server: Varnish
< X-Varnish: 5
< Content-Type: text/html; charset=utf-8
< Retry-After: 5
< Content-Length: 236
< Accept-Ranges: bytes
< Connection: keep-alive
<
{ [236 bytes data]
100 236 100 236 0 0 110k 0 --:--:-- --:--:-- --:--:-- 230k
* Connection #0 to host 192.168.200.12 left intact


This is varnishlog resault in varnish file:

varnish-file:~$ sudo varnishlog -d -g request
* << Request >> 2
- Begin req 1 rxreq
- Timestamp Start: 1576135956.460768 0.000000 0.000000
- Timestamp Req: 1576135956.460768 0.000000 0.000000
- ReqStart 192.168.200.13 22300 a0
- ReqMethod GET
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.13
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.13
- ReqHeader Accept-Encoding: gzip
- ReqHeader X-Varnish: 3
- ReqUnset X-Forwarded-For: 192.168.200.13
- ReqHeader X-Forwarded-For: 192.168.200.13, 192.168.200.13
- VCL_call RECV
- VCL_return hash
- VCL_call HASH
- VCL_return lookup
- VCL_call MISS
- VCL_return fetch
- Link bereq 3 fetch
- Timestamp Fetch: 1576135956.463065 0.002297 0.002297
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespHeader Server: nginx
- RespHeader Date: Thu, 12 Dec 2019 07:33:05 GMT
- RespHeader Content-Type: text/plain
- RespHeader Content-Length: 23521499
- RespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
- RespHeader ETag: "3a1794b0-166e8db"
- RespHeader X-Varnish: 2
- RespHeader Age: 0
- RespHeader Via: 1.1 varnish (Varnish/6.0)
- VCL_call DELIVER
- VCL_return deliver
- Timestamp Process: 1576135956.463114 0.002346 0.000049
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576135956.758920 0.298152 0.295806
- ReqAcct 176 0 176 295 23521499 23521794
- End
** << BeReq >> 3
-- Begin bereq 2 fetch
-- VCL_use boot
-- Timestamp Start: 1576135956.461099 0.000000 0.000000
-- BereqMethod GET
-- BereqURL /Naserfeiz.mp4
-- BereqProtocol HTTP/1.1
-- BereqHeader Host: 192.168.200.13
-- BereqHeader User-Agent: curl/7.47.0
-- BereqHeader Accept: */*
-- BereqHeader newtrack: yes
-- BereqHeader Accept-Encoding: gzip
-- BereqHeader X-Varnish: 3
-- BereqHeader X-Forwarded-For: 192.168.200.13, 192.168.200.13
-- BereqHeader X-Varnish: 3
-- VCL_call BACKEND_FETCH
-- VCL_return fetch
-- BackendOpen 27 boot.swift_proxy_1 192.168.200.11 8080 192.168.200.12 6860
-- BackendStart 192.168.200.11 8080
-- Timestamp Bereq: 1576135956.461955 0.000856 0.000856
-- Timestamp Beresp: 1576135956.462607 0.001508 0.000652
-- BerespProtocol HTTP/1.1
-- BerespStatus 200
-- BerespReason OK
-- BerespHeader Server: nginx
-- BerespHeader Date: Thu, 12 Dec 2019 07:33:05 GMT
-- BerespHeader Content-Type: text/plain
-- BerespHeader Content-Length: 23521499
-- BerespHeader Last-Modified: Sun, 19 Nov 2000 08:52:00 GMT
-- BerespHeader Connection: keep-alive
-- BerespHeader ETag: "3a1794b0-166e8db"
-- BerespHeader Accept-Ranges: bytes
-- TTL RFC 120 10 0 1576135956 1576135956 1576135985 0 0 cacheable
-- VCL_call BACKEND_RESPONSE
-- TTL VCL 86400 10 0 1576135956 cacheable
-- VCL_return deliver
-- Storage file s0
-- Fetch_Body 3 length stream
-- BackendReuse 27 boot.swift_proxy_1
-- Timestamp BerespBody: 1576135956.758956 0.297858 0.296349
-- Length 23521499
-- BereqAcct 206 0 206 241 23521499 23521740
-- End

* << Request >> 5
- Begin req 4 rxreq
- Timestamp Start: 1576136019.230666 0.000000 0.000000
- Timestamp Req: 1576136019.230666 0.000000 0.000000
- ReqStart 192.168.200.12 5452 a0
- ReqMethod PURGE
- ReqURL /Naserfeiz.mp4
- ReqProtocol HTTP/1.1
- ReqHeader Host: 192.168.200.12:8080<http://192.168.200.12:8080/>
- ReqHeader User-Agent: curl/7.47.0
- ReqHeader Accept: */*
- ReqHeader newtrack: yes
- ReqHeader X-Forwarded-For: 192.168.200.12
- VCL_call RECV
- VCL_return purge
- VCL_call HASH
- VCL_return lookup
- VCL_call PURGE
- VCL_return synth
- Timestamp Process: 1576136019.230863 0.000198 0.000198
- RespHeader Date: Thu, 12 Dec 2019 07:33:39 GMT
- RespHeader Server: Varnish
- RespHeader X-Varnish: 5
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespReason Purged
- VCL_call SYNTH
- RespHeader Content-Type: text/html; charset=utf-8
- RespHeader Retry-After: 5
- VCL_return deliver
- RespHeader Content-Length: 236
- Storage malloc Transient
- RespHeader Accept-Ranges: bytes
- RespHeader Connection: keep-alive
- Timestamp Resp: 1576136019.231138 0.000473 0.000275
- ReqAcct 113 0 113 214 236 450
- End

########
PLEASE test this scenario for yourself and you can see that i do correct and the object wont remove from varnish-file
Please do this scenario for yourself

________________________________
From: Guillaume Quintard <guillaume@varnish-software.com<mailto:guillaume@varnish-software.com>>
Sent: Wednesday, December 11, 2019 9:44 AM
To: hamidreza hosseini <hrhosseini@hotmail.com<mailto:hrhosseini@hotmail.com>>
Cc: varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org> <varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org>>
Subject: Re: Varnish doesnt purge

Yes it does, we have already proved it./

Each instance purges and will refetch the object for its respective backend. I explained how to test: start with the origin, change the data and check that you actually get the new object. Then add a varnish layer, purge, check, repeat.

You are testing everything at once and are unable to isolate the issue because you refuse to take things step by step, and spamming won't change that.

I'm sorry but until you have tested each layer individually and identified the one that fails, I won't be able to help.

--
Guillaume Quintard


On Wed, Dec 11, 2019 at 3:08 PM hamidreza hosseini <hrhosseini@hotmail.com<mailto:hrhosseini@hotmail.com>> wrote:
Hi,
please help me to solve this problem, this is the link that i explaine my question completely:
https://varnish-cache.org/lists/pipermail/varnish-misc/2019-December/026754.html

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org>
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org<mailto:varnish-misc@varnish-cache.org>
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
--
Sent from my iPhone