please keep the mailing-list CC'd
you backend is telling Varnish to not cache:
-- BerespHeader Cache-Control: no-cache
which is acted upon in the built-in.vcl:
https://github.com/varnishcache/varnish-cache/blob/6.0/bin/varnishd/builtin.vcl#L161 more info here;
https://varnish-cache.org/docs/trunk/users-guide/vcl-built-in-code.html#vcl-built-in-code and maybe this can help too:
https://info.varnish-software.com/blog/finally-understanding-built-in-vcl --
Guillaume Quintard
On Sun, Jun 13, 2021 at 11:16 PM Hamidreza Hosseini <hrhosseini@hotmail.com>
wrote:
> This is one of hls fragments that I want to be cached:
>
> > wget http://stream.test.local/hls/mystream/1623650629260.ts
>
> ```
> * << Request >> 32770
> - Begin req 32769 rxreq
> - Timestamp Start: 1623650670.552461 0.000000 0.000000
> - Timestamp Req: 1623650670.552461 0.000000 0.000000
> - VCL_use boot
> - ReqStart 192.168.200.10 58016 a0
> - ReqMethod GET
> - ReqURL /hls/mystream/1623650629260.ts
> - ReqProtocol HTTP/1.1
> - ReqHeader User-Agent: Wget/1.20.3 (linux-gnu)
> - ReqHeader Accept: */*
> - ReqHeader Accept-Encoding: identity
> - ReqHeader Host: stream.test.local
> - ReqHeader Connection: Keep-Alive
> - ReqHeader X-Forwarded-For: 192.168.200.10
> - VCL_call RECV
> - VCL_return hash
> - ReqUnset Accept-Encoding: identity
> - VCL_call HASH
> - VCL_return lookup
> - VCL_call MISS
> - VCL_return fetch
> - Link bereq 32771 fetch
> - Timestamp Fetch: 1623650670.557642 0.005181 0.005181
> - RespProtocol HTTP/1.1
> - RespStatus 200
> - RespReason OK
> - RespHeader Server: nginx/1.20.1
> - RespHeader Date: Mon, 14 Jun 2021 06:04:30 GMT
> - RespHeader Content-Type: video/mp2t
> - RespHeader Content-Length: 161868
> - RespHeader Last-Modified: Mon, 14 Jun 2021 06:03:51 GMT
> - RespHeader ETag: "60c6f147-2784c"
> - RespHeader Cache-Control: no-cache
> - RespHeader Access-Control-Allow-Origin: *
> - RespHeader Access-Control-Expose-Headers: Content-Length
> - RespHeader Accept-Ranges: bytes
> - RespHeader X-Varnish: 32770
> - RespHeader Age: 0
> - RespHeader Via: 1.1 varnish (Varnish/6.2)
> - VCL_call DELIVER
> - VCL_return deliver
> - Timestamp Process: 1623650670.557660 0.005199 0.000018
> - Filters
> - RespHeader Connection: keep-alive
> - Timestamp Resp: 1623650670.558417 0.005956 0.000757
> - ReqAcct 179 0 179 406 161868 162274
> - End
> ** << BeReq >> 32771
> -- Begin bereq 32770 fetch
> -- VCL_use boot
> -- Timestamp Start: 1623650670.552655 0.000000 0.000000
> -- BereqMethod GET
> -- BereqURL /hls/mystream/1623650629260.ts
> -- BereqProtocol HTTP/1.1
> -- BereqHeader User-Agent: Wget/1.20.3 (linux-gnu)
> -- BereqHeader Accept: */*
> -- BereqHeader Host: stream.test.local
> -- BereqHeader X-Forwarded-For: 192.168.200.10
> -- BereqHeader Accept-Encoding: gzip
> -- BereqHeader X-Varnish: 32771
> -- VCL_call BACKEND_FETCH
> -- VCL_return fetch
> -- BackendOpen 25 b1 {Backend_ip} 80 {Varnish_ip} 49734
> -- BackendStart {Backend_ip} 80
> -- Timestamp Bereq: 1623650670.552739 0.000084 0.000084
> -- Timestamp Beresp: 1623650670.557325 0.004669 0.004586
> -- BerespProtocol HTTP/1.1
> -- BerespStatus 200
> -- BerespReason OK
> -- BerespHeader Server: nginx/1.20.1
> -- BerespHeader Date: Mon, 14 Jun 2021 06:04:30 GMT
> -- BerespHeader Content-Type: video/mp2t
> -- BerespHeader Content-Length: 161868
> -- BerespHeader Last-Modified: Mon, 14 Jun 2021 06:03:51 GMT
> -- BerespHeader Connection: keep-alive
> -- BerespHeader ETag: "60c6f147-2784c"
> -- BerespHeader Cache-Control: no-cache
> -- BerespHeader Access-Control-Allow-Origin: *
> -- BerespHeader Access-Control-Expose-Headers: Content-Length
> -- BerespHeader Accept-Ranges: bytes
> -- TTL RFC 120 10 0 1623650671 1623650671 1623650670 0 0
> cacheable
> -- VCL_call BACKEND_RESPONSE
> -- TTL VCL 300 10 0 1623650671 cacheable
> -- TTL VCL 30 10 0 1623650671 cacheable
> -- TTL VCL 120 10 0 1623650671 cacheable
> -- TTL VCL 120 10 0 1623650671 uncacheable
> -- VCL_return deliver
> -- Filters
> -- Storage malloc Transient
> -- Fetch_Body 3 length stream
> -- BackendReuse 25 b1
> -- Timestamp BerespBody: 1623650670.558352 0.005697 0.001028
> -- Length 161868
> -- BereqAcct 202 0 202 348 161868 162216
> -- End
>
> ```
>
> ------------------------------
> *From:* Guillaume Quintard <guillaume@varnish-software.com>
> *Sent:* Sunday, June 13, 2021 8:45 AM
> *To:* Hamidreza Hosseini <hrhosseini@hotmail.com>
> *Cc:* varnish-misc <varnish-misc@varnish-cache.org>
> *Subject:* Re: Varnish wouldn't cache HLS fragments
>
> Hi,
>
> Can you share the output of "varnishlog -g request" for one of those
> requests that should be cached please?
>
> Cheers,
>
> --
> Guillaume Quintard
>
> On Sun, Jun 13, 2021, 00:17 Hamidreza Hosseini <hrhosseini@hotmail.com>
> wrote:
>
> Hi,
> I put varnish in front of my http servers to serve Hls streaming, I want
> varnish cache the fragments but not .m3u8 manifest file,
> I configure it but it cache nothing!
> My configuration file:
>
> ```
> vcl 4.1;
>
> import directors;
>
>
> backend b1 {
> .host = "playback-02";
> .probe = {
> .url = "/";
> .timeout = 150 ms;
> .interval = 10s;
> .window = 6;
> .threshold = 5;
> }
> }
>
>
>
> sub vcl_init {
> # we use round robin director for our backend swift proxies
>
> new hls_cluster = directors.round_robin();
> hls_cluster.add_backend(b1);
>
> }
>
> acl purge {
> "localhost";
> }
>
>
> sub vcl_recv {
>
> set req.backend_hint = hls_cluster.backend();
> if (req.method == "PURGE") {
> if (!client.ip ~ purge) {
> return(synth(405,"Not allowed."));
> }
> return (purge);
> }
>
> if (req.url ~ "\.m3u8$") {
> return (pass);
> }
> }
>
>
>
>
>
> sub vcl_backend_response {
> # cache for half of a day
> set beresp.ttl=5m;
> # Don't cache 404 responses
>
> if (bereq.url ~ "\.(aac|dash|m4s|mp4|ts)$") {
> set beresp.ttl = 30s;
> }
>
> if ( beresp.status == 404 ) {
> set beresp.ttl = 120s;
> set beresp.uncacheable = true;
> return (deliver);
> }
> if (beresp.status == 500 || beresp.status == 502 || beresp.status ==
> 503 || beresp.status == 504)
> {
> set beresp.uncacheable = true;
> }
> }
>
> ```
>
> Varnish version:
> varnishd (varnish-6.0.7 revision 525d371e3ea0e0c38edd7baf0f80dc226560f26e)
> Copyright (c) 2006 Verdens Gang AS
> Copyright (c) 2006-2020 Varnish Software AS
>
> Distribution: Ubuntu 20.04 LTS
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
>