Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. Varnish>
  3. Bugs
#1643: corrupt range response
varnish-bugs at varnish-cache
Dec 2, 2014, 2:52 AM
Post #1 of 16 (4204 views)
Permalink
#1643: corrupt range response
--------------------------------------+----------------------
Reporter: Jay | Type: defect
Status: new | Priority: normal
Milestone: Later | Component: varnishd
Version: trunk | Severity: normal
Keywords: byte-range request range |
--------------------------------------+----------------------
A client sends a get-request on a 422kbyte audiofile with the following
header:

{{{
Range:bytes=0-
Accept:*/*
Accept-Encoding:identity;q=1, *;q=0
Accept-Language:en-US,en;q=0.8,nb;q=0.6
Cache-Control:no-cache
Connection:keep-alive
Pragma:no-cach
}}}

The response from varnish however caps the size of the file to 65536
bytes:

{{{
Accept-Ranges:bytes
Age:0
Cache-Control:public,no-cache
Connection:keep-alive
Content-Length:65536
Content-Range:bytes 0-65535/65536
Content-Type:audio/mpeg
Date:Tue, 02 Dec 2014 08:27:40 GMT
ETag:"0297684e1ad01:0"
Last-Modified:Fri, 28 Nov 2014 08:01:30 GMT
Server:Microsoft-IIS/8.5
Via:1.1 varnish-v4
X-Cache:MISS
X-Powered-By:ASP.NET
X-Varnish:82728786
}}}

It doesn't happen if I skip using varnish.
Varnish trunk from 19.11.2014

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1643>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1643: corrupt range response [ In reply to ]
varnish-bugs at varnish-cache
Dec 3, 2014, 9:24 AM
Post #2 of 16 (4157 views)
Permalink
#1643: corrupt range response
--------------------------------------+-----------------------
Reporter: Jay | Owner:
Type: defect | Status: needinfo
Priority: normal | Milestone: Later
Component: varnishd | Version: trunk
Severity: normal | Resolution:
Keywords: byte-range request range |
--------------------------------------+-----------------------
Changes (by fgsch):

* status: new => needinfo


Comment:

Can you please include a full varnishlog capture (client and backend) for
a miss and hit?

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1643#comment:1>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1643: corrupt range response [ In reply to ]
varnish-bugs at varnish-cache
Dec 8, 2014, 3:57 AM
Post #3 of 16 (4149 views)
Permalink
#1643: corrupt range response
--------------------------------------+-----------------------
Reporter: Jay | Owner:
Type: defect | Status: needinfo
Priority: normal | Milestone: Later
Component: varnishd | Version: trunk
Severity: normal | Resolution:
Keywords: byte-range request range |
--------------------------------------+-----------------------

Comment (by lkarsten):

What is the result you are expecting? "Content-Range: 0-65535/432128" ?

Does it work as expected if you disable streaming/cut-through forwarding
for this request?

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1643#comment:2>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1643: corrupt range response [ In reply to ]
varnish-bugs at varnish-cache
Dec 8, 2014, 4:36 AM
Post #4 of 16 (4152 views)
Permalink
#1643: corrupt range response
--------------------------------------+-----------------------
Reporter: Jay | Owner:
Type: defect | Status: needinfo
Priority: normal | Milestone: Later
Component: varnishd | Version: trunk
Severity: normal | Resolution:
Keywords: byte-range request range |
--------------------------------------+-----------------------

Comment (by Jay):

Since the client asked for all of the data, the response should probably
be 0-432127/432128, yes.

The problem disappears If I enable piping on the affected URL.
I'll try to get some more logs.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1643#comment:3>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1643: corrupt range response [ In reply to ]
varnish-bugs at varnish-cache
Dec 8, 2014, 6:11 AM
Post #5 of 16 (4153 views)
Permalink
#1643: corrupt range response
--------------------------------------+-----------------------
Reporter: Jay | Owner:
Type: defect | Status: needinfo
Priority: normal | Milestone: Later
Component: varnishd | Version: trunk
Severity: normal | Resolution:
Keywords: byte-range request range |
--------------------------------------+-----------------------

Comment (by fgsch):

This should be only an issue on miss.
"Range: bytes=0-" is a bit silly. You could try disable streaming in this
case:

{{{
if (bereq.http.range ~ "bytes=0-$") {
set beresp.do_stream = false;
}
}}}

I still would like to see a varnishlog capture if possible for both client
and backend.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1643#comment:4>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1643: corrupt range response [ In reply to ]
varnish-bugs at varnish-cache
Dec 18, 2014, 3:44 PM
Post #6 of 16 (4135 views)
Permalink
#1643: corrupt range response
--------------------------------------+-----------------------
Reporter: Jay | Owner:
Type: defect | Status: needinfo
Priority: normal | Milestone: Later
Component: varnishd | Version: trunk
Severity: normal | Resolution:
Keywords: byte-range request range |
--------------------------------------+-----------------------

Comment (by fgsch):

Replying to [comment:4 fgsch]:
> This should be only an issue on miss.
> "Range: bytes=0-" is a bit silly. You could try disable streaming in
this case:
> [..]

Actually thinking back you could remove the range altogether without
disabling streaming:

{{{
if (bereq.http.range ~ "bytes=0-$") {
unset bereq.http.range;
}
}}}

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1643#comment:5>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1643: corrupt range response [ In reply to ]
varnish-bugs at varnish-cache
Dec 19, 2014, 2:15 AM
Post #7 of 16 (4135 views)
Permalink
#1643: corrupt range response
--------------------------------------+-----------------------
Reporter: Jay | Owner:
Type: defect | Status: needinfo
Priority: normal | Milestone: Later
Component: varnishd | Version: trunk
Severity: normal | Resolution:
Keywords: byte-range request range |
--------------------------------------+-----------------------

Comment (by Jay):

Great. Thanks.

The HTTP request comes from IE/Chrome/Firefox and the audio html-tag or
perhaps javascript (a lot of objects on the page so getting a varnishlog
is difficult).
It is a silly range-request, but it seems to be widely used by client
software. It also seem to be valid according to RFC.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1643#comment:6>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1643: corrupt range response [ In reply to ]
varnish-bugs at varnish-cache
Dec 22, 2014, 4:15 AM
Post #8 of 16 (4121 views)
Permalink
#1643: corrupt range response
--------------------------------------+-----------------------
Reporter: Jay | Owner:
Type: defect | Status: needinfo
Priority: normal | Milestone: Later
Component: varnishd | Version: trunk
Severity: normal | Resolution:
Keywords: byte-range request range |
--------------------------------------+-----------------------

Comment (by slink):

Jay, can you tell us why the client is sending this Range request in the
first place? I'd presume this is to
* test if the server actually does support range requests or
* ensure that the client gets a content-length equivalent (as a 200 could
be a chunked response)

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1643#comment:7>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1643: corrupt range response [ In reply to ]
varnish-bugs at varnish-cache
Dec 22, 2014, 2:01 PM
Post #9 of 16 (4117 views)
Permalink
#1643: corrupt range response
--------------------------------------+-----------------------
Reporter: Jay | Owner:
Type: defect | Status: needinfo
Priority: normal | Milestone: Later
Component: varnishd | Version: trunk
Severity: normal | Resolution:
Keywords: byte-range request range |
--------------------------------------+-----------------------

Comment (by Jay):

It is a part of a page using the builtin audio/html5-player in multiple
browsers. Either the browser is doing it directly or it is some sort of
javascript handling the request.
The reason behind the range-request is not know. Perhaps if the size of
the file was larger, the player would have split the file fetching into
multiple request. The author of the code could have just dropped the
header if fetching the entire file at once, but is seems to be a valid
request anyway.


Per RFC 2616, section 14.35.1 Byte Ranges

If the last-byte-pos value is absent, or if the value is greater than or
equal to the current length of the entity-body, last-byte-pos is taken to
be equal to one less than the current length of the entity- body in bytes.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1643#comment:8>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1643: corrupt range response [ In reply to ]
varnish-bugs at varnish-cache
Jan 5, 2015, 4:12 AM
Post #10 of 16 (4104 views)
Permalink
#1643: corrupt range response
--------------------------------------+--------------------
Reporter: Jay | Owner: phk
Type: defect | Status: new
Priority: normal | Milestone: Later
Component: varnishd | Version: trunk
Severity: normal | Resolution:
Keywords: byte-range request range |
--------------------------------------+--------------------
Changes (by slink):

* status: needinfo => new
* owner: => phk


--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1643#comment:9>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1643: corrupt range response [ In reply to ]
varnish-bugs at varnish-cache
Jan 30, 2015, 7:52 AM
Post #11 of 16 (4037 views)
Permalink
#1643: corrupt range response
--------------------------------------+--------------------
Reporter: Jay | Owner: phk
Type: defect | Status: new
Priority: normal | Milestone: Later
Component: varnishd | Version: trunk
Severity: normal | Resolution:
Keywords: byte-range request range |
--------------------------------------+--------------------

Comment (by jstrom):

Hi,

fgs on IRC thought my issues was related to this bug, so I'll repeat my
findings here:

I'm running varnish 4.0.3 on Mac OS 10.9.5 (dev) with a very basic VCL.
I have a GET request which is pass'ed directly in vcl_recv.
This request comes from FFMPeg and has a "Range: bytes=0-" header.
The backend will reply with "Transfer-Encoding: chunked". One single chunk
with 8220 bytes is present in the response.

Every now and then (2 out of 5 requests maybe), varnish returns the full
response. But some times it returns 206 Partial Content and "Content-
Range: bytes 0-2667/2668".

tcpdump indicates that varnish starts to reply after 3 response TCP
packets has arrived, i..e before full response has been obtained. This
response includes the Content-Range header. then nothing happens until it
has received the rest of the backend packets, at which time the frontend
connection is closed.

My solution for now is just to remove the "Range: bytes=0-" header in the
request, as I don't use that anyway.
One guess from IRC:
{{{
16:22:53 < thierr1> jstrom: still guessing but what could lead to such
a situation may be : varnish receives the range request, makes a
background fetch and has received only 2668 bytes when it replies
to client (streaming mode), so issues a 206 with the current content
it has. From my point of view, this is RFC compliant, but not necessarily
what is expected. Maybe you can check this with tcpdump, you will see
if varnish has received full response when it starts stre
}}}
I agree, it *might* be compliant but definitely not what I expected. Also,
no streaming stuff is enabled in the VCL, so I'd expect full reply.

These are the Client -> Varnish headers:
{{{
GET /some/path/to/file HTTP/1.1
User-Agent: Lavf/56.15.100
Accept: */*
Range: bytes=0-
Connection: close
Host: localhost:8081
Icy-MetaData: 1

HTTP/1.1 206 Partial Content
Date: Fri, 30 Jan 2015 14:14:13 GMT
Server: Apache/2.2.29 (Unix) mod_ssl/2.2.29 OpenSSL/1.0.1j DAV/2 PHP/5.6.4
X-Powered-By: PHP/5.6.4
Content-Type: application/x-mpegURL
X-Varnish: 100148
Age: 0
Via: 1.1 varnish-v4
Transfer-Encoding: chunked
Connection: close
Accept-Ranges: bytes
Content-Range: bytes 0-2667/2668
}}}

These are the Varnish -> Backend headers:
{{{
GET /some/path/to/file HTTP/1.1
User-Agent: Lavf/56.15.100
Accept: */*
Range: bytes=0-
Icy-MetaData: 1
X-Forwarded-For: ::1
Host: example.com
X-Varnish: 100149

HTTP/1.1 200 OK
Date: Fri, 30 Jan 2015 14:14:13 GMT
Server: Apache/2.2.29 (Unix) mod_ssl/2.2.29 OpenSSL/1.0.1j DAV/2 PHP/5.6.4
X-Powered-By: PHP/5.6.4
Transfer-Encoding: chunked
Content-Type: application/x-mpegURL
}}}


Full VCL:
{{{
vcl 4.0;

import std;

backend stream {
.host = "example.com";
.port = "80";
}

sub vcl_recv {
# Pipe unknown requests
if (req.method != "GET" &&
req.method != "HEAD" &&
req.method != "PUT" &&
req.method != "POST" &&
req.method != "TRACE" &&
req.method != "OPTIONS" &&
req.method != "DELETE") {
/* Non-RFC2616 or CONNECT which is weird. */
return (pipe);
}

# Kill range header
#unset req.http.Range;

set req.http.Host = "example.com";
if(req.url ~ ".ts$") {
return(hash);
}
return(pass);
}

sub vcl_backend_response {
if (bereq.url ~ "\.ts$") {
set beresp.uncacheable = false;
set beresp.ttl = 86400s;
}

return (deliver);
}
}}}

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1643#comment:10>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1643: corrupt range response [ In reply to ]
varnish-bugs at varnish-cache
May 5, 2015, 4:47 AM
Post #12 of 16 (3848 views)
Permalink
#1643: corrupt range response
--------------------------------------+---------------------
Reporter: Jay | Owner: phk
Type: defect | Status: closed
Priority: normal | Milestone: Later
Component: varnishd | Version: trunk
Severity: normal | Resolution: fixed
Keywords: byte-range request range |
--------------------------------------+---------------------
Changes (by phk):

* status: new => closed
* resolution: => fixed


Comment:

I overlooked this ticket yesterday. This should be fixed now.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1643#comment:11>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1643: corrupt range response [ In reply to ]
varnish-bugs at varnish-cache
Sep 21, 2015, 3:28 AM
Post #13 of 16 (3548 views)
Permalink
#1643: corrupt range response
--------------------------------------+-----------------------
Reporter: Jay | Owner: phk
Type: defect | Status: reopened
Priority: normal | Milestone: Later
Component: varnishd | Version: trunk
Severity: normal | Resolution:
Keywords: byte-range request range |
--------------------------------------+-----------------------
Changes (by onovy):

* status: closed => reopened
* resolution: fixed =>


Comment:

this bug is still preset in 4.0.3 varnish

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1643#comment:12>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1643: corrupt range response [ In reply to ]
varnish-bugs at varnish-cache
Sep 21, 2015, 3:37 AM
Post #14 of 16 (3548 views)
Permalink
#1643: corrupt range response
--------------------------------------+--------------------
Reporter: Jay | Owner:
Type: defect | Status: new
Priority: normal | Milestone: Later
Component: varnishd | Version: 4.0.3
Severity: normal | Resolution:
Keywords: byte-range request range |
--------------------------------------+--------------------
Changes (by phk):

* owner: phk =>
* status: reopened => new
* version: trunk => 4.0.3


--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1643#comment:13>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1643: corrupt range response [ In reply to ]
varnish-bugs at varnish-cache
Sep 21, 2015, 4:04 AM
Post #15 of 16 (3543 views)
Permalink
#1643: corrupt range response
--------------------------------------+--------------------
Reporter: Jay | Owner:
Type: defect | Status: new
Priority: normal | Milestone: Later
Component: varnishd | Version: 4.0.3
Severity: normal | Resolution:
Keywords: byte-range request range |
--------------------------------------+--------------------

Comment (by lkarsten):

Comment for VS: Look up how much effort it is porting this to 4.0. Since
4.1 is right around the corner, that may be an option.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1643#comment:14>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1643: corrupt range response [ In reply to ]
varnish-bugs at varnish-cache
Sep 21, 2015, 4:05 AM
Post #16 of 16 (3548 views)
Permalink
#1643: corrupt range response
--------------------------------------+-----------------------
Reporter: Jay | Owner: lkarsten
Type: defect | Status: new
Priority: normal | Milestone: Later
Component: varnishd | Version: 4.0.3
Severity: normal | Resolution:
Keywords: byte-range request range |
--------------------------------------+-----------------------
Changes (by lkarsten):

* owner: => lkarsten


--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1643#comment:15>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal