Mailing List Archive: #1781: Error with gzip content on Varnish4

#1781: Error with gzip content on Varnish4

Sep 2, 2015, 2:10 AM

Post #1 of 4 (997 views)

#1781: Error with gzip content on Varnish4
--------------------------+----------------------
Reporter: llavaud | Type: defect
Status: new | Priority: normal
Milestone: | Component: varnishd
Version: 4.1.0-TP1 | Severity: major
Keywords: gzip corrupt |
--------------------------+----------------------
Hello,

i have a problem with some gzip content, i request an url on my Varnish3
and i got a correct gzip response but if i do the same request on my
Varnish4 server i got a corrupt gzip response.

- request on the Varnish3 (3.0.5 on debian wheezy):

{{{
curl -H 'Accept-Encoding: gzip' -H 'Host: www.parents.fr'
http://10.114.3.12/Beaute-Forme/Beaute --silent > /tmp/content.gz
}}}

{{{
stat /tmp/content.gz
File: â€˜/tmp/content.gzâ€™
Size: 11804 Blocks: 24 IO Block: 4096 regular file
Device: 801h/2049d Inode: 175099 Links: 1
Access: (0664/-rw-rw-r--) Uid: ( 1000/laurentl) Gid: ( 1000/laurentl)
Access: 2015-09-01 18:25:01.632840870 +0200
Modify: 2015-09-01 18:25:33.733209689 +0200
Change: 2015-09-01 18:25:33.733209689 +0200
Birth: -
}}}

{{{
gunzip -t /tmp/content.gz
}}}

- request on the Varnish4 (4.1.0-TP1 on debian wheezy):

{{{
curl -H 'Accept-Encoding: gzip' -H 'Host: www.parents.fr'
http://10.114.3.14/Beaute-Forme/Beaute --silent > /tmp/content2.gz
}}}

file is smaller:
{{{
stat /tmp/content2.gz
File: â€˜/tmp/content2.gzâ€™
Size: 11794 Blocks: 24 IO Block: 4096 regular file
Device: 801h/2049d Inode: 175055 Links: 1
Access: (0664/-rw-rw-r--) Uid: ( 1000/laurentl) Gid: ( 1000/laurentl)
Access: 2015-09-01 18:26:38.481953633 +0200
Modify: 2015-09-01 18:26:38.509953954 +0200
Change: 2015-09-01 18:26:38.509953954 +0200
Birth: -
}}}

and corrupt:
{{{
gunzip -t /tmp/content2.gz

gzip: /tmp/content2.gz: invalid compressed data--crc error

gzip: /tmp/content2.gz: invalid compressed data--length error
}}}

i have attached the full varnishlog.

thanks in advance.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1781>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs

Re: #1781: Error with gzip content on Varnish4 [ In reply to ]

varnish-bugs at varnish-cache

Sep 8, 2015, 2:44 AM

Post #2 of 4 (978 views)

Permalink

#1781: Error with gzip content on Varnish4
--------------------------+------------------------
Reporter: llavaud | Owner:
Type: defect | Status: new
Priority: normal | Milestone:
Component: varnishd | Version: 4.1.0-TP1
Severity: major | Resolution:
Keywords: gzip corrupt |
--------------------------+------------------------

Comment (by martin):

Hi, and thanks for the detailed bug report. My attempts at reproducing the
problem though has so far been unsuccessful. As it looks like you can
easily reproduce the issue, I wondered if I could ask you to redo it once
more.

What I notice in the logs you attached is that the objects that is
attempted stitched together appears to be empty objects. Or at least they
end up producing zero bytes worth of payload to the final result. This is
contrary to the logs from Varnish 3 where things are working. So likely
something went wrong during the fetch into cache of the numerous ESI
fragments. In your log though most of these fragments are already in the
cache, so they are cache HITs and the logs thus won't show the steps to
fetch them.

Could I ask you to please redo the test on Varnish 4, with the same
logging as before, but make sure that the cache is completely clean at the
time you run it? (e.g. just started). This would allow us to see the
nature of the objects as they enter the cache (e.g. if they are gzip'ed
before entering or gziped by Varnish, etc.)

Regards,
Martin Blix Grydeland

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1781#comment:1>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs

Re: #1781: Error with gzip content on Varnish4 [ In reply to ]

varnish-bugs at varnish-cache

Sep 8, 2015, 5:40 AM

Post #3 of 4 (977 views)

Permalink

Re: #1781: Error with gzip content on Varnish4 [ In reply to ]

varnish-bugs at varnish-cache

Sep 9, 2015, 12:29 PM

Post #4 of 4 (973 views)

Permalink

#1781: Error with gzip content on Varnish4
--------------------------+----------------------------------------
Reporter: llavaud | Owner: Poul-Henning Kamp <phk@â€¦>
Type: defect | Status: closed
Priority: normal | Milestone:
Component: varnishd | Version: 4.1.0-TP1
Severity: major | Resolution: fixed
Keywords: gzip corrupt |
--------------------------+----------------------------------------
Changes (by Poul-Henning Kamp <phk@â€¦>):

* owner: => Poul-Henning Kamp <phk@â€¦>
* status: new => closed
* resolution: => fixed

Comment:

In [723f8a5219e622c10302a70c0596b34d1116418b]:
{{{
#!CommitTicketReference repository=""
revision="723f8a5219e622c10302a70c0596b34d1116418b"
Propagate gzip CRC upwards from nested ESI includes.

Detected by: Laurent Lavaud
Patched by: martin (with minor shuffle by me)

Fixes: #1781
}}}

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1781#comment:3>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs