Mailing List Archive: Gzip issues with Varnish

Gzip issues with Varnish

Apr 25, 2007, 11:33 AM

Post #1 of 6 (736 views)

Hi,

One of my developer colleagues has experienced an issue at Aftenposten with regards to gzip and varnish.
I think you are experiencing the same issue we had, that is Varnish doesn't really handle gzipped data from the backend intelligently yet. So you could have the following scenario:

0. Varnish startup
1. Client A (with a gzip capable client) requests object 1
2. Varnish will fetch the object from the backend and pass it on to the client, and store the gzipped object in the cache
3. Client B (with a non gzip capable client) requests object 1
4. Varnish will pass on the gzipped object to the client
5. The client will not be able to interpret the data received, and you get various errors..

We "solved" the issue by disabling gzip on the backend until Varnish will handle this properly. For us the sacrifice in bandwidth savings was more than negligible when we take into consideration application server performance.

Regards
--
Denis Braekhus - Teknisk Ansvarlig ABC Startsiden AS
http://www.startsiden.no

Gzip issues with Varnish [ In reply to ]

denis at startsiden

Apr 30, 2007, 8:38 AM

Post #2 of 6 (727 views)

Permalink

----- Dag-Erling Sm?rgrav <des at linpro.no> wrote:
> Denis Br?khus <denis at startsiden.no> writes:
<SNIP>
> Yes, this is exactly what will happen, because Varnish does not yet
> understand the Vary: header which the server uses to indicate that
> the document it returns can vary depending on request headers.

I forgot to mention that it was of course DES who solved the issue for us.

> > We "solved" the issue by disabling gzip on the backend until
> > Varnish will handle this properly. For us the sacrifice in bandwidth
> > savings was more than negligible when we take into consideration
> > application server performance.
> Unfortunately, for some applications, the bandwidth savings are
> considerable... which is why Vary: handling is very high on our
> priority list. We expect to have it working in trunk within a couple
> of months.

Yes I know, that's why i said "for us" :) And I would of course love to have both a speedy appserver and gzipped transfers.
Meanwhile I think this is an issue sites using Varnish should be aware of. And Aftenposten should probably fix their setup..

Regards
--
Denis Braekhus - Teknisk Ansvarlig ABC Startsiden AS
http://www.startsiden.no

Gzip issues with Varnish [ In reply to ]

des at linpro

Apr 30, 2007, 8:40 AM

Post #3 of 6 (726 views)

Permalink

Denis Br?khus <denis at startsiden.no> writes:
> One of my developer colleagues has experienced an issue at
> Aftenposten with regards to gzip and varnish.
>
> I think you are experiencing the same issue we had, that is Varnish
> doesn't really handle gzipped data from the backend intelligently
> yet. So you could have the following scenario:
>
> 0. Varnish startup
> 1. Client A (with a gzip capable client) requests object 1
> 2. Varnish will fetch the object from the backend and pass it on to
> the client, and store the gzipped object in the cache
> 3. Client B (with a non gzip capable client) requests object 1
> 4. Varnish will pass on the gzipped object to the client
> 5. The client will not be able to interpret the data received, and
> you get various errors..

Yes, this is exactly what will happen, because Varnish does not yet
understand the Vary: header which the server uses to indicate that the
document it returns can vary depending on request headers.

> We "solved" the issue by disabling gzip on the backend until Varnish
> will handle this properly. For us the sacrifice in bandwidth savings
> was more than negligible when we take into consideration application
> server performance.

Unfortunately, for some applications, the bandwidth savings are
considerable... which is why Vary: handling is very high on our
priority list. We expect to have it working in trunk within a couple
of months.

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no

Gzip issues with Varnish [ In reply to ]

denis at zeno

May 9, 2007, 9:00 AM

Post #4 of 6 (733 views)

Permalink

Hi

The gzip problem can be circumvented with since revision r1398.

Simply add the following to a varnish vcl script:

sub vcl_hash
{
if (req.http.Accept-Encoding) {
set req.hash += req.http.Accept-Encoding;
}
}

Denis Ahrens

Gzip issues with Varnish [ In reply to ]

phk at phk

May 9, 2007, 9:44 AM

Post #5 of 6 (724 views)

Permalink

In message <922C786D-54C3-480C-88E2-93DC454B8174 at zeno.org>, Denis Ahrens writes
:
>Hi
>
>The gzip problem can be circumvented with since revision r1398.
>
>Simply add the following to a varnish vcl script:
>
>sub vcl_hash
>{
> if (req.http.Accept-Encoding) {
> set req.hash += req.http.Accept-Encoding;
> }
>}

Well, yes, I guess that does it, but depending on how many
differnet Accept-Encoding headers there are out there, your
hit-date may drop and your cache-size explode

Caveat Emptor

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

Gzip issues with Varnish [ In reply to ]

denis at zeno

May 10, 2007, 6:38 AM

Post #6 of 6 (733 views)

Permalink

On 09.05.2007, at 18:44, Poul-Henning Kamp wrote:

> In message <922C786D-54C3-480C-88E2-93DC454B8174 at zeno.org>, Denis
> Ahrens writes
> :
>> Hi
>>
>> The gzip problem can be circumvented with since revision r1398.
>>
>> Simply add the following to a varnish vcl script:
>>
>> sub vcl_hash
>> {
>> if (req.http.Accept-Encoding) {
>> set req.hash += req.http.Accept-Encoding;
>> }
>> }
>
> Well, yes, I guess that does it, but depending on how many
> differnet Accept-Encoding headers there are out there, your
> hit-date may drop and your cache-size explode

Ok, I changed it a little to reflect that:

sub vcl_hash
{
if (req.http.Accept-Encoding ~ "gzip") {
set req.hash += "gzip";
}
else if (req.http.Accept-Encoding ~ "deflate") {
set req.hash += "deflate";
}
}

We only have gzip support, so this should be ok.

Denis Ahrens