Mailing List Archive: Cookie inspection

Cookie inspection

May 28, 2007, 10:05 PM

Post #1 of 5 (599 views)

Hi there,

I have a web application which supports a bunch of interface/website
languages via Accept-Language headers and/or a user-overridden cookie
value.

I'd like to modify req.hash in vcl_hash to take into account the value
of this particular cookie (similar to what Denis does with
Accept-Encoding here:
http://projects.linpro.no/pipermail/varnish-misc/2007-May/000407.html
-- neat! :)

From looking through the varnish-cache code, I don't think this type
of operation is supported, and I'm not sure what the best way to
represent this in VCL is.

Including the entire req.http.Cookie in the hash key is suboptimal as
there are a bunch of other cookies set (session cookies, etc)
depending on other variables.

Doing something like req.http.Cookie.LANGUAGE *could* theoretically do
the right thing (and use http_GetHdrField, although some browsers can
send multiple values for the same cookie name... :) and *looks* right,
but obviously has a very different meaning in VCL where it compiles
to:

VRT_GetHdr(sp, 1, "\020Cookie.LANGUAGE:")

I've created a small patch:

http://treehou.se/~omar/cookie-inspection-1.patch

Which implements VRT_GetHdrField and cookie inspection.

This should be a lot more general, but is this something that varnish
could potentially support?

The other thing I'd like to be able to do is have some sort of 'first
match over an array of header values' VCL construct, so you could vary
the hash by whatever languages in Accept-Language your site supports
-- but the cookie thing would be workable too.

Thanks!

Regards,
Omar

Cookie inspection [ In reply to ]

des at linpro

May 29, 2007, 12:53 AM

Post #2 of 5 (586 views)

Permalink

"Omar Kilani" <omar.kilani at gmail.com> writes:
> I'd like to modify req.hash in vcl_hash to take into account the value
> of this particular cookie [...]
> From looking through the varnish-cache code, I don't think this type
> of operation is supported, and I'm not sure what the best way to
> represent this in VCL is.

Poul-Henning is working on making multi-value headers accessible as
associative arrays, e.g. "req.http.cache-control[max-age]" for
"Cache-control: max-age". This implies not only inspection but also
modification of individual fields. Your patch is a step in the right
direction, though it probably duplicates (or conflicts with) part of
Poul-Henning's work.

In any case, could you please open a ticket with the patch as an
attachment?

> Doing something like req.http.Cookie.LANGUAGE *could* theoretically do
> the right thing (and use http_GetHdrField, although some browsers can
> send multiple values for the same cookie name... :)

AFAIR from RFC2616, if multiple values are provided for the same key,
only the last one applies.

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no

Cookie inspection [ In reply to ]

phk at phk

May 29, 2007, 1:16 AM

Post #3 of 5 (600 views)

Permalink

In message <f0cc3e3e0705282205h3b43065atbc1ee434a6adec94 at mail.gmail.com>, "Omar
Kilani" writes:

>Including the entire req.http.Cookie in the hash key is suboptimal as
>there are a bunch of other cookies set (session cookies, etc)
>depending on other variables.
>
>Doing something like req.http.Cookie.LANGUAGE *could* theoretically do
>the right thing (and use http_GetHdrField, although some browsers can
>send multiple values for the same cookie name... :) and *looks* right,
>but obviously has a very different meaning in VCL where it compiles
>to:

The Planned syntax is:
req.http.Cookie[language]

and just as for http headers, the headers will be read sequentially
and the last found value is used.

>http://treehou.se/~omar/cookie-inspection-1.patch

Not bad :-)

>The other thing I'd like to be able to do is have some sort of 'first
>match over an array of header values' VCL construct, so you could vary
>the hash by whatever languages in Accept-Language your site supports

Does your backend include a proper Vary: header when it selects on
language ?

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

Cookie inspection [ In reply to ]

des at linpro

May 29, 2007, 2:18 AM

Post #4 of 5 (599 views)

Permalink

"Poul-Henning Kamp" <phk at phk.freebsd.dk> writes:
> Does your backend include a proper Vary: header when it selects on
> language ?

Applications which use cookies should emit "Vary: Cookie".

The HTTP protocol provides a mechanism for selecting the language of a
page (client sends Accept-Language, server sends Content-Language), so
there should be no reason to use cookies for that.

The part of the patch that makes ';' equivalent to ',' is problematic,
as ';' is normally used to separate subfields within individual header
values (e.g. "Content-Type: text/html;charset=utf-8"). Whoever wrote
RFC 2109 (which uses ';' instead of ',' to separate multiple values) is
an idiot...

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no

Cookie inspection [ In reply to ]

omar.kilani at gmail

May 29, 2007, 4:01 AM

Post #5 of 5 (598 views)

Permalink

On 5/29/07, Poul-Henning Kamp <phk at phk.freebsd.dk> wrote:
> In message <f0cc3e3e0705282205h3b43065atbc1ee434a6adec94 at mail.gmail.com>, "Omar
> Kilani" writes:
>
> >Doing something like req.http.Cookie.LANGUAGE *could* theoretically do
> >the right thing (and use http_GetHdrField, although some browsers can
> >send multiple values for the same cookie name... :) and *looks* right,
> >but obviously has a very different meaning in VCL where it compiles
> >to:
>
> The Planned syntax is:
> req.http.Cookie[language]
>
> and just as for http headers, the headers will be read sequentially
> and the last found value is used.

That makes a lot more sense. :)

Out of curiosity, are things like this documented on Trac or somewhere else?

I probably missed it searching the tickets - has this work been started?

> >http://treehou.se/~omar/cookie-inspection-1.patch
>
> Not bad :-)

Hehe - I really just meant to illustrate what I was getting at with
the patch. ;)

It took longer to write the email than to code up the patch, which I
think says a lot about the hackability of the code and how easy it is
to understand.

> >The other thing I'd like to be able to do is have some sort of 'first
> >match over an array of header values' VCL construct, so you could vary
> >the hash by whatever languages in Accept-Language your site supports
>
> Does your backend include a proper Vary: header when it selects on
> language ?

Yes, but I'm not sure how Vary on Accept-Language is supposed to work
- does it just add the entire value to the hash key or?

Wouldn't the hit rate be low on the cached content if this was the case?

For example, I'd serve en-US to all these browsers (whose language
settings I haven't changed):

Opera (which sends the following Accept-Language value):

en,ja;q=0.9,fr;q=0.8,de;q=0.7,es;q=0.6,it;q=0.5,nl;q=0.4,sv;q=0.3,nb;q=0.2,da;q=0.1,fi;q=0.1,pt;q=0.1,lv;q=0.1,zh-CN;q=0.1,zh-TW;q=0.1,ko;q=0.1,en;q=0.1

Safari:

en

Camino:

en,ja;q=0.9,fr;q=0.9,de;q=0.8,es;q=0.8,it;q=0.7,nl;q=0.6,sv;q=0.6,nb;q=0.5,da;q=0.4,fi;q=0.4,pt;q=0.3,lv;q=0.3,zh-Hans;q=0.2,zh-Hant;q=0.1,ko;q=0.1

But would they all be cached independently and served different
versions of the same document?

Thanks.

Regards,
Omar