Mailing List Archive

On Tue, Oct 15, 2019 at 2:22 PM Jeff Potter
<jpotter-varnish@codepuppy.com> wrote:
>
>
> Hi All,
>
> This seems like an easy task, but I haven’t been able to figure out how to do it or find any posts online. Is there a way to only send certain headers to a backend?
>
> I.e. in our application, we know we only need X-Forwarded-For and Cookie headers. I know I can unset other known headers (User-Agent, etc) — but how can I unset *all* other headers?
>
> (We’re on VCL format 4.0.)

Hi Jeff,

This is not doable in VCL, this kind of header whitelisting could be
implemented with a VMOD but I'm not aware of any one doing that.

Dridi
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

On 10/15/19 16:21, Jeff Potter wrote:
>
> This seems like an easy task, but I haven’t been able to figure out
> how to do it or find any posts online. Is there a way to only send
> certain headers to a backend?
>
> I.e. in our application, we know we only need X-Forwarded-For and
> Cookie headers. I know I can unset other known headers (User-Agent, etc)
> — but how can I unset *all* other headers?

VMOD re2 has the .hdr_filter() method for the set object:

https://code.uplex.de/uplex-varnish/libvmod-re2

https://code.uplex.de/uplex-varnish/libvmod-re2/blob/master/README.rst#L1775

VOID myset.hdr_filter(HTTP, BOOL whitelist)

The HTTP parameter can be one of req, resp, bereq or beresp. If the
whitelist parameter is true (default true), then only matching headers
are retained. Otherwise it's a blacklist -- matching headers are removed.

So for your use case:

sub vcl_init {
new whitelist = re2.set(anchor=start, case_sensitive=false);
whitelist.add("X-Forwarded-For:");
whitelist.add("Cookie:");
whitelist.add("Host:");
whitelist.compile();
}

sub vcl_backend_fetch {
whitelist.hdr_filter(bereq);
}

I took the liberty of adding the Host header to your whitelist, since
it's required since HTTP/1.1. Even if your backends "happen" to work
without it, I wouldn't leave it out, since it's not well-formed HTTP
otherwise (might stop working, for example, if the backend apps are
upgraded).


HTH,
Geoff
--
** * * UPLEX - Nils Goroll Systemoptimierung

Scheffelstraße 32
22301 Hamburg

Tel +49 40 2880 5731
Mob +49 176 636 90917
Fax +49 40 42949753

http://uplex.de

On Wed, Oct 16, 2019 at 4:08 PM Geoff Simmons <geoff@uplex.de> wrote:
>
> On 10/15/19 16:21, Jeff Potter wrote:
> >
> > This seems like an easy task, but I haven’t been able to figure out
> > how to do it or find any posts online. Is there a way to only send
> > certain headers to a backend?
> >
> > I.e. in our application, we know we only need X-Forwarded-For and
> > Cookie headers. I know I can unset other known headers (User-Agent, etc)
> > — but how can I unset *all* other headers?
>
> VMOD re2 has the .hdr_filter() method for the set object:
>
> https://code.uplex.de/uplex-varnish/libvmod-re2
>
> https://code.uplex.de/uplex-varnish/libvmod-re2/blob/master/README.rst#L1775
>
> VOID myset.hdr_filter(HTTP, BOOL whitelist)
>
> The HTTP parameter can be one of req, resp, bereq or beresp. If the
> whitelist parameter is true (default true), then only matching headers
> are retained. Otherwise it's a blacklist -- matching headers are removed.
>
> So for your use case:
>
> sub vcl_init {
> new whitelist = re2.set(anchor=start, case_sensitive=false);
> whitelist.add("X-Forwarded-For:");
> whitelist.add("Cookie:");
> whitelist.add("Host:");
> whitelist.compile();
> }
>
> sub vcl_backend_fetch {
> whitelist.hdr_filter(bereq);
> }

TIL, thanks!

> I took the liberty of adding the Host header to your whitelist, since
> it's required since HTTP/1.1. Even if your backends "happen" to work
> without it, I wouldn't leave it out, since it's not well-formed HTTP
> otherwise (might stop working, for example, if the backend apps are
> upgraded).

Agreed, there are other control headers that one may want to keep in
the whitelist, otherwise you may break conditional or partial requests,
and everything else I don't remember off the top of my head.


Dridi
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Thanks, Geoff and Dridi! We’ll give this a try.

And Dridi, thanks also for maintaining varnish and this list — “long time lurker; very rare poster” — since I have the microphone, just wanted to send a short note of appreciation.

-Jeff

> On Oct 16, 2019, at 1:36 PM, Dridi Boukelmoune <dridi@varni.sh> wrote:
>
> On Wed, Oct 16, 2019 at 4:08 PM Geoff Simmons <geoff@uplex.de> wrote:
>>
>> On 10/15/19 16:21, Jeff Potter wrote:
>>>
>>> This seems like an easy task, but I haven’t been able to figure out
>>> how to do it or find any posts online. Is there a way to only send
>>> certain headers to a backend?
>>>
>>> I.e. in our application, we know we only need X-Forwarded-For and
>>> Cookie headers. I know I can unset other known headers (User-Agent, etc)
>>> — but how can I unset *all* other headers?
>>
>> VMOD re2 has the .hdr_filter() method for the set object:
>>
>> https://code.uplex.de/uplex-varnish/libvmod-re2
>>
>> https://code.uplex.de/uplex-varnish/libvmod-re2/blob/master/README.rst#L1775
>>
>> VOID myset.hdr_filter(HTTP, BOOL whitelist)
>>
>> The HTTP parameter can be one of req, resp, bereq or beresp. If the
>> whitelist parameter is true (default true), then only matching headers
>> are retained. Otherwise it's a blacklist -- matching headers are removed.
>>
>> So for your use case:
>>
>> sub vcl_init {
>> new whitelist = re2.set(anchor=start, case_sensitive=false);
>> whitelist.add("X-Forwarded-For:");
>> whitelist.add("Cookie:");
>> whitelist.add("Host:");
>> whitelist.compile();
>> }
>>
>> sub vcl_backend_fetch {
>> whitelist.hdr_filter(bereq);
>> }
>
> TIL, thanks!
>
>> I took the liberty of adding the Host header to your whitelist, since
>> it's required since HTTP/1.1. Even if your backends "happen" to work
>> without it, I wouldn't leave it out, since it's not well-formed HTTP
>> otherwise (might stop working, for example, if the backend apps are
>> upgraded).
>
> Agreed, there are other control headers that one may want to keep in
> the whitelist, otherwise you may break conditional or partial requests,
> and everything else I don't remember off the top of my head.
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

On Thu, Oct 17, 2019 at 12:50 PM Jeff Potter
<jpotter-varnish@codepuppy.com> wrote:
>
>
> Thanks, Geoff and Dridi! We’ll give this a try.
>
> And Dridi, thanks also for maintaining varnish and this list — “long time lurker; very rare poster” — since I have the microphone, just wanted to send a short note of appreciation.

Very appreciated too, but you are crediting me much more than I deserve ;-)

PHK, Martin and Nils are the current maintainers and someone from
Uplex is maintaining this list.

Dridi
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc