Mailing List Archive

Using varnish and vouch-proxy together
Hello,

I try to use vouch-proxy and varnish (v7) together to build a authorisation
proxy. vouch-proxy is written to work with nginx ngx_http_auth_request_module

https://github.com/vouch/vouch-proxy
https://nginx.org/en/docs/http/ngx_http_auth_request_module.html

Idea:

inspired from
https://web.archive.org/web/20121124064818/https://adayinthelifeof.nl/2012/07/06/using-varnish-to-offload-and-cache-your-oauth-requests/

- use varnish request restart feature
- intercept original client request and make a GET request to vouch-proxy
validate endpoint
- when validated restore the original request and do a restart

in detail:

# vcl_recv
# restarts == 0
# save req method, url, Content-Length, Content-Type in var
# method := GET
# url := /validate
# backend := vouch-proxy
# remove Content-Length, Content-Type
# restarts > 0
# check vouch-proxy headers (roles, groups)
#
# vcl_deliver
# resp == vouch-proxy,GET,/validate,200
# restore req method, url, Content-Length, Content-Type from var
# forward vouch-proxy response headers to req
# restart (original) req

see attached common-vouch-proxy.vcl

It works for client requests without request body (GET, HEAD, …) but not for
POST, PUT, …. POST, PUT run in timeouts, so I think the request body is lost in
the restarted request. Why is the body gone after restart?

I think it should work with the curl vmod but this is not integrated yet.

Thank you very much in advance
tom

--
Tom Anheyer
Senior Developer

BerlinOnline Stadtportal GmbH & Co. KG
Stefan-Heym-Platz 1
10365 Berlin
Germany

Tel.: +49 30 2327-5210
Fax: +49 30 5771180-95
E-Mail: tom.anheyer@berlinonline.de

berlin.de | berlinonline.net

Amtsgericht Berlin-Charlottenburg, HRA 31951
Sitz der Gesellschaft: Berlin,
Deutschland
USt-IdNr.: DE219483549

Persönlich haftender Gesellschafter:
BerlinOnline Stadtportalbeteiligungsges. mbH
Amtsgericht Berlin-Charlottenburg, HRB 79077
Sitz der Gesellschaft: Berlin, Deutschland

Geschäftsführung: Olf Dziadek, Andreas Mängel
Amtierender Vorsitzender des Aufsichtsrates: Lothar Sattler
Re: Using varnish and vouch-proxy together [ In reply to ]
Hi!

You are correct, the request body is, by default, not cached. To correct
this, you need to use the cache_req_body() function from the std vmod:
https://varnish-cache.org/docs/trunk/reference/vmod_std.html#std-cache-req-body

<self promotion time>
I haven't looked at the vcl yet (or the article), but since you mention
vmod_curl, maybe you can try with vmod_reqwest instead (
https://github.com/gquintard/vmod_reqwest). It's a bit different, but
hopefully more powerful, and I'd love to get some feedback on it.
</self promotion time>

Hope that helps!

On Tue, Jul 26, 2022, 00:03 Tom Anheyer | BerlinOnline <
tom.anheyer@berlinonline.de> wrote:

> Hello,
>
> I try to use vouch-proxy and varnish (v7) together to build a authorisation
> proxy. vouch-proxy is written to work with nginx
> ngx_http_auth_request_module
>
> https://github.com/vouch/vouch-proxy
> https://nginx.org/en/docs/http/ngx_http_auth_request_module.html
>
> Idea:
>
> inspired from
>
> https://web.archive.org/web/20121124064818/https://adayinthelifeof.nl/2012/07/06/using-varnish-to-offload-and-cache-your-oauth-requests/
>
> - use varnish request restart feature
> - intercept original client request and make a GET request to vouch-proxy
> validate endpoint
> - when validated restore the original request and do a restart
>
> in detail:
>
> # vcl_recv
> # restarts == 0
> # save req method, url, Content-Length, Content-Type in var
> # method := GET
> # url := /validate
> # backend := vouch-proxy
> # remove Content-Length, Content-Type
> # restarts > 0
> # check vouch-proxy headers (roles, groups)
> #
> # vcl_deliver
> # resp == vouch-proxy,GET,/validate,200
> # restore req method, url, Content-Length, Content-Type from var
> # forward vouch-proxy response headers to req
> # restart (original) req
>
> see attached common-vouch-proxy.vcl
>
> It works for client requests without request body (GET, HEAD, …) but not
> for
> POST, PUT, …. POST, PUT run in timeouts, so I think the request body is
> lost in
> the restarted request. Why is the body gone after restart?
>
> I think it should work with the curl vmod but this is not integrated yet.
>
> Thank you very much in advance
> tom
>
> --
> Tom Anheyer
> Senior Developer
>
> BerlinOnline Stadtportal GmbH & Co. KG
> Stefan-Heym-Platz 1
> 10365 Berlin
> Germany
>
> Tel.: +49 30 2327-5210
> Fax: +49 30 5771180-95
> E-Mail: tom.anheyer@berlinonline.de
>
> berlin.de | berlinonline.net
>
> Amtsgericht Berlin-Charlottenburg, HRA 31951
> Sitz der Gesellschaft: Berlin,
> Deutschland
> USt-IdNr.: DE219483549
>
> Persönlich haftender Gesellschafter:
> BerlinOnline Stadtportalbeteiligungsges. mbH
> Amtsgericht Berlin-Charlottenburg, HRB 79077
> Sitz der Gesellschaft: Berlin, Deutschland
>
> Geschäftsführung: Olf Dziadek, Andreas Mängel
> Amtierender Vorsitzender des Aufsichtsrates: Lothar Sattler
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
Re: Using varnish and vouch-proxy together [ In reply to ]
Hi Guillaume,

Thumbs up for your cache_req_body() hint. It works perfect now. Thank you very much for your help.

TIL: POST, PUT, … requests are not restart-able by default.

tom


Am Dienstag, dem 26.07.2022 um 00:50 -0700 schrieb Guillaume Quintard:
Hi!

You are correct, the request body is, by default, not cached. To correct this, you need to use the cache_req_body() function from the std vmod: https://varnish-cache.org/docs/trunk/reference/vmod_std.html#std-cache-req-body

<self promotion time>
I haven't looked at the vcl yet (or the article), but since you mention vmod_curl, maybe you can try with vmod_reqwest instead (https://github.com/gquintard/vmod_reqwest). It's a bit different, but hopefully more powerful, and I'd love to get some feedback on it.
</self promotion time>

Hope that helps!

On Tue, Jul 26, 2022, 00:03 Tom Anheyer | BerlinOnline <tom.anheyer@berlinonline.de<mailto:tom.anheyer@berlinonline.de>> wrote:
Hello,

I try to use vouch-proxy and varnish (v7) together to build a authorisation
proxy. vouch-proxy is written to work with nginx ngx_http_auth_request_module

https://github.com/vouch/vouch-proxy
https://nginx.org/en/docs/http/ngx_http_auth_request_module.html

Idea:

inspired from
https://web.archive.org/web/20121124064818/https://adayinthelifeof.nl/2012/07/06/using-varnish-to-offload-and-cache-your-oauth-requests/

- use varnish request restart feature
- intercept original client request and make a GET request to vouch-proxy
validate endpoint
- when validated restore the original request and do a restart

in detail:

# vcl_recv
# restarts == 0
# save req method, url, Content-Length, Content-Type in var
# method := GET
# url := /validate
# backend := vouch-proxy
# remove Content-Length, Content-Type
# restarts > 0
# check vouch-proxy headers (roles, groups)
#
# vcl_deliver
# resp == vouch-proxy,GET,/validate,200
# restore req method, url, Content-Length, Content-Type from var
# forward vouch-proxy response headers to req
# restart (original) req

see attached common-vouch-proxy.vcl

It works for client requests without request body (GET, HEAD, …) but not for
POST, PUT, …. POST, PUT run in timeouts, so I think the request body is lost in
the restarted request. Why is the body gone after restart?

I think it should work with the curl vmod but this is not integrated yet.

Thank you very much in advance
tom


--

Tom Anheyer
Senior Developer

BerlinOnline Stadtportal GmbH & Co. KG
Stefan-Heym-Platz 1
10365 Berlin
Germany

Tel.: +49 30 2327-5210
Fax: +49 30 5771180-95
E-Mail: tom.anheyer@berlinonline.de<mailto:tom.anheyer@berlinonline.de>

berlin.de | berlinonline.net

Amtsgericht Berlin-Charlottenburg, HRA 31951
Sitz der Gesellschaft: Berlin, Deutschland
USt-IdNr.: DE219483549

Persönlich haftender Gesellschafter:
BerlinOnline Stadtportalbeteiligungsges. mbH
Amtsgericht Berlin-Charlottenburg, HRB 79077
Sitz der Gesellschaft: Berlin, Deutschland

Geschäftsführung: Olf Dziadek, Andreas Mängel
Amtierender Vorsitzender des Aufsichtsrates: Lothar Sattler