Mailing List Archive

Looking at a VCL I use to set hostname and this seems to work-

sub vcl_deliver {
set resp.http.X-Cache-Node = server.hostname;
}

Maybe you can use in form of server.environmental_variable

From: varnish-misc <varnish-misc-bounces+charles=beachcamera.com@varnish-cache.org> on behalf of Hugues Alary <hugues@betabrand.com>
Date: Monday, September 9, 2019 at 6:09 PM
To: varnish-misc <varnish-misc@varnish-cache.org>
Subject: Environment variables in VCL

Hi there,

I'm using Varnish in a Kubernetes cluster and my configuration needs to use environment variables.

More specifically, I'm trying to declare a `backend` with a `.host` whose value is an environment variable.

backend b0 {
.host = "$ENVIRONMENT_VARIABLE_HOSTNAME";
.port = "80";
.connect_timeout = 300s;
.first_byte_timeout = 300s;
.between_bytes_timeout = 300s;
}

This doesn't work. It tried using the std.get_env() function but it doesn't work in this context.

Is there any way to use environment variables in VCL, or do I have to create a configuration template?

Thanks for your help,

Cheers,
-Hugues

Hi,

The trouble is that the backends are C structures created when you compile
the VCL, but std.getenv() is called when you use the compile VCL, so that
way too late.

I see two ways:
- use a dynamic backend vmod (like vmod_goto), allowing you to regularly
interrogate DNS a build an elastic director
- create a static backend point to you k8s proxy, and just set
req.http.host to whatever value you want.

The last one is two of course create a template, but remember that varnish
resolves domain names in static backend when it compiles the VCL, so the
resolved IP may not be right all the time.
--
Guillaume Quintard


On Tue, Sep 10, 2019 at 2:17 AM Bender, Charles <charles@beachcamera.com>
wrote:

> Looking at a VCL I use to set hostname and this seems to work-
>
>
>
> sub vcl_deliver {
>
> set resp.http.X-Cache-Node = server.hostname;
>
> }
>
>
>
> Maybe you can use in form of server.environmental_variable
>
>
>
> *From: *varnish-misc <varnish-misc-bounces+charles=
> beachcamera.com@varnish-cache.org> on behalf of Hugues Alary <
> hugues@betabrand.com>
> *Date: *Monday, September 9, 2019 at 6:09 PM
> *To: *varnish-misc <varnish-misc@varnish-cache.org>
> *Subject: *Environment variables in VCL
>
>
>
> Hi there,
>
>
>
> I'm using Varnish in a Kubernetes cluster and my configuration needs to
> use environment variables.
>
>
>
> More specifically, I'm trying to declare a `backend` with a `.host` whose
> value is an environment variable.
>
>
> backend b0 {
> .host = "*$ENVIRONMENT_VARIABLE_HOSTNAME*";
> .port = "80";
> .connect_timeout = 300s;
> .first_byte_timeout = 300s;
> .between_bytes_timeout = 300s;
>
> }
>
>
>
> This doesn't work. It tried using the std.get_env() function but it
> doesn't work in this context.
>
>
>
> *Is there any way to use environment variables in VCL, or do I have to
> create a configuration template?*
>
>
>
> Thanks for your help,
>
>
>
> Cheers,
>
> -Hugues
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>

--------
In message <CAJ6ZYQy5RnBjfCs88re6-hMfsQvyuzbOkHoofQ_c+suQNRQjig@mail.gmail.com>
, Guillaume Quintard writes:

>I see two ways:

There is a third way:

You can use:

include "somefile"

anywhere and everywhere in a VCL program.

Before you start your varnishd, do this in a shell script:

echo "\"$ENVIRONMENT_VARIABLE_HOSTNAME\"" > /somewhere/hostname.vcl

Then in VCL:

backend b0 {
.host = include "/somewhere/hostname.vcl" ;
.port 80
...



--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

True, but that amounts to template processing :-)

--
Guillaume Quintard


On Tue, Sep 10, 2019 at 8:44 AM Poul-Henning Kamp <phk@phk.freebsd.dk>
wrote:

> --------
> In message <
> CAJ6ZYQy5RnBjfCs88re6-hMfsQvyuzbOkHoofQ_c+suQNRQjig@mail.gmail.com>
> , Guillaume Quintard writes:
>
> >I see two ways:
>
> There is a third way:
>
> You can use:
>
> include "somefile"
>
> anywhere and everywhere in a VCL program.
>
> Before you start your varnishd, do this in a shell script:
>
> echo "\"$ENVIRONMENT_VARIABLE_HOSTNAME\"" > /somewhere/hostname.vcl
>
> Then in VCL:
>
> backend b0 {
> .host = include "/somewhere/hostname.vcl" ;
> .port 80
> ...
>
>
>
> --
> Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
> phk@FreeBSD.ORG | TCP/IP since RFC 956
> FreeBSD committer | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by incompetence.
>

Guillaume, Poul-Henning, thank you for your suggestions.

Regarding,
> - create a static backend point to you k8s proxy, and just set
req.http.host to whatever value you want.

I don't think this would work for me since I'm working with a backend
definition, trying to change the IP address (/hostname) to which the
backend points to and not simply trying to change the Host header of the
request.

Cheers,
-Hugues



On Mon, Sep 9, 2019 at 11:54 PM Guillaume Quintard <
guillaume@varnish-software.com> wrote:

> True, but that amounts to template processing :-)
>
> --
> Guillaume Quintard
>
>
> On Tue, Sep 10, 2019 at 8:44 AM Poul-Henning Kamp <phk@phk.freebsd.dk>
> wrote:
>
>> --------
>> In message <
>> CAJ6ZYQy5RnBjfCs88re6-hMfsQvyuzbOkHoofQ_c+suQNRQjig@mail.gmail.com>
>> , Guillaume Quintard writes:
>>
>> >I see two ways:
>>
>> There is a third way:
>>
>> You can use:
>>
>> include "somefile"
>>
>> anywhere and everywhere in a VCL program.
>>
>> Before you start your varnishd, do this in a shell script:
>>
>> echo "\"$ENVIRONMENT_VARIABLE_HOSTNAME\"" >
>> /somewhere/hostname.vcl
>>
>> Then in VCL:
>>
>> backend b0 {
>> .host = include "/somewhere/hostname.vcl" ;
>> .port 80
>> ...
>>
>>
>>
>> --
>> Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
>> phk@FreeBSD.ORG | TCP/IP since RFC 956
>> FreeBSD committer | BSD since 4.3-tahoe
>> Never attribute to malice what can adequately be explained by
>> incompetence.
>>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>

@Hugues

I am also a heavy user in kube/openshift environments and this issue is
addressed by means of templating.

Using tools like helm would easily solve this for you. Helm has the value
and it uses it when rendering the backends file - in the form of a
configmap perhaps.

If you can’t/don’t want to use helm, you can still “render” configuration
by consuming values from the ENV by using envsubst for example. This is how
we dis it before helm. Burn the backends.vcl file AT RUNTIME by passing the
backends.vcl file through envsubst.

Either way, my recommendation is to solve this through templating and not
by trying to access ENV directly from the the backends VCL. I’m saying this
having a proper SDLC in mind.

Of course, dynamic backends and all other options Guillaume and
Paul-Henning have mentioned are solid.

Vlad Rusu

On Tue, 10 Sep 2019 at 22:14, Hugues Alary <hugues@betabrand.com> wrote:

> Guillaume, Poul-Henning, thank you for your suggestions.
>
> Regarding,
> > - create a static backend point to you k8s proxy, and just set
> req.http.host to whatever value you want.
>
> I don't think this would work for me since I'm working with a backend
> definition, trying to change the IP address (/hostname) to which the
> backend points to and not simply trying to change the Host header of the
> request.
>
> Cheers,
> -Hugues
>
>
>
> On Mon, Sep 9, 2019 at 11:54 PM Guillaume Quintard <
> guillaume@varnish-software.com> wrote:
>
>> True, but that amounts to template processing :-)
>>
>> --
>> Guillaume Quintard
>>
>>
>> On Tue, Sep 10, 2019 at 8:44 AM Poul-Henning Kamp <phk@phk.freebsd.dk>
>> wrote:
>>
>>> --------
>>> In message <
>>> CAJ6ZYQy5RnBjfCs88re6-hMfsQvyuzbOkHoofQ_c+suQNRQjig@mail.gmail.com>
>>> , Guillaume Quintard writes:
>>>
>>> >I see two ways:
>>>
>>> There is a third way:
>>>
>>> You can use:
>>>
>>> include "somefile"
>>>
>>> anywhere and everywhere in a VCL program.
>>>
>>> Before you start your varnishd, do this in a shell script:
>>>
>>> echo "\"$ENVIRONMENT_VARIABLE_HOSTNAME\"" >
>>> /somewhere/hostname.vcl
>>>
>>> Then in VCL:
>>>
>>> backend b0 {
>>> .host = include "/somewhere/hostname.vcl" ;
>>> .port 80
>>> ...
>>>
>>>
>>>
>>> --
>>> Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
>>> phk@FreeBSD.ORG | TCP/IP since RFC 956
>>> FreeBSD committer | BSD since 4.3-tahoe
>>> Never attribute to malice what can adequately be explained by
>>> incompetence.
>>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc@varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
--
Sent from my iPhone

Hi Vlad,

Thanks for the suggestion. I was using Helm and I'm actually running into
this issue because I'm migrating away from Helm to use Kustomize. Using
`envsubst` is what I set out to do in the end! I'm not a fan of having to
add an init container just for templating, but ???????????.

Cheers,
-Hugues

On Tue, Sep 10, 2019 at 1:55 PM Vlad Rusu <vlad.rusu@lola.tech> wrote:

> @Hugues
>
> I am also a heavy user in kube/openshift environments and this issue is
> addressed by means of templating.
>
> Using tools like helm would easily solve this for you. Helm has the value
> and it uses it when rendering the backends file - in the form of a
> configmap perhaps.
>
> If you can’t/don’t want to use helm, you can still “render” configuration
> by consuming values from the ENV by using envsubst for example. This is how
> we dis it before helm. Burn the backends.vcl file AT RUNTIME by passing the
> backends.vcl file through envsubst.
>
> Either way, my recommendation is to solve this through templating and not
> by trying to access ENV directly from the the backends VCL. I’m saying this
> having a proper SDLC in mind.
>
> Of course, dynamic backends and all other options Guillaume and
> Paul-Henning have mentioned are solid.
>
> Vlad Rusu
>
> On Tue, 10 Sep 2019 at 22:14, Hugues Alary <hugues@betabrand.com> wrote:
>
>> Guillaume, Poul-Henning, thank you for your suggestions.
>>
>> Regarding,
>> > - create a static backend point to you k8s proxy, and just set
>> req.http.host to whatever value you want.
>>
>> I don't think this would work for me since I'm working with a backend
>> definition, trying to change the IP address (/hostname) to which the
>> backend points to and not simply trying to change the Host header of the
>> request.
>>
>> Cheers,
>> -Hugues
>>
>>
>>
>> On Mon, Sep 9, 2019 at 11:54 PM Guillaume Quintard <
>> guillaume@varnish-software.com> wrote:
>>
>>> True, but that amounts to template processing :-)
>>>
>>> --
>>> Guillaume Quintard
>>>
>>>
>>> On Tue, Sep 10, 2019 at 8:44 AM Poul-Henning Kamp <phk@phk.freebsd.dk>
>>> wrote:
>>>
>>>> --------
>>>> In message <
>>>> CAJ6ZYQy5RnBjfCs88re6-hMfsQvyuzbOkHoofQ_c+suQNRQjig@mail.gmail.com>
>>>> , Guillaume Quintard writes:
>>>>
>>>> >I see two ways:
>>>>
>>>> There is a third way:
>>>>
>>>> You can use:
>>>>
>>>> include "somefile"
>>>>
>>>> anywhere and everywhere in a VCL program.
>>>>
>>>> Before you start your varnishd, do this in a shell script:
>>>>
>>>> echo "\"$ENVIRONMENT_VARIABLE_HOSTNAME\"" >
>>>> /somewhere/hostname.vcl
>>>>
>>>> Then in VCL:
>>>>
>>>> backend b0 {
>>>> .host = include "/somewhere/hostname.vcl" ;
>>>> .port 80
>>>> ...
>>>>
>>>>
>>>>
>>>> --
>>>> Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
>>>> phk@FreeBSD.ORG | TCP/IP since RFC 956
>>>> FreeBSD committer | BSD since 4.3-tahoe
>>>> Never attribute to malice what can adequately be explained by
>>>> incompetence.
>>>>
>>> _______________________________________________
>>> varnish-misc mailing list
>>> varnish-misc@varnish-cache.org
>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc@varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>
> --
> Sent from my iPhone
>