Mailing List Archive

Using user defined variable in backend definition?
Hi,

I'm experimenting with user defined variables, and when using them in regular string concatenation, and in synch output, it works fine. But I would like to use them in the backend definitions, and I simply can't get it to work.

backend myBackend {
.host = var.get("myBackendHost");
}

But that fails with a VCC-compiler error on that line, saying "Expected CSTR got 'var.get'".
I also tried with the "variable" vmod, but it resulted in the same type of error.

Is there any way to get this to work? I would really like to have all environment specific configuration (including backend hostnames) in an environment specific vcl file, and then include it in the main vcl where the backend definitions should be (and use the variables). And I would like to achive this using just VCL (and vmods), so no custom script that does search and replace or anything like that.

So, I would like something like this to work:

default.vcl:
...
import var;

include "environment.vcl";

backend myBackend {
.host = var.get("myBackendHost");
}
...

environment.vcl:
sub vcl_init {
var.set("myBackendHost", "myHostName");
}
Re: Using user defined variable in backend definition? [ In reply to ]
On Mon, Aug 24, 2020 at 5:43 PM Batanun B <batanun@hotmail.com> wrote:
>
> Hi,
>
> I'm experimenting with user defined variables, and when using them in regular string concatenation, and in synch output, it works fine. But I would like to use them in the backend definitions, and I simply can't get it to work.
>
> backend myBackend {
> .host = var.get("myBackendHost");
> }
>
> But that fails with a VCC-compiler error on that line, saying "Expected CSTR got 'var.get'".
> I also tried with the "variable" vmod, but it resulted in the same type of error.
>
> Is there any way to get this to work? I would really like to have all environment specific configuration (including backend hostnames) in an environment specific vcl file, and then include it in the main vcl where the backend definitions should be (and use the variables). And I would like to achive this using just VCL (and vmods), so no custom script that does search and replace or anything like that.
>
> So, I would like something like this to work:
>
> default.vcl:
> ...
> import var;
>
> include "environment.vcl";
>
> backend myBackend {
> .host = var.get("myBackendHost");
> }
> ...
>
> environment.vcl:
> sub vcl_init {
> var.set("myBackendHost", "myHostName");
> }

Hi,

You can't do that, but you can move the backend definition inside
environment.vcl instead to keep your default.vcl the same across all
environments.

Dridi
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Using user defined variable in backend definition? [ In reply to ]
On Mon, Aug 24, 2020 at 11:07 PM Dridi Boukelmoune <dridi@varni.sh> wrote:

> Hi,
>
> You can't do that, but you can move the backend definition inside
> environment.vcl instead to keep your default.vcl the same across all
> environments.

Hi,

Too bad... Strange thing to require hard coded strings. Is there a technical reason for that?

Yeah, I actually ended up doing just that, moving the entire backend definition to the separate file. But I would have preferred having the base structure in the main vcl file, and only the actual host names (and other environment specific configuration) in the separate vcl file. Also, if they were variables, I would be able to use them elsewhere in the vcl (like in synth output, which was my main goal originally). If I want to do that now, I would have to define the same host name multiple times.
Re: Using user defined variable in backend definition? [ In reply to ]
On Tue, Aug 25, 2020 at 9:20 AM Batanun B <batanun@hotmail.com> wrote:
>
> On Mon, Aug 24, 2020 at 11:07 PM Dridi Boukelmoune <dridi@varni.sh> wrote:
>
> > Hi,
> >
> > You can't do that, but you can move the backend definition inside
> > environment.vcl instead to keep your default.vcl the same across all
> > environments.
>
> Hi,
>
> Too bad... Strange thing to require hard coded strings. Is there a technical reason for that?

Yes, definitions need to be constant!

> Yeah, I actually ended up doing just that, moving the entire backend definition to the separate file. But I would have preferred having the base structure in the main vcl file, and only the actual host names (and other environment specific configuration) in the separate vcl file. Also, if they were variables, I would be able to use them elsewhere in the vcl (like in synth output, which was my main goal originally). If I want to do that now, I would have to define the same host name multiple times.

I guess, what you want is constants:

https://github.com/varnishcache/varnish-cache/pull/3134

Specifically, from this example, but redacted to remove test-case syntax:

https://github.com/varnishcache/varnish-cache/pull/3134/files#diff-996416d1d725c18f8a7bf688cc2f4a52

environment.vcl:
const string be_host = "example.com";
const duration be_tmo = 3s;

main vcl:
vcl 4.1;

include "environment.vcl";

backend be {
.host = be_host;
.connect_timeout = be_tmo;
}

Right now you have the option of either moving the whole backend
definition inside environment.vcl or treat your VCL as a template and
populate the environment-specific parts at some stage of your
deployment pipeline.

Cheers,
Dridi
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Using user defined variable in backend definition? [ In reply to ]
On 24/08/2020 19:42, Batanun B wrote:
>
> So, I would like something like this to work:

With the "constant" vmod from https://code.uplex.de/uplex-varnish/varnish-objvar
you can do something like this


/* backends_X.inc.vcl */

backend foo {

??? .host = "127.0.0.1";

}

sub vcl_init {

??? new envbackend = constant.backend(foo);

}



and then in your main vcl include "environment.vcl", which points to some
variant of the above and do

sub vcl_backend_fetch {

??? set bereq.backend = envbackend.get();

}


See
https://code.uplex.de/uplex-varnish/varnish-objvar/blob/master/src/vmod_constant.rst
for more details on the constant vmod


You could do the same thing with a string and the host header or even use a
string and https://github.com/nigoroll/libvmod-dynamic to turn that into a
backend at run time.


hf, Nils
Re: Using user defined variable in backend definition? [ In reply to ]
On 25/08/2020 17:22, Nils Goroll wrote:
> You could do the same thing with a string and the host header or even use a
> string and https://github.com/nigoroll/libvmod-dynamic to turn that into a
> backend at run time.

I noticed that this is probably what you want.

So you might want to consider

/* env.inc.vcl */

sub vcl_init {

??? new be_name = constant.string("foo.bar.com");

}

/* main.vcl */

vcl 4.1;

import constant;??? // I forgot this in my previous email

include "env.inc.vcl";

sub vcl_init {

??? new dyn = dynamic.director();

}

sub vcl_backend_fetch {

??? set bereq.backend = dyn.backend(be_name.get());

}




_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Using user defined variable in backend definition? [ In reply to ]
On 25/08/2020 17:28, Nils Goroll wrote:
> import constant;??? // I forgot this in my previous email

and in this one I forgot

??? import dynamic;


Sorry for posting in a rush

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Using user defined variable in backend definition? [ In reply to ]
Thanks for the suggestions! I'm not sure if we are in the right moment of this project to introduce more competitivity in the form of switching to directors, and a dynamic on top of that. At least not just for this simple thing. But we might need/want to use directors later on, for the dynamic address resolution. And then I might revisit this thread, and look at your example. Thanks!