Mailing List Archive

Reg: global variable in vcl
Hi Team,

I have requirement to use a global variable which i can access through out
VCL (particular vcl). I came across globalvar vmod.
I want to access this variable for all the requests for that vcl. If really
require more information about my requirement I can share.

query:
-I am not able to get any information about this vmod on internet. Means
not used much ? or is there any problem for using this vmod ?
-can you point out exact path/location I can checkout and use this vmod, if
below path is not correct. ( I am using varnish 6.1 )
https://gitlab.com/uplex/varnish/varnish-objvar/tree/6.1

Thank you
Hardik
Re: Reg: global variable in vcl [ In reply to ]
On 5/15/19 13:20, Hardik wrote:
>
> -can you point out exact path/location I can checkout and use this vmod, if
> below path is not correct. ( I am using varnish 6.1 )
> https://gitlab.com/uplex/varnish/varnish-objvar/tree/6.1

That's a branch, you'll want to check out the 6.1 branch.

$ git clone https://code.uplex.de/uplex-varnish/varnish-objvar.git
$ cd varnish-objvar/
$ git checkout 6.1

The you can build the version compatible with 6.1 by following the
instructions in INSTALL.rst.


HTH,
Geoff
--
** * * UPLEX - Nils Goroll Systemoptimierung

Scheffelstraße 32
22301 Hamburg

Tel +49 40 2880 5731
Mob +49 176 636 90917
Fax +49 40 42949753

http://uplex.de
Re: Reg: global variable in vcl [ In reply to ]
Hi Geoff,

Thank you for the hint. I am able to get the vmod src.

Does I have to take care something while using this vmod ? Because first
time going to use and also not getting any information about this.

-Hardik

On Wed, 15 May 2019 at 17:02, Geoff Simmons <geoff@uplex.de> wrote:

> On 5/15/19 13:20, Hardik wrote:
> >
> > -can you point out exact path/location I can checkout and use this vmod,
> if
> > below path is not correct. ( I am using varnish 6.1 )
> >
> https://gitlab.com/uplex/varnish/varnish-objvar/tree/6.1
>
> That's a branch, you'll want to check out the 6.1 branch.
>
> $ git clone https://code.uplex.de/uplex-varnish/varnish-objvar.git
> $ cd varnish-objvar/
> $ git checkout 6.1
>
> The you can build the version compatible with 6.1 by following the
> instructions in INSTALL.rst.
>
>
> HTH,
> Geoff
> --
> ** * * UPLEX - Nils Goroll Systemoptimierung
>
> Scheffelstraße 32
> 22301 Hamburg
>
> Tel +49 40 2880 5731
> Mob +49 176 636 90917
> Fax +49 40 42949753
>
> http://uplex.de
>
>
Re: Reg: global variable in vcl [ In reply to ]
Hi Team,

what is difference between below this ?

https://github.com/varnish/varnish-modules/blob/master/src/vmod_var.c

and
https://code.uplex.de/uplex-varnish/varnish-objvar.git

I am familiar with first link flow wise. Generation of *.vcc file in second
link is totally different then previous one which I am not familiar with.
Someone please clarify confusion that which one should use for global
variable vmod ?

Thank you
Hardik

On Thu, 16 May 2019 at 14:11, Hardik <hetardik.p@gmail.com> wrote:

> Hi Geoff,
>
> Thank you for the hint. I am able to get the vmod src.
>
> Does I have to take care something while using this vmod ? Because first
> time going to use and also not getting any information about this.
>
> -Hardik
>
> On Wed, 15 May 2019 at 17:02, Geoff Simmons <geoff@uplex.de> wrote:
>
>> On 5/15/19 13:20, Hardik wrote:
>> >
>> > -can you point out exact path/location I can checkout and use this
>> vmod, if
>> > below path is not correct. ( I am using varnish 6.1 )
>> >
>> https://gitlab.com/uplex/varnish/varnish-objvar/tree/6.1
>>
>> That's a branch, you'll want to check out the 6.1 branch.
>>
>> $ git clone https://code.uplex.de/uplex-varnish/varnish-objvar.git
>> $ cd varnish-objvar/
>> $ git checkout 6.1
>>
>> The you can build the version compatible with 6.1 by following the
>> instructions in INSTALL.rst.
>>
>>
>> HTH,
>> Geoff
>> --
>> ** * * UPLEX - Nils Goroll Systemoptimierung
>>
>> Scheffelstraße 32
>> 22301 Hamburg
>>
>> Tel +49 40 2880 5731
>> Mob +49 176 636 90917
>> Fax +49 40 42949753
>>
>> http://uplex.de
>>
>>
Re: Reg: global variable in vcl [ In reply to ]
On 5/16/19 10:41, Hardik wrote:
>
> Does I have to take care something while using this vmod ? Because first
> time going to use and also not getting any information about this.

That's a very generic question. Do you have something specific in mind?

objvar is actually a bundle of four VMODs, and they each have
documentation as .rst files in the src/ directory. These get converted
and installed as man pages on installation. So you might want to start
there.

To take a guess at possible answers to the generic question, the main
thing to decide is the scope and mutability of the variables you want to
use. That's what distinguishes the four VMODs:

- constant: global constants (immutable)

- globalvar: global variables (mutable)

- taskvar: variables with task scope - scoped to the client or backend
context of a single request/response transaction

- topvar: ESI top-level scope - these are variables accessible in client
context for an ESI include and all of its sub-includes

You should be aware that the global variables use locking and memory
barriers to prevent simultaneous updates. So if you have a lot of
threads accessing a variable at the same time, you could run into lock
contention.

Conceivably you might want to think about how memory is allocated for
the different scopes. If you only need a couple of variables in VCL, you
won't notice the difference. But if you're using a lot of them, if
you're trying to minimize RAM usage, trying to minimize workspace size
configurations to reduce Varnish's footprint, if you're using a lot of
workspace for other purposes (large ESI trees, other VMODs that consume
workspace ...) ... then it could matter.

The global constants and variables are allocated from the system heap,
for the lifetime of the VCL instance. Task scoped variables are
allocated from the workspace of the client or backend task. ESI
top-level variables are allocated from client workspace, which is
re-used for the entire ESI include tree. Workspaces are reset and
re-used for new request/response transactions.

> what is difference between below this ?
>
> https://github.com/varnish/varnish-modules/blob/master/src/vmod_var.c
>
> and
> https://code.uplex.de/uplex-varnish/varnish-objvar.git
>
> I am familiar with first link flow wise. Generation of *.vcc file in second
> link is totally different then previous one which I am not familiar with.
> Someone please clarify confusion that which one should use for global
> variable vmod ?

VMOD var supports global variables for the STRING, INT and REAL types,
and task-scoped variables for STRING, INT, REAL, DURATION, IP and BACKEND.

The objvar VMODs cover all of the VCL data types, and have the different
options for scope and mutability described above. They also have methods
to undefine or protect a variable (a protected variable may not be
changed), and check if a variable is defined or protected. See the .rst
files for documentation, or the manpage if you install any of the VMODs.


HTH,
Geoff
--
** * * UPLEX - Nils Goroll Systemoptimierung

Scheffelstraße 32
22301 Hamburg

Tel +49 40 2880 5731
Mob +49 176 636 90917
Fax +49 40 42949753

http://uplex.de
Re: Reg: global variable in vcl [ In reply to ]
Hi,

On 17/05/2019 10:16, Hardik wrote:
> what is difference between below this ?
>
> https://github.com/varnish/varnish-modules/blob/master/src/vmod_var.c
>
> and 
> https://code.uplex.de/uplex-varnish/varnish-objvar.git 

I would like to add to Geoff's response:

* Conceptual differences between the vmods:

while the var vmod uses strings to identify variables, the objvar vmods use
objects. The main difference is that objects need to be declared in vcl_init,
which can be an advantage for code clarity, but does not allow for runtime
declaration.

Also, object access should be somewhat faster and scale better.

* with respect to global variables and types:

On 17/05/2019 13:09, Geoff Simmons wrote:
> VMOD var supports global variables for the STRING, INT and REAL types,

not that I would be aware, to my knowledge the var vmod only supports
STRING for global variables.

vmod globalvar from the objvar project support virtually all types.

In general, to achieve consistent concurrent access to global variables, the var
vmod copies the global variable string value to each caller's workspace.

The globalvar vmod uses refcounting to avoid copying. Other than for short
strings, it should thus use less workspace than the "var" vmod and be more
efficient and scalable. (refcounting will most definitely use less workspace for
multiple read access within the same task).

And once I am at it:

> -I am not able to get any information about this vmod on internet. Means not
> used much ?

It is comparably new (initially released 27 October 2018) and also we at UPLEX
are probably not particularly good with marketing, so vmod var will probably be
more widely used simply for existing _much_ longer and being mentioned more often.

The objvar bundle is being used (and sponsored by) some very high traffic sites
and, as mentioned before, was written for optimal flexibility, scalability and
efficiency.

Nils
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Reg: global variable in vcl [ In reply to ]
On 17/05/2019 16:29, Nils Goroll wrote:
> The globalvar vmod uses refcounting to avoid copying.

Risking to get carried away into implementation details: It actually does this
only for types which require it, immediate values like REAL, INT etc. do not
need refcounting.
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Reg: global variable in vcl [ In reply to ]
On 5/17/19 16:29, Nils Goroll wrote:
>
> * with respect to global variables and types:
>
> On 17/05/2019 13:09, Geoff Simmons wrote:
> > VMOD var supports global variables for the STRING, INT and REAL types,
>
> not that I would be aware, to my knowledge the var vmod only supports
> STRING for global variables.

Yeah I think you're right. I was confused by what the docs for VMOD var
say, IMO this could use some better clarity:

-----8<-----
It supports strings, integers and real numbers. There are methods to get
and set each data type.

Global variables have a lifespan that extends across requests and
VCLs, for as long as the vmod is loaded.

The remaining functions have PRIV_TASK lifespan ...
-----8<-----

The first sentence lists STRING, INT and REAL, and then further down
there's a contrast to the "remaining functions" with task scope. Since
the paragraph in the middle says something about global variables, I
thought it meant that there is support for global variables of those
three types.

If I'm reading the .vcc file correctly:

- VMOD var supports six types, not just those three (STRING, INT, REAL,
IP, DURATION and BACKEND).

- It supports task scope for all six types.

- It also supports global scope for STRING.


Best,
Geoff
--
** * * UPLEX - Nils Goroll Systemoptimierung

Scheffelstraße 32
22301 Hamburg

Tel +49 40 2880 5731
Mob +49 176 636 90917
Fax +49 40 42949753

http://uplex.de
Re: Reg: global variable in vcl [ In reply to ]
Thank you Geoff and Nils for detailed explanation. I am trying to use one
of the vmods as per my requirement. If I face any problem will get back to
you. Thanks again.

Hardik

On Fri, 17 May 2019 at 20:05, Nils Goroll <slink@schokola.de> wrote:

> On 17/05/2019 16:29, Nils Goroll wrote:
> > The globalvar vmod uses refcounting to avoid copying.
>
> Risking to get carried away into implementation details: It actually does
> this
> only for types which require it, immediate values like REAL, INT etc. do
> not
> need refcounting.
>