Mailing List Archive

[PATCH] PRIV pointers available in vmod obj methods
Hi,
as discussed at bugwash today, I'm posting a patch that makes private
pointers(PRIV_TASK, PRIV_TOP, PRIV_VCL) available in VMODS objects.
Please refer to trac ticket #1800 for more details.

--
Arianna Aondio
Software Developer | Varnish Software AS
Mobile: +47 98062619
We Make Websites Fly
www.varnish-software.com
Re: [PATCH] PRIV pointers available in vmod obj methods [ In reply to ]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 10/26/2015 01:32 PM, Arianna Aondio wrote:
> Hi, as discussed at bugwash today, I'm posting a patch that makes
> private pointers(PRIV_TASK, PRIV_TOP, PRIV_VCL) available in VMODS
> objects. Please refer to trac ticket #1800 for more details.

As mentioned in the ticket, the patch has tests for the use of
PRIV_TASK in an object method, but none for PRIV_TOP or PRIV_VCL.


Best,
Geoff
- --
** * * UPLEX - Nils Goroll Systemoptimierung

Scheffelstraße 32
22301 Hamburg

Tel +49 40 2880 5731
Mob +49 176 636 90917
Fax +49 40 42949753

http://uplex.de
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCAAGBQJWLiNiAAoJEOUwvh9pJNURbtYQAKw5CwnrBdDGd4vlCyLnT8kB
n1eg4rzpI2KbIw7xOYEzw96GIN1Z8fddzb/ANr4lQxV+lwaa1OtXCC/sjaU5ohcx
Ow9+kcA4vxF08QtNKMt9+WmcxIipYkeTNmSo+sULcQCVSCpTcQP32tg4ll1QVL4T
s/17s+FOuoRUZwphEGuV/82XCCOu/ZRILzP9UGbQ988guiJvftePd9veAQj42M0E
CwoDcJWSkL98ZT57vR1v5O6ycEprjYAil3RkRMZ2b8vFrucYee/gSsIEHiRaD/DB
EEHzdlxpvkOaXwhclBVV3kYMKDJKSEPfpQM0203usz4G6rFIObbt5myZ5Keeir/S
vaX55ICpdIMRFgBFB558WlVjE1uW8JP4KhbJO2v2hcCNm2C2W/FOpANTYP1djUi9
PTAvwURZvK8y4tIepF5anH3WkvaO+6TQc5mQTc0bib7YFUq4EJKIuWLvjPouCTFQ
d7DDwslKbolIunbl+EvIjts66c5zADI3FEgO6TSCZ6Cg4a6i0CC2BJ2C5OtgQP3f
aUruc6ZrAVp5T1318s6CTQnyKPwgw8Erf9sqObbCN9yewZXadtgvl9cPUtjLoYYi
DoINhItcvEDpwxsRmIfin6PsujgPihEL1RU8qZcpF912DDUleuwvelzJ4d8I/C1h
T89grx0bvDHklwckq3C9
=ZWoj
-----END PGP SIGNATURE-----

_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: [PATCH] PRIV pointers available in vmod obj methods [ In reply to ]
--------

Ok, wrapping this subject up:

Right now PRIV_foo is named with a mix of lifetime and visibility,
and we seem to need to be able to control both aspects.

The easiest and most backward compatible, seems to be to make
the naming

PRIV_lifetime[_visibility]

With a default visibility of "VMOD" with other possible values being
"CALL" and "OBJ"

Three of the the current four PRIV's maps naturally:

PRIV_VCL -> PRIV_VCL_VMOD
PRIV_TOP -> PRIV_TOP_VMOD
PRIV_TASK -> PRIV_TASK_VMOD

But:

PRIV_CALL -> PRIV_VCL_CALL

The functionality Geoff requires would be PRIV_TASK_OBJ

If we special-case PRIV_CALL, this is backwards compatible.

Comments ?

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: [PATCH] PRIV pointers available in vmod obj methods [ In reply to ]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 11/04/2015 11:58 AM, Poul-Henning Kamp wrote:
> --------
>
> Ok, wrapping this subject up:
>
> Right now PRIV_foo is named with a mix of lifetime and visibility,
> and we seem to need to be able to control both aspects.
>
> The easiest and most backward compatible, seems to be to make the
> naming
>
> PRIV_lifetime[_visibility]
>
> With a default visibility of "VMOD" with other possible values
> being "CALL" and "OBJ"
>
> Three of the the current four PRIV's maps naturally:
>
> PRIV_VCL -> PRIV_VCL_VMOD PRIV_TOP -> PRIV_TOP_VMOD PRIV_TASK ->
> PRIV_TASK_VMOD
>
> But:
>
> PRIV_CALL -> PRIV_VCL_CALL
>
> The functionality Geoff requires would be PRIV_TASK_OBJ
>
> If we special-case PRIV_CALL, this is backwards compatible.
>
> Comments ?

Sounds like the right solution. If you can indulge me for a few
minutes, I'd like to try re-phrasing it, to make sure we all
understand the same thing. It gets a long-winded further down, because
I want to go through all of the permutations.

3 lifetimes * 3 visibilities mean that we'll have 9 PRIV scopes, more
than twice of what we have now. That will make documentation to aid
VMOD authors important, so this will also be a kind of first draft for
that (unless of course I'm getting it all wrong).

For practical purposes, the VMOD author will need to know:

- - When can the scope be used at all (priv != NULL)
- - When is the priv object new (when will priv->priv == NULL)
- - When will it be destroyed (when is priv->free called)
- - Under what circumstances will a function/method declaring a
parameter with this scope see the same object, either in another
function/method call from the same VMOD interface, in the same VCL, or
in another thread.

PRIV_{lifetime}_* controls the second and third conditions, and
PRIV_*_{visibility} controls the last one.

Is PRIV_*_OBJ legal for function declarations? My guess is no, and
that and that priv==NULL for PRIV_*_OBJ in any context that is not an
object method or init/free function.

(Actually, I think it would be best if generate.py would reject
illegal scopes in the first place, by declining to generate vcc_if.*.
But we presently have priv==NULL in backend contexts for PRIV_TOP, so
I'll just assume that priv==NULL for PRIV_*_OBJ in function calls as
well.)

Also not clear to me -- do PRIV_TOP_* and PRIV_TASK_* extend
visibility over restarts and retries? So if I allocate resources for a
PRIV_TOP_* or PRIV_TASK_* scope during a client request, and there's a
restart, will I see the same resources after the restart? And the same
question for PRIV_TASK_* and retries in backend threads.

I'm guessing that the answer is yes in both cases (at least that's
what makes sense to me).

So for the 9 scopes, this is what I come up with:

PRIV_VCL_VMOD (== current PRIV_VCL)

- - new when called the first time for any func/method in the VMOD
interface after the VCL is loaded
- - destroyed when the VCL is unloaded
- - the same for any func/method in the interface, in any thread, using
the scope during the lifetime of a VCL
- - the same object is passed into a VMOD's event function

PRIV_TOP_VMOD (== PRIV_TOP)

- - new when called the first time by any func/method during a client
request
- - destroyed when the client request is completed
- - the same for any func/method in the interface during a client
request and its ESI subrequests, including after restarts
- - different for different client requests/subrequests (in different
threads)
- - priv==NULL always for backend threads

PRIV_TASK_VMOD (== PRIV_TASK)

- - new when called the first time by any func/method during a client or
backend task
- - destroyed when the client/backend task is completed
- - the same for any func/method in the interface during the same c/b
task, including after restarts and retries
- - different in calls during different c/b tasks (in different threads)

PRIV_VCL_CALL (== PRIV_CALL)

- - new when invoked the first time by any distinct VCL statement after
the VCL is loaded
- - destroyed when the VCL is unloaded
- - the same for that particular call, in any thread, during the
lifetime of VCL
- - different for any other invocation, i.e. by a different VCL
statement, even of the same func/method

PRIV_TOP_CALL

- - new when invoked by a VCL statement during a client request
- - destroyed when the client request is completed
- - the same for that invocation during any ESI subrequest, and after
restarts
- - different for other client requests/subrequests, and for other
invocations of the same func/method
- - priv==NULL in backend threads

PRIV_TASK_CALL

- - new when invoked by a VCL statement during a client or backend task
- - destroyed when the client/backend task is completed
- - for the client side: the same for that invocation during any ESI
subrequest, and after restarts
- - backend side: the same for that invocation after retries
- - different for any other invocation (even of the same func/method)
- - If I'm seeing this right, the usefulness would be to preserve
information for one call over restarts or retries.

PRIV_VCL_OBJ

- - new when called the first time for any method of a specific object
after the VCL is loaded
- - destroyed when the VCL is unloaded
- - the same for any method call of that object, in any thread, during
the lifetime of a VCL
- - priv==NULL for functions

PRIV_TOP_OBJ

- - new when called the first time for any method of a specific object
during a client request
- - destroyed when the client request is completed
- - the same for that object during the client request and its
subrequests, and after restarts
- - different for other client requests/subrequests, and for different
objects
- - priv==NULL for backend threads and functions

PRIV_TASK_OBJ

- - new when called the first time for a specific object during a client
or backend task
- - destroyed when the client/backend task is completed
- - the same for that object during that task, including ESI subrequests
and after restarts on the client side, and after retries on the
backend side
- - different for different objects, and for different c/b tasks in
different threads
- - priv==NULL for functions


Sorry for another long email, and/or if I got it all wrong. I do think
we'll eventually need to understand and document the 9 scopes at this
level of detail.


Best,
Geoff
- --
** * * UPLEX - Nils Goroll Systemoptimierung

Scheffelstraße 32
22301 Hamburg

Tel +49 40 2880 5731
Mob +49 176 636 90917
Fax +49 40 42949753

http://uplex.de
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCAAGBQJWTwThAAoJEOUwvh9pJNUR6mUQAJXMPurMZ84uy7UgsDFBl24Y
m7dfqENa+J7tsO4VLwBR5OJM4Z3T5rk+gBks6t9RI+P+53aPIeBbZNHdbCGZOIDD
jIAfVRSt38nW6nYeuU4/5v0pNFI3zeM7p4AOviqc6K3MOWe+dVSGN3S0pTJ+e5qC
2/QlEj2NmQJwfXaqL0I8zHfad2v5syDLKPmsfs60vooyUFsPRc4jyGpYAMNu5Y/a
uS2H2hK1XpmKMcz1urKz5k4FbqpCYl/2DaoyRWSWBxav79LwLaNaBeDkrTGdU5Iu
vbTjxQpp+de5w/AJ6zpi9bz+iCGiPflSNFzdnUhPpsA86SxeWHT0JZQsD11/ikho
emaUU7u5DarWlquHhcqF6HFFOzS8LRRO1UMFceu1Apj5YhrtGlDMGU0SWTQ+KsS4
Pyx4QejKHB/L2I9OoLFO1bl3yP2xoscoTHmW0d9Lmxtx2yuAN1CHAfcq3nSgvXcH
wB2KoUEFIpjV3OUlyQvfjOXJdIVApDVyG3fpEXyo6cB3mGGtwfRE4XThhG2g7ZxX
osOGGUsYLEi9O+0sCgG7JZkxTlWqgXBgu7Sqb3jSr/YXX6BspfrYBE+7OLg/6zlT
oOLrwrtke3+mCJvJGvXtTiDs0x3eFy6YFc6uX4SA4LF8k7tadlEvvo2twzW5R8r4
WChNwb5Rxn+PQZfvxhh0
=Mo4h
-----END PGP SIGNATURE-----

_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: [PATCH] PRIV pointers available in vmod obj methods [ In reply to ]
I have updated
https://github.com/varnishcache/varnish-cache/wiki/VIP1:-PRIV_*-visibility-and-lifetime-control
with an Alternative Idea

_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: [PATCH] PRIV pointers available in vmod obj methods [ In reply to ]
On Mon, Aug 29, 2016 at 6:12 PM, Nils Goroll <slink@schokola.de> wrote:
> I have updated
> https://github.com/varnishcache/varnish-cache/wiki/VIP1:-PRIV_*-visibility-and-lifetime-control
> with an Alternative Idea

I like the idea of making it both someone else's problem and making it
as easy as possible for them.

_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: [PATCH] PRIV pointers available in vmod obj methods [ In reply to ]
On 26/10/15 13:32, Arianna Aondio wrote:
> Hi,
> as discussed at bugwash today, I'm posting a patch that makes private
> pointers(PRIV_TASK, PRIV_TOP, PRIV_VCL) available in VMODS objects.
> Please refer to trac ticket #1800 for more details.

There was only one line missing to get this for current master, see
https://github.com/varnishcache/varnish-cache/pull/2061 lib/libvcc/vcc_vmod.c


Nils

_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev