Mailing List Archive

VDD summary
Thanks to everybody at the VDD in Oslo yesterday, I think we got a
lot done and we certainly ended up with one of the most actionable
summaries we have ever had.

See you in Hamburg next time:

27.9. optional VDD if we want
28.9. VDD
29.9. optional fun day including TMBG concert *) or techno party depending on taste

*) need to book yourself if you want to go


H2::Send_Frame move to ->error
#1799

Pluggable ops for TCP-Pools and TCP operations
=> Proper way to do this is to generalize and come up with multi-protocol backend support
=> Unknown/large scope: unlikely for a Sept release

#2573 @^#@&^#@-Headers (vmod_header[s])
=> New vmod_header. @Steven writes/proposes .vcc file
=> New parameter: header-charset ? return 400 if not acceptable. Default=what VCL can handle.

SELinux: maintain a policy upstream
=> we ship a policy if we can test it (dridi to figure this out) (That really goes for *anything* in our tree)

CLI:JSON:
=> First step is implement -j in all commands

VFP/VDP in vmods
=> If filter-list is set, setting do_{gzip|gunzip|esi} gives Failure

libvgz compilation warnings

inflate.c:1232:25: error: this statement may fall through [-Werror=implicit-fallthrough=]

state->mode = LENGTH;

~~~~~~~~~~~~^~~~~~~~

either: -Wno-error=implicit-fallthrough= if the code supports it, or just -Wno-error

(new zlib code is still affected by this)


Reported upstream:

https://github.com/madler/zlib/issues/316


=> separate "varnish" CFLAGS (warnings) from the rest

=> don't pass varnish CFLAGS to foreign code

=> don't treat sanitizer flags separately?



Features for Sept 2018 release?

VCL variables? (see https://etherpad.wikimedia.org/p/vdd17q3 lines 103-114)

release proc/docs etc.

* docs:

changes.rst / vs. sphinx/whats-new/changes-X.X.rst

new definition: doc/changes.rst only as a hint to release managers


#1799
=> Pål wrote this: https://github.com/varnishcache/varnish-cache/compare/master...hermunn:master-cache-behavior-vmods and Martin has added some comments. Also, Nils had some comments in an email.
=> Implement func pointer zegerman_catflap() veto function from HSH_Lookup{} slink to write PR
=> Reimplement req.grace, vcl_hit{} becomes "return(deliver)". (Pål does a PR for this)

Director/Refcount/Probe
=> phk suggests BACKEND std.resolve(BACKEND) (slink says it doesn't work for shard directors, which phk thinks is shard's problem, not his.)
-> explained and would be solved by cookie idea below
-> a general mechanism for passing private data from the client to backend side may be interesting --> describe some use cases to phk to persuade him of the usefulness

-> this may be where introducing variables becomes interesting, e.g. declare INT req.var.foo, which then becomes available as an INT as bereq.var.foo
XXX: why wouldn't it simply by var.foo both places ?

=> new CLI command: backend.tell <glob> args
=> ... if glob matches multiple backends, error-returns are ignored
=> ... Should "?" args be magic ?
=> "lightweight" directors for req.backend_hint, storing "cookie" on req.ws which is copied to bereq.ws and reinstated. (?)
=> What happens if backend_hint is shard with shards under it ? How do 2nd-level shards get cookies ?
-> slink volunteers to write PR if phk wants him to

VDP cleanup and API changes (#2673)
=> Sounds fine, consider slink's comments, reiterate, then PHK to review

VRT_CTX, priv->free(), calls outside VCL methods (VDI) (and object destructors?)
=> This one sounds sensible, but requires some homework wrt. breaking APIs and if priv's need broken more (ie: dynamic objects)

(again) native VCC type for regexen
=> VCL_REGEXP, compiler turns CONST-STRING into vre

Next VDD on Norway-Denmark boat ?
No: TMBG concert in Hamburg


--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: VDD summary [ In reply to ]
> => new CLI command: backend.tell <glob> args
> => ... if glob matches multiple backends, error-returns are ignored
> => ... Should "?" args be magic ?

It occurred to me that directors listening to the CLI via a backend.tell
command is a very special case that we could otherwise generalize
to a vmod.cmd CLI command. This would allow modules like xkey
to purge either from VCL or the CLI, just like ban:

vmod.cmd <vmod-name> command [args...]

Example:

vmod.cmd xkey help
help [<command>]
purge [-s] <id> [<id>...]

vmod.cmd xkey help purge
purge [-s] <id> [<id>...]
Perform a purge of all objects matching at least one id,
or a soft purge when the -s flag is present.

vmod.cmd xkey purge 123
42 objects purged.

This way it's up to directors to have their respective VMODs expose
various ways to change their state.

Thanks for the VDD and thanks to Nils for his patience and explanations
of their backend/director advanced needs and how we can keep the
user interface manageable in VCL.

Dridi
_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: VDD summary [ In reply to ]
--------
In message <CABoVN9CUMa3=YzBgVa7s6P5QcUx7-XKJJ8SmiHVmzAOZdasizQ@mail.gmail.com>
, Dridi Boukelmoune writes:
>> => new CLI command: backend.tell <glob> args
>> => ... if glob matches multiple backends, error-returns are ignored
>> => ... Should "?" args be magic ?
>
>It occurred to me that directors listening to the CLI via a backend.tell
>command is a very special case that we could otherwise generalize
>to a vmod.cmd CLI command. This would allow modules like xkey
>to purge either from VCL or the CLI, just like ban:
>
> vmod.cmd <vmod-name> command [args...]

That's a sensible generalization, but in difference from the
backend targeted version, it will be up to the vmod itself to
look a string up to find a backend etc.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: VDD summary [ In reply to ]
On Wed, May 16, 2018 at 9:20 AM, Poul-Henning Kamp <phk@phk.freebsd.dk> wrote:
> --------
> In message <CABoVN9CUMa3=YzBgVa7s6P5QcUx7-XKJJ8SmiHVmzAOZdasizQ@mail.gmail.com>
> , Dridi Boukelmoune writes:
>>> => new CLI command: backend.tell <glob> args
>>> => ... if glob matches multiple backends, error-returns are ignored
>>> => ... Should "?" args be magic ?
>>
>>It occurred to me that directors listening to the CLI via a backend.tell
>>command is a very special case that we could otherwise generalize
>>to a vmod.cmd CLI command. This would allow modules like xkey
>>to purge either from VCL or the CLI, just like ban:
>>
>> vmod.cmd <vmod-name> command [args...]
>
> That's a sensible generalization, but in difference from the
> backend targeted version, it will be up to the vmod itself to
> look a string up to find a backend etc.

And you may have several competing vmods loaded with the same name
thanks to vmod caching. I'm aware of the can of worms it opens
(competing vmods, temperature, active vs labels...) but thought it
would be worth mentioning. Then it's up to the director implementation
to come batteries^Wcli-included and upstream Varnish could set an
example with vmod_directors while Varnish Software could probably find
a couple cases like the one I outlined for vmod_xkey to help the
generalization.

I think that many people would be happy to issue xkey purges via the
CLI much like bans and not have to worry about ACLs or other security
concerns.

Cheers
_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: VDD summary [ In reply to ]
--------
In message <CABoVN9C1cddEwjP+2BEPwQNoY-YaTqxSEoyMurynK_7T7Jq5Fw@mail.gmail.com>
, Dridi Boukelmoune writes:

>And you may have several competing vmods loaded with the same name
>thanks to vmod caching.

Well, due to having multiple loaded VCLs really.

It would have to be possible to say

vmod.tell somevcl::somevmod "bla bla bla"

To target a single one, but it is probably ill-advised to insist
on always only targeting a single one.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: VDD summary [ In reply to ]
On 05/16/2018 09:56 AM, Poul-Henning Kamp wrote:
>
> It would have to be possible to say
>
> vmod.tell somevcl::somevmod "bla bla bla"
>
> To target a single one, but it is probably ill-advised to insist
> on always only targeting a single one.

... and then it's easy to imagine syntax like:

# Mumble to somevmod in every loaded instance
vmod.tell *::somevmod "mumble"

# Mumble to somevmod in the active instance
vmod.tell $$ACTIVE::somevmod "mumble"

Assuming that some string $$ACTIVE can be formed that may not be the
name of any VCL instance.

I doubt that we need somevcl::* or *::*, since it's unlikely that
there's a command that every VMOD's CLI will understand. Unless for some
reason we think of a command that any VMOD with a CLI should support,
which I guess is not out of the question after all. Think of something
like "give me back a version string for the VMOD".


Best,
Geoff
--
** * * UPLEX - Nils Goroll Systemoptimierung

Scheffelstraße 32
22301 Hamburg

Tel +49 40 2880 5731
Mob +49 176 636 90917
Fax +49 40 42949753

http://uplex.de