Here's the decentralized backup of our etherpad at
https://etherpad.wikimedia.org/p/VDD2023q1
Dridi has also created a nicely formatted backup at
https://github.com/varnishcache/varnish-cache/wiki/VDD23Q1
VDD 2023q1 https://etherpad.wikimedia.org/p/VDD2023q1
IRC: #vdd on irc.linpro.no
TOPICS?
happy varnish day
evening programme?
Feb 7 in town
revisit last year's notes
https://etherpad.wikimedia.org/p/VDD2022q2_topics_suggestions
vdd2022q2 #1 varnishlog scalability
===================================
- dridi talks about progress on #1 logging
- keep shared memory, but add option to block varnishd
- different vsm layout for improved concurrency, more page based
- -d option would still be kept, idea of running queries in varnishd is not
on the table any more at this time
- discussion about the nature of the problem
- phk mentions old idea about split log (complete/incomplete tx)
- phk idea to suppress transactions from VCL
- phk idea to use log files only once, open a new one once full
- phk idea keep req grouping in varnishd (into dynamic memory), flush only
once per top level request.
- dridi presents vsl scrubbing idea
- replace log entries or fields with Xes
- discussion about oob info from length - should redaction be a single
fixed token or original length?
tasks
- VIP for scrubbing (dridi)
- VIP batching on toplevel req in varnishd (PHK)
vdd2022q2 #3 Varnish Association
================================
- reiterated over status quo
vdd2022q2 #4 TLS
================
- no progress
- discussion again about the future life of vtest: binary dependency vs. source
dependency vs. submodule
- VS has green light to contribute TLS support
- slink has talked to CF guys at H3 workshop, no known show stoppers
- h2o also has keyless "neverbleed" https://github.com/h2o/neverbleed MIT licensed
- should be one or several VEXTs (openssl, libressl, ...)
- TODO:
- vtest
- VIP the backend interface
- backend definition like vmod function for vext
backend newstuff {
.options = {
.foo = 123;
}
}
options get passed as string at init time
vdd2022q2 #5 H3
===============
- asad: still working on it with msquic
- also researching alternatives
- haproxy also needs patched openssl
Tenants and VCL files
=====================
- hermunn/dridi: want to add a parameter to vcl.load as the base path for
includes with fall back on global vcl_path, ban absolute includes from vcl
vdd2022q2 #6 Extensions
=======================
- do we want extension arguments?
- env vars working find for now
BANs and persistent storages
============================
- loading old storage requires a control system
- control system can assign the "ban id" (current vtim_real)
- CLI command to return min(max(silo(ban_id))) (or per silo max(silo(ban_id))) ?
- feature flag to deny bans from vcl
- multi tenancy support
General idea to ponder: proper VCL "jails" for multi-tenancy
Direction of VCL?
=================
- local, task scoped variables/objects
- vcl functions
- bump vcl version every time? -> NO
- new include version check?
- "umbrella" vcl x.y requires x.Y with Y <= y
for (x in vmod.bla(...)) {
}
- foreach (key[, val[, idx]]) in vmod.something(arguments) {
key type is defined by vmod.something iterator
val type is defined by vmod.something iterator
idx is integer
}
- foreach x in vmod.something(arguments) {
x type is defined by vmod.something iterator
can have x.key x.index, x.whatever
}
# built-in structured fields knowledge?
foreach bla in req.http { // bla has type "http_header"
what members does `bla` have ?
// "literal, string, iterable".fields possible too
if (bla.name == "something") {
for fld in bla.val.fields { // comma, fld has type sf-element (?)
for x in fld.attr { // semi-colon, x has type sf-attr (?)
x.key x.val
}
}
}
}
jobj = vmod_json(input)
for x in jobj.somthing_i_really_want(dont_fail_hard=True) {
}
if (jobj.failed) {
}
foreach bla in req.http {
if (bla.name ~ "(?i)cache-control") {
foreach directive in std.split(bla.val, ", ") {
...
}
}
}
=> Everyone should re-write their complex vcl how they would
want it to look
Fastly-Syntax for SF
resp.http.Cache-Control["max-age"] = 2h;
function foo(myvmod.mytype x) {
}
What about:
for (x in myvmod.bla()) {
foo(x);
}
Also, what if the above is in foo()? Answer: Maybe we will allow recursive
functions, maybe not
# foo takes type returned by vmod?
compliance
==========
Dridi gives a very short talk about a minor change. He will share his 49 page
presentation later.
https://github.com/varnishcache/varnish-cache/files/10707808/vdd23q1-compliance-stuff.pdf
====
DAY2
====
VCL SYNTAX DISCUSSION AROUND FIELD ACCESS
=========================================
if (req.http.Cache-Control[private]) => true if private present
if (req.http.Cache-Control[private] is none) => true if private has no value
if (req.http.Cache-Control[private] == "public") => true if value of private
folds to
string "public"
if (req.http.Cache-Control has private)
if (req.http.Cache-Control[private] is flag) => true if private is present
if (req.http.Cache-Control[private] is not bool) => true if private= is present
if ("private=4"[private] is string) => false
if ("private=*true"[private] is not bool) => false
if ("private"[private]) => bool (true)
if (exists(req.http.Cache-Control[private]))
if (nonzero(req.http.Cache-Control[private]))
if (req.http.Cache-Control[private])
if (req.http.Cache-Control[private].exists)
if (req.http.Cache-Control[private].val)
if (req.http.Cache-Control[private].type)
"private" => True, None, Bool
"private=" => True, "", String
"private=X" => True, "X", String
"private=43" => True, 43, Number
"" => False, None, None
if (req.http.Cache-Control[private]) { //exist
if (req.http.Cache-Control[private].value) { //exist && has value
}
}
==> PHK to write up a strawman
Specifically on issue 3844:
* HEADER == STRING should be equivalent to STRING == HEADER (it is)
* implicit HEADER to BOOL conversion conveys existence (it does)
---
varnishtest cmp
varnish v1 -vcl {
backend be none;
sub vcl_recv {
return (synth(200));
}
sub vcl_synth {
set resp.http.nohdr-eq-empty = req.http.nohdr == "";
set resp.http.empty-eq-nohdr = "" == req.http.nohdr;
}
} -start
client c1 {
txreq
rxresp
expect resp.http.nohdr-eq-empty == true
expect resp.http.empty-eq-nohdr == true
} -start
---
BUGWASH & NEXT VDD
==================
Does it work as is?
- basically yes
- but we want to set up IRC & web bridge with simple basic auth
- should keep logs
@slink to ping @theis about VUG
VUG needs to be a varnish-cache (!) thing
find a sponsor for a venue
VS, UPLEX, fastly, whoever, are welcome as sponsors
10 days after release
idea: 26./27. september
VDD: combine with VUG
sub vcl_connect / vcl_accept / vcl_client_hello
================================================
sub vcl_init {
mytls = new newtls();
mytls.load("/etc/tls/.....pem");
mytls.load("/etc/tls/.....pem");
}
sub vcl_client_hello {
mytls.dynload(tls.sni, ttl=10m); # Dynamicly load cert?
if (tls.sni == "legacysite.asas") {
set tls.ciphers = "ssl, tlsv3";
tls.hint = mytls;
}
set tls.ciphers = "sslv1, tlsv3";
set tls.alpn = "h1,h2,h3=....";
tls.hint = fallback;
return(OK);
}
also: vcl_h[23]_settings, proxy2_tlv
----------------------
VEXT client_ssl has vmod tls
sub vcl_init {
new something = tls.init(...)
something.callback(myfunc)
}
func myfunc(tls.type: arg) {
arg.crytomumblemumble = "bla";
return(tls.OK)
}
-----------------------
sub vcl_accept { // Probably not needed.
if (...) {
filter.use();
}
}
modifying the response body (thi.js)
================================
sub vcl_backend_response {
set beresp.filter_list += "gunzip vmodfoo.spot_foo";
}
sub vcl_backend_body_is_here {
if (vmodfoo.did_you_see_foo) {
if (! beresp.body.try_filter({"regsuball(...) gzip"})) {
return (deliver); # keep the "source" object
}
beresp.body.filter("...");
}
// trailers goes here
}
h2/h3 on the backend, for backend not understanding h1.
=======================================================
we can have h2 in-core, but vext is maybe also viable option.
AGENDA DAY2
===========
h3
add vext_path -> probably just PR it
--
** * * UPLEX - Nils Goroll Systemoptimierung
Scheffelstraße 32
22301 Hamburg
tel +49 40 28805731
mob +49 170 2723133
fax +49 40 42949753
xmpp://slink@jabber.int.uplex.de/
http://uplex.de/
_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
https://etherpad.wikimedia.org/p/VDD2023q1
Dridi has also created a nicely formatted backup at
https://github.com/varnishcache/varnish-cache/wiki/VDD23Q1
VDD 2023q1 https://etherpad.wikimedia.org/p/VDD2023q1
IRC: #vdd on irc.linpro.no
TOPICS?
happy varnish day
evening programme?
Feb 7 in town
revisit last year's notes
https://etherpad.wikimedia.org/p/VDD2022q2_topics_suggestions
vdd2022q2 #1 varnishlog scalability
===================================
- dridi talks about progress on #1 logging
- keep shared memory, but add option to block varnishd
- different vsm layout for improved concurrency, more page based
- -d option would still be kept, idea of running queries in varnishd is not
on the table any more at this time
- discussion about the nature of the problem
- phk mentions old idea about split log (complete/incomplete tx)
- phk idea to suppress transactions from VCL
- phk idea to use log files only once, open a new one once full
- phk idea keep req grouping in varnishd (into dynamic memory), flush only
once per top level request.
- dridi presents vsl scrubbing idea
- replace log entries or fields with Xes
- discussion about oob info from length - should redaction be a single
fixed token or original length?
tasks
- VIP for scrubbing (dridi)
- VIP batching on toplevel req in varnishd (PHK)
vdd2022q2 #3 Varnish Association
================================
- reiterated over status quo
vdd2022q2 #4 TLS
================
- no progress
- discussion again about the future life of vtest: binary dependency vs. source
dependency vs. submodule
- VS has green light to contribute TLS support
- slink has talked to CF guys at H3 workshop, no known show stoppers
- h2o also has keyless "neverbleed" https://github.com/h2o/neverbleed MIT licensed
- should be one or several VEXTs (openssl, libressl, ...)
- TODO:
- vtest
- VIP the backend interface
- backend definition like vmod function for vext
backend newstuff {
.options = {
.foo = 123;
}
}
options get passed as string at init time
vdd2022q2 #5 H3
===============
- asad: still working on it with msquic
- also researching alternatives
- haproxy also needs patched openssl
Tenants and VCL files
=====================
- hermunn/dridi: want to add a parameter to vcl.load as the base path for
includes with fall back on global vcl_path, ban absolute includes from vcl
vdd2022q2 #6 Extensions
=======================
- do we want extension arguments?
- env vars working find for now
BANs and persistent storages
============================
- loading old storage requires a control system
- control system can assign the "ban id" (current vtim_real)
- CLI command to return min(max(silo(ban_id))) (or per silo max(silo(ban_id))) ?
- feature flag to deny bans from vcl
- multi tenancy support
General idea to ponder: proper VCL "jails" for multi-tenancy
Direction of VCL?
=================
- local, task scoped variables/objects
- vcl functions
- bump vcl version every time? -> NO
- new include version check?
- "umbrella" vcl x.y requires x.Y with Y <= y
for (x in vmod.bla(...)) {
}
- foreach (key[, val[, idx]]) in vmod.something(arguments) {
key type is defined by vmod.something iterator
val type is defined by vmod.something iterator
idx is integer
}
- foreach x in vmod.something(arguments) {
x type is defined by vmod.something iterator
can have x.key x.index, x.whatever
}
# built-in structured fields knowledge?
foreach bla in req.http { // bla has type "http_header"
what members does `bla` have ?
// "literal, string, iterable".fields possible too
if (bla.name == "something") {
for fld in bla.val.fields { // comma, fld has type sf-element (?)
for x in fld.attr { // semi-colon, x has type sf-attr (?)
x.key x.val
}
}
}
}
jobj = vmod_json(input)
for x in jobj.somthing_i_really_want(dont_fail_hard=True) {
}
if (jobj.failed) {
}
foreach bla in req.http {
if (bla.name ~ "(?i)cache-control") {
foreach directive in std.split(bla.val, ", ") {
...
}
}
}
=> Everyone should re-write their complex vcl how they would
want it to look
Fastly-Syntax for SF
resp.http.Cache-Control["max-age"] = 2h;
function foo(myvmod.mytype x) {
}
What about:
for (x in myvmod.bla()) {
foo(x);
}
Also, what if the above is in foo()? Answer: Maybe we will allow recursive
functions, maybe not
# foo takes type returned by vmod?
compliance
==========
Dridi gives a very short talk about a minor change. He will share his 49 page
presentation later.
https://github.com/varnishcache/varnish-cache/files/10707808/vdd23q1-compliance-stuff.pdf
====
DAY2
====
VCL SYNTAX DISCUSSION AROUND FIELD ACCESS
=========================================
if (req.http.Cache-Control[private]) => true if private present
if (req.http.Cache-Control[private] is none) => true if private has no value
if (req.http.Cache-Control[private] == "public") => true if value of private
folds to
string "public"
if (req.http.Cache-Control has private)
if (req.http.Cache-Control[private] is flag) => true if private is present
if (req.http.Cache-Control[private] is not bool) => true if private= is present
if ("private=4"[private] is string) => false
if ("private=*true"[private] is not bool) => false
if ("private"[private]) => bool (true)
if (exists(req.http.Cache-Control[private]))
if (nonzero(req.http.Cache-Control[private]))
if (req.http.Cache-Control[private])
if (req.http.Cache-Control[private].exists)
if (req.http.Cache-Control[private].val)
if (req.http.Cache-Control[private].type)
"private" => True, None, Bool
"private=" => True, "", String
"private=X" => True, "X", String
"private=43" => True, 43, Number
"" => False, None, None
if (req.http.Cache-Control[private]) { //exist
if (req.http.Cache-Control[private].value) { //exist && has value
}
}
==> PHK to write up a strawman
Specifically on issue 3844:
* HEADER == STRING should be equivalent to STRING == HEADER (it is)
* implicit HEADER to BOOL conversion conveys existence (it does)
---
varnishtest cmp
varnish v1 -vcl {
backend be none;
sub vcl_recv {
return (synth(200));
}
sub vcl_synth {
set resp.http.nohdr-eq-empty = req.http.nohdr == "";
set resp.http.empty-eq-nohdr = "" == req.http.nohdr;
}
} -start
client c1 {
txreq
rxresp
expect resp.http.nohdr-eq-empty == true
expect resp.http.empty-eq-nohdr == true
} -start
---
BUGWASH & NEXT VDD
==================
Does it work as is?
- basically yes
- but we want to set up IRC & web bridge with simple basic auth
- should keep logs
@slink to ping @theis about VUG
VUG needs to be a varnish-cache (!) thing
find a sponsor for a venue
VS, UPLEX, fastly, whoever, are welcome as sponsors
10 days after release
idea: 26./27. september
VDD: combine with VUG
sub vcl_connect / vcl_accept / vcl_client_hello
================================================
sub vcl_init {
mytls = new newtls();
mytls.load("/etc/tls/.....pem");
mytls.load("/etc/tls/.....pem");
}
sub vcl_client_hello {
mytls.dynload(tls.sni, ttl=10m); # Dynamicly load cert?
if (tls.sni == "legacysite.asas") {
set tls.ciphers = "ssl, tlsv3";
tls.hint = mytls;
}
set tls.ciphers = "sslv1, tlsv3";
set tls.alpn = "h1,h2,h3=....";
tls.hint = fallback;
return(OK);
}
also: vcl_h[23]_settings, proxy2_tlv
----------------------
VEXT client_ssl has vmod tls
sub vcl_init {
new something = tls.init(...)
something.callback(myfunc)
}
func myfunc(tls.type: arg) {
arg.crytomumblemumble = "bla";
return(tls.OK)
}
-----------------------
sub vcl_accept { // Probably not needed.
if (...) {
filter.use();
}
}
modifying the response body (thi.js)
================================
sub vcl_backend_response {
set beresp.filter_list += "gunzip vmodfoo.spot_foo";
}
sub vcl_backend_body_is_here {
if (vmodfoo.did_you_see_foo) {
if (! beresp.body.try_filter({"regsuball(...) gzip"})) {
return (deliver); # keep the "source" object
}
beresp.body.filter("...");
}
// trailers goes here
}
h2/h3 on the backend, for backend not understanding h1.
=======================================================
we can have h2 in-core, but vext is maybe also viable option.
AGENDA DAY2
===========
h3
add vext_path -> probably just PR it
--
** * * UPLEX - Nils Goroll Systemoptimierung
Scheffelstraße 32
22301 Hamburg
tel +49 40 28805731
mob +49 170 2723133
fax +49 40 42949753
xmpp://slink@jabber.int.uplex.de/
http://uplex.de/
_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev