Mailing List Archive

#1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked
#1800: PRIV_TASK in a vmod object method leads to code that cannot be
compiled/linked
-------------------+--------------------
Reporter: geoff | Type: defect
Status: new | Priority: normal
Milestone: Later | Component: vmod
Version: 4.1.0 | Severity: normal
Keywords: |
-------------------+--------------------
This diff in the vcc declaration of vmod_debug:

{{{
diff --git a/lib/libvmod_debug/vmod.vcc b/lib/libvmod_debug/vmod.vcc
index dad36ec..9dd9f5b 100644
--- a/lib/libvmod_debug/vmod.vcc
+++ b/lib/libvmod_debug/vmod.vcc
@@ -89,6 +89,8 @@ $Method TIME .date()

You never know when you need a date.

+$Method VOID .task(PRIV_TASK)
+
$Function VOID rot52(HTTP hdr)

Encrypt the HTTP header with quad-ROT13 encryption,
}}}

... causes about 25 vtc tests to fail, all of them with this error
message:

{{{
**** v1 0.2 CLI RX| Message from VCC-compiler:\n
**** v1 0.2 CLI RX| Could not load VMOD debug\n
**** v1 0.2 CLI RX| \tFile name: /home/geoff/Develop/varnish/varnish-
cache/lib/libvmod_debug/.libs/libvmod_debug.so\n
**** v1 0.2 CLI RX| \tdlerror:: /home/geoff/Develop/varnish/varnish-
cache/lib/libvmod_debug/.libs/libvmod_debug.so: undefined symbol:
vmod_obj_task\n
**** v1 0.2 CLI RX| ('input' Line 5 Pos 16)\n
**** v1 0.2 CLI RX| import debug from
"/home/geoff/Develop/varnish/varnish-
cache/lib/libvmod_debug/.libs/libvmod_debug.so";\n
**** v1 0.2 CLI RX|
---------------#####-------------------------------------------------------------------------------------------\n
**** v1 0.2 CLI RX| \n
**** v1 0.2 CLI RX| Running VCC-compiler failed, exited with 2\n
**** v1 0.2 CLI RX| VCL compilation failed
---- v1 0.2 FAIL VCL does not compile
}}}

In my own VMOD I'm getting a compile error for the C code generated for
the VMOD:

{{{
**** v1 0.3 CLI RX| Message from C-compiler:\n
**** v1 0.3 CLI RX| vgc.c: In function
\xe2\x80\x98VGC_function_vcl_backend_response\xe2\x80\x99:\n
**** v1 0.3 CLI RX| vgc.c:1178:31: error:
\xe2\x80\x98VGC_vmod_bar\xe2\x80\x99 undeclared (first use in this
function)\n
**** v1 0.3 CLI RX| VRT_priv_task(ctx, &VGC_vmod_bar),\n
**** v1 0.3 CLI RX| ^\n
**** v1 0.3 CLI RX| vgc.c:1178:31: note: each undeclared identifier is
reported only once for each function it appears in\n
**** v1 0.3 CLI RX| vgc.c: In function
\xe2\x80\x98VGC_function_vcl_recv\xe2\x80\x99:\n
**** v1 0.3 CLI RX| vgc.c:1429:31: error:
\xe2\x80\x98VGC_vmod_foobar\xe2\x80\x99 undeclared (first use in this
function)\n
**** v1 0.3 CLI RX| VRT_priv_task(ctx, &VGC_vmod_foobar),\n
**** v1 0.3 CLI RX| ^\n
**** v1 0.3 CLI RX| vgc.c:1439:31: error:
\xe2\x80\x98VGC_vmod_snafu\xe2\x80\x99 undeclared (first use in this
function)\n
**** v1 0.3 CLI RX| [2 lines truncated]\n
**** v1 0.3 CLI RX| Running C-compiler failed, exited with 1\n
**** v1 0.3 CLI RX| VCL compilation failed
---- v1 0.3 FAIL VCL does not compile
}}}

The vmodtool doesn't throw any errors, but the generated code is
defective. Evidently a symbol is generated for the private object that is
not declared.

The use case is a VMOD object that is intended to have state across method
calls within a "task" (client or backend context). In my case, it's for a
regex VMOD with a backref call meant to refer back to captured expressions
in a previous match call (https://code.uplex.de/uplex-varnish/libvmod-re).
The VCC interface is something like this:

{{{
$Object regex(STRING)
$Method BOOL .match(PRIV_TASK, STRING)
$Method STRING .backref(PRIV_TASK, INT)
}}}

With usage like this:

{{{
import re;

sub vcl_init {
# compile a regex
new myregex = re.regex("foo(bar)");
}

sub vcl_recv {
if (myregex.match(req.http.Foo)) {
set req.http.Bar = myregex.backref(1);
}
}}}

The immediate defect is that code is generated that either can't be
compiled or linked. More generally, it seems very intuitive to me that
object methods should be able to maintain a state with task scope -- the
regex example is basic stuff. Up to now, I'd been maintaining state by
using pthread-local keys, but I thought using PRIV_TASK would be the more
natural, Varnishy solution.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1800>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked [ In reply to ]
#1800: PRIV_TASK in a vmod object method leads to code that cannot be
compiled/linked
--------------------+--------------------
Reporter: geoff | Owner:
Type: defect | Status: new
Priority: normal | Milestone: Later
Component: vmod | Version: 4.1.0
Severity: normal | Resolution:
Keywords: |
--------------------+--------------------

Comment (by fgsch):

This works fine for me. Do you have the C counterpart in vmod_debug.c?

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1800#comment:1>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked [ In reply to ]
#1800: PRIV_TASK in a vmod object method leads to code that cannot be
compiled/linked
--------------------+--------------------
Reporter: geoff | Owner:
Type: defect | Status: new
Priority: normal | Milestone: Later
Component: vmod | Version: 4.1.0
Severity: normal | Resolution:
Keywords: |
--------------------+--------------------

Comment (by geoff):

Holy crap, what a stupid mistake. Nope, forgot about that part, I'm sorry.

OK, let me try again, to see if I can reproduce the problem with my own
VMOD.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1800#comment:2>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked [ In reply to ]
#1800: PRIV_TASK in a vmod object method leads to code that cannot be
compiled/linked
--------------------+--------------------
Reporter: geoff | Owner:
Type: defect | Status: new
Priority: normal | Milestone: Later
Component: vmod | Version: 4.1.0
Severity: normal | Resolution:
Keywords: |
--------------------+--------------------

Comment (by geoff):

OK, I've attached a patch to demonstrate the problem, which matches the
problem with my own VMOD.

The patch adds these two methods to the object "obj" in vmod debug:

{{{
$Method VOID .task_set(PRIV_TASK, INT)

$Method INT .task_get(PRIV_TASK, INT fallback=-1)
}}}

Set an integer in task scope, and retrieve it in the same scope.
m00000.vtc tests it:

{{{
sub vcl_init {
new foo = debug.obj("foo");
}

sub vcl_deliver {
# ...
foo.task_set(4711);
set resp.http.task = foo.task_get();
}
}}}

The generated code fails to compile with this message:

{{{
**** v1 0.2 CLI RX| Message from C-compiler:\n
**** v1 0.2 CLI RX| vgc.c: In function
\xe2\x80\x98VGC_function_vcl_deliver\xe2\x80\x99:\n
**** v1 0.2 CLI RX| vgc.c:1352:29: error:
\xe2\x80\x98VGC_vmod_foo\xe2\x80\x99 undeclared (first use in this
function)\n
**** v1 0.2 CLI RX| VRT_priv_task(ctx, &VGC_vmod_foo),\n
**** v1 0.2 CLI RX| ^\n
**** v1 0.2 CLI RX| vgc.c:1352:29: note: each undeclared identifier is
reported only once for each function it appears in\n
**** v1 0.2 CLI RX| Running C-compiler failed, exited with 1\n
**** v1 0.2 CLI RX| VCL compilation failed
---- v1 0.2 FAIL VCL does not compile
}}}

So please try to forget what I said about linking (duh), the problem is in
the generated C code. A symbol `VGC_vmod_<object name>` is used for the
priv object, but is not declared.

As I said above, the uncompilable code is the immediate problem, but it
also seems intuitive that object methods should be able to make use of
task scope -- in other words, rather than having the vmodtool generate an
error, it would be better to make PRIV_TASK possible in a method.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1800#comment:3>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked [ In reply to ]
#1800: PRIV_TASK in a vmod object method leads to code that cannot be
compiled/linked
--------------------+--------------------
Reporter: geoff | Owner:
Type: defect | Status: new
Priority: normal | Milestone: Later
Component: vmod | Version: 4.1.0
Severity: normal | Resolution:
Keywords: |
--------------------+--------------------

Comment (by daghf):

Hi

I had a look, and this comes down to overly simple parsing in
vcc_expr.c:vcc_priv_arg, where we've forgot to take VMOD objects into
account.

The parsing code blindly assumes the VMOD name to be the initial portion
of the function invocation string leading up to the first dot.

This works fine for regular function calls ("vmodname.foo(..)"), but
breaks down for VMOD objects, where the parsed result is the arbitrary
name of the object instance ("my_object.foo(..)").

We need to expand on this to do a lookup that tells us if the parsed name
is a VMOD object, and if so return the corresponding VMOD name.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1800#comment:4>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked [ In reply to ]
#1800: PRIV_TASK in a vmod object method leads to code that cannot be
compiled/linked
--------------------+--------------------
Reporter: geoff | Owner:
Type: defect | Status: new
Priority: normal | Milestone: Later
Component: vmod | Version: 4.1.0
Severity: normal | Resolution:
Keywords: |
--------------------+--------------------

Comment (by daghf):

Forgot to mention: This also applies for PRIV_VCL and PRIV_TOP.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1800#comment:5>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked [ In reply to ]
#1800: PRIV_TASK in a vmod object method leads to code that cannot be
compiled/linked
--------------------+---------------------
Reporter: geoff | Owner: aondio
Type: defect | Status: new
Priority: normal | Milestone: Later
Component: vmod | Version: 4.1.0
Severity: normal | Resolution:
Keywords: |
--------------------+---------------------
Changes (by aondio):

* owner: => aondio


--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1800#comment:6>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked [ In reply to ]
#1800: PRIV_TASK in a vmod object method leads to code that cannot be
compiled/linked
--------------------+---------------------
Reporter: geoff | Owner: aondio
Type: defect | Status: new
Priority: normal | Milestone: Later
Component: vmod | Version: 4.1.0
Severity: normal | Resolution:
Keywords: |
--------------------+---------------------

Comment (by geoff):

aondio, thanks, I can confirm that make check succeeds for Varnish with
the patch, and also that my use of PRIV_TASK in a vmod object method
succeeds.

But I think that a few things are still missing, as we talked about in
email.

Since the patch also addresses the use of PRIV_VCL and PRIV_TOP in
methods, shouldn't there be tests for those as well? (The only tests are
for PRIV_TASK.)

The patch doesn't provide for the use of PRIV_CALL in a method, which
makes sense because you don't need that for an object (just store your
private data in the object). But someone might declare PRIV_CALL for a
method in their vcc spec anyway. I think the right thing in that case
would be for the vmodtool to an emit error with a helpful hint. (Otherwise
what would happen? The same vgc compile-time error that I encountered
above?)

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1800#comment:7>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1800: PRIV_TASK in a vmod object method leads to code that cannot be compiled/linked [ In reply to ]
#1800: PRIV_TASK in a vmod object method leads to code that cannot be
compiled/linked
--------------------+---------------------
Reporter: geoff | Owner: aondio
Type: defect | Status: new
Priority: normal | Milestone: Later
Component: vmod | Version: 4.1.0
Severity: normal | Resolution:
Keywords: |
--------------------+---------------------

Comment (by geoff):

daghf mentioned in IRC today that PRIV_CALL in an object method works fine
without the patch. I gave it a try and confirm that it's all right.

So all that I think is missing are tests to verify PRIV_VCL and PRIV_TOP
in object methods.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1800#comment:8>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs