Mailing List Archive

Re: vtest status
--------
In message <58d74416-8e0f-4a35-bb7c-67f02a680fdc@haproxy.com>, Frederic Lecaille writes:

>I hope everything is going well on your side.

Well, getting better: One of my close friends died from stomach-cancer
right before X-mas and that threw all plans askew.

>So, if you need some help to make vtest project progress, please
>do not hesitate to ask.

I think what we need most right now, is to decide what shape vtest
should take?

I want to stress that I want this to be a collective decision,
so don't take anything I write here as edict or diktat.

Personally I am not very keen on turning vtest a "real" project
with releases, package-building and all that, because it would cause
me a lot of work which I don't think brings enough advantages.

The problem for me is that vtc_varnish has a very incestuos
relationship with varnish internals, internals which we do not want
to turn into documented or even open APIs, and that means that
vtc_varnish and varnishd must match versions.

A stand-alone vtest-package would either need to be compiled against
a specific version of varnish, or have some way to dynamically load
vtc_varnish from from the varnish source/package it is being used
against.

But that just moves the problem to the other side of the line, where
we need to open the internals of vtest up as a documented and
versioned API...

Some day (H3?) all that work may make sense, but I don't feel we
are there yet, at least not as far as cost/benefit is concerned.

My suggestion for now, is to let vtest live as a "source code
library" on github and not build and distribute it as stand-alone
packages.

Instead it will be up to the projects which use it to import
and build in their own projects.

That way HAproxy and leave vtc_varnish out of their compilation so
varnish does not become a build-dependency (or you can conditionally
compile vtc_varnish in, if it is already there.)

And we can compile it in Varnish including vtc_varnish, and include
vtc_haproxy in similar fashion. (actually, compilation of
vtc_haproxy does not need haproxy to be installed, does it ?)

We should still provide a Makefile in the vtest github project which
compiles as much as possible (ie: vtc_varnish if it can find varnish
installed), that will make work on the shared sources easier for
all of us.

If we decide to do things that way, maybe you should call your
compiled version something like "hatest", while we stick with
"varnishtest", so we can reserve the "vtest" name for the future
runtime-extensible all-singing-and-dancing thing ?


Poul-Henning

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: vtest status [ In reply to ]
On 1/9/19 11:36 AM, Poul-Henning Kamp wrote:
> --------
> In message <58d74416-8e0f-4a35-bb7c-67f02a680fdc@haproxy.com>, Frederic Lecaille writes:
>
>> I hope everything is going well on your side.
>
> Well, getting better: One of my close friends died from stomach-cancer
> right before X-mas and that threw all plans askew.
>
>> So, if you need some help to make vtest project progress, please
>> do not hesitate to ask.
>
> I think what we need most right now, is to decide what shape vtest
> should take?

Yes!

> I want to stress that I want this to be a collective decision,
> so don't take anything I write here as edict or diktat.
>
> Personally I am not very keen on turning vtest a "real" project
> with releases, package-building and all that, because it would cause
> me a lot of work which I don't think brings enough advantages.
>
> The problem for me is that vtc_varnish has a very incestuos
> relationship with varnish internals, internals which we do not want
> to turn into documented or even open APIs, and that means that
> vtc_varnish and varnishd must match versions.
>
> A stand-alone vtest-package would either need to be compiled against
> a specific version of varnish, or have some way to dynamically load
> vtc_varnish from from the varnish source/package it is being used
> against.
>
> But that just moves the problem to the other side of the line, where
> we need to open the internals of vtest up as a documented and
> versioned API...
>
> Some day (H3?) all that work may make sense, but I don't feel we
> are there yet, at least not as far as cost/benefit is concerned.
>
> My suggestion for now, is to let vtest live as a "source code
> library" on github and not build and distribute it as stand-alone
> packages.

Currently, vtest convert.sh script make copies of lib/libvarnish/*.c
which are already compiled when compiling varnish sources providing
lib/libvarnish.a library.

Could not this vtest library be compiled against lib/libvarnish.a
library?

> Instead it will be up to the projects which use it to import
> and build in their own projects.
>
> That way HAproxy and leave vtc_varnish out of their compilation so
> varnish does not become a build-dependency (or you can conditionally
> compile vtc_varnish in, if it is already there.)
>
> And we can compile it in Varnish including vtc_varnish, and include
> vtc_haproxy in similar fashion. (actually, compilation of
> vtc_haproxy does not need haproxy to be installed, does it ?)

Of course not, it does not.

> We should still provide a Makefile in the vtest github project which
> compiles as much as possible (ie: vtc_varnish if it can find varnish
> installed), that will make work on the shared sources easier for
> all of us.

Yes, I agree.

> If we decide to do things that way, maybe you should call your
> compiled version something like "hatest", while we stick with
> "varnishtest", so we can reserve the "vtest" name for the future
> runtime-extensible all-singing-and-dancing thing ?

All these information sound fair to me.

As far as we do not have to make copies of file from varnish sources
to compile vtest this would help to its maintenance.


Fred.
_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: vtest status [ In reply to ]
--------
In message <1ca72b15-d581-4556-713a-a95106aa2d09@haproxy.com>, Frederic Lecaille writes:

>> My suggestion for now, is to let vtest live as a "source code
>> library" on github and not build and distribute it as stand-alone
>> packages.
>
>Currently, vtest convert.sh script make copies of lib/libvarnish/*.c
>which are already compiled when compiling varnish sources providing
>lib/libvarnish.a library.

The convert.sh script is really only meant to show me what the
difference is between the copy of vtest in the varnish cache
project and in the vtest project.

>Could not this vtest library be compiled against lib/libvarnish.a
>library?

The problem is that if you install a varnish package you do not
have libvarnish.a, it is an internal static library.

>As far as we do not have to make copies of file from varnish sources
>to compile vtest this would help to its maintenance.

Precisely.

Watch out for a couple of commits...

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: vtest status [ In reply to ]
--------
In message <1ca72b15-d581-4556-713a-a95106aa2d09@haproxy.com>, Frederic Lecaille writes:

Try the stuff I just committed.

If you say "make vtest" you get no varnishsupport

If you say "make varnishtest" and point the variable at your
varnish source tree, you get varnish support.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: vtest status [ In reply to ]
Hi Poul-Henning!

On Wed, Jan 09, 2019 at 10:09:15PM +0000, Poul-Henning Kamp wrote:
> --------
> In message <1ca72b15-d581-4556-713a-a95106aa2d09@haproxy.com>, Frederic Lecaille writes:
>
> Try the stuff I just committed.
>
> If you say "make vtest" you get no varnishsupport
>
> If you say "make varnishtest" and point the variable at your
> varnish source tree, you get varnish support.

Indeed, it's much easier now :-) It's the first time I can build it.
I had to use this since there are probably still options that the user
has to tune :

make CC='gcc -DHAVE_SYS_VFS_H'

I think we can plan on adding a CFLAGS or whatever variable and enumerate
the most common defines that users may need. They're all supposed to be
sufficiently autonomous developers to be able to figure what they need
to pass to build it.

We also need to make this "python3" thing a variable in the makefile. On
my previous distro, it was called "python". Same, developers will just
have to force the variable if they want.

Everything looks good this way, I'll spread the word here to the team.
We may propose you some patches for the few points above, and after this
I think we can update our doc in haproxy to refer to the vtest repo and
suggest how to build it.

Thank you!
Willy
_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: vtest status [ In reply to ]
On 1/9/19 10:49 PM, Poul-Henning Kamp wrote:
> --------
> In message <1ca72b15-d581-4556-713a-a95106aa2d09@haproxy.com>, Frederic Lecaille writes:
>
>>> My suggestion for now, is to let vtest live as a "source code
>>> library" on github and not build and distribute it as stand-alone
>>> packages.
>>
>> Currently, vtest convert.sh script make copies of lib/libvarnish/*.c
>> which are already compiled when compiling varnish sources providing
>> lib/libvarnish.a library.
>
> The convert.sh script is really only meant to show me what the
> difference is between the copy of vtest in the varnish cache
> project and in the vtest project.

Ok.
>> Could not this vtest library be compiled against lib/libvarnish.a
>> library?

to be more accurate: lib/libvarnish/libvarnish.a

> The problem is that if you install a varnish package you do not
> have libvarnish.a, it is an internal static library.

Yes, I know that lib/libvarnishapi is exported.

I thought we would still have to compile the varnish sources before
compiling vtest, and give to this latter the path to
lib/libvarnish/libvarnish.a.

Anyway if we do not need the varnish sources anymore to compile
vtest this means varnish is perfectly modularized.

Great!
_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: vtest status [ In reply to ]
On 1/9/19 11:09 PM, Poul-Henning Kamp wrote:
> --------
> In message <1ca72b15-d581-4556-713a-a95106aa2d09@haproxy.com>, Frederic Lecaille writes:
>
> Try the stuff I just committed.
>
> If you say "make vtest" you get no varnishsupport
>
> If you say "make varnishtest" and point the variable at your
> varnish source tree, you get varnish support.
>

Everything is OK.
Thank you a lot Poul-Henning.


Fred
_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: vtest status [ In reply to ]
--------
In message <e4b67848-e764-0aa3-cea8-6a48c0845cef@haproxy.com>, Frederic Lecaille writes:
>On 1/9/19 11:09 PM, Poul-Henning Kamp wrote:
>> --------
>> In message <1ca72b15-d581-4556-713a-a95106aa2d09@haproxy.com>, Frederic Lecaille writes:
>>
>> Try the stuff I just committed.
>>
>> If you say "make vtest" you get no varnishsupport
>>
>> If you say "make varnishtest" and point the variable at your
>> varnish source tree, you get varnish support.
>
>Everything is OK.
>Thank you a lot Poul-Henning.

Cool.

I'll update the README with some context...

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: vtest status [ In reply to ]
On 1/10/19 6:49 AM, Willy TARREAU wrote:
> Hi Poul-Henning!
>
> On Wed, Jan 09, 2019 at 10:09:15PM +0000, Poul-Henning Kamp wrote:
>> --------
>> In message <1ca72b15-d581-4556-713a-a95106aa2d09@haproxy.com>, Frederic Lecaille writes:
>>
>> Try the stuff I just committed.
>>
>> If you say "make vtest" you get no varnishsupport
>>
>> If you say "make varnishtest" and point the variable at your
>> varnish source tree, you get varnish support.
>
> Indeed, it's much easier now :-) It's the first time I can build it.
> I had to use this since there are probably still options that the user
> has to tune :
>
> make CC='gcc -DHAVE_SYS_VFS_H'
>
> I think we can plan on adding a CFLAGS or whatever variable and enumerate
> the most common defines that users may need. They're all supposed to be
> sufficiently autonomous developers to be able to figure what they need
> to pass to build it.
>
> We also need to make this "python3" thing a variable in the makefile. On
> my previous distro, it was called "python". Same, developers will just
> have to force the variable if they want.

As the python scripts run when compiling vtest is python2 compatible,
we can replace "python3" by "python" which points to python2 or
python3, depending on the distro.
_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: vtest status [ In reply to ]
On 1/10/19 6:49 AM, Willy TARREAU wrote:
> Hi Poul-Henning!
>
> On Wed, Jan 09, 2019 at 10:09:15PM +0000, Poul-Henning Kamp wrote:
>> --------
>> In message <1ca72b15-d581-4556-713a-a95106aa2d09@haproxy.com>, Frederic Lecaille writes:
>>
>> Try the stuff I just committed.
>>
>> If you say "make vtest" you get no varnishsupport
>>
>> If you say "make varnishtest" and point the variable at your
>> varnish source tree, you get varnish support.
>
> Indeed, it's much easier now :-) It's the first time I can build it.
> I had to use this since there are probably still options that the user
> has to tune :
>
> make CC='gcc -DHAVE_SYS_VFS_H'

with
make CC='gcc -O2 -s -D_GNU_SOURCE -DHAVE_SYS_VFS_H'

we do not have anymore these warnings:


src/vtc_process.c:652:11: warning: implicit declaration of function
‘posix_openpt’ [-Wimplicit-function-declaration]
master = posix_openpt(O_RDWR|O_NOCTTY);
^~~~~~~~~~~~
In file included from src/vtc.h:41:0,
from src/vtc_process.c:49:
src/vtc_process.c:654:5: warning: implicit declaration of function
‘grantpt’ [-Wimplicit-function-declaration]
AZ(grantpt(master));
^
lib/vas.h:61:8: note: in definition of macro ‘assert’
if (!(e)) { \
^
src/vtc_process.c:654:2: note: in expansion of macro ‘AZ’
AZ(grantpt(master));
^~
src/vtc_process.c:655:5: warning: implicit declaration of function
‘unlockpt’ [-Wimplicit-function-declaration]
AZ(unlockpt(master));
^
lib/vas.h:61:8: note: in definition of macro ‘assert’
if (!(e)) { \
^
src/vtc_process.c:655:2: note: in expansion of macro ‘AZ’
AZ(unlockpt(master));
^~
src/vtc_process.c:656:14: warning: implicit declaration of function
‘ptsname’ [-Wimplicit-function-declaration]
slavename = ptsname(master);
^~~~~~~
src/vtc_process.c:656:12: warning: assignment makes pointer from integer
without a cast [-Wint-conversion]
slavename = ptsname(master);
^
lib/vfil.c: In function ‘VFIL_fsinfo’:
lib/vfil.c:198:16: error: storage size of ‘fsst’ isn’t known
struct statfs fsst;
_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: vtest status [ In reply to ]
--------
In message <4f28e66f-b12c-6192-7307-f88e6b2bc62e@haproxy.com>, Frederic Lecaille writes:

>As the python scripts run when compiling vtest is python2 compatible,
>we can replace "python3" by "python" which points to python2 or
>python3, depending on the distro.

I did it slightly different, because some systems have given up on the
"bare" python name.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: vtest status [ In reply to ]
--------
In message <f80786e4-75c5-e80a-7da8-3af23dcc0ada@haproxy.com>, Frederic Lecaille writes:

>> Indeed, it's much easier now :-) It's the first time I can build it.
>> I had to use this since there are probably still options that the user
>> has to tune :
>>
>> make CC='gcc -DHAVE_SYS_VFS_H'
>
>with
> make CC='gcc -O2 -s -D_GNU_SOURCE -DHAVE_SYS_VFS_H'
>
>we do not have anymore these warnings:

Better ?

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: vtest status [ In reply to ]
On 1/10/19 10:57 AM, Poul-Henning Kamp wrote:
> --------
> In message <f80786e4-75c5-e80a-7da8-3af23dcc0ada@haproxy.com>, Frederic Lecaille writes:
>
>>> Indeed, it's much easier now :-) It's the first time I can build it.
>>> I had to use this since there are probably still options that the user
>>> has to tune :
>>>
>>> make CC='gcc -DHAVE_SYS_VFS_H'
>>
>> with
>> make CC='gcc -O2 -s -D_GNU_SOURCE -DHAVE_SYS_VFS_H'
>>
>> we do not have anymore these warnings:
>
> Better ?
>

Yes, I have seen this commit:

https://github.com/vtest/VTest/commit/9433a1f8fb8687465dd611cb0a941f0d7865e707

But I am not sure that _GNU_SOURCE may be enabled only Linux systems.
To me it is Glibc specific.

For instance on my FreeBSD 11 system, the config.h of varnish cache
defines _GNU_SOURCE.

Nothing really important, vtest compiles like a charm.

Thanks.


_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
Re: vtest status [ In reply to ]
--------
In message <6190b3ce-77db-5e6e-7c33-2a571517c9d8@haproxy.com>, Frederic Lecaille writes:

>For instance on my FreeBSD 11 system, the config.h of varnish cache
>defines _GNU_SOURCE.

I truly hate and detest the autocrap tools[1], and that is just
one of the many, many, many reasons why.

Poul-Henning

[1] See: https://queue.acm.org/detail.cfm?id=2349257

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev