Mailing List Archive: "vtest" script for review/experimentation

"vtest" script for review/experimentation

Nov 14, 2016, 2:28 AM

Post #1 of 3 (760 views)

I'm trying to construct a light-weight cross-platform tinderbox
facility which doesn't require jvm or other esoterics.

First step is this 'vtest.sh' script:

http://phk.freebsd.dk/misc/vtest.sh

It basically runs in a loop, git pulls, runs autogen.des and
make distcheck after which a report is sent of to the varnish
project server.

I would appreciate if people would take a few minutes to
review and possibly test this script to see that it is
as portable as I hope.

Sending to the varnish project server is done with a ssh
key built by the script for the same purpose. Needless
to say I need a copy of that key before the scp will
work, but the script will tell you that...

The script should run fine without root permission, but
since we would also like coverage of the JAIL bits I hope
we can get some people to run it as root also, and therefore
it needs to be reviewed and scrutinized properly.

Obviously, running a "make distcheck" can never be "safe"
in any meaning of the word, in both theory and practice
we can put any kind of shit in our Makefiles, but at the
very minimum the vtest.sh script should not increase
exposure beyond that.

The backend side is very rudimentary still, but I have
a script which can output this:

1479111273 718531c +autogen +distcheck FreeBSD_11.0-CURRENT_amd64 phk
1477252091 718531c +autogen +distcheck FreeBSD_11.0-RELEASE-p1_arm phk
1479116010 1a989e2 +autogen +distcheck FreeBSD_11.0-CURRENT_amd64 phk
1477255339 1a989e2 +autogen -distcheck FreeBSD_11.0-RELEASE-p1_arm phk
c00049 ---- v1 11.1 Not true: n_object (4) == 3 (3)

More to follow on that side as I get some test-data to work with.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev

Re: "vtest" script for review/experimentation [ In reply to ]

dridi at varni

Nov 14, 2016, 3:30 AM

Post #2 of 3 (754 views)

Permalink

> I would appreciate if people would take a few minutes to
> review and possibly test this script to see that it is
> as portable as I hope.

As the official shell over-enthusiast I will review it.

First nitpick:

> nice make distclean > /dev/null 2>&1 || true
> nice /usr/bin/time sh autogen.des

The first distclean is redundant, it is done by autogen.des already,
unless you want to avoid timing the distclean too.

I would replace `/usr/bin/time` by `command time` to pick the binary
from the PATH, or `command time -v` if it's portable.

I would log and create the report in $TMPDIR.

In failedtests() you assume that all tests are VTCs but a distcheck
will include C tests as well, although they fail less often.

Further review later, that was a quick skim through the script.

> The script should run fine without root permission, but
> since we would also like coverage of the JAIL bits I hope
> we can get some people to run it as root also, and therefore
> it needs to be reviewed and scrutinized properly.

Bringing an "esoteric" here, what about capabilities to run as a
pseudo-root? Where your process thinks it's run by root, and has root
privileges in (let's call it) its container.

Can you pretend to be root in a FreeBSD jail and have actual
privileges limited to inside the jail?

Otherwise there's VMs...

Dridi

_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev

Re: "vtest" script for review/experimentation [ In reply to ]

phk at phk

Nov 14, 2016, 3:39 AM

Post #3 of 3 (754 views)

Permalink

--------
In message <CABoVN9CaM7RB3YVOpwmi7=6trZT=W-p4KTn9j9u0q_bnfyP4NA@mail.gmail.com>, Dridi Boukel
moune writes:

>As the official shell over-enthusiast I will review it.

Cool.

>First nitpick:
>
>> nice make distclean > /dev/null 2>&1 || true
>> nice /usr/bin/time sh autogen.des
>
>The first distclean is redundant, it is done by autogen.des already,
>unless you want to avoid timing the distclean too.

That was the point.

>I would replace `/usr/bin/time` by `command time` to pick the binary
>from the PATH, or `command time -v` if it's portable.

Right now I don't use the output of the time for anything, the idea
was to collect the timing info and include it in the report.

>I would log and create the report in $TMPDIR.

yeah, probably a good idea as well.

>In failedtests() you assume that all tests are VTCs but a distcheck
>will include C tests as well, although they fail less often.

I've been wondering if we should wrap the C tests in a VTC, for
sake of uniformity, but since I can not even remember when a C test
has failed last time, it has priority well down the list.

>Can you pretend to be root in a FreeBSD jail and have actual
>privileges limited to inside the jail?

That's basically what jails does...

My intent with this script is that it does not know or need to know
about its privileges (that's a 100% surefire way to punch holes
in your own code).

The idea is simply to run make distcheck and report the result.

What distcheck investigates, should not affect this script.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

_______________________________________________
varnish-dev mailing list
varnish-dev@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev