Mailing List Archive

How to run a specific test from the testsuite?
Dear Perl folks :

Well for the sake of fun over the holiday season I dragged out an old
old Sun SPARCStation 20 and played around with it. Turns out that the
latest Perl 5.34.0 will compile just fine. Slowly. Six hours in fact.
However it does compile fine! Then I want to see the testsuite run.
While watching Perl 5.34.0 running the testsuite I was surprised
to see :

.
.
.
cpan/autodie/t/utime ............................................. #
Failed test 'We can utime a file just fine.'
# at t/utime.t line 16.
# Can't utime(undef, undef,
'/export/home/dclarke/local/build/perl-5.34.0_SunOS5.8_sun4m.002/cpan/autodie/t/touch_me'):
Permission denied at t/utime.t line 15
# Failed test 'utime fails correctly on a 'true' failure.'
# at t/utime.t line 20.
# got: '0'
# expected: '1'
Can't utime('1641079886', '1592175685',
'/export/home/dclarke/local/build/perl-5.34.0_SunOS5.8_sun4m.002/cpan/autodie/t/touch_me'):
Not owner at t/utime.t line 24
# Looks like your test exited with 1 just after 4.
FAILED at test 2
.
.
.

I am curious if there is a magic incantation wherein I can see the
details of that test verbosely? I am very surprised to see that I am
not the owner of that file but perhaps this is because I am doing a
build "out of tree" wherein the actual sources are located in the
another path with whatever ownership was in the distribution tarball:

triton # ls -lad /export/home/dclarke/local/build/perl-5.34.0
drwxr-xr-x 25 perl perl 5120 May 20 2021
/export/home/dclarke/local/build/perl-5.34.0
triton #

Yes I did create a group and user named "perl" with uid:gid equal to
1000:1000 which I saw within the extracted tarball. I am guessing that
is the reason for the test failure. Seems strange given that the test is
not going after the source path but the correct build and test path?
Otherwise I see everything is going along swimmingly.

--
Dennis Clarke
RISC-V/SPARC/PPC/ARM/CISC
UNIX and Linux spoken
GreyBeard and suspenders optional
Re: How to run a specific test from the testsuite? [ In reply to ]
On 1/2/22 5:01 PM, Dennis Clarke via perl5-porters wrote:
>
> Dear Perl folks :
>
> Well for the sake of fun over the holiday season I dragged out an old
> old Sun SPARCStation 20 and played around with it. Turns out that the
> latest Perl 5.34.0 will compile just fine. Slowly. Six hours in fact.
> However it does compile fine! Then I want to see the testsuite run.
> While watching Perl 5.34.0 running the testsuite I was surprised
> to see :
>
> .
> .
> .
> cpan/autodie/t/utime ............................................. #
> Failed test 'We can utime a file just fine.'
> #   at t/utime.t line 16.
> # Can't utime(undef, undef,
> '/export/home/dclarke/local/build/perl-5.34.0_SunOS5.8_sun4m.002/cpan/autodie/t/touch_me'):
> Permission denied at t/utime.t line 15
> #   Failed test 'utime fails correctly on a 'true' failure.'
> #   at t/utime.t line 20.
> #          got: '0'
> #     expected: '1'
> Can't utime('1641079886', '1592175685',
> '/export/home/dclarke/local/build/perl-5.34.0_SunOS5.8_sun4m.002/cpan/autodie/t/touch_me'):
> Not owner at t/utime.t line 24
> # Looks like your test exited with 1 just after 4.
> FAILED at test 2
> .
> .
> .
>
> I am curious if there is a magic incantation wherein I can see the
> details of that test verbosely?  I am very surprised to see that I am
> not the owner of that file but perhaps this is because I am doing a
> build "out of tree" wherein the actual sources are located in the
> another path with whatever ownership was in the distribution tarball:
>
> triton # ls -lad /export/home/dclarke/local/build/perl-5.34.0
> drwxr-xr-x  25 perl     perl        5120 May 20  2021
> /export/home/dclarke/local/build/perl-5.34.0
> triton #
>
> Yes I did create a group and user named "perl" with uid:gid equal to
> 1000:1000 which I saw within the extracted tarball. I am guessing that
> is the reason for the test failure. Seems strange given that the test is
> not going after the source path but the correct build and test path?
> Otherwise I see everything is going along swimmingly.
>

Perhaps you're creating problems for yourself? When I download and
unpack a tarball (I'm on FreeBSD), I get:

#####
$ wget https://www.cpan.org/src/5.0/perl-5.34.0.tar.gz
$ tar xzvf perl-5.34.0.tar.gz
$ cd perl-5.34.0
$ ls -l cpan/autodie/t/utime.t
-r--r--r-- 1 jkeenan jkeenan 840 Jun 14 2020 cpan/autodie/t/utime.t
#####

... and at this point I've never had to create a new 'user:group' or had
any problem saying:

#####
$ sh ./Configure -des -Dusedevel
$ make test_prep
$ make test_harness
#####

Once you've gotten as far as 'make test_prep' above, you should be able
to run an individual test program (with verbose output) via:

#####
$ cd t; ./perl harness -v ../cpan/autodie/t/utime.t; cd -
#####
Re: How to run a specific test from the testsuite? [ In reply to ]
On 1/2/22 17:35, James E Keenan wrote:
> On 1/2/22 5:01 PM, Dennis Clarke via perl5-porters wrote:
>>
>> Dear Perl folks :
>>
>> Well for the sake of fun over the holiday season I dragged out an old
.
.
.
>
> Perhaps you're creating problems for yourself?  When I download and
> unpack a tarball (I'm on FreeBSD), I get:
>

You extracted the tarball as yourself. Therefore you did not protect the
sources from changes. I created the uid:gid "perl" user/group just for
the fun of having a name assigned to the value 1000. No other purpose.


--
Dennis Clarke
RISC-V/SPARC/PPC/ARM/CISC
UNIX and Linux spoken
GreyBeard and suspenders optional
Re: How to run a specific test from the testsuite? [ In reply to ]
On Mon, Jan 3, 2022 at 9:01 AM Dennis Clarke via perl5-porters <
perl5-porters@perl.org> wrote:

>
> I am curious if there is a magic incantation wherein I can see the
> details of that test verbosely?
>

To run that test script alone you can, from the top level source directory,
run:
../perl -I./lib cpan/autodie/t/utime.t

If the verbosity you seek is not supplied, then you can insert appropriate
code into utime.t to provide it.

In general, you can run individual test scripts that way, but sometimes
they have to be run from a location other than the top level perl source
directory.
For example, it would not be unusual if you had to run that script from one
level up from the "t" directory as:
../../perl -I../../lib t/utime.t
(For this particular test file I found it unnecessary to move out of the
top level.)

Cheers,
Rob
Re: How to run a specific test from the testsuite? [ In reply to ]
On 1/2/22 23:06, sisyphus wrote:
> On Mon, Jan 3, 2022 at 9:01 AM Dennis Clarke via perl5-porters <
> perl5-porters@perl.org> wrote:
>
>>
>> I am curious if there is a magic incantation wherein I can see the
>> details of that test verbosely?
>>
>
> To run that test script alone you can, from the top level source
directory,
> run:
> ../perl -I./lib cpan/autodie/t/utime.t
>
> If the verbosity you seek is not supplied, then you can insert
appropriate
> code into utime.t to provide it.
>
> In general, you can run individual test scripts that way, but sometimes
> they have to be run from a location other than the top level perl source
> directory.
> For example, it would not be unusual if you had to run that script
from one
> level up from the "t" directory as:
> ../../perl -I../../lib t/utime.t
> (For this particular test file I found it unnecessary to move out of the
> top level.)
>
> Cheers,
> Rob

Excellent! Thank you very much Sir. What I should have done was wait
the full day for the tests to complete. The final report on this old
wonderful machine looks like this :

.
.
.
Failed 11 tests out of 2432, 99.55% okay.
../cpan/autodie/t/chmod.t
../cpan/autodie/t/chown.t
../cpan/autodie/t/utime.t
../ext/POSIX/t/iv_const.t
../ext/POSIX/t/sigaction.t
re/fold_grind_8.t
re/fold_grind_aa.t
re/fold_grind_l.t
re/fold_grind_u.t
re/pat_advanced.t
re/pat_psycho.t
### Since not all tests were successful, you may want to run some of
### them individually and examine any diagnostic messages they produce.
### See the INSTALL document's section on "make test".
### You have a good chance to get more information by running
### ./perl harness
### in the 't' directory since most (>=80%) of the tests succeeded.
### You may have to set your dynamic library search path,
### LD_LIBRARY_PATH, to point to the build directory:
### setenv LD_LIBRARY_PATH `pwd`:$LD_LIBRARY_PATH; cd t; ./perl harness
### LD_LIBRARY_PATH=`pwd`:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; cd
t; ./perl harness
### export LD_LIBRARY_PATH=`pwd`:$LD_LIBRARY_PATH; cd t; ./perl harness
### for csh-style shells, like tcsh; or for traditional/modern
### Bourne-style shells, like bash, ksh, and zsh, respectively.
Elapsed: 73239 sec
u=837.88 s=190.76 cu=67562.42 cs=4886.09 scripts=2432 tests=1182828
gmake: *** [makefile:802: test] Error 1

real 74377.39
user 69364.75
sys 5240.62
triton$

Not quite a full day but given that the compile took over six hours I
can just round up to the nearest day :)

So at first glance I see some tests that are in fact actually trying to
modify files in the source directory. That is a strict no no given that
I am building "out of tree" and the sources are protected and owned by
the "perl" user and group uid:gid of 1000. Thus :

1) test cpan/autodie/t/chmod.t

triton$ ls -lap cpan/autodie/t/chmod.t
lrwxrwxrwx 1 dclarke devl 67 Jan 2 01:03
cpan/autodie/t/chmod.t ->
/export/home/dclarke/local/build/perl-5.34.0/cpan/autodie/t/chmod.t*
triton$

That is clearly a link to a file in the actual source tree.

triton$ cat ../perl-5.34.0/cpan/autodie/t/chmod.t
#!/usr/bin/perl -w
use strict;
use Test::More tests => 7;
use constant NO_SUCH_FILE => "this_file_had_better_not_exist";
use constant ERROR_REGEXP => qr{Can't chmod\(0755, '${\(NO_SUCH_FILE)}'\):};
use constant SINGLE_DIGIT_ERROR_REGEXP => qr{Can't chmod\(0010,
'${\(NO_SUCH_FILE)}'\):};
use autodie;

# This tests RT #50423, Debian #550462

eval { chmod(0755, NO_SUCH_FILE); };
isa_ok($@, 'autodie::exception', 'exception thrown for chmod');
like($@, ERROR_REGEXP, "Message should include numeric mode in octal form");

eval { chmod(8, NO_SUCH_FILE); };
isa_ok($@, 'autodie::exception', 'exception thrown for chmod');
like($@, SINGLE_DIGIT_ERROR_REGEXP, "Message should include numeric mode
in octal form");

eval { chmod(0755, $0); };
ok(! $@, "We can chmod ourselves just fine.");

eval { chmod(0755, $0, NO_SUCH_FILE) };
isa_ok($@, 'autodie::exception', 'chmod exception on any file failure.');
is($@->return,1,"Confirm autodie on a 'true' chown failure.");


Here we can just create a local directory inside the build area and
then run that test to see what happens :


triton$ mkdir foobar
triton$ cp -p ../perl-5.34.0/cpan/autodie/t/chmod.t foobar
triton$ ls -lap foobar
total 16
drwxr-xr-x 2 dclarke devl 512 Jan 3 15:58 ./
drwxr-xr-x 27 dclarke devl 6144 Jan 3 15:58 ../
-r-xr-xr-x 1 dclarke devl 962 Jun 14 2020 chmod.t
triton$

triton$ cd foobar
triton$
triton$ pwd
/export/home/dclarke/local/build/perl-5.34.0_SunOS5.8_sun4m.002/foobar
triton$

triton$
LD_LIBRARY_PATH=/export/home/dclarke/local/build/perl-5.34.0_SunOS5.8_sun4m.002
\
> ../perl -I../lib ./chmod.t
1..7
ok 1 - 'exception thrown for chmod' isa 'autodie::exception'
ok 2 - Message should include numeric mode in octal form
ok 3 - 'exception thrown for chmod' isa 'autodie::exception'
ok 4 - Message should include numeric mode in octal form
ok 5 - We can chmod ourselves just fine.
ok 6 - 'chmod exception on any file failure.' isa 'autodie::exception'
ok 7 - Confirm autodie on a 'true' chown failure.
triton$

So that works just fine.

I suspect all the tests that need to modify themselves in any way at all
will also "just work"(tm).

That leaves the ones that timeout due to long thrashing on an old old
machine.

Is there a way to tell the test harness to just keep on chugging and
wait for a result?

--
Dennis Clarke
RISC-V/SPARC/PPC/ARM/CISC
UNIX and Linux spoken
GreyBeard and suspenders optional



PS: Here is the report from perl -V :

triton$
triton$ LD_LIBRARY_PATH=`pwd` ./perl -I./lib -V
Summary of my perl5 (revision 5 version 34 subversion 0) configuration:

Platform:
osname=solaris
osvers=2.8
archname=sun4-solaris-64int
uname='sunos triton 5.8 generic_117350-62 sun4m sparc
sunw,sparcstation-20 '
config_args='-Dprefix=/export/home/dclarke/local -Dmksymlinks
-Accflags=-m32 -g -O0 -mno-app-regs -mhard-float -mno-faster-structs
-mstd-struct-return -mcpu=v8 -mno-v8plus -mno-vis
-L/export/home/dclarke/local/lib -I/export/home/dclarke/local/include'
hint=recommended
useposix=true
d_sigaction=define
useithreads=undef
usemultiplicity=undef
use64bitint=define
use64bitall=undef
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
Compiler:
cc='/opt/csw/gcc4/bin/gcc'
ccflags ='-m32 -g -O0 -mno-app-regs -mhard-float
-mno-faster-structs -mstd-struct-return -mcpu=v8 -mno-v8plus -mno-vis
-L/export/home/dclarke/local/lib -I/export/home/dclarke/local/include
-fwrapv -fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
optimize='-O0 -g'
cppflags='-m32 -g -O0 -mno-app-regs -mhard-float
-mno-faster-structs -mstd-struct-return -mcpu=v8 -mno-v8plus -mno-vis
-L/export/home/dclarke/local/lib -I/export/home/dclarke/local/include
-fwrapv -fno-strict-aliasing -pipe'
ccversion=''
gccversion='4.5.1'
gccosandvers=''
intsize=4
longsize=4
ptrsize=4
doublesize=8
byteorder=87654321
doublekind=4
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=16
longdblkind=2
ivtype='long long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='off_t'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries:
ld='/opt/csw/gcc4/bin/gcc'
ldflags =' '
libpth=/export/home/dclarke/local/lib /opt/bw/lib /opt/csw/lib
/opt/csw/gcc4/lib /usr/lib /usr/ccs/lib
libs=-lpthread -lsocket -lnsl -ldl -lm -lc
perllibs=-lpthread -lsocket -lnsl -ldl -lm -lc
libc=/lib/libc.so
so=so
useshrplib=true
libperl=libperl.so
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_dlopen.xs
dlext=so
d_dlsymun=undef
ccdlflags=' -R
/export/home/dclarke/local/lib/perl5/5.34.0/sun4-solaris-64int/CORE'
cccdlflags='-fPIC -O0 -g'
lddlflags='-G'


Characteristics of this binary (from libperl):
Compile-time options:
HAS_TIMES
PERLIO_LAYERS
PERL_COPY_ON_WRITE
PERL_DONT_CREATE_GVSV
PERL_MALLOC_WRAP
PERL_OP_PARENT
PERL_PRESERVE_IVUV
USE_64_BIT_INT
USE_LARGE_FILES
USE_LOCALE
USE_LOCALE_COLLATE
USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC
USE_LOCALE_TIME
USE_PERLIO
USE_PERL_ATOF
Built under solaris
Compiled at Jan 2 2022 04:33:54
@INC:
./lib

/export/home/dclarke/local/lib/perl5/site_perl/5.34.0/sun4-solaris-64int
/export/home/dclarke/local/lib/perl5/site_perl/5.34.0
/export/home/dclarke/local/lib/perl5/5.34.0/sun4-solaris-64int
/export/home/dclarke/local/lib/perl5/5.34.0
triton$ echo $?
0
triton$
Re: How to run a specific test from the testsuite? [ In reply to ]
On Mon, 03 Jan 2022 11:38:55 -0500, Dennis Clarke via perl5-porters wrote:

> That leaves the ones that timeout due to long thrashing on an old old
> machine.
>
> Is there a way to tell the test harness to just keep on chugging and
> wait for a result?

There's an environment variable which I've set on one of my Raspberry
Pis, where some of the tests would time out:
PERL_TEST_TIME_OUT_FACTOR=3


Cheers,
gregor

--
.''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org
: :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06
`. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe
`- NP: Tom Waits: Everything You Can Think
Re: How to run a specific test from the testsuite? [ In reply to ]
On Sun, 2 Jan 2022 at 23:01, Dennis Clarke via perl5-porters <
perl5-porters@perl.org> wrote:

>
> Dear Perl folks :
>
> Well for the sake of fun over the holiday season I dragged out an old
> old Sun SPARCStation 20 and played around with it. Turns out that the
> latest Perl 5.34.0 will compile just fine. Slowly. Six hours in fact.
> However it does compile fine! Then I want to see the testsuite run.
> While watching Perl 5.34.0 running the testsuite I was surprised
> to see :
>
> .
> .
> .
> cpan/autodie/t/utime ............................................. #
> Failed test 'We can utime a file just fine.'
> # at t/utime.t line 16.
> # Can't utime(undef, undef,
> '/export/home/dclarke/local/build/perl-5.34.0_SunOS5.8_sun4m.002/cpan/autodie/t/touch_me'):
>
> Permission denied at t/utime.t line 15
> # Failed test 'utime fails correctly on a 'true' failure.'
> # at t/utime.t line 20.
> # got: '0'
> # expected: '1'
> Can't utime('1641079886', '1592175685',
> '/export/home/dclarke/local/build/perl-5.34.0_SunOS5.8_sun4m.002/cpan/autodie/t/touch_me'):
>
> Not owner at t/utime.t line 24
> # Looks like your test exited with 1 just after 4.
> FAILED at test 2
> .
> .
> .
>
> I am curious if there is a magic incantation wherein I can see the
> details of that test verbosely?


IIRC:

TEST_ARGS="-v -re utime" make test_harness

Check the Makefile for `make test_reonly` and see what it does, i believe
it boils down to the above.

Yves