Mailing List Archive: Installing tests?

Installing tests?

Aug 18, 2023, 9:41 AM

Post #1 of 6 (103 views)

Hi all,

This has come up at conferences before, but I've not seen it discussed here
(or at least, I can't find it in the archives. Apologies if this has been
done to death).

When I install a module, it often has tons of dependencies which
potentially impact modules I've already installed. It's particularly bad if
it updates dual-life modules and something breaks.

It would be nice if I could do "cpan Some::Module" and also install the
tests. Later, I install some other module and I run the tests for all of
the modules I've already installed.

I've been doing testing long enough that I know that some tests will not be
easily installable (or might not be installable at all), but having a good
solution is better than having no solution.

Is this at all feasible? Or is there some other group I should be asking?

Best,
Ovid
--
Curtis "Ovid" Poe
--
CTO, All Around the World
World-class software development and consulting
https://allaroundtheworld.fr/

Re: Installing tests? [ In reply to ]

leonerd at leonerd

Aug 18, 2023, 9:45 AM

Post #2 of 6 (103 views)

Permalink

On Fri, 18 Aug 2023 18:41:01 +0200
Ovid <curtis.poe@gmail.com> wrote:

> Hi all,
>
> This has come up at conferences before, but I've not seen it
> discussed here (or at least, I can't find it in the archives.
> Apologies if this has been done to death).
>
> When I install a module, it often has tons of dependencies which
> potentially impact modules I've already installed. It's particularly
> bad if it updates dual-life modules and something breaks.
>
> It would be nice if I could do "cpan Some::Module" and also install
> the tests. Later, I install some other module and I run the tests for
> all of the modules I've already installed.
>
> I've been doing testing long enough that I know that some tests will
> not be easily installable (or might not be installable at all), but
> having a good solution is better than having no solution.
>
> Is this at all feasible? Or is there some other group I should be
> asking?

I've often wanted a smoker that can do that kind of tracking.

It feels much more like a CI-shaped question, than a CPAN / core perl
question.

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/

Re: Installing tests? [ In reply to ]

dj.p5p at avoiding

Aug 18, 2023, 10:03 AM

Post #3 of 6 (103 views)

Permalink

On 8/18/23 12:41, Ovid wrote:
> Hi all,
>
> This has come up at conferences before, but I've not seen it discussed
> here (or at least, I can't find it in the archives. Apologies if this
> has been done to death).
>
> When I install a module, it often has tons of dependencies which
> potentially impact modules I've already installed. It's particularly
> bad if it updates dual-life modules and something breaks.
>
> It would be nice if I could do "cpan Some::Module" and also install
> the tests. Later, I install some other module and I run the tests for
> all of the modules I've already installed.
>
> I've been doing testing long enough that I know that some tests will
> not be easily installable (or might not be installable at all), but
> having a good solution is better than having no solution.
>
> Is this at all feasible? Or is there some other group I should be asking?
>
> Best,
> Ovid
>

This following is an (untested) hack, but might it work in your case?

   # 1. initial install in a local directory to not pollute site, but
to install all of the dependencies
   # 1.1 save all downloaded dists in order to reuse them in following
steps, in case CPAN has been updated after this step
   # 3. reinstall everything, using all of the saved dists, so that all
modules see the same environment
   # 4. final installation to site

   cpanm -l local --save-dists ./minicpan <modules> && \
   cpanm -l local --from ./minicpan –reinstall <modules> && \
   cpanm –from ./minicpan <modules

Re: Installing tests? [ In reply to ]

fawaka at gmail

Aug 18, 2023, 10:05 AM

Post #4 of 6 (103 views)

Permalink

On Fri, Aug 18, 2023 at 6:41?PM Ovid <curtis.poe@gmail.com> wrote:

> Hi all,
>
> This has come up at conferences before, but I've not seen it discussed
> here (or at least, I can't find it in the archives. Apologies if this has
> been done to death).
>
> When I install a module, it often has tons of dependencies which
> potentially impact modules I've already installed. It's particularly bad if
> it updates dual-life modules and something breaks.
>
> It would be nice if I could do "cpan Some::Module" and also install the
> tests. Later, I install some other module and I run the tests for all of
> the modules I've already installed.
>
> I've been doing testing long enough that I know that some tests will not
> be easily installable (or might not be installable at all), but having a
> good solution is better than having no solution.
>
> Is this at all feasible? Or is there some other group I should be asking?
>

It's more of a toolchain discussion than a p5p one TBH

But yeah we have thought about it before at some PTS/QAH (though not a
recent one). A naive implementation that installs the contents of t/* would
work most of the time but would fail terribly some other cases (e.g. that
use a corpus/ for test data).

I'm not sure this would work as a default-on feature, but it could easily
be made to work on an opt-in basis.

Leon

Re: Installing tests? [ In reply to ]

darren at darrenduncan

Aug 18, 2023, 11:22 AM

Post #5 of 6 (103 views)

Permalink

On 2023-08-18 9:41 a.m., Ovid wrote:
> Hi all,
>
> This has come up at conferences before, but I've not seen it discussed here (or
> at least, I can't find it in the archives. Apologies if this has been done to
> death).
>
> When I install a module, it often has tons of dependencies which potentially
> impact modules I've already installed. It's particularly bad if it updates
> dual-life modules and something breaks.
>
> It would be nice if I could do "cpan Some::Module" and also install the tests.
> Later, I install some other module and I run the tests for all of the modules
> I've already installed.
>
> I've been doing testing long enough that I know that some tests will not be
> easily installable (or might not be installable at all), but having a good
> solution is better than having no solution.
>
> Is this at all feasible? Or is there some other group I should be asking?

I agree that this would be a great idea, and implementation is the question.

I had been thinking about installable tests myself before, but it was in a
different scenario of the tests defining a standard that possibly multiple
implementations may conform to, or that I otherwise wanted to distribute
separately from the thing being tested.

What you are proposing does seem more like a toolchain type question, and would
seem to involve a lot of work.

This also seems like a general problem or need faced by many programming
languages, not just Perl.

-- Darren Duncan

Re: Installing tests? [ In reply to ]

grinnz at gmail

Aug 18, 2023, 4:32 PM

Post #6 of 6 (103 views)

Permalink

On Fri, Aug 18, 2023 at 1:06?PM Leon Timmermans <fawaka@gmail.com> wrote:

> On Fri, Aug 18, 2023 at 6:41?PM Ovid <curtis.poe@gmail.com> wrote:
>
>> Hi all,
>>
>> This has come up at conferences before, but I've not seen it discussed
>> here (or at least, I can't find it in the archives. Apologies if this has
>> been done to death).
>>
>> When I install a module, it often has tons of dependencies which
>> potentially impact modules I've already installed. It's particularly bad if
>> it updates dual-life modules and something breaks.
>>
>> It would be nice if I could do "cpan Some::Module" and also install the
>> tests. Later, I install some other module and I run the tests for all of
>> the modules I've already installed.
>>
>> I've been doing testing long enough that I know that some tests will not
>> be easily installable (or might not be installable at all), but having a
>> good solution is better than having no solution.
>>
>> Is this at all feasible? Or is there some other group I should be asking?
>>
>
> It's more of a toolchain discussion than a p5p one TBH
>
> But yeah we have thought about it before at some PTS/QAH (though not a
> recent one). A naive implementation that installs the contents of t/* would
> work most of the time but would fail terribly some other cases (e.g. that
> use a corpus/ for test data).
>
> I'm not sure this would work as a default-on feature, but it could easily
> be made to work on an opt-in basis.
>

Agreed. Like most things there's the easy way which would work for dists
with normal tests (just store t/ somewhere and require the dist to specify
metadata similar to x_static_install if it supports running those tests
after installation) and the hard way which would actually work for dists
that don't opt into it (store the entire original tarball* somewhere).

* because the supported way to run CPAN dist tests is perl Makefile.PL &&
make test, which requires a significant portion of the original files to be
present, and there is no guarantee even the files that end up installed are
the same ones in the original lib.

-Dan