Mailing List Archive: What exactly do we expect from an RC release?

What exactly do we expect from an RC release?

May 1, 2021, 4:23 PM

Post #1 of 4 (256 views)

It's never been clear to me what sort of QA we hope to happen once an RC
release is made each spring. Nor is it evident what QA we actually do
-- and whether that differs from that we do for a monthly development
release.

Does an RC release get tested on more platforms than a monthly dev
release? Does an RC release get tested on more configurations than a
monthly dev release? Do we test its impact on CPAN more than we do a
monthly dev release?

When I first became a committer I assumed that the answer to these
question was "Yes" and that eventually people who had been at it longer
than I would disclose the specifics. But up until now I've never seen
anything done specifically for RC releases above and beyond what we do
for a monthly dev release.

Having done CPAN-river-testing from 2018 to 2020, I know that testing
against CPAN is non-trivial. And whatever code you've set up to handle
monthly dev releases needs some modification to handle the different
naming convention for RC releases. So I'm not expecting us to put
together some more intensive QA procedure on the fly. For the time
being, I'd simply like some discussion on the subject line: What
exactly do we expect from an RC release?

Thank you very much.
Jim Keenan

Re: What exactly do we expect from an RC release? [ In reply to ]

dom at earth

May 2, 2021, 2:30 AM

Post #2 of 4 (256 views)

Permalink

On Sat, May 01, 2021 at 07:23:30PM -0400, James E Keenan wrote:
> It's never been clear to me what sort of QA we hope to happen once an RC
> release is made each spring. Nor is it evident what QA we actually do --
> and whether that differs from that we do for a monthly development release.
>
> Does an RC release get tested on more platforms than a monthly dev release?
> Does an RC release get tested on more configurations than a monthly dev
> release? Do we test its impact on CPAN more than we do a monthly dev
> release?

It's only a single data point, but the RC releases are typically the
first time that Debian integrates the new changes for testing. We
upload RCs to the 'experimental' distribution and use that to perform
widespread rebuild testing against the Debian archive. This exercises
perl a good deal, and typically picks up problems with CPAN modules that
have already been found and reported upstream, picks up issues with third
party integrations (often those embedding perl), as well as Debian
specific issues, and less often picks up new issues that we report
upstream. Whilst Debian only uses a limited number of configurations (a
single nondebug+debug one, and a build of libperl), we test on what I
suspect is more (Linux) architectures than dev releases (eg the list at [1]).

It's also a chance for people to make use of that new release during
manual testing, but because of the deep integration between perl and the
rest of Debian, and the way we use the experimental distribution to
manage rebuilds (which involves a dependency tree 7 levels deep for the
XS/libperl builds) the release doesn't get much "daily use" type testing
at this stage.

Cheers
Dominic

[1] <https://buildd.debian.org/status/package.php?p=perl&suite=sid>

Re: What exactly do we expect from an RC release? [ In reply to ]

perl5-porters at perl

May 2, 2021, 10:04 AM

Post #3 of 4 (256 views)

Permalink

??????? Original Message ???????
On Sunday, May 2, 2021 4:30 AM, Dominic Hargreaves <dom@earth.li> wrote:

> On Sat, May 01, 2021 at 07:23:30PM -0400, James E Keenan wrote:
>
> > It's never been clear to me what sort of QA we hope to happen once an RC
> > release is made each spring. Nor is it evident what QA we actually do --
> > and whether that differs from that we do for a monthly development release.
> > Does an RC release get tested on more platforms than a monthly dev release?
> > Does an RC release get tested on more configurations than a monthly dev
> > release? Do we test its impact on CPAN more than we do a monthly dev
> > release?
>
> It's only a single data point, but the RC releases are typically the
> first time that Debian integrates the new changes for testing. We
> upload RCs to the 'experimental' distribution and use that to perform
> widespread rebuild testing against the Debian archive. This exercises
> perl a good deal, and typically picks up problems with CPAN modules that
> have already been found and reported upstream, picks up issues with third
> party integrations (often those embedding perl), as well as Debian
> specific issues, and less often picks up new issues that we report
> upstream. Whilst Debian only uses a limited number of configurations (a
> single nondebug+debug one, and a build of libperl), we test on what I
> suspect is more (Linux) architectures than dev releases (eg the list at [1]).
>
> It's also a chance for people to make use of that new release during
> manual testing, but because of the deep integration between perl and the
> rest of Debian, and the way we use the experimental distribution to
> manage rebuilds (which involves a dependency tree 7 levels deep for the
> XS/libperl builds) the release doesn't get much "daily use" type testing
> at this stage.
>
> Cheers
> Dominic
>
> [1] https://buildd.debian.org/status/package.php?p=perl&suite=sid

Wow. Thank you, Dominic. This seems like extremely important information.

Does anyone know if similar insights exist for OpenBSD, FreeBSD ports (which has a perl-devel port currently @ 5.33.8.9*), and pkgsrc (quarterly releases, I believe)? Same information for other places perl is packaged would also be helpful, especially on Windows & Mac 'package' managers (e.g., Strawberry Perl, Chocolate, Homebrew, etc).

As I've offered before, I am happy to track this somewhere. Question is - where is the best place for it? I suspect it should be on GH somewhere. I am thinking, start with the distros that include perl by default (thank you!!) and work out to the ones that include them in their default package managers (again, thank you!).

(Jim - let me know if you'd find this information is valuable for you, and I'll contact you offline about how best to present/preserve it.)

Notes:

* https://www.freshports.org/lang/perl5-devel/

Cheers,
Brett

PS: first post with this email address, old one is now 'reveive-only'

--
oodler@cpan.org
?https://github.com/oodler577
#pdl #p5p #p7-dev #native @ irc.perl.org

Sent with ProtonMail Secure Email.

Re: What exactly do we expect from an RC release? [ In reply to ]

Stromeko at nexgo

May 2, 2021, 11:05 AM

Post #4 of 4 (256 views)

Permalink

James E Keenan writes:
> For the time being, I'd simply like some discussion on the subject
> line: What exactly do we expect from an RC release?

I can't help you with figuring out what you what you do or should expect
from it, but I usually use the RC releases to pull up my patch stack and
do a full compile and test cycle when and if I plan to relase that
version for Cygwin. I suspect most packagers for other distributions will
have a similar habit. If I get to that point early enough I'll report on
the mailing list whether I had any problems (I usually do not).

Achim.
--
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf microQ V2.22R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada