Here are the (delayed) minutes from 9 days ago. Again, I prioritised my
finite time hustling RFCs and other things over getting minutes done.
Neil, Nick and Rick present.
## Help wanted testing try/catch
We briefly discussed how we can make it easier for more people to "help with
the heavy lifting" as a general model. A good example was the new try/catch
feature. A lot of people seem keen to have this, but if everyone assumes
that someone else is going to test it, then bugs will lurk, and we'll never
get it "out of experimental". Paul wrote more at
https://www.nntp.perl.org/group/perl.perl5.porters/2021/06/msg260462.html
The bottom line is "your language needs you!" - please "try" this out in
real code so that we can "catch" the bugs.
(The bad puns will continue until the testing improves...)
## Actions from last time
Most had been done. Rik has now posted the gory details of ternaries and the
AASIGN/SASSIGN ops, and thoughts on pragmata, features, and version bundles.
Nick *still* hasn't written up a list of obscure corner cases to test with.
## RFCs
We considered the outstanding issues for n-at-a-time for, resolved them, and
agreed that the RFC moves to the "accepted" state. This means that "we think
this plan looks viable" and it's ready to implement. It is now implemented,
and awaiting review as https://github.com/Perl/perl5/pull/18925
Specifically, we asked "does this need a feature gate?" and "does this need
an experimental warning?" Existing precedents on this:
* ~~ is not feature gated, but is warning as experimental pair slicing was
* neither gated nor warned - Rik says: I was sure it was the correct design,
and the only question was bugs; experimental warnings and feature gates
aren't about bugs.
* It seems that for these we're wanting warnings as "might be buggy", but
heck, *anything* we change might be buggy, so we're not unanimously keen
on singling out the *implementation* of simple new features as special
here. warnings discourage people from using it and shaking out bugs -
long term we need the bugs gone
* For example, try/catch lurked for a while, and it seems that no-one really
tested it
* Being able to retroactively disable experimental warnings might help here
[*] - if we can deliver on this, then we'd have fewer reservations about
initially shipping with warnings
We considered warning on -w -np and agreed that the RFC was "rejected"
* How much value is this going to add, and for whom?
* We are discouraging the use of -w (ie we recommend `use warnings;`), so
why are we gating this warning on -w? (Because, obviously, for CLI flag
warnings there's no choice)
* Does the change the warning would recommend remove some surprise? No, not
really.
* Instead it takes code that works and suggests more work.
We discussed some questions about process, but didn't come to firm
answers. Such as
* "If we get to 100 approved RFCs, could we actually say what the future of
perl is?" given that we can't know when/whether any individual RFC will
get implemented.
* Will we get overwhelmed if too many RFCs arrive for discussion at the same
time?
* Can we avoid worrying about formal limits on RFCs in each state (prior to
approved) - we hope that there will be natural rate limiting
We are strongly of the opinion that the first step is not a draft RFC, but a
much shorter "elevator pitch". We quickly settled on a "four paragraph"
e-mail:
1. Here's the problem that I want to solve
2. Here's the syntax that I'm proposing
3. Here are the benefits
4. Here are potential problems
If a paragraph is just one sentence, that's great.
What we want to avoid is people diligently using the template, but as a
result creating verbose "form letters" which are long enough that most
people skip reading them, and hence the intended rapid feedback doesn't
happen.
What we want is a rapid answer to "is there any appetite for discussing
this?" and if there isn't a "no" then you draft an RFC. (At the time, we
considered whether "even silence" is not-a-"no". Since then, Nick now has a
slightly more detailed suggestion for the current PSC to consider)
If we get rapid yes/more info/CPAN/no feedback within 48 hours, we think
that we won't need any formal documented to track the state of "pre-RFCs"
We intended to trial this with Rik submitting an "elevator pitch" for
everything slices.
## 5.34.1
We discussed the GDBM_File performance issue in
https://github.com/Perl/perl5/issues/18884
As it's caused by the external library, not perl code, it was a coincidence
that was discovered by 5.34.0, and hence it's not an issue for 5.34.1
However, since the meeting it seems that the fix for the bug in the current
gdbm library (version 1.20) requires a change in the GDBM_File XS code, so
potentially there is a 5.34.1 backport needed. See
https://github.com/Perl/perl5/issues/18915
We discussed how the .N (>0) releases work. We need to be careful with maint
releases - they are *somewhat like* dev releases, but far more dangerous.
The current process predates our migration to GitHub, and started with a web
frontend that wrote out an XML file to track "backport votes", and share the
work of deciding what was necessary and safe to backport. We wondered
whether we can make anything simpler by finding features of GitHub that
could help, but we failed to assign an "action item" to any one individual,
so, whoops, it has stalled.
## Indexing permissions on core modules
Neil noticed that the PAUSE permissions on Text::Tabs and Text::Wrap seemed
a bit strange, given that the modules are shipped as dual life in core, and
hence potentially might need critical fixes. Neil was going to contact the
author to find a resolution, and also check all the other upstream blead
code to see that the PAUSE permissions made sense.
## Quirks
Neil has been working on a document collating many Perl features that are
often viewed as misfeatures. The intent is to have a common baseline for
discussing what we think are oddities of the language, so that we start from
the same place when discussing what some "emergent behaviour" or similar is,
why it ended up how it is today, and whether we want to do anything about it
(or even can - maybe the costs of changing it are too high). This is nearly
ready to go public. (Hardware issues have knocked Neil mostly offline for
the past week, hence it hasn't been finished.)
## Minimum perl versions (and encouraging adoption of our new features)
We discussed how it's frustrating that each year we ship a new version of
Perl, but the take up of the improvements is limited because folks shipping
code to CPAN are reluctant to use them because they favour supporting the
long tail of older versions. At some level, this conflict of incentives
actually makes it arguably pointless to try to add features/fix the
deficiencies of the language, which isn't what we want.
The rough conclusion was that the trade offs that the toolchain authors
choose is sensible for the toolchain, but we don't think that other modules
should adopt the toolchain end conclusion *unthinkingly*.
## Supported Platforms
Neil had made good progress in collating feedback on "supported platforms",
and was close to having a coherent document to propose. The overview is that
we have 4 tiers, currently described as
1. Battle tested
2. Stable
3. Unmaintained
4. Unknown
To be in tier 1, a platform needs to provide reliable Continuous
Integration, so that we get test results promptly for changes made to
blead. GitHub doesn't offer CI for FreeBSD and OpenBSD, but we'd like to
make these tier 1, so we need to approach them to ask if they have resources
we can use.
We also noted that our list of current platforms is based on hints files and
similar. We have hints files for platforms that have been dead for over 10
years - these are in our "tier 4 list" and we propose to remove these.
However, we don't (yet) have a table of $^O values for platforms that we
have already removed. For example, BeOS is long gone, but there are still
some tests that check for $^O matching 'BeOS'. This could usefully go.
So we also need to collate the "known dead" values of $^O, so that anyone
working on the code base can easily confirm that special-case code is dead
and can be removed.
## Data-Dumper issue queue
Data::Dumper is "upstream blead". Unlike several other modules with the same
state, its code is not mirrored into a specific repository, and hence no
repository exists for a ticket queue. As a result, we intend to change the
distribution META.json to point bug reports to the main perl issue queue, as
that is where the code is. Hence the existing (default) rt.cpan.org queue is
no longer active. There are currently about 30 issues in there. Jim Keenan
has already reviewed the tickets, but is keen to avoid importing tickets
which could be solved instead:
https://github.com/Perl/perl5/pull/18832#issuecomment-860210283
He's asked for help - some bugs might need fixes in Dumper.xs
PSC is keen for this to happen - volunteer(s) needed, but also to avoid the
process stalling and hence never completing. So we're suggesting a timebox -
do as much as possible by 31st August 2021, and then import every ticket
that hasn't been fixed by then.
## postderef_qq
Hugo asked for some clarification on this. Rik was to follow up with a
specific answer to Hugo (and has done so). Nick was to follow up with
starting to document Perl's general design principles, which are being
mentioned piecemeal or in passing as we discuss various RFCs. Nick has not
done this yet.
## Security fuzzing and typeglob assignment
Nick noted that typeglob assignment bugs feature disproportionately highly
in the security bugs we get reported, compared with how often they are used
in production code. This seems to be because the short syntax for typeglobs
makes them likely to get hit by fuzzers. The bugs *are* legitimate, but we
(and the researchers reporting them) then have to spend time investigating
them, usually to find that they are due to the stack not reference counted,
and could only be exploited if you can already inject code (in which case,
it's already game over), so they are not in need of a security fix. Nick was
wondering about ways to systematically fix at least some of these bugs, but
hasn't had time to go further on this yet. (RFCs took priority)
## Footnote - "retroactively disable experimental warnings"
When discussing the stability of experimental features, Rik has often noted
that the first perl version that offered lexical subroutines had an
implementation with serious bugs, such that you shouldn't use it. But a
later release (v5.22.0) fixed these, and as nothing changed for two
releases, it's no longer considered as experimental, and doesn't warn. But
with hindsight we can say you *can* safely use lexical subs in v5.22.0 or
later, despite the warning, because we now know that it's good. (The warning
went away in v5.26.0.)
Hence at times we've wondered whether we could implement some way to
formalise this - split out some part of the implementation of the warnings
pragma, such that
your code to ask to disable warnings for proven successful experiments we
could upload updates about "success" to CPAN as a dual life module
That way, we'd get a two year "head start" on the widespread use of new
features. At the point when experiments end, you could count on at least
some distributions having already shipped a perl "new" enough to use them
safely.
We'd not minuted this previously because we didn't have a firmer plan, or
any need to reference what is a thought experiment. We do now.
finite time hustling RFCs and other things over getting minutes done.
Neil, Nick and Rick present.
## Help wanted testing try/catch
We briefly discussed how we can make it easier for more people to "help with
the heavy lifting" as a general model. A good example was the new try/catch
feature. A lot of people seem keen to have this, but if everyone assumes
that someone else is going to test it, then bugs will lurk, and we'll never
get it "out of experimental". Paul wrote more at
https://www.nntp.perl.org/group/perl.perl5.porters/2021/06/msg260462.html
The bottom line is "your language needs you!" - please "try" this out in
real code so that we can "catch" the bugs.
(The bad puns will continue until the testing improves...)
## Actions from last time
Most had been done. Rik has now posted the gory details of ternaries and the
AASIGN/SASSIGN ops, and thoughts on pragmata, features, and version bundles.
Nick *still* hasn't written up a list of obscure corner cases to test with.
## RFCs
We considered the outstanding issues for n-at-a-time for, resolved them, and
agreed that the RFC moves to the "accepted" state. This means that "we think
this plan looks viable" and it's ready to implement. It is now implemented,
and awaiting review as https://github.com/Perl/perl5/pull/18925
Specifically, we asked "does this need a feature gate?" and "does this need
an experimental warning?" Existing precedents on this:
* ~~ is not feature gated, but is warning as experimental pair slicing was
* neither gated nor warned - Rik says: I was sure it was the correct design,
and the only question was bugs; experimental warnings and feature gates
aren't about bugs.
* It seems that for these we're wanting warnings as "might be buggy", but
heck, *anything* we change might be buggy, so we're not unanimously keen
on singling out the *implementation* of simple new features as special
here. warnings discourage people from using it and shaking out bugs -
long term we need the bugs gone
* For example, try/catch lurked for a while, and it seems that no-one really
tested it
* Being able to retroactively disable experimental warnings might help here
[*] - if we can deliver on this, then we'd have fewer reservations about
initially shipping with warnings
We considered warning on -w -np and agreed that the RFC was "rejected"
* How much value is this going to add, and for whom?
* We are discouraging the use of -w (ie we recommend `use warnings;`), so
why are we gating this warning on -w? (Because, obviously, for CLI flag
warnings there's no choice)
* Does the change the warning would recommend remove some surprise? No, not
really.
* Instead it takes code that works and suggests more work.
We discussed some questions about process, but didn't come to firm
answers. Such as
* "If we get to 100 approved RFCs, could we actually say what the future of
perl is?" given that we can't know when/whether any individual RFC will
get implemented.
* Will we get overwhelmed if too many RFCs arrive for discussion at the same
time?
* Can we avoid worrying about formal limits on RFCs in each state (prior to
approved) - we hope that there will be natural rate limiting
We are strongly of the opinion that the first step is not a draft RFC, but a
much shorter "elevator pitch". We quickly settled on a "four paragraph"
e-mail:
1. Here's the problem that I want to solve
2. Here's the syntax that I'm proposing
3. Here are the benefits
4. Here are potential problems
If a paragraph is just one sentence, that's great.
What we want to avoid is people diligently using the template, but as a
result creating verbose "form letters" which are long enough that most
people skip reading them, and hence the intended rapid feedback doesn't
happen.
What we want is a rapid answer to "is there any appetite for discussing
this?" and if there isn't a "no" then you draft an RFC. (At the time, we
considered whether "even silence" is not-a-"no". Since then, Nick now has a
slightly more detailed suggestion for the current PSC to consider)
If we get rapid yes/more info/CPAN/no feedback within 48 hours, we think
that we won't need any formal documented to track the state of "pre-RFCs"
We intended to trial this with Rik submitting an "elevator pitch" for
everything slices.
## 5.34.1
We discussed the GDBM_File performance issue in
https://github.com/Perl/perl5/issues/18884
As it's caused by the external library, not perl code, it was a coincidence
that was discovered by 5.34.0, and hence it's not an issue for 5.34.1
However, since the meeting it seems that the fix for the bug in the current
gdbm library (version 1.20) requires a change in the GDBM_File XS code, so
potentially there is a 5.34.1 backport needed. See
https://github.com/Perl/perl5/issues/18915
We discussed how the .N (>0) releases work. We need to be careful with maint
releases - they are *somewhat like* dev releases, but far more dangerous.
The current process predates our migration to GitHub, and started with a web
frontend that wrote out an XML file to track "backport votes", and share the
work of deciding what was necessary and safe to backport. We wondered
whether we can make anything simpler by finding features of GitHub that
could help, but we failed to assign an "action item" to any one individual,
so, whoops, it has stalled.
## Indexing permissions on core modules
Neil noticed that the PAUSE permissions on Text::Tabs and Text::Wrap seemed
a bit strange, given that the modules are shipped as dual life in core, and
hence potentially might need critical fixes. Neil was going to contact the
author to find a resolution, and also check all the other upstream blead
code to see that the PAUSE permissions made sense.
## Quirks
Neil has been working on a document collating many Perl features that are
often viewed as misfeatures. The intent is to have a common baseline for
discussing what we think are oddities of the language, so that we start from
the same place when discussing what some "emergent behaviour" or similar is,
why it ended up how it is today, and whether we want to do anything about it
(or even can - maybe the costs of changing it are too high). This is nearly
ready to go public. (Hardware issues have knocked Neil mostly offline for
the past week, hence it hasn't been finished.)
## Minimum perl versions (and encouraging adoption of our new features)
We discussed how it's frustrating that each year we ship a new version of
Perl, but the take up of the improvements is limited because folks shipping
code to CPAN are reluctant to use them because they favour supporting the
long tail of older versions. At some level, this conflict of incentives
actually makes it arguably pointless to try to add features/fix the
deficiencies of the language, which isn't what we want.
The rough conclusion was that the trade offs that the toolchain authors
choose is sensible for the toolchain, but we don't think that other modules
should adopt the toolchain end conclusion *unthinkingly*.
## Supported Platforms
Neil had made good progress in collating feedback on "supported platforms",
and was close to having a coherent document to propose. The overview is that
we have 4 tiers, currently described as
1. Battle tested
2. Stable
3. Unmaintained
4. Unknown
To be in tier 1, a platform needs to provide reliable Continuous
Integration, so that we get test results promptly for changes made to
blead. GitHub doesn't offer CI for FreeBSD and OpenBSD, but we'd like to
make these tier 1, so we need to approach them to ask if they have resources
we can use.
We also noted that our list of current platforms is based on hints files and
similar. We have hints files for platforms that have been dead for over 10
years - these are in our "tier 4 list" and we propose to remove these.
However, we don't (yet) have a table of $^O values for platforms that we
have already removed. For example, BeOS is long gone, but there are still
some tests that check for $^O matching 'BeOS'. This could usefully go.
So we also need to collate the "known dead" values of $^O, so that anyone
working on the code base can easily confirm that special-case code is dead
and can be removed.
## Data-Dumper issue queue
Data::Dumper is "upstream blead". Unlike several other modules with the same
state, its code is not mirrored into a specific repository, and hence no
repository exists for a ticket queue. As a result, we intend to change the
distribution META.json to point bug reports to the main perl issue queue, as
that is where the code is. Hence the existing (default) rt.cpan.org queue is
no longer active. There are currently about 30 issues in there. Jim Keenan
has already reviewed the tickets, but is keen to avoid importing tickets
which could be solved instead:
https://github.com/Perl/perl5/pull/18832#issuecomment-860210283
He's asked for help - some bugs might need fixes in Dumper.xs
PSC is keen for this to happen - volunteer(s) needed, but also to avoid the
process stalling and hence never completing. So we're suggesting a timebox -
do as much as possible by 31st August 2021, and then import every ticket
that hasn't been fixed by then.
## postderef_qq
Hugo asked for some clarification on this. Rik was to follow up with a
specific answer to Hugo (and has done so). Nick was to follow up with
starting to document Perl's general design principles, which are being
mentioned piecemeal or in passing as we discuss various RFCs. Nick has not
done this yet.
## Security fuzzing and typeglob assignment
Nick noted that typeglob assignment bugs feature disproportionately highly
in the security bugs we get reported, compared with how often they are used
in production code. This seems to be because the short syntax for typeglobs
makes them likely to get hit by fuzzers. The bugs *are* legitimate, but we
(and the researchers reporting them) then have to spend time investigating
them, usually to find that they are due to the stack not reference counted,
and could only be exploited if you can already inject code (in which case,
it's already game over), so they are not in need of a security fix. Nick was
wondering about ways to systematically fix at least some of these bugs, but
hasn't had time to go further on this yet. (RFCs took priority)
## Footnote - "retroactively disable experimental warnings"
When discussing the stability of experimental features, Rik has often noted
that the first perl version that offered lexical subroutines had an
implementation with serious bugs, such that you shouldn't use it. But a
later release (v5.22.0) fixed these, and as nothing changed for two
releases, it's no longer considered as experimental, and doesn't warn. But
with hindsight we can say you *can* safely use lexical subs in v5.22.0 or
later, despite the warning, because we now know that it's good. (The warning
went away in v5.26.0.)
Hence at times we've wondered whether we could implement some way to
formalise this - split out some part of the implementation of the warnings
pragma, such that
your code to ask to disable warnings for proven successful experiments we
could upload updates about "success" to CPAN as a dual life module
That way, we'd get a two year "head start" on the widespread use of new
features. At the point when experiments end, you could count on at least
some distributions having already shipped a perl "new" enough to use them
safely.
We'd not minuted this previously because we didn't have a firmer plan, or
any need to reference what is a thought experiment. We do now.