The purpose of P5P is "the development and maintenance of Perl" — easy, right? The "perlpolicy" doc expands on this in some areas, but in PSC discussions we still found ourselves revisiting this, and I tried various times to write up our thoughts.
Various points related to this have come up in discussions on P5P, so I think it might be helpful to see how much of this we agree on. Think of this as P5P's mission statement, or manifesto.
We think our priorities are, in decreasing order:
1. Deal with security issues promptly (Security).
2. Keep perl code running (Reliability).
3. Improve the language for developers who are maintaining existing code or writing new code (Effectiveness).
4. Provide reasons for people to return to Perl, or to start writing Perl (Growth).
One of the points of contention is the priority of Reliability vs Effectiveness: there's a lot of old code out there that no-one cares about (any more) – do we really think keeping that code running is more important than extending the language to support today's active developers? Let's look at the priorities in turn, and then revisit this question.
Security
If security bugs are reported or discovered, then they are given appropriate priority, and they will often outweigh other considerations. When addressing security issues, we hope that the fix won't impact the other considerations. Sometimes that's not possible though, and security wins out. For example, "no . in @INC".
This is not only "fix security bugs", but also "try and prevent security bugs in Perl".
Reliability
Once someone has written Perl code and it has been "put into production", we should do our best to ensure that it continues to work when they upgrade to a new version of Perl. Perl has a reputation for backward compatibility, earned over decades. That reputation is valuable, and we should only spend that capital in return for something sufficiently valuable to warrant it.
- Aim for backwards compatibility when introducing new features.
- Fix bugs, while remembering that sometimes bugs become relied upon, and fixing those ones can break code.
- Be mindful of the breakage of CPAN code, which is our best proxy for how proprietary code may break.
This doesn't mean that we must be 100% backwards compatible and never break any CPAN modules, but that we should only deviate from that when we've carefully considered the impact. If a change would break three twenty-year old modules on CPAN that no-one seems to use, that's a different proposition from breaking a module that's far up the CPAN River.
Furthermore, if we do decide that it's ok to break something, we should try to reduce the inconvenience it causes by providing advance notice, clear documentation, and, when possible, tooling or other assistance.
It's easy to think that perl is just about what you do in perl, but over the years perl has been used, and is still being used for a broad range of things. I think it would help us to have a shared picture of those, but that's for another day.
Effectiveness
This is about continuously evolving and improving Perl, to make the lives of active Perl developers better: increase productivity, prevent common bugs, improve performance, etc.
Some of this is about filling gaps in the features provided by Perl, which developers may be used to from other languages.
This can also be small incremental improvements in the language, particularly where we have the evidence to support them, for example where people regularly make the same mistake, or there's an commonly used idiom. To borrow a phrase from Nicholas, if we think something should be in the FAQ, then we should also consider whether it's something the language should provide.
This might also be by including additional modules in the core. There are differing thoughts here: some want to see more batteries included, but every additional core module is additional maintenance and release work.
When adding new features or feature bundles, we also need to think about all the existing code out there and how we help it bridge the gap without requiring a stressful flag-day effort.
Growth
Many of the features that we'd like to see in Perl are things that have been provided by plenty of other languages for years. We're playing catch-up. Many of these features aren't likely to attract significant numbers of new developers, but the aggregate effect of them will hopefully bring us up to par.
Considering the leaky bucket model, some of the things we can do for growth are really about reducing leakage.
That said, there are no doubt changes we could make to give Perl an edge over other languages, some of which have been called for in recent discussions. But we have to remember the resources we have available and that we're a volunteer organisation.
What will really attract new Perl developers in greater numbers is CPAN distributions for emerging domains. Lots of people have learned Python because of the machine learning support. There's a chicken and egg problem here, but it's not one for P5P to try and solve.
It's also tempting to bundle Effectiveness and Growth together, but I think it's worth separating them, as (a) I think the things that will attract significant numbers of developers aren't the language, and (b) some significant changes to the language possibly would attract new developers, but at the cost of existing developers, which in general isn't a trade-off I think we should make.
Final Thoughts
These aren't gates, but an order in which we should consider things. It doesn't mean that we can't consider changes that will break things, but that we have to make an informed decision, and that some considerations are more important than others. A sexy new feature that might attract hordes of new developers won't be acceptable if it makes Perl less secure, or inconveniences significant numbers of our existing users or current developers.
None of this is surprising or new. It's an attempt to write down what I think most people feel. Some of these ideas are already built-in to how new capabilities are rolled out, with feature guards, for example, and the "use vN" version bundles.
So, should reliability of old code come ahead of the e of current developers? There's surely some existing perl code in the wild that no one cares about any more (let's call it "abandoned code"), and if we broke it, its owners wouldn't care. But we've no way of identifying all the abandoned code, so it's unrealistic that we could define changes that would only break abandoned code. And if we tried this, then people would naturally worry that at some point in the future we might sacrifice their code's reliability to the new feature gods, and we'd lose perhaps the strongest pillar of Perl's reputation.
In addition to the priorities outlined here, we think there are principles that we apply as well. For example, in designing new language features, we should ensure they are consistent with the existing language.
Various points related to this have come up in discussions on P5P, so I think it might be helpful to see how much of this we agree on. Think of this as P5P's mission statement, or manifesto.
We think our priorities are, in decreasing order:
1. Deal with security issues promptly (Security).
2. Keep perl code running (Reliability).
3. Improve the language for developers who are maintaining existing code or writing new code (Effectiveness).
4. Provide reasons for people to return to Perl, or to start writing Perl (Growth).
One of the points of contention is the priority of Reliability vs Effectiveness: there's a lot of old code out there that no-one cares about (any more) – do we really think keeping that code running is more important than extending the language to support today's active developers? Let's look at the priorities in turn, and then revisit this question.
Security
If security bugs are reported or discovered, then they are given appropriate priority, and they will often outweigh other considerations. When addressing security issues, we hope that the fix won't impact the other considerations. Sometimes that's not possible though, and security wins out. For example, "no . in @INC".
This is not only "fix security bugs", but also "try and prevent security bugs in Perl".
Reliability
Once someone has written Perl code and it has been "put into production", we should do our best to ensure that it continues to work when they upgrade to a new version of Perl. Perl has a reputation for backward compatibility, earned over decades. That reputation is valuable, and we should only spend that capital in return for something sufficiently valuable to warrant it.
- Aim for backwards compatibility when introducing new features.
- Fix bugs, while remembering that sometimes bugs become relied upon, and fixing those ones can break code.
- Be mindful of the breakage of CPAN code, which is our best proxy for how proprietary code may break.
This doesn't mean that we must be 100% backwards compatible and never break any CPAN modules, but that we should only deviate from that when we've carefully considered the impact. If a change would break three twenty-year old modules on CPAN that no-one seems to use, that's a different proposition from breaking a module that's far up the CPAN River.
Furthermore, if we do decide that it's ok to break something, we should try to reduce the inconvenience it causes by providing advance notice, clear documentation, and, when possible, tooling or other assistance.
It's easy to think that perl is just about what you do in perl, but over the years perl has been used, and is still being used for a broad range of things. I think it would help us to have a shared picture of those, but that's for another day.
Effectiveness
This is about continuously evolving and improving Perl, to make the lives of active Perl developers better: increase productivity, prevent common bugs, improve performance, etc.
Some of this is about filling gaps in the features provided by Perl, which developers may be used to from other languages.
This can also be small incremental improvements in the language, particularly where we have the evidence to support them, for example where people regularly make the same mistake, or there's an commonly used idiom. To borrow a phrase from Nicholas, if we think something should be in the FAQ, then we should also consider whether it's something the language should provide.
This might also be by including additional modules in the core. There are differing thoughts here: some want to see more batteries included, but every additional core module is additional maintenance and release work.
When adding new features or feature bundles, we also need to think about all the existing code out there and how we help it bridge the gap without requiring a stressful flag-day effort.
Growth
Many of the features that we'd like to see in Perl are things that have been provided by plenty of other languages for years. We're playing catch-up. Many of these features aren't likely to attract significant numbers of new developers, but the aggregate effect of them will hopefully bring us up to par.
Considering the leaky bucket model, some of the things we can do for growth are really about reducing leakage.
That said, there are no doubt changes we could make to give Perl an edge over other languages, some of which have been called for in recent discussions. But we have to remember the resources we have available and that we're a volunteer organisation.
What will really attract new Perl developers in greater numbers is CPAN distributions for emerging domains. Lots of people have learned Python because of the machine learning support. There's a chicken and egg problem here, but it's not one for P5P to try and solve.
It's also tempting to bundle Effectiveness and Growth together, but I think it's worth separating them, as (a) I think the things that will attract significant numbers of developers aren't the language, and (b) some significant changes to the language possibly would attract new developers, but at the cost of existing developers, which in general isn't a trade-off I think we should make.
Final Thoughts
These aren't gates, but an order in which we should consider things. It doesn't mean that we can't consider changes that will break things, but that we have to make an informed decision, and that some considerations are more important than others. A sexy new feature that might attract hordes of new developers won't be acceptable if it makes Perl less secure, or inconveniences significant numbers of our existing users or current developers.
None of this is surprising or new. It's an attempt to write down what I think most people feel. Some of these ideas are already built-in to how new capabilities are rolled out, with feature guards, for example, and the "use vN" version bundles.
So, should reliability of old code come ahead of the e of current developers? There's surely some existing perl code in the wild that no one cares about any more (let's call it "abandoned code"), and if we broke it, its owners wouldn't care. But we've no way of identifying all the abandoned code, so it's unrealistic that we could define changes that would only break abandoned code. And if we tried this, then people would naturally worry that at some point in the future we might sacrifice their code's reliability to the new feature gods, and we'd lose perhaps the strongest pillar of Perl's reputation.
In addition to the priorities outlined here, we think there are principles that we apply as well. For example, in designing new language features, we should ensure they are consistent with the existing language.