Hi! I said I would try to start going through, piece by piece, what I think is a viable way to plan for the future of Perl. I think this topic is way too big to eat in one bite, and that we need to break it down into pieces. It's a thing that can be broken down in a lot of different ways, but I have kind of an idea of how I'd like to go through it all. Please bear with me. If I end up saying, "Let's come back to that," I promise I'll put it on a list, and that I think we really come back to it, but that it would be a problem or distraction at the moment. Again: I think if we try to talk about everything all at once with everybody on the list, it'll be a lot of noise. Let's see how we can do!
For as long as I've been on perl5-porters, there's been a tension between reliability and improvement. I wrote about this about six years ago in the "perlpolicy v2 thread <http://markmail.org/thread/syv7ls43bzhzuvjc>". Here's part of what I said:
> The two priority that are at opposition here are something like this:
> * Perpetual reliability, because it's infuriating when a documented behavior goes away, after you've had code using it quietly and productively for years
> * Continual improvement, because when you learn lessons about what doesn't work, you want to benefit from them without throwing away the entire project.
I stand by what I wrote in that email, and I won't rehash it. You can read it yourself, if you like. My point, though, was that we should have a policy that acknowledged this tension, and we should review changes to the language in light of it.
That thread also talks about what it means for a changes to be lack backward compatibility. It means that your program changes behavior when you upgrade the language without changing the program. Adding a new warning is an incompatible change. Removing a feature is an incompatible change. Even adding a feature can be an incompatible change. Let's consider the lifespan of `$*`.
Way back in v5.8 and earlier, $* was a global variable. Roughly, setting it to true made all regex act like /m was applied.
As you can imagine, this was sort of a common footgun, and was deprecated for ages. It was diasbled in v5.10.0 — but actually in v5.9.0, very early on in the (very long) v5.10.0 development cycle. Referencing $* at all got you a compile time warning: "$* is no longer supported".
This remained true in v5.20, when we introduced postfix dereferencing. Given a globref, you might write $globref->$* but this would emit a warning about $* and act like you tried to call a method named "". You had to turn on the postderef feature to get a glob deref.
A few years later, we turned on postderef by default, meaning that ->$* did not mean "a method call with the empty-named method", but meant "dereference as glob." This *would* have broken code that might have worked before:
~$ perl -le 'print $];' \
-e 'sub Foo::AUTOLOAD { print "Auto: <$Foo::AUTOLOAD>" }' \
-e '$x = bless {}, "Foo";' \
-e '$x->$*'
$* is no longer supported at -e line 4.
5.020003
Auto: <Foo::>
…then…
~$ perl -le 'print $];' \
-e 'sub Foo::AUTOLOAD { print "Auto: <$Foo::AUTOLOAD>" }' \
-e '$x = bless {}, "Foo";' \
> -e '$x->$*'
5.024004
Not a SCALAR reference at -e line 4.
Turning off postfix deref was no longer even possible. In other circumstances, $* was treated normally, acting like a normal global variable, but warning when used.
Finally, in v5.30, using $* became a fatal compile-time error.
These are a bunch of backward incompatible changes, all more or less motivated by "make the language easier to use" and, as incompatible changes, with some baggage of "someone out there will be inconvenienced." I was both inconvenienced *and* helped by possibly all of these changes. Some people might fall into only one of those groups, or neither.
$* is a nice example because it's connected to so many changes, but it's not the only place we've changed defaults, either to deprecate things, to remove deprecated things, or to just change behavior. (For example, the meaning of "scalar %hash" changed between v5.24 and v5.26.) Some of these changes change runtime behavior of the code, some change the legal syntax of Perl. The "only perl can parse Perl" rule will apply to these in different amounts.
I think that was Perl moves forward, we need to continue to evaluate what changes like these are, on the whole, beneficial. That's a complicated problem to measure, and it won't be something I think we can properly quantify but I think it's something we can actually look at, think about, and come to consensus on when it comes to individual changes. We need to consider what the benefit is, who benefits, who gets inconvenienced, how it could improve the language over time, and how inconvenient the inconvenience is.
The current feature bundle contains: bitwise, current_sub, evalbytes, fc, indirect, postderef_qq, say, state, switch (good grief), unicode_eval, and unicode_strings. I think some of these are things that should become default parts of the language. It's helpful to people who write code to a recent version of perl only. It makes it possible to eventually make the feature always-on.
Not every feature is likely to work that way. The road to turning on unicode_strings by default is unclear. While it'd be nice to fix the semantics of string operations once and for all, the inconvenience is quite high, because we easily provide a single helpful warning about what to do, or whether the user is going to see things change, and whether it'd be for the worse.
So I think, after saying all this, the first big question is: Is there a general agreement that there are kinds of changes we've made (or will make) to the language that we can ease into making the default, through some multi-step process? We may need to hash out individual changes' paths forward, but if there is an overwhelming opposition to changing these sorts of defaults *at all*, then I think a lot of this conversation has to be entirely reconsidered.
--
rjbs
For as long as I've been on perl5-porters, there's been a tension between reliability and improvement. I wrote about this about six years ago in the "perlpolicy v2 thread <http://markmail.org/thread/syv7ls43bzhzuvjc>". Here's part of what I said:
> The two priority that are at opposition here are something like this:
> * Perpetual reliability, because it's infuriating when a documented behavior goes away, after you've had code using it quietly and productively for years
> * Continual improvement, because when you learn lessons about what doesn't work, you want to benefit from them without throwing away the entire project.
I stand by what I wrote in that email, and I won't rehash it. You can read it yourself, if you like. My point, though, was that we should have a policy that acknowledged this tension, and we should review changes to the language in light of it.
That thread also talks about what it means for a changes to be lack backward compatibility. It means that your program changes behavior when you upgrade the language without changing the program. Adding a new warning is an incompatible change. Removing a feature is an incompatible change. Even adding a feature can be an incompatible change. Let's consider the lifespan of `$*`.
Way back in v5.8 and earlier, $* was a global variable. Roughly, setting it to true made all regex act like /m was applied.
As you can imagine, this was sort of a common footgun, and was deprecated for ages. It was diasbled in v5.10.0 — but actually in v5.9.0, very early on in the (very long) v5.10.0 development cycle. Referencing $* at all got you a compile time warning: "$* is no longer supported".
This remained true in v5.20, when we introduced postfix dereferencing. Given a globref, you might write $globref->$* but this would emit a warning about $* and act like you tried to call a method named "". You had to turn on the postderef feature to get a glob deref.
A few years later, we turned on postderef by default, meaning that ->$* did not mean "a method call with the empty-named method", but meant "dereference as glob." This *would* have broken code that might have worked before:
~$ perl -le 'print $];' \
-e 'sub Foo::AUTOLOAD { print "Auto: <$Foo::AUTOLOAD>" }' \
-e '$x = bless {}, "Foo";' \
-e '$x->$*'
$* is no longer supported at -e line 4.
5.020003
Auto: <Foo::>
…then…
~$ perl -le 'print $];' \
-e 'sub Foo::AUTOLOAD { print "Auto: <$Foo::AUTOLOAD>" }' \
-e '$x = bless {}, "Foo";' \
> -e '$x->$*'
5.024004
Not a SCALAR reference at -e line 4.
Turning off postfix deref was no longer even possible. In other circumstances, $* was treated normally, acting like a normal global variable, but warning when used.
Finally, in v5.30, using $* became a fatal compile-time error.
These are a bunch of backward incompatible changes, all more or less motivated by "make the language easier to use" and, as incompatible changes, with some baggage of "someone out there will be inconvenienced." I was both inconvenienced *and* helped by possibly all of these changes. Some people might fall into only one of those groups, or neither.
$* is a nice example because it's connected to so many changes, but it's not the only place we've changed defaults, either to deprecate things, to remove deprecated things, or to just change behavior. (For example, the meaning of "scalar %hash" changed between v5.24 and v5.26.) Some of these changes change runtime behavior of the code, some change the legal syntax of Perl. The "only perl can parse Perl" rule will apply to these in different amounts.
I think that was Perl moves forward, we need to continue to evaluate what changes like these are, on the whole, beneficial. That's a complicated problem to measure, and it won't be something I think we can properly quantify but I think it's something we can actually look at, think about, and come to consensus on when it comes to individual changes. We need to consider what the benefit is, who benefits, who gets inconvenienced, how it could improve the language over time, and how inconvenient the inconvenience is.
The current feature bundle contains: bitwise, current_sub, evalbytes, fc, indirect, postderef_qq, say, state, switch (good grief), unicode_eval, and unicode_strings. I think some of these are things that should become default parts of the language. It's helpful to people who write code to a recent version of perl only. It makes it possible to eventually make the feature always-on.
Not every feature is likely to work that way. The road to turning on unicode_strings by default is unclear. While it'd be nice to fix the semantics of string operations once and for all, the inconvenience is quite high, because we easily provide a single helpful warning about what to do, or whether the user is going to see things change, and whether it'd be for the worse.
So I think, after saying all this, the first big question is: Is there a general agreement that there are kinds of changes we've made (or will make) to the language that we can ease into making the default, through some multi-step process? We may need to hash out individual changes' paths forward, but if there is an overwhelming opposition to changing these sorts of defaults *at all*, then I think a lot of this conversation has to be entirely reconsidered.
--
rjbs