Mailing List Archive

RIP Smartmatch

On 12/2/2022 8:12 AM, Paul "LeoNerd" Evans wrote:
> [With my PSC hat on]
>
> In Perl 5.10 we added `~~`, known as smartmatch, as a sort-of copy of
> what is now in Raku (known as "Perl 6" at the time). Initially looked
> at favourably, design problems were eventually found around it that
> mean it is a poor match for the different type system that Perl 5 has.
> This operator (and the related `given/when` syntax) sat in
> "experimental" status ever since Perl 5.18, while we attempted to work
> out what to do with it.
>
> The time has now come to finally declare an end to this experiment. It
> has not been possible to provide a meaningful, coherent set of
> semantics for this operator that sits well with users. Because of this,
> we feel it best to move the operator into "deprecated" status,
> announcing that we *will* be removing it in a later version of perl.
> This is currently scheduled for two releases later (Perl 5.42 in the
> current schedule).
>
> By the time that Perl 5.38 is released, we aim to have some updated
> documentation that explains the deprecated feature and the reasons
> behind it, and also gives advice to users who are already using this
> syntax on what they can write instead to replace it.
>
> We also aim to provide some thoughts on what new features may be added
> to Perl in the future to provide a better solution to these common
> situations that smartmatch and `given/when` are currently used for.
>

On Fri, 2 Dec 2022 16:12:35 +0000
"Paul \"LeoNerd\" Evans" <leonerd@leonerd.org.uk> wrote:

> [With my PSC hat on]
...
> We also aim to provide some thoughts on what new features may be added
> to Perl in the future to provide a better solution to these common
> situations that smartmatch and `given/when` are currently used for.

[.And now with my personal hat on. These are just my own thoughts and
not the collected decisions of the PSC as a whole.]

I've been working on a number of CPAN module that implement syntax
extensions, which could be good design candidates for something to
provide in core.

Each of these has been designed, not to be a *total* replacement of
everything that smartmatch or given/when can do (because indeed, most
of the problem with them is the huge wide-ranging set of conflicting
features and behaviours they had), but instead to attack one small
specific problem area and find a nice neat solution to *that* problem
in a way that can be combined with the others, in programmer-controlled
(rather than runtime value-controlled) ways.

* The `match/case` statement
https://metacpan.org/pod/Syntax::Keyword::Match

The main use-case for given/when is to ask the question "does this
scalar value match any of these possible things?" - much like a
C-style switch statement. For this scenario, I imagined a new pair of
keywords used in a similar style to given/when. A key difference
here is that they don't use smartmatch, but instead use an operator
that the programmer specified upfront. That way, there can be no
surprises that it does "odd things" when it encounters some weird
value you weren't expecting. If you expected string equality
matching, you *know* you are getting string equality matching
because it says so right there:

my $var = "abc";
match($var : eq) {
case("abc") { $ok++ }
case("def") { fail('Not this one sorry'); }
}

* The `in` meta-operator
https://metacpan.org/pod/Syntax::Operator::In

One thing people liked to use `~~` for is to ask "is this value a
member of that list of values?"; a sortof set-element-of operator.
While it's been possible to ask this question using List::Util::any
ever since perl 5.6 (and likely before - I don't remember that far
back in time), people still aren't satisfied with that notation:

if($x ~~ @numbers) { ... }

if(any { $x == $_ } @numbers) { ... }

For those folks, I have written an `in` meta-operator. Motivated in
much the same way as match/case above, there can be no ambiguity on
"is this matching with number or string equality here?" because it
says right upfront in the operator name itself. You tell it - `==`
or `eq`.

if($x in:== @numbers) { ... }

* The `equ` and `===` operators
https://metacpan.org/pod/Syntax::Operator::Equ

Once the above things are taken into account, there isn't much left
that's "obvious" about smartmatch or given/when, that isn't already
handled. One remaining problem is that given/when and smartmatch can
distinguish undef from both the number zero and the empty string.
Neither of the above things can do that.

For this problem I imagined two new operators, `equ` and `===`, which
are slight variants of the familiar `eq` and `==` but which consider
undef to be a distinct value separate from empty or zero.

if($n === 0) { say "n is zero" }
else { say "n is undefined, or non-zero" }

Useful on their own perhaps, but they become more useful when you
realise you can use them with `match` or `in`:

match($x : equ) {
case(undef) { say "x is undef" }
case("") { say "x is the empty string" }
default { say "x is a non-empty string" }
}

if($x in:=== (0..9)) { say "x is a defined single-digit number" }


To recap on above: This set of ideas isn't supposed to cover every
possible use-case of smartmatch and given/when. Rather, it is intended
to cover a wide range of practical useful things that people *actually*
use the syntax for. Each was designed in the face of real practical
problems. Furthermore, match/case is already in use in a number of CPAN
modules, and actual real-world programs that I run all the time:

https://metacpan.org/module/Syntax::Keyword::Match/requires

I believe that, between them, this set of ideas and syntax could be a
useful addition to the Perl language, as a much more predictable and
controllable alternative to smartmatch.

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/

On 2022-12-02 9:38 a.m., Paul "LeoNerd" Evans wrote:
> ... new pair of
> keywords used in a similar style to given/when ... use an operator
> that the programmer specified upfront. That way, there can be no
> surprises that it does "odd things" when it encounters some weird
> value you weren't expecting. If you expected string equality
> matching, you *know* you are getting string equality matching
> because it says so right there:
>
> my $var = "abc";
> match($var : eq) {
> case("abc") { $ok++ }
> case("def") { fail('Not this one sorry'); }
> }

I agree fully that this is the right way to go.

I would further say that it must be mandatory for the programmer to specify the
comparison operator to use. There must not be an implicit default for leaving
that out, because this would be full of surprises, saving a few characters isn't
worth the ambiguity.

I will also add that this construct must be generalized enough to support
anonymous inline defined operators such as map/grep/sort/etc do with their block
syntaxes, and that this is not limited to things for which an infix operator
syntax exists.

One can bikeshed the exact syntax.

> ... a sortof set-element-of operator ... `in` meta-operator. Motivated in
> much the same way as match/case above, there can be no ambiguity on
> "is this matching with number or string equality here?" because it
> says right upfront in the operator name itself. You tell it - `==`
> or `eq`.
>
> if($x in:== @numbers) { ... }

I also like this idea, especially the programmer-defined operator to use. And
everything I said about mandatoryness and anonymous operators applies here too.
But in contrast to the switch-case type expression above, this list membership
thing may be best to remain a module without dedicated language syntax, because
there are many variations or list operations that it would be good to have that
flexibility with, and it would be hard to pick a good set for core, and special
syntax doesn't really add much over List::Util etc, or maybe such syntax should
be a List::Util enhancement instead.

> ... `equ` and `===`, which
> are slight variants of the familiar `eq` and `==` but which consider
> undef to be a distinct value separate from empty or zero.
>
> if($n === 0) { say "n is zero" }
> else { say "n is undefined, or non-zero" }
>
> Useful on their own perhaps, but they become more useful when you
> realise you can use them with `match` or `in`:

I agree that having these undef-distinguishing variants would be a very good
thing to have in core contemporary to `eq` and `==`. One might bikeshed on
their syntax, but in principle taking the existing operator names and adding 1
character for the variant seems to be an elegant choice.

So in summary, I agree with the match-case and eqv/=== additions to core, but I
feel the "in" should be left for the likes of List::Util. The first 2 have a
high win for core syntax AND they are relatively uncomplicated, while the last 1
is a lot more complicated in its variants and best left in modules, albeit
possibly core-bundled modules.

-- Darren Duncan

On 2022-12-02 21:11, Darren Duncan wrote:

> [...]
> So in summary, I agree with the match-case and eqv/=== additions to
> core, but I feel the "in" should be left for the likes of List::Util.
> The first 2 have a high win for core syntax AND they are relatively
> uncomplicated, while the last 1 is a lot more complicated in its
> variants and best left in modules, albeit possibly core-bundled modules.

I'd love to have the 'in' into Perl, with a default of 'eq', much like
how 'sort' defaults to 'cmp'.

And I still mourn for 'dor' to not be there yet.

-- Ruud

P.S. I always rejected usage of: lexical_topic; switch; indirect;
multidimensional; bareword_filehandles; looks_like_number; $@ checks.

I strongly support weeding out value-based semantics, like is done with
bitwise. And I'm longing for more mature signatures, so I no longer have
to code like \my($x,$y)=\(@_);
:)

Op 02-12-2022 om 18:38 schreef Paul "LeoNerd" Evans:
>
> I believe that, between them, this set of ideas and syntax could be a
> useful addition to the Perl language, as a much more predictable and
> controllable alternative to smartmatch.
>

Wouldn't this syntax be worthwhile to be generalised, so it can be used
for user defined subs as well?


F.i.

sub my_in($needle :use($op), @haystack)

{

    for (@haystack) {

        return $_ if $_ :$op $needle;

}


Note this is a brainstorm idea, the above has some obvious problems.
First there is a syntax problem, as written it clashes with ternaries.
Secondly, different operations would need $needle to be either on the
LHS or RHS, but I already mailed about that in another separate mail,
better to discuss that problem there first.


Obviously it would be great if this would be somewhere in the future. I
bring it up now because if we want it, it might have effect on the
syntax of the currently proposed 'in'.


HTH,

M4

[. resend, sent it to Paul instead of p5p, sorry Paul ]


Op 02-12-2022 om 18:38 schreef Paul "LeoNerd" Evans:

> * The `in` meta-operator
> https://metacpan.org/pod/Syntax::Operator::In
>
> One thing people liked to use `~~` for is to ask "is this value a
> member of that list of values?"; a sortof set-element-of operator.
> While it's been possible to ask this question using List::Util::any
> ever since perl 5.6 (and likely before - I don't remember that far
> back in time), people still aren't satisfied with that notation:
>
> if($x ~~ @numbers) { ... }
> if(any { $x == $_ } @numbers) { ... }
>
> For those folks, I have written an `in` meta-operator. Motivated in
> much the same way as match/case above, there can be no ambiguity on
> "is this matching with number or string equality here?" because it
> says right upfront in the operator name itself. You tell it - `==`
> or `eq`.
>
> if($x in:== @numbers) { ... }


A thought occured to me of a possible problem here. Consider:

    $its_in = $needle in:=~ qr/haystack/;

v.s.

    $its_in = $needle in:< @haystack;

Wouldn't the first to be the RHS of the op, and the second the LHS?
Wouldn't this create a problem? Or do I simply misunderstand?


M4

On Sun, 4 Dec 2022 15:08:45 +0100
Martijn Lievaart <m@rtij.nl> wrote:

> A thought occured to me of a possible problem here. Consider:
>
>     $its_in = $needle in:=~ qr/haystack/;
>
> v.s.
>
>     $its_in = $needle in:< @haystack;
>
> Wouldn't the first to be the RHS of the op, and the second the LHS?
> Wouldn't this create a problem? Or do I simply misunderstand?

I'm not sure I see the problem.

Actually for several reasons. Firstly, `in` doesn't permit arbitrary
operators. Only `eq`, `==`, or anything custom registered with
XS::Parse::Infix that says it is an equality operator (thus permitting
`equ` and `===`). It would not permit `<` or even `=~`.

Secondly I don't at all follow what you mean about "be the RHS of the
op" here. Can you explain it some more?

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/

Paul "LeoNerd" Evans wrote:
>we feel it best to move the operator into "deprecated" status,
>announcing that we *will* be removing it in a later version of perl.

FWIW, I approve of this move. I thought it was still possible to salvage
a usable smartmatch operator by something close to what we tried in
5.27.7, but it is telling that there has been no similar effort in the
intervening five years. With no such fix in sight, it is right that
smartmatch be declared a failure and be removed.

-zefram