Mailing List Archive

arity checking (was Re: PSC #049 2022-01-07)
This might seem flippant or obvious but it's not:

There is only one perl core, so it can only have one default.


Whereas different use cases often benefit from different defaults.

(And different people have different common use cases. And this isn't a
problem with modules on CPAN, as there can be more than one module to solve
the same problems.)

On Sun, Jan 16, 2022 at 11:58:00AM +0000, Paul "LeoNerd" Evans wrote:
> On Sun, 16 Jan 2022 12:23:53 +0100
> demerphq <demerphq@gmail.com> wrote:

I think that this is the key insight:

> > If one could introspect on the arity of a sub then to some level this
> > could be avoided. GEg, when the new framework accepts a sub designed
> > for the old 1 argument API, it could internally ensure that sub was
> > called with 1 argument, and call the subs intended for more than 1
> > argument as required. If you can't introspect on the arity then you
> > can not harden yourself for this. Eg, a forward looking dev could say
> > "my callback framework insists the callback accepts a slurpy argument
> > list" and warn or die when someone passes in one that doesnt have
> > artiy==-1 (-1 for infinity), and one less prescient could do
> > something else internally, something like:
> >
> > my $arity=arity($sub_ref);
> > if ($arity == 1) {
> > $sub_ref->($key)
> > }
> > elsif ($arity < 0 or $arity == 2 ) {
> > $sub_ref->($key,$index);
> > }


> Could be used with any of the following, without error:
>
> ->each(sub ($item) { ... });
> ->each(sub ($item, $idx) { ... });
> ->each(sub ($item, $idx, $c) { ... });
>
>
> That said, my ideal wish would be for arity-checking to appear as a
> default-fatal exception from the caller's perspective; so the caller
> could do something like:
>
> sub each ($code) {
> ...
> {
> no fatal 'signatures::max_arity';
> $code->($item, $index, $context);
> }
> }
>
> To, within that small lexical scope, disarm the fatality of maximum
> arity checking.

Why is maximum arity checking special?

Am I also allowed

no fatal 'signatures::min_arity';

to disable minimum arity checking?

I'm not sure if *that* is useful, but constraints as flat text in package
names doesn't scale up. "Does it take a slurpy hash?" is also a question
that calling code might care about, and how many positional parameters
before said hash, because that can be a way to extend APIs.


Really you want to know you have a problem *before* you call the callback
(ie what Yves wrote - you want to be able to flag a problem at object
construction time, which might well be program startup. Not fail at runtime,
unattended, into a log file or even /dev/null)

and generally I thought it was considered that exceptions should be for
things that went wrong, not control logic



Also, *if* you have arity introspection (not just "no fatal"), you can
implement more nuanced support for changed callbacks, such as filling in
missing parameters, or calling the callback if the "missing" extra
parameter is undef, but hard failing if the caller needed to pass non-undef
but the "old style" callback would ignore it.




And I don't think that being able to turn of arity checks - ie some

use feature 'no_arity_check';

really solves the problem. Because if the default default is strict arity
checks, then however much you document the API as "provide a callback that
accepts future extra parameters", someone using the API is going to get it
"wrong" and call it with a strict arity callback.

And it's all going to work (and pass tests) for months, until the time when
you extend the API (as planned and documented) and the client code that
passes a strict arity callback breaks. At runtime.


Hence I think that optional parameters (whatever syntax enables them) really
is just a band aid. At best it doesn't solve the underlying problem. At
worst it will create a false sense of security.

Nicholas Clark
Re: arity checking (was Re: PSC #049 2022-01-07) [ In reply to ]
On Fri, 21 Jan 2022 at 14:12, Nicholas Clark <nick@ccl4.org> wrote:

> Really you want to know you have a problem *before* you call the callback
> (ie what Yves wrote - you want to be able to flag a problem at object
> construction time, which might well be program startup. Not fail at
> runtime,
> unattended, into a log file or even /dev/null)
>

So am i correct in understanding that you would be ok with exposing an API
to check the subs arity signature? If so then I'll try to think up
one (others could too).

cheers
Yves
Re: arity checking (was Re: PSC #049 2022-01-07) [ In reply to ]
On Fri, Jan 21, 2022 at 03:15:31PM +0100, demerphq wrote:
> On Fri, 21 Jan 2022 at 14:12, Nicholas Clark <nick@ccl4.org> wrote:
>
> > Really you want to know you have a problem *before* you call the callback
> > (ie what Yves wrote - you want to be able to flag a problem at object
> > construction time, which might well be program startup. Not fail at
> > runtime,
> > unattended, into a log file or even /dev/null)
> >
>
> So am i correct in understanding that you would be ok with exposing an API
> to check the subs arity signature? If so then I'll try to think up
> one (others could too).

But this is a general problem, not just one with signatured subs. A
"normal" callback sub could be written as:

sub callback {
croak unless @_ == 2;
(my $self, $foo) = @_;
...
}

How would the caller (say an event handler module), which has been upgraded
to pass an optional next arg, know that the callback sub it wanted to call
would croak on that extra arg?

I think in the past that strong opinions have been expressed against
allowing a signature API which a caller can use to inspect a sub, on the
grounds that really the sub signature is an implementation detail - just a
fancy and fast way of writing a @_ unpacker, and in general, doesn't form
a promised API.

--
You're only as old as you look.
Re: arity checking (was Re: PSC #049 2022-01-07) [ In reply to ]
On Fri, 21 Jan 2022 at 17:26, Dave Mitchell <davem@iabyn.com> wrote:

> On Fri, Jan 21, 2022 at 03:15:31PM +0100, demerphq wrote:
> > On Fri, 21 Jan 2022 at 14:12, Nicholas Clark <nick@ccl4.org> wrote:
> >
> > > Really you want to know you have a problem *before* you call the
> callback
> > > (ie what Yves wrote - you want to be able to flag a problem at object
> > > construction time, which might well be program startup. Not fail at
> > > runtime,
> > > unattended, into a log file or even /dev/null)
> > >
> >
> > So am i correct in understanding that you would be ok with exposing an
> API
> > to check the subs arity signature? If so then I'll try to think up
> > one (others could too).
>
> But this is a general problem, not just one with signatured subs.


If you read the scenario that lead to this you will see that actually they
are very very different. One is a "doctor it hurts when is stick a fork in
my eye", and one is a "doctor it hurts when YOU stick a fork in my eye".

So consider, I own a framework whose api is that I accept a callback from
the caller. I tell the caller that my api will call his sub with 1
argument, lets say "key".

You write a callback which takes one argument, and you pass it into the
framework. All is good.

Later on I get a request from a user to pass in two arguments to the
callers, "key" and "idx".

With the unsignatured subs UNLESS you put the guard clause you just stated
into the callback all will be good. Old users of the api will continue to
work as they will just ignore the new argument. New users of the api who
are aware of the new parameter will get the argument and all will be good.

But with signatured subs all of a sudden the old users will break as they
violate the arity check. And the framework can't "defend" against this by
validating the signature. So basically signatured subs become useless in a
callback api without a way to introspect the artity.



> A
> "normal" callback sub could be written as:
>
> sub callback {
> croak unless @_ == 2;
> (my $self, $foo) = @_;
> ...
> }
>
> How would the caller (say an event handler module), which has been upgraded
> to pass an optional next arg, know that the callback sub it wanted to call
> would croak on that extra arg?
>

In this case it is self-inflicted. The person supplying the sub into the
framework broke the framework and gets to keep the pieces. The *framework*
owner isnt breaking the *callers* code, the caller is doing itself by
adding a constraint the framework author didnt ask for.

But in the signature case the framework author ends up in a trap, the only
way they can deal with this is change the api, eg, maybe the original code
had

$obj->new(callback=>$sub);

now they add a new argument and have to support:

$obj->new(callback=>$sub);
$obj->new(callback_2arg => $sub);

that starts getting ugly. If they could introspect on the artity of the sub
then this wouldnt be an issue at all. And everything would work fine.


>
> I think in the past that strong opinions have been expressed against
> allowing a signature API which a caller can use to inspect a sub, on the
> grounds that really the sub signature is an implementation detail - just a
> fancy and fast way of writing a @_ unpacker, and in general, doesn't form
> a promised API.
>

I think the case explained in the previous thread renders this argument
somewhat religious and not necessary and certainly not practical. If perl
itself can complain and check the artity then it can expose a function to
do the same.

cheers,
Yves
Re: arity checking (was Re: PSC #049 2022-01-07) [ In reply to ]
On Fri, Jan 21, 2022 at 11:26 AM Dave Mitchell <davem@iabyn.com> wrote:

> On Fri, Jan 21, 2022 at 03:15:31PM +0100, demerphq wrote:
> > On Fri, 21 Jan 2022 at 14:12, Nicholas Clark <nick@ccl4.org> wrote:
> >
> > > Really you want to know you have a problem *before* you call the
> callback
> > > (ie what Yves wrote - you want to be able to flag a problem at object
> > > construction time, which might well be program startup. Not fail at
> > > runtime,
> > > unattended, into a log file or even /dev/null)
> > >
> >
> > So am i correct in understanding that you would be ok with exposing an
> API
> > to check the subs arity signature? If so then I'll try to think up
> > one (others could too).
>
> But this is a general problem, not just one with signatured subs. A
> "normal" callback sub could be written as:
>
> sub callback {
> croak unless @_ == 2;
> (my $self, $foo) = @_;
> ...
> }
>
> How would the caller (say an event handler module), which has been upgraded
> to pass an optional next arg, know that the callback sub it wanted to call
> would croak on that extra arg?
>

The difference is a practical one: no one would pass a callback like your
example because it's asking for breakage and adds nothing, but most would
use a signature like ($self, $foo) in a callback documented to accept those
arguments.

-Dan
Re: arity checking (was Re: PSC #049 2022-01-07) [ In reply to ]
On Fri, Jan 21, 2022 at 06:29:27PM +0100, demerphq wrote:
> So consider, I own a framework whose api is that I accept a callback from
> the caller. I tell the caller that my api will call his sub with 1
> argument, lets say "key".

There's the crunch. If your framework promises to call always with one
arg, then if you call the callback with 2 args, you've broken your
promise.

If you've promised 1 arg, then the user of your framework should be able to
pass either of these subs:

sub callback { croak unless @_ == 1; my ($x) = @_ }
sub callback ($x) { }

If the framework promises it will pass at *least* one arg, then these
should both be valid:

sub callback { croak unless @_ >= 1; my ($x) = @_ }
sub callback ($x, @) { }

If the framework says you can't trust it in any fashion, then both of
these would do:

sub callback { my ($x) = @_ }
sub callback ($x=undef, @) { }

There is no fundamental difference between each pair of subs. In the first
pair, the writer of the sub has made a conscious choice to write a
subroutine with strict arity checking, both signatured and unsignatured
variants.

Now you can of course argue that signatures make it *easier* to
inadvertently write subroutines which have arity checking when with
hindsight it would have been easier to migrate to a new version of the
framework if you had disabled arity checking. But its not a fundamental
design problem with signatures.

If a framework author wants to break a promise about arity, they could
always call it with an eval:

if (defined $extra) {
eval { $self->$callback($x, $y, $extra) };
goto fallback if $@ =~ /Too many arguments/;
}
else {
fallback:
$self->$callback($x, $y);
}

which is as much (or little) hacky as introspecting the callback to see if
its a sub with a declared arity range.


--
You're only as old as you look.
Re: arity checking (was Re: PSC #049 2022-01-07) [ In reply to ]
On Sat, 22 Jan 2022 at 01:49, Dave Mitchell <davem@iabyn.com> wrote:

> On Fri, Jan 21, 2022 at 06:29:27PM +0100, demerphq wrote:
> > So consider, I own a framework whose api is that I accept a callback from
> > the caller. I tell the caller that my api will call his sub with 1
> > argument, lets say "key".
>
> There's the crunch. If your framework promises to call always with one
> arg, then if you call the callback with 2 args, you've broken your
> promise.
>

Do you have examples of such frameworks, where there's an explicit promise
of never adding more args? (rather than saying "this takes 1 argument" with
"at the moment" being implicit)

If those exist, I believe those are the exception, rather than the norm.

Anyway, this thread seems to be repeating what was discussed very recently
- see
https://www.nntp.perl.org/group/perl.perl5.porters/2022/01/msg262413.html
for an example use-case.

Your eval workaround assumes that the directly-called sub is the only
possible thing which can throw the exception - what if that's triggered by
a sub further down?

I ended up writing
https://metacpan.org/dist/Acme-Signature-Arity/view/lib/Acme/Signature/Arity.pod
as a proof-of-concept API, but would hope that core Perl will expose some
sort of arity inspection in due course. Ignoring this library-extensibility
use-case while simultaneously promoting signatures as a complete
replacement for @_-based subs seems less than ideal to me.
Re: arity checking (was Re: PSC #049 2022-01-07) [ In reply to ]
On Fri, 21 Jan 2022 17:49:40 +0000
Dave Mitchell <davem@iabyn.com> wrote:

> If a framework author wants to break a promise about arity, they could
> always call it with an eval:
>
> if (defined $extra) {
> eval { $self->$callback($x, $y, $extra) };
> goto fallback if $@ =~ /Too many arguments/;
> }
> else {
> fallback:
> $self->$callback($x, $y);
> }
>
> which is as much (or little) hacky as introspecting the callback to
> see if its a sub with a declared arity range.

Weelll - except that it will invoke fallback whenever an exception
matching that string is thrown from *anywhere* inside the callback.
It's not sensitive to just arity checking of $callback itself, but
might accidentally trigger many function-calls nested deeply inside it.

That's quite a hacky solution indeed ;) Lets step back a little.

Signatures work great in a normal "function-call" world wherein the API
shape (the interface definition) is given by the definition of the
function; the callers simply must adhere to it. Here, we put the API
shape on the definition of the function and all works just fine.

The situation it doesn't work in is this callback idea; the places
where the API shape is defined by the caller of some coderef, and
must be adhered to by implementors of those callback functions.

Consider in a statically-typed language, where the API signatures of
all of these things can be given upfront. Perhaps in that world we'd
have a way to define optional vs. required parameters. But as it stands
in Perl we don't. In this callback scenario the caller defines the
interface implicitly by their action. So far we don't have a way for the
caller to say "hey, I'm gonna pass you extra information but I don't
mind if you ignore it".

This was the motivation behind my original suggestion of some pragma
that turns off arity checking from the point of view of the caller:

{
no fatal 'arity::max';
$self->$callback($item, $idx); # $idx is optional
}

TEAM on CPAN has provided another hacky solution (this one inspects
optrees for the OP_ARGCHECK op and parses out the fields of it):

coderef_ignoring_extra($callback)->($self, $item, $idx);

https://metacpan.org/pod/Acme::Signature::Arity#coderef_ignoring_extra

Of course these solutions aren't very nice, partly because they don't
encode the fact that $item must be handled, but $idx may not be.
Perhaps we can do better?


What would be lovely would be if we could encode a position in a
function call site, to say "from this point onwards, don't worry if the
invoked sub does not handle this argument. They are optional". I wonder
about allowing the keyword `optional` in the middle of a function or
method call argument list:

$self->$callback($item, optional $idx);

A function/method call with one of these 'optional' words inside it is
then handled slightly differently. If the invoked sub fails to handle
at least all of the arguments before the 'optional' then that's fatal
as it is now, but the caller is saying "I don't mind if the callee
ignores any of the values after here". Thus $callback could be defined
by any of

sub ($self, $item) { ... }
sub ($self, $item, $idx) { ... }
sub ($self, $item, $) { ... }
sub ($self, $item, @) { ... }

and all would be just fine.

((It would otherwise be a syntax error for 'optional' to appear outside
of a function call argument list, or multiple of them, but that's just
book-keeping)).


This design shape stresses the inversion of the normal design shape of
function calls:

When dealing with callback scenarios it is the CALLER of the coderef
who decides the API shape, not the CALLEE. It is therefore up to the
caller to encode what is optional about it.


Now I'm not hugely fixed on this particular syntax as a solution to the
problem, but I feel that the principle of "caller specifies API" is one
we should explore.

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/
Re: arity checking (was Re: PSC #049 2022-01-07) [ In reply to ]
On Sat, 22 Jan 2022 02:20:21 +0800
Tom Molesworth via perl5-porters <perl5-porters@perl.org> wrote:

> Your eval workaround assumes that the directly-called sub is the only
> possible thing which can throw the exception - what if that's
> triggered by a sub further down?
>
> I ended up writing
> https://metacpan.org/dist/Acme-Signature-Arity/view/lib/Acme/Signature/Arity.pod
> as a proof-of-concept API, but would hope that core Perl will expose
> some sort of arity inspection in due course. Ignoring this
> library-extensibility use-case while simultaneously promoting
> signatures as a complete replacement for @_-based subs seems less
> than ideal to me.

Ah; I just addressed both of those points in my reply just now ;)

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/
Re: arity checking (was Re: PSC #049 2022-01-07) [ In reply to ]
So, up front, I'm not sure if I should write this message because I'm not sure
how much value I'm adding, and I failed to write a short message...



On Fri, Jan 21, 2022 at 05:49:40PM +0000, Dave Mitchell wrote:

> Now you can of course argue that signatures make it *easier* to
> inadvertently write subroutines which have arity checking when with
> hindsight it would have been easier to migrate to a new version of the
> framework if you had disabled arity checking. But its not a fundamental
> design problem with signatures.

I agree. With caveats.

It's that comment I made previously:

There is only one perl core, so it can only have one default.
Whereas different use cases often benefit from different defaults.


because in the case of signatures:

On Fri, Jan 21, 2022 at 12:33:49PM -0500, Dan Book wrote:
> On Fri, Jan 21, 2022 at 11:26 AM Dave Mitchell <davem@iabyn.com> wrote:

> > But this is a general problem, not just one with signatured subs. A
> > "normal" callback sub could be written as:
> >
> > sub callback {
> > croak unless @_ == 2;
> > (my $self, $foo) = @_;
> > ...
> > }
> >
> > How would the caller (say an event handler module), which has been upgraded
> > to pass an optional next arg, know that the callback sub it wanted to call
> > would croak on that extra arg?
> >
>
> The difference is a practical one: no one would pass a callback like your
> example because it's asking for breakage and adds nothing, but most would
> use a signature like ($self, $foo) in a callback documented to accept those
> arguments.


I'm assuming (and I think that I'm correct in assuming) that most folks
writing code will see this:

sub foo {
my ($bar, $baz) = @_;
...
}

and think that they can rewrite it as

sub foo ($bar, $baz) {
...
}


when (of course) the latter approach creates the strict arity checking.
*And* for nearly cases, the strict arity checking is the right default.


So this combined with how callbacks typically evolve is creating a trap.
And this isn't a "with hindsight, we ..." trap. It's foreseeable.


On Fri, Jan 21, 2022 at 04:26:18PM +0000, Dave Mitchell wrote:

> I think in the past that strong opinions have been expressed against
> allowing a signature API which a caller can use to inspect a sub, on the
> grounds that really the sub signature is an implementation detail - just a
> fancy and fast way of writing a @_ unpacker, and in general, doesn't form
> a promised API.

I don't remember *this* specifically. (I might have missed it)

I *do* remember a desire to inspect signatures as strings, and even treat
them roughly as such, and pushback that signatures are *code*, not flat
strings, and so can't be handed round as strings. But the objection I
remember (and agreed with) was the "strings", not fundamentally the
introspection.


I think it important to say that, as signatures can contain arbitrary code
(and I think it important that they can, for enabling setting defaults
without trying to instead create a Domain Specific Language that morphs into
a *different* Turing complete programming language) then they can't be 100%
introspectable. And then that obviously means that there's going to be some
level of discussion on where are the "rigidly defined areas of doubt an
uncertainty". In that:

On Sat, Jan 22, 2022 at 02:20:21AM +0800, Tom Molesworth via perl5-porters wrote:

> I ended up writing
> https://metacpan.org/dist/Acme-Signature-Arity/view/lib/Acme/Signature/Arity.pod
> as a proof-of-concept API, but would hope that core Perl will expose some
> sort of arity inspection in due course. Ignoring this library-extensibility
> use-case while simultaneously promoting signatures as a complete
> replacement for @_-based subs seems less than ideal to me.


Aside - I didn't think that we *were* promoting signatures as a complete
replacement for @_-based subs. I always thought of signatures as the "easy
things easy" part, where @_ remains for "hard things possible". In that, I'm
predicting that the syntax of signatures will get too complex to be easily
readable, if it's extended to cover all rarely used things that one can do
with @_.



Thanks for the module. This is useful because you chose a different line for
"doubt" than I was thinking, and that's useful. In that, your API can
distinguish between these two:

sub foo($bar, @args) { my ($baz) = @args; say "$bar: $baz" }

sub foo(@args) { my ($bar, $baz) = @args; say "$bar: $baz" }


and these two:

sub foo($bar, $baz = "world", ,@) { say "$bar: $baz" }

sub foo($bar, @args) { my $baz = shift @args // "world"; say "$bar: $baz" }


whereas I thought that I'd view these choices as "implementation details",
and so I'd limit things to roughly:

1) min count of positional arguments
2) max count of positional arguments (might be infinite)

and not expose

1) where the named arguments stop
2) whether "infinite" is a named array, or just discarded


but then...

I realised that introspecting "does it end with a slurpy hash" gets sort of
interesting too, because that means that not all large parameter counts are
valid. The simple and wrong answer would be `$m + 2 * $n`. But optional
positional parameters followed by a slurpy hash...

./perl -Ilib -wE 'use feature "signatures"; sub foo($bar, $baz = "world", ,%) { say "$bar: $baz" } foo("Mandatory", "Optional", "Whoops!")'
The signatures feature is experimental at -e line 1.
Odd name/value argument for subroutine 'main::foo' at -e line 1


So

1) I don't know exactly what is the published interface, and what is
implementation
2) And this is only positional parameters. It's not future-proofed for also
describing named parameters (mandatory and optional. eg does one even
need to "announce" the names of optional named parameters? Is that
implementation details? Do I want "your" code to object to my callback
because I have an optional named parameter you don't use, but you wrote
your validator to reject any sub reference with non-zero optional named
params?)

On Fri, Jan 21, 2022 at 03:15:31PM +0100, demerphq wrote:
> On Fri, 21 Jan 2022 at 14:12, Nicholas Clark <nick@ccl4.org> wrote:
>
> > Really you want to know you have a problem *before* you call the callback
> > (ie what Yves wrote - you want to be able to flag a problem at object
> > construction time, which might well be program startup. Not fail at
> > runtime,
> > unattended, into a log file or even /dev/null)
> >
>
> So am i correct in understanding that you would be ok with exposing an API
> to check the subs arity signature? If so then I'll try to think up
> one (others could too).

Yes, I'd be OK with it. But as above

1) I don't know what it should expose
2) and I don't think I (alone) get to say "we" want this
3) and I really can't promise to be around in a timely fashion to resolve
design questions or review anything (which makes me feel bad about saying
"yes", because I feel that "yes" implies "and I have time to stay involved")

On Fri, Jan 21, 2022 at 06:28:19PM +0000, Paul "LeoNerd" Evans wrote:

> What would be lovely would be if we could encode a position in a
> function call site, to say "from this point onwards, don't worry if the
> invoked sub does not handle this argument. They are optional". I wonder
> about allowing the keyword `optional` in the middle of a function or
> method call argument list:
>
> $self->$callback($item, optional $idx);
>
> A function/method call with one of these 'optional' words inside it is
> then handled slightly differently. If the invoked sub fails to handle
> at least all of the arguments before the 'optional' then that's fatal
> as it is now, but the caller is saying "I don't mind if the callee
> ignores any of the values after here". Thus $callback could be defined
> by any of

It bugs me that implementing this efficiently would be a right pain.

Currently a signature sub looks like this:

$ ./perl -Ilib -MO=Concise,foo,-exec -wE 'use feature "signatures"; sub foo($bar, $baz = "world", ,%) { say "$bar: $baz" }'
The signatures feature is experimental at -e line 1.
main::foo:
1 <;> nextstate(main 6 -e:1) v:%,us,fea=15
2 <+> argcheck(2,1,%) v
3 <;> nextstate(main 4 -e:1) v:%,us,fea=15
4 <+> argelem(0)[$bar:4,6] v/SV
5 <;> nextstate(main 5 -e:1) v:%,us,fea=15
6 <|> argdefelem(other->7)[1] sK
7 <$> const(PV "world") s
8 <+> argelem(1)[$baz:5,6] vKS/SV
9 <;> nextstate(main 6 -e:1) v:%,us,fea=15
a <0> padrange[$bar:4,6; $baz:5,6] /range=2
b <+> multiconcat(": ",-1,2,-1)[t5] sK/STRINGIFY
c <@> say sK
d <1> leavesub[1 ref] K/REFC,1
-e syntax OK

Dave suggests that for common cases all of that argument setup can be reduced
to a single op (with data for a finite state machine). Maybe not with my
default value shown. But generally, I guess we get to this:

1 <;> setup[arm waving]
2 <;> nextstate(main 6 -e:1) v:%,us,fea=15
3 <0> padrange[$bar:4,6; $baz:5,6] /range=2
4 <+> multiconcat(": ",-1,2,-1)[t5] sK/STRINGIFY
5 <@> say sK
6 <1> leavesub[1 ref] K/REFC,1
-e syntax OK


as the optree of the subroutine. And nothing in pp_entersub *or* the optrees
of the calling code needs to know the difference between the two, and always
passes flat arguments just like perl 5.000 to blead.

Hence the implementation is free to swap to the latter whenever it needs to.
But more importantly, that "setup" OP only takes the list of arguments. It
doesn't care what the *callee* is doing, so the only logic it needs to
encode is the (known at compile time)(single) logic of the signature author.
vary


Whereas if the *callee* is also allowed to influence what is optional, then

1) a side channel has to exist for the *callee* to pass that onward too
2) the signature processing code must handle it (more branches


Yes, one *could* them optimise this by having two optree entry points for
each CV - one with no callee extra info (fast path) and one for with,
both of which have ->op_next land at the first (real) nextstate op of the
body, but this is (also) getting more complex.

> Now I'm not hugely fixed on this particular syntax as a solution to the
> problem, but I feel that the principle of "caller specifies API" is one
> we should explore.

Agree.

I like the idea of a (limited) signature introspection API better as a
solution to the problem.

Nicholas Clark
Re: arity checking (was Re: PSC #049 2022-01-07) [ In reply to ]
I've just encountered another fine use-case for wanting to know the
min/max arity count of a sub before calling it.

I'm writing an nroff parser... I'll spare you the full details but in
brief I have a plaintext input that consists of a short two-letter
"directive" code, followed by some positional arguments:

.TH NAME 3 "date here"
.SH HEADING
.SS Subheading

etc...

My code to parse/dispatch these basically takes the two-letter
directive code and the args in an array, and does:

my $method = $self->can( "handle_directive_$d" ) or
$self->die( "Unrecognised directive .$d" );
return $self->$method( @args );

I can then go off and implement

sub handle_directive_TH {
my $self = shift;
my ( $title, $section, $date ) = @_;
...
}

# and so on for handle_directive_SH, handle_directive_SS and many
# many others

Without sub signatures I get no min/max arity checking of these things.
I could hack that in manually via some lookup table or whatever, but
it'd be great if I can just use the sub signatures themselves:

# date is optional
sub handle_directive_TH ( $self, $title, $section, $date = undef )
{
...

Problem is if I do that, any arity failures appear to come from
*perl* land in the dispatch logic in the toplevel. My process dies with

Too few arguments for subroutine 'main::handle_directive_TH' at
Some/Perl/File.pm line 1234.

That's not very friendly to my end-users. I want them to see a failure
as generated by my parser logic; something like:

Too few arguments for .TH directive on line 1 at:
.TH TITLE
^

If perl provided me a way to query the min/max arity of a coderef
without invoking it, well this is easy enough to write:

my $method = $self->can( "handle_directive_$d" ) or
$self->die( "Unrecognised directive .$d" );

my ( $minargs, $maxargs ) = builtin::sub_arity $method;
$self->die( "Too few arguments for .$d directive" ) if 1+@args < $minargs;
$self->die( "Too many arguments for .$d directive" ) if 1+@args > $minargs;

return $self->$method( @args );

Perl currently does not provide this. So I can't.

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/
Re: arity checking (was Re: PSC #049 2022-01-07) [ In reply to ]
2022-2-15 2:12 Paul "LeoNerd" Evans <leonerd@leonerd.org.uk> wrote:

>
> my ( $minargs, $maxargs ) = builtin::sub_arity $method;
> $self->die( "Too few arguments for .$d directive" ) if 1+@args <
> $minargs;
> $self->die( "Too many arguments for .$d directive" ) if 1+@args >
> $minargs;
>
> return $self->$method( @args );
>
> Perl currently does not provide this. So I can't.
>
>
If signatures enable arity checking, there is a strong need for the API to
get the minimum and maximum number of arguments.

This is because generally speaking, module authors know the details of Perl
language, but module users don't know the details.
Re: arity checking (was Re: PSC #049 2022-01-07) [ In reply to ]
2022-2-15 10:50 Yuki Kimoto <kimoto.yuki@gmail.com> wrote:

>
> 2022-2-15 2:12 Paul "LeoNerd" Evans <leonerd@leonerd.org.uk> wrote:
>
>>
>> my ( $minargs, $maxargs ) = builtin::sub_arity $method;
>> $self->die( "Too few arguments for .$d directive" ) if 1+@args <
>> $minargs;
>> $self->die( "Too many arguments for .$d directive" ) if 1+@args >
>> $minargs;
>>
>> return $self->$method( @args );
>>
>> Perl currently does not provide this. So I can't.
>>
>>
> If signatures enable arity checking, there is a strong need for the API
> to get the minimum and maximum number of arguments.
>
> This is because generally speaking, module authors know the details of
> Perl language, but module users don't know the details.
>
>
Additional Description:

In this way, @_ can be suppressed because the definition of a signature is
determined at compile time;

Callbacks can receive the correct arguments.

$cb->(splice @args, 0, $maxargs);
Re: arity checking (was Re: PSC #049 2022-01-07) [ In reply to ]
2022-2-15 11:07 Yuki Kimoto <kimoto.yuki@gmail.com> wrote:

>
> 2022-2-15 10:50 Yuki Kimoto <kimoto.yuki@gmail.com> wrote:
>
>>
>> 2022-2-15 2:12 Paul "LeoNerd" Evans <leonerd@leonerd.org.uk> wrote:
>>
>>>
>>> my ( $minargs, $maxargs ) = builtin::sub_arity $method;
>>> $self->die( "Too few arguments for .$d directive" ) if 1+@args <
>>> $minargs;
>>> $self->die( "Too many arguments for .$d directive" ) if 1+@args >
>>> $minargs;
>>>
>>> return $self->$method( @args );
>>>
>>> Perl currently does not provide this. So I can't.
>>>
>>>
>> If signatures enable arity checking, there is a strong need for the API
>> to get the minimum and maximum number of arguments.
>>
>> This is because generally speaking, module authors know the details of
>> Perl language, but module users don't know the details.
>>
>>
> Additional Description:
>
> In this way, @_ can be suppressed because the definition of a signature is
> determined at compile time;
>
> Callbacks can receive the correct arguments.
>
> $cb->(splice @args, 0, $maxargs);
>
>
>
Additional Description 2:

If arity checking is enabled, and it is included to v5.36

# End user(test.pl)
use v5.36;

use Foo;

Foo->execute(sub ($foo) {
# ...
});

# The module author (This module supports Perl 5.10)
package Foo;

sub execute {
my ($cb) = @_;

$cb->(1);
}

The module author adds an argument after a year.

package Foo;

sub execute {
my ($cb) = @_;

# Adds an argument
$cb->(1, 2);
}

The "test.pl" crashes by the message "Too many arguments". The "test.pl"
maybe a production code.

In order that the module author pass arguments safely, dual life module "
buildin.pm" and the function to pass arguments safely.

# Pass arguments safely
package Foo;

use builtin 'call_cb_safe';

sub execute {
my ($cb) = @_;

# Pass arguments safely
call_cb_safe($cb, (1, 2));
}

The "call_cb_safe" passes the safe arguments to the callback by seeing the
definition of the signatures of the callback.

If the callback doesn't have signatures, all arguments are passed.
Re: arity checking (was Re: PSC #049 2022-01-07) [ In reply to ]
On Thu, 17 Feb 2022 at 11:19, Yuki Kimoto <kimoto.yuki@gmail.com> wrote:

> In order that the module author pass arguments safely, dual life module "
> buildin.pm" and the function to pass arguments safely.
>
> # Pass arguments safely
> package Foo;
>
> use builtin 'call_cb_safe';
>
> sub execute {
> my ($cb) = @_;
>
> # Pass arguments safely
> call_cb_safe($cb, (1, 2));
> }
>

Yes, but this has been covered several times on the list recently.

See https://metacpan.org/pod/Acme::Signature::Arity#coderef_ignoring_extra
for this specific case - I think your example would look like this:

package Foo;

use Acme::Signature::Arity qw(coderef_ignoring_extra);

sub execute {
my ($cb) = @_;

# Pass arguments safely
coderef_ignoring_extra($cb)->(1, 2);
}

(or use `min_arity`/`max_arity` to apply different rules)
Re: arity checking (was Re: PSC #049 2022-01-07) [ In reply to ]
2022-2-17 12:31 Tom Molesworth <tom@deriv.com> wrote:

> On Thu, 17 Feb 2022 at 11:19, Yuki Kimoto <kimoto.yuki@gmail.com> wrote:
>
>> In order that the module author pass arguments safely, dual life module "
>> buildin.pm" and the function to pass arguments safely.
>>
>> # Pass arguments safely
>> package Foo;
>>
>> use builtin 'call_cb_safe';
>>
>> sub execute {
>> my ($cb) = @_;
>>
>> # Pass arguments safely
>> call_cb_safe($cb, (1, 2));
>> }
>>
>
> Yes, but this has been covered several times on the list recently.
>
> See https://metacpan.org/pod/Acme::Signature::Arity#coderef_ignoring_extra
> for this specific case - I think your example would look like this:
>
> package Foo;
>
> use Acme::Signature::Arity qw(coderef_ignoring_extra);
>
> sub execute {
> my ($cb) = @_;
>
> # Pass arguments safely
> coderef_ignoring_extra($cb)->(1, 2);
> }
>
> (or use `min_arity`/`max_arity` to apply different rules)
>
>
Tom

Oh, You have already mentioned it.

I understand how to use coderef_ignoring_extra now.