Mailing List Archive

Refaliasing and closures [was: PSC #093: 2023-01-13]
On Fri, 13 Jan 2023 11:57:11 -0500
"Ricardo Signes" <perl.p5p@rjbs.manxome.org> wrote:

> We talked about the refaliasing
> feature, which remains broken on closures, a total showstopper.

An example of the breakage:

$ perl -E 'use feature qw(refaliasing);
my @arr;
sub boo { print "boo sees: @arr\n" }
\@arr = [1,2,3];
boo();
print "main sees: @arr\n";'

Aliasing via reference is experimental at -e line 1.
boo sees:
main sees: 1 2 3

This means that the main code vs. the function saw different things as
the actual @arr array. There's not a lot we can do about this. It
sortof comes down to the reasons why you can't `local` a `my` variable.
You shouldn't be able to do this here either but it was allowed. Oops.

> Paul
> wonders if we can reduce the scope of how the feature can be used.
> For example: what if you *must* also use declared_refs?

Whereas hypothetically if you do this it could be handled better:

my \@arr = [1, 2, 3];
sub boo { print "boo sees: @arr\n" }
...

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/
Re: Refaliasing and closures [was: PSC #093: 2023-01-13] [ In reply to ]
Hi there,

On Mon, 16 Jan 2023, Paul "LeoNerd" Evans wrote:

> ... hypothetically if you do this it could be handled better:
>
> my \@arr = [1, 2, 3];
> sub boo { print "boo sees: @arr\n" }
> ...

The code above looks confusing and potentially fragile to me. If the
'refalias' feature is what would provide it, then I don't want it.

I've looked at

https://github.com/perl/perl5/issues/14150

and if there's evidence of any general desire to go in this direction
it's eluded me. Is there such evidence? Where?

In characteristically plodding style I'd much prefer to write something
which makes the intent both obvious and unambiguous:

$ perl -e 'my $aref=[1,2]; sub boo {print "array=[@{$aref}]\n";} print "running\n"; boo;'
running boo
array=1 2
$

My take on this is that

(a) I've never felt the need for this thing and

(b) it's a decade since its inception and it still doesn't work

so it serves our interests best to abandon the experiment.

--

73,
Ged.
Re: Refaliasing and closures [was: PSC #093: 2023-01-13] [ In reply to ]
On 1/16/23 09:04, G.W. Haywood via perl5-porters wrote:
> Hi there,
>
> On Mon, 16 Jan 2023, Paul "LeoNerd" Evans wrote:
>
>> ... hypothetically if you do this it could be handled better:
>>
>>    my \@arr = [1, 2, 3];
>>    sub boo { print "boo sees: @arr\n" }
>>    ...
>
> The code above looks confusing and potentially fragile to me.  If the
> 'refalias' feature is what would provide it, then I don't want it.
>
> I've looked at
>
> https://github.com/perl/perl5/issues/14150
>
> and if there's evidence of any general desire to go in this direction
> it's eluded me.  Is there such evidence?  Where?
>
> In characteristically plodding style I'd much prefer to write something
> which makes the intent both obvious and unambiguous:
>
> $ perl -e 'my $aref=[1,2]; sub boo {print "array=[@{$aref}]\n";} print
> "running\n"; boo;'
> running boo
> array=1 2
> $
>
> My take on this is that
>
> (a) I've never felt the need for this thing and
>
> (b) it's a decade since its inception and it still doesn't work
>
> so it serves our interests best to abandon the experiment.
>

I’m in the other camp.

I find that sample code clear; I use ref aliases quite a bit to simplify
expressions, and the lack of support in closures introduces a serious
inconsistency in the closure model. The closure shouldn’t care if @array
is a ref alias or a real array.
Re: Refaliasing and closures [was: PSC #093: 2023-01-13] [ In reply to ]
On 2023-01-16 15:04, G.W. Haywood via perl5-porters wrote:
> Hi there,
>
> On Mon, 16 Jan 2023, Paul "LeoNerd" Evans wrote:
>
>> ... hypothetically if you do this it could be handled better:
>>
>>    my \@arr = [1, 2, 3];
>>    sub boo { print "boo sees: @arr\n" }
>>    ...
>
> The code above looks confusing and potentially fragile to me.  If the
> 'refalias' feature is what would provide it, then I don't want it.
>
> I've looked at
>
> https://github.com/perl/perl5/issues/14150
>
> and if there's evidence of any general desire to go in this direction
> it's eluded me.  Is there such evidence?  Where?
>
> In characteristically plodding style I'd much prefer to write something
> which makes the intent both obvious and unambiguous:
>
> $ perl -e 'my $aref=[1,2]; sub boo {print "array=[@{$aref}]\n";} print
> "running\n"; boo;'
> running boo
> array=1 2
> $
>
> My take on this is that
>
> (a) I've never felt the need for this thing and
>
> (b) it's a decade since its inception and it still doesn't work
>
> so it serves our interests best to abandon the experiment.
>

I'd like to keep it.

I prefer non-scalars for non-scalar data structures,
as a data-type dimension/restriction.

-- Ruud
Re: Refaliasing and closures [was: PSC #093: 2023-01-13] [ In reply to ]
On Tue, 17 Jan 2023 at 22:08, Diab Jerius <dj@avoiding.work> wrote:
>
>
> On 1/16/23 09:04, G.W. Haywood via perl5-porters wrote:
> > Hi there,
> >
> > On Mon, 16 Jan 2023, Paul "LeoNerd" Evans wrote:
> >
> >> ... hypothetically if you do this it could be handled better:
> >>
> >> my \@arr = [1, 2, 3];
> >> sub boo { print "boo sees: @arr\n" }
> >> ...
> >
> > The code above looks confusing and potentially fragile to me. If the
> > 'refalias' feature is what would provide it, then I don't want it.
> >
> > I've looked at
> >
> > https://github.com/perl/perl5/issues/14150
> >
> > and if there's evidence of any general desire to go in this direction
> > it's eluded me. Is there such evidence? Where?
> >
> > In characteristically plodding style I'd much prefer to write something
> > which makes the intent both obvious and unambiguous:
> >
> > $ perl -e 'my $aref=[1,2]; sub boo {print "array=[@{$aref}]\n";} print
> > "running\n"; boo;'
> > running boo
> > array=1 2
> > $
> >
> > My take on this is that
> >
> > (a) I've never felt the need for this thing and
> >
> > (b) it's a decade since its inception and it still doesn't work
> >
> > so it serves our interests best to abandon the experiment.
> >
>
> I’m in the other camp.
>
> I find that sample code clear; I use ref aliases quite a bit to simplify
> expressions, and the lack of support in closures introduces a serious
> inconsistency in the closure model. The closure shouldn’t care if @array
> is a ref alias or a real array.

I agree. It is interesting that it does, it suggests that some kind of
optimization is kicking in which means the code isnt indirecting on
the pad slot that represents @array, and instead is binding to it
directly. So when you alias the ref the closure remains pointing at
the previous AV.

--
perl -Mre=debug -e "/just|another|perl|hacker/"
Re: Refaliasing and closures [was: PSC #093: 2023-01-13] [ In reply to ]
On Wed, Jan 18, 2023 at 05:59:09PM +0100, demerphq wrote:
> I agree. It is interesting that it does, it suggests that some kind of
> optimization is kicking in which means the code isnt indirecting on
> the pad slot that represents @array, and instead is binding to it
> directly. So when you alias the ref the closure remains pointing at
> the previous AV.

Each closure has its own pad, with one of the pad's slots holding a
pointer to the shared (closed-over) SV. If the 'main' lexical is then
aliased, i.e. PL_curpad[i] in the current pad is temporarily made to
point to something else, then the PL_curpad[i]'s of all the closures which
have closed over the same variable don't get automatically updated too.
And I don't see an obvious or efficient way of making it happen.

This flaw isn't unique to refaliasing; it's always been present in other
parts of perl which do aliasing, such as for():

my $x = "X";
my $y = "Y";

my $s = sub { print "y=$y in closure\n" };

$s->();
for $y ($x) {
print "y=$y in loop\n";
$s->();
}

which outputs:

y=Y in closure
y=X in loop
y=Y in closure # many people would expect y=X

Similar things apply with

local *y = \$x;

So this is a general "feature" of perl's closures and aliasing, and
not a specific problem with refaliasing.

I think even without closure support, refaliasing is a useful feature:

sub f {
my ($ary_ref, ...) = @_;
my \@a = $ary_ref;
$a[0] =....;
}

and with my proposed signature aliasing extension which does the same
thing but cleaner:

sub f (\@a, ...) {
$a[0] = ...;
}

there would still be the same issue with any closures.

So personally I think refaliasing should become non-experimental, but with
a note that its behaviour with closures is undefined and liable to change.


--
"Foul and greedy Dwarf - you have eaten the last candle."
-- "Hordes of the Things", BBC Radio.
Re: Refaliasing and closures [was: PSC #093: 2023-01-13] [ In reply to ]
On Tue, 24 Jan 2023 17:57:21 +0000
Dave Mitchell <davem@iabyn.com> wrote:

> Each closure has its own pad, with one of the pad's slots holding a
> pointer to the shared (closed-over) SV. If the 'main' lexical is then
> aliased, i.e. PL_curpad[i] in the current pad is temporarily made to
> point to something else, then the PL_curpad[i]'s of all the closures
> which have closed over the same variable don't get automatically
> updated too. And I don't see an obvious or efficient way of making it
> happen.

Yup, that.

This is the reason why `local` was never made to work with normal
lexical variables, because it has the exact same problem with closures.

> So personally I think refaliasing should become non-experimental, but
> with a note that its behaviour with closures is undefined and liable
> to change.

Yeah, this might be a way forward.

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/
Re: Refaliasing and closures [was: PSC #093: 2023-01-13] [ In reply to ]
On Tue, Jan 24, 2023, at 13:14, Paul "LeoNerd" Evans wrote:
> [davem wrote]:
> > So personally I think refaliasing should become non-experimental, but
> > with a note that its behaviour with closures is undefined and liable
> > to change.
>
> Yeah, this might be a way forward.

Can we give it something like "enclosed alias will not stay shared" warning?

--
rjbs
Re: Refaliasing and closures [was: PSC #093: 2023-01-13] [ In reply to ]
On Tue, Jan 24, 2023 at 10:40:08PM -0500, Ricardo Signes wrote:
> Can we give it something like "enclosed alias will not stay shared" warning?

Not easily. The information isn't readily available at either compile time
nor run time. But it might be possible to add more compile-time
information (flags or whatever) to make such a warning possible.

But we would need to agree on what the 'bad' thing is which should
actually trigger the warning. For compile time, is it for when the
compiler has both seen that a lexical variable has been marked as
can-be-closed-over, *and* is used in a lexical alias situation? E.g. in
the following, the compiler issues a single compile-time warning at the
indicated places:

my ($x, $y);
my $s = sub { $y };
\$y = \$x; # $y may not stay shared during aliasing

my ($x, $y);
\$y = \$x;
my $s = sub {
$y; # $y may not stay shared during aliasing
};

Or perhaps it warns when finishing up compiling each sub or file scope,
doing a scan of the newly-compiled pad looking, for each lexical, for the
presence of multiple flags ("used in closure"; "used in alias", etc).

Or should it only warn at run-time, if and when an anonymous sub has
actually been created *and* the variable aliased? So this wouldn't warn:

if (cond) {
my $s = sub { $y };
...
}
else {
\$y = \$x;
...
}`

Should it also warn on other types of aliasing? E.g.

my ($x, $y);
my $s = sub { $y };
for $y ($x) { ... } # $y may not stay shared during aliasing

But that would warn on pre-existing and possibly non-harmful code.

--
Technology is dominated by two types of people: those who understand what
they do not manage, and those who manage what they do not understand.
Re: Refaliasing and closures [was: PSC #093: 2023-01-13] [ In reply to ]
A crazy solution: lexical globs. The declaration "my *foo" would make
$foo, @foo, %foo, and &foo refer to the appropriate slots in the lexical
glob. Refaliasing or localising these would then be visible both inside
and outside of any closures.

/Bo Lindbergh