Mailing List Archive

Still wading through all the messages that appeared over the weekend,
but this is the most subtly amusing thing I've read today (actually for
some days):

On Sun, Jun 20, 2021 at 01:22:16PM +0100, Paul "LeoNerd" Evans wrote:

> As to implementation, I have no idea how to create this. `defer {}`'s
> current implementation is just to use SAVEDESTRUCTOR_X() which pushes
> to the same savestack as things like variable scope clearing and
> `local` uses. That's strictly a stack, and you can't splice into it
> lower down. There have variously been times I have wanted such
> an ability, but equally other times (most notably while implementing
> async/await) when I have been thankful for its absence.

"With this hat on I want feature X"
"With that hat on I hate my other self"


Most of the Perl internals is like this. It's a love-hate relationship with
yourself, depending on whether you're authoring code for CPAN or
(re)implementing core functionality.

Nicholas Clark

On Mon, 21 Jun 2021 at 11:16, Eirik Berg Hanssen <
Eirik-Berg.Hanssen@allverden.no> wrote:

> On Mon, Jun 21, 2021 at 3:37 AM Ben Bullock <benkasminbullock@gmail.com>
> wrote:
>
>> I went back and checked, and I've actually said this in all of the
>> messages I've sent on this thread, and so far nobody has pointed out the
>> mistake in my understanding. Is there some way of sneaking out of if{}
>> without arriving at the end of the code?
>>
>
> Yes? Plenty of ways: An exception may sneak out (all the way to the
> first dynamically enclosing eval or similar); a return may sneak out (all
> the way out of the subroutine call); a next/last/redo may sneak out (all
> the way out of the first dynamically enclosing loop with matching label, if
> given); a goto LABEL may sneak out to some arbitrary LABEL.
>

No, that's not my point. If you have something like this

while (1) {
if (something) {
suddenlyexitif;
print "Did not execute.\n";
}
print "Did execute\n";
}

you have to come up with something at suddenlyexitif which leaves the if
block which does not execute "print "Did not execute.\n"" but does execute
"print "Did execute\n"", without adding labels.

On Mon, 21 Jun 2021 at 11:08, Tom Molesworth <tom@binary.com> wrote:

> If you want something to execute in the parent scope, that can be done by
> putting the defer above the if() statement. How would the following be
> implemented with your suggested change in behaviour?
>
> if(condition) {
> open my $fh, '>', 'output' or die;
> defer { $fh->close and send_email_from_file('output') }
> ... arbitrary code which may include next/return/die/goto ...
> }
> open my $fh, '>', 'output' or die;
>

I concede that you would be better off with the current implementation
rather than my suggestion if that is what you want to do with defer.

2021-6-21 9:24 Tom Molesworth <tom@deriv.com> wrote:

> On Mon, 21 Jun 2021 at 07:55, Yuki Kimoto <kimoto.yuki@gmail.com> wrote:
>
>> How much does defer block affect current performance of Perl?
>>
>
> The module is available on CPAN:
>
> https://metacpan.org/pod/Syntax::Keyword::Defer
>
> so this would be the best place to start if there are specific benchmarks
> you would like to try.
>
> Addition of the keyword should have no measurable impact on existing code.
>
> Performance is generally faster than the equivalent
> create-instance-and-hook-DESTROY functionality provided by modules such as
> Scope::Guard.
>
>
I want to ask a question about real logical cost.

For example,

if (UNLIKELY(scope_has_deffer_blocks)) {

}

This may be 1 operation if the if block is not executed.

On Sun, 20 Jun 2021 at 14:23, Paul "LeoNerd" Evans <leonerd@leonerd.org.uk>
wrote:

> sub wibble {
> uplevel::defer { say "This runs after Hello" }
> }
>
> {
> wibble();
> say "Hello";
> }
>
>
>
would in be better to use behaviour already used with redo/next/last ?

sub foo {
LABEL: {
if ($cond) { defer LABEL { ... } }
}
}

how about some kind of way to inject coderefs into the call stack, perhaps
by using C<caller(n)> as an l-value?

sub wibble{ caller(0) = sub { say "This runs after Hello" } }

alternately

sub wibble{ push caller(0), sub { say "This runs after Hello" } }


> sub wibble {
>> uplevel::defer { say "This runs after Hello" }
>> }
>>
>> {
>> wibble();
>> say "Hello";
>> }
>>
>
--
"Lay off that whiskey, and let that cocaine be!" -- Johnny Cash

On Sun, Jun 20, 2021 at 8:23 AM Paul "LeoNerd" Evans <leonerd@leonerd.org.uk>
wrote:

> On Sat, 19 Jun 2021 22:41:24 -0400
> "Ricardo Signes" <perl.p5p@rjbs.manxome.org> wrote:
>
> > But when unitcheck was introduced, I remember thinking clearly and
> > repeatedly that I'd have used it far often if it had been possible to
> > inject the unitcheck upward one scope. Honestly, I'd have to do a
> > fair bit more thinking about it to remember the use cases I had in
> > mind. But you suggest (I think *very* hopefully) that we could
> > eventually get away from timely destruction if we had better defer
> > blocks. I think often we'd like defer blocks to be produced by
> > reusable code, which does hint toward the idea of "defer this until
> > my calling scope is complete."
> >
> > Do you think this is an area in which we should consider nudging the
> > capabilities of this proposal?
>
> An exciting question. There's two parts to the answer:
>
> On the question of how to spell the syntax, I could imagine if we're
> going into the realm of core-provided namespaced functions and keywords
> (e.g. previous discussions on `string::trim` and `ref::type`) I could
> imagine an uplevel:: space for doing that kind of thing:
>
> sub wibble {
> uplevel::defer { say "This runs after Hello" }
> }
>
> {
> wibble();
> say "Hello";
> }
>
> As to implementation, I have no idea how to create this. `defer {}`'s
> current implementation is just to use SAVEDESTRUCTOR_X() which pushes
> to the same savestack as things like variable scope clearing and
> `local` uses. That's strictly a stack, and you can't splice into it
> lower down. There have variously been times I have wanted such
> an ability, but equally other times (most notably while implementing
> async/await) when I have been thankful for its absence.
>
> I don't think I want to add this to the main body of the current
> proposal, but I can at least add a note in "future direction" to
> suggest that such an extension might become useful.
>

I concur with this, and will add that such a concept would also be
incredibly useful for the `local` operator, so maybe both utilities could
be enhanced the same way.

-Dan

>
>>
> I concur with this, and will add that such a concept would also be
> incredibly useful for the `local` operator, so maybe both utilities could
> be enhanced the same way.
>
> -Dan
>

I hate to play the killjoy, but a stack-hopping defer would represent both
a massive opportunity for exploitation and an explosion of spooky action at
a distance bugs.

--
"Lay off that whiskey, and let that cocaine be!" -- Johnny Cash

On Tue, Jun 22, 2021 at 9:34 PM David Nicol <davidnicol@gmail.com> wrote:

>
>
>>>
>> I concur with this, and will add that such a concept would also be
>> incredibly useful for the `local` operator, so maybe both utilities could
>> be enhanced the same way.
>>
>> -Dan
>>
>
> I hate to play the killjoy, but a stack-hopping defer would represent both
> a massive opportunity for exploitation and an explosion of spooky action at
> a distance bugs.
>

Something to reason out ("not reasonably viable" is a possible conclusion)
when such functionality is proposed, which as noted should definitely not
be done as part of this current effort.

-Dan

On Tue, 22 Jun 2021 at 20:16, David Nicol <davidnicol@gmail.com> wrote:

>
> how about some kind of way to inject coderefs into the call stack, perhaps
> by using C<caller(n)> as an l-value?
>
> sub wibble{ caller(0) = sub { say "This runs after Hello" } }
>
>
>
This syntax is limited to callers only.

From discussion here - it's clear to me that many of you think in the terms
of context oriented programming (without realizing it ...)

Looks like there are two kind of defer usage:
- code layout oriented
- data scope oriented

Although defer may be legit observing code layout.majority of usage may be
related to data frame cleanup (aka: defer (whenever $dbh is defined) { })

Speaking about loop exit operators: how it behaves in with dump ?

This still covers only "programmer-perfect" code, as implemented - doesn't
cover tests, reuse, where whole change one may need
is attach defer to addressable context from outside (see "define meta
operator")

On Wed, Jun 16, 2021 at 11:15:40PM +0100, Paul "LeoNerd" Evans wrote:
> Here's an RFC formalization of my currently-open github feature issue:

Thanks.

> https://github.com/Perl/perl5/issues/17949
>
> which was itself an issue tracking the previous mailing list
> conversation:
>
> https://www.nntp.perl.org/group/perl.perl5.porters/2020/06/msg257611.html
>
> If we accept this one into the RFC process, I'll likely close the
> github issue and point at the RFC instead.
>
> ---
>
> # defer {} syntax
>
> ## Pre?mble

The intent was that this section is machine parseable.

Technically this is the new PSC's call now, but

I know that you like diaereses, and I get that this spelling makes it clear
that it's two vowel sounds, but

* I don't mind what the section header is
* I didn't want it to have an ambiguous name
* I didn't want to unify on a name that isn't the common spelling

I can't find anywhere that suggests "pre?mble" as an alternative spelling.
eg

https://dictionary.cambridge.org/dictionary/english/preamble

(interestingly they have coordinate as "(also mainly UK co-ordinate)":
https://dictionary.cambridge.org/dictionary/english/coordinate

Even newyorker.com is using preamble, and, well, they send themselves up
about diaereses:

https://www.newyorker.com/culture/culture-desk/the-curse-of-the-diaeresis

So can we pick a different header name for this section that has one
unambiguous spelling, that is all ASCII? "Metadata" would do.

> Author: Paul Evans <PEVANS>
> Sponsor:
> ID: 0004

Technically according to the plan an RFC doesn't get an official ID until it
is accepted into the process. Right now things are slow enough that it
didn't matter.

> Status: Draft

I realised that everyone keeps talking about "Draft" RFCs when they are
drafting an RFC (well, d'oh) whereas the intent was that the is a clear
status that was "not yet accepted". So...

I swapped the names for the status of the first step, and for "not yet
accepted. "Exploratory" is now the first step in the repository, and "Draft"
is what comes before. So that "Status" line is retrospectively correct.

The status is now also "Provisional" because the outgoing PSC were pretty
confident "we think that this idea is worth implementing"


I've pushed a large update to the RFC repository that brings your RFC in.

Nicholas Clark

On Wed, Jun 16, 2021 at 11:15:40PM +0100, Paul "LeoNerd" Evans wrote:

> ## Motivation


> It would be nice to offer a native core syntax for this common
> behaviour.


: ((TODO - I admit I'm not very clear on how to split my wording between
: the Motivation section, and this one))

It seems much easier in theory that in practice. I *think* that it goes
roughly here (splitting your paragraph). Above this point is "problem I want
to solve" and below here is "my proposed solution".

Except that the next sentence flows nicely as part of the reasoning of *why*
so having the next line read "A simple explicit syntax for this" instead
would make a cleaner separation.


> A simple `defer { BLOCK }` syntax removes from the user any
> requirement to think about storing the guard object in a lexical
> variable, or worry about making sure it really does get released at the
> right time.


I don't think it really matters if you change the wording you had. But
take the TODO out - on balance I think you got it about as right as is
possible, but I still think that it's important to attempt to separate

Motivation - The existing problem
Rationale - My solution, and why it is good

even if those aren't the best names for the two.


> As well as being more convenient for users, it would help decouple Perl
> core's object behaviours, like `DESTROY`, from this task. If less code
> relies on DESTROY, it may one day be possible to change the way that
> part of the code works, without worrying so much about code that relies
> on it breaking.

IIRC after discussion you took this out, but I think that a variant of it
does belong here. Something like:

As well as being more convenient for experienced programmers, it would
make it easier to explain the code to newcomers. as it makes the desired
behaviour explicit, rather than behaviour emergent from deliberate
combination of side effects of core's object behaviours.
(ie what `DESTROY` is, and the timing of implicit object destruction)

Nicholas Clark

On Wed, Jun 16, 2021 at 11:15:40PM +0100, Paul "LeoNerd" Evans wrote:


Yes, I really like the idea.

> ## Prototype Implementation
>
> CPAN module `Syntax::Keyword::Defer` as already mentioned.
>
> In addition, I have a mostly-complete branch of bleadperl (somewhat
> behind since I haven't updated it for the 5.34 release yet) at
>
> https://github.com/leonerd/perl5/tree/defer

It's really useful to have a prototype implementation to play with.


There were a couple of things that I noticed

## The `return` error is runtime:

$ ./perl -Ilib -E 'use feature "defer"; sub foo { defer { return; } say "Hi"}; foo()'
defer is experimental at -e line 1.
Hi
Can't "return" out of a defer block at -e line 1.


Could this be detected at compile time (the peephole optimiser? Or even the
tokeniser?) and be a runtime error?

I realise that `last` and `next` are also errors if they attempt to exit
the defer block, but loops are allowed inside defer blocks, so I doubt
that they are as easy to reliably detect at compile time.


## Error handling differs between `eval` and not.

I think that this would be the first time in perl that the behaviour between
`eval` and not eval differs. In that without eval:

$ ./perl -Ilib -E 'use feature "defer"; sub foo { defer { die "One"; } defer { die "Two" } say "Hi"}; foo(); say "Bye"'
defer is experimental at -e line 1.
defer is experimental at -e line 1.
Hi
Two at -e line 1.
One at -e line 1.


We get both exceptions to STDERR and then the process exits:


$ ./perl -Ilib -E 'use feature "defer"; sub foo { defer { die "One"; } defer { die "Two" } say "Hi"}; eval { foo() }; say "Bye"'
defer is experimental at -e line 1.
defer is experimental at -e line 1.
Hi
Bye



whereas $@ can only capture one:

$ ./perl -Ilib -E 'use feature "defer"; sub foo { defer { die "One"; } defer { die "Two" } say "Hi"}; eval { foo() }; say "\$@ is $@"; say "Bye"'
defer is experimental at -e line 1.
defer is experimental at -e line 1.
Hi
$@ is One at -e line 1.

Bye
$


and if the block exit is caused by an exception:

$ ./perl -Ilib -E 'use feature "defer"; sub foo { defer { die "One"; } defer { die "Two" } die "Three"}; eval { foo() }; say "\$@ is $@"; say "Bye"'
defer is experimental at -e line 1.
defer is experimental at -e line 1.
$@ is One at -e line 1.

Bye
$ ./perl -Ilib -E 'use feature "defer"; sub foo { defer { die "One"; } defer { die "Two" } die "Three"}; foo(); say "\$@ is $@"; say "Bye"'
defer is experimental at -e line 1.
defer is experimental at -e line 1.
Three at -e line 1.
Two at -e line 1.
One at -e line 1.


it's consistent - the last error encountered is in $@. But errors get
eaten.

I'm struggling to suggest a better solution. I *think* that possibly $@
ought to be the *first* error encountered. As Klortho glibly put it:

#11916 Always ignore the second error message unless the meaning is obvious

but what's bugging me is that the second (or subsequent errors) might be
fallout from the problem that caused the first exception and error message,
but if that error message is lost (and not reported) it could be rather hard
to figure out that actual problem.


Do we also need to create a variable @@ which exposes the stack of exceptions
encountered? (With $@ set to one of them, last as you currently have it,
first as I'm suggesting, and unchanged for existing problematic situations,
such as exceptions firing in DESTROY)

Nicholas Clark

the multiple exceptions behavior looks like a bug to be repaired, not a
feature to be accommodated.

IMO The first exception thrown from deferred code should prevent later
defer blocks in that scope from happening, just as if they were later in
the code text, instead of getting rearranged there through defer sugar.

sub foo { say "Hi"; die "two"; die "one" } # should this not be equivalent?

On Fri, Jul 2, 2021 at 4:45 AM Nicholas Clark <nick@ccl4.org> wrote:

>
> ## Error handling differs between `eval` and not.
>
> I think that this would be the first time in perl that the behaviour
> between
> `eval` and not eval differs. In that without eval:
>
> $ ./perl -Ilib -E 'use feature "defer"; sub foo { defer { die "One"; }
> defer { die "Two" } say "Hi"}; foo(); say "Bye"'
> defer is experimental at -e line 1.
> defer is experimental at -e line 1.
> Hi
> Two at -e line 1.
> One at -e line 1.
>
>
> We get both exceptions to STDERR and then the process exits:
>

--
"Lay off that whiskey, and let that cocaine be!" -- Johnny Cash

On Fri, Jul 2, 2021 at 12:58 PM David Nicol <davidnicol@gmail.com> wrote:

>
> the multiple exceptions behavior looks like a bug to be repaired, not a
> feature to be accommodated.
>
> IMO The first exception thrown from deferred code should prevent later
> defer blocks in that scope from happening, just as if they were later in
> the code text, instead of getting rearranged there through defer sugar.
>
> sub foo { say "Hi"; die "two"; die "one" } # should this not be
> equivalent?
>

No; the defer blocks were reached, so their scope cleanup should occur
whenever the scope is left, in this case because of an exception
propagating.

-Dan

On Wed, Jun 16, 2021 at 11:15:40PM +0100, Paul "LeoNerd" Evans wrote:

Just a few random thoughts and observations:

First, I approve of the idea and the name.

* What value does defer return (in either scalar and list context)? E.g.

sub foo { defer { BLOCK } }
my $x = foo(); # what is $x?
my @x = foo(); # what is @x?

# (while I assume this is a syntax error):
my $x = defer { BLOCK };

* Does your implementation handle Deparse ok? (and B, Concise etc?)

* Does it peephole optimise the BLOCK? And does it test for it?

I usually test for this by putting an $a[0] in the code block and then
checking in t/perf/opcount.t that there is at least one aelemfast op
(which is what small constant array indices are optimised to).

* is it called on die(), exit(), exception, etc?

Is it clearly documented (and tested) what it does for every possible type
of scope exit? I see it croaks on last etc. Destructors are called on
exit() - is a defer block too?

* what happens if the sub gets deleted mid-flow:

sub foo {
defer { delete $::{foo} }
defer { .. do more stuff .. } # might this SEGV?
}


--
Decaffeinated coffee is like dehydrated water