Mailing List Archive

Pre-RFC: Invalid reference access
Hi all,

We've discussed this before, but I can't see there's an RFC for it.

I'd love to see warnings when references are used in ways that we probably
did not intend.

Stringified references don't warn:

$ perl -Mwarnings -E 'my $foo = bless {}, "asdf"; say "$foo"'
asdf=HASH(0x14f80aa80)


Numification warns if you do that to a string:

$ perl -Mwarnings -E 'my $foo = "asdf"; say $foo + 1'
Argument "asdf" isn't numeric in addition (+) at -e line 1.
1


But not with a reference:

$ perl -Mwarnings -E 'my $foo = []; say $foo + 1'
5754627201


Regexes:

$ perl -Mwarnings -E 'my $foo = []; say $foo =~ /\d/'
1


Do we consider this a problem? (I do). If so, what are all the cases where
we'd need to catch this and warn? What would the warnings category be?

Best,
Curtis "Ovid" Poe
--
CTO, All Around the World
World-class software development and consulting
https://allaroundtheworld.fr/
Re: Pre-RFC: Invalid reference access [ In reply to ]
On 2023-01-18 14:45, Ovid wrote:
> Hi all,
>
> We've discussed this before, but I can't see there's an RFC for it.
>
> I'd love to see warnings when references are used in ways that we
> probably did not intend.
>
> Stringified references don't warn:
>
> $ perl -Mwarnings -E 'my $foo = bless {}, "asdf"; say "$foo"'
> asdf=HASH(0x14f80aa80)
>
>
> Numification warns if you do that to a string:
>
> $ perl -Mwarnings -E 'my $foo = "asdf"; say $foo + 1'
> Argument "asdf" isn't numeric in addition (+) at -e line 1.
> 1
>
>
> But not with a reference:
>
> $ perl -Mwarnings -E 'my $foo = []; say $foo + 1'
> 5754627201
>
>
> Regexes:
>
> $ perl -Mwarnings -E 'my $foo = []; say $foo =~ /\d/'
> 1
>
>
> Do we consider this a problem? (I do). If so, what are all the cases
> where we'd need to catch this and warn? What would the warnings
> category be?
>

This is also why I almost only use %s (and some %.*f) with sprintf.
(to avoid the auto-casting of formats like %d)

-- Ruud
Re: Pre-RFC: Invalid reference access [ In reply to ]
On Wed, 18 Jan 2023 at 14:45, Ovid <curtis.poe@gmail.com> wrote:
>
> Hi all,
>
> We've discussed this before, but I can't see there's an RFC for it.
>
> I'd love to see warnings when references are used in ways that we probably did not intend.
>
> Stringified references don't warn:
>
> $ perl -Mwarnings -E 'my $foo = bless {}, "asdf"; say "$foo"'
> asdf=HASH(0x14f80aa80)

I dont see a problem here. If I print out a ref im doing so
deliberately, and I definitely don't want warnings from it.

Also this shouldnt warn:

my %obj_hash;
$obj_hash{$foo}= $foo;

and as far as I know we dont have a special way to stringify things

>
> Numification warns if you do that to a string:
>
> $ perl -Mwarnings -E 'my $foo = "asdf"; say $foo + 1'
> Argument "asdf" isn't numeric in addition (+) at -e line 1.
> 1

I agree adding 1 to a ref is weird and it would be nice if we can
warn, but again I would not expect print 0+$ref to warn, for the same
reason as above:

my %obj_cache;
$obj_cache{0+$foo}= $foo;


>
> But not with a reference:
>
> $ perl -Mwarnings -E 'my $foo = []; say $foo + 1'
> 5754627201

Yeah, it would be nice if normal addition did warn.

>
> Regexes:
>
> $ perl -Mwarnings -E 'my $foo = []; say $foo =~ /\d/'
> 1

As written like it would be nice if we warned. But consider code like this:

die "Object $foo doesn't have work as we want";
if ($@=~/\d/) { }

>
> Do we consider this a problem? (I do). If so, what are all the cases where we'd need to catch this and warn? What would the warnings category be?

MY guess is it is a can of worms to separate the "good" cases from the
"bad" cases. Maybe I am being insufficiently imaginative tho.

Yves


--
perl -Mre=debug -e "/just|another|perl|hacker/"
Re: Pre-RFC: Invalid reference access [ In reply to ]
On Wed, 18 Jan 2023 14:45:04 +0100
Ovid <curtis.poe@gmail.com> wrote:

> Hi all,
>
> We've discussed this before, but I can't see there's an RFC for it.
>
> I'd love to see warnings when references are used in ways that we
> probably did not intend.
>
> Stringified references don't warn:
>
> $ perl -Mwarnings -E 'my $foo = bless {}, "asdf"; say "$foo"'
> asdf=HASH(0x14f80aa80)

For that you want my `no stringification` CPAN module.

I've always suggested it should be core.

It is just lacking a decent name. Ideally it would be a strictness flag

use strict 'strings';

but then people started arguing about our inability to ever add new
strictness flags because a plain `use strict` apparently means `use
strict ':all'` and so we can't add any more.

Or something.

I think it's a stupid reason - I'd love `use strict` to mean
`use strict ':5.8'` and we could then add new strictness flags in
versioned bundles.

If we agreed to the principle of the idea I could probably implement
the lot of it in a day or two.

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/
Re: Pre-RFC: Invalid reference access [ In reply to ]
Hi there,

On Wed, 18 Jan 2023, Ovid wrote:

> I'd love to see warnings when references are used in ways that we
> probably did not intend.

+1

> Stringified references don't warn:
>
> $ perl -Mwarnings -E 'my $foo = bless {}, "asdf"; say "$foo"'
> asdf=HASH(0x14f80aa80)

Ah. I'd always considered this a 'feature', and I use it. Probably
I'm just being lazy (I think I read somewhere that was a Good Thing)
but probably I shouldn't use it that way.

> Numification warns if you do that to a string:
>
> $ perl -Mwarnings -E 'my $foo = "asdf"; say $foo + 1'
> Argument "asdf" isn't numeric in addition (+) at -e line 1.
> 1

Fine.

> But not with a reference:
>
> $ perl -Mwarnings -E 'my $foo = []; say $foo + 1'
> 5754627201

I made it 367344747833 the first time but it's obviously broken. :/

> Regexes:
>
> $ perl -Mwarnings -E 'my $foo = []; say $foo =~ /\d/'
> 1

Also broken.

> Do we consider this a problem? (I do).

Agreed.

> ...what are all the cases where we'd need to catch this and warn?

If you mean all the cases where it isn't caught and should be, then I
have no idea and I'd need to spend (probably quite a lot of) time on
it that I don't have. Is whacking moles as they surface an option?

> What would the warnings category be?

For the two you've shown, I'd go for 'exiting'. :)

--

73,
Ged.
Re: Pre-RFC: Invalid reference access [ In reply to ]
On Wed, Jan 18, 2023 at 2:45 PM Ovid <curtis.poe@gmail.com> wrote:
>
> Hi all,
>
> We've discussed this before, but I can't see there's an RFC for it.
>
> I'd love to see warnings when references are used in ways that we probably did not intend.
>
> Stringified references don't warn:
>
> $ perl -Mwarnings -E 'my $foo = bless {}, "asdf"; say "$foo"'
> asdf=HASH(0x14f80aa80)
>
>
> Numification warns if you do that to a string:
>
> $ perl -Mwarnings -E 'my $foo = "asdf"; say $foo + 1'
> Argument "asdf" isn't numeric in addition (+) at -e line 1.
> 1
>
>
> But not with a reference:
>
> $ perl -Mwarnings -E 'my $foo = []; say $foo + 1'
> 5754627201
>
>
> Regexes:
>
> $ perl -Mwarnings -E 'my $foo = []; say $foo =~ /\d/'
> 1
>
>
> Do we consider this a problem? (I do). If so, what are all the cases where we'd need to catch this and warn? What would the warnings category be?
>

I think this would make a lot of sense as a new warning category, but
I think stringifying and numifying refs is done intentionally often
enough that it would be a problem to enable by default. This then
brings up the issue of trying to separate "all" warnings (which "use
warnings;" is documented as enabling) from "extended" warnings or
something, which has been proposed before but a satisfactory solution
has never been settled on.
Re: Pre-RFC: Invalid reference access [ In reply to ]
On Wed, Jan 18, 2023 at 3:04 PM Paul "LeoNerd" Evans <leonerd@leonerd.org.uk>
wrote:

> > Stringified references don't warn:
> >
> > $ perl -Mwarnings -E 'my $foo = bless {}, "asdf"; say "$foo"'
> > asdf=HASH(0x14f80aa80)
>
> For that you want my `no stringification` CPAN module.


Can you provide a link? I don't see it: https://metacpan.org/author/PEVANS

--
Curtis "Ovid" Poe
--
CTO, All Around the World
World-class software development and consulting
https://allaroundtheworld.fr/
Re: Pre-RFC: Invalid reference access [ In reply to ]
On Wed, Jan 18, 2023 at 9:27 AM Graham Knop <haarg@haarg.org> wrote:

> On Wed, Jan 18, 2023 at 2:45 PM Ovid <curtis.poe@gmail.com> wrote:
> >
> > Hi all,
> >
> > We've discussed this before, but I can't see there's an RFC for it.
> >
> > I'd love to see warnings when references are used in ways that we
> probably did not intend.
> >
> > Stringified references don't warn:
> >
> > $ perl -Mwarnings -E 'my $foo = bless {}, "asdf"; say "$foo"'
> > asdf=HASH(0x14f80aa80)
> >
> >
> > Numification warns if you do that to a string:
> >
> > $ perl -Mwarnings -E 'my $foo = "asdf"; say $foo + 1'
> > Argument "asdf" isn't numeric in addition (+) at -e line 1.
> > 1
> >
> >
> > But not with a reference:
> >
> > $ perl -Mwarnings -E 'my $foo = []; say $foo + 1'
> > 5754627201
> >
> >
> > Regexes:
> >
> > $ perl -Mwarnings -E 'my $foo = []; say $foo =~ /\d/'
> > 1
> >
> >
> > Do we consider this a problem? (I do). If so, what are all the cases
> where we'd need to catch this and warn? What would the warnings category be?
> >
>
> I think this would make a lot of sense as a new warning category, but
> I think stringifying and numifying refs is done intentionally often
> enough that it would be a problem to enable by default. This then
> brings up the issue of trying to separate "all" warnings (which "use
> warnings;" is documented as enabling) from "extended" warnings or
> something, which has been proposed before but a satisfactory solution
> has never been settled on.
>

Relevant: https://github.com/Perl/perl5/issues/18543

-Dan
Re: Pre-RFC: Invalid reference access [ In reply to ]
On Wed, Jan 18, 2023 at 6:33 PM Dan Book <grinnz@gmail.com> wrote:

>
>> Relevant: https://github.com/Perl/perl5/issues/18543
>

And that has a link to stringification.pm
<https://metacpan.org/release/PEVANS/stringification-0.01_004/view/lib/stringification.pm>,
which doesn't show up in Paul's module list
<https://metacpan.org/author/PEVANS> presumably because the module is a
"development release" (though it's over a decade old).

So it seems like we have two things:

1. Find a better way to surface to developers that they're abusing
references
2. New warnings and strictures would be useful,

For the second, in a similar idea to what you proposed (though with a
different interface), I had a pre-rfc for warnings::extended
<https://www.nntp.perl.org/group/perl.perl5.porters/2022/10/msg264879.html>,
but it laid there and twitched. No one seemed that keen, so I dropped it.

LeoNerd: I wouldn't mind seeing a version of stringification.pm that let me
warn instead of died.

Best,
Ovid
--
Curtis "Ovid" Poe
--
CTO, All Around the World
World-class software development and consulting
https://allaroundtheworld.fr/
Re: Pre-RFC: Invalid reference access [ In reply to ]
On Thu, 19 Jan 2023 08:43:02 +0100
Ovid <curtis.poe@gmail.com> wrote:

> And that has a link to stringification.pm
> <https://metacpan.org/release/PEVANS/stringification-0.01_004/view/lib/stringification.pm>,
> which doesn't show up in Paul's module list
> <https://metacpan.org/author/PEVANS> presumably because the module is
> a "development release" (though it's over a decade old).

Yeah; metacpan doesn't index modules that *only* have a development
release.

> LeoNerd: I wouldn't mind seeing a version of stringification.pm that
> let me warn instead of died.

I'd consider it. Unsure how you'd request it - module import option?

I don't want to encourage people to actually *use* that implementation
of the module though. It's hooooooorrrrrrrible. Core perl doesn't have
enough hooks to properly implement it so I had to go about it a
terrible way and basically replace every single core op. Yuck! A far
better core implementation is about 10 lines of core code.

Plus a decade of bikeshedding about what it should be called, that I
really couldn't be bothered with.

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/
Re: Pre-RFC: Invalid reference access [ In reply to ]
On Thu, 19 Jan 2023, 18:04 Paul "LeoNerd" Evans, <leonerd@leonerd.org.uk>
wrote:

> On Thu, 19 Jan 2023 08:43:02 +0100
> Ovid <curtis.poe@gmail.com> wrote:
>
> > And that has a link to stringification.pm
> > <
> https://metacpan.org/release/PEVANS/stringification-0.01_004/view/lib/stringification.pm
> >,
> > which doesn't show up in Paul's module list
> > <https://metacpan.org/author/PEVANS> presumably because the module is
> > a "development release" (though it's over a decade old).
>
> Yeah; metacpan doesn't index modules that *only* have a development
> release.
>
> > LeoNerd: I wouldn't mind seeing a version of stringification.pm that
> > let me warn instead of died.
>
> I'd consider it. Unsure how you'd request it - module import option?
>
> I don't want to encourage people to actually *use* that implementation
> of the module though. It's hooooooorrrrrrrible. Core perl doesn't have
> enough hooks to properly implement it so I had to go about it a
> terrible way and basically replace every single core op. Yuck! A far
> better core implementation is about 10 lines of core code.
>

Post the patch. Let's start the ball rolling.

>
> Plus a decade of bikeshedding about what it should be called, that I
> really couldn't be bothered with.
>

Oh come now. A decade? Let's not exaggerate huh, it would be a few years at
most. :-)

Yves

>
Re: Pre-RFC: Invalid reference access [ In reply to ]
On Thu, Jan 19, 2023 at 08:43:02AM +0100, Ovid wrote:
> 1. Find a better way to surface to developers that they're abusing
> references

The behaviour of a reference in a string or numeric context is documented
(and is sometimes useful!).

> 2. New warnings and strictures would be useful,

I'm happy with the general concept of adding new warnings and strictures
that are not enabled by default via 'use strict' etc, but which get
enabled in conjunction with 'use feature X' and/or 'use v5.XX'.

I would be be ok with the numeric/string use of references triggering
warnings or croaking in the presence of such a new stricture. I wouldn't
be happy with them warning/croaking in the presence of a plain "use strict
/ warnings".


--
Counsellor Troi states something other than the blindingly obvious.
-- Things That Never Happen in "Star Trek" #16