Mailing List Archive

On Sat, 17 Sept 2022, 11:57 Paul "LeoNerd" Evans, <leonerd@leonerd.org.uk>
wrote:

> On Sat, 17 Sep 2022 03:53:53 -0700
> Darren Duncan <darren@darrenduncan.net> wrote:
>
> > > I don't see what a builtin would add.
> > >
> > > eval "use $module; 1"
> > >
> > > Works fine.
> >
> > Why? Its because having to use string eval for something so simple
> > and basic is a TERRIBLE design, a huge code smell. The proposal of a
> > proper basic runtime-defined load that DOESN'T use string eval is
> > sorely needed. This is insecure and error prone like SQL injection
> > from straight interpolation of variables into the raw source when it
> > should just be a plain value or argument.
>
> +1. (Well, +2 if I could ;) )
>
> Plus also Yves' code didnt' handle errors; you forgot to
>
> eval "use $module; 1" or die $@;


I didn't forget anything. I assumed anyone on this list understood that was
implied by the 1. I'm on a mobile phone, typing code is a pita. Sorry I
didn't spell it out.

So all I see is a bunch of folks sticking this load function in an eval.

Whatever, I don't care that much. But please don't call it "load".
"module_load" or "package_load".

I'm a serous anti-fan of lazy loading patterns in library level code,
especially in a web context it is an antipattern that slows things down and
consumes excess memory. I understand and appreciate that for scripts it can
be the opposite. One thing that would sell me on adding this if we came up
with a way to resolve the conflict between those two use cases. Can we
figure out a way that a module consumer can reliably tell modules that they
should *pre*load its dependencies and not lazy load them?

Yves

>
>

On 2022-09-17 7:02 a.m., demerphq wrote:
> So all I see is a bunch of folks sticking this load function in an eval.

I interpret this that you see many people not understanding how to use the new
function properly and that they incorrectly assume they have to put it in an
eval to make it work like "require $foo" etc. So that's up to education.

> Whatever, I don't care that much. But please don't call it "load". "module_load"
> or "package_load".

Personally I'm not stuck on any particular name, and I welcome a separate
discussion to bikeshed the name, but on the RFC itself.

> I'm a serous anti-fan of lazy loading patterns in library level code, especially
> in a web context it is an antipattern that slows things down and consumes excess
> memory. I understand and appreciate that for scripts it can be the opposite. One
> thing that would sell me on adding this if we came up with a way to resolve the
> conflict between those two use cases. Can we figure out a way that a module
> consumer can reliably tell modules that they should *pre*load its dependencies
> and not lazy load them?

A key usage scenario I see for these new methods is when we have a library with
a plugin architecture such that the specific plugin module we want to use is not
known except at runtime and it is a user configuration data file etc that is
naming the module to use.

For example, it would be a cleaner way to implement something like the DBI
module which can use the new methods to load any DBMS driver module the user
names via connect().

For another example, it would be a cleaner way to implement localization type
modules that implement each user language as a Perl module, an example being my
Locale::KeyedText module on CPAN.

With the new builtins available, I wouldn't have to resort to these shenanigans:

sub template_module_is_loaded {
my ($self, $module_name) = @_;
...
no strict 'refs';
return scalar keys %{$module_name . '::'};
}

sub load_template_module {
my ($self, $module_name) = @_;
...
# Note: We have to invoke this 'require' in an eval string
# because we need the bareword semantics, where 'require'
# will munge the package name into file system paths.
eval "require $module_name;";
$self->_die_with_msg( ... )
if $@;
return;
}

-- Darren Duncan

> > these are no ordinary methods, these are methods invoked on internal data.
> > It's similar to My::Class::->meta->..., but instead of having meta
> > method (polluting user's namespace)
> > it will signal that this is internal method.
>
> What you're saying here sounds wrong to me. Its coming across like these
> methods are unique and nothing like them has ever existed in Perl before.

I will use example from raku and its Grammar, problem I ran into
whenwriting SQL parser:
- you cannot create token CREATE, because it is used by language
- you cannot create token before, because class method before is used
by language to handle lookaround

With star syntax it will be clearly visible
- this method is user method (and part its intended interface)
- this method is a hook defined by language to augment default behaviour

That's what I meant mentioning "pollute user's namespace" and "internal data"

> While I'm a strong supporter of having stronger typing in Perl, in the meantime
> why is it not good enough for a regular string scalar like `"Module::Foo"` not
> suffice to represent a namespace?

Perl already supports trailing ::, even with special warning, so that
is good candidate
for namespace literal (more details in mentioned pre-rfc), eg:
Module::Foo::

> Especially since Perl already does this, as far as I know, which is why you can
> say `$class->foo()` etc.

yes, and that's reason why I suggested star syntax, to distinguish
static methods called on class
and methods called on namespace itself

> I would also argue that creating special Perl syntax for namespace literals
> should be done FIRST, and BEFORE considering this star-syntax.

Agree, though these works are independent

> If we have an actual distinct type for a namespace, would that mean the
> rationale for the star-syntax goes away, or do you still see its important
> regardless?

I think star-syntax (or other variant, I used star only because it
looks nice on my monitor)
is important regardless of implementation namespace literal - this
methods are not parts
of given namespace API.

> I believe the core implementation of builtin::load_module() etc should NOT be
> overloadable, and the builtin version has only one behavior.
>
> If we want multiple fundamental behaviors, that's provided by having either
> multiple distinctly named builtin::foo() and/or some of them take extra
> arguments to determine it.

builtin methods are pretty similar to operators

builtin methods should be overloadable (though load_module will not be
because it will be executed before any overload will be defined)

Best regards,
Brano

On Sat, 17 Sept 2022, 21:31 Darren Duncan, <darren@darrenduncan.net> wrote:

> On 2022-09-17 7:02 a.m., demerphq wrote:
> > So all I see is a bunch of folks sticking this load function in an eval.
>
> I interpret this that you see many people not understanding how to use the
> new
> function properly and that they incorrectly assume they have to put it in
> an
> eval to make it work like "require $foo" etc. So that's up to education.
>

Er, no. It's a direct consequence of Paul's comment that load should die if
it fails. I start to get the impression you folks advocating for this
aren't on the same page with each other.

> Whatever, I don't care that much. But please don't call it "load".
> "module_load"
> > or "package_load".
>
> Personally I'm not stuck on any particular name, and I welcome a separate
> discussion to bikeshed the name, but on the RFC itself.
>
> > I'm a serous anti-fan of lazy loading patterns in library level code,
> especially
> > in a web context it is an antipattern that slows things down and
> consumes excess
> > memory. I understand and appreciate that for scripts it can be the
> opposite. One
> > thing that would sell me on adding this if we came up with a way to
> resolve the
> > conflict between those two use cases. Can we figure out a way that a
> module
> > consumer can reliably tell modules that they should *pre*load its
> dependencies
> > and not lazy load them?
>
> A key usage scenario I see for these new methods is when we have a library
> with
> a plugin architecture such that the specific plugin module we want to use
> is not
> known except at runtime and it is a user configuration data file etc that
> is
> naming the module to use.
>

This is exactly the kind of antipattern I object to in a library and
exactly the kind of thing that if you are doing performance sensitive multi
process stuff that causes serious problems and is pretty much exactly a
description of the kind of problem I'd like to see solved.

>
> For example, it would be a cleaner way to implement something like the DBI
> module which can use the new methods to load any DBMS driver module the
> user
> names via connect().
>

I don't consider that good practice so it doesnt really sell me. If you are
using a specific dbd then use it. Sql isn't that portable and DBIs
abstraction does not insulate you from the portability issues that come
from SQL.

>
> For another example, it would be a cleaner way to implement localization
> type
> modules that implement each user language as a Perl module, an example
> being my
> Locale::KeyedText module on CPAN.
>
> With the new builtins available, I wouldn't have to resort to these
> shenanigans:
>
> sub template_module_is_loaded {
> my ($self, $module_name) = @_;
> ...
> no strict 'refs';
> return scalar keys %{$module_name . '::'};
> }
>

This just looks like bad api design to me. If I needed something like that
I'd ensure there is a well defined method to call in the module that
returned a list of whatever was needed. Introspecting on the package stash
doesn't prove anything. Anybody can do 'sub Module::thing {}' anywhere and
break this logic.

I don't see how it relates to having a load builtin either. Loading
package foo doesn't mean that any code will end up in the foo namespace.
Package foo can populate any or no namespaces. Package names and stashes
are only coincidentally related to each other, and are not necessarily
related.



> sub load_template_module {
> my ($self, $module_name) = @_;
> ...
> # Note: We have to invoke this 'require' in an eval string
> # because we need the bareword semantics, where 'require'
> # will munge the package name into file system paths.
> eval "require $module_name;";
> $self->_die_with_msg( ... )
> if $@;
> return;
> }
>

I've done things like this, but I''d never touch $@ without a false return
from an eval. Even if it might work in modern perls it's just confusing to
newbies. Treating $@ like $! Is good practice. I've seen maintainer coders
stick arbitrary code in between the eval and $@ access, which as $@ is
volatile can completely change the result. I'd consider that a bug waiting
to happen if it were in a communal code base.

So, it's not clear to me what you expect. At the start of this mail you
suggested that the proposed builtin would not need an eval, but here you
gave a function that rethrows the exception and would require an eval if I
speculatively want to load something, a pattern which I have seen far more
than the cases you describe. Eg "use this module and if it's present import
a sub, and if it's not use a roll-your-own replacement".

So does this proposed new function die if it fails to load or not?

Yves

On 2022-09-17 2:29 p.m., demerphq wrote:
> On Sat, 17 Sept 2022, 21:31 Darren Duncan wrote:
>
> On 2022-09-17 7:02 a.m., demerphq wrote:
> > So all I see is a bunch of folks sticking this load function in an eval.
>
> I interpret this that you see many people not understanding how to use the new
> function properly and that they incorrectly assume they have to put it in an
> eval to make it work like "require $foo" etc.  So that's up to education.
>
> Er, no. It's a direct consequence of Paul's comment that load should die if it
> fails. I start to get the impression you folks advocating for this aren't on the
> same page with each other.

There is no conflict here because I assume there would be multiple versions of
the functionality and users can pick the version that does or doesn't die on
failure depending on what they want.

> A key usage scenario I see for these new methods is when we have a library with
> a plugin architecture such that the specific plugin module we want to use is
> not
> known except at runtime and it is a user configuration data file etc that is
> naming the module to use.
>
> This is exactly the kind of antipattern I object to in a library and exactly the
> kind of thing that if you are doing performance sensitive multi process stuff
> that causes serious problems and is pretty much exactly a description of the
> kind of problem I'd like to see solved.

I'm not saying a plugin architecture is always appropriate, but there are
legitimate reasons for a core tool to not have to have knowledge of every
third-party thing that might be used with it.

Why should Adobe Photoshop ship with a list of all the possible third-party
plugins baked into its source code?

> For example, it would be a cleaner way to implement something like the DBI
> module which can use the new methods to load any DBMS driver module the user
> names via connect().
>
> I don't consider that good practice so it doesnt really sell me. If you are
> using a specific dbd then use it. Sql isn't that portable and DBIs abstraction
> does not insulate you from the portability issues that come from SQL.

I actually agree with you, and feel that DBI's design is backwards. Instead a
better design is that DBI is just an interface definition that each database
driver adopts, and that users should just be using the driver module directly,
eg they say "use DBD::Oracle" and "DBD::Oracle->connect()" or whatever.

And despite appearances, DBI is actually worse because it actually does have
intimate knowledge of possible drivers, a list of such baked into its source
code, or it did last I looked.

However, I raised this as a well known example of a plugin architecture.

> For another example, it would be a cleaner way to implement localization type
> modules that implement each user language as a Perl module, an example being my
> Locale::KeyedText module on CPAN.
>
> With the new builtins available, I wouldn't have to resort to these shenanigans:
...
> This just looks like bad api design to me. If I needed something like that I'd
> ensure there is a well defined method to call in the module that returned a list
> of whatever was needed. Introspecting on the package stash doesn't prove
> anything. Anybody can do 'sub Module::thing {}' anywhere and break this logic.
>
>  I don't see how it relates to having a load builtin either. Loading package
> foo doesn't mean that any code will end up in the foo namespace. Package foo can
> populate any or no namespaces. Package names and stashes are only coincidentally
> related to each other, and are not necessarily related.
...
> I've done things like this, but I''d never touch $@ without a false return from
> an eval. Even if it might work in modern perls it's just confusing to newbies.
> Treating $@ like $! Is good practice. I've seen maintainer coders stick
> arbitrary code in between the eval and $@ access, which as $@ is volatile can
> completely change the result. I'd consider that a bug waiting to happen if it
> were in a communal code base.
>
> So, it's not clear to me what you expect. At the start of this mail you
> suggested that the proposed builtin would not need an eval, but here you gave a
> function that rethrows the exception and would require an eval if I
> speculatively want to load something, a pattern which I have seen far more than
> the cases you describe. Eg "use this module and if it's present import a sub,
> and if it's not use a roll-your-own replacement".

My mentioning Locale::KeyedText was just meant to be another example of how
checking for the use of a package use or loading one is much more contrived than
it ought to be. The fact that I happen to throw an exception on failure is
beside the point.

Also, Locale::KeyedText is an old CPAN module I no longer update or expect
anyone to use, and it doesn't reflect the latest good design. It was based on
trying to make available the MacOS feature of externally installable language
resources for applications where no language is privileged over others, eg no
replacing English text with other languages.

> So does this proposed new function die if it fails to load or not?

I assume there would be multiple versions so users can choose.

The whole proposal is for a set of routines, not just one.

-- Darren Duncan

On Fri, Sep 16, 2022 at 10:13 AM Paul "LeoNerd" Evans
<leonerd@leonerd.org.uk> wrote:
>
> On Fri, 16 Sep 2022 09:54:27 -0400
> "Ricardo Signes" <perl.p5p@rjbs.manxome.org> wrote:
>
> > This became a formal proposal
> > <https://github.com/Perl/RFCs/blob/main/rfcs/rfc0006.md>, which I
> > think is more or less ready to implement. *Who would like to
> > implement it?* Many people who write in Perl 5 would benefit from
> > such a builtin, and I believe it should be relatively straightforward.
>
> I don't want to add it to my plate of things, but I'd be happy to
> handhold a volunteer through the overall process.

I took a stab at it here:

https://github.com/Perl/perl5/pull/20316

-- Matthew Horsfall (alh)

On 2022-09-18 7:06 p.m., Matthew Horsfall (alh) wrote:
> On Fri, Sep 16, 2022 at 10:13 AM Paul "LeoNerd" Evans
> <leonerd@leonerd.org.uk> wrote:
>>
>> On Fri, 16 Sep 2022 09:54:27 -0400
>> "Ricardo Signes" <perl.p5p@rjbs.manxome.org> wrote:
>>
>>> This became a formal proposal
>>> <https://github.com/Perl/RFCs/blob/main/rfcs/rfc0006.md>, which I
>>> think is more or less ready to implement. *Who would like to
>>> implement it?* Many people who write in Perl 5 would benefit from
>>> such a builtin, and I believe it should be relatively straightforward.
>>
>> I don't want to add it to my plate of things, but I'd be happy to
>> handhold a volunteer through the overall process.
>
> I took a stab at it here:
>
> https://github.com/Perl/perl5/pull/20316
>
> -- Matthew Horsfall (alh)

Thank you for doing that.

Are you planning to also do other related routines besides "load" itself?

Also there's still the open question of whether we want this to have a different
name for "load", however changing that can be done in a subsequent pull request
rather than holding up this one.

-- Darren Duncan

On Sun, Sep 18, 2022, at 22:06, Matthew Horsfall (alh) wrote:
> I took a stab at it here:
>
> https://github.com/Perl/perl5/pull/20316

???? Thanks, Matthew! I'll build this soon.

--
rjbs

On Sun, Sep 18, 2022, at 23:34, Darren Duncan wrote:
> Are you planning to also do other related routines besides "load" itself?

The RFC calls for no other related routines.

> Also there's still the open question of whether we want this to have a different
> name for "load", however changing that can be done in a subsequent pull request
> rather than holding up this one.

The specified name is "load".

--
rjbs

> Also there's still the open question of whether we want this to have a different
> name for "load", however changing that can be done in a subsequent pull request
> rather than holding up this one.
>
>
> The specified name is "load".
>

so there will never be anything else that can be loaded ?
(eg: XSloader move to builting)

> --
> rjbs

On Mon, Sep 19, 2022, at 09:00, Branislav Zahradník wrote:
> > The specified name is "load".
>
> so there will never be anything else that can be loaded ?
> (eg: XSloader move to builting)

"Nothing else will ever be loaded" does not follow from "this case is called load."

--
rjbs

On Mon, 19 Sept 2022, 14:38 Ricardo Signes, <perl.p5p@rjbs.manxome.org>
wrote:

> On Mon, Sep 19, 2022, at 09:00, Branislav Zahradník wrote:
>
> > The specified name is "load".
>
> so there will never be anything else that can be loaded ?
> (eg: XSloader move to builting)
>
>
> "Nothing else will ever be loaded" does not follow from "this case is
> called load."
>

But it makes adding new cases more confusing and will be a builtin that
will clash with many preexisting subs with that name. It should be called
something more specific.

Yves

I wonder if it wouldn’t be more flexible to have

module_file_of (‘Foo::Bar’);

and then allow that to be loaded into require or used for any other desired purpose, rather than having a special “load” routine.

e.g.,

my $x = module_name_of('Foo::Bar');
require $x;

would be equivalent to

require Foo::Bar;



Or, maybe not.

To be honest, I’m not sure to what other purposes this function could be put, but it strikes me as the wrong thing to have some kind of data munging that can only be accessed through a separate function (as though we only had printf and not sprintf).

--
Aaron Priven, aaron@priven.com, www.priven.com/aaron

On Mon, 19 Sept 2022 at 18:36, Aaron Priven <aaron@priven.com> wrote:
>
> I wonder if it wouldn’t be more flexible to have
>
> module_file_of (‘Foo::Bar’);

+1

this is not bad idea

>
> and then allow that to be loaded into require or used for any other desired purpose, rather than having a special “load” routine.
>
> e.g.,
>
> my $x = module_name_of('Foo::Bar');
> require $x;
>
> would be equivalent to
>
> require Foo::Bar;
>

if I understand it correctly, load should behave like:
my $loaded = eval { require $x; 1 }

On Sat, 17 Sep 2022 11:58:47 +0100
"Paul \"LeoNerd\" Evans" <leonerd@leonerd.org.uk> wrote:

> If it isn't explicitly spelled out in words, then yes it ought to be
> so.
>
> builtin::load($module);
>
> or
>
> use builtin 'load';
> load($module);

Aaaaactually, having said that, I am now in the middle of starting to
write the metaprogramming API RFC. It would be quite a good natural fit
for that set of API instead.

So hold that thought until I get something draft-worthy online...

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/