Mailing List Archive

On Fri, Sep 3, 2021 at 12:49 PM Ricardo Signes <perl.p5p@rjbs.manxome.org>
wrote:

> Porters,
>
> For unexciting reasons, Module::Runtime has come up a few times on recent
> PSC calls. We talked about Module::Runtime, Class::Load, and I don't know
> how many other related topics.
>
> We ended up with this question: Isn't this something the core should
> provide?
>
> Now, I'm not saying the core should provide every single thing provided by
> all the modules in this problem area. I'm saying that maybe 99% of use of
> these libraries would be addressed by providing one or two small
> concessions in the core language. I don't know what the second concession
> of that one or two is, but I know the first one is acknowledging that the
> current behavior of "require EXPR" isn't as useful as it could be. When
> given a string, it's much more common to want to load a module, not a file.
>
> So, let's say we were to implement a new require-like builtin that we
> wanted to be a core language feature and live forever. Would it be
> something other than something really close to Module::Runtime's
> use_module? I have some thoughts, but I think the basic question here is:
>
> Is it enough to say "v5.x will provide load, which will take a module
> name and optional version, then load that module (or die trying) and call
> ->VERSION on it.
>

I think that would solve 95% of use cases that currently need
Module::Runtime (or a worse alternative). Name and return value to be
bikeshed notwithstanding.

-Dan

> On Sep 3, 2021, at 12:48 PM, Ricardo Signes <perl.p5p@rjbs.manxome.org> wrote:
>
> So, let's say we were to implement a new require-like builtin that we wanted to be a core language feature and live forever. Would it be something other than something really close to Module::Runtime's use_module? I have some thoughts, but I think the basic question here is:
>
> Is it enough to say "v5.x will provide load, which will take a module name and optional version, then load that module (or die trying) and call ->VERSION on it.

There’s an opportunity here.

Occasionally I’ve implemented plugin-like things where I require() a module that isn’t actually “required”. To differentiate “no such module exists” from “failed to load module” I’ve had to resort to pattern-matches on the thrown error, which is ugly.

I know the topic of structured or otherwise-machine-parsable errors is a quagmire, but would it be feasible to implement this new runtime module loader in such a way that a caller can reliably differentiate those two fundamentally-distinct failure states?

-FG

On Fri, Sep 3, 2021 at 1:30 PM Felipe Gasper <felipe@felipegasper.com>
wrote:

>
> > On Sep 3, 2021, at 12:48 PM, Ricardo Signes <perl.p5p@rjbs.manxome.org>
> wrote:
> >
> > So, let's say we were to implement a new require-like builtin that we
> wanted to be a core language feature and live forever. Would it be
> something other than something really close to Module::Runtime's
> use_module? I have some thoughts, but I think the basic question here is:
> >
> > Is it enough to say "v5.x will provide load, which will take a module
> name and optional version, then load that module (or die trying) and call
> ->VERSION on it.
>
> There’s an opportunity here.
>
> Occasionally I’ve implemented plugin-like things where I require() a
> module that isn’t actually “required”. To differentiate “no such module
> exists” from “failed to load module” I’ve had to resort to pattern-matches
> on the thrown error, which is ugly.
>
> I know the topic of structured or otherwise-machine-parsable errors is a
> quagmire, but would it be feasible to implement this new runtime module
> loader in such a way that a caller can reliably differentiate those two
> fundamentally-distinct failure states?


haarg was working on a Module::Runtime replacement that has this capability
with a function called "try_require_module", which returns true on success,
false on not found or insufficient version, and an exception on other
errors. I think it is a great API, but unfortunately has not been put
through the rigors of CPAN workshopping yet.

-Dan

On Fri, Sep 3, 2021 at 10:49 AM Dan Book <grinnz@gmail.com> wrote:

>
> haarg was working on a Module::Runtime replacement that has this
> capability with a function called "try_require_module", which returns true
> on success, false on not found or insufficient version, and an exception on
> other errors. I think it is a great API, but unfortunately has not been put
> through the rigors of CPAN workshopping yet.
>

IMO the toolchain gang should take on the goal of producing an RFC. Graham
has done much of the work (investigative, as well as code) on this already
so we should be able to get to that point "soon".

On Fri, Sep 3, 2021 at 6:49 PM Ricardo Signes <perl.p5p@rjbs.manxome.org> wrote:
>
> So, let's say we were to implement a new require-like builtin that we wanted to be a core language feature and live forever. Would it be something other than something really close to Module::Runtime's use_module? I have some thoughts, but I think the basic question here is:
>
> Is it enough to say "v5.x will provide load, which will take a module name and optional version, then load that module (or die trying) and call ->VERSION on it.

I think a core function to handle this would be a welcome addition.
But I don't think that an equivalent to use_module is enough.

One of the common uses people have for runtime loading is conditional
loading. For example, providing fallback functions if an XS module is
not available. This is often done using something like

if (eval { require Ref::Util::XS }) {
...;
}

The problem with this is that it doesn't distinguish between the
various possible failures when loading the module. It's intended to
catch missing module errors, but it will also catch compilation
errors. This causes problems in debugging, as any later code that
tries to load Ref::Util::XS code will get errors like "Attempt to
reload Ref/Util/XS.pm aborted.", hiding the compilation error.
Currently, Module::Runtime handles optional loading by parsing the
error message given by perl to detect "Can't locate" errors.

This could be considered part of the general problem perl has with its
errors being strings. A proper exception system would allow filtering
these trivially. It could also be considered entirely separate from
loading a stringy module name, since it applies equally to require
with a bareword module name.

A simple "require $module_name" may be the most common case, but I
think optional loading is too common to ignore if we're going to add
this to core.

> On Sep 12, 2021, at 12:11, Graham Knop <haarg@haarg.org> wrote:
>
> ?On Fri, Sep 3, 2021 at 6:49 PM Ricardo Signes <perl.p5p@rjbs.manxome.org> wrote:
>>
>> So, let's say we were to implement a new require-like builtin that we wanted to be a core language feature and live forever. Would it be something other than something really close to Module::Runtime's use_module? I have some thoughts, but I think the basic question here is:
>>
>> Is it enough to say "v5.x will provide load, which will take a module name and optional version, then load that module (or die trying) and call ->VERSION on it.
>
> I think a core function to handle this would be a welcome addition.
> But I don't think that an equivalent to use_module is enough.
>
> One of the common uses people have for runtime loading is conditional
> loading. For example, providing fallback functions if an XS module is
> not available. This is often done using something like
>
> if (eval { require Ref::Util::XS }) {
> ...;
> }
>
> The problem with this is that it doesn't distinguish between the
> various possible failures when loading the module. It's intended to
> catch missing module errors, but it will also catch compilation
> errors. This causes problems in debugging, as any later code that
> tries to load Ref::Util::XS code will get errors like "Attempt to
> reload Ref/Util/XS.pm aborted.", hiding the compilation error.
> Currently, Module::Runtime handles optional loading by parsing the
> error message given by perl to detect "Can't locate" errors.
>
> This could be considered part of the general problem perl has with its
> errors being strings. A proper exception system would allow filtering
> these trivially. It could also be considered entirely separate from
> loading a stringy module name, since it applies equally to require
> with a bareword module name.
>
> A simple "require $module_name" may be the most common case, but I
> think optional loading is too common to ignore if we're going to add
> this to core.

If I may, another suggestion/request: have the built-in return the namespace itself. This way it’s easy to do stuff like:

my $class = 'My::Class';
my $thing = need($class)->new();

-FG

On Sun, Sep 12, 2021 at 12:23 PM Felipe Gasper <felipe@felipegasper.com>
wrote:

>
> If I may, another suggestion/request: have the built-in return the
> namespace itself. This way it’s easy to do stuff like:
>
> my $class = 'My::Class';
> my $thing = need($class)->new();
>

This is easily sensible for the version which only returns or dies like
require, which does not have a need for a significant return value
(use_module from Module::Runtime does this). A version which uses the
return value to indicate whether it was successful could also use the
package name as the "success" return value, only because '' and '0' are not
valid package names.

-Dan

> On Sep 12, 2021, at 12:36, Dan Book <grinnz@gmail.com> wrote:
>
> ?
>> On Sun, Sep 12, 2021 at 12:23 PM Felipe Gasper <felipe@felipegasper.com> wrote:
>>
>> If I may, another suggestion/request: have the built-in return the namespace itself. This way it’s easy to do stuff like:
>>
>> my $class = 'My::Class';
>> my $thing = need($class)->new();
>
> This is easily sensible for the version which only returns or dies like require, which does not have a need for a significant return value (use_module from Module::Runtime does this). A version which uses the return value to indicate whether it was successful could also use the package name as the "success" return value, only because '' and '0' are not valid package names.

There are enough reasons for “I can’t do that, Dave” when loading a module that ISTM a return-in-failure would have granularity problems.

I think the most natural interface is computer-friendly exceptions of some kind: objects, code-prefixed strings, etc.

-F

On Sun, Sep 12, 2021 at 1:27 PM Felipe Gasper <felipe@felipegasper.com>
wrote:

>
>
> On Sep 12, 2021, at 12:36, Dan Book <grinnz@gmail.com> wrote:
>
> ?
> On Sun, Sep 12, 2021 at 12:23 PM Felipe Gasper <felipe@felipegasper.com>
> wrote:
>
>>
>> If I may, another suggestion/request: have the built-in return the
>> namespace itself. This way it’s easy to do stuff like:
>>
>> my $class = 'My::Class';
>> my $thing = need($class)->new();
>>
>
> This is easily sensible for the version which only returns or dies like
> require, which does not have a need for a significant return value
> (use_module from Module::Runtime does this). A version which uses the
> return value to indicate whether it was successful could also use the
> package name as the "success" return value, only because '' and '0' are not
> valid package names.
>
>
> There are enough reasons for “I can’t do that, Dave” when loading a module
> that ISTM a return-in-failure would have granularity problems.
>
> I think the most natural interface is computer-friendly exceptions of some
> kind: objects, code-prefixed strings, etc.
>

Yes, exception objects would be a solution. But I am talking about the
interface described in the rest of the thread. Exceptions would be thrown
for issues other than missing module or insufficient version; only in those
cases would false be returned.

-Dan

On Fri, Sep 3, 2021, at 3:26 PM, Karen Etheridge wrote:
> On Fri, Sep 3, 2021 at 10:49 AM Dan Book <grinnz@gmail.com> wrote:
>>
>> haarg was working on a Module::Runtime replacement that has this capability with a function called "try_require_module", which returns true on success, false on not found or insufficient version, and an exception on other errors. I think it is a great API, but unfortunately has not been put through the rigors of CPAN workshopping yet.
>
> IMO the toolchain gang should take on the goal of producing an RFC. Graham has done much of the work (investigative, as well as code) on this already so we should be able to get to that point "soon".

Graham wrote into this thread, but I don't think it seems like we're expecting an RFC, so I'd rather this not stagnate.

It seems like the thrust of Graham's post was "yes, a `load STRING` would be useful, but we want to distinguish between kinds of failure." I agree, and I think we want that from `require`, too, and I also think that "an exception system would solve this" is the kind of thing that can stymie a lot of stuff because it wants to plumb stuff into all parts of Perl.

Instead, I propose that the next step here is to provide a load-at-runtime built-in that expects the name of a module and dies on error. At some future time, if we overhaul exceptions, we can fix both it and "require" then. In the meantime, I don't think it's economical to make load-at-runtime wait on an exception overhaul.

Thoughts? Shall I write the RFC?

--
rjbs

Ricardo Signes writes:

> I propose that the next step here is to provide a load-at-runtime
> built-in that expects the name of a module and dies on error. At some
> future time, if we overhaul exceptions, we can fix both it and
> "require" then. In the meantime, I don't think it's economical to
> make load-at-runtime wait on an exception overhaul.
>
> Thoughts?

Sounds wonderful.

> Shall I write the RFC?

Yes please!

Smylers

On Sun, 12 Sep 2021 12:35:19 -0400
Dan Book <grinnz@gmail.com> wrote:

> only because
> '' and '0' are not valid package names.

Au contraire:

https://metacpan.org/dist/Scalar-Defer/source/lib/Scalar/Defer.pm#L16

https://metacpan.org/dist/Scalar-Defer/source/lib/Scalar/Defer.pm#L137

https://metacpan.org/dist/Scalar-Defer/source/lib/Scalar/Defer.pm#L25

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/

On Fri, Jan 28, 2022 at 8:58 AM Paul "LeoNerd" Evans <leonerd@leonerd.org.uk>
wrote:

> On Sun, 12 Sep 2021 12:35:19 -0400
> Dan Book <grinnz@gmail.com> wrote:
>
> > only because
> > '' and '0' are not valid package names.
>
> Au contraire:
>
> https://metacpan.org/dist/Scalar-Defer/source/lib/Scalar/Defer.pm#L16
>
> https://metacpan.org/dist/Scalar-Defer/source/lib/Scalar/Defer.pm#L137
>
> https://metacpan.org/dist/Scalar-Defer/source/lib/Scalar/Defer.pm#L25


I don't remember the context but I was probably being imprecise - they
cannot be declared in a package statement or written to fully-qualify a
symbol, but of course you can pass whatever you want to bless.

-Dan

On Fri, Sep 3, 2021, at 12:48, Ricardo Signes wrote:
> Is it enough to say "v5.x will provide `load`, which will take a module name and optional version, then load that module (or die trying) and call ->VERSION on it.

This became a formal proposal <https://github.com/Perl/RFCs/blob/main/rfcs/rfc0006.md>, which I think is more or less ready to implement. *Who would like to implement it?* Many people who write in Perl 5 would benefit from such a builtin, and I believe it should be relatively straightforward.

If we don't have any implementation in progress in the next couple months, this proposal will be dumped into Expired. status.

--
rjbs

On Fri, 16 Sep 2022 09:54:27 -0400
"Ricardo Signes" <perl.p5p@rjbs.manxome.org> wrote:

> This became a formal proposal
> <https://github.com/Perl/RFCs/blob/main/rfcs/rfc0006.md>, which I
> think is more or less ready to implement. *Who would like to
> implement it?* Many people who write in Perl 5 would benefit from
> such a builtin, and I believe it should be relatively straightforward.

I don't want to add it to my plate of things, but I'd be happy to
handhold a volunteer through the overall process.

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/

IMHO better solution will be to provide namespace specific methods, eg:
Foo::Bar::->*is_loaded
Foo::Bar::->*load

grammar rule:
NAMESPACE_LITERAL ARROW STAR method

and so on

Advantage:
- namespace methods will not pollute namespace for other methods
- namespace methods will always be prefixed with *

Best regards,
Brano

On 2022-09-16 7:44 a.m., Branislav Zahradník wrote:
> IMHO better solution will be to provide namespace specific methods, eg:
> Foo::Bar::->*is_loaded
> Foo::Bar::->*load
>
> grammar rule:
> NAMESPACE_LITERAL ARROW STAR method
>
> and so on
>
> Advantage:
> - namespace methods will not pollute namespace for other methods
> - namespace methods will always be prefixed with *

Is this a situation where we want to start having special syntax for particular
method calls? If so, we probably want to look at such a feature more
holistically, not just for this use case. I also recall discussions of
something similar, specifically adding Raku-like twigils to Perl, was rejected
in Corinna design discussions in respect to object attributes. Also, how would
this proposed design work in situations where the package name is only known at
runtime, such as because it came from a configuration file? -- Darren Duncan

On 2022-09-16 5:50 p.m., Darren Duncan wrote:
> On 2022-09-16 7:44 a.m., Branislav Zahradník wrote:
>> IMHO better solution will be to provide namespace specific methods, eg:
>> Foo::Bar::->*is_loaded
>> Foo::Bar::->*load
>>
>> grammar rule:
>> NAMESPACE_LITERAL ARROW STAR method
>>
>> and so on
>>
>> Advantage:
>> - namespace methods will not pollute namespace for other methods
>> - namespace methods will always be prefixed with *
>
> Is this a situation where we want to start having special syntax for particular
> method calls?  If so, we probably want to look at such a feature more
> holistically, not just for this use case.  I also recall discussions of
> something similar, specifically adding Raku-like twigils to Perl, was rejected
> in Corinna design discussions in respect to object attributes.  Also, how would
> this proposed design work in situations where the package name is only known at
> runtime, such as because it came from a configuration file? -- Darren Duncan

I feel the simplest solution is just alter the original RFC to be
builtin::load($package_name) etc rather than unqualified "load". That seems the
most appropriate and simplest way to do this. This is the kind of thing that
builtin:: is for. -- Darren Duncan

> > Advantage:
> > - namespace methods will not pollute namespace for other methods
> > - namespace methods will always be prefixed with *
>
> Is this a situation where we want to start having special syntax for particular
> method calls?

these are no ordinary methods, these are methods invoked on internal data.
It's similar to My::Class::->meta->..., but instead of having meta
method (polluting user's namespace)
it will signal that this is internal method.

As far as Perl doesn't distinguish between namespace and instance of
namespace special syntax
is needed. Either non-OOP (using builtin) or this prefixed. Having *
prefix is cheap (from implementation
of grammar) and yet it provides reading hints, eg:

Module::Foo::->*load;
Module::Foo::->load;

Executing following on metacpan-cpan-extracted:
ack -l 'sub load\b' | fgrep -v 'Module::Install' | wc -l
5546

There are more questions about this, a few:
- do you want to be more OOP friendly ?
- do you want to support only `load $module` or something else as well?
- do you want to support per-namespace overload?
- do you want another module methods
- related by not part of `load` -> anonymous namespaces

# OOP:

There are many other operations one would like to execute on
namespace, eg (Moose):
Module::Foo::->meta->add_method
Module::Foo::->meta->get_method_list

do you want all of them in builtin namespace?

# load something else:

in case it will reside in builtin, it should say what it should load (and how)
Examples for module:

builtin::load_module (Foo::);
builtin::load_module (Foo::, dies => 0); # returns true/false instead,
yet still populating $@)

# per-namespace overload

Makes no sense for `'load_module` but may for others, eg (showing
intention, no real implementation):

package Foo {
sub AUTOLOAD { ... }
sub * get_method_list {
qw[ foo bar baz ];
}
}

I already wrote pre-rfc about namespace literals, see "Pre-RFC:
namespace literal as first class citizen" (sent 2022-06-30)

Brano

On 2022-09-17 12:50 a.m., Branislav Zahradník wrote:
>>> Advantage:
>>> - namespace methods will not pollute namespace for other methods
>>> - namespace methods will always be prefixed with *
>>
>> Is this a situation where we want to start having special syntax for particular
>> method calls?
>
> these are no ordinary methods, these are methods invoked on internal data.
> It's similar to My::Class::->meta->..., but instead of having meta
> method (polluting user's namespace)
> it will signal that this is internal method.

What you're saying here sounds wrong to me. Its coming across like these
methods are unique and nothing like them has ever existed in Perl before.

Also the whole idea of "internal data" as distinct from regular data doesn't
make sense. In reality there is no such distinction. I say that every kind of
data made visible to users of ordinary Perl code is normal data.

Also the existing method of "builtin::foo" already prevents pollution of the
user's namespace, so why not just use that instead of introducing this new
syntax? I feel that the bar for adding this star-syntax needs to be much
higher, it should be saved for much more valuable use cases where it would be
used very frequently.

> As far as Perl doesn't distinguish between namespace and instance of
> namespace special syntax
> is needed. Either non-OOP (using builtin) or this prefixed.

While I'm a strong supporter of having stronger typing in Perl, in the meantime
why is it not good enough for a regular string scalar like `"Module::Foo"` not
suffice to represent a namespace?

Especially since Perl already does this, as far as I know, which is why you can
say `$class->foo()` etc.

I would also argue that creating special Perl syntax for namespace literals
should be done FIRST, and BEFORE considering this star-syntax.

So we currently have literal syntax for integers, floats, and strings, and
booleans and undef in a way, so lets add a new literal syntax for namespace
literals, and for that matter it should have its own core data type of course,
like how int/float/string/bool do now.

If we have an actual distinct type for a namespace, would that mean the
rationale for the star-syntax goes away, or do you still see its important
regardless?

> # per-namespace overload
>
> Makes no sense for `'load_module` but may for others, eg (showing
> intention, no real implementation):

I believe the core implementation of builtin::load_module() etc should NOT be
overloadable, and the builtin version has only one behavior.

If we want multiple fundamental behaviors, that's provided by having either
multiple distinctly named builtin::foo() and/or some of them take extra
arguments to determine it.

Beyond that, anyone is free to define other alternate loading behaviors in Perl
as wrappers over the builtins, rather than overloads.

> I already wrote pre-rfc about namespace literals, see "Pre-RFC:
> namespace literal as first class citizen" (sent 2022-06-30)

Glad to hear it. I have yet to read the details of your proposal but in
principle would support that much more than the star-call workaround for their
absense or builtin:: option.

-- Darren Duncan

On Fri, 3 Sept 2021, 17:49 Ricardo Signes, <perl.p5p@rjbs.manxome.org>
wrote:

> Porters,
>
> For unexciting reasons, Module::Runtime has come up a few times on recent
> PSC calls. We talked about Module::Runtime, Class::Load, and I don't know
> how many other related topics.
>
> We ended up with this question: Isn't this something the core should
> provide?
>


I'm confused. The core does provide this. It's called require/use in eval.

I don't see what a builtin would add.



> Now, I'm not saying the core should provide every single thing provided by
> all the modules in this problem area. I'm saying that maybe 99% of use of
> these libraries would be addressed by providing one or two small
> concessions in the core language. I don't know what the second concession
> of that one or two is, but I know the first one is acknowledging that the
> current behavior of "require EXPR" isn't as useful as it could be. When
> given a string, it's much more common to want to load a module, not a file.
>

eval "use $module; 1"

Works fine.

>
> So, let's say we were to implement a new require-like builtin that we
> wanted to be a core language feature and live forever. Would it be
> something other than something really close to Module::Runtime's
> use_module? I have some thoughts, but I think the basic question here is:
>
> Is it enough to say "v5.x will provide load, which will take a module
> name and optional version, then load that module (or die trying) and call
> ->VERSION on it.
>

Weird. In all 20 years I've been hacking perl I don't think I ever wanted
to do that even once.

Yves

On 2022-09-17 2:37 a.m., demerphq wrote:
> On Fri, 3 Sept 2021, 17:49 Ricardo Signes wrote:
> For unexciting reasons, Module::Runtime has come up a few times on recent
> PSC calls.  We talked about Module::Runtime, Class::Load, and I don't know
> how many other related topics.
>
> We ended up with this question:  Isn't this something the core should provide?
>
> I'm confused. The core does provide this. It's called require/use in eval.
>
> I don't see what a builtin would add.
>
> eval "use $module; 1"
>
> Works fine.

Why? Its because having to use string eval for something so simple and basic is
a TERRIBLE design, a huge code smell. The proposal of a proper basic
runtime-defined load that DOESN'T use string eval is sorely needed. This is
insecure and error prone like SQL injection from straight interpolation of
variables into the raw source when it should just be a plain value or argument.
-- Darren Duncan

On Sat, 17 Sep 2022 03:53:53 -0700
Darren Duncan <darren@darrenduncan.net> wrote:

> > I don't see what a builtin would add.
> >
> > eval "use $module; 1"
> >
> > Works fine.
>
> Why? Its because having to use string eval for something so simple
> and basic is a TERRIBLE design, a huge code smell. The proposal of a
> proper basic runtime-defined load that DOESN'T use string eval is
> sorely needed. This is insecure and error prone like SQL injection
> from straight interpolation of variables into the raw source when it
> should just be a plain value or argument.

+1. (Well, +2 if I could ;) )

Plus also Yves' code didnt' handle errors; you forgot to

eval "use $module; 1" or die $@;

and already we see why

builtin::load $module;

would be much nicer.

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/

On Fri, 16 Sep 2022 17:55:36 -0700
Darren Duncan <darren@darrenduncan.net> wrote:

> I feel the simplest solution is just alter the original RFC to be
> builtin::load($package_name) etc rather than unqualified "load".
> That seems the most appropriate and simplest way to do this. This is
> the kind of thing that builtin:: is for.

Oh, I hadn't quite picked up or commented on that subtlety.

The original RFC was mostly written before the "builtin" mechanism was
finalised. I think in everyone's head who was looking at the original
writing, we all imagined it would be in `builtin::` - I've just been
assuming that while reading it and hadn't actually noticed.

If it isn't explicitly spelled out in words, then yes it ought to be so.

builtin::load($module);

or

use builtin 'load';
load($module);

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/

On Sat, 17 Sept 2022, 11:57 Paul "LeoNerd" Evans, <leonerd@leonerd.org.uk>
wrote:

> On Sat, 17 Sep 2022 03:53:53 -0700
> Darren Duncan <darren@darrenduncan.net> wrote:
>
> > > I don't see what a builtin would add.
> > >
> > > eval "use $module; 1"
> > >
> > > Works fine.
> >
> > Why? Its because having to use string eval for something so simple
> > and basic is a TERRIBLE design, a huge code smell. The proposal of a
> > proper basic runtime-defined load that DOESN'T use string eval is
> > sorely needed. This is insecure and error prone like SQL injection
> > from straight interpolation of variables into the raw source when it
> > should just be a plain value or argument.
>
> +1. (Well, +2 if I could ;) )
>
> Plus also Yves' code didnt' handle errors; you forgot to
>
> eval "use $module; 1" or die $@;


I didn't forget anything. I assumed anyone on this list understood that was
implied by the 1. I'm on a mobile phone, typing code is a pita. Sorry I
didn't spell it out.

So all I see is a bunch of folks sticking this load function in an eval.

Whatever, I don't care that much. But please don't call it "load".
"module_load" or "package_load".

I'm a serous anti-fan of lazy loading patterns in library level code,
especially in a web context it is an antipattern that slows things down and
consumes excess memory. I understand and appreciate that for scripts it can
be the opposite. One thing that would sell me on adding this if we came up
with a way to resolve the conflict between those two use cases. Can we
figure out a way that a module consumer can reliably tell modules that they
should *pre*load its dependencies and not lazy load them?

Yves

>
>

On 2022-09-17 7:02 a.m., demerphq wrote:
> So all I see is a bunch of folks sticking this load function in an eval.

I interpret this that you see many people not understanding how to use the new
function properly and that they incorrectly assume they have to put it in an
eval to make it work like "require $foo" etc. So that's up to education.

> Whatever, I don't care that much. But please don't call it "load". "module_load"
> or "package_load".

Personally I'm not stuck on any particular name, and I welcome a separate
discussion to bikeshed the name, but on the RFC itself.

> I'm a serous anti-fan of lazy loading patterns in library level code, especially
> in a web context it is an antipattern that slows things down and consumes excess
> memory. I understand and appreciate that for scripts it can be the opposite. One
> thing that would sell me on adding this if we came up with a way to resolve the
> conflict between those two use cases. Can we figure out a way that a module
> consumer can reliably tell modules that they should *pre*load its dependencies
> and not lazy load them?

A key usage scenario I see for these new methods is when we have a library with
a plugin architecture such that the specific plugin module we want to use is not
known except at runtime and it is a user configuration data file etc that is
naming the module to use.

For example, it would be a cleaner way to implement something like the DBI
module which can use the new methods to load any DBMS driver module the user
names via connect().

For another example, it would be a cleaner way to implement localization type
modules that implement each user language as a Perl module, an example being my
Locale::KeyedText module on CPAN.

With the new builtins available, I wouldn't have to resort to these shenanigans:

sub template_module_is_loaded {
my ($self, $module_name) = @_;
...
no strict 'refs';
return scalar keys %{$module_name . '::'};
}

sub load_template_module {
my ($self, $module_name) = @_;
...
# Note: We have to invoke this 'require' in an eval string
# because we need the bareword semantics, where 'require'
# will munge the package name into file system paths.
eval "require $module_name;";
$self->_die_with_msg( ... )
if $@;
return;
}

-- Darren Duncan

> > these are no ordinary methods, these are methods invoked on internal data.
> > It's similar to My::Class::->meta->..., but instead of having meta
> > method (polluting user's namespace)
> > it will signal that this is internal method.
>
> What you're saying here sounds wrong to me. Its coming across like these
> methods are unique and nothing like them has ever existed in Perl before.

I will use example from raku and its Grammar, problem I ran into
whenwriting SQL parser:
- you cannot create token CREATE, because it is used by language
- you cannot create token before, because class method before is used
by language to handle lookaround

With star syntax it will be clearly visible
- this method is user method (and part its intended interface)
- this method is a hook defined by language to augment default behaviour

That's what I meant mentioning "pollute user's namespace" and "internal data"

> While I'm a strong supporter of having stronger typing in Perl, in the meantime
> why is it not good enough for a regular string scalar like `"Module::Foo"` not
> suffice to represent a namespace?

Perl already supports trailing ::, even with special warning, so that
is good candidate
for namespace literal (more details in mentioned pre-rfc), eg:
Module::Foo::

> Especially since Perl already does this, as far as I know, which is why you can
> say `$class->foo()` etc.

yes, and that's reason why I suggested star syntax, to distinguish
static methods called on class
and methods called on namespace itself

> I would also argue that creating special Perl syntax for namespace literals
> should be done FIRST, and BEFORE considering this star-syntax.

Agree, though these works are independent

> If we have an actual distinct type for a namespace, would that mean the
> rationale for the star-syntax goes away, or do you still see its important
> regardless?

I think star-syntax (or other variant, I used star only because it
looks nice on my monitor)
is important regardless of implementation namespace literal - this
methods are not parts
of given namespace API.

> I believe the core implementation of builtin::load_module() etc should NOT be
> overloadable, and the builtin version has only one behavior.
>
> If we want multiple fundamental behaviors, that's provided by having either
> multiple distinctly named builtin::foo() and/or some of them take extra
> arguments to determine it.

builtin methods are pretty similar to operators

builtin methods should be overloadable (though load_module will not be
because it will be executed before any overload will be defined)

Best regards,
Brano

On Sat, 17 Sept 2022, 21:31 Darren Duncan, <darren@darrenduncan.net> wrote:

> On 2022-09-17 7:02 a.m., demerphq wrote:
> > So all I see is a bunch of folks sticking this load function in an eval.
>
> I interpret this that you see many people not understanding how to use the
> new
> function properly and that they incorrectly assume they have to put it in
> an
> eval to make it work like "require $foo" etc. So that's up to education.
>

Er, no. It's a direct consequence of Paul's comment that load should die if
it fails. I start to get the impression you folks advocating for this
aren't on the same page with each other.

> Whatever, I don't care that much. But please don't call it "load".
> "module_load"
> > or "package_load".
>
> Personally I'm not stuck on any particular name, and I welcome a separate
> discussion to bikeshed the name, but on the RFC itself.
>
> > I'm a serous anti-fan of lazy loading patterns in library level code,
> especially
> > in a web context it is an antipattern that slows things down and
> consumes excess
> > memory. I understand and appreciate that for scripts it can be the
> opposite. One
> > thing that would sell me on adding this if we came up with a way to
> resolve the
> > conflict between those two use cases. Can we figure out a way that a
> module
> > consumer can reliably tell modules that they should *pre*load its
> dependencies
> > and not lazy load them?
>
> A key usage scenario I see for these new methods is when we have a library
> with
> a plugin architecture such that the specific plugin module we want to use
> is not
> known except at runtime and it is a user configuration data file etc that
> is
> naming the module to use.
>

This is exactly the kind of antipattern I object to in a library and
exactly the kind of thing that if you are doing performance sensitive multi
process stuff that causes serious problems and is pretty much exactly a
description of the kind of problem I'd like to see solved.

>
> For example, it would be a cleaner way to implement something like the DBI
> module which can use the new methods to load any DBMS driver module the
> user
> names via connect().
>

I don't consider that good practice so it doesnt really sell me. If you are
using a specific dbd then use it. Sql isn't that portable and DBIs
abstraction does not insulate you from the portability issues that come
from SQL.

>
> For another example, it would be a cleaner way to implement localization
> type
> modules that implement each user language as a Perl module, an example
> being my
> Locale::KeyedText module on CPAN.
>
> With the new builtins available, I wouldn't have to resort to these
> shenanigans:
>
> sub template_module_is_loaded {
> my ($self, $module_name) = @_;
> ...
> no strict 'refs';
> return scalar keys %{$module_name . '::'};
> }
>

This just looks like bad api design to me. If I needed something like that
I'd ensure there is a well defined method to call in the module that
returned a list of whatever was needed. Introspecting on the package stash
doesn't prove anything. Anybody can do 'sub Module::thing {}' anywhere and
break this logic.

I don't see how it relates to having a load builtin either. Loading
package foo doesn't mean that any code will end up in the foo namespace.
Package foo can populate any or no namespaces. Package names and stashes
are only coincidentally related to each other, and are not necessarily
related.



> sub load_template_module {
> my ($self, $module_name) = @_;
> ...
> # Note: We have to invoke this 'require' in an eval string
> # because we need the bareword semantics, where 'require'
> # will munge the package name into file system paths.
> eval "require $module_name;";
> $self->_die_with_msg( ... )
> if $@;
> return;
> }
>

I've done things like this, but I''d never touch $@ without a false return
from an eval. Even if it might work in modern perls it's just confusing to
newbies. Treating $@ like $! Is good practice. I've seen maintainer coders
stick arbitrary code in between the eval and $@ access, which as $@ is
volatile can completely change the result. I'd consider that a bug waiting
to happen if it were in a communal code base.

So, it's not clear to me what you expect. At the start of this mail you
suggested that the proposed builtin would not need an eval, but here you
gave a function that rethrows the exception and would require an eval if I
speculatively want to load something, a pattern which I have seen far more
than the cases you describe. Eg "use this module and if it's present import
a sub, and if it's not use a roll-your-own replacement".

So does this proposed new function die if it fails to load or not?

Yves

On 2022-09-17 2:29 p.m., demerphq wrote:
> On Sat, 17 Sept 2022, 21:31 Darren Duncan wrote:
>
> On 2022-09-17 7:02 a.m., demerphq wrote:
> > So all I see is a bunch of folks sticking this load function in an eval.
>
> I interpret this that you see many people not understanding how to use the new
> function properly and that they incorrectly assume they have to put it in an
> eval to make it work like "require $foo" etc.  So that's up to education.
>
> Er, no. It's a direct consequence of Paul's comment that load should die if it
> fails. I start to get the impression you folks advocating for this aren't on the
> same page with each other.

There is no conflict here because I assume there would be multiple versions of
the functionality and users can pick the version that does or doesn't die on
failure depending on what they want.

> A key usage scenario I see for these new methods is when we have a library with
> a plugin architecture such that the specific plugin module we want to use is
> not
> known except at runtime and it is a user configuration data file etc that is
> naming the module to use.
>
> This is exactly the kind of antipattern I object to in a library and exactly the
> kind of thing that if you are doing performance sensitive multi process stuff
> that causes serious problems and is pretty much exactly a description of the
> kind of problem I'd like to see solved.

I'm not saying a plugin architecture is always appropriate, but there are
legitimate reasons for a core tool to not have to have knowledge of every
third-party thing that might be used with it.

Why should Adobe Photoshop ship with a list of all the possible third-party
plugins baked into its source code?

> For example, it would be a cleaner way to implement something like the DBI
> module which can use the new methods to load any DBMS driver module the user
> names via connect().
>
> I don't consider that good practice so it doesnt really sell me. If you are
> using a specific dbd then use it. Sql isn't that portable and DBIs abstraction
> does not insulate you from the portability issues that come from SQL.

I actually agree with you, and feel that DBI's design is backwards. Instead a
better design is that DBI is just an interface definition that each database
driver adopts, and that users should just be using the driver module directly,
eg they say "use DBD::Oracle" and "DBD::Oracle->connect()" or whatever.

And despite appearances, DBI is actually worse because it actually does have
intimate knowledge of possible drivers, a list of such baked into its source
code, or it did last I looked.

However, I raised this as a well known example of a plugin architecture.

> For another example, it would be a cleaner way to implement localization type
> modules that implement each user language as a Perl module, an example being my
> Locale::KeyedText module on CPAN.
>
> With the new builtins available, I wouldn't have to resort to these shenanigans:
...
> This just looks like bad api design to me. If I needed something like that I'd
> ensure there is a well defined method to call in the module that returned a list
> of whatever was needed. Introspecting on the package stash doesn't prove
> anything. Anybody can do 'sub Module::thing {}' anywhere and break this logic.
>
>  I don't see how it relates to having a load builtin either. Loading package
> foo doesn't mean that any code will end up in the foo namespace. Package foo can
> populate any or no namespaces. Package names and stashes are only coincidentally
> related to each other, and are not necessarily related.
...
> I've done things like this, but I''d never touch $@ without a false return from
> an eval. Even if it might work in modern perls it's just confusing to newbies.
> Treating $@ like $! Is good practice. I've seen maintainer coders stick
> arbitrary code in between the eval and $@ access, which as $@ is volatile can
> completely change the result. I'd consider that a bug waiting to happen if it
> were in a communal code base.
>
> So, it's not clear to me what you expect. At the start of this mail you
> suggested that the proposed builtin would not need an eval, but here you gave a
> function that rethrows the exception and would require an eval if I
> speculatively want to load something, a pattern which I have seen far more than
> the cases you describe. Eg "use this module and if it's present import a sub,
> and if it's not use a roll-your-own replacement".

My mentioning Locale::KeyedText was just meant to be another example of how
checking for the use of a package use or loading one is much more contrived than
it ought to be. The fact that I happen to throw an exception on failure is
beside the point.

Also, Locale::KeyedText is an old CPAN module I no longer update or expect
anyone to use, and it doesn't reflect the latest good design. It was based on
trying to make available the MacOS feature of externally installable language
resources for applications where no language is privileged over others, eg no
replacing English text with other languages.

> So does this proposed new function die if it fails to load or not?

I assume there would be multiple versions so users can choose.

The whole proposal is for a set of routines, not just one.

-- Darren Duncan

On Fri, Sep 16, 2022 at 10:13 AM Paul "LeoNerd" Evans
<leonerd@leonerd.org.uk> wrote:
>
> On Fri, 16 Sep 2022 09:54:27 -0400
> "Ricardo Signes" <perl.p5p@rjbs.manxome.org> wrote:
>
> > This became a formal proposal
> > <https://github.com/Perl/RFCs/blob/main/rfcs/rfc0006.md>, which I
> > think is more or less ready to implement. *Who would like to
> > implement it?* Many people who write in Perl 5 would benefit from
> > such a builtin, and I believe it should be relatively straightforward.
>
> I don't want to add it to my plate of things, but I'd be happy to
> handhold a volunteer through the overall process.

I took a stab at it here:

https://github.com/Perl/perl5/pull/20316

-- Matthew Horsfall (alh)

On 2022-09-18 7:06 p.m., Matthew Horsfall (alh) wrote:
> On Fri, Sep 16, 2022 at 10:13 AM Paul "LeoNerd" Evans
> <leonerd@leonerd.org.uk> wrote:
>>
>> On Fri, 16 Sep 2022 09:54:27 -0400
>> "Ricardo Signes" <perl.p5p@rjbs.manxome.org> wrote:
>>
>>> This became a formal proposal
>>> <https://github.com/Perl/RFCs/blob/main/rfcs/rfc0006.md>, which I
>>> think is more or less ready to implement. *Who would like to
>>> implement it?* Many people who write in Perl 5 would benefit from
>>> such a builtin, and I believe it should be relatively straightforward.
>>
>> I don't want to add it to my plate of things, but I'd be happy to
>> handhold a volunteer through the overall process.
>
> I took a stab at it here:
>
> https://github.com/Perl/perl5/pull/20316
>
> -- Matthew Horsfall (alh)

Thank you for doing that.

Are you planning to also do other related routines besides "load" itself?

Also there's still the open question of whether we want this to have a different
name for "load", however changing that can be done in a subsequent pull request
rather than holding up this one.

-- Darren Duncan

On Sun, Sep 18, 2022, at 22:06, Matthew Horsfall (alh) wrote:
> I took a stab at it here:
>
> https://github.com/Perl/perl5/pull/20316

???? Thanks, Matthew! I'll build this soon.

--
rjbs

On Sun, Sep 18, 2022, at 23:34, Darren Duncan wrote:
> Are you planning to also do other related routines besides "load" itself?

The RFC calls for no other related routines.

> Also there's still the open question of whether we want this to have a different
> name for "load", however changing that can be done in a subsequent pull request
> rather than holding up this one.

The specified name is "load".

--
rjbs

> Also there's still the open question of whether we want this to have a different
> name for "load", however changing that can be done in a subsequent pull request
> rather than holding up this one.
>
>
> The specified name is "load".
>

so there will never be anything else that can be loaded ?
(eg: XSloader move to builting)

> --
> rjbs

On Mon, Sep 19, 2022, at 09:00, Branislav Zahradník wrote:
> > The specified name is "load".
>
> so there will never be anything else that can be loaded ?
> (eg: XSloader move to builting)

"Nothing else will ever be loaded" does not follow from "this case is called load."

--
rjbs

On Mon, 19 Sept 2022, 14:38 Ricardo Signes, <perl.p5p@rjbs.manxome.org>
wrote:

> On Mon, Sep 19, 2022, at 09:00, Branislav Zahradník wrote:
>
> > The specified name is "load".
>
> so there will never be anything else that can be loaded ?
> (eg: XSloader move to builting)
>
>
> "Nothing else will ever be loaded" does not follow from "this case is
> called load."
>

But it makes adding new cases more confusing and will be a builtin that
will clash with many preexisting subs with that name. It should be called
something more specific.

Yves

I wonder if it wouldn’t be more flexible to have

module_file_of (‘Foo::Bar’);

and then allow that to be loaded into require or used for any other desired purpose, rather than having a special “load” routine.

e.g.,

my $x = module_name_of('Foo::Bar');
require $x;

would be equivalent to

require Foo::Bar;



Or, maybe not.

To be honest, I’m not sure to what other purposes this function could be put, but it strikes me as the wrong thing to have some kind of data munging that can only be accessed through a separate function (as though we only had printf and not sprintf).

--
Aaron Priven, aaron@priven.com, www.priven.com/aaron

On Mon, 19 Sept 2022 at 18:36, Aaron Priven <aaron@priven.com> wrote:
>
> I wonder if it wouldn’t be more flexible to have
>
> module_file_of (‘Foo::Bar’);

+1

this is not bad idea

>
> and then allow that to be loaded into require or used for any other desired purpose, rather than having a special “load” routine.
>
> e.g.,
>
> my $x = module_name_of('Foo::Bar');
> require $x;
>
> would be equivalent to
>
> require Foo::Bar;
>

if I understand it correctly, load should behave like:
my $loaded = eval { require $x; 1 }

On Sat, 17 Sep 2022 11:58:47 +0100
"Paul \"LeoNerd\" Evans" <leonerd@leonerd.org.uk> wrote:

> If it isn't explicitly spelled out in words, then yes it ought to be
> so.
>
> builtin::load($module);
>
> or
>
> use builtin 'load';
> load($module);

Aaaaactually, having said that, I am now in the middle of starting to
write the metaprogramming API RFC. It would be quite a good natural fit
for that set of API instead.

So hold that thought until I get something draft-worthy online...

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/