Let me know if something like the following exists. I am interested in
helping to marshal whatever iterative *documentation* process might be
involved in production some sort of thing that can be pointed to, in the
vein of "so you want to add a feature..."
I must preface this by saying we're all adults here. At least I think
so. I appreciate everyone's hard work and true kindness in tolerating my
recent spate of mails and also pointing me in the right direction. None
of the following is targeting individual efforts. And its all said out
of appreciation for all that comes with perl/Perl and the community.
The motivation has come to me over the last 12 hours after, reading:
1. the trim PR on github
2.
https://leonerds-code.blogspot.com/2021/02/writing-perl-core-feature.html
(thank you kind person who send me this link)
3. the infix 'isa' implementation
4. https://perlmonks.org/?node_id=861966 (Far More Than Everything
You've Ever Wanted to Know about Prototypes in Perl - by T. Christiansen)
5. spending time "putting pen to paper" with the context switching thing
since I can work out the semantics-context stuff out in perl "userland"
If there is anything that I am envisioning, I have never seen it; so I
will describe it. If something of this nature does exists, I'd like to
evaluate it and see if it's what I would like to see as a total n00b. I
have "fresh" but not "totally ignorant" eyes - and though I have much
ignorance, I will use this as a strength.
The sketch of the document I am envisioning essentially does the following:
I. Find Your Path
This section would attempt to help the reader to determine the end state
of the "feature". E.g., options things like: CPAN, "dual life" (included
in the perl distro), high level "core" feature (ala, 'trim', 'isa' noted
above), perl "run time", perl API, data guts, other low level stuffs
(I'd need help identifying this stack of turtles). I might use the term
"perl userland" to mean "something a perl consumer might want direct
access to in their scripting activities". "perl runtime" would mean what
it does traditionally.
Some paths are related above, some are not. For example if the "goal" is
a "dual life" module, you clearly need to get it on CPAN it needs to
prove its worth. This document wouldn't cover how to "decide when to
promote", only that a promotion is necessary; to be "dual life" one
needs to be promoted from CPAN. And we know well, anyone can put
anything on CPAN.
It would effective present a "decision" tree or a set of Socratic type
questions:
* what functionality is desired?
* where does this functionality fit best (userland, runtime, "kernel", etc)?
* how does it look to use said functionality?
* does said functionality sit in a higher level, but require fundamental
lower level support?
* what's the highest "level" on which this functionality can be implmented?
* how important is performance to your first iteration?
* really, how important is performance to your first iteration?
* ..
* ...
The goal: find the best place to implement the feature with the desired
interface/semantics so that it can be proven a) unique, b) useful, c)
compelling to use, etc. (more on the "goal" later).
II. Choose Your Weapon
This section assumes the interested party is sufficiently past step 1
and would suggest things like combining "put it on CPAN" + the wisdom in
#4 (prototypes and presenting natural feeling 'keywords'). In any case,
I'd treat this section as completely as my current world view allows.
And I can iterate with anyone interested. But the goal would be to
create a high level perldoc - i.e., if nothing like this exists. If
something like this exists, please show me and I will likely start there
to achieve, in part, what I am envisioning here.
The point of the documentation is to establish a pathway. The point of
the path way is to simultaneously a) find the right "place" for the
feature implementation and b) more easily detect when lower level
support of a feature is needed. The charge to add infix additions is a
*perfect* example that points to extended support for prototypes (and
possibly the 'override' mechanism).
Please let me know the current state of our documentation regarding
this, I am committing to doing this if it would be beneficial - and
guess what, it *doesn't* have to be a perldoc for me to be happy with
it. I just want the document that I think would have really pulled me
in. It just so happens I am very persistent. Or maddeningly hard headed;
different people have vastly different opinions on this topic.
III. Efficiency && Efficacy
It is my estimation that the majority of things that have been
laboriously baked into core or implemented in the way suggested by #2
could have been done as: i) a module, ii) using prototypes as intended
in #4, and iii) being patient, iv) humbly accepting the fact that no one
thinks your function is as cool as you do.
Try::Tiny is clearly to me the best example of this of a success story.
It is the platinum standard, IMO of how it should work. It started as a
module. It unashamedly uses protoypes. And it proved to be one of the
most popular modules in CPAN for reasons related to following the right
path and being an actually useful thing. Look at how it's implemented.
Moreover, consider the interface options. Until last night I zero idea
that I could coerce a block into a subroutine ref, let alone an array of
them. OH THE POWER.
I must take a hard look at two other "things" that took IMO, the less
righteous path; you know what I am going to say: 'trim' and 'isa'. Oh
and 'say'. Of those: 'trim' and 'say' could have been modules leveraging
prototypes "templating" ability. 'isa' "should have been" a module - but
it could not because prototypes simply do not support the infix
notation. To make this point *even* stronger, I looked slack jawed at
was was required to implement 'trim' in the PR. Assuming this was
necessary (I can't be the judge there); the complexity of the code was
unbelievable; almost as if it was necessary to implement a regex parser
inline. That this was required points more strongly to the failings of
the perl API to implement something so conceptually simple. But is also
clearly points to where 'trim' really belongs. In that place where one
can easily reach for regular expressions.
I think we could avoid (and could have avoided) a tremendous amount
"misfeatures" and "misappropriations" of the past if there was a clear
and judicious path to adding features.
In summary:
0) The goal should always tend "up" in terms of where the implementation
is. The lower level one goes, the less it should be changing. If a
feature's "place" is clear, but that place doesn't provide sufficiently
high level support, *THEN* is when you go under the covers. An example
of this is #4 provides an outlet for things that should act like "perl
keywords" (like trim) and clearly points to it being in a module. For
"isa", #4 can't handle infix. So to me, that points to the notion that
infix operators should be able to be handled by #4; but can't. So go
under the covers and make it so it can be. E.g.,the natural way to
extend #4 for infix would be simply to add a template "hook" on the LHS
of the sub name + maybe even an extension to the overload module:
use overload
'===' -> \&eqn;
sub ($) eqn ($) {
my ($a, $b) = @_;
...
return $something;
}
That feels nice and fell out of looking at a little bit of historical
precedence. As a "template" it would be a violation to something like:
sub eqn ($ SOMESPECIALTHING $); # anti-pattern prototype
That is not a template, that is an incoherent declaration to what is
actually wanted. So I ask, is it worth time to implement "pluggable
infix" operators - or to extend prototype template coercion semantics to
support a parameter list on the LHS of the new op? And all things
considered equal, is it not consistent with what we already do to allow
'overload' to take advantage? IMO, *this* is where the effort will be. I
can't speak on the actual work involved, but I can absolutely foresee
the additional benefit and whole host of new infix operators. In short,
we now get to create all sorts of weird things on purpose:
What would this do?
if ($lhs_thing1 '(_8(|)' $rhs_hing2) {
say 'doh!';
}
Or better, and I " really"x1000 hope the text spacing is preserved:
if ($lhs_thing1 q{__&__
/ \
| |
| (o)(o)
C .---_)
| |.___|
| \__/
/_____\
/_____/ \
/ \} $lhs_thing2) {
say 'doh!';
}
The enhanced ascii perl art alone is worth enabling this all by itself.
1) I find it is the "interface" that that people seem to care most
about; some interfaces require lower level support than #4+CPAN can give
-i.e., "perlish" feeling functions and keywords (e.g., try/tiny uses
prototypes and this is really what demonstrated their power). IOW, most
are looking for "creating [a] user-defined functions that behave in much
the same way that Perl's own built-in functions behave with respect to
their effects upon the parser and upon implicit contexts" - trim/tromp
is most definitely in this camp. "isa" is not since it is 'infix' and
now I see why LeoNerd is looking at creating "pluggable" infix - seems
like what is really needed is an extension to prototypes in #4 to
provide infix semantics. Maybe a good lead to follow.
2) Another goal of this process would be to shake out those with less
honorable intents; be the pride, etc. (not accusing anyone here of
that!!!!! to be clear.). But we are human beings and humans are biased
by all sorts of things. If done right, we'll have a nice sieve in place
for eventually and fairly determining where a "feature" in whatever form
*actually* belongs. This saves everyone time, frustration, and mostly
time. Best of all, it would greatly improve the "market testing" phase
of features. It allows us to test more features in more perlish ways;
and basically wherever the process "stops" for a "feature" is where it
should stay. Probably CPAN; but the selected few could make it to dual
life. Really interesting features might demand more low level "under the
covers" support; etc, etc. Seems pretty harmonious to me.
Incidentally, #4 was a game changer for me; even without the other
things I enumerated above. I also do wish they had just called
prototypes, "parameter context templates". Maybe it's not too late to
change the name; seems to reason they are not mutually exclusive with
"function signatures" and are in fact much more "perlish" that the
signature thing, which as far as I can tell has no way to do the "type"
coercion that "parameter context templates" can. Additionally,
"parameter context templates" is meant specifically to, "creating
user-defined functions that behave in much the same way that Perl's own
built-in functions behave with respect to their effects upon the parser
and upon implicit contexts". So maybe the reason why "signatures" have
not been made a default feature is because it seems to not pay proper
respects to what "parameter context templates" are for. I am not
prepared to do an analysis on prototypes vs signatures right now, but I
have started to think about it. Signatures necessary need to be
compatible with prototypes, even if that condition alone is not
sufficient to conclude that saga.
If you made it this far, you probably forgot that this was really about
documenting a sane and humane pathway for features. Let's return to that
and I am ready to get to work on that.
Thank you,
Brett
helping to marshal whatever iterative *documentation* process might be
involved in production some sort of thing that can be pointed to, in the
vein of "so you want to add a feature..."
I must preface this by saying we're all adults here. At least I think
so. I appreciate everyone's hard work and true kindness in tolerating my
recent spate of mails and also pointing me in the right direction. None
of the following is targeting individual efforts. And its all said out
of appreciation for all that comes with perl/Perl and the community.
The motivation has come to me over the last 12 hours after, reading:
1. the trim PR on github
2.
https://leonerds-code.blogspot.com/2021/02/writing-perl-core-feature.html
(thank you kind person who send me this link)
3. the infix 'isa' implementation
4. https://perlmonks.org/?node_id=861966 (Far More Than Everything
You've Ever Wanted to Know about Prototypes in Perl - by T. Christiansen)
5. spending time "putting pen to paper" with the context switching thing
since I can work out the semantics-context stuff out in perl "userland"
If there is anything that I am envisioning, I have never seen it; so I
will describe it. If something of this nature does exists, I'd like to
evaluate it and see if it's what I would like to see as a total n00b. I
have "fresh" but not "totally ignorant" eyes - and though I have much
ignorance, I will use this as a strength.
The sketch of the document I am envisioning essentially does the following:
I. Find Your Path
This section would attempt to help the reader to determine the end state
of the "feature". E.g., options things like: CPAN, "dual life" (included
in the perl distro), high level "core" feature (ala, 'trim', 'isa' noted
above), perl "run time", perl API, data guts, other low level stuffs
(I'd need help identifying this stack of turtles). I might use the term
"perl userland" to mean "something a perl consumer might want direct
access to in their scripting activities". "perl runtime" would mean what
it does traditionally.
Some paths are related above, some are not. For example if the "goal" is
a "dual life" module, you clearly need to get it on CPAN it needs to
prove its worth. This document wouldn't cover how to "decide when to
promote", only that a promotion is necessary; to be "dual life" one
needs to be promoted from CPAN. And we know well, anyone can put
anything on CPAN.
It would effective present a "decision" tree or a set of Socratic type
questions:
* what functionality is desired?
* where does this functionality fit best (userland, runtime, "kernel", etc)?
* how does it look to use said functionality?
* does said functionality sit in a higher level, but require fundamental
lower level support?
* what's the highest "level" on which this functionality can be implmented?
* how important is performance to your first iteration?
* really, how important is performance to your first iteration?
* ..
* ...
The goal: find the best place to implement the feature with the desired
interface/semantics so that it can be proven a) unique, b) useful, c)
compelling to use, etc. (more on the "goal" later).
II. Choose Your Weapon
This section assumes the interested party is sufficiently past step 1
and would suggest things like combining "put it on CPAN" + the wisdom in
#4 (prototypes and presenting natural feeling 'keywords'). In any case,
I'd treat this section as completely as my current world view allows.
And I can iterate with anyone interested. But the goal would be to
create a high level perldoc - i.e., if nothing like this exists. If
something like this exists, please show me and I will likely start there
to achieve, in part, what I am envisioning here.
The point of the documentation is to establish a pathway. The point of
the path way is to simultaneously a) find the right "place" for the
feature implementation and b) more easily detect when lower level
support of a feature is needed. The charge to add infix additions is a
*perfect* example that points to extended support for prototypes (and
possibly the 'override' mechanism).
Please let me know the current state of our documentation regarding
this, I am committing to doing this if it would be beneficial - and
guess what, it *doesn't* have to be a perldoc for me to be happy with
it. I just want the document that I think would have really pulled me
in. It just so happens I am very persistent. Or maddeningly hard headed;
different people have vastly different opinions on this topic.
III. Efficiency && Efficacy
It is my estimation that the majority of things that have been
laboriously baked into core or implemented in the way suggested by #2
could have been done as: i) a module, ii) using prototypes as intended
in #4, and iii) being patient, iv) humbly accepting the fact that no one
thinks your function is as cool as you do.
Try::Tiny is clearly to me the best example of this of a success story.
It is the platinum standard, IMO of how it should work. It started as a
module. It unashamedly uses protoypes. And it proved to be one of the
most popular modules in CPAN for reasons related to following the right
path and being an actually useful thing. Look at how it's implemented.
Moreover, consider the interface options. Until last night I zero idea
that I could coerce a block into a subroutine ref, let alone an array of
them. OH THE POWER.
I must take a hard look at two other "things" that took IMO, the less
righteous path; you know what I am going to say: 'trim' and 'isa'. Oh
and 'say'. Of those: 'trim' and 'say' could have been modules leveraging
prototypes "templating" ability. 'isa' "should have been" a module - but
it could not because prototypes simply do not support the infix
notation. To make this point *even* stronger, I looked slack jawed at
was was required to implement 'trim' in the PR. Assuming this was
necessary (I can't be the judge there); the complexity of the code was
unbelievable; almost as if it was necessary to implement a regex parser
inline. That this was required points more strongly to the failings of
the perl API to implement something so conceptually simple. But is also
clearly points to where 'trim' really belongs. In that place where one
can easily reach for regular expressions.
I think we could avoid (and could have avoided) a tremendous amount
"misfeatures" and "misappropriations" of the past if there was a clear
and judicious path to adding features.
In summary:
0) The goal should always tend "up" in terms of where the implementation
is. The lower level one goes, the less it should be changing. If a
feature's "place" is clear, but that place doesn't provide sufficiently
high level support, *THEN* is when you go under the covers. An example
of this is #4 provides an outlet for things that should act like "perl
keywords" (like trim) and clearly points to it being in a module. For
"isa", #4 can't handle infix. So to me, that points to the notion that
infix operators should be able to be handled by #4; but can't. So go
under the covers and make it so it can be. E.g.,the natural way to
extend #4 for infix would be simply to add a template "hook" on the LHS
of the sub name + maybe even an extension to the overload module:
use overload
'===' -> \&eqn;
sub ($) eqn ($) {
my ($a, $b) = @_;
...
return $something;
}
That feels nice and fell out of looking at a little bit of historical
precedence. As a "template" it would be a violation to something like:
sub eqn ($ SOMESPECIALTHING $); # anti-pattern prototype
That is not a template, that is an incoherent declaration to what is
actually wanted. So I ask, is it worth time to implement "pluggable
infix" operators - or to extend prototype template coercion semantics to
support a parameter list on the LHS of the new op? And all things
considered equal, is it not consistent with what we already do to allow
'overload' to take advantage? IMO, *this* is where the effort will be. I
can't speak on the actual work involved, but I can absolutely foresee
the additional benefit and whole host of new infix operators. In short,
we now get to create all sorts of weird things on purpose:
What would this do?
if ($lhs_thing1 '(_8(|)' $rhs_hing2) {
say 'doh!';
}
Or better, and I " really"x1000 hope the text spacing is preserved:
if ($lhs_thing1 q{__&__
/ \
| |
| (o)(o)
C .---_)
| |.___|
| \__/
/_____\
/_____/ \
/ \} $lhs_thing2) {
say 'doh!';
}
The enhanced ascii perl art alone is worth enabling this all by itself.
1) I find it is the "interface" that that people seem to care most
about; some interfaces require lower level support than #4+CPAN can give
-i.e., "perlish" feeling functions and keywords (e.g., try/tiny uses
prototypes and this is really what demonstrated their power). IOW, most
are looking for "creating [a] user-defined functions that behave in much
the same way that Perl's own built-in functions behave with respect to
their effects upon the parser and upon implicit contexts" - trim/tromp
is most definitely in this camp. "isa" is not since it is 'infix' and
now I see why LeoNerd is looking at creating "pluggable" infix - seems
like what is really needed is an extension to prototypes in #4 to
provide infix semantics. Maybe a good lead to follow.
2) Another goal of this process would be to shake out those with less
honorable intents; be the pride, etc. (not accusing anyone here of
that!!!!! to be clear.). But we are human beings and humans are biased
by all sorts of things. If done right, we'll have a nice sieve in place
for eventually and fairly determining where a "feature" in whatever form
*actually* belongs. This saves everyone time, frustration, and mostly
time. Best of all, it would greatly improve the "market testing" phase
of features. It allows us to test more features in more perlish ways;
and basically wherever the process "stops" for a "feature" is where it
should stay. Probably CPAN; but the selected few could make it to dual
life. Really interesting features might demand more low level "under the
covers" support; etc, etc. Seems pretty harmonious to me.
Incidentally, #4 was a game changer for me; even without the other
things I enumerated above. I also do wish they had just called
prototypes, "parameter context templates". Maybe it's not too late to
change the name; seems to reason they are not mutually exclusive with
"function signatures" and are in fact much more "perlish" that the
signature thing, which as far as I can tell has no way to do the "type"
coercion that "parameter context templates" can. Additionally,
"parameter context templates" is meant specifically to, "creating
user-defined functions that behave in much the same way that Perl's own
built-in functions behave with respect to their effects upon the parser
and upon implicit contexts". So maybe the reason why "signatures" have
not been made a default feature is because it seems to not pay proper
respects to what "parameter context templates" are for. I am not
prepared to do an analysis on prototypes vs signatures right now, but I
have started to think about it. Signatures necessary need to be
compatible with prototypes, even if that condition alone is not
sufficient to conclude that saga.
If you made it this far, you probably forgot that this was really about
documenting a sane and humane pathway for features. Let's return to that
and I am ready to get to work on that.
Thank you,
Brett