Mailing List Archive

On Fri, Sep 08, 2023 at 12:24:58PM +0200, Ovid wrote:
> I don't know if a native stack trace implemented in the Perl core would
> solve the problems, but if we were to add "official" stack traces in the
> core, allowing people to pass a callback or frame subclass for filtering, I
> think this would be a benefit.
>
> If we later want to add a native exception system, native stack traces
> would likely be a prerequisite.

I'm not really sure in this context what you consider to be a "native"
stack trace which isn't already provided by perl?

Carp and Devel::StackTrace are just wrappers of varying heaviness around
perl's built-in call stack introspection facility: caller() and @DB::args.

Here's a minimal perl stack tracer:

f(qw(here's some args for f));
sub f { g(qw(args for g)) }
sub g { h(qw(h's args)) }
sub h { dump_stack() }

sub dump_stack {
my @c;
my $i = 0;
package DB;
while (@c = caller(++$i)) {
print "$c[3](", join(', ', @DB::args), ")\n";
}
}

which outputs:

main::h(h's, args)
main::g(args, for, g)
main::f(here's, some, args, for, f)

Most of the value that Carp adds is around sanely displaying weird
arguments (undef, long strings, typeglobs, tied/overloaded for example).

What else would you want perl to natively provide?


--
The Enterprise successfully ferries an alien VIP from one place to another
without serious incident.
-- Things That Never Happen in "Star Trek" #7

On Fri, Sep 08, 2023 at 11:56:00AM +0100, Dave Mitchell wrote:
> Most of the value that Carp adds is around sanely displaying weird
> arguments (undef, long strings, typeglobs, tied/overloaded for example).

I forgot to mention that perl's introspection of caller args is inherently
flaky. It will be better once PERL_RC_STACK becomes the norm, but it
can still return weird rubbish.

The biggest issue is that caller() tries to reconstruct the original $_[0]
etc even after a shift; so in:

sub f { my $self = shift; ... }

it actually inspects back before the current $_[0] looking for something
which might have been an argument. What it finds might actually have been
freed and/or reallocated in the meantime.

--
Wesley Crusher gets beaten up by his classmates for being a smarmy git,
and consequently has a go at making some friends of his own age for a
change.
-- Things That Never Happen in "Star Trek" #18

On Fri, 08 Sep 2023 12:56:00 +0200, Dave Mitchell <davem@iabyn.com> wrote:

> On Fri, Sep 08, 2023 at 12:24:58PM +0200, Ovid wrote:
>> I don't know if a native stack trace implemented in the Perl core would
>> solve the problems, but if we were to add "official" stack traces in the
>> core, allowing people to pass a callback or frame subclass for filtering, I
>> think this would be a benefit.
>>
>> If we later want to add a native exception system, native stack traces
>> would likely be a prerequisite.
>
> I'm not really sure in this context what you consider to be a "native"
> stack trace which isn't already provided by perl?
>
> Carp and Devel::StackTrace are just wrappers of varying heaviness around
> perl's built-in call stack introspection facility: caller() and @DB::args.
>
> ...
>
> What else would you want perl to natively provide?

i think what ovid is getting at that there could be value in moderately configurable stacktrace generators implemented in the perl c core, with the hope that those could be considerably faster than the only current way of getting the data out of the internals, then munging it at the perl level

no idea if that would actually bear out, but that's my understanding

--
With regards,
Christian Walde

> On Sep 8, 2023, at 08:54, Christian Walde <walde.christian@gmail.com> wrote:
>
> ?On Fri, 08 Sep 2023 12:56:00 +0200, Dave Mitchell <davem@iabyn.com> wrote:
>
>>> On Fri, Sep 08, 2023 at 12:24:58PM +0200, Ovid wrote:
>>> I don't know if a native stack trace implemented in the Perl core would
>>> solve the problems, but if we were to add "official" stack traces in the
>>> core, allowing people to pass a callback or frame subclass for filtering, I
>>> think this would be a benefit.
>>>
>>> If we later want to add a native exception system, native stack traces
>>> would likely be a prerequisite.
>>
>> I'm not really sure in this context what you consider to be a "native"
>> stack trace which isn't already provided by perl?
>>
>> Carp and Devel::StackTrace are just wrappers of varying heaviness around
>> perl's built-in call stack introspection facility: caller() and @DB::args.
>>
>> ...
>>
>> What else would you want perl to natively provide?
>
> i think what ovid is getting at that there could be value in moderately configurable stacktrace generators implemented in the perl c core, with the hope that those could be considerably faster than the only current way of getting the data out of the internals, then munging it at the perl level
>
> no idea if that would actually bear out, but that's my understanding

Would an XS module provide comparable functionality?

-FG

On Fri, 8 Sept 2023 at 14:56, Felipe Gasper via perl5-porters <
perl5-porters@perl.org> wrote:

>
> > On Sep 8, 2023, at 08:54, Christian Walde <walde.christian@gmail.com>
> wrote:
> >
> > ?On Fri, 08 Sep 2023 12:56:00 +0200, Dave Mitchell <davem@iabyn.com>
> wrote:
> >
> >>> On Fri, Sep 08, 2023 at 12:24:58PM +0200, Ovid wrote:
> >>> I don't know if a native stack trace implemented in the Perl core would
> >>> solve the problems, but if we were to add "official" stack traces in
> the
> >>> core, allowing people to pass a callback or frame subclass for
> filtering, I
> >>> think this would be a benefit.
> >>>
> >>> If we later want to add a native exception system, native stack traces
> >>> would likely be a prerequisite.
> >>
> >> I'm not really sure in this context what you consider to be a "native"
> >> stack trace which isn't already provided by perl?
> >>
> >> Carp and Devel::StackTrace are just wrappers of varying heaviness around
> >> perl's built-in call stack introspection facility: caller() and
> @DB::args.
> >>
> >> ...
> >>
> >> What else would you want perl to natively provide?
> >
> > i think what ovid is getting at that there could be value in moderately
> configurable stacktrace generators implemented in the perl c core, with the
> hope that those could be considerably faster than the only current way of
> getting the data out of the internals, then munging it at the perl level
> >
> > no idea if that would actually bear out, but that's my understanding
>
> Would an XS module provide comparable functionality?
>

Yeah, this could be implemented as an XS module.

I think the problem with this kind of request is that there are an awful
lot of possible options that could be applied. Do you show arguments or
not? If you do, do you show the raw value or do you filter them, or do you
transform them to show type only, etc. Do you show hints flags? Do you show
the file and line? Do you munge the filename? Do you show the full stack?
Do you not? How do you interoperate with eval, etc. It turns out that
there are lot of options there...

Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

On Fri, Sep 8, 2023 at 3:01?PM demerphq <demerphq@gmail.com> wrote:

>
>> > i think what ovid is getting at that there could be value in moderately
>> configurable stacktrace generators implemented in the perl c core, with the
>> hope that those could be considerably faster than the only current way of
>> getting the data out of the internals, then munging it at the perl level
>>
>
Yes, that's what I was hoping.


> Yeah, this could be implemented as an XS module.
>
> I think the problem with this kind of request is that there are an awful
> lot of possible options that could be applied. Do you show arguments or
> not? If you do, do you show the raw value or do you filter them, or do you
> transform them to show type only, etc. Do you show hints flags? Do you show
> the file and line? Do you munge the filename? Do you show the full stack?
> Do you not? How do you interoperate with eval, etc. It turns out that
> there are lot of options there...
>

What I'm hoping was that some of this could be native to a C implementation
with "default" behavior, with a possibility to override some bits in Perl,
but still have the ability to have the rest done in C. Of course, if you
override too much, you'll not get much benefit over just having a pure Perl
version. That means a core version might be fast, with minor customization
available, but if you need to upgrade, there are still CPAN modules
available.

*Most* of the time, I just want a quick stack trace with the ability to
hide scalar values so that PII doesn't leak. Thus, I can generate a stack
trace and write it to the logs with less concern about exposing secrets,
and with less concern that it's a performance or memory hit if I do it a
lot (which happens in the nightmare case of a bug that can't be reproduced
outside of production, for example).

But whether or not that is realistic, I can't say. For a few exception
objects I've found, a common anti-pattern is to use exceptions for flow
control (throw, catch it up the stack, and ignore the error). Those
exceptions often have a stack trace, so a faster, reliable solution would
be nice for them (though they should stop generating those damned traces in
that case).

That being said, I fully acknowledge that this might be too little benefit
for the reward.

Best,
Ovid

--
Curtis "Ovid" Poe
--
CTO, All Around the World
World-class software development and consulting
https://allaroundtheworld.fr/

On Fri, 8 Sep 2023 17:42:00 +0200
Ovid <curtis.poe@gmail.com> wrote:

> That being said, I fully acknowledge that this might be too little
> benefit for the reward.

I think this may hit the nail on the head ;)

As compared: object systems, type^W value constraints, :export
attributes, builtin functions for X, Y and Z, a metaprogramming API...
we're running a little thin on the ground for folks to actually be
implementing anything else.

I don't think fundamentally anyone would complain about the idea of "Do
foo but faster", we have to consider how often/likely the average user
is to "do foo" and how much "faster" it could be made for the given
amount of effort.

At least for me, stacktraces fall below the criteria to bother spending
a lot of time on just at the moment, when there's so many other things
to be done. But of course that's not to stop anyone else who wishes to
have a go.

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/

On Fri, Sep 08, 2023 at 04:52:16PM +0100, Paul "LeoNerd" Evans wrote:
> At least for me, stacktraces fall below the criteria to bother spending
> a lot of time on just at the moment, when there's so many other things
> to be done. But of course that's not to stop anyone else who wishes to
> have a go.

On the other hand, the current 'package DB; @args = @DB::args' is a
horrible hack. Perhaps once signatures have been properly implemented
(with the original args kept on the stack rather than quietly being copied
into @_ and accessed from there) it might be time to revisit the official
API that perl provides to access the stack args, and come up up with
something better? For example something that can extract the names of
named parameters.

But I think creating exception objects which can have stack trace info
attached to them or similar, is a tad ambitious.

PS:

This quick-n-dirty sample code creates a full stack trace about 75,000
times per second on my laptop. It sanitises the output by just listing the
class or type of the arg, such as,

./p:5 Foo::f(Foo, ARRAY, SCALAR, SCALAR, SCALAR, SCALAR, SCALAR)

It shows that such code can be small and fast.

Here goes:

package Foo;

f(bless({}, 'Foo'), [], qw(here's some args for f));

sub f { my $self = shift; $self->g(qw(args for g)) }
sub g { my $self = shift; $self->h(qw(h's args)) }
sub h {
my $self = shift;
for my $i (1..100_000) {
dump_stack();
}
}

sub dump_stack {
my @c;
my $i = 0;
my @frames;
package DB;
while (@c = caller(++$i)) {
my @args = map ref ? ref : ref \$_, @DB::args;
push @frames, "$c[1]:$c[2] $c[3](" . join(', ', @args) . ")\n";
}
}

--
The Enterprise successfully ferries an alien VIP from one place to another
without serious incident.
-- Things That Never Happen in "Star Trek" #7

On Fri, 8 Sept 2023 at 18:52, Dave Mitchell <davem@iabyn.com> wrote:

> On Fri, Sep 08, 2023 at 04:52:16PM +0100, Paul "LeoNerd" Evans wrote:
> > At least for me, stacktraces fall below the criteria to bother spending
> > a lot of time on just at the moment, when there's so many other things
> > to be done. But of course that's not to stop anyone else who wishes to
> > have a go.
>
> On the other hand, the current 'package DB; @args = @DB::args' is a
> horrible hack. Perhaps once signatures have been properly implemented
> (with the original args kept on the stack rather than quietly being copied
> into @_ and accessed from there) it might be time to revisit the official
> API that perl provides to access the stack args, and come up up with
> something better? For example something that can extract the names of
> named parameters.
>
> But I think creating exception objects which can have stack trace info
> attached to them or similar, is a tad ambitious.
>
> PS:
>
> This quick-n-dirty sample code creates a full stack trace about 75,000
> times per second on my laptop. It sanitises the output by just listing the
> class or type of the arg, such as,
>
> ./p:5 Foo::f(Foo, ARRAY, SCALAR, SCALAR, SCALAR, SCALAR, SCALAR)
>
> It shows that such code can be small and fast.
>
> Here goes:
>
> package Foo;
>
> f(bless({}, 'Foo'), [], qw(here's some args for f));
>
> sub f { my $self = shift; $self->g(qw(args for g)) }
> sub g { my $self = shift; $self->h(qw(h's args)) }
> sub h {
> my $self = shift;
> for my $i (1..100_000) {
> dump_stack();
> }
> }
>
> sub dump_stack {
> my @c;
> my $i = 0;
> my @frames;
> package DB;
> while (@c = caller(++$i)) {
> my @args = map ref ? ref : ref \$_, @DB::args;
> push @frames, "$c[1]:$c[2] $c[3](" . join(', ', @args) .
> ")\n";
> }
> }
>

That stack isnt very deep and the args on the stack aren't complex, and as
you said what it does with those arguments isnt very fancy. Also there are
a lot of subtleties that code leaves out. A typical use case for a stack
trace dump is to be called from a $SIG{__DIE__} handler, and in that
context you probably want to be careful about dealing with eval frames.
Either because you specifically *don't* want to serialize the stack from a
specific eval frame, or because you *do* want to do so, or a mixture of
both, where you want to ignore some eval frames at the top of the stack but
not those at the bottom. So $^S comes up as a factor, and within a
web-server there is a very good chance you actually want to let the
developer decide if a given eval should disable the stack-trace or not, so
you end up needing code that counts the number of eval frames in the stack,
and provide a way for users to control how many evals to ignore. So a
simple implementation probably acquires additional functionality over time.
Consider that a good stack trace dumper used in a corporate setting
probably shouldnt accidentally dump credit card numbers and other sensitive
data, and you end up needing a fair bit of logic.

cheers,
Yves



--
perl -Mre=debug -e "/just|another|perl|hacker/"

On 2023-09-08 17:42, Ovid wrote:
> /Most/ of the time, I just want a quick stack trace with the ability
> to hide scalar values so that PII doesn't leak.

That means that the PII-values need to be marked as such.

You can for example use Variable::Magic to make a value look like
"secret" or 0xDEADBEEF,
unless the call came from a trusted source.

-- Ruud