Mailing List Archive

PL_localizing doesn't nest
This is a bug report for perl from zefram@fysh.org,
generated with the help of perlbug 1.43 running under perl 5.37.10.


-----------------------------------------------------------------
<!--[Please describe your issue here]-->

A small number of Perl\'s behaviours depend on the PL\_localizing flag\,
which is set when performing get or set magic as part of a \"local\"
operation\. But this flag is not well behaved\. For example\, the
warning about using undef as a string is suppressed by PL\_localizing\,
and here\'s some behaviour involving nested magical localisations\:

```
$ perl -lwe 'package Foo { sub TIESCALAR { bless({}) } sub FETCH { print "fetching"; if($::p) { local $!; } "abc$a" } sub STORE {} } tie $x, "Foo"; print $x; { local $x; } print $p = 1; { local $x; } print 2'
fetching
Use of uninitialized value $Foo::a in concatenation (.) or string at -e line 1.
abc
fetching
1
fetching
Use of uninitialized value $Foo::a in concatenation (.) or string at -e line 1.
2
```

The Foo\:\:FETCH method stringifies an undef value\, which normally
warns\. That warning is seen in the first fetch operation\, in \"print
\$x\"\. The warning is suppressed in the second fetch operation\, for
the first \"local \$x\"\. But then the third fetch operation\, for the
second \"local \$x\"\, generates the warning again\. The difference
between the two localisations of \$x is that for the second one the
FETCH method localises \(and restores\) \$\!\, in a way that should
amount to a no\-op\.

The underlying issue is that the setting of PL\_localizing doesn\'t take
account of the way these operations can be nested\. There\'s no
restoration of PL\_localizing to its former value\; rather\, it gets
unconditionally cleared when the magic operation completes\. In the
above test case\, PL\_localizing \=\= 1 when the FETCH method starts
running for either localisation\, but the inner localisation of \$\! has
the effect of clearing PL\_localizing\, so it has different values when
the undef stringification occurs for those two localisations of \$x\.

Clearly\, in the above the two localisations of \$x should behave
identically\. PL\_localizing should have the same value both times at
the point when the undef stringification happens\. But I\'m not sure
what value that should be\. One possibility is that PL\_localizing
should be 1\, as it currently is when the FETCH method starts running\.
In that case the localisation of \$\! \(and so localisations in
general\) should be saving and restoring PL\_localizing\.

However\, looking at the more significant ways in which PL\_localizing
gets used\, I tend towards the view that PL\_localizing shoud never be
non\-zero when arbitrary Perl ops are executing\. It looks like the
non\-zero PL\_localizing is meant to have a very limited scope\, during
which only magic get\/set methods written in C run\. In that case the
tie magic should be saving and clearing PL\_localizing before calling
the tied method\, and restoring it when the method returns\.

More significant ways in which PL\_localizing gets used include
suppressing tainting\. I don\'t think tainting should be suppressed in
cases like the first attempt in this test case\:

```
$ perl -Tlwe 'package Foo { sub TIESCALAR { bless({}) } sub FETCH { open(my $f, ">", "abc".substr($^X,0,0)) and print "opened"; } sub STORE {} } tie $x, "Foo"; { local $x; } print $x'
opened
Insecure dependency in open while running with -T switch at -e line 1.
```

<!--[Please do not change anything below this line]-->
<!------------------------------------------------------------------- -->


---
**Flags**
- category=core
- severity=low
---
**Perl configuration**
```
Site configuration information for perl 5.37.10:

Configured by zefram at Tue Mar 21 14:35:23 GMT 2023.

Summary of my perl5 (revision 5 version 37 subversion 10) configuration:

Platform:
osname=linux
osvers=5.10.0-21-amd64
archname=x86_64-linux-thread-multi
uname='linux barba.rous.org 5.10.0-21-amd64 #1 smp debian 5.10.162-1 (2023-01-21) x86_64 gnulinux '
config_args='-des -Dprefix=/home/zefram/usr/perl/perl_install/perl-5.37.10-i64-f52 -Duselargefiles -Dusethreads -Uafs -Ud_csh -Uusesfio -Uusenm -Duseshrplib -Dusedevel -Uversiononly -Ui_db'
hint=recommended
useposix=true
d_sigaction=define
useithreads=define
usemultiplicity=define
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
Compiler:
cc='cc'
ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
optimize='-O2'
cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
ccversion=''
gccversion='10.2.1 20210110'
gccosandvers=''
intsize=4
longsize=8
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=16
longdblkind=3
ivtype='long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='off_t'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries:
ld='cc'
ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib
libs=-lpthread -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc
perllibs=-lpthread -ldl -lm -lcrypt -lutil -lc
libc=libc-2.31.so
so=so
useshrplib=true
libperl=libperl.so
gnulibc_version='2.31'
Dynamic Linking:
dlsrc=dl_dlopen.xs
dlext=so
d_dlsymun=undef
ccdlflags='-Wl,-E -Wl,-rpath,/home/zefram/usr/perl/perl_install/perl-5.37.10-i64-f52/lib/5.37.10/x86_64-linux-thread-multi/CORE'
cccdlflags='-fPIC'
lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'


---
@INC for perl 5.37.10:
/home/zefram/usr/perl/perl_install/perl-5.37.10-i64-f52/lib/site_perl/5.37.10/x86_64-linux-thread-multi
/home/zefram/usr/perl/perl_install/perl-5.37.10-i64-f52/lib/site_perl/5.37.10
/home/zefram/usr/perl/perl_install/perl-5.37.10-i64-f52/lib/5.37.10/x86_64-linux-thread-multi
/home/zefram/usr/perl/perl_install/perl-5.37.10-i64-f52/lib/5.37.10

---
Environment for perl 5.37.10:
HOME=/home/zefram
LANG (unset)
LANGUAGE (unset)
LD_LIBRARY_PATH (unset)
LOGDIR (unset)
PATH=/home/zefram/usr/perl/perl_install/perl-5.37.10-i64-f52/bin:/home/zefram/usr/perl/util:/home/zefram/pub/x86_64-unknown-linux-gnu/bin:/home/zefram/pub/common/bin:/usr/bin:/bin:/usr/local/bin:/usr/games
PERL_BADLANG (unset)
SHELL=/usr/bin/zsh
```
Re: PL_localizing doesn't nest [ In reply to ]
On Mon, Apr 10, 2023 at 6:31?PM zefram via perl5-porters <
perl5-porters@perl.org> wrote:

This message has once again failed to render in gmail as specified.

-Dan

>
Re: PL_localizing doesn't nest [ In reply to ]
On Mon, 10 Apr 2023 18:46:22 -0400
Dan Book <grinnz@gmail.com> wrote:

> On Mon, Apr 10, 2023 at 6:31?PM zefram via perl5-porters <
> perl5-porters@perl.org> wrote:
>
> This message has once again failed to render in gmail as specified.

Possibly gmail is upset about:

Content-Type: text/plain (actually GitHub Flavored Markdown, but the Perl maintainers prefer to receive it labelled as plain text)

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/
Re: PL_localizing doesn't nest [ In reply to ]
>
> Content-Type: text/plain (actually GitHub Flavored Markdown, but the Perl
> maintainers prefer to receive it labelled as plain text)


This syntax violates the RFC for 'Content-Type'.
Re: PL_localizing doesn't nest [ In reply to ]
Dan Book wrote:
>This message has once again failed to render in gmail as specified.

Interesting. Please give more detail: how does it appear? By "once
again" are you referring to "refaliasing disrespects readonlyness",
which you complained about at the time?

This time the body of the message is GitHub Flavored Markdown, but
there's a Content-Type header which specifies it as text/plain, and
the Content-Type header also has a comment explaining the mislabelling.
Adding this header with the comment is the new bit that I'm trying this
time round, as I said I would in the aftermath of the previous bug report.
The comment ought to be acceptable to all MUAs, but if it's preventing
Gmail from treating the body as plain text (which I think is what you're
saying, but I'm not clear) then it'll have to go.

-zefram
Re: PL_localizing doesn't nest [ In reply to ]
Karen Etheridge wrote:
>> Content-Type: text/plain (actually GitHub Flavored Markdown, but the Perl
>> maintainers prefer to receive it labelled as plain text)
>
>This syntax violates the RFC for 'Content-Type'.

Which RFC are you referring to? I believe RFC 2045 is the relevant one.
It refers to RFC 822's rules for structured header fields, which permit
comments. RFC 2045 also explicitly says, in section 5.1 on the syntax
of Content-Type, that comments are permitted, and gives an example of
a Content-Type header including a comment.

-zefram
Re: PL_localizing doesn't nest [ In reply to ]
On Mon, Apr 10, 2023 at 7:12?PM Zefram via perl5-porters <
perl5-porters@perl.org> wrote:

> Dan Book wrote:
> >This message has once again failed to render in gmail as specified.
>
> Interesting. Please give more detail: how does it appear? By "once
> again" are you referring to "refaliasing disrespects readonlyness",
> which you complained about at the time?
>
> This time the body of the message is GitHub Flavored Markdown, but
> there's a Content-Type header which specifies it as text/plain, and
> the Content-Type header also has a comment explaining the mislabelling.
> Adding this header with the comment is the new bit that I'm trying this
> time round, as I said I would in the aftermath of the previous bug report.
> The comment ought to be acceptable to all MUAs, but if it's preventing
> Gmail from treating the body as plain text (which I think is what you're
> saying, but I'm not clear) then it'll have to go.
>

Correct: it is not being recognized as a specification of plaintext, so the
body appears in an unpreviewable attachment.

-Dan
Re: PL_localizing doesn't nest [ In reply to ]
On Tue, 11 Apr 2023 00:17:50 +0100
Zefram via perl5-porters <perl5-porters@perl.org> wrote:

> Karen Etheridge wrote:
> >> Content-Type: text/plain (actually GitHub Flavored Markdown, but
> >> the Perl maintainers prefer to receive it labelled as plain text)
> >
> >This syntax violates the RFC for 'Content-Type'.
>
> Which RFC are you referring to? I believe RFC 2045 is the relevant
> one. It refers to RFC 822's rules for structured header fields, which
> permit comments. RFC 2045 also explicitly says, in section 5.1 on
> the syntax of Content-Type, that comments are permitted, and gives an
> example of a Content-Type header including a comment.

Just because the RFCs mention it really doesn't mean gmail will respect
it.

Gmail is known to totally disobey a lot of established email precedent.
There are RFCs about the "In-Reply-To" and "References" headers, and
gmail totally ignores those and simply does some string matches on the
"Subject" header in order to make "threading".

It's one among about 50 reasons I don't use it personally...

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/
Re: PL_localizing doesn't nest [ In reply to ]
Dan Book wrote:
>Correct: it is not being recognized as a specification of plaintext, so the
>body appears in an unpreviewable attachment.

Bummer. I'd still like to put a comment in that vicinity, but it can
go in a separate header of no semantic value. Thank you for the feedback.

So far it's looking like the winning formula is Markdown body labelled as
plain text using very conservative headers, as I did with "initfields CV
visibility causes mayhem". That one got no complaints, but also no one
explicitly saying it was good. Would anyone like to give an explicit
comment on that format?

Other than putting in conservative MIME headers and a comment header,
I have no further variation itching to be tried. If the format of
"initfields CV visibility causes mayhem" is agreeable, then I think
we're done with the experimentation, and I can put together a patch for
perlbug to reflect our findings.

-zefram
Re: PL_localizing doesn't nest [ In reply to ]
On Mon, Apr 10, 2023 at 7:41?PM Zefram via perl5-porters <
perl5-porters@perl.org> wrote:

> Other than putting in conservative MIME headers and a comment header,
> I have no further variation itching to be tried. If the format of
> "initfields CV visibility causes mayhem" is agreeable, then I think
> we're done with the experimentation, and I can put together a patch for
> perlbug to reflect our findings.
>

That message worked fine for me.

-Dan