Mailing List Archive: 'make install' fails on a perl configured with '-fsanitize=address'

'make install' fails on a perl configured with '-fsanitize=address'

Jul 10, 2021, 5:06 PM

Post #1 of 7 (312 views)

Today I attempted to build and install a perl built with
AddressSanitizer on FreeBSD-12. I configured by adding the following 3
switches to a standard ./Configure invocation:

#####
-Accflags='-fsanitize=address'
-Aldflags='-fsanitize=address'
-Alddflags='-fsanitize=address'
#####

'make' completed successfully, leading to the following value for
'config_args':

#####
$ ./perl -Ilib -V:config_args
config_args='-des -Dusedevel -Accflags='-fsanitize=address'
-Aldflags='-fsanitize=address' -Alddflags='-fsanitize=address'
-Dprefix=/home/jkeenan/testing/v5.35.1 -Uversiononly -Dman1dir=none
-Dman3dir=none';

'make' was actually invoked via 'make install':

#####
make -j8 install
#####

... so after 'make' itself completed, installation began. However,
every time I tried this, the underlying 'installperl' process stalled,
usually about here:

#####
...

/home/jkeenan/testing/v5.35.1/lib/perl5/5.35.1/amd64-freebsd/Filter/Util/Call.pm
/home/jkeenan/testing/v5.35.1/lib/perl5/5.35.1/Module/Load.pm
/home/jkeenan/testing/v5.35.1/lib/perl5/5.35.1/Module/Loaded.pm
/home/jkeenan/testing/v5.35.1/lib/perl5/5.35.1/Module/CoreList.pm
[hang]
#####

I called 'ps' to try to find out what was going on.

#####
$ ps auxwww | ack '(^USER.*VSZ|installperl)'
USER PID %CPU %MEM VSZ RSS TT STAT STARTED
TIME COMMAND
jkeenan 59025 100.0 2.9 21474917848 239388 0 R 23:46
1:05.03 ./perl -Ilib -I. installperl --destdir=
jkeenan 59793 0.0 0.1 11048 5016 2 R+ 23:47
0:00.00 /usr/local/bin/perl /usr/local/bin/ack (^USER.*VSZ|installperl)
#####

I'm not very surprised that 'installperl' was using 100% of CPU. There
was nothing much else happening on the machine at that point, so why
shouldn't all of CPU be devoted to this task.

The high values for 'VSZ' and 'RSS' look suspicious, but I don't have
the expertise to determine their relevance.

The partition is only 32% full, so lack of disk space is not the
explanation.

This installation process works perfectly fine when I configure and
build without the 3 asan-related Configure switches listed above.

Can anyone suggest why 'make install' fails on a perl built with
AddressSanitizer?

Thank you very much.
Jim Keenan

Re: 'make install' fails on a perl configured with '-fsanitize=address' [ In reply to ]

hv at crypt

Jul 12, 2021, 7:50 AM

Post #2 of 7 (312 views)

Permalink

James E Keenan <jkeenan@pobox.com> wrote:
:Today I attempted to build and install a perl built with
:AddressSanitizer on FreeBSD-12. I configured by adding the following 3
:switches to a standard ./Configure invocation:
:
:#####
:-Accflags='-fsanitize=address'
:-Aldflags='-fsanitize=address'
:-Alddflags='-fsanitize=address'

You probably don't want lddflags:

% ldd -fsanitize=address /bin/ls
ldd: unrecognized option `-fsanitize=address'
Try `ldd --help' for more information.
%

Leaving that out, I was unable to reproduce the problem on my Ubuntu system;
however I also did not run parallel make, just:

./Configure -des -Dcc=gcc -Dprefix=/opt/blead -Accflags='-fsanitize=address' \
-Aldflags='-fsanitize=address' -Dusedevel -Uversiononly
make install

.. so if removing the lddflags option doesn't fix it, next would be to
try it without the -j8. (It would also be useful to know whether this
uses gcc or clang on your system.)

Hugo

Re: 'make install' fails on a perl configured with '-fsanitize=address' [ In reply to ]

jkeenan at pobox

Jul 12, 2021, 10:47 AM

Post #3 of 7 (312 views)

Permalink

On 7/12/21 10:50 AM, hv@crypt.org wrote:
> James E Keenan <jkeenan@pobox.com> wrote:
> :Today I attempted to build and install a perl built with
> :AddressSanitizer on FreeBSD-12. I configured by adding the following 3
> :switches to a standard ./Configure invocation:
> :
> :#####
> :-Accflags='-fsanitize=address'
> :-Aldflags='-fsanitize=address'
> :-Alddflags='-fsanitize=address'
>
> You probably don't want lddflags:
>
> % ldd -fsanitize=address /bin/ls
> ldd: unrecognized option `-fsanitize=address'
> Try `ldd --help' for more information.
> %
>

FWIW, on FreeBSD this command does do something:
#####
$ ldd -fsanitize=address /bin/ls
sanitize=addresssanitize=addresssanitize=address
#####
Perhaps not something useful ... but something.

> Leaving that out, I was unable to reproduce the problem on my Ubuntu system;
> however I also did not run parallel make, just:
>
> ./Configure -des -Dcc=gcc -Dprefix=/opt/blead -Accflags='-fsanitize=address' \
> -Aldflags='-fsanitize=address' -Dusedevel -Uversiononly
> make install
>
> .. so if removing the lddflags option doesn't fix it, next would be to
> try it without the -j8.

I set 'export TEST_JOBS=1' and re-ran; 'make install' failed at the same
point.

(It would also be useful to know whether this
> uses gcc or clang on your system.)
>

clang is the default C-compiler on FreeBSD.
#####
$ clang --version
FreeBSD clang version 10.0.1 (git@github.com:llvm/llvm-project.git
llvmorg-10.0.1-0-gef32c611aa2)
Target: x86_64-unknown-freebsd12.2
Thread model: posix
InstalledDir: /usr/bin
#####

Re: 'make install' fails on a perl configured with '-fsanitize=address' [ In reply to ]

doughera at lafayette

Jul 12, 2021, 2:03 PM

Post #4 of 7 (312 views)

Permalink

On Mon, Jul 12, 2021 at 01:47:37PM -0400, James E Keenan wrote:

> > James E Keenan <jkeenan@pobox.com> wrote:
> > :Today I attempted to build and install a perl built with
> > :AddressSanitizer on FreeBSD-12. I configured by adding the following 3
> > :switches to a standard ./Configure invocation:
> > :
> > :#####
> > :-Accflags='-fsanitize=address'
> > :-Aldflags='-fsanitize=address'
> > :-Alddflags='-fsanitize=address'
> >

Adding -fsantize=address also hung for me on FreeBSD-13.0-RELEASE-p3, with
clang-11.0.1. The hang happened during 'make test' on t/op/defins.t,
while running the test below. It looks to be hanging on the glob('*') call.

$seen = 0;
while (my $name = glob('*'))
{
$seen++ if $name eq $wanted_filename;
}
cmp_ok($seen,'==',1,'saw file in glob while()');

I hope to spend more time tracking this down later this week.

--
Andy Dougherty doughera@lafayette.edu

Re: 'make install' fails on a perl configured with '-fsanitize=address' [ In reply to ]

jkeenan at pobox

Jul 12, 2021, 2:33 PM

Post #5 of 7 (312 views)

Permalink

On 7/10/21 8:06 PM, James E Keenan wrote:
> Today I attempted to build and install a perl built with
> AddressSanitizer on FreeBSD-12. I configured by adding the following 3
> switches to a standard ./Configure invocation:
>
> #####
> -Accflags='-fsanitize=address'
> -Aldflags='-fsanitize=address'
> -Alddflags='-fsanitize=address'
> #####
>
> 'make' completed successfully, leading to the following value for
> 'config_args':
>
> #####
> $ ./perl -Ilib -V:config_args
> config_args='-des -Dusedevel -Accflags='-fsanitize=address'
> -Aldflags='-fsanitize=address' -Alddflags='-fsanitize=address'
> -Dprefix=/home/jkeenan/testing/v5.35.1 -Uversiononly -Dman1dir=none
> -Dman3dir=none';
>
> 'make' was actually invoked via 'make install':
>
> #####
> make -j8 install
> #####
>
> ... so after 'make' itself completed, installation began. However,
> every time I tried this, the underlying 'installperl' process stalled,
> usually about here:
>
> #####
> ...
>
> /home/jkeenan/testing/v5.35.1/lib/perl5/5.35.1/amd64-freebsd/Filter/Util/Call.pm
>
> /home/jkeenan/testing/v5.35.1/lib/perl5/5.35.1/Module/Load.pm
> /home/jkeenan/testing/v5.35.1/lib/perl5/5.35.1/Module/Loaded.pm
> /home/jkeenan/testing/v5.35.1/lib/perl5/5.35.1/Module/CoreList.pm
> [hang]
> #####
>
> I called 'ps' to try to find out what was going on.
>
> #####
> $ ps auxwww | ack '(^USER.*VSZ|installperl)'
> USER      PID %CPU %MEM         VSZ     RSS TT STAT STARTED TIME COMMAND
> jkeenan 59025 100.0 2.9 21474917848 239388 0 R    23:46 1:05.03
> ./perl -Ilib -I. installperl --destdir=
> jkeenan 59793   0.0 0.1       11048    5016 2 R+   23:47 0:00.00
> /usr/local/bin/perl /usr/local/bin/ack (^USER.*VSZ|installperl)
> #####
>
> I'm not very surprised that 'installperl' was using 100% of CPU. There
> was nothing much else happening on the machine at that point, so why
> shouldn't all of CPU be devoted to this task.
>
> The high values for 'VSZ' and 'RSS' look suspicious, but I don't have
> the expertise to determine their relevance.
>
> The partition is only 32% full, so lack of disk space is not the
> explanation.
>
> This installation process works perfectly fine when I configure and
> build without the 3 asan-related Configure switches listed above.
>
> Can anyone suggest why 'make install' fails on a perl built with
> AddressSanitizer?
>
> Thank you very much.
> Jim Keenan

Much to my surprise, I was able to build and install a perl configured
with asan at tag v5.35.1 on the same FreeBSD-12 machine but using
*clang11* as my C compiler rather than the default clang10.

I have not yet run 'make test' on this build, however.

Re: 'make install' fails on a perl configured with '-fsanitize=address' [ In reply to ]

doughera at lafayette

Jul 15, 2021, 12:59 PM

Post #6 of 7 (312 views)

Permalink

On Mon, Jul 12, 2021 at 05:03:42PM -0400, Andy Dougherty wrote:
> On Mon, Jul 12, 2021 at 01:47:37PM -0400, James E Keenan wrote:
>
> > > James E Keenan <jkeenan@pobox.com> wrote:
> > > :Today I attempted to build and install a perl built with
> > > :AddressSanitizer on FreeBSD-12. I configured by adding the following 3
> > > :switches to a standard ./Configure invocation:
> > > :
> > > :#####
> > > :-Accflags='-fsanitize=address'
> > > :-Aldflags='-fsanitize=address'
> > > :-Alddflags='-fsanitize=address'
> > >
>
> Adding -fsantize=address also hung for me on FreeBSD-13.0-RELEASE-p3, with
> clang-11.0.1. The hang happened during 'make test' on t/op/defins.t,
> while running the test below. It looks to be hanging on the glob('*') call.
>
> $seen = 0;
> while (my $name = glob('*'))
> {
> $seen++ if $name eq $wanted_filename;
> }
> cmp_ok($seen,'==',1,'saw file in glob while()');
>
> I hope to spend more time tracking this down later this week.

Unfortunately, I have made no useful progress. It hangs similarly
in much earlier perl versions as well, and I have not been able to
find any way around it yet.

--
Andy Dougherty doughera@lafayette.edu

Re: 'make install' fails on a perl configured with '-fsanitize=address' [ In reply to ]

jkeenan at pobox

Jul 15, 2021, 1:16 PM

Post #7 of 7 (312 views)

Permalink

On 7/12/21 5:33 PM, James E Keenan wrote:
> On 7/10/21 8:06 PM, James E Keenan wrote:
>> Today I attempted to build and install a perl built with
>> AddressSanitizer on FreeBSD-12. I configured by adding the following
>> 3 switches to a standard ./Configure invocation:
>>
>> #####
>> -Accflags='-fsanitize=address'
>> -Aldflags='-fsanitize=address'
>> -Alddflags='-fsanitize=address'
>> #####
>>
>> 'make' completed successfully, leading to the following value for
>> 'config_args':
>>
>> #####
>> $ ./perl -Ilib -V:config_args
>> config_args='-des -Dusedevel -Accflags='-fsanitize=address'
>> -Aldflags='-fsanitize=address' -Alddflags='-fsanitize=address'
>> -Dprefix=/home/jkeenan/testing/v5.35.1 -Uversiononly -Dman1dir=none
>> -Dman3dir=none';
>>
>> 'make' was actually invoked via 'make install':
>>
>> #####
>> make -j8 install
>> #####
>>
>> ... so after 'make' itself completed, installation began. However,
>> every time I tried this, the underlying 'installperl' process stalled,
>> usually about here:
>>
>> #####
>> ...
>>
>> /home/jkeenan/testing/v5.35.1/lib/perl5/5.35.1/amd64-freebsd/Filter/Util/Call.pm
>>
>>    /home/jkeenan/testing/v5.35.1/lib/perl5/5.35.1/Module/Load.pm
>>    /home/jkeenan/testing/v5.35.1/lib/perl5/5.35.1/Module/Loaded.pm
>>    /home/jkeenan/testing/v5.35.1/lib/perl5/5.35.1/Module/CoreList.pm
>> [hang]
>> #####
>>
>> I called 'ps' to try to find out what was going on.
>>
>> #####
>> $ ps auxwww | ack '(^USER.*VSZ|installperl)'
>> USER      PID %CPU %MEM         VSZ     RSS TT STAT STARTED TIME
>> COMMAND
>> jkeenan 59025 100.0 2.9 21474917848 239388 0 R    23:46 1:05.03
>> ./perl -Ilib -I. installperl --destdir=
>> jkeenan 59793   0.0 0.1       11048    5016 2 R+   23:47 0:00.00
>> /usr/local/bin/perl /usr/local/bin/ack (^USER.*VSZ|installperl)
>> #####
>>
>> I'm not very surprised that 'installperl' was using 100% of CPU.
>> There was nothing much else happening on the machine at that point, so
>> why shouldn't all of CPU be devoted to this task.
>>
>> The high values for 'VSZ' and 'RSS' look suspicious, but I don't have
>> the expertise to determine their relevance.
>>
>> The partition is only 32% full, so lack of disk space is not the
>> explanation.
>>
>> This installation process works perfectly fine when I configure and
>> build without the 3 asan-related Configure switches listed above.
>>
>> Can anyone suggest why 'make install' fails on a perl built with
>> AddressSanitizer?
>>
>> Thank you very much.
>> Jim Keenan
>
> Much to my surprise, I was able to build and install a perl configured
> with asan at tag v5.35.1 on the same FreeBSD-12 machine but using
> *clang11* as my C compiler rather than the default clang10.
>
> I have not yet run 'make test' on this build, however.

Again, to my surprise, 'make test' passed -- with a runtime of over 11
hours!