-------------8<---------- Start Bug Report ------------8<----------
1. Problem Description:
To reproduce the problem, start from Debian vanilla server (I used
testing (buster)).
Install apache2, libapache2-mod-perl2, libev-perl
(I also used development version of perl, mod_perl and EV with the
same result)
Create 2 perl files with following content:
handler1.pl:
#!/usr/bin/perl
my $pwd = `/bin/pwd`;
1;
handler2.pl:
#!/usr/bin/perl
use EV;
my $w = EV::signal 'CHLD', sub {
print "SIGCHLD received\n";
};
1;
Reference them in two PerlRequire directive, either
in /etc/apache2/apache2.conf or in a separate conf file
in /etc/apache2/conf-enabled:
# order is important
PerlRequire "/path/to/handler1.pl"
PerlRequire "/path/to/handler2.pl"
==> Apache will crash on startup.
I did an in-depth analysis of this bug. Here is what happens:
The backtick operator in handler1.pl internally calls waitpid, which
internally waits for the SIGCHLD signal.
EV::signal 'CHLD' installs a signal handler using sigaction (see
EV-4.22.0: libev/ev.c::ev_signal_start). There is no provision to remove
this signal handler. The handler is located in EV.so, loaded by the XS
loader.
apache2 goes through 3 stages on startup: pre-config, destroy-config,
create-config
In the pre-config stage, the 2 PerlRequire commands are executed and
the signal handler is installed at process level.
In the destroy-config stage, all modules are removed, including
mod_perl, which consists in removing all XS modules (see mod_perl:
src/modules/perl/modperl_interp.c::modperl_interp_destroy())
The removal of EV.so does not restore the signal handler as there is
no cleanup in EV.so.
At this point the signal handler is still registered at process level
but points to invalid memory!
In the create-config stage, the config is recreated, handler1.pl is
reloaded and the backtick operator triggers a SIGCHLD, which causes a
SIGEV because the signal handler is not in memory (the EV.so library
will only be loaded when the second PerlRequire is executed).
Conclusion:
Unloading the XS modules relies on the modules to cleanup the global
state properly but it's clearly not the case of the EV module.
I tested a fix that consists in skipping the unloading of the XS
modules. While it works for my case, it is not a proper solution either
as there are circumstances where it would cause other bugs.
Ideally the EV module should register a cleanup function to restore
the signal handlers on unload.
There are similar potential issues when an XS module register some op
checker in PL_check static array: unloading the XS module does not
remove the handler because Perl does not provide an API to do it.
Hopefully, in this case unloading mod_perl will normally causes
libperl to unload as well, which cleans the PL_check array.
Clearly, Perl is not designed to be unloaded-reloaded.
2. Used Components and their Configuration:
*** mod_perl version 2.000011
*** using /home/ben/src/mod_perl-2.0/lib/Apache2/BuildConfig.pm
*** Makefile.PL options:
MP_APR_LIB => aprext
MP_APXS => /usr/bin/apxs
MP_COMPAT_1X => 1
MP_GENERATE_XS => 1
MP_LIBNAME => mod_perl
MP_USE_DSO => 1
*** /usr/sbin/apache2 -V
Server version: Apache/2.4.34 (Debian)
Server built: 2018-07-27T19:37:37
Server's Module Magic Number: 20120211:79
Server loaded: APR 1.6.3, APR-UTIL 1.6.1
Compiled using: APR 1.6.3, APR-UTIL 1.6.1
Architecture: 64-bit
Server MPM: event
threaded: yes (fixed thread count)
forked: yes (variable process count)
Server compiled with....
-D APR_HAS_SENDFILE
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
-D APR_USE_SYSVSEM_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D DYNAMIC_MODULE_LIMIT=256
-D HTTPD_ROOT="/etc/apache2"
-D SUEXEC_BIN="/usr/lib/apache2/suexec"
-D DEFAULT_PIDLOG="/var/run/apache2.pid"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="mime.types"
-D SERVER_CONFIG_FILE="apache2.conf"
*** /usr/bin/ldd /usr/sbin/apache2
linux-vdso.so.1 (0x00007ffdc0dfa000)
libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f9abcf5b000)
libaprutil-1.so.0 => /lib/x86_64-linux-gnu/libaprutil-1.so.0
(0x00007f9abcd2f000)
libapr-1.so.0 => /lib/x86_64-linux-gnu/libapr-1.so.0
(0x00007f9abccf6000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
(0x00007f9abccd5000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9abcb18000)
libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f9abcb0f000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f9abcb03000)
libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1
(0x00007f9abcac9000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f9abcac4000)
libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1
(0x00007f9abc892000)
/lib64/ld-linux-x86-64.so.2 (0x00007f9abd085000)
*** (apr|apu)-config linking info
-L/usr/lib/x86_64-linux-gnu -laprutil-1
-L/usr/lib/x86_64-linux-gnu -lapr-1
*** /usr/local/bin/perl -V
Summary of my perl5 (revision 5 version 29 subversion 2) configuration:
Commit id: 48ae8dc39c5996cfd74bb0fefa5026f48ef8445e
Platform:
osname=linux
osvers=4.17.0-1-amd64
archname=x86_64-linux-thread-multi
uname='linux testing 4.17.0-1-amd64 #1 smp debian 4.17.8-1
(2018-07-20) x86_64 gnulinux '
config_args='-es -Dusethreads -A ccflags=-fPIC'
hint=recommended
useposix=true
d_sigaction=define
useithreads=define
usemultiplicity=define
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
bincompat5005=undef
Compiler:
cc='cc'
ccflags ='-D_REENTRANT -D_GNU_SOURCE -fPIC -fwrapv
-fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
optimize='-O2'
cppflags='-D_REENTRANT -D_GNU_SOURCE -fPIC -fwrapv
-fno-strict-aliasing -pipe -fstack-protector-strong
-I/usr/local/include'
ccversion=''
gccversion='8.2.0'
gccosandvers=''
intsize=4
longsize=8
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=16
longdblkind=3
ivtype='long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='off_t'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries:
ld='cc'
ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/8/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /lib64 /usr/lib64
libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.27.so
so=so
useshrplib=true
libperl=libperl.so
gnulibc_version='2.27'
Dynamic Linking:
dlsrc=dl_dlopen.xs
dlext=so
d_dlsymun=undef
ccdlflags='-Wl,-E
-Wl,-rpath,/usr/local/lib/perl5/5.29.2/x86_64-linux-thread-multi/CORE'
cccdlflags='-fPIC'
lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'
Characteristics of this binary (from libperl):
Compile-time options:
HAS_TIMES
MULTIPLICITY
PERLIO_LAYERS
PERL_COPY_ON_WRITE
PERL_DONT_CREATE_GVSV
PERL_IMPLICIT_CONTEXT
PERL_MALLOC_WRAP
PERL_OP_PARENT
PERL_PRESERVE_IVUV
USE_64_BIT_ALL
USE_64_BIT_INT
USE_ITHREADS
USE_LARGE_FILES
USE_LOCALE
USE_LOCALE_COLLATE
USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC
USE_LOCALE_TIME
USE_PERLIO
USE_PERL_ATOF
USE_REENTRANT_API
Built under linux
Compiled at Aug 14 2018 23:09:13
%ENV:
PERL_LWP_USE_HTTP_10="1"
@INC:
/usr/local/lib/perl5/site_perl/5.29.2/x86_64-linux-thread-multi
/usr/local/lib/perl5/site_perl/5.29.2
/usr/local/lib/perl5/5.29.2/x86_64-linux-thread-multi
/usr/local/lib/perl5/5.29.2
*** Packages of interest status:
Apache2 : -
Apache2::Request : -
CGI : -
ExtUtils::MakeMaker: 7.34
LWP : -
mod_perl : -
mod_perl2 : 2.000011
3. This is the core dump trace: (if you get a core dump):
root@testing:/var/log/apache2# gdb /usr/sbin/apache2
GNU gdb (Debian 8.1-4) 8.1
(gdb) run -DONE_PROCESS -DNO_DETACH -k start
Starting program: /usr/sbin/apache2 -DONE_PROCESS -DNO_DETACH -k start
[Thread debugging using libthread_db enabled]
Using host libthread_db library
"/lib/x86_64-linux-gnu/libthread_db.so.1".
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff6fe3d90 in ?? ()
(gdb) bt
#0 0x00007ffff6fe3d90 in ?? ()
#1 <signal handler called>
#2 0x00007ffff7cde214 in __waitpid (pid=15194, stat_loc=0x7fffffffe2c4,
options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:30
#3 0x00007ffff6d68025 in Perl_wait4pid ()
from /usr/local/lib/perl5/5.29.2/x86_64-linux-thread-multi/CORE/libperl.so
#4 0x00007ffff6d6892e in Perl_my_pclose ()
from /usr/local/lib/perl5/5.29.2/x86_64-linux-thread-multi/CORE/libperl.so
#5 0x00007ffff6dd5b06 in Perl_pp_backtick ()
from /usr/local/lib/perl5/5.29.2/x86_64-linux-thread-multi/CORE/libperl.so
#6 0x00007ffff6d84bf6 in Perl_runops_standard ()
from /usr/local/lib/perl5/5.29.2/x86_64-linux-thread-multi/CORE/libperl.so
#7 0x00007ffff6cf9726 in Perl_eval_sv ()
from /usr/local/lib/perl5/5.29.2/x86_64-linux-thread-multi/CORE/libperl.so
#8 0x00007ffff6cf9cf5 in Perl_require_pv ()
from /usr/local/lib/perl5/5.29.2/x86_64-linux-thread-multi/CORE/libperl.so
#9 0x00007ffff779ecce in modperl_require_file ()
from /usr/lib/apache2/modules/mod_perl.so
#10 0x00007ffff7797e4e in modperl_config_apply_PerlRequire ()
from /usr/lib/apache2/modules/mod_perl.so
#11 0x00007ffff77959ac in modperl_startup.localalias ()
from /usr/lib/apache2/modules/mod_perl.so
#12 0x00007ffff77957c3 in modperl_startup.localalias ()
from /usr/lib/apache2/modules/mod_perl.so
#13 0x00007ffff7795c26 in modperl_init ()
from /usr/lib/apache2/modules/mod_perl.so
#14 0x00007ffff7795d96 in modperl_hook_init ()
from /usr/lib/apache2/modules/mod_perl.so
#15 0x00005555555af2b3 in ap_run_open_logs ()
#16 0x000055555558aef2 in main ()
This report was generated by /usr/local/bin/mp2bug on Wed Aug 15
21:27:41 2018 GMT.
-------------8<---------- End Bug Report --------------8<----------
1. Problem Description:
To reproduce the problem, start from Debian vanilla server (I used
testing (buster)).
Install apache2, libapache2-mod-perl2, libev-perl
(I also used development version of perl, mod_perl and EV with the
same result)
Create 2 perl files with following content:
handler1.pl:
#!/usr/bin/perl
my $pwd = `/bin/pwd`;
1;
handler2.pl:
#!/usr/bin/perl
use EV;
my $w = EV::signal 'CHLD', sub {
print "SIGCHLD received\n";
};
1;
Reference them in two PerlRequire directive, either
in /etc/apache2/apache2.conf or in a separate conf file
in /etc/apache2/conf-enabled:
# order is important
PerlRequire "/path/to/handler1.pl"
PerlRequire "/path/to/handler2.pl"
==> Apache will crash on startup.
I did an in-depth analysis of this bug. Here is what happens:
The backtick operator in handler1.pl internally calls waitpid, which
internally waits for the SIGCHLD signal.
EV::signal 'CHLD' installs a signal handler using sigaction (see
EV-4.22.0: libev/ev.c::ev_signal_start). There is no provision to remove
this signal handler. The handler is located in EV.so, loaded by the XS
loader.
apache2 goes through 3 stages on startup: pre-config, destroy-config,
create-config
In the pre-config stage, the 2 PerlRequire commands are executed and
the signal handler is installed at process level.
In the destroy-config stage, all modules are removed, including
mod_perl, which consists in removing all XS modules (see mod_perl:
src/modules/perl/modperl_interp.c::modperl_interp_destroy())
The removal of EV.so does not restore the signal handler as there is
no cleanup in EV.so.
At this point the signal handler is still registered at process level
but points to invalid memory!
In the create-config stage, the config is recreated, handler1.pl is
reloaded and the backtick operator triggers a SIGCHLD, which causes a
SIGEV because the signal handler is not in memory (the EV.so library
will only be loaded when the second PerlRequire is executed).
Conclusion:
Unloading the XS modules relies on the modules to cleanup the global
state properly but it's clearly not the case of the EV module.
I tested a fix that consists in skipping the unloading of the XS
modules. While it works for my case, it is not a proper solution either
as there are circumstances where it would cause other bugs.
Ideally the EV module should register a cleanup function to restore
the signal handlers on unload.
There are similar potential issues when an XS module register some op
checker in PL_check static array: unloading the XS module does not
remove the handler because Perl does not provide an API to do it.
Hopefully, in this case unloading mod_perl will normally causes
libperl to unload as well, which cleans the PL_check array.
Clearly, Perl is not designed to be unloaded-reloaded.
2. Used Components and their Configuration:
*** mod_perl version 2.000011
*** using /home/ben/src/mod_perl-2.0/lib/Apache2/BuildConfig.pm
*** Makefile.PL options:
MP_APR_LIB => aprext
MP_APXS => /usr/bin/apxs
MP_COMPAT_1X => 1
MP_GENERATE_XS => 1
MP_LIBNAME => mod_perl
MP_USE_DSO => 1
*** /usr/sbin/apache2 -V
Server version: Apache/2.4.34 (Debian)
Server built: 2018-07-27T19:37:37
Server's Module Magic Number: 20120211:79
Server loaded: APR 1.6.3, APR-UTIL 1.6.1
Compiled using: APR 1.6.3, APR-UTIL 1.6.1
Architecture: 64-bit
Server MPM: event
threaded: yes (fixed thread count)
forked: yes (variable process count)
Server compiled with....
-D APR_HAS_SENDFILE
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
-D APR_USE_SYSVSEM_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D DYNAMIC_MODULE_LIMIT=256
-D HTTPD_ROOT="/etc/apache2"
-D SUEXEC_BIN="/usr/lib/apache2/suexec"
-D DEFAULT_PIDLOG="/var/run/apache2.pid"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="mime.types"
-D SERVER_CONFIG_FILE="apache2.conf"
*** /usr/bin/ldd /usr/sbin/apache2
linux-vdso.so.1 (0x00007ffdc0dfa000)
libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f9abcf5b000)
libaprutil-1.so.0 => /lib/x86_64-linux-gnu/libaprutil-1.so.0
(0x00007f9abcd2f000)
libapr-1.so.0 => /lib/x86_64-linux-gnu/libapr-1.so.0
(0x00007f9abccf6000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
(0x00007f9abccd5000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f9abcb18000)
libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f9abcb0f000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f9abcb03000)
libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1
(0x00007f9abcac9000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f9abcac4000)
libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1
(0x00007f9abc892000)
/lib64/ld-linux-x86-64.so.2 (0x00007f9abd085000)
*** (apr|apu)-config linking info
-L/usr/lib/x86_64-linux-gnu -laprutil-1
-L/usr/lib/x86_64-linux-gnu -lapr-1
*** /usr/local/bin/perl -V
Summary of my perl5 (revision 5 version 29 subversion 2) configuration:
Commit id: 48ae8dc39c5996cfd74bb0fefa5026f48ef8445e
Platform:
osname=linux
osvers=4.17.0-1-amd64
archname=x86_64-linux-thread-multi
uname='linux testing 4.17.0-1-amd64 #1 smp debian 4.17.8-1
(2018-07-20) x86_64 gnulinux '
config_args='-es -Dusethreads -A ccflags=-fPIC'
hint=recommended
useposix=true
d_sigaction=define
useithreads=define
usemultiplicity=define
use64bitint=define
use64bitall=define
uselongdouble=undef
usemymalloc=n
default_inc_excludes_dot=define
bincompat5005=undef
Compiler:
cc='cc'
ccflags ='-D_REENTRANT -D_GNU_SOURCE -fPIC -fwrapv
-fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
optimize='-O2'
cppflags='-D_REENTRANT -D_GNU_SOURCE -fPIC -fwrapv
-fno-strict-aliasing -pipe -fstack-protector-strong
-I/usr/local/include'
ccversion=''
gccversion='8.2.0'
gccosandvers=''
intsize=4
longsize=8
ptrsize=8
doublesize=8
byteorder=12345678
doublekind=3
d_longlong=define
longlongsize=8
d_longdbl=define
longdblsize=16
longdblkind=3
ivtype='long'
ivsize=8
nvtype='double'
nvsize=8
Off_t='off_t'
lseeksize=8
alignbytes=8
prototype=define
Linker and Libraries:
ld='cc'
ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/8/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib /lib64 /usr/lib64
libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.27.so
so=so
useshrplib=true
libperl=libperl.so
gnulibc_version='2.27'
Dynamic Linking:
dlsrc=dl_dlopen.xs
dlext=so
d_dlsymun=undef
ccdlflags='-Wl,-E
-Wl,-rpath,/usr/local/lib/perl5/5.29.2/x86_64-linux-thread-multi/CORE'
cccdlflags='-fPIC'
lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'
Characteristics of this binary (from libperl):
Compile-time options:
HAS_TIMES
MULTIPLICITY
PERLIO_LAYERS
PERL_COPY_ON_WRITE
PERL_DONT_CREATE_GVSV
PERL_IMPLICIT_CONTEXT
PERL_MALLOC_WRAP
PERL_OP_PARENT
PERL_PRESERVE_IVUV
USE_64_BIT_ALL
USE_64_BIT_INT
USE_ITHREADS
USE_LARGE_FILES
USE_LOCALE
USE_LOCALE_COLLATE
USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC
USE_LOCALE_TIME
USE_PERLIO
USE_PERL_ATOF
USE_REENTRANT_API
Built under linux
Compiled at Aug 14 2018 23:09:13
%ENV:
PERL_LWP_USE_HTTP_10="1"
@INC:
/usr/local/lib/perl5/site_perl/5.29.2/x86_64-linux-thread-multi
/usr/local/lib/perl5/site_perl/5.29.2
/usr/local/lib/perl5/5.29.2/x86_64-linux-thread-multi
/usr/local/lib/perl5/5.29.2
*** Packages of interest status:
Apache2 : -
Apache2::Request : -
CGI : -
ExtUtils::MakeMaker: 7.34
LWP : -
mod_perl : -
mod_perl2 : 2.000011
3. This is the core dump trace: (if you get a core dump):
root@testing:/var/log/apache2# gdb /usr/sbin/apache2
GNU gdb (Debian 8.1-4) 8.1
(gdb) run -DONE_PROCESS -DNO_DETACH -k start
Starting program: /usr/sbin/apache2 -DONE_PROCESS -DNO_DETACH -k start
[Thread debugging using libthread_db enabled]
Using host libthread_db library
"/lib/x86_64-linux-gnu/libthread_db.so.1".
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff6fe3d90 in ?? ()
(gdb) bt
#0 0x00007ffff6fe3d90 in ?? ()
#1 <signal handler called>
#2 0x00007ffff7cde214 in __waitpid (pid=15194, stat_loc=0x7fffffffe2c4,
options=0) at ../sysdeps/unix/sysv/linux/waitpid.c:30
#3 0x00007ffff6d68025 in Perl_wait4pid ()
from /usr/local/lib/perl5/5.29.2/x86_64-linux-thread-multi/CORE/libperl.so
#4 0x00007ffff6d6892e in Perl_my_pclose ()
from /usr/local/lib/perl5/5.29.2/x86_64-linux-thread-multi/CORE/libperl.so
#5 0x00007ffff6dd5b06 in Perl_pp_backtick ()
from /usr/local/lib/perl5/5.29.2/x86_64-linux-thread-multi/CORE/libperl.so
#6 0x00007ffff6d84bf6 in Perl_runops_standard ()
from /usr/local/lib/perl5/5.29.2/x86_64-linux-thread-multi/CORE/libperl.so
#7 0x00007ffff6cf9726 in Perl_eval_sv ()
from /usr/local/lib/perl5/5.29.2/x86_64-linux-thread-multi/CORE/libperl.so
#8 0x00007ffff6cf9cf5 in Perl_require_pv ()
from /usr/local/lib/perl5/5.29.2/x86_64-linux-thread-multi/CORE/libperl.so
#9 0x00007ffff779ecce in modperl_require_file ()
from /usr/lib/apache2/modules/mod_perl.so
#10 0x00007ffff7797e4e in modperl_config_apply_PerlRequire ()
from /usr/lib/apache2/modules/mod_perl.so
#11 0x00007ffff77959ac in modperl_startup.localalias ()
from /usr/lib/apache2/modules/mod_perl.so
#12 0x00007ffff77957c3 in modperl_startup.localalias ()
from /usr/lib/apache2/modules/mod_perl.so
#13 0x00007ffff7795c26 in modperl_init ()
from /usr/lib/apache2/modules/mod_perl.so
#14 0x00007ffff7795d96 in modperl_hook_init ()
from /usr/lib/apache2/modules/mod_perl.so
#15 0x00005555555af2b3 in ap_run_open_logs ()
#16 0x000055555558aef2 in main ()
This report was generated by /usr/local/bin/mp2bug on Wed Aug 15
21:27:41 2018 GMT.
-------------8<---------- End Bug Report --------------8<----------