Mailing List Archive

[Bug 67379] Debugging Memory leak in httpd , leads to generate core
https://bz.apache.org/bugzilla/show_bug.cgi?id=67379

shahbhav1993@gmail.com changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |shahbhav1993@gmail.com

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 67379] Debugging Memory leak in httpd , leads to generate core [ In reply to ]
https://bz.apache.org/bugzilla/show_bug.cgi?id=67379

--- Comment #1 from Yann Ylavic <ylavic.dev@gmail.com> ---
(In reply to shahbhav1993 from comment #0)

> -bash# httpd -M
> Loaded Modules:
> ...
> auth_ax_module (shared)
> auth_jwt_module (shared)
> axapi_1_module (shared)
> ...
> web_apis_module (shared)
> ...

Did you try without these (third-party) modules?


> -bash# httpd -V
> Server version: Apache/2.4.54 (Unix)
> Server built: Jun 30 2022 13:43:50
> Server's Module Magic Number: 20120211:124
> Server loaded: APR 1.7.4, APR-UTIL 1.6.3, PCRE 8.35 2014-04-04
> Compiled using: APR 1.5.2, APR-UTIL 1.5.2, PCRE 8.32 2012-11-30
> Architecture: 64-bit
> Server MPM: prefork
> threaded: no
> forked: yes (variable process count)

This says prefork but gdb traces seem to indicate MPM event (explicit
LoadModule?), which one is used exactly?


> I can see /proc/parent_httpd_pid/statm output keep on increasing below I
> have attached sample output.
> 2023-08-25 10:45:02 179536 7679 5872 158 0 15097 0
> 2023-08-25 10:45:30 192359 7712 5901 158 0 15097 0
> 2023-08-25 10:50:30 218033 7723 5906 158 0 15125 0
> ...
> 2023-08-29 03:18:21 33181755 69652 59222 158 0 23737 0
> 2023-08-29 03:23:21 33220220 69689 59248 158 0 23733 0
> 2023-08-29 03:28:21 33245866 69769 59322 158 0 23733 0
> 2023-08-29 03:33:21 33284361 69813 59356 158 0 23759 0

This is pretty stable for me with both MPM prefork and event, but without the
third-party modules.

> I have tried to attach gdb. but not getting useful information.
> dump_all_pools command not working for me. what to do for that.
> I have attached logs for gdb.
> (gdb) dump_all_pools
> Undefined command: "dump_all_pools". Try "help".

You probably need the .gdbinit file provided by httpd
(https://github.com/apache/httpd/blob/trunk/.gdbinit) in your gdb environment
or working directory.

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 67379] Debugging Memory leak in httpd , leads to generate core [ In reply to ]
https://bz.apache.org/bugzilla/show_bug.cgi?id=67379

--- Comment #2 from shahbhav1993@gmail.com ---
(In reply to Yann Ylavic from comment #1)
> (In reply to shahbhav1993 from comment #0)
>
> > -bash# httpd -M
> > Loaded Modules:
> > ...
> > auth_ax_module (shared)
> > auth_jwt_module (shared)
> > axapi_1_module (shared)
> > ...
> > web_apis_module (shared)
> > ...
>
> Did you try without these (third-party) modules?
I didn't try. I will try to remove these 4 modules, which you highlighted.
Just to confirm other modules other than these 4, are supported by apache?
>
>
> > -bash# httpd -V
> > Server version: Apache/2.4.54 (Unix)
> > Server built: Jun 30 2022 13:43:50
> > Server's Module Magic Number: 20120211:124
> > Server loaded: APR 1.7.4, APR-UTIL 1.6.3, PCRE 8.35 2014-04-04
> > Compiled using: APR 1.5.2, APR-UTIL 1.5.2, PCRE 8.32 2012-11-30
> > Architecture: 64-bit
> > Server MPM: prefork
> > threaded: no
> > forked: yes (variable process count)
>
> This says prefork but gdb traces seem to indicate MPM event (explicit
> LoadModule?), which one is used exactly?

I am using prefork. here is log from gdb.
oaded symbols for /etc/httpd/modules/mod_cgid.so
Reading symbols from /etc/httpd/modules/mod_unique_id.so...Missing separate
debuginfo for /etc/httpd/modules/mod_unique_id.so
Try: yum --enablerepo='*debug*' install
/usr/lib/debug/.build-id/44/d70e82fd0afc88216182b8840f2965753b745d.debug
Reading symbols from /etc/httpd/modules/mod_unique_id.so...(no debugging
symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /etc/httpd/modules/mod_unique_id.so
0x00007f4fe7c2eb23 in __select_nocancel () from /lib64/libc.so.6
(gdb) c
Continuing.

Program received signal SIGHUP, Hangup.
0x00007f4fe7c2eb23 in __select_nocancel () from /lib64/libc.so.6
(gdb) bt
#0 0x00007f4fe7c2eb23 in __select_nocancel () at /lib64/libc.so.6
#1 0x00007f4fe83558a5 in apr_sleep () at /lib64/libapr-1.so.0
#2 0x000055f5eb44a059 in ap_wait_or_timeout ()
#3 0x00007f3526282682 in prefork_run () at
/etc/httpd/modules/mod_mpm_prefork.so
#4 0x000055f5eb4494ae in ap_run_mpm ()
#5 0x000055f5eb44127d in main ()
(gdb) c
Continuing.

Program received signal SIGHUP, Hangup.
0x00007f4fe7b6f657 in kill () from /lib64/libc.so.6
(gdb) bt
#0 0x00007f4fe7b6f657 in kill () at /lib64/libc.so.6
#1 0x00007f3526282dc1 in prefork_run () at
/etc/httpd/modules/mod_mpm_prefork.so
#2 0x000055f5eb4494ae in ap_run_mpm ()
#3 0x000055f5eb44127d in main ()
(gdb) c
Continuing.

Program received signal SIGPIPE, Broken pipe.
0x00007f4fe81196e0 in __write_nocancel () from /lib64/libpthread.so.0
(gdb) bt full
#0 0x00007f4fe81196e0 in __write_nocancel () at /lib64/libpthread.so.0
#1 0x00007f4fe8345e4c in apr_file_write () at /lib64/libapr-1.so.0
#2 0x00007f4fe834462e in apr_file_write_full () at /lib64/libapr-1.so.0
#3 0x000055f5eb46e4c4 in log_error_core ()
#4 0x000055f5eb46ed78 in ap_log_error_ ()
#5 0x00007f3526282e0a in prefork_run () at
/etc/httpd/modules/mod_mpm_prefork.so
#6 0x000055f5eb4494ae in ap_run_mpm ()
#7 0x000055f5eb44127d in main ()
(gdb) c
Continuing.
warning: Could not load shared library symbols for 4 libraries, e.g.
../lib/libnetsnmpagent.so.15.
Use the "info sharedlibrary" command to see the complete listing.
Do you need "set solib-search-path" or "set sysroot"?
[Detaching after fork from child process 28341]
[Detaching after fork from child process 28345]
[Detaching after fork from child process 28350]
[Detaching after fork from child process 28354]
[Detaching after fork from child process 28365]
[Detaching after fork from child process 28366]
[Detaching after fork from child process 28367]
[Detaching after fork from child process 28368]
[Detaching after fork from child process 28369]
[Detaching after fork from child process 28370]

Program received signal SIGINT, Interrupt.
0x00007f4fe7c2eb23 in __select_nocancel () from /lib64/libc.so.6
(gdb) bt full
#0 0x00007f4fe7c2eb23 in __select_nocancel () at /lib64/libc.so.6
#1 0x00007f4fe83558a5 in apr_sleep () at /lib64/libapr-1.so.0
#2 0x000055f5eb44a059 in ap_wait_or_timeout ()
#3 0x00007f3520651682 in prefork_run () at
/etc/httpd/modules/mod_mpm_prefork.so
#4 0x000055f5eb4494ae in ap_run_mpm ()
#5 0x000055f5eb44127d in main ()
(gdb)

>
>
> > I can see /proc/parent_httpd_pid/statm output keep on increasing below I
> > have attached sample output.
> > 2023-08-25 10:45:02 179536 7679 5872 158 0 15097 0
> > 2023-08-25 10:45:30 192359 7712 5901 158 0 15097 0
> > 2023-08-25 10:50:30 218033 7723 5906 158 0 15125 0
> > ...
> > 2023-08-29 03:18:21 33181755 69652 59222 158 0 23737 0
> > 2023-08-29 03:23:21 33220220 69689 59248 158 0 23733 0
> > 2023-08-29 03:28:21 33245866 69769 59322 158 0 23733 0
> > 2023-08-29 03:33:21 33284361 69813 59356 158 0 23759 0
>
> This is pretty stable for me with both MPM prefork and event, but without
> the third-party modules.

this output increase for me when , I do httpd -k restart every 3 min interval,
every restart it increase gradually. are you doing restart and includes other
shared modules without third-party modules?
do you mean without these 4 modules auth_ax_module (shared) ,auth_jwt_module
(shared), axapi_1_module (shared), web_apis_module (shared) you tried?
If so I will try without these four module.

> > I have tried to attach gdb. but not getting useful information.
> > dump_all_pools command not working for me. what to do for that.
> > I have attached logs for gdb.
> > (gdb) dump_all_pools
> > Undefined command: "dump_all_pools". Try "help".
>
> You probably need the .gdbinit file provided by httpd
> (https://github.com/apache/httpd/blob/trunk/.gdbinit) in your gdb
> environment or working directory.

how to include this while doing gdb?
I usually do "gdb -p pid"

Thanks & Regards,
Bhav

--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org