Mailing List Archive

Author: stevehay
Date: Fri Jul 21 10:16:16 2023
New Revision: 1911170

URL: http://svn.apache.org/viewvc?rev=1911170&view=rev
Log:
Remove use of Linux::Smaps - it is slow and uses wrong statistics anyway

Patch by Zefram <zefram@fysh.org> on CPAN RT#93757

Modified:
perl/Apache-SizeLimit/trunk/Makefile.PL
perl/Apache-SizeLimit/trunk/README
perl/Apache-SizeLimit/trunk/lib/Apache/SizeLimit.pm
perl/Apache-SizeLimit/trunk/lib/Apache/SizeLimit/Core.pm
perl/Apache-SizeLimit/trunk/lib/Apache2/SizeLimit.pm
perl/Apache-SizeLimit/trunk/t/apache/all.t
perl/Apache-SizeLimit/trunk/t/apache2/all.t
perl/Apache-SizeLimit/trunk/t/response/TestApache/basic.pm
perl/Apache-SizeLimit/trunk/t/response/TestApache2/basic.pm

Modified: perl/Apache-SizeLimit/trunk/Makefile.PL
URL: http://svn.apache.org/viewvc/perl/Apache-SizeLimit/trunk/Makefile.PL?rev=1911170&r1=1911169&r2=1911170&view=diff
==============================================================================
--- perl/Apache-SizeLimit/trunk/Makefile.PL (original)
+++ perl/Apache-SizeLimit/trunk/Makefile.PL Fri Jul 21 10:16:16 2023
@@ -23,9 +23,6 @@ else {
unless ( $ARGV[0] eq '--dist' || $ENV{MOD_PERL_2_BUILD} ) {
if ( $Config{'osname'} eq 'linux' ) {
$prereqs{'Linux::Pid'} = 0;
- if ( -e '/proc/self/smaps' ) {
- $prereqs{'Linux::Smaps'} = 0;
- }
}
elsif ( $Config{'osname'} =~ /(bsd|aix)/i ) {
$prereqs{'BSD::Resource'} = 0;

Modified: perl/Apache-SizeLimit/trunk/README
URL: http://svn.apache.org/viewvc/perl/Apache-SizeLimit/trunk/README?rev=1911170&r1=1911169&r2=1911170&view=diff
==============================================================================
--- perl/Apache-SizeLimit/trunk/README (original)
+++ perl/Apache-SizeLimit/trunk/README Fri Jul 21 10:16:16 2023
@@ -140,77 +140,6 @@ PER-PLATFORM BEHAVIOR
"Apache2::SizeLimit->set_check_interval()" to reduce how often this read
happens.

- As of linux 2.6, /proc/self/statm does not report the amount of memory
- shared by the copy-on-write mechanism as shared memory. This means that
- decisions made based on shared memory as reported by that interface are
- inherently wrong.
-
- However, as of the 2.6.14 release of the kernel, there is
- /proc/self/smaps entry for each process. /proc/self/smaps reports
- various sizes for each memory segment of a process and allows us to
- count the amount of shared memory correctly.
-
- If "Apache2::SizeLimit" detects a kernel that supports /proc/self/smaps
- and the "Linux::Smaps" module is installed it will use that module
- instead of /proc/self/statm.
-
- Reading /proc/self/smaps is expensive compared to /proc/self/statm. It
- must look at each page table entry of a process. Further, on
- multiprocessor systems the access is synchronized with spinlocks. Again,
- you might consider using "Apache2::SizeLimit->set_check_interval()".
-
- Copy-on-write and Shared Memory
- The following example shows the effect of copy-on-write:
-
- <Perl>
- require Apache2::SizeLimit;
- package X;
- use strict;
- use Apache2::Const -compile => qw(OK);
-
- my $x = "a" x (1024*1024);
-
- sub handler {
- my $r = shift;
- my ($size, $shared) = $Apache2::SizeLimit->_check_size();
- $x =~ tr/a/b/;
- my ($size2, $shared2) = $Apache2::SizeLimit->_check_size();
- $r->content_type('text/plain');
- $r->print("1: size=$size shared=$shared\n");
- $r->print("2: size=$size2 shared=$shared2\n");
- return OK;
- }
- </Perl>
-
- <Location /X>
- SetHandler modperl
- PerlResponseHandler X
- </Location>
-
- The parent Apache process allocates memory for the string in $x. The
- "tr"-command then overwrites all "a" with "b" if the handler is called
- with an argument. This write is done in place, thus, the process size
- doesn't change. Only $x is not shared anymore by means of copy-on-write
- between the parent and the child.
-
- If /proc/self/smaps is available curl shows:
-
- r2@s93:~/work/mp2> curl http://localhost:8181/X
- 1: size=13452 shared=7456
- 2: size=13452 shared=6432
-
- Shared memory has lost 1024 kB. The process' overall size remains
- unchanged.
-
- Without /proc/self/smaps it says:
-
- r2@s93:~/work/mp2> curl http://localhost:8181/X
- 1: size=13052 shared=3628
- 2: size=13052 shared=3636
-
- One can see the kernel lies about the shared memory. It simply doesn't
- count copy-on-write pages as shared.
-
solaris 2.6 and above
For solaris we simply retrieve the size of /proc/self/as, which contains
the address-space image of the process, and convert to KB. Shared memory
@@ -275,7 +204,6 @@ DEPRECATED APIS
* $Apache2::SizeLimit::MIN_SHARE_SIZE
* $Apache2::SizeLimit::MAX_UNSHARED_SIZE
* $Apache2::SizeLimit::CHECK_EVERY_N_REQUESTS
- * $Apache2::SizeLimit::USE_SMAPS

Direct use of these globals is deprecated, but will continue to work for
the foreseeable future.

Modified: perl/Apache-SizeLimit/trunk/lib/Apache/SizeLimit.pm
URL: http://svn.apache.org/viewvc/perl/Apache-SizeLimit/trunk/lib/Apache/SizeLimit.pm?rev=1911170&r1=1911169&r2=1911170&view=diff
==============================================================================
--- perl/Apache-SizeLimit/trunk/lib/Apache/SizeLimit.pm (original)
+++ perl/Apache-SizeLimit/trunk/lib/Apache/SizeLimit.pm Fri Jul 21 10:16:16 2023
@@ -34,7 +34,6 @@ use Apache::SizeLimit::Core qw(
$MIN_SHARE_SIZE
$CHECK_EVERY_N_REQUESTS
$START_TIME
- $USE_SMAPS
$VERSION
$REQUEST_COUNT
);
@@ -323,78 +322,6 @@ are worried about performance, you can c
Apache::SizeLimit->set_check_interval() >> to reduce how often this
read happens.

-As of linux 2.6, F</proc/self/statm> does not report the amount of
-memory shared by the copy-on-write mechanism as shared memory. This
-means that decisions made based on shared memory as reported by that
-interface are inherently wrong.
-
-However, as of the 2.6.14 release of the kernel, there is
-F</proc/self/smaps> entry for each process. F</proc/self/smaps>
-reports various sizes for each memory segment of a process and allows
-us to count the amount of shared memory correctly.
-
-If C<Apache::SizeLimit> detects a kernel that supports
-F</proc/self/smaps> and the C<Linux::Smaps> module is installed it
-will use that module instead of F</proc/self/statm>.
-
-Reading F</proc/self/smaps> is expensive compared to
-F</proc/self/statm>. It must look at each page table entry of a
-process. Further, on multiprocessor systems the access is
-synchronized with spinlocks. Again, you might consider using C<<
-Apache::SizeLimit->set_check_interval() >>.
-
-=head3 Copy-on-write and Shared Memory
-
-The following example shows the effect of copy-on-write:
-
- <Perl>
- require Apache::SizeLimit;
- package X;
- use strict;
- use Apache::Constants qw(OK);
-
- my $x = "a" x (1024*1024);
-
- sub handler {
- my $r = shift;
- my ($size, $shared) = $Apache::SizeLimit->_check_size();
- $x =~ tr/a/b/;
- my ($size2, $shared2) = $Apache::SizeLimit->_check_size();
- $r->content_type('text/plain');
- $r->print("1: size=$size shared=$shared\n");
- $r->print("2: size=$size2 shared=$shared2\n");
- return OK;
- }
- </Perl>
-
- <Location /X>
- SetHandler modperl
- PerlResponseHandler X
- </Location>
-
-The parent Apache process allocates memory for the string in
-C<$x>. The C<tr>-command then overwrites all "a" with "b" if the
-handler is called with an argument. This write is done in place, thus,
-the process size doesn't change. Only C<$x> is not shared anymore by
-means of copy-on-write between the parent and the child.
-
-If F</proc/self/smaps> is available curl shows:
-
- r2@s93:~/work/mp2> curl http://localhost:8181/X
- 1: size=13452 shared=7456
- 2: size=13452 shared=6432
-
-Shared memory has lost 1024 kB. The process' overall size remains unchanged.
-
-Without F</proc/self/smaps> it says:
-
- r2@s93:~/work/mp2> curl http://localhost:8181/X
- 1: size=13052 shared=3628
- 2: size=13052 shared=3636
-
-One can see the kernel lies about the shared memory. It simply doesn't
-count copy-on-write pages as shared.
-
=head2 solaris 2.6 and above

For solaris we simply retrieve the size of F</proc/self/as>, which
@@ -475,8 +402,6 @@ memory size limits:

=item * $Apache::SizeLimit::CHECK_EVERY_N_REQUESTS

-=item * $Apache::SizeLimit::USE_SMAPS
-
=back

Direct use of these globals is deprecated, but will continue to work

Modified: perl/Apache-SizeLimit/trunk/lib/Apache/SizeLimit/Core.pm
URL: http://svn.apache.org/viewvc/perl/Apache-SizeLimit/trunk/lib/Apache/SizeLimit/Core.pm?rev=1911170&r1=1911169&r2=1911170&view=diff
==============================================================================
--- perl/Apache-SizeLimit/trunk/lib/Apache/SizeLimit/Core.pm (original)
+++ perl/Apache-SizeLimit/trunk/lib/Apache/SizeLimit/Core.pm Fri Jul 21 10:16:16 2023
@@ -24,7 +24,6 @@ use Exporter;
use vars qw(
$VERSION
$REQUEST_COUNT
- $USE_SMAPS

$MAX_PROCESS_SIZE
$MAX_UNSHARED_SIZE
@@ -41,7 +40,6 @@ use vars qw(
@EXPORT_OK = qw(
$VERSION
$REQUEST_COUNT
- $USE_SMAPS
$MAX_PROCESS_SIZE
$MAX_UNSHARED_SIZE
$MIN_SHARE_SIZE
@@ -140,17 +138,8 @@ BEGIN {
}
elsif ($Config{'osname'} eq 'linux') {
_load('Linux::Pid');
-
*_platform_getppid = \&_linux_getppid;
-
- if (eval { require Linux::Smaps && Linux::Smaps->new($$) }) {
- $USE_SMAPS = 1;
- *_platform_check_size = \&_linux_smaps_size_check;
- }
- else {
- $USE_SMAPS = 0;
- *_platform_check_size = \&_linux_size_check;
- }
+ *_platform_check_size = \&_linux_size_check;
}
elsif ($Config{'osname'} =~ /(?:bsd|aix)/i) {
# on OSX, getrusage() is returning 0 for proc & shared size.
@@ -170,17 +159,6 @@ BEGIN {
}
}

-sub _linux_smaps_size_check {
- my $class = shift;
-
- return $class->_linux_size_check() unless $USE_SMAPS;
-
- my $s = Linux::Smaps->new($$)->all;
- return ($s->size,
- $s->shared_clean + $s->shared_dirty,
- $s->private_clean + $s->private_dirty);
-}
-
sub _linux_size_check {
my $class = shift;


Modified: perl/Apache-SizeLimit/trunk/lib/Apache2/SizeLimit.pm
URL: http://svn.apache.org/viewvc/perl/Apache-SizeLimit/trunk/lib/Apache2/SizeLimit.pm?rev=1911170&r1=1911169&r2=1911170&view=diff
==============================================================================
--- perl/Apache-SizeLimit/trunk/lib/Apache2/SizeLimit.pm (original)
+++ perl/Apache-SizeLimit/trunk/lib/Apache2/SizeLimit.pm Fri Jul 21 10:16:16 2023
@@ -42,7 +42,6 @@ use Apache::SizeLimit::Core qw(
$MIN_SHARE_SIZE
$CHECK_EVERY_N_REQUESTS
$START_TIME
- $USE_SMAPS
$VERSION
$REQUEST_COUNT
);
@@ -326,78 +325,6 @@ are worried about performance, you can c
Apache2::SizeLimit->set_check_interval() >> to reduce how often this
read happens.

-As of linux 2.6, F</proc/self/statm> does not report the amount of
-memory shared by the copy-on-write mechanism as shared memory. This
-means that decisions made based on shared memory as reported by that
-interface are inherently wrong.
-
-However, as of the 2.6.14 release of the kernel, there is
-F</proc/self/smaps> entry for each process. F</proc/self/smaps>
-reports various sizes for each memory segment of a process and allows
-us to count the amount of shared memory correctly.
-
-If C<Apache2::SizeLimit> detects a kernel that supports
-F</proc/self/smaps> and the C<Linux::Smaps> module is installed it
-will use that module instead of F</proc/self/statm>.
-
-Reading F</proc/self/smaps> is expensive compared to
-F</proc/self/statm>. It must look at each page table entry of a
-process. Further, on multiprocessor systems the access is
-synchronized with spinlocks. Again, you might consider using C<<
-Apache2::SizeLimit->set_check_interval() >>.
-
-=head3 Copy-on-write and Shared Memory
-
-The following example shows the effect of copy-on-write:
-
- <Perl>
- require Apache2::SizeLimit;
- package X;
- use strict;
- use Apache2::Const -compile => qw(OK);
-
- my $x = "a" x (1024*1024);
-
- sub handler {
- my $r = shift;
- my ($size, $shared) = $Apache2::SizeLimit->_check_size();
- $x =~ tr/a/b/;
- my ($size2, $shared2) = $Apache2::SizeLimit->_check_size();
- $r->content_type('text/plain');
- $r->print("1: size=$size shared=$shared\n");
- $r->print("2: size=$size2 shared=$shared2\n");
- return OK;
- }
- </Perl>
-
- <Location /X>
- SetHandler modperl
- PerlResponseHandler X
- </Location>
-
-The parent Apache process allocates memory for the string in
-C<$x>. The C<tr>-command then overwrites all "a" with "b" if the
-handler is called with an argument. This write is done in place, thus,
-the process size doesn't change. Only C<$x> is not shared anymore by
-means of copy-on-write between the parent and the child.
-
-If F</proc/self/smaps> is available curl shows:
-
- r2@s93:~/work/mp2> curl http://localhost:8181/X
- 1: size=13452 shared=7456
- 2: size=13452 shared=6432
-
-Shared memory has lost 1024 kB. The process' overall size remains unchanged.
-
-Without F</proc/self/smaps> it says:
-
- r2@s93:~/work/mp2> curl http://localhost:8181/X
- 1: size=13052 shared=3628
- 2: size=13052 shared=3636
-
-One can see the kernel lies about the shared memory. It simply doesn't
-count copy-on-write pages as shared.
-
=head2 solaris 2.6 and above

For solaris we simply retrieve the size of F</proc/self/as>, which
@@ -478,8 +405,6 @@ memory size limits:

=item * $Apache2::SizeLimit::CHECK_EVERY_N_REQUESTS

-=item * $Apache2::SizeLimit::USE_SMAPS
-
=back

Direct use of these globals is deprecated, but will continue to work

Modified: perl/Apache-SizeLimit/trunk/t/apache/all.t
URL: http://svn.apache.org/viewvc/perl/Apache-SizeLimit/trunk/t/apache/all.t?rev=1911170&r1=1911169&r2=1911170&view=diff
==============================================================================
--- perl/Apache-SizeLimit/trunk/t/apache/all.t (original)
+++ perl/Apache-SizeLimit/trunk/t/apache/all.t Fri Jul 21 10:16:16 2023
@@ -15,9 +15,6 @@ sub my_need {

if ( $Config{'osname'} eq 'linux' ) {
$ok = need_module('Linux::Pid');
- if ( -e '/proc/self/smaps' ) {
- $ok &= need_module('Linux::Smaps');
- }
}
elsif ( $Config{'osname'} =~ /(bsd|aix)/i ) {
$ok &= need_module('BSD::Resource');

Modified: perl/Apache-SizeLimit/trunk/t/apache2/all.t
URL: http://svn.apache.org/viewvc/perl/Apache-SizeLimit/trunk/t/apache2/all.t?rev=1911170&r1=1911169&r2=1911170&view=diff
==============================================================================
--- perl/Apache-SizeLimit/trunk/t/apache2/all.t (original)
+++ perl/Apache-SizeLimit/trunk/t/apache2/all.t Fri Jul 21 10:16:16 2023
@@ -15,9 +15,6 @@ sub my_need {

if ( $Config{'osname'} eq 'linux' ) {
$ok = need_module('Linux::Pid');
- if ( -e '/proc/self/smaps' ) {
- $ok &= need_module('Linux::Smaps');
- }
}
elsif ( $Config{'osname'} =~ /(bsd|aix)/i ) {
$ok &= need_module('BSD::Resource');

Modified: perl/Apache-SizeLimit/trunk/t/response/TestApache/basic.pm
URL: http://svn.apache.org/viewvc/perl/Apache-SizeLimit/trunk/t/response/TestApache/basic.pm?rev=1911170&r1=1911169&r2=1911170&view=diff
==============================================================================
--- perl/Apache-SizeLimit/trunk/t/response/TestApache/basic.pm (original)
+++ perl/Apache-SizeLimit/trunk/t/response/TestApache/basic.pm Fri Jul 21 10:16:16 2023
@@ -30,12 +30,8 @@ sub handler {
cmp_ok( $size, '>', 0, 'proc size is reported > 0' );

{
- # test with USE_SMAPS=0
- my $smaps = $Apache::SizeLimit::USE_SMAPS;
- $Apache::SizeLimit::USE_SMAPS = 0;
my ( $size, $shared ) = Apache::SizeLimit->_check_size();
cmp_ok( $size, '>', 0, 'proc size is reported > 0' );
- $Apache::SizeLimit::USE_SMAPS = $smaps;
}

SKIP:

Modified: perl/Apache-SizeLimit/trunk/t/response/TestApache2/basic.pm
URL: http://svn.apache.org/viewvc/perl/Apache-SizeLimit/trunk/t/response/TestApache2/basic.pm?rev=1911170&r1=1911169&r2=1911170&view=diff
==============================================================================
--- perl/Apache-SizeLimit/trunk/t/response/TestApache2/basic.pm (original)
+++ perl/Apache-SizeLimit/trunk/t/response/TestApache2/basic.pm Fri Jul 21 10:16:16 2023
@@ -30,12 +30,8 @@ sub handler {
cmp_ok( $size, '>', 0, 'proc size is reported > 0' );

{
- # test with USE_SMAPS=0
- my $smaps = $Apache2::SizeLimit::USE_SMAPS;
- $Apache2::SizeLimit::USE_SMAPS = 0;
my ( $size, $shared ) = Apache2::SizeLimit->_check_size();
cmp_ok( $size, '>', 0, 'proc size is reported > 0' );
- $Apache2::SizeLimit::USE_SMAPS = $smaps;
}

SKIP: