Mailing List Archive: Memory leak in KinoSearch::Index::PostingsWriter::add_batch

Memory leak in KinoSearch::Index::PostingsWriter::add_batch

daniel at electricrain

Jul 30, 2007, 10:13 AM

Post #1 of 10 (2987 views)

There appears to be a memory leak in add_batch, which I've been unable to
track down. If I comment out the XS code in PostingsWriter that calls:

kino_PostWriter_add_batch(self, batch, &field_name, doc_num, doc_boost, length_norm);

My indexing process stays at around 45M, vs 100M+ with that call enabled for
14k documents.

Unfortunately I don't understand where kino_PostWriter_add_batch() is defined.

Not doing anything too special.. subclassing ::Schema, with 4 fields, one of
which is non-indexed. There doesn't seem to be a way to periodically flush
the index to disk (and out of memory).

Using current svn tree (0.20-4)

-D
--
<dsully> please describe web 2.0 to me in 2 sentences or less.
<jwb> you make all the content. they keep all the revenue.

Memory leak in KinoSearch::Index::PostingsWriter::add_batch [ In reply to ]

Jul 30, 2007, 2:35 PM

Post #2 of 10 (2901 views)

On 7/30/07, Dan Sully <daniel@electricrain.com> wrote:
> Unfortunately I don't understand where kino_PostWriter_add_batch() is defined.

Presumably Marvin is busy with something, so I'll try an interim
answer. I haven't looked much at the Index creation side of things,
but that would be defined in c_src/KinoSearch/Index/PostingsWriter.c
as PostWriter_add_batch().
The "kino_" part is caused by the BoilerPlater layer, which defines a
class hierarchy for the C level objects, and probably can be ignored
here.

Glancing at the code, it sees like you can specify a 'mem_thresh'
parameter when creating the PostWriter object. This gets passed
through the the underlying MemPool object
(c_src/KinoSearch/Util/MemoryPool.c). If it's a true leak (as opposed
to just large buffer) it might be easiest to track down with
valgrind.

Good luck!

Nathan Kurz
nate@verse.com

Memory leak in KinoSearch::Index::PostingsWriter::add_batch [ In reply to ]

marvin at rectangular

Jul 30, 2007, 3:54 PM

Post #3 of 10 (2895 views)

On Jul 30, 2007, at 2:24 PM, Nathan Kurz wrote:

> On 7/30/07, Dan Sully <daniel@electricrain.com> wrote:
>> Unfortunately I don't understand where kino_PostWriter_add_batch()
>> is defined.
>
> Presumably Marvin is busy with something, so I'll try an interim
> answer.

Thanks, Nathan. I have a double deadline Wednesday: testing begins
and the project manager leaves. :\ It's a fun gig and I'm enjoying
it, but I'm also looking forward to having more time to work on KS
again as it finishes up. Sorry I haven't gotten out a reply to you;
your emails can be challenging. ;)

> I haven't looked much at the Index creation side of things,
> but that would be defined in c_src/KinoSearch/Index/PostingsWriter.c
> as PostWriter_add_batch().

Yes. FWIW, this section of code is pretty involved and could
withstand some simplification. ;)

> (c_src/KinoSearch/Util/MemoryPool.c). If it's a true leak (as opposed
> to just large buffer) it might be easiest to track down with
> valgrind.

It's either a true leak, or something else is wrong -- Dan's numbers
are definitely bigger than they should be. I think around 45-50
megs ought to be worst case for an InvIndexer in 0.20_04 (and we can
bring that down in future releases).

Tracking leaks in a Perl app with valgrind is a little tricky.

* Linux only.
* A debugging perl is required.
* Set the PERL_DESTRUCT_LEVEL environment variable to 2.
* You probably need a suppressions file like the one
in $KS_DIST/devel/p588_valgrind.supp.
* Valgrind won't catch leaks from circular refs at the Perl
level, as those get cleaned up during global destruct.

perlhack has more details. Check the bleadperl version, as it has
some recent doc patches.

http://search.cpan.org/~rgarcia/perl-5.9.5/pod/perlhack.pod

[... time passes ...]

I just ran the KS test suite under valgrind, using the script
$KS_DIST/devel/valgrind_test.plx, and it came up clean. So we'll
need some way of duplicating the problem.

Thanks for the report, Dan. I'll attend to this when I can take a
break from fighting fires at $contract_job.

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/

Memory leak in KinoSearch::Index::PostingsWriter::add_batch [ In reply to ]

daniel at electricrain

Jul 30, 2007, 4:04 PM

Post #4 of 10 (2911 views)

* Marvin Humphrey shaped the electrons to say...

>It's either a true leak, or something else is wrong -- Dan's numbers
>are definitely bigger than they should be. I think around 45-50
>megs ought to be worst case for an InvIndexer in 0.20_04 (and we can
>bring that down in future releases).
>
>Tracking leaks in a Perl app with valgrind is a little tricky.

I'm trying to run under valgrind right now.. current size is 186M.

>Thanks for the report, Dan. I'll attend to this when I can take a
>break from fighting fires at $contract_job.

No problem.

-D
--
<dsully> please describe web 2.0 to me in 2 sentences or less.
<jwb> you make all the content. they keep all the revenue.

Memory leak in KinoSearch::Index::PostingsWriter::add_batch [ In reply to ]

daniel at electricrain

Jul 30, 2007, 4:05 PM

Post #5 of 10 (2920 views)

* Nathan Kurz shaped the electrons to say...

>answer. I haven't looked much at the Index creation side of things,
>but that would be defined in c_src/KinoSearch/Index/PostingsWriter.c
>as PostWriter_add_batch().
>The "kino_" part is caused by the BoilerPlater layer, which defines a
>class hierarchy for the C level objects, and probably can be ignored
>here.

Aha! I didn't look above the perl/ tree.

>Glancing at the code, it sees like you can specify a 'mem_thresh'
>parameter when creating the PostWriter object. This gets passed
>through the the underlying MemPool object
>(c_src/KinoSearch/Util/MemoryPool.c). If it's a true leak (as opposed
>to just large buffer) it might be easiest to track down with
>valgrind.

Yeah, tried passing in 2k for the mem_thresh, but that didn't work.

-D
--
<dsully> please describe web 2.0 to me in 2 sentences or less.
<jwb> you make all the content. they keep all the revenue.

Memory leak in KinoSearch::Index::PostingsWriter::add_batch [ In reply to ]

daniel at electricrain

Jul 30, 2007, 4:24 PM

Post #6 of 10 (2912 views)

* Dan Sully shaped the electrons to say...

>>Tracking leaks in a Perl app with valgrind is a little tricky.
>
>I'm trying to run under valgrind right now.. current size is 186M.

Alas, this is not going to work.. the process (with debugperl, etc) quickly
balloons to multiple gigs, and I've run out of ram.

-D
--
<dsully> please describe web 2.0 to me in 2 sentences or less.
<jwb> you make all the content. they keep all the revenue.

Memory leak in KinoSearch::Index::PostingsWriter::add_batch [ In reply to ]

marvin at rectangular

Jul 30, 2007, 4:28 PM

Post #7 of 10 (2909 views)

On Jul 30, 2007, at 4:13 PM, Dan Sully wrote:

> Alas, this is not going to work.. the process (with debugperl, etc)
> quickly
> balloons to multiple gigs, and I've run out of ram.

How about running with a subset of your document collection? As soon
as a byte gets lost, Valgrind notices.

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/

Memory leak in KinoSearch::Index::PostingsWriter::add_batch [ In reply to ]

daniel at electricrain

Jul 30, 2007, 4:38 PM

Post #8 of 10 (2909 views)

* Marvin Humphrey shaped the electrons to say...

>How about running with a subset of your document collection? As soon
>as a byte gets lost, Valgrind notices.

I've managed to get it to index 5 documents. :)

valgrind output is attached.

-D
--
<dsully> please describe web 2.0 to me in 2 sentences or less.
<jwb> you make all the content. they keep all the revenue.
-------------- next part --------------
==4137== Memcheck, a memory error detector.
==4137== Copyright (C) 2002-2006, and GNU GPL'd, by Julian Seward et al.
==4137== Using LibVEX rev 1606, a library for dynamic binary translation.
==4137== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP.
==4137== Using valgrind-3.2.0-Debian, a dynamic binary instrumentation framework.
==4137== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et al.
==4137==
==4137== My PID = 4137, parent PID = 21731. Prog and args are:
==4137== debugperl
==4137== mail-indexer
==4137==
--4137--
--4137-- Command line
--4137-- debugperl
--4137-- mail-indexer
--4137-- Startup, with flags:
--4137-- --suppressions=/usr/lib/valgrind/debian-libc6-dbg.supp
--4137-- --suppressions=/usr/lib/valgrind/p588_valgrind.supp
--4137-- --leak-check=full
--4137-- --show-reachable=yes
--4137-- --log-file-exactly=valgrind.txt
--4137-- -v
--4137-- Contents of /proc/version:
--4137-- Linux version 2.6.17-10-generic (root@king) (gcc version 4.1.2 20060928 (prerelease) (Ubuntu 4.1.1-13ubuntu5)) #2 SMP Tue Dec 5 21:16:35 UTC 2006 (Ubuntu 2.6.17-10.34-generic)
--4137-- Arch and hwcaps: AMD64, amd64-sse2
--4137-- Valgrind library directory: /usr/lib/valgrind
--4137-- Reading syms from /usr/bin/debugperl (0x400000)
--4137-- Reading syms from /lib/ld-2.4.so (0x4000000)
--4137-- Reading debug info from /lib/ld-2.4.so...
--4137-- ... CRC mismatch (computed 9697110F wanted 12D86C19)
--4137-- object doesn't have a symbol table
--4137-- Reading syms from /usr/lib/valgrind/amd64-linux/memcheck (0x38000000)
--4137-- object doesn't have a dynamic symbol table
--4137-- Reading suppressions file: /usr/lib/valgrind/debian-libc6-dbg.supp
--4137-- Reading suppressions file: /usr/lib/valgrind/p588_valgrind.supp
--4137-- Reading suppressions file: /usr/lib/valgrind/default.supp
--4137-- Reading syms from /usr/lib/valgrind/amd64-linux/vgpreload_core.so (0x491D000)
--4137-- Reading syms from /usr/lib/valgrind/amd64-linux/vgpreload_memcheck.so (0x4A1E000)
--4137-- Reading syms from /lib/libdl-2.4.so (0x4B24000)
--4137-- Reading debug info from /lib/libdl-2.4.so...
--4137-- ... CRC mismatch (computed 815B85A3 wanted 6E76141D)
--4137-- object doesn't have a symbol table
--4137-- Reading syms from /lib/libm-2.4.so (0x4C28000)
--4137-- Reading debug info from /lib/libm-2.4.so...
--4137-- ... CRC mismatch (computed 2B892E9C wanted 50A7C6DA)
--4137-- object doesn't have a symbol table
--4137-- Reading syms from /lib/libpthread-2.4.so (0x4DA9000)
--4137-- Reading debug info from /lib/libpthread-2.4.so...
--4137-- ... CRC mismatch (computed B930304E wanted 6B816E25)
--4137-- Reading syms from /lib/libc-2.4.so (0x4EBF000)
--4137-- Reading debug info from /lib/libc-2.4.so...
--4137-- ... CRC mismatch (computed 8109B5C7 wanted 8B3079D3)
--4137-- object doesn't have a symbol table
--4137-- Reading syms from /lib/libcrypt-2.4.so (0x50FF000)
--4137-- Reading debug info from /lib/libcrypt-2.4.so...
--4137-- ... CRC mismatch (computed 683CAEF7 wanted A51A8ADE)
--4137-- object doesn't have a symbol table
--4137-- REDIR: 0x4F35390 (memset) redirected to 0x4A21830 (memset)
--4137-- REDIR: 0x4F35AB0 (memcpy) redirected to 0x4A21FC0 (memcpy)
--4137-- REDIR: 0x4F344B0 (rindex) redirected to 0x4A21420 (rindex)
--4137-- REDIR: 0x4F2F660 (malloc) redirected to 0x4A2078C (malloc)
--4137-- REDIR: 0x4F35200 (memmove) redirected to 0x4A21860 (memmove)
--4137-- REDIR: 0x4F340C0 (strlen) redirected to 0x4A215D0 (strlen)
--4137-- REDIR: 0x4F34340 (strncmp) redirected to 0x4A21630 (strncmp)
--4137-- REDIR: 0x4F33B30 (strcmp) redirected to 0x4A216A0 (strcmp)
--4137-- REDIR: 0x4F33980 (index) redirected to 0x4A21510 (index)
--4137-- REDIR: 0x4F36440 (strchrnul) redirected to 0x4A218C0 (strchrnul)
--4137-- REDIR: 0x4F2D830 (free) redirected to 0x4A2039C (free)
--4137-- REDIR: 0x4F34BB0 (memchr) redirected to 0x4A21760 (memchr)
--4137-- REDIR: 0x4F2FAE0 (realloc) redirected to 0x4A2083D (realloc)
--4137-- REDIR: 0x4F341B0 (strnlen) redirected to 0x4A215A0 (strnlen)
--4137-- REDIR: 0x4F357B0 (stpcpy) redirected to 0x4A21DD0 (stpcpy)
--4137-- REDIR: 0x4F33B70 (strcpy) redirected to 0x4A21B20 (strcpy)
--4137-- REDIR: 0x4F36310 (rawmemchr) redirected to 0x4A218F0 (rawmemchr)
--4137-- REDIR: 0xFFFFFFFFFF600400 (???) redirected to 0x380275F1 (???)
--4137-- REDIR: 0x4F2F2F0 (calloc) redirected to 0x4A1FAB1 (calloc)
==4137== Invalid read of size 8
==4137== at 0x4014CB0: (within /lib/ld-2.4.so)
==4137== by 0x400A29D: (within /lib/ld-2.4.so)
==4137== by 0x4005F94: (within /lib/ld-2.4.so)
==4137== by 0x40081D5: (within /lib/ld-2.4.so)
==4137== by 0x401119C: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4010C0A: (within /lib/ld-2.4.so)
==4137== by 0x4B251A9: (within /lib/libdl-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4B2571C: (within /lib/libdl-2.4.so)
==4137== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==4137== by 0x41DEF8: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==4137== Address 0x5718780 is 32 bytes inside a block of size 38 alloc'd
==4137== at 0x4A2080E: malloc (vg_replace_malloc.c:149)
==4137== by 0x4007573: (within /lib/ld-2.4.so)
==4137== by 0x400812E: (within /lib/ld-2.4.so)
==4137== by 0x401119C: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4010C0A: (within /lib/ld-2.4.so)
==4137== by 0x4B251A9: (within /lib/libdl-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4B2571C: (within /lib/libdl-2.4.so)
==4137== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==4137== by 0x41DEF8: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==4137== by 0x4C53CD: Perl_pp_entersub (pp_hot.c:2877)
--4137-- Reading syms from /usr/lib/perl/5.8.8/auto/Fcntl/Fcntl.so (0x5766000)
--4137-- object doesn't have a symbol table
==4137==
==4137== Invalid read of size 8
==4137== at 0x4014CCA: (within /lib/ld-2.4.so)
==4137== by 0x400A29D: (within /lib/ld-2.4.so)
==4137== by 0x4005F94: (within /lib/ld-2.4.so)
==4137== by 0x40081D5: (within /lib/ld-2.4.so)
==4137== by 0x401119C: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4010C0A: (within /lib/ld-2.4.so)
==4137== by 0x4B251A9: (within /lib/libdl-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4B2571C: (within /lib/libdl-2.4.so)
==4137== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==4137== by 0x41DEF8: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==4137== Address 0x5974610 is 40 bytes inside a block of size 41 alloc'd
==4137== at 0x4A2080E: malloc (vg_replace_malloc.c:149)
==4137== by 0x4007573: (within /lib/ld-2.4.so)
==4137== by 0x400812E: (within /lib/ld-2.4.so)
==4137== by 0x401119C: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4010C0A: (within /lib/ld-2.4.so)
==4137== by 0x4B251A9: (within /lib/libdl-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4B2571C: (within /lib/libdl-2.4.so)
==4137== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==4137== by 0x41DEF8: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==4137== by 0x4C53CD: Perl_pp_entersub (pp_hot.c:2877)
--4137-- Reading syms from /usr/lib/perl/5.8.8/auto/List/Util/Util.so (0x5A69000)
--4137-- object doesn't have a symbol table
--4137-- Reading syms from /usr/lib/perl5/auto/KinoSearch/KinoSearch.so (0x5B72000)
--4137-- object doesn't have a symbol table
--4137-- Reading syms from /usr/lib/perl5/auto/Clone/Clone.so (0x5E02000)
--4137-- object doesn't have a symbol table
--4137-- Reading syms from /usr/lib/perl/5.8.8/auto/Time/HiRes/HiRes.so (0x6005000)
--4137-- object doesn't have a symbol table
--4137-- REDIR: 0xFFFFFFFFFF600000 (???) redirected to 0x380275E7 (???)
--4137-- Reading syms from /usr/lib/perl/5.8.8/auto/IO/IO.so (0x630A000)
--4137-- object doesn't have a symbol table
--4137-- Reading syms from /usr/lib/perl5/auto/Compress/Zlib/Zlib.so (0x650F000)
--4137-- object doesn't have a symbol table
==4137==
==4137== Invalid read of size 8
==4137== at 0x4014CE4: (within /lib/ld-2.4.so)
==4137== by 0x400A29D: (within /lib/ld-2.4.so)
==4137== by 0x4005F94: (within /lib/ld-2.4.so)
==4137== by 0x40081D5: (within /lib/ld-2.4.so)
==4137== by 0x400B45C: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x400BB2B: (within /lib/ld-2.4.so)
==4137== by 0x40111FC: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4010C0A: (within /lib/ld-2.4.so)
==4137== by 0x4B251A9: (within /lib/libdl-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== Address 0x647FEA0 is 16 bytes inside a block of size 19 alloc'd
==4137== at 0x4A2080E: malloc (vg_replace_malloc.c:149)
==4137== by 0x400865F: (within /lib/ld-2.4.so)
==4137== by 0x400B45C: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x400BB2B: (within /lib/ld-2.4.so)
==4137== by 0x40111FC: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4010C0A: (within /lib/ld-2.4.so)
==4137== by 0x4B251A9: (within /lib/libdl-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4B2571C: (within /lib/libdl-2.4.so)
==4137== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
--4137-- Reading syms from /usr/lib/libz.so.1.2.3 (0x661F000)
--4137-- Reading debug info from /usr/lib/libz.so.1.2.3...
--4137-- ... CRC mismatch (computed 21C34893 wanted 515F07D3)
--4137-- object doesn't have a symbol table
--4137-- Reading syms from /usr/lib/perl5/auto/Lingua/Stem/Snowball/Snowball.so (0x6A35000)
--4137-- object doesn't have a symbol table
--4137-- Reading syms from /usr/lib/perl/5.8.8/auto/MIME/Base64/Base64.so (0x6C78000)
--4137-- object doesn't have a symbol table
--4137-- Reading syms from /usr/lib/perl5/auto/HTML/Parser/Parser.so (0x767B000)
--4137-- object doesn't have a symbol table
--4137-- Reading syms from /usr/lib/perl5/auto/Term/ReadKey/ReadKey.so (0x7786000)
--4137-- object doesn't have a symbol table
==4137==
==4137== ERROR SUMMARY: 11 errors from 3 contexts (suppressed: 72 from 1)
==4137==
==4137== 2 errors in context 1 of 3:
==4137== Invalid read of size 8
==4137== at 0x4014CE4: (within /lib/ld-2.4.so)
==4137== by 0x400A29D: (within /lib/ld-2.4.so)
==4137== by 0x4005F94: (within /lib/ld-2.4.so)
==4137== by 0x40081D5: (within /lib/ld-2.4.so)
==4137== by 0x400B45C: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x400BB2B: (within /lib/ld-2.4.so)
==4137== by 0x40111FC: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4010C0A: (within /lib/ld-2.4.so)
==4137== by 0x4B251A9: (within /lib/libdl-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== Address 0x647FEA0 is 16 bytes inside a block of size 19 alloc'd
==4137== at 0x4A2080E: malloc (vg_replace_malloc.c:149)
==4137== by 0x400865F: (within /lib/ld-2.4.so)
==4137== by 0x400B45C: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x400BB2B: (within /lib/ld-2.4.so)
==4137== by 0x40111FC: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4010C0A: (within /lib/ld-2.4.so)
==4137== by 0x4B251A9: (within /lib/libdl-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4B2571C: (within /lib/libdl-2.4.so)
==4137== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==4137==
==4137== 2 errors in context 2 of 3:
==4137== Invalid read of size 8
==4137== at 0x4014CB0: (within /lib/ld-2.4.so)
==4137== by 0x400A29D: (within /lib/ld-2.4.so)
==4137== by 0x4005F94: (within /lib/ld-2.4.so)
==4137== by 0x40081D5: (within /lib/ld-2.4.so)
==4137== by 0x401119C: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4010C0A: (within /lib/ld-2.4.so)
==4137== by 0x4B251A9: (within /lib/libdl-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4B2571C: (within /lib/libdl-2.4.so)
==4137== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==4137== by 0x41DEF8: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==4137== Address 0x5718780 is 32 bytes inside a block of size 38 alloc'd
==4137== at 0x4A2080E: malloc (vg_replace_malloc.c:149)
==4137== by 0x4007573: (within /lib/ld-2.4.so)
==4137== by 0x400812E: (within /lib/ld-2.4.so)
==4137== by 0x401119C: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4010C0A: (within /lib/ld-2.4.so)
==4137== by 0x4B251A9: (within /lib/libdl-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4B2571C: (within /lib/libdl-2.4.so)
==4137== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==4137== by 0x41DEF8: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==4137== by 0x4C53CD: Perl_pp_entersub (pp_hot.c:2877)
==4137==
==4137== 7 errors in context 3 of 3:
==4137== Invalid read of size 8
==4137== at 0x4014CCA: (within /lib/ld-2.4.so)
==4137== by 0x400A29D: (within /lib/ld-2.4.so)
==4137== by 0x4005F94: (within /lib/ld-2.4.so)
==4137== by 0x40081D5: (within /lib/ld-2.4.so)
==4137== by 0x401119C: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4010C0A: (within /lib/ld-2.4.so)
==4137== by 0x4B251A9: (within /lib/libdl-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4B2571C: (within /lib/libdl-2.4.so)
==4137== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==4137== by 0x41DEF8: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==4137== Address 0x5974610 is 40 bytes inside a block of size 41 alloc'd
==4137== at 0x4A2080E: malloc (vg_replace_malloc.c:149)
==4137== by 0x4007573: (within /lib/ld-2.4.so)
==4137== by 0x400812E: (within /lib/ld-2.4.so)
==4137== by 0x401119C: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4010C0A: (within /lib/ld-2.4.so)
==4137== by 0x4B251A9: (within /lib/libdl-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4B2571C: (within /lib/libdl-2.4.so)
==4137== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==4137== by 0x41DEF8: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==4137== by 0x4C53CD: Perl_pp_entersub (pp_hot.c:2877)
--4137--
--4137-- supp: 72 Fedora-Core-5-hack3-ld24
==4137==
==4137== IN SUMMARY: 11 errors from 3 contexts (suppressed: 72 from 1)
==4137==
==4137== malloc/free: in use at exit: 27,238 bytes in 516 blocks.
==4137== malloc/free: 602,958 allocs, 602,442 frees, 808,085,025 bytes allocated.
==4137==
==4137== searching for pointers to 516 not-freed blocks.
==4137== checked 556,824 bytes.
==4137==
==4137==
==4137== 19 bytes in 1 blocks are still reachable in loss record 1 of 9
==4137== at 0x4A2080E: malloc (vg_replace_malloc.c:149)
==4137== by 0x400865F: (within /lib/ld-2.4.so)
==4137== by 0x400B45C: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x400BB2B: (within /lib/ld-2.4.so)
==4137== by 0x40111FC: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4010C0A: (within /lib/ld-2.4.so)
==4137== by 0x4B251A9: (within /lib/libdl-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4B2571C: (within /lib/libdl-2.4.so)
==4137== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==4137==
==4137==
==4137== 32 bytes in 1 blocks are still reachable in loss record 2 of 9
==4137== at 0x4A1FB37: calloc (vg_replace_malloc.c:279)
==4137== by 0x4B2577A: (within /lib/libdl-2.4.so)
==4137== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==4137== by 0x41DEF8: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==4137== by 0x4C53CD: Perl_pp_entersub (pp_hot.c:2877)
==4137== by 0x4993BF: Perl_runops_debug (dump.c:1459)
==4137== by 0x425351: S_call_body (perl.c:2731)
==4137== by 0x424DAA: Perl_call_sv (perl.c:2646)
==4137== by 0x42ADDA: S_call_list_body (perl.c:5262)
==4137== by 0x42A7C9: Perl_call_list (perl.c:5191)
==4137== by 0x470AF0: Perl_newATTRSUB (op.c:4520)
==4137== by 0x46B4BA: Perl_utilize (op.c:3096)
==4137==
==4137==
==4137== 40 bytes in 1 blocks are still reachable in loss record 3 of 9
==4137== at 0x4A2080E: malloc (vg_replace_malloc.c:149)
==4137== by 0x400BBDC: (within /lib/ld-2.4.so)
==4137== by 0x40111FC: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4010C0A: (within /lib/ld-2.4.so)
==4137== by 0x4B251A9: (within /lib/libdl-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4B2571C: (within /lib/libdl-2.4.so)
==4137== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==4137== by 0x41DEF8: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==4137== by 0x4C53CD: Perl_pp_entersub (pp_hot.c:2877)
==4137== by 0x4993BF: Perl_runops_debug (dump.c:1459)
==4137==
==4137==
==4137== 460 bytes in 11 blocks are still reachable in loss record 4 of 9
==4137== at 0x4A2080E: malloc (vg_replace_malloc.c:149)
==4137== by 0x4007573: (within /lib/ld-2.4.so)
==4137== by 0x400812E: (within /lib/ld-2.4.so)
==4137== by 0x401119C: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4010C0A: (within /lib/ld-2.4.so)
==4137== by 0x4B251A9: (within /lib/libdl-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4B2571C: (within /lib/libdl-2.4.so)
==4137== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==4137== by 0x41DEF8: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==4137== by 0x4C53CD: Perl_pp_entersub (pp_hot.c:2877)
==4137==
==4137==
==4137== 479 bytes in 12 blocks are still reachable in loss record 5 of 9
==4137== at 0x4A2080E: malloc (vg_replace_malloc.c:149)
==4137== by 0x400A34B: (within /lib/ld-2.4.so)
==4137== by 0x4005F94: (within /lib/ld-2.4.so)
==4137== by 0x40081D5: (within /lib/ld-2.4.so)
==4137== by 0x401119C: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4010C0A: (within /lib/ld-2.4.so)
==4137== by 0x4B251A9: (within /lib/libdl-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4B2571C: (within /lib/libdl-2.4.so)
==4137== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==4137== by 0x41DEF8: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==4137==
==4137==
==4137== 632 bytes in 11 blocks are still reachable in loss record 6 of 9
==4137== at 0x4A2080E: malloc (vg_replace_malloc.c:149)
==4137== by 0x400BD62: (within /lib/ld-2.4.so)
==4137== by 0x40111FC: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4010C0A: (within /lib/ld-2.4.so)
==4137== by 0x4B251A9: (within /lib/libdl-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4B2571C: (within /lib/libdl-2.4.so)
==4137== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==4137== by 0x41DEF8: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==4137== by 0x4C53CD: Perl_pp_entersub (pp_hot.c:2877)
==4137== by 0x4993BF: Perl_runops_debug (dump.c:1459)
==4137==
==4137==
==4137== 936 bytes in 12 blocks are still reachable in loss record 7 of 9
==4137== at 0x4A1FB37: calloc (vg_replace_malloc.c:279)
==4137== by 0x400ECD1: (within /lib/ld-2.4.so)
==4137== by 0x4011472: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4010C0A: (within /lib/ld-2.4.so)
==4137== by 0x4B251A9: (within /lib/libdl-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4B2571C: (within /lib/libdl-2.4.so)
==4137== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==4137== by 0x41DEF8: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==4137== by 0x4C53CD: Perl_pp_entersub (pp_hot.c:2877)
==4137== by 0x4993BF: Perl_runops_debug (dump.c:1459)
==4137==
==4137==
==4137== 10,730 bytes in 455 blocks are definitely lost in loss record 8 of 9
==4137== at 0x4A2080E: malloc (vg_replace_malloc.c:149)
==4137== by 0x499A85: Perl_safesysmalloc (util.c:78)
==4137== by 0x49B7DF: Perl_savepvn (util.c:789)
==4137== by 0x42D8C7: Perl_gv_fetchpv (gv.c:754)
==4137== by 0x42D31B: Perl_gv_stashpvn (gv.c:671)
==4137== by 0x46AE5A: Perl_package (op.c:3010)
==4137== by 0x45FE6C: Perl_yyparse (perly.y:415)
==4137== by 0x51BC30: S_doeval (pp_ctl.c:2894)
==4137== by 0x51ED2E: Perl_pp_require (pp_ctl.c:3429)
==4137== by 0x4993BF: Perl_runops_debug (dump.c:1459)
==4137== by 0x425351: S_call_body (perl.c:2731)
==4137== by 0x424DAA: Perl_call_sv (perl.c:2646)
==4137==
==4137==
==4137== 13,910 bytes in 12 blocks are still reachable in loss record 9 of 9
==4137== at 0x4A1FB37: calloc (vg_replace_malloc.c:279)
==4137== by 0x400A0E4: (within /lib/ld-2.4.so)
==4137== by 0x4005F94: (within /lib/ld-2.4.so)
==4137== by 0x40081D5: (within /lib/ld-2.4.so)
==4137== by 0x401119C: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4010C0A: (within /lib/ld-2.4.so)
==4137== by 0x4B251A9: (within /lib/libdl-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4B2571C: (within /lib/libdl-2.4.so)
==4137== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==4137== by 0x41DEF8: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==4137==
==4137== LEAK SUMMARY:
==4137== definitely lost: 10,730 bytes in 455 blocks.
==4137== possibly lost: 0 bytes in 0 blocks.
==4137== still reachable: 16,508 bytes in 61 blocks.
==4137== suppressed: 0 bytes in 0 blocks.
--4137-- memcheck: sanity checks: 2810 cheap, 113 expensive
--4137-- memcheck: auxmaps: 0 auxmap entries (0k, 0M) in use
--4137-- memcheck: auxmaps: 0 searches, 0 comparisons
--4137-- memcheck: SMs: n_issued = 6761 (108176k, 105M)
--4137-- memcheck: SMs: n_deissued = 6196 (99136k, 96M)
--4137-- memcheck: SMs: max_noaccess = 524287 (8388592k, 8191M)
--4137-- memcheck: SMs: max_undefined = 75 (1200k, 1M)
--4137-- memcheck: SMs: max_defined = 375 (6000k, 5M)
--4137-- memcheck: SMs: max_non_DSM = 598 (9568k, 9M)
--4137-- memcheck: max sec V bit nodes: 19 (1k, 0M)
--4137-- memcheck: set_sec_vbits8 calls: 892 (new: 19, updates: 873)
--4137-- memcheck: max shadow mem size: 13713k, 13M
--4137-- translate: fast SP updates identified: 28,418 ( 85.0%)
--4137-- translate: generic_known SP updates identified: 3,314 ( 9.9%)
--4137-- translate: generic_unknown SP updates identified: 1,685 ( 5.0%)
--4137-- tt/tc: 2,093,137 tt lookups requiring 2,538,048 probes
--4137-- tt/tc: 2,093,137 fast-cache updates, 2 flushes
--4137-- transtab: new 33,464 (1,010,028 -> 17,717,959; ratio 175:10) [0 scs]
--4137-- transtab: dumped 0 (0 -> ??)
--4137-- transtab: discarded 0 (0 -> ??)
--4137-- scheduler: 281,017,496 jumps (bb entries).
--4137-- scheduler: 2,810/3,302,520 major/minor sched events.
--4137-- sanity: 2811 cheap, 113 expensive checks.
--4137-- exectx: 30,011 lists, 10,240 contexts (avg 0 per list)
--4137-- exectx: 1,196,788 searches, 6,697,839 full compares (5,596 per 1000)
--4137-- exectx: 677 cmp2, 1,017 cmp4, 0 cmpAll

Memory leak in KinoSearch::Index::PostingsWriter::add_batch [ In reply to ]

marvin at rectangular

Jul 30, 2007, 4:59 PM

Post #9 of 10 (2911 views)

On Jul 30, 2007, at 4:28 PM, Dan Sully wrote:

> * Marvin Humphrey shaped the electrons to say...
>
>> How about running with a subset of your document collection? As
>> soon as a byte gets lost, Valgrind notices.
>
> I've managed to get it to index 5 documents. :)
>
> valgrind output is attached.

I don't see anything in there that's KS related, so I think we'll
need to find a threshold big enough to trip the problem, but not so
big that it dies. We might start seeing stuff at around 30 megs.
Once we hit 70 or 80, I'd be surprised if nothing showed.

FWIW, the DynaLoader stuff isn't being caught by the KS suppressions
file:

==4137== 40 bytes in 1 blocks are still reachable in loss record 3 of 9
==4137== at 0x4A2080E: malloc (vg_replace_malloc.c:149)
==4137== by 0x400BBDC: (within /lib/ld-2.4.so)
==4137== by 0x40111FC: (within /lib/ld-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4010C0A: (within /lib/ld-2.4.so)
==4137== by 0x4B251A9: (within /lib/libdl-2.4.so)
==4137== by 0x400D1E5: (within /lib/ld-2.4.so)
==4137== by 0x4B2571C: (within /lib/libdl-2.4.so)
==4137== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==4137== by 0x41DEF8: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==4137== by 0x4C53CD: Perl_pp_entersub (pp_hot.c:2877)
==4137== by 0x4993BF: Perl_runops_debug (dump.c:1459)

If you want to clean things up, you can generate a suppressions file
with --gen_suppressions with this 5-document pass. Info here:

http://valgrind.org/docs/manual/manual-core.html#manual-core.suppress
http://valgrind.org/docs/manual/writing-tools.html#writing-
tools.suppressions
http://valgrind.org/docs/manual/mc-manual.html#mc-manual.suppfiles

However, we'll probably be able to see what's going on without that.

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/

Memory leak in KinoSearch::Index::PostingsWriter::add_batch [ In reply to ]

daniel at electricrain

Jul 30, 2007, 5:19 PM

Post #10 of 10 (2888 views)

* Marvin Humphrey shaped the electrons to say...

>I don't see anything in there that's KS related, so I think we'll
>need to find a threshold big enough to trip the problem, but not so
>big that it dies. We might start seeing stuff at around 30 megs.
>Once we hit 70 or 80, I'd be surprised if nothing showed.

30 megs leaked? If I try and scan 100 documents with valgrind, that's when
the memory bumps up to gigabytes.. Just did 30 documents, output attached.
Doesn't look much different than the other one.

If it will help, I can give you access to the server.

-D
--
<dsully> please describe web 2.0 to me in 2 sentences or less.
<jwb> you make all the content. they keep all the revenue.
-------------- next part --------------
==8800== Memcheck, a memory error detector.
==8800== Copyright (C) 2002-2006, and GNU GPL'd, by Julian Seward et al.
==8800== Using LibVEX rev 1606, a library for dynamic binary translation.
==8800== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP.
==8800== Using valgrind-3.2.0-Debian, a dynamic binary instrumentation framework.
==8800== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et al.
==8800==
==8800== My PID = 8800, parent PID = 21731. Prog and args are:
==8800== debugperl
==8800== mail-indexer
==8800==
--8800--
--8800-- Command line
--8800-- debugperl
--8800-- mail-indexer
--8800-- Startup, with flags:
--8800-- --suppressions=/usr/lib/valgrind/debian-libc6-dbg.supp
--8800-- --suppressions=/usr/lib/valgrind/p588_valgrind.supp
--8800-- --leak-check=full
--8800-- --show-reachable=yes
--8800-- --log-file-exactly=valgrind.txt
--8800-- -v
--8800-- Contents of /proc/version:
--8800-- Linux version 2.6.17-10-generic (root@king) (gcc version 4.1.2 20060928 (prerelease) (Ubuntu 4.1.1-13ubuntu5)) #2 SMP Tue Dec 5 21:16:35 UTC 2006 (Ubuntu 2.6.17-10.34-generic)
--8800-- Arch and hwcaps: AMD64, amd64-sse2
--8800-- Valgrind library directory: /usr/lib/valgrind
--8800-- Reading syms from /usr/bin/debugperl (0x400000)
--8800-- Reading syms from /lib/ld-2.4.so (0x4000000)
--8800-- Reading debug info from /lib/ld-2.4.so...
--8800-- ... CRC mismatch (computed 9697110F wanted 12D86C19)
--8800-- object doesn't have a symbol table
--8800-- Reading syms from /usr/lib/valgrind/amd64-linux/memcheck (0x38000000)
--8800-- object doesn't have a dynamic symbol table
--8800-- Reading suppressions file: /usr/lib/valgrind/debian-libc6-dbg.supp
--8800-- Reading suppressions file: /usr/lib/valgrind/p588_valgrind.supp
--8800-- Reading suppressions file: /usr/lib/valgrind/default.supp
--8800-- Reading syms from /usr/lib/valgrind/amd64-linux/vgpreload_core.so (0x491D000)
--8800-- Reading syms from /usr/lib/valgrind/amd64-linux/vgpreload_memcheck.so (0x4A1E000)
--8800-- Reading syms from /lib/libdl-2.4.so (0x4B24000)
--8800-- Reading debug info from /lib/libdl-2.4.so...
--8800-- ... CRC mismatch (computed 815B85A3 wanted 6E76141D)
--8800-- object doesn't have a symbol table
--8800-- Reading syms from /lib/libm-2.4.so (0x4C28000)
--8800-- Reading debug info from /lib/libm-2.4.so...
--8800-- ... CRC mismatch (computed 2B892E9C wanted 50A7C6DA)
--8800-- object doesn't have a symbol table
--8800-- Reading syms from /lib/libpthread-2.4.so (0x4DA9000)
--8800-- Reading debug info from /lib/libpthread-2.4.so...
--8800-- ... CRC mismatch (computed B930304E wanted 6B816E25)
--8800-- Reading syms from /lib/libc-2.4.so (0x4EBF000)
--8800-- Reading debug info from /lib/libc-2.4.so...
--8800-- ... CRC mismatch (computed 8109B5C7 wanted 8B3079D3)
--8800-- object doesn't have a symbol table
--8800-- Reading syms from /lib/libcrypt-2.4.so (0x50FF000)
--8800-- Reading debug info from /lib/libcrypt-2.4.so...
--8800-- ... CRC mismatch (computed 683CAEF7 wanted A51A8ADE)
--8800-- object doesn't have a symbol table
--8800-- REDIR: 0x4F35390 (memset) redirected to 0x4A21830 (memset)
--8800-- REDIR: 0x4F35AB0 (memcpy) redirected to 0x4A21FC0 (memcpy)
--8800-- REDIR: 0x4F344B0 (rindex) redirected to 0x4A21420 (rindex)
--8800-- REDIR: 0x4F2F660 (malloc) redirected to 0x4A2078C (malloc)
--8800-- REDIR: 0x4F35200 (memmove) redirected to 0x4A21860 (memmove)
--8800-- REDIR: 0x4F340C0 (strlen) redirected to 0x4A215D0 (strlen)
--8800-- REDIR: 0x4F34340 (strncmp) redirected to 0x4A21630 (strncmp)
--8800-- REDIR: 0x4F33B30 (strcmp) redirected to 0x4A216A0 (strcmp)
--8800-- REDIR: 0x4F33980 (index) redirected to 0x4A21510 (index)
--8800-- REDIR: 0x4F36440 (strchrnul) redirected to 0x4A218C0 (strchrnul)
--8800-- REDIR: 0x4F2D830 (free) redirected to 0x4A2039C (free)
--8800-- REDIR: 0x4F34BB0 (memchr) redirected to 0x4A21760 (memchr)
--8800-- REDIR: 0x4F2FAE0 (realloc) redirected to 0x4A2083D (realloc)
--8800-- REDIR: 0x4F341B0 (strnlen) redirected to 0x4A215A0 (strnlen)
--8800-- REDIR: 0x4F357B0 (stpcpy) redirected to 0x4A21DD0 (stpcpy)
--8800-- REDIR: 0x4F33B70 (strcpy) redirected to 0x4A21B20 (strcpy)
--8800-- REDIR: 0x4F36310 (rawmemchr) redirected to 0x4A218F0 (rawmemchr)
--8800-- REDIR: 0xFFFFFFFFFF600400 (???) redirected to 0x380275F1 (???)
--8800-- REDIR: 0x4F2F2F0 (calloc) redirected to 0x4A1FAB1 (calloc)
--8800-- Reading syms from /usr/lib/perl/5.8.8/auto/Fcntl/Fcntl.so (0x5766000)
--8800-- object doesn't have a symbol table
--8800-- Reading syms from /usr/lib/perl/5.8.8/auto/List/Util/Util.so (0x5A69000)
--8800-- object doesn't have a symbol table
--8800-- Reading syms from /usr/lib/perl5/auto/KinoSearch/KinoSearch.so (0x5B72000)
--8800-- Reading syms from /usr/lib/perl5/auto/Clone/Clone.so (0x5E02000)
--8800-- object doesn't have a symbol table
--8800-- Reading syms from /usr/lib/perl/5.8.8/auto/Time/HiRes/HiRes.so (0x6005000)
--8800-- object doesn't have a symbol table
--8800-- REDIR: 0xFFFFFFFFFF600000 (???) redirected to 0x380275E7 (???)
--8800-- Reading syms from /usr/lib/perl/5.8.8/auto/IO/IO.so (0x630A000)
--8800-- object doesn't have a symbol table
--8800-- Reading syms from /usr/lib/perl5/auto/Compress/Zlib/Zlib.so (0x650F000)
--8800-- object doesn't have a symbol table
==8800== Invalid read of size 8
==8800== at 0x4014CE4: (within /lib/ld-2.4.so)
==8800== by 0x400A29D: (within /lib/ld-2.4.so)
==8800== by 0x4005F94: (within /lib/ld-2.4.so)
==8800== by 0x40081D5: (within /lib/ld-2.4.so)
==8800== by 0x400B45C: (within /lib/ld-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x400BB2B: (within /lib/ld-2.4.so)
==8800== by 0x40111FC: (within /lib/ld-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x4010C0A: (within /lib/ld-2.4.so)
==8800== by 0x4B251A9: (within /lib/libdl-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== Address 0x647FF00 is 16 bytes inside a block of size 19 alloc'd
==8800== at 0x4A2080E: malloc (vg_replace_malloc.c:149)
==8800== by 0x400865F: (within /lib/ld-2.4.so)
==8800== by 0x400B45C: (within /lib/ld-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x400BB2B: (within /lib/ld-2.4.so)
==8800== by 0x40111FC: (within /lib/ld-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x4010C0A: (within /lib/ld-2.4.so)
==8800== by 0x4B251A9: (within /lib/libdl-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x4B2571C: (within /lib/libdl-2.4.so)
==8800== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
--8800-- Reading syms from /usr/lib/libz.so.1.2.3 (0x661F000)
--8800-- Reading debug info from /usr/lib/libz.so.1.2.3...
--8800-- ... CRC mismatch (computed 21C34893 wanted 515F07D3)
--8800-- object doesn't have a symbol table
--8800-- Reading syms from /usr/lib/perl5/auto/Lingua/Stem/Snowball/Snowball.so (0x6A35000)
--8800-- object doesn't have a symbol table
--8800-- Reading syms from /usr/lib/perl/5.8.8/auto/MIME/Base64/Base64.so (0x6C78000)
--8800-- object doesn't have a symbol table
--8800-- Reading syms from /usr/lib/perl5/auto/HTML/Parser/Parser.so (0x767B000)
--8800-- object doesn't have a symbol table
--8800-- Reading syms from /usr/lib/perl5/auto/Term/ReadKey/ReadKey.so (0x7786000)
--8800-- object doesn't have a symbol table
==8800==
==8800== ERROR SUMMARY: 2 errors from 1 contexts (suppressed: 81 from 2)
==8800==
==8800== 2 errors in context 1 of 1:
==8800== Invalid read of size 8
==8800== at 0x4014CE4: (within /lib/ld-2.4.so)
==8800== by 0x400A29D: (within /lib/ld-2.4.so)
==8800== by 0x4005F94: (within /lib/ld-2.4.so)
==8800== by 0x40081D5: (within /lib/ld-2.4.so)
==8800== by 0x400B45C: (within /lib/ld-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x400BB2B: (within /lib/ld-2.4.so)
==8800== by 0x40111FC: (within /lib/ld-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x4010C0A: (within /lib/ld-2.4.so)
==8800== by 0x4B251A9: (within /lib/libdl-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== Address 0x647FF00 is 16 bytes inside a block of size 19 alloc'd
==8800== at 0x4A2080E: malloc (vg_replace_malloc.c:149)
==8800== by 0x400865F: (within /lib/ld-2.4.so)
==8800== by 0x400B45C: (within /lib/ld-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x400BB2B: (within /lib/ld-2.4.so)
==8800== by 0x40111FC: (within /lib/ld-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x4010C0A: (within /lib/ld-2.4.so)
==8800== by 0x4B251A9: (within /lib/libdl-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x4B2571C: (within /lib/libdl-2.4.so)
==8800== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
--8800--
--8800-- supp: 72 Fedora-Core-5-hack3-ld24
--8800-- supp: 9 DynaLoader 6
==8800==
==8800== IN SUMMARY: 2 errors from 1 contexts (suppressed: 81 from 2)
==8800==
==8800== malloc/free: in use at exit: 27,238 bytes in 516 blocks.
==8800== malloc/free: 1,282,727 allocs, 1,282,211 frees, 10,840,858,152 bytes allocated.
==8800==
==8800== searching for pointers to 516 not-freed blocks.
==8800== checked 556,824 bytes.
==8800==
==8800==
==8800== 19 bytes in 1 blocks are still reachable in loss record 1 of 9
==8800== at 0x4A2080E: malloc (vg_replace_malloc.c:149)
==8800== by 0x400865F: (within /lib/ld-2.4.so)
==8800== by 0x400B45C: (within /lib/ld-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x400BB2B: (within /lib/ld-2.4.so)
==8800== by 0x40111FC: (within /lib/ld-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x4010C0A: (within /lib/ld-2.4.so)
==8800== by 0x4B251A9: (within /lib/libdl-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x4B2571C: (within /lib/libdl-2.4.so)
==8800== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==8800==
==8800==
==8800== 32 bytes in 1 blocks are still reachable in loss record 2 of 9
==8800== at 0x4A1FB37: calloc (vg_replace_malloc.c:279)
==8800== by 0x4B2577A: (within /lib/libdl-2.4.so)
==8800== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==8800== by 0x41DEF8: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==8800== by 0x4C53CD: Perl_pp_entersub (pp_hot.c:2877)
==8800== by 0x4993BF: Perl_runops_debug (dump.c:1459)
==8800== by 0x425351: S_call_body (perl.c:2731)
==8800== by 0x424DAA: Perl_call_sv (perl.c:2646)
==8800== by 0x42ADDA: S_call_list_body (perl.c:5262)
==8800== by 0x42A7C9: Perl_call_list (perl.c:5191)
==8800== by 0x470AF0: Perl_newATTRSUB (op.c:4520)
==8800== by 0x46B4BA: Perl_utilize (op.c:3096)
==8800==
==8800==
==8800== 40 bytes in 1 blocks are still reachable in loss record 3 of 9
==8800== at 0x4A2080E: malloc (vg_replace_malloc.c:149)
==8800== by 0x400BBDC: (within /lib/ld-2.4.so)
==8800== by 0x40111FC: (within /lib/ld-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x4010C0A: (within /lib/ld-2.4.so)
==8800== by 0x4B251A9: (within /lib/libdl-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x4B2571C: (within /lib/libdl-2.4.so)
==8800== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==8800== by 0x41DEF8: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==8800== by 0x4C53CD: Perl_pp_entersub (pp_hot.c:2877)
==8800== by 0x4993BF: Perl_runops_debug (dump.c:1459)
==8800==
==8800==
==8800== 460 bytes in 11 blocks are still reachable in loss record 4 of 9
==8800== at 0x4A2080E: malloc (vg_replace_malloc.c:149)
==8800== by 0x4007573: (within /lib/ld-2.4.so)
==8800== by 0x400812E: (within /lib/ld-2.4.so)
==8800== by 0x401119C: (within /lib/ld-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x4010C0A: (within /lib/ld-2.4.so)
==8800== by 0x4B251A9: (within /lib/libdl-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x4B2571C: (within /lib/libdl-2.4.so)
==8800== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==8800== by 0x41DEF8: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==8800== by 0x4C53CD: Perl_pp_entersub (pp_hot.c:2877)
==8800==
==8800==
==8800== 479 bytes in 12 blocks are still reachable in loss record 5 of 9
==8800== at 0x4A2080E: malloc (vg_replace_malloc.c:149)
==8800== by 0x400A34B: (within /lib/ld-2.4.so)
==8800== by 0x4005F94: (within /lib/ld-2.4.so)
==8800== by 0x40081D5: (within /lib/ld-2.4.so)
==8800== by 0x401119C: (within /lib/ld-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x4010C0A: (within /lib/ld-2.4.so)
==8800== by 0x4B251A9: (within /lib/libdl-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x4B2571C: (within /lib/libdl-2.4.so)
==8800== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==8800== by 0x41DEF8: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==8800==
==8800==
==8800== 632 bytes in 11 blocks are still reachable in loss record 6 of 9
==8800== at 0x4A2080E: malloc (vg_replace_malloc.c:149)
==8800== by 0x400BD62: (within /lib/ld-2.4.so)
==8800== by 0x40111FC: (within /lib/ld-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x4010C0A: (within /lib/ld-2.4.so)
==8800== by 0x4B251A9: (within /lib/libdl-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x4B2571C: (within /lib/libdl-2.4.so)
==8800== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==8800== by 0x41DEF8: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==8800== by 0x4C53CD: Perl_pp_entersub (pp_hot.c:2877)
==8800== by 0x4993BF: Perl_runops_debug (dump.c:1459)
==8800==
==8800==
==8800== 936 bytes in 12 blocks are still reachable in loss record 7 of 9
==8800== at 0x4A1FB37: calloc (vg_replace_malloc.c:279)
==8800== by 0x400ECD1: (within /lib/ld-2.4.so)
==8800== by 0x4011472: (within /lib/ld-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x4010C0A: (within /lib/ld-2.4.so)
==8800== by 0x4B251A9: (within /lib/libdl-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x4B2571C: (within /lib/libdl-2.4.so)
==8800== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==8800== by 0x41DEF8: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==8800== by 0x4C53CD: Perl_pp_entersub (pp_hot.c:2877)
==8800== by 0x4993BF: Perl_runops_debug (dump.c:1459)
==8800==
==8800==
==8800== 10,730 bytes in 455 blocks are definitely lost in loss record 8 of 9
==8800== at 0x4A2080E: malloc (vg_replace_malloc.c:149)
==8800== by 0x499A85: Perl_safesysmalloc (util.c:78)
==8800== by 0x49B7DF: Perl_savepvn (util.c:789)
==8800== by 0x42D8C7: Perl_gv_fetchpv (gv.c:754)
==8800== by 0x42D31B: Perl_gv_stashpvn (gv.c:671)
==8800== by 0x46AE5A: Perl_package (op.c:3010)
==8800== by 0x45FE6C: Perl_yyparse (perly.y:415)
==8800== by 0x51BC30: S_doeval (pp_ctl.c:2894)
==8800== by 0x51ED2E: Perl_pp_require (pp_ctl.c:3429)
==8800== by 0x4993BF: Perl_runops_debug (dump.c:1459)
==8800== by 0x425351: S_call_body (perl.c:2731)
==8800== by 0x424DAA: Perl_call_sv (perl.c:2646)
==8800==
==8800==
==8800== 13,910 bytes in 12 blocks are still reachable in loss record 9 of 9
==8800== at 0x4A1FB37: calloc (vg_replace_malloc.c:279)
==8800== by 0x400A0E4: (within /lib/ld-2.4.so)
==8800== by 0x4005F94: (within /lib/ld-2.4.so)
==8800== by 0x40081D5: (within /lib/ld-2.4.so)
==8800== by 0x401119C: (within /lib/ld-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x4010C0A: (within /lib/ld-2.4.so)
==8800== by 0x4B251A9: (within /lib/libdl-2.4.so)
==8800== by 0x400D1E5: (within /lib/ld-2.4.so)
==8800== by 0x4B2571C: (within /lib/libdl-2.4.so)
==8800== by 0x4B25120: dlopen (in /lib/libdl-2.4.so)
==8800== by 0x41DEF8: XS_DynaLoader_dl_load_file (DynaLoader.xs:191)
==8800==
==8800== LEAK SUMMARY:
==8800== definitely lost: 10,730 bytes in 455 blocks.
==8800== possibly lost: 0 bytes in 0 blocks.
==8800== still reachable: 16,508 bytes in 61 blocks.
==8800== suppressed: 0 bytes in 0 blocks.
--8800-- memcheck: sanity checks: 7431 cheap, 298 expensive
--8800-- memcheck: auxmaps: 0 auxmap entries (0k, 0M) in use
--8800-- memcheck: auxmaps: 0 searches, 0 comparisons
--8800-- memcheck: SMs: n_issued = 137311 (2196976k, 2145M)
--8800-- memcheck: SMs: n_deissued = 136489 (2183824k, 2132M)
--8800-- memcheck: SMs: max_noaccess = 524287 (8388592k, 8191M)
--8800-- memcheck: SMs: max_undefined = 35 (560k, 0M)
--8800-- memcheck: SMs: max_defined = 380 (6080k, 5M)
--8800-- memcheck: SMs: max_non_DSM = 952 (15232k, 14M)
--8800-- memcheck: max sec V bit nodes: 22 (1k, 0M)
--8800-- memcheck: set_sec_vbits8 calls: 1476 (new: 22, updates: 1454)
--8800-- memcheck: max shadow mem size: 19377k, 18M
--8800-- translate: fast SP updates identified: 28,517 ( 84.9%)
--8800-- translate: generic_known SP updates identified: 3,342 ( 9.9%)
--8800-- translate: generic_unknown SP updates identified: 1,708 ( 5.0%)
--8800-- tt/tc: 4,789,450 tt lookups requiring 6,163,239 probes
--8800-- tt/tc: 4,789,450 fast-cache updates, 2 flushes
--8800-- transtab: new 33,738 (1,018,374 -> 17,862,208; ratio 175:10) [0 scs]
--8800-- transtab: dumped 0 (0 -> ??)
--8800-- transtab: discarded 0 (0 -> ??)
--8800-- scheduler: 743,174,202 jumps (bb entries).
--8800-- scheduler: 7,431/7,346,016 major/minor sched events.
--8800-- sanity: 7432 cheap, 298 expensive checks.
--8800-- exectx: 30,011 lists, 10,285 contexts (avg 0 per list)
--8800-- exectx: 2,556,335 searches, 11,164,155 full compares (4,367 per 1000)
--8800-- exectx: 677 cmp2, 1,017 cmp4, 0 cmpAll