Mailing List Archive: memory leak in KinoSearch::Util::BitVector (v .15)

memory leak in KinoSearch::Util::BitVector (v .15)

Aug 15, 2007, 4:56 PM

Post #1 of 3 (1385 views)

Am running into a bit of a snafu using a BitCollector in a persistent
environment. After calling the following code, memory utilization
grows arithmetically:

my $bit_vector = $collector->get_bit_vector();
my $result_ids = $bit_vector->to_arrayref();

From what I can tell, the array returned to perl is never freed up.
From BitVector.pm:

Kino_BitVec_to_array(BitVector* bit_vec) {
U32 num = 0;
AV *out_av;

out_av = newAV();
while (1) {
num = Kino_BitVec_next_set_bit(bit_vec, num);
if (num == KINO_BITVEC_SENTINEL)
break;
av_push( out_av, newSViv(num) );
num++;
}
return out_av;
}

Anyone run into this problem? Any suggestions for whether I'm on
base, or potentially how to solve for this?

Thanks,

Matthew Berk
Lead Search Architect
t 206.331.3528
f 206.331.3695
matthew@openlist.com

Marchex, Inc.
http://www.marchex.com

This email message and any attachments are solely for intended
recipients, and may contain information that is privileged and
confidential. If you are not the intended recipient, any
dissemination, distribution or copying is strictly prohibited. If you
believe that you may have received this message in error, please
immediately notify the sender by replying to this e-mail message.

memory leak in KinoSearch::Util::BitVector (v .15) [ In reply to ]

matthew at openlist

Aug 15, 2007, 5:16 PM

Post #2 of 3 (1323 views)

Permalink

Just doing a bit of digging. Would adding sv_2mortal make a difference?

PPCODE:
out_av = Kino_BitVec_to_array(bit_vec);
/* XPUSHs(sv_2mortal(newRV_noinc( (SV*)out_av ))); */
XPUSHs(sv_2mortal(newRV_noinc( (SV*)out_av )));
XSRETURN(1);

Thanks again,

Matthew

On Aug 15, 2007, at 4:39 PM, Matthew Berk wrote:

> Am running into a bit of a snafu using a BitCollector in a
> persistent environment. After calling the following code, memory
> utilization grows arithmetically:
>
> my $bit_vector = $collector->get_bit_vector();
> my $result_ids = $bit_vector->to_arrayref();
>
> From what I can tell, the array returned to perl is never freed up.
> From BitVector.pm:
>
> Kino_BitVec_to_array(BitVector* bit_vec) {
> U32 num = 0;
> AV *out_av;
>
> out_av = newAV();
> while (1) {
> num = Kino_BitVec_next_set_bit(bit_vec, num);
> if (num == KINO_BITVEC_SENTINEL)
> break;
> av_push( out_av, newSViv(num) );
> num++;
> }
> return out_av;
> }
>
> Anyone run into this problem? Any suggestions for whether I'm on
> base, or potentially how to solve for this?
>
> Thanks,
>
> Matthew Berk
> Lead Search Architect
> t 206.331.3528
> f 206.331.3695
> matthew@openlist.com
>
> Marchex, Inc.
> http://www.marchex.com
>
> This email message and any attachments are solely for intended
> recipients, and may contain information that is privileged and
> confidential. If you are not the intended recipient, any
> dissemination, distribution or copying is strictly prohibited. If
> you believe that you may have received this message in error,
> please immediately notify the sender by replying to this e-mail
> message.
>
>
>
> _______________________________________________
> KinoSearch mailing list
> KinoSearch@rectangular.com
> http://www.rectangular.com/mailman/listinfo/kinosearch

Matthew Berk
Lead Search Architect
t 206.331.3528
f 206.331.3695
matthew@openlist.com

Marchex, Inc.
http://www.marchex.com

This email message and any attachments are solely for intended
recipients, and may contain information that is privileged and
confidential. If you are not the intended recipient, any
dissemination, distribution or copying is strictly prohibited. If you
believe that you may have received this message in error, please
immediately notify the sender by replying to this e-mail message.

memory leak in KinoSearch::Util::BitVector (v .15) [ In reply to ]

marvin at rectangular

Aug 16, 2007, 6:04 AM

Post #3 of 3 (1328 views)

Permalink

On Aug 15, 2007, at 5:00 PM, Matthew Berk wrote:

> Just doing a bit of digging. Would adding sv_2mortal make a
> difference?
>
> PPCODE:
> out_av = Kino_BitVec_to_array(bit_vec);
> /* XPUSHs(sv_2mortal(newRV_noinc( (SV*)out_av ))); */
> XPUSHs(sv_2mortal(newRV_noinc( (SV*)out_av )));
> XSRETURN(1);

This originally confused me, because the commented-out line is
identical to the one that follows. In the real text, though, the
sv_2mortal is indeed missing.

XPUSHs(newRV_noinc( (SV*)out_av ));

The Valgrind testing I do before each release did not pick this up. :
( Presumably Perl reclaims the leaked elements during global
destruction, at least under PERL_DESTRUCT_LEVEL=2.

Your assessment and remedy are both correct, Matthew. I am swamped
right now, but I will release KS 0.16 with this fix when I get a
moment. Thank you.

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/