Mailing List Archive

> From: aburlison@cix.compulink.co.uk (Alan Burlison)
>
> I have rebuilt beta1 with patches a-d and Tim's memory stats patch
> applied. It *definitely* leaks - memory stats follow:
>
> Memory allocation statistics after compilation: (buckets 8..16384)
> 17968 free: 118 34 59 14 15 2 2 2 4 0 0 0
> 129488 used: 138 222 1093 690 49 6 10 14 1 0 0 1
> Memory allocation statistics after execution: (buckets 8..16384)
> 40704 free: 388 638 228 118 18 12 2 0 3 0 0 0
> 6357248 used: 636 2434 8028 4138 350 28 1326 4918 2 1 0 1
>
> This was generated by only processing the first 1000 records of a 17,000
> record file. 5.001m processes the entire 17,000 records without growing
> to more than 750K. beta1 is up to 6.3Mb with less than a tenth of the
> file processed.

And most of that space is in the 1k bucket (eg ~ 0.5..1.0Kb). Does that help
pin down what is being leaked?

I notice that you are getting allocations in you 8 byte bucket.
Could you post your myconfig output. Just curious (and you don't
say if the perl has -DDEBUGGING enabled.

> Some other funnies I noticed: firstly, running Configure with a
> -D DEBUGGING_MSTATS does not add this to cflags, despite what the
> documentation says.

Er, I don;t think the documentation says that (but it could probably
do with being clarified). It defines a _Configure_ variable called
DEBUGGING_MSTATS. What you wanted to do was more along the lines of:

Configure -D cppflags='-DDEBUGGING_MSTATS'

Tim (recovering after our works Xmas party last night :-)

On Fri, 8 Dec 1995, Alan Burlison wrote:

> I have rebuilt beta1 with patches a-d and Tim's memory stats patch
> applied. It *definitely* leaks - memory stats follow:
>
> Some other funnies I noticed: firstly, running Configure with a
> -D DEBUGGING_MSTATS does not add this to cflags, despite what the
> documentation says.

Huh? I just tried
sh Configure -Dccflags='-D DEBUGGING_MSTATS'
and the default ccflags suggested by Configure included -D
DEBUGGING_MSTATS.

Perhaps you tried
sh Configure -D DEBUGGING_MSTATS
that will define a config.sh variable $DEBUGGING_MSTATS to 'define',
but since none of the .SH files use $DEBUGGING_MSTATS, it won't make
any difference.

(Mind you, I haven't actually looked at Tim's patch yet. It's
conceivable that he might have also patched .SH files to use
$DEBUGGING_MSTATS, but I really, really doubt it.)

Anyway, if you could suggest any changes to the documentation that
would have clarified this for you, by all means feel free to submit a
patch.

Andy Dougherty doughera@lafcol.lafayette.edu

On Fri, 8 Dec 1995, Alan Burlison wrote:

> I have rebuilt beta1 with patches a-d and Tim's memory stats patch
> applied. It *definitely* leaks - memory stats follow:
>
> Some other funnies I noticed: firstly, running Configure with a
> -D DEBUGGING_MSTATS does not add this to cflags, despite what the
> documentation says.

Huh? I just tried
sh Configure -Dccflags='-D DEBUGGING_MSTATS'
and the default ccflags suggested by Configure included -D
DEBUGGING_MSTATS.

Perhaps you tried
sh Configure -D DEBUGGING_MSTATS
that will define a config.sh variable $DEBUGGING_MSTATS to 'define',
but since none of the .SH files use $DEBUGGING_MSTATS, it won't make
any difference.

(Mind you, I haven't actually looked at Tim's patch yet. It's
conceivable that he might have also patched .SH files to use
$DEBUGGING_MSTATS, but I really, really doubt it.)

Anyway, if you could suggest any changes to the documentation that
would have clarified this for you, by all means feel free to submit a
patch.

Andy Dougherty doughera@lafcol.lafayette.edu

In-Reply-To: <Pine.SUN.3.91.951208095928.4911A-100000@lafcol>
> Huh? I just tried
> sh Configure -Dccflags='-D DEBUGGING_MSTATS'
> and the default ccflags suggested by Configure included -D
> DEBUGGING_MSTATS.
>
> Perhaps you tried
> sh Configure -D DEBUGGING_MSTATS

Oops, yes I did - rereading the doc I can see where I misinterpreted it.
I claim the late hour at which I was doing the rebuild as an excuse :-)

Alan Burlison aburlison@cix.compulink.co.uk

In-Reply-To: <9512081125.AA06592@toad.ig.co.uk>
Tim,

> And most of that space is in the 1k bucket (eg ~ 0.5..1.0Kb). Does that
> help pin down what is being leaked?

Sort of. The program reads in a data file and splits it into records &
fields. Fields are <= 10 chars long on average. A few are up to 100
chars.

> Could you post your myconfig output. Just curious (and you don't
> say if the perl has -DDEBUGGING enabled.

Here it is, and no, -DDEBUGGING was not enabled

Summary of my perl5 (patchlevel 2) configuration:
Platform:
osname=linux, osver=1, archname=i486-linux
uname='linux fubar 1.2.13 #2 sat oct 28 19:55:36 gmt 1995 i486 '
hint=previous
Compiler:
cc='gcc', optimize='-O2', ld='gcc'
cppflags='-D__USE_BSD_SIGNAL -Dbool=char -DHAS_BOOL
-DDEBUGGING_MSTATS'
ccflags ='-D__USE_BSD_SIGNAL -Dbool=char -DHAS_BOOL
-DDEBUGGING_MSTATS'
ldflags =''
stdchar='char', d_stdstdio=define, usevfork=false
voidflags=15, castflags=0, d_casti32=undef, d_castneg=define
intsize=4, alignbytes=4, usemymalloc=y, randbits=31
Libraries:
so=so
libpth=/lib /usr/lib
libs=-lndbm -lgdbm -ldbm -ldb -ldl -lm -lc -lcrypt -lbsd
libc=/usr/lib/libc.so.5
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef
cccdlflags='-fpic', ccdlflags='-rdynamic', lddlflags='-shared'

The data structure I am using is (briefly) described below.

Base class: Record. This holds a field-based representation of an input
record. This class is virtual. Two derived classes are used - TSRecord
and FLRecord, for tab-seperated and fixed-length fields respectively.
These derived classes don't add any data, just functionality, and don't
have their own new() methods. Data structure of a Record object is:

Record->{'data' => {field name => str value, field name => str value, ..},
'invalid' => { field name => integer, field name => integer, ..},
'num_errors' => integer,
'num_warns' => integer }

These TSRecord and FLRecord objects are in turn stored inside another
object, which holds all the subrecords making up a message record. The
data structure of this outer class is:

Msg330->{ 'insert' =>
{ record class =>
{ record type => [ Record object, Record object, ... ],
record type => [ Record object, Record object, ... ],
... },
record class =>
{ record type => [ Record object, Record object, ... ],
record type => [ Record object, Record object, ... ],
... },
... },
'delete'
{ record class =>
{ record type => integer,
record type => integer,
... },
record class =>
{ record type => integer,
record type => integer,
... },
... },
'num_errors' => integer,
'num_warns' => integer }
}

I will try to produce a cut-down version of the program that shows the
problem, but I am away in Paris nest week, so it may well be the week
after before I can devote much time to this :-(

Alan Burlison aburlison@cix.compulink.co.uk