Mailing List Archive: possible Memleak in dbmail cvs 2.0 patch attached

possible Memleak in dbmail cvs 2.0 patch attached

Jan 21, 2004, 9:36 AM

Post #1 of 7 (935 views)

Dear Dev,

I belive I have found in 2.0cvs a few places where the db result is not
freed under mysql, This was causing my imapd to get outragiously large,
like in the 100Mb range, after pulling only about 10000 headers. The
patch I have attached also fixes a minor issue with dbmail and mysql,
and that is that after an overnight the mysql server may timeout the
connections and there isn't a reconnect check in the code for
check_connection like there is in the pgsql driver. I added that as well
as added the mysql specific auto-reconnect option. There is also a minor
missing free in the the authsql.c with the escaped user str.

I also belive there is a leak still with one of the allocated list's
(list.c) not being list freeded, All this memory work was traced with
dmalloc which is a fantastic package, and allows for logfiles per pid as
well as line numbers.

Please review the diff and apply if it suites. I really apreciate
everyones hard work on this project and I am very excited with the
progress and overall stablity of the 2.0 branch as it is now. I have
been running it (after my patches) for about a week now under light load
but have put it under some major stress with the postal stress test
package and it has done very well.

Does the dbmail list know of a better testing suite that fully supports
imap or may have more features?

Thanks,
Leif Jackson

--
Jones Jackson Consulting http://www.jjcons.com
Programmer && Linux Enthusiast ljackson@jjcons.com
"#define QUESTION ((bb) || !(bb))" - Shakespeare
http://www.jjcons.com/pgpkeys.html

Re: possible Memleak in dbmail cvs 2.0 patch attached [ In reply to ]

ilja at ic-s

Jan 22, 2004, 9:07 AM

Post #2 of 7 (918 views)

Permalink

Hi

Leif Jackson wrote:

> Dear Dev,
>
> I belive I have found in 2.0cvs a few places where the db result is not
> freed under mysql, This was causing my imapd to get outragiously large,
> like in the 100Mb range, after pulling only about 10000 headers. The
> patch I have attached also fixes a minor issue with dbmail and mysql,
> and that is that after an overnight the mysql server may timeout the
> connections and there isn't a reconnect check in the code for
> check_connection like there is in the pgsql driver. I added that as well
> as added the mysql specific auto-reconnect option. There is also a minor
> missing free in the the authsql.c with the escaped user str.
Great work. I'll have a look at it tomorrow morning!
>
> I also belive there is a leak still with one of the allocated list's
> (list.c) not being list freeded, All this memory work was traced with
> dmalloc which is a fantastic package, and allows for logfiles per pid as
> well as line numbers.
Sounds good. their website (dmalloc.com) is unreachable at the moment,
but I'll take a look at it.

>
> Please review the diff and apply if it suites. I really apreciate
> everyones hard work on this project and I am very excited with the
> progress and overall stablity of the 2.0 branch as it is now. I have
> been running it (after my patches) for about a week now under light load
> but have put it under some major stress with the postal stress test
> package and it has done very well.
I'll review it asap.
>
> Does the dbmail list know of a better testing suite that fully supports
> imap or may have more features?
I mostly use my normal mail client and sometimes some python scripts.
There's no substitute for testing in production though. I've been using
dbmail 2.0 in parallel with dbmail 1.2 for 1 1/2 months now. My mail
gets sent to both dbmail 2.0 and dbmail 1.2 (so I get to read every
email twice ;) ).

Thanks for the work,
Ilja
--
IC&S
Stadhouderslaan 57
3583 JD Utrecht

PGP-key:
http://www.ic-s.nl/keys/ilja.txt

Re: possible Memleak in dbmail cvs 2.0 patch attached [ In reply to ]

aaron at serendipity

Jan 22, 2004, 9:18 AM

Post #3 of 7 (923 views)

Permalink

How are you doing this? I'd like to do something similar, and figure a little
shell script would do the trick, but it's really, really low on my todo list
:-\ Could you post your config / script?

Thanks!
Aaron

Ilja Booij <ilja@ic-s.nl> said:
[cuts to my question]
> >
> > Does the dbmail list know of a better testing suite that fully supports
> > imap or may have more features?
> I mostly use my normal mail client and sometimes some python scripts.
> There's no substitute for testing in production though. I've been using
> dbmail 2.0 in parallel with dbmail 1.2 for 1 1/2 months now. My mail
> gets sent to both dbmail 2.0 and dbmail 1.2 (so I get to read every
> email twice ;) ).

--

Re: possible Memleak in dbmail cvs 2.0 patch attached [ In reply to ]

ilja at ic-s

Jan 22, 2004, 9:37 AM

Post #4 of 7 (917 views)

Permalink

sending to both dbmail 1.2 and 2.0?

dbmail 2.0 server is on a different machine:
I have an alias in dbmail 1.2 setup which sends all mail for
ilja@ic-s.nl to ilja@dbmail2_0machine

really easy, and works like a charm. I guess that using both
on one machine would be harder (Although you could send it to
a different maildomain on the same machine, and let postfix use
a different transport for that domain.

Is this enough info?

Ilja

Aaron Stone wrote:

> How are you doing this? I'd like to do something similar, and figure a little
> shell script would do the trick, but it's really, really low on my todo list
> :-\ Could you post your config / script?
>
> Thanks!
> Aaron
>
>
> Ilja Booij <ilja@ic-s.nl> said:
> [cuts to my question]
>
>>> Does the dbmail list know of a better testing suite that fully supports
>>>imap or may have more features?
>>
>>I mostly use my normal mail client and sometimes some python scripts.
>>There's no substitute for testing in production though. I've been using
>>dbmail 2.0 in parallel with dbmail 1.2 for 1 1/2 months now. My mail
>>gets sent to both dbmail 2.0 and dbmail 1.2 (so I get to read every
>>email twice ;) ).
>
>

--
IC&S
Stadhouderslaan 57
3583 JD Utrecht

PGP-key:
http://www.ic-s.nl/keys/ilja.txt

Re: possible Memleak in dbmail cvs 2.0 patch attached [ In reply to ]

ljackson at jjcons

Jan 22, 2004, 10:08 AM

Post #5 of 7 (920 views)

Permalink

Ilja Booij wrote:
>
> Hi
>
> Leif Jackson wrote:
>

> > I also belive there is a leak still with one of the allocated list's
> > (list.c) not being list freeded, All this memory work was traced with
> > dmalloc which is a fantastic package, and allows for logfiles per pid as
> > well as line numbers.
> Sounds good. their website (dmalloc.com) is unreachable at the moment,
> but I'll take a look at it.
>
I see this as well I have put the most current release of dmalloc on my
site
http://www.jjcons.com/dmalloc-5.2.4.tgz I have also attached a diffrent
patch which I used to do
my work but since it added a header file and a few other things I didn't
want it in the first diff.

bascialy you just need to patch with this patch, which includes my
changes for the db free's and then edit the makefile to define -DDMALLOC
and linke the libdmallocth.a or the shared lib if you wish. Then in the
mydebug.h file you will see the options I used to get these results.
Note, I do my devel on an sun solaris 9 box with gcc.

> > Does the dbmail list know of a better testing suite that fully supports
> > imap or may have more features?
> I mostly use my normal mail client and sometimes some python scripts.
> There's no substitute for testing in production though. I've been using
> dbmail 2.0 in parallel with dbmail 1.2 for 1 1/2 months now. My mail
> gets sent to both dbmail 2.0 and dbmail 1.2 (so I get to read every
> email twice ;) ).

Thanks for the info. I was getting ready to write a test suite in perl
using the parrallel fork manager modules but I wanted to check with this
before I duplicated work on a test suite.

>
> Thanks for the work,

Your very welcome thanks for such a great package! :)

-leif

--
Jones Jackson Consulting http://www.jjcons.com
Programmer && Linux Enthusiast ljackson@jjcons.com
"#define QUESTION ((bb) || !(bb))" - Shakespeare
http://www.jjcons.com/pgpkeys.html

Re: possible Memleak in dbmail cvs 2.0 patch attached [ In reply to ]

ljackson at jjcons

Jan 22, 2004, 10:49 AM

Post #6 of 7 (917 views)

Permalink

Oops, my bad... didn't attach anything... here is the patch with dmalloc
stuff.

> Ilja Booij wrote:
>>
>> Hi
>>
>> Leif Jackson wrote:
>>
>
>> > I also belive there is a leak still with one of the allocated list's
>> > (list.c) not being list freeded, All this memory work was traced with
>> > dmalloc which is a fantastic package, and allows for logfiles per pid
>> as
>> > well as line numbers.
>> Sounds good. their website (dmalloc.com) is unreachable at the moment,
>> but I'll take a look at it.
>>
> I see this as well I have put the most current release of dmalloc on my
> site
> http://www.jjcons.com/dmalloc-5.2.4.tgz I have also attached a diffrent
> patch which I used to do
> my work but since it added a header file and a few other things I didn't
> want it in the first diff.
>
> bascialy you just need to patch with this patch, which includes my
> changes for the db free's and then edit the makefile to define -DDMALLOC
> and linke the libdmallocth.a or the shared lib if you wish. Then in the
> mydebug.h file you will see the options I used to get these results.
> Note, I do my devel on an sun solaris 9 box with gcc.
>
>
>> > Does the dbmail list know of a better testing suite that fully
>> supports
>> > imap or may have more features?
>> I mostly use my normal mail client and sometimes some python scripts.
>> There's no substitute for testing in production though. I've been using
>> dbmail 2.0 in parallel with dbmail 1.2 for 1 1/2 months now. My mail
>> gets sent to both dbmail 2.0 and dbmail 1.2 (so I get to read every
>> email twice ;) ).
>
> Thanks for the info. I was getting ready to write a test suite in perl
> using the parrallel fork manager modules but I wanted to check with this
> before I duplicated work on a test suite.
>
>>
>> Thanks for the work,
>
> Your very welcome thanks for such a great package! :)
>
> -leif
>
> --
> Jones Jackson Consulting http://www.jjcons.com
> Programmer && Linux Enthusiast ljackson@jjcons.com
> "#define QUESTION ((bb) || !(bb))" - Shakespeare
> http://www.jjcons.com/pgpkeys.html
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>

Re: possible Memleak in dbmail cvs 2.0 patch attached [ In reply to ]

ilja at ic-s

Jan 29, 2004, 4:09 AM

Post #7 of 7 (918 views)

Permalink

Hi,

I've put Leif's patches (with some small changes) in DBMail2.0 CVS.

Ilja

Leif Jackson wrote:

> Oops, my bad... didn't attach anything... here is the patch with dmalloc
> stuff.
>
>
>>Ilja Booij wrote:
>>
>>>Hi
>>>
>>>Leif Jackson wrote:
>>>
>>
>>>> I also belive there is a leak still with one of the allocated list's
>>>>(list.c) not being list freeded, All this memory work was traced with
>>>>dmalloc which is a fantastic package, and allows for logfiles per pid
>>>
>>>as
>>>
>>>>well as line numbers.
>>>
>>>Sounds good. their website (dmalloc.com) is unreachable at the moment,
>>>but I'll take a look at it.
>>>
>>
>>I see this as well I have put the most current release of dmalloc on my
>>site
>>http://www.jjcons.com/dmalloc-5.2.4.tgz I have also attached a diffrent
>>patch which I used to do
>>my work but since it added a header file and a few other things I didn't
>>want it in the first diff.
>>
>>bascialy you just need to patch with this patch, which includes my
>>changes for the db free's and then edit the makefile to define -DDMALLOC
>>and linke the libdmallocth.a or the shared lib if you wish. Then in the
>>mydebug.h file you will see the options I used to get these results.
>>Note, I do my devel on an sun solaris 9 box with gcc.
>>
>>
>>
>>>> Does the dbmail list know of a better testing suite that fully
>>>
>>>supports
>>>
>>>>imap or may have more features?
>>>
>>>I mostly use my normal mail client and sometimes some python scripts.
>>>There's no substitute for testing in production though. I've been using
>>>dbmail 2.0 in parallel with dbmail 1.2 for 1 1/2 months now. My mail
>>>gets sent to both dbmail 2.0 and dbmail 1.2 (so I get to read every
>>>email twice ;) ).
>>
>>Thanks for the info. I was getting ready to write a test suite in perl
>>using the parrallel fork manager modules but I wanted to check with this
>>before I duplicated work on a test suite.
>>
>>
>>>Thanks for the work,
>>
>>Your very welcome thanks for such a great package! :)
>>
>>-leif
>>
>>--
>>Jones Jackson Consulting http://www.jjcons.com
>>Programmer && Linux Enthusiast ljackson@jjcons.com
>>"#define QUESTION ((bb) || !(bb))" - Shakespeare
>>http://www.jjcons.com/pgpkeys.html
>>_______________________________________________
>>Dbmail-dev mailing list
>>Dbmail-dev@dbmail.org
>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>

--
IC&S
Stadhouderslaan 57
3583 JD Utrecht

PGP-key:
http://www.ic-s.nl/keys/ilja.txt

Mailing List Archive

Attached Files:

Attached Files: