In recent past I worked on an idea that I had
to eliminate the frequent fork() / exec()
calls made by tcpserver for incoming emails
Summary
-------------
1. the entire smtpd code was written as a function smtp_init()
and compiled as /usr/lib/indimail/plugins/qmail_smtpd.so
2. the entire rblsmtpd code was written as a function rblsmtpd()
and compiled as /usr/lib/indimail/plugins/rblsmtpd.so
3. wrote a function load_shared() for tcpserver which is just
a 40 line function using dlopen(). To avoid messing up
this email link to this function is here
https://sourceforge.net/p/indimail/code/ci/master/tree/ucspi-tcp-0.88/load_shared.c
How it works
------------------
You declare environment variables PLUGIN0, PLUGIN1, ... to load
the plugins using dlopen. Normally tcpserver will load a function with
the basename of the plugin after removing ".so" from the name.
One can fine tune the behaviour for each
plugin by defining additional variables (but not necessary) like
PLUGIN0_init, PLUGIN0_dir
The value of PLUGIN0, PLUGIN1, etc variables should
point to the path. e.g.
PLUGIN0=/usr/lib/indimail/plugins/qmail_smtpd.so
PLUGIN1=/usr/lib/indimail/plugins/rblsmtpd.so
PLUGIN0_init=smtp_init
PLUGIN0_dir=/etc/indimail/control
The above causes tcpserver to load qmail_smtpd.so
and execute rblsmtpd() function for rbl and smtp_init() function
for each and every connection. BTW my supervise run file
for qmail-smtpd is like this
QMAILDUID=`/usr/bin/id -u indimail`
NOFILESGID=`/usr/bin/id -g indimail`
ME=`head -1 /etc/indimail/control/me`
HOSTNAME=`uname -n`
if [ -z "$QMAILDUID" -o -z "$NOFILESGID" -o -z "$ME" ]; then
echo QMAILDUID, NOFILESGID, or ME is unset in
echo /service/qmail-smtpd.25/run
sleep 5
exit 1
fi
if [ ! -f /etc/indimail/control/rcpthosts ]; then
echo "No /etc/indimail/control/rcpthosts!"
echo "Refusing to start SMTP listener because it'll create an open relay"
sleep 5
exit 1
fi
exec /usr/bin/envdir /service/qmail-smtpd.25/variables sh -c "
exec /usr/bin/softlimit -m \$SOFT_MEM -o 1024 \
/usr/bin/tcpserver -v -h -R -l \$LOCALIP \
-x /etc/indimail/tcp.smtp.cdb \
-c /service/qmail-smtpd.25/variables/MAXDAEMONS -o -b \$MAXDAEMONS \
-u $QMAILDUID -g $NOFILESGID \$LOCALIP \$PORT \$RBLCOMMAND \
/usr/lib/indimail/plugins/qmail_smtpd.so $HOSTNAME \$AUTHMODULES
/bin/false 2>&1"
Problem Statement
===============
1. There are duplicate functions between the shared
library of qmail_smtpd.so and rblsmtpd.so. This should
not have been an issue as I am using the flags
RTLD_NOW|RTLD_LOCAL|RTLD_NODELETE
I had hoped that RTLD_LOCAL will ensure that symbols
local to qmail_smtpd.so will not be used by rblsmtpd()
and vice versa.
2. However my tcpserver code had a control_readline()
function which was slightly different than the
control_readline() function used in qmail_smptd.so.
On CentOS7 I found that, smtp_init() function loaded
from qmail_smtpd.so started using the tcpserver's
control_readlne() function and it resulted in a BUG
3. To resolve this issue, I made both the functions same
4. I did objdump on qmail_smtpd.so and rblsmtpd.so and
found additional duplicate functions, which I believe
shoudn't be a problem ??? I used the command
$ objdump -T rblsmtpd.so |grep ".text" |awk '{print $7}' > /tmp/rbl
$ objdump -T qmail_smtpd.so |grep ".text"|awk '{print $7}' > /tmp/smtp
$ cat /tmp/smtp /tmp/rbl |sort |uniq -c|sort -k1|grep ".* 2"
2 alloc
2 alloc_free
2 alloc_re
2 byte_copy
2 byte_copyr
2 byte_diff
2 byte_zero
2 case_diffb
2 case_diffs
2 commands
2 env_get
2 error_str
2 fmt_ulong
2 ndelay_on
2 open_read
2 scan_ulong
2 scan_xlong
2 sig_catch
2 stralloc_append
2 stralloc_cat
2 stralloc_catb
2 stralloc_cats
2 stralloc_copy
2 stralloc_copyb
2 stralloc_copys
2 stralloc_ready
2 stralloc_readyplus
2 str_chr
2 str_diff
2 str_len
2 str_start
5. I need to do the same for global variables
but stuck because of lack of knowledge
6. I have done some deep reading on dlopen()
and have additionally added the flag
RTLD_DEEPBIND available since
glibc 2.3.4. Removing the duplicate
functions and RTLD_DEEPBIND has
fixed the bug i encountered on CentOS7
7. I have saved on the exec call and copying
of program space when exec happens
but loading of control files still happens
in the child after fork(). My idea is to
write a separate shared object just
to load all the control files. But it involves
some tedious work due to so many global
variables, especially the stralloc variables
Worries
======
1. Maybe I have done something stupid here
eventhough it works for me
2. I don't have the full knowledge of how
the ld loader works and half baked
knowledge is not good.
3. I believe this will require community effort,
and I am ready to work under someone's
guidance. I believe this can be done
for qmail/netqmail as indimail is basically
DJB's qmail with some meddling of my own.
I do believe that this (along with loading
of control files in the parent) will result
in some good performance improvement.
Kai Peter did reach out to me and Erwin
But somehow, we never got in touch as
I have to figure out how to use IRC. But
I am ready to play a developer role if
someone steps in to revive qmail. I have
looked at sendmail/postfix code and
feel it would be a shame have a far
superior code lose out.
Copying Kai and Eh in this email as I find them still
interested in working to improve qmail and I do once
in a while look at Eh's code. If I borrow his code, I do mention
it in the indimail CREDITS file
--
Regards Manvendra - http://www.indimail.org
GPG Pub Key
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC7CBC760014D250C
to eliminate the frequent fork() / exec()
calls made by tcpserver for incoming emails
Summary
-------------
1. the entire smtpd code was written as a function smtp_init()
and compiled as /usr/lib/indimail/plugins/qmail_smtpd.so
2. the entire rblsmtpd code was written as a function rblsmtpd()
and compiled as /usr/lib/indimail/plugins/rblsmtpd.so
3. wrote a function load_shared() for tcpserver which is just
a 40 line function using dlopen(). To avoid messing up
this email link to this function is here
https://sourceforge.net/p/indimail/code/ci/master/tree/ucspi-tcp-0.88/load_shared.c
How it works
------------------
You declare environment variables PLUGIN0, PLUGIN1, ... to load
the plugins using dlopen. Normally tcpserver will load a function with
the basename of the plugin after removing ".so" from the name.
One can fine tune the behaviour for each
plugin by defining additional variables (but not necessary) like
PLUGIN0_init, PLUGIN0_dir
The value of PLUGIN0, PLUGIN1, etc variables should
point to the path. e.g.
PLUGIN0=/usr/lib/indimail/plugins/qmail_smtpd.so
PLUGIN1=/usr/lib/indimail/plugins/rblsmtpd.so
PLUGIN0_init=smtp_init
PLUGIN0_dir=/etc/indimail/control
The above causes tcpserver to load qmail_smtpd.so
and execute rblsmtpd() function for rbl and smtp_init() function
for each and every connection. BTW my supervise run file
for qmail-smtpd is like this
QMAILDUID=`/usr/bin/id -u indimail`
NOFILESGID=`/usr/bin/id -g indimail`
ME=`head -1 /etc/indimail/control/me`
HOSTNAME=`uname -n`
if [ -z "$QMAILDUID" -o -z "$NOFILESGID" -o -z "$ME" ]; then
echo QMAILDUID, NOFILESGID, or ME is unset in
echo /service/qmail-smtpd.25/run
sleep 5
exit 1
fi
if [ ! -f /etc/indimail/control/rcpthosts ]; then
echo "No /etc/indimail/control/rcpthosts!"
echo "Refusing to start SMTP listener because it'll create an open relay"
sleep 5
exit 1
fi
exec /usr/bin/envdir /service/qmail-smtpd.25/variables sh -c "
exec /usr/bin/softlimit -m \$SOFT_MEM -o 1024 \
/usr/bin/tcpserver -v -h -R -l \$LOCALIP \
-x /etc/indimail/tcp.smtp.cdb \
-c /service/qmail-smtpd.25/variables/MAXDAEMONS -o -b \$MAXDAEMONS \
-u $QMAILDUID -g $NOFILESGID \$LOCALIP \$PORT \$RBLCOMMAND \
/usr/lib/indimail/plugins/qmail_smtpd.so $HOSTNAME \$AUTHMODULES
/bin/false 2>&1"
Problem Statement
===============
1. There are duplicate functions between the shared
library of qmail_smtpd.so and rblsmtpd.so. This should
not have been an issue as I am using the flags
RTLD_NOW|RTLD_LOCAL|RTLD_NODELETE
I had hoped that RTLD_LOCAL will ensure that symbols
local to qmail_smtpd.so will not be used by rblsmtpd()
and vice versa.
2. However my tcpserver code had a control_readline()
function which was slightly different than the
control_readline() function used in qmail_smptd.so.
On CentOS7 I found that, smtp_init() function loaded
from qmail_smtpd.so started using the tcpserver's
control_readlne() function and it resulted in a BUG
3. To resolve this issue, I made both the functions same
4. I did objdump on qmail_smtpd.so and rblsmtpd.so and
found additional duplicate functions, which I believe
shoudn't be a problem ??? I used the command
$ objdump -T rblsmtpd.so |grep ".text" |awk '{print $7}' > /tmp/rbl
$ objdump -T qmail_smtpd.so |grep ".text"|awk '{print $7}' > /tmp/smtp
$ cat /tmp/smtp /tmp/rbl |sort |uniq -c|sort -k1|grep ".* 2"
2 alloc
2 alloc_free
2 alloc_re
2 byte_copy
2 byte_copyr
2 byte_diff
2 byte_zero
2 case_diffb
2 case_diffs
2 commands
2 env_get
2 error_str
2 fmt_ulong
2 ndelay_on
2 open_read
2 scan_ulong
2 scan_xlong
2 sig_catch
2 stralloc_append
2 stralloc_cat
2 stralloc_catb
2 stralloc_cats
2 stralloc_copy
2 stralloc_copyb
2 stralloc_copys
2 stralloc_ready
2 stralloc_readyplus
2 str_chr
2 str_diff
2 str_len
2 str_start
5. I need to do the same for global variables
but stuck because of lack of knowledge
6. I have done some deep reading on dlopen()
and have additionally added the flag
RTLD_DEEPBIND available since
glibc 2.3.4. Removing the duplicate
functions and RTLD_DEEPBIND has
fixed the bug i encountered on CentOS7
7. I have saved on the exec call and copying
of program space when exec happens
but loading of control files still happens
in the child after fork(). My idea is to
write a separate shared object just
to load all the control files. But it involves
some tedious work due to so many global
variables, especially the stralloc variables
Worries
======
1. Maybe I have done something stupid here
eventhough it works for me
2. I don't have the full knowledge of how
the ld loader works and half baked
knowledge is not good.
3. I believe this will require community effort,
and I am ready to work under someone's
guidance. I believe this can be done
for qmail/netqmail as indimail is basically
DJB's qmail with some meddling of my own.
I do believe that this (along with loading
of control files in the parent) will result
in some good performance improvement.
Kai Peter did reach out to me and Erwin
But somehow, we never got in touch as
I have to figure out how to use IRC. But
I am ready to play a developer role if
someone steps in to revive qmail. I have
looked at sendmail/postfix code and
feel it would be a shame have a far
superior code lose out.
Copying Kai and Eh in this email as I find them still
interested in working to improve qmail and I do once
in a while look at Eh's code. If I borrow his code, I do mention
it in the indimail CREDITS file
--
Regards Manvendra - http://www.indimail.org
GPG Pub Key
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC7CBC760014D250C
