Mailing List Archive: Benchmarked tcpserver dlopen() -> qmail_smtpd() VS tcpserver --> fork() --> exec qmail-smtpd

Benchmarked tcpserver dlopen() -> qmail_smtpd() VS tcpserver --> fork() --> exec qmail-smtpd

Apr 4, 2017, 7:11 AM

Post #1 of 14 (3568 views)

On 2 April 2017 at 20:47, Charles Cazabon
<search-web-for-address@pyropus.ca> wrote:
> Manvendra Bhangui <mbhangui@gmail.com> wrote:
>> On 1 April 2017 at 22:36, Charles Cazabon wrote:
>>
>> > reasonable operating systems and even vaguely-modern hardware. Do you
>> > actually have data showing that this minimal overhead is a bottleneck in
>> > your situation? If not, why are you trying to eliminate it?
>>
>> I do have a situation where I open around 23 control files in the
>> qmail-smtpd process. I want to move this to the main() of the
>> tcpserver process and avoid loading it again and again.
> [...]
>> No I do not have any data. It is definitely not a bottleneck. It's
>> just that the idea of loading the control files just once does look
>> good.
>
> I'm afraid you're fooling yourself. Any control files that are opened and
> read regularly - i.e. if you're getting at least a few SMTP connections every
> minute - will be satisfied from the cache. And qmail's reading of those files
> doesn't use a lot of CPU time or other resources.
>
> So you aren't really saving CPU, or anything else for that matter. You've
> just introduced complexity, and probably security problems, for no real
> benefit.
>
> I *strongly* urge you to profile the performance of vanilla qmail/tcpserver
> before you decide to "fix" something and try to make it "look good". I would
> bet dollars to doughnuts that you will find you are trying to optimize
> something which is only responsible for 0.01% of the program's work in the
> first place.
>
> As djb himself says, measure, don't speculate.

So finally I measured and now I have data, and it so consistent, that
I too am surprised. Using dlopen() to load the control files in
tcpserver gives a huge, huge gain if the control files are large. I
finally managed to do some extreme testing. In cases where the load
was light, the dlopen() method gives a very minor performance
improvement over case where tcpserver uses pathexec func to to exec
qmail-smtpd. In the extreme case where I have loaded a 25 Mib
badmailfrom, there is a huge difference. In the case of indimail-mta,
I loaded 50 Mib of control file. Every invocation of qmail-smtpd,
using the traditional fork()/exec() method, loads the control file and
my laptop (sony viao with hybrid sshd) becomes unresponsive for almost
an hour. netqmail also gives the same pathetic response. The testing
was done by firing, 2000 swaks perl program in parallel, for sending
emails through SMTP. I used indimail-mta's qmail-nullqueue to throw
the emails into void. For netqmail too, I defined QMAILQUEUE to
indimail-mta's nullqueue to keep the testing parameters equal.

In all the tests, dlopen() gave far superior response compared to the
existing tcpserver fork() --> exec() mechanism and the load never shot
above 170. In case of tcpserver doing fork() and then exec of
qmail-smtpd(), with each invocation of qmail-smtpd loading the 25 Mib
control file, the load shot up to > 1000 and after that laptop used to
remain frozen till the qmail-smtpd processes completed. I used a Sony
Viao latop with 8 GB Ram, 1Tb sshd 2012 model.

Using dlopen, I never was able to see qmail-smtpd() in the top
processes as the smtp function is now being done by tcpserver itself.
However, I did not notice the top screen getting filled with tcpserver
processes. In the fork() exec() mechanism, top used to show only
qmail-smtpd() processes.

The result for these tests are tabulated in the online google sheet below.
https://docs.google.com/spreadsheets/d/1lw0V2NRkUHxBVJuqiigVm-ggYiTiPJ9AoE8dMwviyew/edit?usp=sharing

I used this script for testing.
#!/bin/sh
count=0
while true
do
swaks --from testuser01@example.com --to testuser02@example.com >
/dev/null 2>&1 &
swaks --from testuser01@example.com --to testuser02@example.com >
/dev/null 2>&1 &
swaks --from testuser01@example.com --to testuser02@example.com >
/dev/null 2>&1 &
swaks --from testuser01@example.com --to testuser02@example.com >
/dev/null 2>&1 &
swaks --from testuser01@example.com --to testuser02@example.com >
/dev/null 2>&1 &
swaks --from testuser01@example.com --to testuser02@example.com >
/dev/null 2>&1 &
swaks --from testuser01@example.com --to testuser02@example.com >
/dev/null 2>&1 &
swaks --from testuser01@example.com --to testuser02@example.com >
/dev/null 2>&1 &
swaks --from testuser01@example.com --to testuser02@example.com >
/dev/null 2>&1 &
swaks --from testuser01@example.com --to testuser02@example.com >
/dev/null 2>&1 &
count=`expr $count + 1`
if [ $count -eq 200 ] ; then
break
fi
done
wait

The size of my control files for testing was huge (deliberately 25 Mib).

$ ls -l /etc/indimail/control/blackholedsenders
/etc/indimail/control/badmailfrom /var/qmail/control/badmailfrom
-rw-rw-r--. 1 mbhangui mbhangui 26889998 Apr 4 11:00
/etc/indimail/control/badmailfrom
lrwxrwxrwx. 1 root root 11 Apr 4 18:58
/etc/indimail/control/blackholedsenders -> badmailfrom
-rw-r--r--. 1 root root 26889998 Apr 4 17:03
/var/qmail/control/badmailfrom

Now the bad news :( I tried dlmopen() and it is not working. strace
gives the error "process tcpserver runs in x32 mode". There are no 32
bit libraries loaded. Just don't know that is happening and so I have
gone back to using dlopen() which uses the same namespace as
tcpserver's namespace.

Re: Benchmarked tcpserver dlopen() -> qmail_smtpd() VS tcpserver --> fork() --> exec qmail-smtpd [ In reply to ]

malte at tancred

Apr 4, 2017, 7:50 AM

Post #2 of 14 (3568 views)

Permalink

On 4 Apr 2017, at 16:11, Manvendra Bhangui wrote:

> On 2 April 2017 at 20:47, Charles Cazabon
> <search-web-for-address@pyropus.ca> wrote:
...
>> As djb himself says, measure, don't speculate.

> So finally I measured and now I have data, and it so consistent, that
> I too am surprised. Using dlopen() to load the control files in
> tcpserver gives a huge, huge gain if the control files are large.

Nice work.

Just a thought though. It seems to me that control files weren't
designed
to handle huge lists of this sort and that a better approach would be to
move to a more efficient file format or database, say, cdb, sqlite, LMDB
or postgresql.

Regards,
Malte

Re: Benchmarked tcpserver dlopen() -> qmail_smtpd() VS tcpserver --> fork() --> exec qmail-smtpd [ In reply to ]

mbhangui at gmail

Apr 4, 2017, 8:25 AM

Post #3 of 14 (3568 views)

Permalink

On 4 April 2017 at 20:20, Malte Tancred <malte@tancred.com> wrote:
> On 4 Apr 2017, at 16:11, Manvendra Bhangui wrote:
>
>> On 2 April 2017 at 20:47, Charles Cazabon
>> <search-web-for-address@pyropus.ca> wrote:
>
> ...
>>>
>>> As djb himself says, measure, don't speculate.
>
>
>> So finally I measured and now I have data, and it so consistent, that
>> I too am surprised. Using dlopen() to load the control files in
>> tcpserver gives a huge, huge gain if the control files are large.
>
>
> Nice work.
>
> Just a thought though. It seems to me that control files weren't designed
> to handle huge lists of this sort and that a better approach would be to
> move to a more efficient file format or database, say, cdb, sqlite, LMDB
> or postgresql.

Yes. I use mysql. So if a control file is named as badmailfrom.sql,
then qmail-smtpd looks for a table named badmailfrom and reads
badmailfrom.sql for the user, password, database and port/socket.
Advantage of postgresql/mysql is that just one control file like
badmailfrom, etc need to be maintained for all your qmail relay
servers running in your network. I deliberately used a 25 Mib file to
expose the performance issue that can occur in a fork() --> exec
design. My quest is to find an answer for a private namespace for the
loaded shared libraries so that a malicious code be prevented from
modifying tcpserver's global variables or exposing functions with the
same name in tcpserver. The man page for dlmopen suggests that dlmopen
is the answer. However, I have not been able to make dlmopen() work.
tcpserver runs with 100% cpu and even strace doesn't work. Sometimes
technology can be so difficult :(

Re: Benchmarked tcpserver dlopen() -> qmail_smtpd() VS tcpserver --> fork() --> exec qmail-smtpd [ In reply to ]

search-web-for-address at pyropus

Apr 4, 2017, 10:39 AM

Post #4 of 14 (3568 views)

Permalink

Manvendra Bhangui <mbhangui@gmail.com> wrote:
> >
> > I *strongly* urge you to profile the performance of vanilla
> > qmail/tcpserver before you decide to "fix" something and try to make it
> > "look good". I would bet dollars to doughnuts that you will find you are
> > trying to optimize something which is only responsible for 0.01% of the
> > program's work in the first place.
> >
> > As djb himself says, measure, don't speculate.
>
> So finally I measured and now I have data, and it so consistent, that
> I too am surprised. Using dlopen() to load the control files in
> tcpserver gives a huge, huge gain

Unexpected. Have you increased the size of the default buffer used for
reading those files? IIRC it's 64 bytes, and many patches up it to some tens
of K. I don't remember if we did that in netqmail.

> qmail-smtpd. In the extreme case where I have loaded a 25 Mib
> badmailfrom

You're Holding It Wrong.

qmail's control-file handling is absolutely, completely, and totally NOT
designed for multi-megabyte files, and is completely unsuited to be used as
such. Given that, your numbers are completely invalid; you're not using the
correct tool for the job, so it isn't surprising that it doesn't work very
well.

If you need stupidly-large control files, you should be using one of the
patches that replaces the simple in-memory arrays with a CDB lookup. Your
"overhead" loading these control files will disappear.

Charles
--
--------------------------------------------------------------------------
Charles Cazabon
GPL'ed software available at: http://pyropus.ca/software/
Read http://pyropus.ca/personal/writings/12-steps-to-qmail-list-bliss.html
--------------------------------------------------------------------------

Re: Benchmarked tcpserver dlopen() -> qmail_smtpd() VS tcpserver --> fork() --> exec qmail-smtpd [ In reply to ]

search-web-for-address at pyropus

Apr 4, 2017, 10:42 AM

Post #5 of 14 (3568 views)

Permalink

Manvendra Bhangui <mbhangui@gmail.com> wrote:
> badmailfrom, there is a huge difference. In the case of indimail-mta,
> I loaded 50 Mib of control file.

And what was your control-file read buffer size? 64 bytes? That's nearly
800,000 syscalls just to read the bloody file. No wonder you think it's slow.

Charles
--
--------------------------------------------------------------------------
Charles Cazabon
GPL'ed software available at: http://pyropus.ca/software/
Read http://pyropus.ca/personal/writings/12-steps-to-qmail-list-bliss.html
--------------------------------------------------------------------------

Re: Benchmarked tcpserver dlopen() -> qmail_smtpd() VS tcpserver --> fork() --> exec qmail-smtpd [ In reply to ]

mbhangui at gmail

Apr 4, 2017, 11:05 AM

Post #6 of 14 (3568 views)

Permalink

On 4 April 2017 at 23:12, Charles Cazabon
<search-web-for-address@pyropus.ca> wrote:
> Manvendra Bhangui <mbhangui@gmail.com> wrote:
>> badmailfrom, there is a huge difference. In the case of indimail-mta,
>> I loaded 50 Mib of control file.
>
> And what was your control-file read buffer size? 64 bytes? That's nearly
> 800,000 syscalls just to read the bloody file. No wonder you think it's slow.
>

static char inbuf[64];

.....
buffer_init(&ss, read, fd, inbuf, sizeof inbuf);

BTW I don't use such huge files without using cdb or using MySQL. The
control.c which I have already uses .cdb if the control file name has
.cdb extension and MySQL if the file has .sql extension. The point I
am trying to make is that in the dlopen() method, I load the control
file just once in the lifetime - Just once. The crazy 25 MiB control
file was just an experiment. Even in the normal scenario where the
control files are just few KiB, I am seeing a very small performance
gain. I have the code read, working and tested. It is just that I am
struggling to make the tcpserver symbols private. Yesterday I
discovered that one can make the namespace private by using dlmopen()
call instead of dlopen(). However, I have been struggling to make
dlmopen() work with tcpserver. The tcpserver goes into consuming 100%
cpu. With the help of gdb, I have just figured out the reason. The
culprit is prot_uid() and prot_gid() functions in tcpserver. They use
int data type for uid and gid. The moment I call setgid(), setgroups()
or setuid(), these calls do not return and goes into an infinite loop.
It is a situation like errno.h. I need to change the data type for uid
and gid to uid_t and gid_t as they are defined in sys/types.h. If I do
that, tcpserver is working perfectly and securely without its
namespace exposed to other externally loaded library and hence as safe
as execv().

I think this could be a bug with dlmopen() or the kernel that I have
uncovered. Even strace stops working as soon as setuid(), setgid() or
setgroups() gets called, with an error
strace: process runs in x32 mode

This entire exercise has been an academic exercise and an attempt to
learn about how the unix loader works
Cheers :)
and even as I write this, I think I have figured out the

Re: Benchmarked tcpserver dlopen() -> qmail_smtpd() VS tcpserver --> fork() --> exec qmail-smtpd [ In reply to ]

feh at fehcom

Apr 4, 2017, 12:09 PM

Post #7 of 14 (3568 views)

Permalink

Hi Manvendra,

> Am 04.04.2017 um 19:39 schrieb Charles Cazabon <search-web-for-address@pyropus.ca>:
>
> Manvendra Bhangui <mbhangui@gmail.com> wrote:
>>>
>>> I *strongly* urge you to profile the performance of vanilla
>>> qmail/tcpserver before you decide to "fix" something and try to make it
>>> "look good". I would bet dollars to doughnuts that you will find you are
>>> trying to optimize something which is only responsible for 0.01% of the
>>> program's work in the first place.
>>>
>>> As djb himself says, measure, don't speculate.
>>
>> So finally I measured and now I have data, and it so consistent, that
>> I too am surprised. Using dlopen() to load the control files in
>> tcpserver gives a huge, huge gain
>
> Unexpected. Have you increased the size of the default buffer used for
> reading those files? IIRC it's 64 bytes, and many patches up it to some tens
> of K. I don't remember if we did that in netqmail.
>
>> qmail-smtpd. In the extreme case where I have loaded a 25 Mib
>> badmailfrom
>
> You're Holding It Wrong.
>
> qmail's control-file handling is absolutely, completely, and totally NOT
> designed for multi-megabyte files, and is completely unsuited to be used as
> such. Given that, your numbers are completely invalid; you're not using the
> correct tool for the job, so it isn't surprising that it doesn't work very
> well.
>
> If you need stupidly-large control files, you should be using one of the
> patches that replaces the simple in-memory arrays with a CDB lookup. Your
> "overhead" loading these control files will disappear.
>

Absolutely ACK. And -- according to your list of control files -- at least some of them can be replaced by environment variables and can be merged into one control file (badmail*). This is the way Spamcontrol did it and s/qmail does it. See: http://fehcom.de/sqmail/man/qmail-smtpd.html

regards.
--eh.

PS: Apart from that: a 25K badmail file is ridiculous. Something is wrong with your mail architecture.

Dr. Erwin Hoffmann | FEHCom | http://www.fehcom.de | PGP Key-Id: EE00CF65

Re: Benchmarked tcpserver dlopen() -> qmail_smtpd() VS tcpserver --> fork() --> exec qmail-smtpd [ In reply to ]

mbhangui at gmail

Apr 4, 2017, 1:06 PM

Post #8 of 14 (3568 views)

Permalink

On 5 April 2017 at 00:39, Erwin Hoffmann <feh@fehcom.de> wrote:
> Absolutely ACK. And -- according to your list of control files -- at least some of them can be replaced by environment variables and can be merged into one control file (badmail*). This is the way Spamcontrol did it and s/qmail does it.

That's how indimail does it too :). Environment variable BADMAILFROM
and BLACKHOLEDSENDERS. In addition, it looks at the file extension. If
it is .cdb, it uses cdb search. If it has .sql extension, then it
connects to mysql as per database name, tablename, user, socket, etc
stored in badmail.sql file. So no issues here.

> PS: Apart from that: a 25K badmail file is ridiculous. Something is wrong with your mail architecture.
>

No. I don't use a 25 Mib badmail file. That example was to make
apparent the penalty imposed in the current architecture where every
invocation of qmail-smtpd reads this file. I just load the badmail
file in tcpserver and the data is available to each and every child
without the need to again open it. The disadvantage - If the file
changes, I have to send tcpserver SIGHUP. Even for small control
files, I have been able to see small performance improvement by
loading the control files just once. David who replied in an earlier
post suggested using inotify and also suggested using a control file
server using UNIX Domain sockets. One of these days I am gonna use TCP
IP socket. With so many relay servers, one will have to maintain this
control file at just one location and also cache the file locally in
case the control file server is not available. But that's for another
day.

My dlopen() architecture was completed months ago but I did not
succeed in making the tcpserver symbols private to just the tcpserver
executable. If the symbols can be private how can that be less secure
then using exec() ?

So yesterday I discovered dlmopen() which allows you to create a
private namespace but I have been struggling to make it work with
tcpserver. Had a painful session completed few mins ago with gdb. The
moment I use dlmopen(), tcpserver goes at the top, consuming 100% cpu.
gdb showed that it never returned from the function prot_gid() defined
in prot.c. I figured out that due to NPTL, Native Posix Thread
Library, setgroups, setgid, setuid, etc are all wrappers which are
supposed to set the gid, uid of all threads in case of multi-threaded
processes. The setgroups() and setgid() implementation on linux
definitely has a bug. setgroups(), setgid() hang in all cases if
dlmopen has been called. setuid() hangs only if it is passed int
instead of uid_t. The original prot.c uses int instead of uid_t. Since
tcpserver is not a multi-threaded application I have fixed the file
prot.c in the ucsp-tcp-0.88 as following to bypass the NPTL wrappers
in glibc. WIth this change, it is bliss/nirvana for me at the moment
:)

#include "hasshsgr.h"
#include "prot.h"
#include <unistd.h>
#include <grp.h>

#include <syscall.h>
#define SYS_SETGROUPS(NGRUP,MYGIDSET) syscall(SYS_setgroups, NGRUP, MYGIDSET)
#define SYS_SETGID(MYGID) syscall(SYS_setgid, MYGID)
#define SYS_SETUID(MYUID) syscall(SYS_setuid, MYUID)

int
prot_gid(gid_t gid)
{
#ifdef HASSHORTSETGROUPS
short x[2];
x[0] = gid;
x[1] = 73; /*- catch errors */
if (SYS_SETGROUPS(1, x) == -1)
return -1;
#else
if (SYS_SETGROUPS(1, &gid) == -1)
return -1;
#endif
return SYS_SETGID(gid); /*- _should_ be redundant, but on some
systems it isn't */
}

int
prot_uid(uid_t uid)
{
return SYS_SETUID(uid);
}

--
Regards Manvendra - http://www.indimail.org
GPG Pub Key
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC7CBC760014D250C

Re: Benchmarked tcpserver dlopen() -> qmail_smtpd() VS tcpserver --> fork() --> exec qmail-smtpd [ In reply to ]

search-web-for-address at pyropus

Apr 4, 2017, 2:20 PM

Post #9 of 14 (3568 views)

Permalink

Manvendra Bhangui <mbhangui@gmail.com> wrote:
>
> No. I don't use a 25 Mib badmail file. That example was to make
> apparent the penalty imposed in the current architecture where every
> invocation of qmail-smtpd reads this file.

So to summarize: you manufactured a circumstance where qmail's control-file
architecture was a spectacularly bad fit for the artifical circumstance. And
then you used that to justify your development of your feature?

That's fine; inventing and solving weird problems can be fun. It's basically
intellecutal masturbation at that point.

But...

> My dlopen() architecture was completed months ago but I did not
> succeed in making the tcpserver symbols private to just the tcpserver
> executable. If the symbols can be private how can that be less secure
> then using exec() ?

<SFX: KLAXON SOUNDS>

Making the symbols private affects how the linker (static and dynamic) loads
and resolves names/symbols in the object code.

It does *not* make the code or data private. Anyone who can trigger a heap
overflow, stack overflow, use-after-free, or other bug in your code can still
modify data and modify/execute code behind those non-public symbols. You
could easily turn yourself into an open relay, or open your private control
information up to attackers on the net, or leak private information from your
db server to an attacker, etc.

i.e. you appear to have not understood when I said that the fork & exec design
of tcpserver was a security feature.

I *strongly* recommend you not expose this code of yours to the 'net at large
until you have had a competent security review done on it, as you seem not to
understand the security implications of your changes.

Charles
--
--------------------------------------------------------------------------
Charles Cazabon
GPL'ed software available at: http://pyropus.ca/software/
Read http://pyropus.ca/personal/writings/12-steps-to-qmail-list-bliss.html
--------------------------------------------------------------------------

Re: Benchmarked tcpserver dlopen() -> qmail_smtpd() VS tcpserver --> fork() --> exec qmail-smtpd [ In reply to ]

mbhangui at gmail

Apr 4, 2017, 3:02 PM

Post #10 of 14 (3568 views)

Permalink

On 5 April 2017 at 02:50, Charles Cazabon
<search-web-for-address@pyropus.ca> wrote:
> Manvendra Bhangui <mbhangui@gmail.com> wrote:
>>
>> No. I don't use a 25 Mib badmail file. That example was to make
>> apparent the penalty imposed in the current architecture where every
>> invocation of qmail-smtpd reads this file.
>
> So to summarize: you manufactured a circumstance where qmail's control-file
> architecture was a spectacularly bad fit for the artifical circumstance. And
> then you used that to justify your development of your feature?
>
> That's fine; inventing and solving weird problems can be fun. It's basically
> intellecutal masturbation at that point.
>

Yes Charles, you are right about that. It is fun. You may not
remember, but I got into understanding qmail more than 17 years ago. I
knew nothing about qmail and it was your timely help and explanation
about how qmail works, got me into this. I was trying to migrate from
my company's ISOCOR messaging platform. You even gave me a script to
convert from mailbox to Maildir. When I went live, the stock qmail was
just not able to handle the deluge of emails. We had around 3 million
customers. The first thing I did was to put multiple queues (instead
of just one /var/qmail/queue). There is a lot I learned from Russ
Nelson too, but looks like he has disappeared. And I cannot forget
Dave Sills handbook.

> But...
>
>> My dlopen() architecture was completed months ago but I did not
>> succeed in making the tcpserver symbols private to just the tcpserver
>> executable. If the symbols can be private how can that be less secure
>> then using exec() ?
>
> <SFX: KLAXON SOUNDS>
>
> Making the symbols private affects how the linker (static and dynamic) loads
> and resolves names/symbols in the object code.
>
> It does *not* make the code or data private. Anyone who can trigger a heap
> overflow, stack overflow, use-after-free, or other bug in your code can still
> modify data and modify/execute code behind those non-public symbols. You
> could easily turn yourself into an open relay, or open your private control
> information up to attackers on the net, or leak private information from your
> db server to an attacker, etc.
>
> i.e. you appear to have not understood when I said that the fork & exec design
> of tcpserver was a security feature.
>
> I *strongly* recommend you not expose this code of yours to the 'net at large
> until you have had a competent security review done on it, as you seem not to
> understand the security implications of your changes.

That has been the original intention of my first email in this thread.
I update the source code using git and it is public. Anyday more heads
are better than one.

https://sourceforge.net/p/indimail/code/ci/master/tree/ucspi-tcp-0.88/

The relevant files are tcpserver.c, tcpserver_plugin.c, load_shared.c,
dlnamespace.c, rblsmtpd.c. This entire feature can be turned off by
removing conf-dlopen

This was the whole point of my original post and I am again puting it.
Worries
======
1. Maybe I have done something stupid here
eventhough it works for me

2. I don't have the full knowledge of how
the ld loader works and half baked
knowledge is not good.

Re: Benchmarked tcpserver dlopen() -> qmail_smtpd() VS tcpserver --> fork() --> exec qmail-smtpd [ In reply to ]

feh at fehcom

Apr 4, 2017, 3:15 PM

Post #11 of 14 (3568 views)

Permalink

Hi,

> Am 05.04.2017 um 00:02 schrieb Manvendra Bhangui <mbhangui@gmail.com>:
>
> When I went live, the stock qmail was
> just not able to handle the deluge of emails. We had around 3 million
> customers. The first thing I did was to put multiple queues (instead
> of just one /var/qmail/queue).

Yes, the funny 'Silly Qmail Syndrom'. André tried to overcome this; me too with a different approach.

Anyway, the current store&forward paradigm of emails needs to be redesigned. To put everything on disk and to look THEN who is going to potentially receive the message might no be a clever choice (greetings to IM2000 -;)) in our current 'messaging' world.

Regards.
--eh.

Dr. Erwin Hoffmann | FEHCom | http://www.fehcom.de | PGP Key-Id: EE00CF65

Re: Benchmarked tcpserver dlopen() -> qmail_smtpd() VS tcpserver --> fork() --> exec qmail-smtpd [ In reply to ]

mbhangui at gmail

Apr 5, 2017, 1:19 AM

Post #12 of 14 (3556 views)

Permalink

On 5 April 2017 at 03:45, Erwin Hoffmann <feh@fehcom.de> wrote:
> Hi,
>
>
>> Am 05.04.2017 um 00:02 schrieb Manvendra Bhangui <mbhangui@gmail.com>:
>>
>> When I went live, the stock qmail was
>> just not able to handle the deluge of emails. We had around 3 million
>> customers. The first thing I did was to put multiple queues (instead
>> of just one /var/qmail/queue).
>
> Yes, the funny 'Silly Qmail Syndrom'. André tried to overcome this; me too with a different approach.
>
Curious to know your approach. I used a patch (ext-todo by Andreas
Hanssen and big-todo from Russ). In addition I used the current time
in seconds since 1970 and the number of queues as the modulus operator
to push emails in different queues, with each queue having qmail-send
running.

--
Regards Manvendra - http://www.indimail.org
GPG Pub Key
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC7CBC760014D250C

Re: Benchmarked tcpserver dlopen() -> qmail_smtpd() VS tcpserver --> fork() --> exec qmail-smtpd [ In reply to ]

kp at lists

Apr 5, 2017, 5:12 AM

Post #13 of 14 (3555 views)

Permalink

On 2017-04-04 23:20, Charles Cazabon wrote:
>
> That's fine; inventing and solving weird problems can be fun. It's
> basically
> intellecutal masturbation at that point.
>
The m* word has an evil touch at least in german ... perhaps in other
languages not.

Initially I wouldn't jump in here, because I wasn't able to give an
answer to Manvendra's core question. But as nobody did it ... points to
the nature of lists.

I think I know enough about *qmail that I can safely point out: I know
nothing about *qmail! Means I cannot and I will not compare me with the
developers of netqmail and/or patch authors. And my eQmail wouldn't
exist at all if there would be a continuous, coordinated, less
conservative development of netqmail. Fortunately nobody have to wait
for this. eQmail was published because it was looking to me that the
community has a need for. Some more thoughs:

1. Performance

I never have had a performance issue with *qmail. Even when, I would
setup another (high end) server with load balancing. It doesn't make any
sense to me to save a nanosecond (or less) by adding complexity to the
code. The readability of the sources is much more important than
performance. Not to forget that the 'djblibs' are important for *qmail's
performance.

2. Security

Sure, software have to be secure. But it is not a unique feature of
*qmail anymore. Nobody can seriously say that e.g. Postfix is insecure.
So the user has a choice here. On the other side nearly all users have
to **believe** in the security of software. And much more if a software
is written in a very individual style. But the biggest security risk to
me is the (inexperienced, ...) admin always - bad that this is (nearly)
not changeable.

I tried out other MTA's in the past (either OpenSMTPd) and was comming
back to *qmail. Not because of the two points above, but of the general
design. One of *my* rules is to carefully select the software which will
be installed. *qmail fits these, I can easily use (my own) wrappers or
plugins. Btw., I don't use djbdns or daemontools - but that's another
story. I'm not following the main street always.

Anyway, opinions are different. Just my POV.

Kai

--
Sent with eQmail-1.10-beta1

Re: Benchmarked tcpserver dlopen() -> qmail_smtpd() VS tcpserver --> fork() --> exec qmail-smtpd [ In reply to ]

mbhangui at gmail

Apr 5, 2017, 6:28 AM

Post #14 of 14 (3555 views)

Permalink

On 5 April 2017 at 17:42, Kai Peter <kp@lists.openqmail.org> wrote:
> On 2017-04-04 23:20, Charles Cazabon wrote:
>>
>>
>> That's fine; inventing and solving weird problems can be fun. It's
>> basically
>> intellecutal masturbation at that point.
>>
> The m* word has an evil touch at least in german ... perhaps in other
> languages not.
>

I have learned to chill and not take offence. Different countries,
different cultures. In my country there are worse things happening.
Nothing is sacred in this world, Koran, Bible, Gita, qmail, postifx,
or my favourite programming language. Nothing matters more than
knowledge. It is just that over the years I have got tired of
modifying generic code like tcpserver. exec is one way of achieving
the common code in tcpserver and leave out non-generic things to
external code. This external code could be a shell script, could be a
compiled binary and also a shared library. After all in all
probability if you do ldd on your executable you will find few shared
libraries already linked duing the linking stage by the ld. Yes the
original tcpserver doesn't have any external library linked to it. But
then it has also become a source of frustration, especially "NO TLS
?". Seriously? No IPV6, I can still manage.

> Initially I wouldn't jump in here, because I wasn't able to give an answer
> to Manvendra's core question. But as nobody did it ... points to the nature
> of lists.
>
> I think I know enough about *qmail that I can safely point out: I know
> nothing about *qmail! Means I cannot and I will not compare me with the
> developers of netqmail and/or patch authors. And my eQmail wouldn't exist at
> all if there would be a continuous, coordinated, less conservative
> development of netqmail. Fortunately nobody have to wait for this. eQmail
> was published because it was looking to me that the community has a need
> for. Some more thoughs:

We are in the same boat. In 1999 the company I worked for - the
largest private ISP in India, went bust along with the dot-com burst.
We had no money to buy new software and hardware and we were paying 1
USD per mailbox to Critical Path for their ISOCOR platform. Tried
sendmail, qmail and postfix. As someone who lives C programming, fell
in love with DJB's code. Yes it is difficult but very carefully
written. Inspite of this carefully written code, it just failed to
perform. The silly qmail syndrome was a disaster. We lost few customer
the first week, but the management was behind me. Probably didn't
sleep for the whole week but managed to tweak the code to make it
qmail work for me. The biggest problem was the queue. I have enough
examples which tell me that qmail needs to be treated like some holy
grail and prevented from modifications. It is better to make mistakes
and learn on the way rather than living in the fear of not modifying
it and learn nothing.

> 1. Performance
>
> I never have had a performance issue with *qmail. Even when, I would setup
> another (high end) server with load balancing. It doesn't make any sense to
> me to save a nanosecond (or less) by adding complexity to the code. The
> readability of the sources is much more important than performance. Not to
> forget that the 'djblibs' are important for *qmail's performance.

Simplicity and readability goes a long way in getting new programmers
understand it. DJB was thorough with the design. In almost all cases,
you will not require to modify. But I have had enough situations where
the stock qmail or netqmail have not been able to handle, especially
when you don't have money to invest in good servers, fast disks and
large memory. In 2000, my team and me had to support 2.2 million users
on just one Sun Solaris Enterprise 3500 box, which incidentally also
ran a badly written, CPU sucking, webmail code. truss used to be
favourite friend those days. Hardware is much better and much cheaper.
Since 2006, I haven't had a situation where I had to bother about
qmail performance. No one does and it really does not matter to most,
whether they run postfix, sendmail. Since sendmail gets installed by
just typing yum/dnf/apt-get, no one really bothers about qmail. The
truth is that it is orphaned.

> Sent with eQmail-1.10-beta1

Nice signature.

--
Regards Manvendra - http://www.indimail.org
GPG Pub Key
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC7CBC760014D250C

Mailing List Archive

Attached Files:

Attached Files: