Mailing List Archive

s/qmail 4.1 is out
Hi qmailer,

I finished and published s/qmail 4.1. In case you are interested in a modern Qmail system, you might wont to consider it.

https://www.fehcom.de/sqmail/sqmail.html

Here, you find the new enhancements documented and the downloadable package(s) - fehQlibs-17 are required prior of installation.

Greylisting for qmail-smtpd and TLSA/DANE lookup and automatic X.509 certificate validation for qmail-remote is now included.
DKIM is still on my todo list, but scheduled for the forthcoming release.

s/qmail 4.1 is known to work well on all current Linux OS, OpenBSD, FreeBSD (of course) and is 64 bit clean while supporting the ARM architecture.

Enjoy!

regards.
--eh.


Dr. Erwin Hoffmann | FEHCom | http://www.fehcom.de | PGP Key-Id 7E4034BE
Re: s/qmail 4.1 is out [ In reply to ]
On Sat, 3 Apr 2021 at 18:52, Erwin Hoffmann <feh@fehcom.de> wrote:
>
> Hi qmailer,
>
> I finished and published s/qmail 4.1. In case you are interested in a modern Qmail system, you might wont to consider it.
>
> https://www.fehcom.de/sqmail/sqmail.html
>
> Here, you find the new enhancements documented and the downloadable package(s) - fehQlibs-17 are required prior of installation.
>
> Greylisting for qmail-smtpd and TLSA/DANE lookup and automatic X.509 certificate validation for qmail-remote is now included.
> DKIM is still on my todo list, but scheduled for the forthcoming release.

TLSA/DANE lookup is a welcome feature. Another feature caught my attention.
- Distributed queueing: n:1, 1:n n:m with qualified authentication and
authorization (enhanced 'QMQ').

What exactly is distributed queuing and qmail multiple queues?






--
Regards Manvendra - http://www.indimail.org
GPG Pub Key
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC7CBC760014D250C
Re: s/qmail 4.1 is out [ In reply to ]
Hi Manvendra,

I see, you are busy with Notqmail ....


> Am 03.04.2021 um 19:04 schrieb Manvendra Bhangui <mbhangui@gmail.com>:
>
> On Sat, 3 Apr 2021 at 18:52, Erwin Hoffmann <feh@fehcom.de> wrote:
>>
>> Hi qmailer,
>>
>> I finished and published s/qmail 4.1. In case you are interested in a modern Qmail system, you might wont to consider it.
>>
>> https://www.fehcom.de/sqmail/sqmail.html
>>
>> Here, you find the new enhancements documented and the downloadable package(s) - fehQlibs-17 are required prior of installation.
>>
>> Greylisting for qmail-smtpd and TLSA/DANE lookup and automatic X.509 certificate validation for qmail-remote is now included.
>> DKIM is still on my todo list, but scheduled for the forthcoming release.
>
> TLSA/DANE lookup is a welcome feature. Another feature caught my attention.
> - Distributed queueing: n:1, 1:n n:m with qualified authentication and
> authorization (enhanced 'QMQ').
>
> What exactly is distributed queuing and qmail multiple queues?
>

When I was in charge for a Germany's federal state central email service (as consultant and developer; not admin), we faced the following situation:

a) A central email hub (based on qmail + spamcontrol).
b) Inbound and outbound mail service including (of course spam and virus scanning).
c) Maybe it was 4 core CPU (Pentium) with 32 bit CentOS (and gcc 4.7.2 driving my crazy; see remarks on my web site).
d) Recipient extension was working while using my qmail-smtpam + ldapam.pl script against the ministries (mostly Exchange) servers.

Back now 14 year or so, spam storms were daily business, the MTA had to deal > 100 000 mails/day (could be 500 000).

Inband bandwidth was plenty, but outband was restricted; many bounces as well (sitting in the queue and can't be delivered).

Thus, my idea was to decouple the entire thing, since the bottleneck was pre-processing and a huge number of messages in the queue (even with bigtodo; ext-todo was not implemented yet).

This gave rise to the following:

a) Developing the idea of a 'bounce queue' and let qmail recognize bounces (part of standard s/qmail now and since Spamcontrol 2.4).
b) Setting up several 'mini-qmail' systems (on the same host) taking individually care about delivery to the ministries (with a different rule set).
c) Transmissions between the different qmail MTA instances is given by the QMQP.
d) Rather using QMTP instead of QMQP, TLS encryption and authentication is possible (you can see it in the header of this email; mailing lists may however filter it).

I've called that 'QMQ'. Since these days, Spamcontrol and later s/qmail includes a script to deploy additional instances easy and on the fly.

Given today's approach, it is like a provisioning customized Docker images and starting those; though without any 'jail-ing' and VMs.

This actually worked extraordinary well: Mailstorms could be managed while parking mails on (in-active) instances and keeping the standard delivery channels free, while sweeping away the unwanted malls.

In short: QMQ is not a 'technique' but rather an administrative tool.

Sidenote:

Well, there exists a patch for Qmail to provide additional 'delivery channels'. Right now Qmail and s/qmail has two: remote and local (not considering bounces). In the forthcoming releases of s/qmail, I will use this concept in a smarter way to cope better with today's privacy concerns.


Yeah, I know, i need to provide a full description of QMQ; the documentation web site is still empty. Probably, I will use this explanation for a first start.

https://www.fehcom.de/sqmail/sqmaildoc.html


Anybody is free to use these ideas even for let's say Notqmail. However, trying to port DANE/TLSA to standard qmail would be a challenge. It took me some time to adjust DJB's DNS routines to be working with s/qmail and to benefit from the solution.


My next project is to finish djbdnscurve6-37 and then go for CurveDNS support in tinydns. tinydns right now provides already support for TLSA records.

Regards.
--eh.










>
>
>
>
>
> --
> Regards Manvendra - http://www.indimail.org
> GPG Pub Key
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC7CBC760014D250C

Dr. Erwin Hoffmann | FEHCom | http://www.fehcom.de | PGP Key-Id 7E4034BE
Re: s/qmail 4.1 is out [ In reply to ]
On Sun, 4 Apr 2021 at 01:06, Erwin Hoffmann <feh@fehcom.de> wrote:
> > TLSA/DANE lookup is a welcome feature. Another feature caught my attention.
> > - Distributed queueing: n:1, 1:n n:m with qualified authentication and
> > authorization (enhanced 'QMQ').
> >
> > What exactly is distributed queuing and qmail multiple queues?
> >
>
> When I was in charge for a Germany's federal state central email service (as consultant and developer; not admin), we faced the following situation:
>
> a) A central email hub (based on qmail + spamcontrol).
> b) Inbound and outbound mail service including (of course spam and virus scanning).
> c) Maybe it was 4 core CPU (Pentium) with 32 bit CentOS (and gcc 4.7.2 driving my crazy; see remarks on my web site).
> d) Recipient extension was working while using my qmail-smtpam + ldapam.pl script against the ministries (mostly Exchange) servers.
>
> Back now 14 year or so, spam storms were daily business, the MTA had to deal > 100 000 mails/day (could be 500 000).
>
> Inband bandwidth was plenty, but outband was restricted; many bounces as well (sitting in the queue and can't be delivered).
>
> Thus, my idea was to decouple the entire thing, since the bottleneck was pre-processing and a huge number of messages in the queue (even with bigtodo; ext-todo was not implemented yet).
>
> This gave rise to the following:
>
> a) Developing the idea of a 'bounce queue' and let qmail recognize bounces (part of standard s/qmail now and since Spamcontrol 2.4).
> b) Setting up several 'mini-qmail' systems (on the same host) taking individually care about delivery to the ministries (with a different rule set).
> c) Transmissions between the different qmail MTA instances is given by the QMQP.
> d) Rather using QMTP instead of QMQP, TLS encryption and authentication is possible (you can see it in the header of this email; mailing lists may however filter it).
>
> I've called that 'QMQ'. Since these days, Spamcontrol and later s/qmail includes a script to deploy additional instances easy and on the fly.
>

Thank you for taking the time to reply. It seems both of us faced the
same problem years ago with qmail - dealing with high injection
rates.I had exactly the same problem in 2000. I did it bit differently
in indimail. decoupled qmail-send, qmail-lspawn, qmail-rspawn from
/var/qmail/queue. A script would create multiple queues (queue1,
queue2, queue3 and so on). The rc script was modified to start
multiple instances of qmail-send, qmail-lspawn, qmail-rspawn. Along
with multiple inbound servers that would just hold the queue, the high
injection rates were tamed. Those days I didn't try out QMTP, QMQP
which would have been slightly better. The QMAILQUEUE patch was
exploited to distribute incoming mails evenly amongst the multiple
queues.

> Given today's approach, it is like a provisioning customized Docker images and starting those; though without any 'jail-ing' and VMs.
>
> This actually worked extraordinary well: Mailstorms could be managed while parking mails on (in-active) instances and keeping the standard delivery channels free, while sweeping away the unwanted malls.
>
> In short: QMQ is not a 'technique' but rather an administrative tool.

Almost all email administrators ultimately end up writing scripts that
solve day to day problems :)

>
> Sidenote:
>
> Well, there exists a patch for Qmail to provide additional 'delivery channels'. Right now Qmail and s/qmail has two: remote and local (not considering bounces). In the forthcoming releases of s/qmail, I will use this concept in a smarter way to cope better with today's privacy concerns.

Another interesting idea. I too will give this a thought.

>
> Anybody is free to use these ideas even for let's say Notqmail. However, trying to port DANE/TLSA to standard qmail would be a challenge. It took me some time to adjust DJB's DNS routines to be working with s/qmail and to benefit from the solution.
>
>
> My next project is to finish djbdnscurve6-37 and then go for CurveDNS support in tinydns. tinydns right now provides already support for TLSA records.
>

Thank you and all the best for djbdnscurve / tinydns
Re: s/qmail 4.1 is out [ In reply to ]
Hi Manvendra,

(it should be very late in India ...)

we are both qmail veterans; working on 25ys old software ...

Interesting to see, that things can still be kept alive, though the qmail users tend to diminish in favour of postfix and exim.

But it is always good to have same alternatives - and the true choice to use them!

I'll try to keep s/qmail competitive with the other MTA; however only having the time to follow the mainstream. There are a lot of experiments going one, which would be interesting to follow.

Hm. Maybe I should put s/qmail (aka aqmail) on Github or Gitlab as collaboration edition and let people use this as base for enhancements. Need to talk to Kai.

Regards.
--eh.


> Am 03.04.2021 um 23:05 schrieb Manvendra Bhangui <mbhangui@gmail.com>:
>
> On Sun, 4 Apr 2021 at 01:06, Erwin Hoffmann <feh@fehcom.de> wrote:
>>> TLSA/DANE lookup is a welcome feature. Another feature caught my attention.
>>> - Distributed queueing: n:1, 1:n n:m with qualified authentication and
>>> authorization (enhanced 'QMQ').
>>>
>>> What exactly is distributed queuing and qmail multiple queues?
>>>
>>
>> When I was in charge for a Germany's federal state central email service (as consultant and developer; not admin), we faced the following situation:
>>
>> a) A central email hub (based on qmail + spamcontrol).
>> b) Inbound and outbound mail service including (of course spam and virus scanning).
>> c) Maybe it was 4 core CPU (Pentium) with 32 bit CentOS (and gcc 4.7.2 driving my crazy; see remarks on my web site).
>> d) Recipient extension was working while using my qmail-smtpam + ldapam.pl script against the ministries (mostly Exchange) servers.
>>
>> Back now 14 year or so, spam storms were daily business, the MTA had to deal > 100 000 mails/day (could be 500 000).
>>
>> Inband bandwidth was plenty, but outband was restricted; many bounces as well (sitting in the queue and can't be delivered).
>>
>> Thus, my idea was to decouple the entire thing, since the bottleneck was pre-processing and a huge number of messages in the queue (even with bigtodo; ext-todo was not implemented yet).
>>
>> This gave rise to the following:
>>
>> a) Developing the idea of a 'bounce queue' and let qmail recognize bounces (part of standard s/qmail now and since Spamcontrol 2.4).
>> b) Setting up several 'mini-qmail' systems (on the same host) taking individually care about delivery to the ministries (with a different rule set).
>> c) Transmissions between the different qmail MTA instances is given by the QMQP.
>> d) Rather using QMTP instead of QMQP, TLS encryption and authentication is possible (you can see it in the header of this email; mailing lists may however filter it).
>>
>> I've called that 'QMQ'. Since these days, Spamcontrol and later s/qmail includes a script to deploy additional instances easy and on the fly.
>>
>
> Thank you for taking the time to reply. It seems both of us faced the
> same problem years ago with qmail - dealing with high injection
> rates.I had exactly the same problem in 2000. I did it bit differently
> in indimail. decoupled qmail-send, qmail-lspawn, qmail-rspawn from
> /var/qmail/queue. A script would create multiple queues (queue1,
> queue2, queue3 and so on). The rc script was modified to start
> multiple instances of qmail-send, qmail-lspawn, qmail-rspawn. Along
> with multiple inbound servers that would just hold the queue, the high
> injection rates were tamed. Those days I didn't try out QMTP, QMQP
> which would have been slightly better. The QMAILQUEUE patch was
> exploited to distribute incoming mails evenly amongst the multiple
> queues.
>
>> Given today's approach, it is like a provisioning customized Docker images and starting those; though without any 'jail-ing' and VMs.
>>
>> This actually worked extraordinary well: Mailstorms could be managed while parking mails on (in-active) instances and keeping the standard delivery channels free, while sweeping away the unwanted malls.
>>
>> In short: QMQ is not a 'technique' but rather an administrative tool.
>
> Almost all email administrators ultimately end up writing scripts that
> solve day to day problems :)
>
>>
>> Sidenote:
>>
>> Well, there exists a patch for Qmail to provide additional 'delivery channels'. Right now Qmail and s/qmail has two: remote and local (not considering bounces). In the forthcoming releases of s/qmail, I will use this concept in a smarter way to cope better with today's privacy concerns.
>
> Another interesting idea. I too will give this a thought.
>
>>
>> Anybody is free to use these ideas even for let's say Notqmail. However, trying to port DANE/TLSA to standard qmail would be a challenge. It took me some time to adjust DJB's DNS routines to be working with s/qmail and to benefit from the solution.
>>
>>
>> My next project is to finish djbdnscurve6-37 and then go for CurveDNS support in tinydns. tinydns right now provides already support for TLSA records.
>>
>
> Thank you and all the best for djbdnscurve / tinydns

Dr. Erwin Hoffmann | FEHCom | http://www.fehcom.de | PGP Key-Id 7E4034BE