Mailing List Archive: Identifying used patches

Identifying used patches

Mar 25, 2016, 7:26 PM

Post #1 of 12 (4391 views)

Hi all.

This is a bit unusual, I know. (Feel free to skip to "The actual issue").

Background:
Several years ago (in 2011), I made my "final" qmail installation.
I put up my recipe on my dedicated qmail Web-site, so I thought everything would be good.
Around 4 months later, an error in the HFS+ format caused my Web-directory to instantly disappear in probably less than a nanosecond.
All my Web-sites were lost.
The remaining part of the server was intact, though, so my servers kept running.
I managed to power down the server and make a DD backup of the remains.

I had some backups of mosts sites, but I had to purchase Data Rescue 3, so I could recover some of my latest Web-site work; I think I got around 50% restored (some of the sites were improved because of this event, though).
I also invested in a backup-utility and now run a daily rsync backup, so I've learned a few lessons. ;)

The actual issue:
Today, 4 years later, I subscribed to the debian-user mailing list, but my qmail could not send my posts to the lists.
It seems that my qmail does not have the "ANY DNS" patch, so I tried rebuilding my qmail, just to find out that the sources I've kept are not the sources that my currently running qmail was built from.

I get the following error:
"SMTP Protocol Error: 553 sorry, that domain isn't in my list of allowed rcpthosts; no valid cert for gatewaying (#5.7.1)"

After spending 14 days of trying to reconstruct the qmail using various patches, I think I need a little help identifying how I built the last one.

My setup:
I am denying any authorization/authentication on port 25.
I require secure authorization/authentication on port 587.

I have applied my own extension to RCPTCHECK, don't let this confuse you (I added extra environment variables).

If I telnet into port 587 and type 'ehlo testing', I get the following response:
---8<-----8<-----8<-----
250-example.com
250-STARTTLS
250-PIPELINING
250-8BITMIME
250-SIZE 0
250 AUTH LOGIN PLAIN
--->8----->8----->8-----

Notice that there's no "AUTH=LOGIN PLAIN"

My environment variables for starting qmail-smtpd on port 25 are as follows:

---8<-----8<-----8<-----
ALLOW_INSECURE_AUTH=0
AUTH=0
DENY_TLS=1
FORCETLS=0
RCPTCHECK=/var/qmail/bin/rcptchk
REQUIRE_AUTH=0
SSL=0
TCPLOCALHOST=mail.example.com
TCPLOCALIP=10.0.1.0
TCPLOCALPORT=25
--->8----->8----->8-----

... and for port 587 ...

---8<-----8<-----8<-----
ALLOW_INSECURE_AUTH=0
AUTH=1
DENY_TLS=0
FORCETLS=1
REQUIRE_AUTH=1
SSL=0
TCPLOCALHOST=mail.example.com
TCPLOCALIP=10.0.1.0
TCPLOCALPORT=587
--->8----->8----->8-----

Notice in the environment variables that I do not use the patches that require me to set "SMTPAUTH=!" either.

The build-date is Dec. 7, 2011, which means the patches I used must be before that date.

Those alone should actually provide great info.

My actual question is:
Which patch(es) use "ALLOW_INSECURE_AUTH", "DENY_TLS" and "REQUIRE_AUTH" ?

Normally, I would only apply a minimum set of patches, which means I'm not likely to use a "combined patch".
Is it *only* the patches from John M. Simpson, or are there others too ?
-A lot of arrows point in that direction, which means that I might have stopped rolling my own recipe and used John's patches with my own RCPTCHECK added (I would expect that John's patch would include the "ANY DNS" patch, though).

Additional notes:
I moved from using my own password verification to using vpopmail, so I could easily use Dovecot and Squirrelmail.

Love
Jens

Re: Identifying used patches [ In reply to ]

ed-qmail at s5h

Mar 26, 2016, 1:52 AM

Post #2 of 12 (4330 views)

Permalink

On Sat, Mar 26, 2016 at 03:26:57AM +0100, Jens Bauer wrote:
> "SMTP Protocol Error: 553 sorry, that domain isn't in my list of
> allowed rcpthosts; no valid cert for gatewaying (#5.7.1)"

> I am denying any authorization/authentication on port 25.
> I require secure authorization/authentication on port 587.

> My actual question is:
> Which patch(es) use "ALLOW_INSECURE_AUTH", "DENY_TLS" and "REQUIRE_AUTH" ?
>
> Normally, I would only apply a minimum set of patches, which means I'm not likely to use a "combined patch".
> Is it *only* the patches from John M. Simpson, or are there others too ?
> -A lot of arrows point in that direction, which means that I might have stopped rolling my own recipe and used John's patches with my own RCPTCHECK added (I would expect that John's patch would include the "ANY DNS" patch, though).

From what you've said above, I would suggest you get the latest (7.10)
combined patch[1] and install from his instructions using the various
run scripts[2].

1: <https://qmail.jms1.net/patches/combined-details.shtml>
2: <https://qmail.jms1.net/scripts/>

Surprisingly (or not so) that combined set has served me well. The only
change I've added is some TLS options to prevent SSL2[3].

3: <https://www.usenix.org.uk/content/qmail.html>

Using John's patches will at least get you a good stable system with
minimal effort.

--
Best regards,
Ed http://www.s5h.net/

Re: Identifying used patches [ In reply to ]

jens-lists at gpio

Mar 26, 2016, 3:06 AM

Post #3 of 12 (4330 views)

Permalink

Hi Ed.

Thank you for the quick answer and suggestions.

Note: I am already running qmail; I still have the binaries running, but I need to make it work with debian.org.
I do not have the recipe for re-building the exact same version of qmail that I have currently running.

Can you confirm that John's patch is the *only* patch supporting DENY_TLS and ALLOW_INSECURE_AUTH and REQUIRE_AUTH ?

If 7.10 is the one I already have installed, then it will not help me installing it again, because it already has the ANY-to-CNAME patch.
-So if that's what I have installed already, then the problem is elsewhere.

Anyone wanting to test if their qmail works with debian.org and lists.debian.org could try writing a small test e-mail to the debian-user list-bot:

To: List Bot <debian-user-request@lists.debian.org>
Subject: Hello
Nice weather. I'm just testing my MTA.

(I get a response if I send the message via telnet, but if I send it through qmail-smtpd, it never leaves my server).
If noone can get a reply from the above mentioned list-bot, I think we might have a bug to find and fix.

Love
Jens

On Sat, 26 Mar 2016 08:52:07 +0000, ed wrote:
> On Sat, Mar 26, 2016 at 03:26:57AM +0100, Jens Bauer wrote:
>> "SMTP Protocol Error: 553 sorry, that domain isn't in my list of
>> allowed rcpthosts; no valid cert for gatewaying (#5.7.1)"
>
>> I am denying any authorization/authentication on port 25.
>> I require secure authorization/authentication on port 587.
>
>> My actual question is:
>> Which patch(es) use "ALLOW_INSECURE_AUTH", "DENY_TLS" and "REQUIRE_AUTH" ?
>>
>> Normally, I would only apply a minimum set of patches, which means
>> I'm not likely to use a "combined patch".
>> Is it *only* the patches from John M. Simpson, or are there others too ?
>> -A lot of arrows point in that direction, which means that I might
>> have stopped rolling my own recipe and used John's patches with my
>> own RCPTCHECK added (I would expect that John's patch would include
>> the "ANY DNS" patch, though).
>
> From what you've said above, I would suggest you get the latest (7.10)
> combined patch[1] and install from his instructions using the various
> run scripts[2].
>
> 1: <https://qmail.jms1.net/patches/combined-details.shtml>
> 2: <https://qmail.jms1.net/scripts/>
>
> Surprisingly (or not so) that combined set has served me well. The only
> change I've added is some TLS options to prevent SSL2[3].
>
> 3: <https://www.usenix.org.uk/content/qmail.html>
>
> Using John's patches will at least get you a good stable system with
> minimal effort.
>
> --
> Best regards,
> Ed http://www.s5h.net/
>

Re: Identifying used patches [ In reply to ]

feh at fehcom

Mar 26, 2016, 4:34 AM

Post #4 of 12 (4327 views)

Permalink

Hi Jens,

you may have a look at my qmail follow-up s/qmail (http://fehcom.de/sqmail.html).

It does the trick as well (among others).

regards.
--eh.

> Am 26.03.2016 um 11:06 schrieb Jens Bauer <jens-lists@gpio.dk>:
>
> Hi Ed.
>
> Thank you for the quick answer and suggestions.
>
> Note: I am already running qmail; I still have the binaries running, but I need to make it work with debian.org.
> I do not have the recipe for re-building the exact same version of qmail that I have currently running.
>
> Can you confirm that John's patch is the *only* patch supporting DENY_TLS and ALLOW_INSECURE_AUTH and REQUIRE_AUTH ?
>
> If 7.10 is the one I already have installed, then it will not help me installing it again, because it already has the ANY-to-CNAME patch.
> -So if that's what I have installed already, then the problem is elsewhere.
>
> Anyone wanting to test if their qmail works with debian.org and lists.debian.org could try writing a small test e-mail to the debian-user list-bot:
>
> To: List Bot <debian-user-request@lists.debian.org>
> Subject: Hello
> Nice weather. I'm just testing my MTA.
>
> (I get a response if I send the message via telnet, but if I send it through qmail-smtpd, it never leaves my server).
> If noone can get a reply from the above mentioned list-bot, I think we might have a bug to find and fix.
>
>
> Love
> Jens
>
> On Sat, 26 Mar 2016 08:52:07 +0000, ed wrote:
>> On Sat, Mar 26, 2016 at 03:26:57AM +0100, Jens Bauer wrote:
>>> "SMTP Protocol Error: 553 sorry, that domain isn't in my list of
>>> allowed rcpthosts; no valid cert for gatewaying (#5.7.1)"
>>
>>> I am denying any authorization/authentication on port 25.
>>> I require secure authorization/authentication on port 587.
>>
>>> My actual question is:
>>> Which patch(es) use "ALLOW_INSECURE_AUTH", "DENY_TLS" and "REQUIRE_AUTH" ?
>>>
>>> Normally, I would only apply a minimum set of patches, which means
>>> I'm not likely to use a "combined patch".
>>> Is it *only* the patches from John M. Simpson, or are there others too ?
>>> -A lot of arrows point in that direction, which means that I might
>>> have stopped rolling my own recipe and used John's patches with my
>>> own RCPTCHECK added (I would expect that John's patch would include
>>> the "ANY DNS" patch, though).
>>
>> From what you've said above, I would suggest you get the latest (7.10)
>> combined patch[1] and install from his instructions using the various
>> run scripts[2].
>>
>> 1: <https://qmail.jms1.net/patches/combined-details.shtml>
>> 2: <https://qmail.jms1.net/scripts/>
>>
>> Surprisingly (or not so) that combined set has served me well. The only
>> change I've added is some TLS options to prevent SSL2[3].
>>
>> 3: <https://www.usenix.org.uk/content/qmail.html>
>>
>> Using John's patches will at least get you a good stable system with
>> minimal effort.
>>
>> --
>> Best regards,
>> Ed http://www.s5h.net/
>>
>
Dr. Erwin Hoffmann | FEHCom | http://www.fehcom.de | PGP Key-Id: EE00CF65

Re: Identifying used patches [ In reply to ]

ed-qmail at s5h

Mar 26, 2016, 7:37 AM

Post #5 of 12 (4326 views)

Permalink

On Sat, Mar 26, 2016 at 11:06:14AM +0100, Jens Bauer wrote:
> Hi Ed.
>
> Thank you for the quick answer and suggestions.
>
> Note: I am already running qmail; I still have the binaries running, but I need to make it work with debian.org.
> I do not have the recipe for re-building the exact same version of qmail that I have currently running.
>
> Can you confirm that John's patch is the *only* patch supporting DENY_TLS and ALLOW_INSECURE_AUTH and REQUIRE_AUTH ?

JMS combines other patches and makes some improvements, so chances are
that patch exists outside of the combined patch.

Sent test mail to debian mail server without issue. Though, I think
they greylist so may take a few minutes.

--
Best regards,
Ed http://www.s5h.net/

Re: Identifying used patches [ In reply to ]

jens-lists at gpio

Mar 26, 2016, 8:10 AM

Post #6 of 12 (4326 views)

Permalink

Hi Erwin and Ed.

Believe me, I've been thinking about this more than once. ;)
-But I need to make sure that I stay compatible with my current installation.
I'm pretty sure that I will switch, when I make a fresh install on my new Linux server, though (I've prepared myself already and subscribed to s/qmail, as you probably already know).

I've now discovered that I am not using John M. Simpsons's patch. (Fortunately for everyone else; because that means his version will most likely work with debian.org and other domains).

As I found the string 'netqmail' in the binary, this is also an indication that I'm using another version.
I made a very neat 2-liner for this:
---8<-----8<-----8<----- [ file: ~/bin/bcat ]
#!/bin/bash
sed -E -e $'s/[^\40-\176]+/\xc2\xb7/g' $@
--->8----->8----->8-----
(You can change \xc2\xb7 to a single space if you wish; but the above allows you to clearly see where the real spaces are).

I've examined the strings in the qmail-smtpd binary and a number of those environment variables I supply do not exist in the binary.
That means I only made my launch-control file compatible with John's patch.

Ed's tests also confirms that I'm not on a JMS version.

So all that's left is that I now have to do a lot of hard work. ;)
Thank you John, Ed and Erwin for the pointers and hints; I think I'm on the right track now.

Note for everyone: I was just attacked by Dmitry Kaplanov at 185.103.253.223.
He apparantly has a bot that go through all your domains on your server in order to guess the passwords for the following usernames:
'admin@domain.tld',
'test@domain.tld',
'webmaster@domain.tld',
'info@domain.tld',
'administrator@domain.tld',
'postmaster@domain.tld'
'domain@domain.tld'
-He's trying that on port 587, but he stops attacking, when you flood his IP (yez, I know - it wasn't nice of me to do that).
(it might be a good idea to set up an auto-block trap on those account names - or on any accounts that do not exist on your server).

Love
Jens

On Sat, 26 Mar 2016 12:34:09 +0100, Erwin Hoffmann wrote:
> Hi Jens,
>
> you may have a look at my qmail follow-up s/qmail
> (http://fehcom.de/sqmail.html).
>
> It does the trick as well (among others).
>
> regards.
> --eh.

Re: Identifying used patches [ In reply to ]

amb-sendok-1461606275.kbmjaihplgnnkohnlgei at bradfo

Mar 26, 2016, 10:44 AM

Post #7 of 12 (4311 views)

Permalink

Thus said Jens Bauer on Sat, 26 Mar 2016 11:06:14 +0100:

> (I get a response if I send the message via telnet, but if I send it
> through qmail-smtpd, it never leaves my server).

What do your logs say?

When I send an email to debian-user-request@lists.debian.org I see the
following in my logs:

2016-03-26 11:35:20.319739500 delivery 21223: deferral: 82.195.75.100_does_not_like_recipient./Remote_host_said:_450_4.2.0_<debian-user-request@lists.debian.org>:_Recipient_address_rejected:_Greylisted,_see_http://postgrey.schweikert.ch/help/lists.debian.org.html/Giving_up_on_82.195.75.100./

Which tells me that they are using some kind of grey listing.
Eventually, this email should be delivered when qmail retries delivery.
I'm confused though, how did you pass the grey listing test using just
telnet? Did you try more than once? Did you see the above temporary
failure error with telnet?

My email does eventually make it:

2016-03-26 11:41:52.917416500 delivery 21224: success: 82.195.75.100_accepted_message./Remote_host_said:_250_2.0.0_Ok:_queued_as_6F22F0F/

And I have a mail in my inbox:

Subject: Re: Hello
From: debian-user-request@lists.debian.org
Date: Sat, 26 Mar 2016 17:41:58 +0000 (UTC)

If there is a bug, it's not in my qmail (mostly stock qmail with one or
two patches at most).

Andy
--
TAI64 timestamp: 4000000056f6caa6

Re: Identifying used patches [ In reply to ]

jens-lists at gpio

Mar 26, 2016, 11:18 AM

Post #8 of 12 (4311 views)

Permalink

Hi Andy.

Good news: I'm quite positive that I've finally identified the patch. :)
It seems I'm using Roberto's patch from 2011-10-06 along with my own RCPTCHECK extension.
As far as I can tell, I did not apply anything else.

Looking at Roberto's ChangeLog, I can see that the ANY-to-CNAME patch was applied in 2013 (more than one year after I built the binary I'm running currently).
That means it is most likely because I need to apply the ANY-to-CNAME, that I have the problem.

I can build a binary, which is 4 bytes smaller than the one which is running, but gcc seems to make a few things upside down (for some reason, it rearranges the strings in bottom-top order instead of top-bottom), so I may be unable to build an exact matching binary.

I've also found one of my newest build-scripts, which indicates that I haven't applied the ANY-to-CNAME DNS-patch.

On 26 Mar 2016 11:44:34 -0600, Andy Bradford wrote:
> Thus said Jens Bauer on Sat, 26 Mar 2016 11:06:14 +0100:
>
>> (I get a response if I send the message via telnet, but if I send it
>> through qmail-smtpd, it never leaves my server).
>
> What do your logs say?

Ehm ... nada. Silence, quiet, nothing.
(I would expect that it's system.log I should expect any messages in; I have '/var/log/mail.log' and '/var/log/system.log')
I know it's queued, though, because at some point I rebuilt qmail, and it sent the entire queue, but that version didn't allow me to use TLS.

> When I send an email to debian-user-request@lists.debian.org I see the
> following in my logs:
>
> 2016-03-26 11:35:20.319739500 delivery 21223: deferral:
> 82.195.75.100_does_not_like_recipient./Remote_host_said:_450_4.2.0_<debian-user-request@lists.debian.org>:_Recipient_address_rejected:_Greylisted,_see_http://postgrey.schweikert.ch/help/lists.debian.org.html/Giving_up_on_82.195.75.100./
>
> Which tells me that they are using some kind of grey listing.

I think my qmail-smtpd does not even get there. It's probably stuck in the DNS-lookup.

> Eventually, this email should be delivered when qmail retries delivery.
> I'm confused though, how did you pass the grey listing test using just
> telnet? Did you try more than once? Did you see the above temporary
> failure error with telnet?

Uhm, I didn't experience any problems; I can't explain why. :/
-Perhaps the server white-listed me after I subscribed the first time (which was before I sent a message using telnet).

> My email does eventually make it:
>
> 2016-03-26 11:41:52.917416500 delivery 21224: success:
> 82.195.75.100_accepted_message./Remote_host_said:_250_2.0.0_Ok:_queued_as_6F22F0F/
>
> And I have a mail in my inbox:
>
> Subject: Re: Hello
> From: debian-user-request@lists.debian.org
> Date: Sat, 26 Mar 2016 17:41:58 +0000 (UTC)

It's great to see that most qmail users do not have the problem. Thanks for confirming that this works. :)

Love
Jens

Re: Identifying used patches [ In reply to ]

kristen at atmyhome

Mar 26, 2016, 3:15 PM

Post #9 of 12 (4306 views)

Permalink

On 3/26/16 12:52 AM, ed wrote:
> On Sat, Mar 26, 2016 at 03:26:57AM +0100, Jens Bauer wrote:
>> "SMTP Protocol Error: 553 sorry, that domain isn't in my list of
>> allowed rcpthosts; no valid cert for gatewaying (#5.7.1)"
>
>> I am denying any authorization/authentication on port 25.
>> I require secure authorization/authentication on port 587.
>
>> My actual question is:
>> Which patch(es) use "ALLOW_INSECURE_AUTH", "DENY_TLS" and "REQUIRE_AUTH" ?
>>
>> Normally, I would only apply a minimum set of patches, which means I'm not likely to use a "combined patch".
>> Is it *only* the patches from John M. Simpson, or are there others too ?
>> -A lot of arrows point in that direction, which means that I might have stopped rolling my own recipe and used John's patches with my own RCPTCHECK added (I would expect that John's patch would include the "ANY DNS" patch, though).
>
> From what you've said above, I would suggest you get the latest (7.10)
> combined patch[1] and install from his instructions using the various
> run scripts[2].
>
> 1: <https://qmail.jms1.net/patches/combined-details.shtml>
> 2: <https://qmail.jms1.net/scripts/>
>
> Surprisingly (or not so) that combined set has served me well. The only
> change I've added is some TLS options to prevent SSL2[3].
>
> 3: <https://www.usenix.org.uk/content/qmail.html>
>
> Using John's patches will at least get you a good stable system with
> minimal effort.
>

Jens,

I will voice my support of Jim Simpson's patches. I use these alone and
I have an almost spam free server. The system is solid and is easy to build.

I know you want to repair your system first. I did send an email to the
debian list as requested. First it was greylisted, then accepted. I am
on some debian mail list (Debian-PPC, Debian-Women) but not the one you
are subscribe to.

Best of luck

Kristen

Re: Identifying used patches [ In reply to ]

jens-lists at gpio

Mar 26, 2016, 9:29 PM

Post #10 of 12 (4304 views)

Permalink

Hi Kristen.

On Sat, 26 Mar 2016 14:15:09 -0800, Kristen wrote:
> On 3/26/16 12:52 AM, ed wrote:
>>
>> Using John's patches will at least get you a good stable system with
>> minimal effort.
>
> Jens,
>
> I will voice my support of Jim Simpson's patches. I use these alone and
> I have an almost spam free server. The system is solid and is easy to build.

John M. Simpson has done a great job on his patches - there's no doubt!
But there are more things to consider for me than 'choosing a distribution'.

I already have an effective spam control system, which I designed myself and have actively improved over the last 6 years.

There are advantages to all of the 3 major distributions:
1: John's is tested for a long time, he's a *very* experienced maintainer with good security background, it's quite stable.
2: Roberto's is tested for quite a while, he's an experienced maintainer, it has some support for IPv6 support, which might be required where I move my servers to; it has updates, that John's does not have. In addition, he tend to keep an eye on Erwin's latest work. ;)
3: Erwin's is in active development. It may not have been tested for long, but Erwin is also very experienced and has a full IPv6 support and is also 'alive' and Erwin also have a strong experience with security.

If I have to re-install, I believe the best thing I could do is to move to Erwin's s/qmail and Qsmtp.
I see s/qmail as a "new product"; new products might have some initial issues, but they're usually never major problems.
It's fairly quick and uncomplicated to update from a git-repository compared to patches.
If I (or anyone else) finds a problem, the problem will be solved at the distribution-point and there will not be a patch-on-patch problem anymore.

In my opinion, staying with John Simpson's distribution is absolutely a good choice.
He also taught me how to make a proper qmail setup (thank you for your great guides, John, they've saved months of work!)

> I know you want to repair your system first. I did send an email to the
> debian list as requested. First it was greylisted, then accepted. I am
> on some debian mail list (Debian-PPC, Debian-Women) but not the one you
> are subscribe to.

All debian lists use the same mail-bot, so I have no doubt it works. :)
(I might be rebuilding and testing my qmail tonight)

By the way, do you also get an auto-responder from <ing@ing.nl> *every* time you post to the list ?
I've tried to make it turn itself off, since it seem to reply to everything with the subject intact, but unfortunately I couldn't make it confirm unsubscription.
I've also sent an email to the email associated with the domain name and to abuse@domain, but so far no response.
-If they had just subsribed to 'gcc-help', they would have realized how bad an idea it is to subscribe with an email address that is sometimes switched to an auto-responder.

Love
Jens

Re: Identifying used patches [ In reply to ]

amb-sendok-1461648002.fajhopdffbfoecbbjngn at bradfo

Mar 26, 2016, 10:20 PM

Post #11 of 12 (4294 views)

Permalink

Thus said Jens Bauer on Sun, 27 Mar 2016 06:29:22 +0200:

> By the way, do you also get an auto-responder from <ing@ing.nl>
> *every* time you post to the list ?

I used to, until I added it to badmailfrom. :-)

Andy
--
TAI64 timestamp: 4000000056f76da6

Re: Identifying used patches [ In reply to ]

jens-lists at gpio

Mar 26, 2016, 11:16 PM

Post #12 of 12 (4295 views)

Permalink

Hi Andy.

On 26 Mar 2016 23:20:02 -0600, Andy Bradford wrote:
> Thus said Jens Bauer on Sun, 27 Mar 2016 06:29:22 +0200:
>> By the way, do you also get an auto-responder from <ing@ing.nl>
>> *every* time you post to the list ?
>
> I used to, until I added it to badmailfrom. :-)

I could easily block it too, but I already sent them a few emails, so I'd like it to be possible for them to respond.
-After all, they've done it by accident and I'd like to give them a chance to correct the problem. ;)

Love
Jens

Mailing List Archive

Attached Files:

Attached Files: