Mailing List Archive

exim can't handle 521 response from remote MX
Large email provider in my country uses 521 response at their MX for
some kind of delaying. They don't care that its against rfc1846

rfc1846 says:" A host which sends a 521 greeting message MUST NOT be
listed as an MX record for any domain"

# nc mx.poczta.onet.pl 25
220-mx.poczta.onet.pl ESMTP
521 5.7.1 Service unavailable; client [144.76.50.172] blocked using
postscreenbl.opbl.onet.pl.local

Exim can't handle this and does not even log anything in that
situation. Exim does not retry delivery and after 72 hours fails and
returns message with

all hosts for 'onet.pl' have been failing for a long time (and
retry time not reached)

How can I make exim to log those delivery attempts that end with 521
response? How can I make exim retry deliveries that ended with 521
response?

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: exim can't handle 521 response from remote MX [ In reply to ]
Hi,

krzf83--- via Exim-users <exim-users@exim.org> (Fr 27 Aug 2021 13:10:01 CEST):
> Large email provider in my country uses 521 response at their MX for
> some kind of delaying. They don't care that its against rfc1846
>
> rfc1846 says:" A host which sends a 521 greeting message MUST NOT be
> listed as an MX record for any domain"
>
> # nc mx.poczta.onet.pl 25
> 220-mx.poczta.onet.pl ESMTP
> 521 5.7.1 Service unavailable; client [144.76.50.172] blocked using
> postscreenbl.opbl.onet.pl.local

From m PoV, they clearly state that they do not want connections from
your IP. Not now, and not later.

> Exim can't handle this and does not even log anything in that
> situation. Exim does not retry delivery and after 72 hours fails and
> returns message with

I believe, Exim logs that.

> all hosts for 'onet.pl' have been failing for a long time (and
> retry time not reached)
>
> How can I make exim to log those delivery attempts that end with 521
> response? How can I make exim retry deliveries that ended with 521
> response?

5xx means: permanent failure. Period. If their intention is something
else, it is up to them.

Maybe some of them is reading on the mailops list. Try contacting them
there.

Best regards from Dresden/Germany
Viele Grüße aus Dresden
Heiko Schlittermann
--
SCHLITTERMANN.de ---------------------------- internet & unix support -
Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
gnupg encrypted messages are welcome --------------- key ID: F69376CE -
Re: exim can't handle 521 response from remote MX [ In reply to ]
>
> 5xx means: permanent failure. Period.


Then exim should return message to sender immeadetly but it does not.
Instead exim remembers that that remote mx is "failing for long time" and
does not even try to deliver new mails! If exim for some reason does retry
then it logs "Remote host closed connection in response to EHLO" and won't
retry it again for any emails for that domain! Only after 72 your it
returns email to sender.

pt., 27 sie 2021 o 14:37 Heiko Schlittermann via Exim-users <
exim-users@exim.org> napisa?(a):
>
> Hi,
>
> krzf83--- via Exim-users <exim-users@exim.org> (Fr 27 Aug 2021 13:10:01
CEST):
> > Large email provider in my country uses 521 response at their MX for
> > some kind of delaying. They don't care that its against rfc1846
> >
> > rfc1846 says:" A host which sends a 521 greeting message MUST NOT be
> > listed as an MX record for any domain"
> >
> > # nc mx.poczta.onet.pl 25
> > 220-mx.poczta.onet.pl ESMTP
> > 521 5.7.1 Service unavailable; client [144.76.50.172] blocked using
> > postscreenbl.opbl.onet.pl.local
>
> From m PoV, they clearly state that they do not want connections from
> your IP. Not now, and not later.
>
> > Exim can't handle this and does not even log anything in that
> > situation. Exim does not retry delivery and after 72 hours fails and
> > returns message with
>
> I believe, Exim logs that.
>
> > all hosts for 'onet.pl' have been failing for a long time (and
> > retry time not reached)
> >
> > How can I make exim to log those delivery attempts that end with 521
> > response? How can I make exim retry deliveries that ended with 521
> > response?
>
> 5xx means: permanent failure. Period. If their intention is something
> else, it is up to them.
>
> Maybe some of them is reading on the mailops list. Try contacting them
> there.
>
> Best regards from Dresden/Germany
> Viele Grüße aus Dresden
> Heiko Schlittermann
> --
> SCHLITTERMANN.de ---------------------------- internet & unix support -
> Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
> gnupg encrypted messages are welcome --------------- key ID: F69376CE -
> --
> ## List details at https://lists.exim.org/mailman/listinfo/exim-users
> ## Exim details at http://www.exim.org/
> ## Please use the Wiki with this list - http://wiki.exim.org/
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: exim can't handle 521 response from remote MX [ In reply to ]
On 02/09/2021 20:25, krzf83--- via Exim-users wrote:
>>> Large email provider in my country uses 521 response at their MX for
>>> some kind of delaying. They don't care that its against rfc1846
>>>
>>> rfc1846 says:" A host which sends a 521 greeting message MUST NOT be
>>> listed as an MX record for any domain"
>>>
>>> # nc mx.poczta.onet.pl 25
>>> 220-mx.poczta.onet.pl ESMTP
>>> 521 5.7.1 Service unavailable; client [144.76.50.172] blocked using
>>> postscreenbl.opbl.onet.pl.local

That is not a consistent response. The first line is a 220 (with a flag
saying it will be a multi-line response). The second is a 521 (without
the flag, hence the last line).

All the lines of the response should have the same code
(RFC 5321 section 4.2.1 last para:
"In a multiline reply, the reply code on each of the lines MUST be the
same."
)

For what it's worth, testing from here gets a two-line response but
with 220 for both.

--
Cheers,
Jeremy

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: exim can't handle 521 response from remote MX [ In reply to ]
Hello.

On Thu, Sep 02, 2021 at 09:25:20PM +0200, krzf83--- via Exim-users wrote:
> Then exim should return message to sender immeadetly but it does not.
> Instead exim remembers that that remote mx is "failing for long time" and
> does not even try to deliver new mails! If exim for some reason does retry
> then it logs "Remote host closed connection in response to EHLO" and won't
> retry it again for any emails for that domain! Only after 72 your it
> returns email to sender.

Such behaviour of SMTP server --

> > > # nc mx.poczta.onet.pl 25
> > > 220-mx.poczta.onet.pl ESMTP
> > > 521 5.7.1 Service unavailable; client [144.76.50.172] blocked using
> > > postscreenbl.opbl.onet.pl.local

is a direct violation of SMTP protocol, which has no place for different
status codes in a single reply. This reply may be treated as a temporary
error (as Exim does), for example, as transient data corruption.
Client's reaction is undefined.

Server should NOT begin it's greeting with 220 to make legitime rejection.
Special code 554 is reserved for rejection on greeting - RFC2821, par.3.1:

The SMTP protocol allows a server to formally reject a transaction
while still allowing the initial connection as follows: a 554
response MAY be given in the initial connection opening message
instead of the 220.

Server also must not use extended status codes until client's EHLO.
--
Eugene Berdnikov

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: exim cant handle 521 response from remote MX [ In reply to ]
Maybe Exim should get a feature where you can define "protocol overrides" for domains, like so it
ignore specific protocol violations and such.
So if you have problem with a slightly misbehaving remote mail server, you can in a acl define how
exim should treat its responses (for example, treat it as temporary rejection, permanent rejection,
or just ignore it) - at different stages in the connection.

-----Ursprungligt meddelande-----
Fr?n: Jeremy Harris via Exim-users <exim-users@exim.org>
Skickat: den 3 september 2021 00:46
Till: exim-users@exim.org
?mne: Re: [exim] exim cant handle 521 response from remote MX

On 02/09/2021 20:25, krzf83--- via Exim-users wrote:
>>> Large email provider in my country uses 521 response at their MX for
>>> some kind of delaying. They don't care that its against rfc1846
>>>
>>> rfc1846 says:" A host which sends a 521 greeting message MUST NOT be
>>> listed as an MX record for any domain"
>>>
>>> # nc mx.poczta.onet.pl 25
>>> 220-mx.poczta.onet.pl ESMTP
>>> 521 5.7.1 Service unavailable; client [144.76.50.172] blocked using
>>> postscreenbl.opbl.onet.pl.local

That is not a consistent response. The first line is a 220 (with a flag
saying it will be a multi-line response). The second is a 521 (without
the flag, hence the last line).

All the lines of the response should have the same code
(RFC 5321 section 4.2.1 last para:
"In a multiline reply, the reply code on each of the lines MUST be the
same."
)

For what it's worth, testing from here gets a two-line response but
with 220 for both.

--
Cheers,
Jeremy

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: exim can't handle 521 response from remote MX [ In reply to ]
On 2 Sep 2021, at 23:24, Jeremy Harris via Exim-users <exim-users@exim.org> wrote:
> On 02/09/2021 20:25, krzf83--- via Exim-users wrote:
>> # nc mx.poczta.onet.pl 25
>> 220-mx.poczta.onet.pl ESMTP
>> 521 5.7.1 Service unavailable; client [144.76.50.172] blocked using
>> postscreenbl.opbl.onet.pl.local
>
> That is not a consistent response. The first line is a 220 (with a flag
> saying it will be a multi-line response). The second is a 521 (without
> the flag, hence the last line).
>
> All the lines of the response should have the same code
> (RFC 5321 section 4.2.1 last para:
> "In a multiline reply, the reply code on each of the lines MUST be the
> same."
> )

This is postscreen, see:
http://www.postfix.org/POSTSCREEN_README.html

The site can configure it correctly to delay the enforcement until a mail transaction, but have instead gone with “drop”. Because the DNSBL check is run in parallel with pre-greeting tests, you get the “teaser” 220- followed by the 521 failure. It’s obviously wrong, but perhaps Exim should consider the last line of a response instead of the first for purposes of evaluation?

Cheers,
Sabahattin


--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: exim can't handle 521 response from remote MX [ In reply to ]
On 04/09/2021 15:03, Sabahattin Gucukoglu via Exim-users wrote:
> perhaps Exim should consider the last line of a response instead of the first for purposes of evaluation?

I don't see a coherent argument for either direction,
when they differ.

If postscreen is doing it so wrongly, it is the thing that needs fixing.
--
Cheers,
Jeremy

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: exim can't handle 521 response from remote MX [ In reply to ]
On Sat, Sep 04, 2021 at 03:42:39PM +0100, Jeremy Harris via Exim-users wrote:
> On 04/09/2021 15:03, Sabahattin Gucukoglu via Exim-users wrote:
> > perhaps Exim should consider the last line of a response instead of the first for purposes of evaluation?
>
> I don't see a coherent argument for either direction,
> when they differ.
>
> If postscreen is doing it so wrongly, it is the thing that needs fixing.

The postscreen(8) behaviour is unlikely to change.

Absent a time-machine, and given that the ultimate decision is made
after the initial banner and greet pause, and that refusing SMTP service
(521 banner) is supposed to only happen to botnet and similar clients,
the postscreen(8) service has no choice but to appear to change its mind
after the initial "220-".

We hope that any clients that are erroneously rejected legitimate MTAs
will get the hint, but the primary audience for postscreen bad news are
botnet SMTP engines, and what they make of the dropped connections is of
little interest.

FWIW, Postfix has always taken the SMTP status code from the last line
of a multi-line server response, but as you noted there is no RFC
requirement to do so, and the code is expected to not vary from line to
line.

This does mean that the few Postfix systems that enable greet pauses in
postscreen(8), and then reject a legitimate MTA they judge to have a
poor IP reputation, might end up dealing with repeated retries from such
MTAs.

Choosing the last response code has some advantages, but Exim is under
no obligation to do so.

--
Viktor.

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: exim can't handle 521 response from remote MX [ In reply to ]
On 4 Sep 2021, at 15:42, Jeremy Harris via Exim-users <exim-users@exim.org> wrote:
> On 04/09/2021 15:03, Sabahattin Gucukoglu via Exim-users wrote:
>> perhaps Exim should consider the last line of a response instead of the first for purposes of evaluation?
>
> I don't see a coherent argument for either direction,
> when they differ.
>
> If postscreen is doing it so wrongly, it is the thing that needs fixing.

I agree, it is, but Exim should be robust, all the same. Deferring mail when not needed is obviously undesirable.

Postscreen’s entire reason for being is to defend against the Internet’s SMTP barbarians, so being standards-compliant is perhaps understandably not a high priority. I don’t approve, myself, and haven’t used it, but I appreciate why it exists. The site should configure it right, but by its nature it cannot co-exist with a valid SMTP server process while efficiently disposing a new client, and if it rejects a site by policy, it’s not obliged to accommodate it.

Cheers,
Sabahattin


--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: exim can't handle 521 response from remote MX [ In reply to ]
--On Saturday, September 4, 2021 16:25 +0100 Sabahattin
Gucukoglu via Exim-users <exim-users@exim.org> wrote:

>> If postscreen is doing it so wrongly, it is the thing that
>> needs fixing.
>
> I agree, it is, but Exim should be robust, all the same.
> Deferring mail when not needed is obviously undesirable.

Not so obviously. If one believes this might be a
misconfiguration error (i.e., the fault of whomever set up the
destination server and its MX configuration in the DNS) it would
make perfect sense to do the lookup again and retry a different
MX server or even to assume that the configuration problem is
temporary.

But, again, a multiline reply with different codes on the lines
is sufficiently broken but whatever the client does in response
does not violate the spec.

Incidentally, the reference in my prior note to Section 4.2.4.2
in RFC 5321 was incorrect. That section appears only in
rfc5321bis [1].

> Postscreen's entire reason for being is to defend against
> the Internet's SMTP barbarians, so being standards-compliant
> is perhaps understandably not a high priority. I don't
> approve, myself, and haven't used it, but I appreciate why
> it exists. The site should configure it right, but by its
> nature it cannot co-exist with a valid SMTP server process
> while efficiently disposing a new client, and if it rejects a
> site by policy, it's not obliged to accommodate it.

If it is configured to respond on an SMTP port, it still has
some small obligation to avoid violating the spec for no good
reason. See below.


--On Saturday, September 4, 2021 11:23 -0400 Viktor Dukhovni via
Exim-users <exim-users@exim.org> wrote:

>...
> Absent a time-machine, and given that the ultimate decision is
> made after the initial banner and greet pause, and that
> refusing SMTP service (521 banner) is supposed to only happen
> to botnet and similar clients, the postscreen(8) service has
> no choice but to appear to change its mind after the initial
> "220-".
>...

If, by "change its mind", you mean "send a response sequence
with different codes", not true. First, if it cared about the
SMTP spec (and I understand the reasons why it might not), it
should accumulate whatever information it thinks useful before
sending the initial connection response and then reply with
either 220 or 521 (or something else) as it thinks appropriate,
not try to mix them. Second, it could return 220 (normally
considered the correct response if it accepts mail from anyone)
and then return 521 reply codes to any further commands until
either those commands stopped coming or it go fed up and just
closed the connection.

It does occur to me that a "no mail accepted right now" code
might help to clarify the situation. Watch for rfc5321bis-04.

john

[1]
https://datatracker.ietf.org/doc/draft-ietf-emailcore-rfc5321bis/

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: exim can't handle 521 response from remote MX [ In reply to ]
On Sat, Sep 04, 2021 at 11:23:02AM -0400, Viktor Dukhovni via Exim-users wrote:
> FWIW, Postfix has always taken the SMTP status code from the last line
> of a multi-line server response, but as you noted there is no RFC
> requirement to do so, and the code is expected to not vary from line to
> line.

IMHO, this discussion should be a motive for Exim and Postfix developers
to add checks for consistency of SMTP status codes in multiline answers.

Inconsistent protocol messages are indicators of software bugs, so they
should lead to some attempts to work around (if possible) or switch to
status "communication impossible -- permanent failure".
--
Eugene Berdnikov

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: exim can't handle 521 response from remote MX [ In reply to ]
On Sat, Sep 04, 2021 at 08:28:29PM +0300, Evgeniy Berdnikov via Exim-users wrote:

> IMHO, this discussion should be a motive for Exim and Postfix developers
> to add checks for consistency of SMTP status codes in multiline answers.
>
> Inconsistent protocol messages are indicators of software bugs, so they
> should lead to some attempts to work around (if possible) or switch to
> status "communication impossible -- permanent failure".

That's not going to happen in Postfix. The basic philosophy is that
Postfix is the business of routing email messages, not gratuitous RFC
enforcement. The postscreen(8) behaviour observed is not RFC-compliant,
but it is a not a bug, it is a deliberate trade-off among various
imperfect options.

The status code on the last line of a multi-line response is and will be
taken as authoritative, regardless of any other status codes on prior
lines of the response. This has worked well enough for 22+ years.

The one related thing that might be worth changing is the default of:

http://www.postfix.org/postconf.5.html#smtp_skip_5xx_greeting

there used to be Microsoft Exchange servers that erroneously replied
with 5XX greetings under load, if that's no longer the case, we could
accept 5XX greetings as dispositive.

Thus, FWIW, just the like an Exim client that retries the message under
the reported conditions, a Postfix client in its default configuration
would also retry, but for a different reason. It would see the 521
reply, but then treat that as though it were 421 and either try the next
MX host or defer.

--
Viktor.

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: exim can't handle 521 response from remote MX [ In reply to ]
On Sat, Sep 04, 2021 at 01:18:17PM -0400, John C Klensin wrote:

> > Absent a time-machine, and given that the ultimate decision is
> > made after the initial banner and greet pause, and that
> > refusing SMTP service (521 banner) is supposed to only happen
> > to botnet and similar clients, the postscreen(8) service has
> > no choice but to appear to change its mind after the initial
> > "220-".
>
> If, by "change its mind", you mean "send a response sequence
> with different codes", not true. First, if it cared about the
> SMTP spec (and I understand the reasons why it might not), it
> should accumulate whatever information it thinks useful before
> sending the initial connection response and then reply with
> either 220 or 521 (or something else) as it thinks appropriate,
> not try to mix them.

The greet pause test is *specifically* designed to detect botnet spam
engines that don't wait for the complete multi-line response, and start
talking as soon as they detect the first line of the response. That's
why the pause is after, and not before, "220-". This is also why the
final response code is unavoidably different from the initial.

> Second, it could return 220 (normally considered the correct response
> if it accepts mail from anyone) and then return 521 reply codes to any
> further commands until either those commands stopped coming or it go
> fed up and just closed the connection.

Once the client is believed to be an undesirable source of email
connections, we don't bother with niceties, and drop the connection.
The "521" is a best effort concession to rough protocol conformance.

Clients blacklisted by postscreen(8) don't get to talk SMTP the real
Postfix SMTP server.

> It does occur to me that a "no mail accepted right now" code
> might help to clarify the situation. Watch for rfc5321bis-04.

In practice, there are just 5 response codes in SMTP:

2XY
4XY vs 421
5XY vs 521

No values of "XY" other than "21" make any difference to most
implementations. So I don't see much point in new response codes.

What could perhaps be clarified further is the meaning of 5XY greetings.
Should a sending system that encounters a 5XX greeting defer or bounce
the message envelope. The right answer depends on understanding when
5XX greetings occur in recent practice.

--
Viktor.

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: exim can't handle 521 response from remote MX [ In reply to ]
On Sat, 4 Sep 2021, Viktor Dukhovni via Exim-users wrote:

> On Sat, Sep 04, 2021 at 01:18:17PM -0400, John C Klensin wrote:
>
>>> Absent a time-machine, and given that the ultimate decision is
>>> made after the initial banner and greet pause, and that
>>> refusing SMTP service (521 banner) is supposed to only happen
>>> to botnet and similar clients, the postscreen(8) service has
>>> no choice but to appear to change its mind after the initial
>>> "220-".
>>
>> If, by "change its mind", you mean "send a response sequence
>> with different codes", not true. First, if it cared about the
>> SMTP spec (and I understand the reasons why it might not), it
>> should accumulate whatever information it thinks useful before
>> sending the initial connection response and then reply with
>> either 220 or 521 (or something else) as it thinks appropriate,
>> not try to mix them.
>
> The greet pause test is *specifically* designed to detect botnet spam
> engines that don't wait for the complete multi-line response, and start
> talking as soon as they detect the first line of the response. That's
> why the pause is after, and not before, "220-". This is also why the
> final response code is unavoidably different from the initial.

Are you saying that applies in this case ?
If so, then exim is replying during the greet pause, which is a real bug ?

--
Andrew C. Aitchison Kendal, UK
andrew@aitchison.me.uk

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: exim can't handle 521 response from remote MX [ In reply to ]
On 4 Sep 2021, at 3:00 pm, Andrew C Aitchison <andrew@aitchison.me.uk> wrote:

> > The greet pause test is *specifically* designed to detect botnet spam
> > engines that don't wait for the complete multi-line response, and start
> > talking as soon as they detect the first line of the response. That's
> > why the pause is after, and not before, "220-". This is also why the
> > final response code is unavoidably different from the initial.
>
> Are you saying that applies in this case ?
> If so, then exim is replying during the greet pause, which is a real bug ?

No, in this case, during or at the end of the greet pause this particular
systems was likely also configured to perform RBL checks and the like, and
the final "go away" response is a result of IP reputation, not a greet-pause
violation. I expect that Exim handles multi-line responses correctly.

Regardless, the final status was not known at the time of the initial
"220-" greeting, and the "521" final line was the earliest opportunity
for the bad news. At that point there is no point in continuing an SMTP
conversation, the client is presumed to be a botnet node or equivalent.
Rather than just drop the connection, a "521 go away" is used to finish
the multi-line response.

--
Viktor.

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: exim can't handle 521 response from remote MX [ In reply to ]
(Note two responses combined below)

--On Saturday, September 4, 2021 14:07 -0400 Viktor Dukhovni via
Exim-users <exim-users@exim.org> wrote:

>...
> The status code on the last line of a multi-line response is
> and will be taken as authoritative, regardless of any other
> status codes on prior lines of the response. This has worked
> well enough for 22+ years.

Viktor,

As I assume you may have guessed given that you follow
EMAILCORE, my main interest in this right now is to think about
what changes, if any, are needed in 5431bis. Watch for a note
on that list and some changes in -04 that reflect this
conversation, for which thanks to everyone. From that
particular perspective and purpose, as soon as someone says "for
my specific application or bright idea, it does not matter what
the standard says", I sort of lose interest.

However, while (apparently unlike many of the rest of you) I
have not spent any significant time in more than a decade
pouring over logs looking for mail transaction behavior
anomalies, I don't believe "worked well enough for 22+ years"
actually conveys much information. When I was last looking at
those logs, the number of times I saw a server returning a
multiline reply with mixed codes was zero or very close to it.
If all of the codes are the same, as SMTP requires, then things
will work well no matter which one is picked. Now, if you were
to say "there haven't been any problems since this behavior
first became common N years ago", that would be useful
information. But...

--and--

--On Saturday, September 4, 2021 20:00 +0100 Andrew C Aitchison
via Exim-users <exim-users@exim.org> wrote:

>> The greet pause test is *specifically* designed to detect
>> botnet spam engines that don't wait for the complete
>> multi-line response, and start talking as soon as they detect
>> the first line of the response. That's why the pause is
>> after, and not before, "220-". This is also why the final
>> response code is unavoidably different from the initial.
>
> Are you saying that applies in this case ?
> If so, then exim is replying during the greet pause, which is
> a real bug ?

Yes, if exim, or anything else, were responding before the last
line of a valid multiliine response was received, that would be,
as I understand the standard, a bug. However, if an SMTP client
saw one or more lines of a multiline response that started with
a particular code and then a line with some other code (whether
it carried a continuation indicator or not), well that is
outside the spec and nothing that client did, including closing
the connection or sending an immediate command (especially if it
were QUIT), would violate the spec clearly enough to be an
unambiguous bug.

john






--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: exim can't handle 521 response from remote MX [ In reply to ]
On Sat, Sep 04, 2021 at 04:51:29PM -0400, John C Klensin wrote:

> As I assume you may have guessed given that you follow
> EMAILCORE, my main interest in this right now is to think about
> what changes, if any, are needed in 5431bis. Watch for a note
> on that list and some changes in -04 that reflect this
> conversation, for which thanks to everyone.

Will do, thanks.

> From that particular perspective and purpose, as soon as someone says
> "for my specific application or bright idea, it does not matter what
> the standard says", I sort of lose interest.

FWIW, the "does not matter" in question is very narrowly scoped to the
fine-grained detail in the second and third digits of 3-digit SMTP
responses.

You might recall the "what dogs hear" analogy from an earlier thread on
emailcore. Many an SMTP client doesn't look beyond the first digit of
the SMTP server's response. Postfix is among them, perhaps Exim is not?

We do strive to emit the expected 2XY/4XY/5XY codes, but expect others
to use them consistently in return.

> However, while (apparently unlike many of the rest of you) I
> have not spent any significant time in more than a decade
> pouring over logs looking for mail transaction behavior
> anomalies, I don't believe "worked well enough for 22+ years"
> actually conveys much information.

What's worked well in this context is using the response from the last
line. Actually emitting a different response code on the last last is
a much more recent "innovation", and is used very narrowly to turn away
abusive botnet nodes without the cost of tying up a heavy-weight SMTP
server process to handle the connection.

The postscreen(8) service is an optional feature, that is off by
default, and greet pauses are also off by default, even when
postscreen(8) is enabled.

Legitimate MTAs are typically not turned away by postscreen(8), so
seeing the "220-" followed by a "521" is by far the exception rather
than the rule, and if a legitimate MTA ends up retrying the message,
that could be argued to be a feature, the undeserved IP reputation
might have been resolved by then.

Indeed Postfix (as a client) defaults to retrying (another MX or defer
to later) after a 5XX greeting. So Exim is not doing anything
unexpected.

> When I was last looking at those logs, the number of times I saw a
> server returning a multiline reply with mixed codes was zero or very
> close to it.

This both recent and unusual when the client is not a botnet, ...

> If all of the codes are the same, as SMTP requires, then things will
> work well no matter which one is picked. Now, if you were to say
> "there haven't been any problems since this behavior first became
> common N years ago", that would be useful information. But...

The variable multi-line response code server-side behaviour is new with
postscreen(8), which was first released in Feb 2011.

As mentioned above, it should be rather rare for a legitimate MTA as a
client to see such responses. Users of postscreen(8) should be cautious
to not make it too aggressive in its policies. The intent is to reduce
the number of bad connections that make it through to the real SMTP
servers, not eliminate all possibility of unwanted clients getting
through. Light-weight first stage of defense in depth.

--
Viktor.

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: exim can't handle 521 response from remote MX [ In reply to ]
On Sat, Sep 04, 2021 at 05:14:22PM -0400, Viktor Dukhovni via Exim-users wrote:
> As mentioned above, it should be rather rare for a legitimate MTA as a
> client to see such responses. Users of postscreen(8) should be cautious
> to not make it too aggressive in its policies. The intent is to reduce
> the number of bad connections that make it through to the real SMTP
> servers, not eliminate all possibility of unwanted clients getting
> through. Light-weight first stage of defense in depth.

IMHO, strict and clear rules of behaviour, expressed in SMTP protocol,
are much more significant for community then miserable profit for one
rare and specific case, achived by protocol violation.

Moreover, in this case there are many other protocol-compliant solutions.
Rejection may be done on MAIL FROM, server may close connection, etc.
--
Eugene Berdnikov

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: exim can't handle 521 response from remote MX [ In reply to ]
--On Saturday, September 4, 2021 17:14 -0400 Viktor Dukhovni via
Exim-users <exim-users@exim.org> wrote:

>...
>> From that particular perspective and purpose, as soon as
>> someone says "for my specific application or bright idea, it
>> does not matter what the standard says", I sort of lose
>> interest.
>
> FWIW, the "does not matter" in question is very narrowly
> scoped to the fine-grained detail in the second and third
> digits of 3-digit SMTP responses.

I was actually thinking more about the comments that postscreen
was going to do what it did for good reasons and was not going
to change. But, as to the codes...

> You might recall the "what dogs hear" analogy from an earlier
> thread on emailcore. Many an SMTP client doesn't look beyond
> the first digit of the SMTP server's response. Postfix is
> among them, perhaps Exim is not?

IIR (running out of time and cycles today), 5321 already says
that the first digit is the important one and that it is
necessary to pay attention to other digits only if they are
important to the client or elsewhere in the mail handling
system. I've seen multiple situations in which making the
distinctions implied by those additional digits in cases you
didn't list are important, but most involve systems receiving
and processing NDNs, not SMTP clients themselves. My sense is
that, if Postfix or some other SMTP client does not find it
useful to process past the first digit, it is just fine,
provides useful future-proofing, and is completely consistent
with the spec. I do hope, however, that when Postfix is acting
as a server, it is careful about the codes that it generates,
supplying as much information as possible in the primary codes
and, ideally, adding Enhanced Status codes where they would
provide additional information.

> We do strive to emit the expected 2XY/4XY/5XY codes, but
> expect others to use them consistently in return.

See above.

>> However, while (apparently unlike many of the rest of you) I
>> have not spent any significant time in more than a decade
>> pouring over logs looking for mail transaction behavior
>> anomalies, I don't believe "worked well enough for 22+ years"
>> actually conveys much information.
>
> What's worked well in this context is using the response from
> the last line. Actually emitting a different response code on
> the last last is a much more recent "innovation", and is used
> very narrowly to turn away abusive botnet nodes without the
> cost of tying up a heavy-weight SMTP server process to handle
the connection.

I think this is what I said. I was just pointing out that, if
the codes on all of the response lines are the same, picking the
first one, the last one, or the one in the middle would all have
worked equally well.

> The postscreen(8) service is an optional feature, that is off
> by default, and greet pauses are also off by default, even when
> postscreen(8) is enabled.

Good.

> Legitimate MTAs are typically not turned away by
> postscreen(8), so seeing the "220-" followed by a "521" is by
> far the exception rather than the rule, and if a legitimate
> MTA ends up retrying the message, that could be argued to be a
> feature, the undeserved IP reputation might have been resolved
> by then.

ON the other hand, if you actually wanted that retry, you might
reasonably return something in the 4yz range, which would
encourage it. In any case, if a legitimate MTA does not
understand postscreen's private conventions - whether about
switching codes in a multiline response, a delay between the
first line and subsequent ones, the particular choice of codes
or something else, I don't think there is any grounds for
complain about what it does or does not do. next.

> Indeed Postfix (as a client) defaults to retrying (another MX
> or defer to later) after a 5XX greeting. So Exim is not doing
> anything unexpected.
>
>> When I was last looking at those logs, the number of times I
>> saw a server returning a multiline reply with mixed codes was
>> zero or very close to it.
>
> This both recent and unusual when the client is not a botnet,
> ...
>
>> If all of the codes are the same, as SMTP requires, then
>> things will work well no matter which one is picked. Now,
>> if you were to say "there haven't been any problems since
>> this behavior first became common N years ago", that would be
>> useful information. But...
>
> The variable multi-line response code server-side behaviour is
> new with postscreen(8), which was first released in Feb 2011.
>
> As mentioned above, it should be rather rare for a legitimate
> MTA as a client to see such responses. Users of postscreen(8)
> should be cautious to not make it too aggressive in its
> policies. The intent is to reduce the number of bad
> connections that make it through to the real SMTP servers, not
> eliminate all possibility of unwanted clients getting through.
> Light-weight first stage of defense in depth.

Good. And good whether I would choose to run such a filtering
system or not.

john


--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/