Mailing List Archive

On Sat, 24 Feb 2024, graeme vetterlein via Exim-users wrote:

> Starting in mid February 2024, gmail tightened up its policies and I for one
> started getting bounced email.
>
> After about 4 lots of 12 hour days I figured out I was not setting errors_to=
> in my userforward
> so my "RCPT TO" envelope headerwas wrong/missing.
>
> Having fixed that, I now started getting failures sent to my postmaster (who
> I'd pointed errors_to at)
>
> ------------- cut here------------
> could not be delivered to one or more of its recipients. The following
> address(es) failed:
>
> ? auser@gmail.com
> ??? (generated from auser.home@abox.home)
> ??? host smtp.forwardemail.net [2001:19f0:ac01:d3c:5400:4ff:fe72:47c4]
> ??? SMTP error from remote mail server after end of data:
> ??? 550 From header must end with @mydomain.com
> ------------- cut here------------
>
>
> The phrase "From header" is suitably vague, do they mean the "MAIL FROM"?
> envelope header
> or the From header in the mail body headers ?
>
> This only seems to effect mails that have been forwarded via a a users
> ~/.forward.
>
> What I REALLY REALLY would like to do is watch the SMTP conversation between
> my exim4 and its smarterhost

Not directly the smtp conversation, but you can turn on -d debugging
in an ACL, with
control = debug/<options>
so you have full debugging just for messages to gmail.


> (it's probably something I'm sending, so I want as much to see *my* side of
> the conversation)
>
> I can see over many years people have been asking to do this and been told to
> use wireshark....which I eventually
> did. However , in the past this was fine but nowadays it does a STARTTLS and
> I can see no more.
>
> Is there really no way I can see the SMTP conversation? If I could just see
> what I sent it before said "error"
> I feel I could save another block of long nights.

--
Andrew C. Aitchison Kendal, UK
andrew@aitchison.me.uk

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-users.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-users-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

On 24/02/2024 02:12, graeme vetterlein via Exim-users wrote:
> The phrase "From header" is suitably vague, do they mean the "MAIL FROM"  envelope header
> or the From header in the mail body headers ?

I'd guess the latter. I usually phrase it as "From: hders"
to try to make the point, personally.

Wrt. Andrew's suggestion of debug mode: look specifically
for "SMTP>>" and "SMTP<<" lines in the debug.

For reference on Wireshark - that is very much a last resort.
It *is* possible to get Exim to dump TLS session keys to
a file which Wireshark can be told to use for decode of a
capture... Obviously given the security aspect, admin rights
are needed before Exim will do this. For anyone wanting
to do this: it's the usual SSLKEYLOGFILE (or however it
is spelled) env var, and you need to config Exim to keep
that in the environment, and be in debug mode.
--
Cheers,
Jeremy


--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-users.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-users-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

On 24/02/2024 02:12, graeme vetterlein via Exim-users wrote:
> 550 From header must end with @mydomain.com

Oh, I forgot to say: Look into "submission mode" and getting
plain account names qualified.
--
Cheers,
Jeremy


--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-users.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-users-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

On 2024-02-23 at 21:12:12 UTC-0500 (Sat, 24 Feb 2024 02:12:12 +0000)
graeme vetterlein via Exim-users <graeme.exim4@vetterlein.com>
is rumored to have said:

> Starting in mid February 2024, gmail tightened up its policies and I
> for one
> started getting bounced email.
>
> After about 4 lots of 12 hour days I figured out I was not setting
> errors_to= in my userforward
> so my "RCPT TO" envelope headerwas wrong/missing.'

The argument to RCPT TO is not a header. A "header" in email is a
logical line of text before the first blank line of the message with a
header name followed by ': ' and the header content.

The argument to RCPT TO is simply an email address. The argument to MAIL
FROM is an email address. Neither one is a header. Either may OR MAY NOT
match a message header's content.


> Having fixed that, I now started getting failures sent to my
> postmaster (who I'd pointed errors_to at)
>
> ------------- cut here------------
> could not be delivered to one or more of its recipients. The following
> address(es) failed:
>
>   auser@gmail.com
>     (generated from auser.home@abox.home)
>     host smtp.forwardemail.net
> [2001:19f0:ac01:d3c:5400:4ff:fe72:47c4]
>     SMTP error from remote mail server after end of data:
>     550 From header must end with @mydomain.com
> ------------- cut here------------
>
>
> The phrase "From header" is suitably vague,

It is not in any way vague.

> do they mean the "MAIL FROM"  envelope header

Again for emphasis: that is NOT a header.

> or the From header in the mail body headers ?

That's the only place a header exists.

Another clue is that this was "after the end of data" which means that
the server already gave a positive reply to MAIL FROM and saved this
error until it has actually seen the headers.

> This only seems to effect mails that have been forwarded via a a users
> ~/.forward.

Because that does nothing to modify the headers.

>
> What I REALLY REALLY would like to do is watch the SMTP conversation
> between my exim4 and its smarterhost
>
> (it's probably something I'm sending, so I want as much to see *my*
> side of the conversation)

I strongly doubt this is a technical problem in what Exim is doing in
SMTP. This reads like a policy issue: your account is only allowed to
send mail with a From header using your own domain.

Talk to the people who run forwardemail.net. Or read their docs. Or
change your account config with them. In any case, this is THEIR
POLICY.

> I can see over many years people have been asking to do this and been
> told to use wireshark....which I eventually
> did. However , in the past this was fine but nowadays it does a
> STARTTLS and I can see no more.
>
> Is there really no way I can see the SMTP conversation? If I could
> just see what I sent it before said "error"
> I feel I could save another block of long nights.

It sent the message, after getting a 'go ahead' response to DATA. It
ended the message with <CRLF>.<CRLF>, which is referred to in the error
message as "end of data"

The underlying cause here is that traditional transparent forwarding
(using .forward or /etc/aliases) does absolutely nothing to the message
data, but simply relays the message using the same envelope addresses
that the message arrived with. This has become increasingly incompatible
with the efforts to reduce spam by enhancing the authenticability of
messages. The first step was SPF, which allows domain owners to publish
a limited set of IP addresses that are authorized to use the domain in
MAIL FROM and how to interpret violations of that limit. SPF breaks
traditional forwarding. It has always been known to do so. There is a
mechanism for forwarding MTAs called SRS which was developed with SPF to
allow MTAs to rewrite envelope senders to the MTA's domain while
preserving bounce-ability. If you use transparent forwarding for mail
originating from a domain with a restrictive SPF record, it will fail.

We've also got DKIM to sign messages and DMARC which requires either SPF
or DKIM to succeed AND for the succeeding mechanism to use the domain
in the From header. Google and Yahoo have both said that they will be
requiring mail from large senders to be authenticatable via those
mechanisms to be accepted at all. It is expected that with those
behemoths getting tough, many others will finally start enforcing

In short: the mechanism you are using to forward email in the simplest
and oldest way is not sustainable.

--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-users.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-users-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

On 2024-02-24, graeme vetterlein via Exim-users <exim-users@lists.exim.org> wrote:

> What I REALLY REALLY would like to do is watch the SMTP conversation
> between my exim4 and its smarterhost
>
> (it's probably something I'm sending, so I want as much to see *my* side
> of the conversation)
>
> I can see over many years people have been asking to do this and been
> told to use wireshark.

really?

> ...which I eventually did. However , in the past this was fine but nowadays it does a STARTTLS
> and I can see no more. Is there really no way I can see the SMTP conversation? If I could just
> see what I sent it before said "error"
> I feel I could save another block of long nights.

There is a way to tell exim (well the crypto library actually) to save
the decryption keys so that wireshark can see inside the encryption.


However my normal approach is different.

turn off immediate deliveries by setting 'queue_only=yes' in main
config

Then send a test email, and record its ID (you can find that in exim
main log)

then do

exim -v -M Messag-eIdentifier-Here

and exim will attempt delivery and emit a cleartext transcript of the SMTP session.


--
Jasen.
???????? ????? ???????

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-users.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-users-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/