Mailing List Archive

On 16 Apr 2019, at 04:27, Konstantin Boyandin (lists) via Exim-users <exim-users@exim.org> wrote:
> Is there any Exim setting capable of re-formatting long headers (i.e., split them into several lines, each below 998 character limit) etc such headers, but allowing the incoming mail nonetheless?

I feel your pain; I had to migrate 20000 or so student mailboxes from Gmail to Exchange Online (Office 365) a couple of years back and there were thousands of messages with over-long lines (but not headers!) and EOL would not accept them. We used imapsync which allows in-flight modification of the message, so we split at 998 chars and added \r\n as a line break. Anyhow...

I don’t think there is a generic ‘fold over-long header lines’ option, as that interferes with the mantra of not changing the message in flight.

Main config option ‘header_line_maxsize’ is both your friend and enemy here - the default is 0; you clearly have it set to 998.

If you want to fold a header, you’ll need to remove and replace the header:

* Set global option ‘header_line_maxsize' to 0 (or don’t specify it at all)
* Set an ACL variable (in the DATA ACL) to contain the value of the header if it’s longer than 998 chars
* Use sg on that variable to add ‘\r\n ‘ every 990 characters
* Use headers_remove and headers_add on the appropriate transport to replace the over-long line header

HOWEVER: I don’t think this is advisable, as you could break DKIM signing if the over-long header is included in the signature. You may also need to experiment on whether this is best applied using header_thing or rheader_thing - probably the latter.

Graeme
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Hi, Konstantin.

On Tue, Apr 16, 2019 at 03:27:57AM +0000, Konstantin Boyandin (lists) via Exim-users wrote:
> One of users complained at having their message rejected by our mail
> server (Exim 4.91), providing the below server response:
>
> "Mail delivery error. This email issue occurs because your email header
> was rejected by recipient server.
> The cause of rejection is because email reference header is more than
> 998 character. The 998 character limit is due to limitations in many
> implementations that send, receive, or store IMF messages which simply
> cannot handle more than 998 characters on a line. Receiving
> implementations would do well to handle an arbitrarily large number of
> characters in a line for robustness sake."

Probably config of your Exim contains in acl_check_data something like:

deny
condition = ${if > {$max_received_linelength}{998}}
message = Mail delivery error. This email issue occurs because...
...

If you turn it off, your Exim could receive improperly formatted mail.

> The user's emails contain:
>
> X-Mailer: Microsoft Outlook 15.0
>
> Is there any Exim setting capable of re-formatting long headers
> (i.e., split them into several lines, each below 998 character
> limit) etc such headers, but allowing the incoming mail nonetheless?

Exim has no such built-in capability (AFAIK).

As for MS Outlook, it has bugs with mail formatting from version 10.
At most cases the "References:" header is broken and contain garbage.
There is no sense to reformat it, because this garbage reflects only
memory corruption and has no valid message-ids. You have better to remove
References: header completely. Put in your config something like this:

acl_check_data:
...
warn
set acl_m_ref_len = ${strlen:$h_references:}
condition = ${if > {$acl_m_ref_len}{998}}
logwrite = References header has length=$acl_m_ref_len, removed
remove_header = references

> We have quite long "threads" in local mail archives, with really
> many References: entries, but still not exceeding 998 characters per
> line.

Mail client is responsible for keeping References: withing reasonable
number of entries and below line length limit.
--
Eugene Berdnikov

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

On 16 Apr 2019, at 5:18, Graeme Fowler via Exim-users wrote:

> HOWEVER: I don’t think this is advisable, as you could break DKIM
> signing if the over-long header is included in the signature.

Because header folding is a permissible in-flight message modification
per RFC822 & its successors, the DKIM 'relaxed' header canonicalization
(which is almost universally used) protects signatures from breakage by
folding.

--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Available For Hire: https://linkedin.com/in/billcole

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

On 16/04/2019 10:18, Graeme Fowler via Exim-users wrote:
> On 16 Apr 2019, at 04:27, Konstantin Boyandin (lists) via Exim-users <exim-users@exim.org> wrote:
>> Is there any Exim setting capable of re-formatting long headers (i.e., split them into several lines, each below 998 character limit) etc such headers, but allowing the incoming mail nonetheless?
>
> I feel your pain; I had to migrate 20000 or so student mailboxes from Gmail to Exchange Online (Office 365) a couple of years back and there were thousands of messages with over-long lines (but not headers!) and EOL would not accept them. We used imapsync which allows in-flight modification of the message, so we split at 998 chars and added \r\n as a line break. Anyhow...

The 998 limit is for de-folded lines. Go over that and you'd need
multiple header lines; to do which you'd have to be duplicating
header_names: as well as adding linebreaks. And yes, that would
very likely break DKIM signatures.

The folding limit is 78.

No, there's no builtin in Exim to do that. You could program
it in ACL, but it's some fair amount of work - especially if
you want to try to identify "high level semantic break points"
as suggested by RFC for line-folding.
--
Cheers,
Jeremy


--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Hi Evgeniy,

On 2019-04-16 09:20, Evgeniy Berdnikov via Exim-users wrote:
> Hi, Konstantin.
>
> On Tue, Apr 16, 2019 at 03:27:57AM +0000, Konstantin Boyandin (lists)
> via Exim-users wrote:
>> One of users complained at having their message rejected by our mail
>> server (Exim 4.91), providing the below server response:
>>
>> "Mail delivery error. This email issue occurs because your email
>> header
>> was rejected by recipient server.
>> The cause of rejection is because email reference header is more than
>> 998 character. The 998 character limit is due to limitations in many
>> implementations that send, receive, or store IMF messages which simply
>> cannot handle more than 998 characters on a line. Receiving
>> implementations would do well to handle an arbitrarily large number of
>> characters in a line for robustness sake."
>
> Probably config of your Exim contains in acl_check_data something
> like:
>
> deny
> condition = ${if > {$max_received_linelength}{998}}
> message = Mail delivery error. This email issue occurs because...
> ...
>
> If you turn it off, your Exim could receive improperly formatted mail.

Indeed ti is. However, I think it's a bad idea to accept incorrectly
formatted mail, there will be cascade of further problems

>> The user's emails contain:
>>
>> X-Mailer: Microsoft Outlook 15.0
>>
>> Is there any Exim setting capable of re-formatting long headers
>> (i.e., split them into several lines, each below 998 character
>> limit) etc such headers, but allowing the incoming mail nonetheless?
>
> Exim has no such built-in capability (AFAIK).
>
> As for MS Outlook, it has bugs with mail formatting from version 10.
> At most cases the "References:" header is broken and contain garbage.
> There is no sense to reformat it, because this garbage reflects only
> memory corruption and has no valid message-ids. You have better to
> remove
> References: header completely. Put in your config something like this:
>
> acl_check_data:
> ...
> warn
> set acl_m_ref_len = ${strlen:$h_references:}
> condition = ${if > {$acl_m_ref_len}{998}}
> logwrite = References header has length=$acl_m_ref_len, removed
> remove_header = references

Yes, I applied the above check/removal of extremely long References:,
that's definitely is the lesser evil.

>> We have quite long "threads" in local mail archives, with really
>> many References: entries, but still not exceeding 998 characters per
>> line.
>
> Mail client is responsible for keeping References: withing reasonable
> number of entries and below line length limit.

During many a years (12+) that was the case. We have just recently
stumbled upon a recipient whose mail system doesn't behave well.

Thanks to everyone having responded, for ideas and explanations.

Sincerely,
Konstantin

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

On Tue, Apr 16, 2019 at 04:38:54PM +0100, Jeremy Harris via Exim-users wrote:

> The 998 limit is for de-folded lines. Go over that and you'd need
> multiple header lines; to do which you'd have to be duplicating
> header_names: as well as adding linebreaks. And yes, that would
> very likely break DKIM signatures.

That'd be a rather unexpected Exim behaviour. SMTP limits "physical"
lines to 1000 bytes (including the CRLF), but there is no such limit
on folded headers, and many headers are "single instance" and must
not be present more than once.

Postfix allows a *folded* header of up to 100k bytes by default,
and the limit is configurable. This is a per-logical-heade limit.

The overall header size is not limited, but IIRC more than 64KB of
headers total may not work once milters are configured, due to
limitations of the milter protocol.

--
Viktor.

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

On Thu, Apr 18, 2019 at 11:17 AM Viktor Dukhovni via Exim-users <
exim-users@exim.org> wrote:

> On Tue, Apr 16, 2019 at 04:38:54PM +0100, Jeremy Harris via Exim-users
> wrote:
>
> > The 998 limit is for de-folded lines. Go over that and you'd need
> > multiple header lines; to do which you'd have to be duplicating
> > header_names: as well as adding linebreaks. And yes, that would
> > very likely break DKIM signatures.
>
> That'd be a rather unexpected Exim behaviour. SMTP limits "physical"
> lines to 1000 bytes (including the CRLF), but there is no such limit
> on folded headers, and many headers are "single instance" and must
> not be present more than once.
>

It's expected *iff* you defold the header lines, which should not be done.

Does Exim actually defold multi-line headers and *then* impose the 1000
byte line length limit?

If so, that's a bug (imposing the per-line limit on the defolded header),
since it's no longer about message format processing.

--
Jan
--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/