Mailing List Archive

[Transport error]: message has lines too long for transport
Hi all

We are an italian hosting provider (https://host.it) and we use exim to relay e-mails from
our hosting servers.
Exim is installed and configured through the directadmin control panel
(https://www.directadmin.com), so the main configuration is managed using the
directadmin custombuild subsystem.

One week ago, we upgraded to exim 4.95 and suddenly, some customers (using microsoft
outlook, nonetheless), started to experience the following error for *some* of their
e-mails:

message has lines too long for transport
Reporting-MTA: dns; web017.shared.host.it

I received some examples of such e-mails from our customers service, and it appear that
the problem is with some badly formatted headers.

Unfortunately, we can't throw those customers out of the window :-) so we are searching
how to expand the line limits for the transports.
However, I'm struggling to understand, from the documentation, what the correct solution
is. From what I was able to understand, we should modify the remote_smtp and
remote_smtp_forward_transport sections to solve the problem, with the message_linelength_limit
parameter set to something like "4096".

Consider also, that the main exim configuration, is handled directly by directadmin and
it's generated from it's internal "templates". In general, we can safely change a configuration
file called /etc/exim.variables.conf.custom to put a series of overrides to the default
configuration.

Do we need to modify the directadmin templates or can we use one of the "custom" files
in /etc to achieve the same result?

Thank you and best regards.

Andrea

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: [Transport error]: message has lines too long for transport [ In reply to ]
On 18/11/2021 10:35, Andrea Biscuola via Exim-users wrote:
> One week ago, we upgraded to exim 4.95 and suddenly, some customers (using microsoft
> outlook, nonetheless), started to experience the following error for *some* of their
> e-mails:
>
> message has lines too long for transport
> Reporting-MTA: dns; web017.shared.host.it
>
> I received some examples of such e-mails from our customers service, and it appear that
> the problem is with some badly formatted headers.
>
> Unfortunately, we can't throw those customers out of the window :-) so we are searching
> how to expand the line limits for the transports.

Warning: you would be sending onward messages that are explicitly
exceeding the limits established by the relevant standards.

Any such might be lost (or worse, cause "unintended actions") on
other systems they traverse after yours.

Are you certain you want to be responsible for that?
You might wish to consult your legal staff.

> However, I'm struggling to understand, from the documentation, what the correct solution
> is. From what I was able to understand, we should modify the remote_smtp and
> remote_smtp_forward_transport sections to solve the problem, with the message_linelength_limit
> parameter set to something like "4096".

The smtp transport has a relevant option.

> Do we need to modify the directadmin templates or can we use one of the "custom" files
> in /etc to achieve the same result?

That is a matter for directadmin and your installation, and is not part of Exim.
--
Cheers,
Jeremy

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: [Transport error]: message has lines too long for transport [ In reply to ]
On 2021-11-18, Andrea Biscuola via Exim-users <exim-users@exim.org> wrote:
> Hi all
>
> We are an italian hosting provider (https://host.it) and we use exim to relay e-mails from
> our hosting servers.
> Exim is installed and configured through the directadmin control panel
> (https://www.directadmin.com), so the main configuration is managed using the
> directadmin custombuild subsystem.
>
> One week ago, we upgraded to exim 4.95 and suddenly, some customers (using microsoft
> outlook, nonetheless), started to experience the following error for *some* of their
> e-mails:

Outlook is not email software. Outlook is X.400 software with partial email
capability tacked on.

> message has lines too long for transport
> Reporting-MTA: dns; web017.shared.host.it
>
> I received some examples of such e-mails from our customers service, and it appear that
> the problem is with some badly formatted headers.

> Unfortunately, we can't throw those customers out of the window :-) so we are searching
> how to expand the line limits for the transports.

Turn off 8BITMIME. that may be sufficient to prevent this outlook bug.
perhaps configure two different submission ports one with 8BITMIME
disabled for the benefit of those afflicted with outlook and one with
all the features enabled.

> However, I'm struggling to understand, from the documentation, what the correct solution
> is. From what I was able to understand, we should modify the remote_smtp and
> remote_smtp_forward_transport sections to solve the problem, with the message_linelength_limit
> parameter set to something like "4096".

The important documentation is here:
https://datatracker.ietf.org/doc/html/rfc5322#section-2.1.1

> Consider also, that the main exim configuration, is handled directly by directadmin and
> it's generated from it's internal "templates". In general, we can safely change a configuration
> file called /etc/exim.variables.conf.custom to put a series of overrides to the default
> configuration.
>
> Do we need to modify the directadmin templates or can we use one of the "custom" files
> in /etc to achieve the same result?

Increasing that limit will break SMTP and will probably just result in the refusals
taking longer to reach the sender.

--
Jasen.

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: [Transport error]: message has lines too long for transport [ In reply to ]
On Thu, 18 Nov 2021 21:03:43 -0000 (UTC)
Jasen Betts via Exim-users <exim-users@exim.org> wrote:

> On 2021-11-18, Andrea Biscuola via Exim-users <exim-users@exim.org> wrote:
> > Hi all
> >
> > We are an italian hosting provider (https://host.it) and we use exim to relay e-mails from
> > our hosting servers.
> > Exim is installed and configured through the directadmin control panel
> > (https://www.directadmin.com), so the main configuration is managed using the
> > directadmin custombuild subsystem.
> >
> > One week ago, we upgraded to exim 4.95 and suddenly, some customers (using microsoft
> > outlook, nonetheless), started to experience the following error for *some* of their
> > e-mails:
>
> Outlook is not email software. Outlook is X.400 software with partial email
> capability tacked on.

We all agree on that.

>
> > message has lines too long for transport
> > Reporting-MTA: dns; web017.shared.host.it
> >
> > I received some examples of such e-mails from our customers service, and it appear that
> > the problem is with some badly formatted headers.
>
> > Unfortunately, we can't throw those customers out of the window :-) so we are searching
> > how to expand the line limits for the transports.
>
> Turn off 8BITMIME. that may be sufficient to prevent this outlook bug.
> perhaps configure two different submission ports one with 8BITMIME
> disabled for the benefit of those afflicted with outlook and one with
> all the features enabled.

On another thread it was mentioned that it could be possible to fold the "References"
header, splitting it on multiple lines.

I think it could be a better solution than increasing the limit, would that work?

>
> > However, I'm struggling to understand, from the documentation, what the correct solution
> > is. From what I was able to understand, we should modify the remote_smtp and
> > remote_smtp_forward_transport sections to solve the problem, with the message_linelength_limit
> > parameter set to something like "4096".
>
> The important documentation is here:
> https://datatracker.ietf.org/doc/html/rfc5322#section-2.1.1
>

I already checked the relevant RFC, thanks.

> > Consider also, that the main exim configuration, is handled directly by directadmin and
> > it's generated from it's internal "templates". In general, we can safely change a configuration
> > file called /etc/exim.variables.conf.custom to put a series of overrides to the default
> > configuration.
> >
> > Do we need to modify the directadmin templates or can we use one of the "custom" files
> > in /etc to achieve the same result?
>
> Increasing that limit will break SMTP and will probably just result in the refusals
> taking longer to reach the sender.
>

Andrea

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: [Transport error]: message has lines too long for transport [ In reply to ]
(Adding to this thread because we only just now hit this issue …)

> On Nov 18, 2021, at 15:00, Jeremy Harris via Exim-users <exim-users@exim.org> wrote:
>
> On 18/11/2021 10:35, Andrea Biscuola via Exim-users wrote:
>> One week ago, we upgraded to exim 4.95 and suddenly, some customers (using microsoft
>> outlook, nonetheless), started to experience the following error for *some* of their
>> e-mails:
>> message has lines too long for transport
>> Reporting-MTA: dns; web017.shared.host.it
>> I received some examples of such e-mails from our customers service, and it appear that
>> the problem is with some badly formatted headers.
>> Unfortunately, we can't throw those customers out of the window :-) so we are searching
>> how to expand the line limits for the transports.
>
> Warning: you would be sending onward messages that are explicitly
> exceeding the limits established by the relevant standards.
>
> Any such might be lost (or worse, cause "unintended actions") on
> other systems they traverse after yours.
>
> Are you certain you want to be responsible for that?
> You might wish to consult your legal staff.

RFC 5322 doesn’t stipulate server behaviour upon receipt of a message that violates this limitation.

On the contrary, the same section (2.1.1) that gives the 998-character limit adds, in the next paragraph:

----
Receiving implementations would do well to handle an arbitrarily large number of characters in a line for robustness sake.
----

While Exim’s new default behaviour is reasonable, it doesn’t seem to follow from the relevant standard. Outlook is in violation, to be sure, but servers that accept arbitrarily long lines don’t seem to be (by that virtue alone, anyhow).

-FG

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: [Transport error]: message has lines too long for transport [ In reply to ]
On 18.11.2021 21:00, Jeremy Harris via Exim-users wrote:> On 18/11/2021
10:35, Andrea Biscuola via Exim-users wrote:
>> is. From what I was able to understand, we should modify the
>> remote_smtp and
>> remote_smtp_forward_transport sections to solve the problem, with the
>> message_linelength_limit
>> parameter set to something like "4096".
>
> The smtp transport has a relevant option.

I just hit the same problem as Andrea. So

- I wonder why did you decide that the limit on incoming mails can
still be defined in configuration, but for outgoing mails it must be
forced in the code?

- What do you think about implementing re-folding headers to make mail
RFC-compliant again (ie. fix what outlook spolied)?

- I guess that with the new check you could update RCF.conform

https://github.com/Exim/exim/blob/a1108b5118d32e969c5fe91b2110944f7483a7cb/doc/doc-misc/RFC.conform#L53

best regards
--
Marcin Gryszkalis, PGP 0xA5DBEEC7 http://fork.pl/gpg.txt

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: [Transport error]: message has lines too long for transport [ In reply to ]
On Thu, 27 Jan 2022, Marcin Gryszkalis via Exim-users wrote:

> On 18.11.2021 21:00, Jeremy Harris via Exim-users wrote:> On 18/11/2021
> 10:35, Andrea Biscuola via Exim-users wrote:
>>> is. From what I was able to understand, we should modify the
>>> remote_smtp and
>>> remote_smtp_forward_transport sections to solve the problem, with the
>>> message_linelength_limit
>>> parameter set to something like "4096".
>>
>> The smtp transport has a relevant option.
>
> I just hit the same problem as Andrea. So
>
> - I wonder why did you decide that the limit on incoming mails can still be
> defined in configuration, but for outgoing mails it must be forced in the
> code?
>
> - What do you think about implementing re-folding headers to make mail
> RFC-compliant again (ie. fix what outlook spolied)?

Unfortunately that can break DKIM signatures.

> - I guess that with the new check you could update RCF.conform
>
> https://github.com/Exim/exim/blob/a1108b5118d32e969c5fe91b2110944f7483a7cb/doc/doc-misc/RFC.conform#L53
>
> best regards
> --
> Marcin Gryszkalis, PGP 0xA5DBEEC7 http://fork.pl/gpg.txt
>
> --
> ## List details at https://lists.exim.org/mailman/listinfo/exim-users
> ## Exim details at http://www.exim.org/
> ## Please use the Wiki with this list - http://wiki.exim.org/
>

--
Andrew C. Aitchison Kendal, UK
andrew@aitchison.me.uk

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: [Transport error]: message has lines too long for transport [ In reply to ]
On 27/01/2022 18:41, Marcin Gryszkalis via Exim-users wrote:
> On 18.11.2021 21:00, Jeremy Harris via Exim-users wrote:> On 18/11/2021 10:35, Andrea Biscuola via Exim-users wrote:
> >> is. From what I was able to understand, we should modify the
> >> remote_smtp and
> >> remote_smtp_forward_transport sections to solve the problem, with the
> >> message_linelength_limit
> >> parameter set to something like "4096".
> >
> > The smtp transport has a relevant option.
>
> I just hit the same problem as Andrea. So
>
>  - I wonder why did you decide that the limit on incoming mails can still be defined in configuration,

I'm not sure where you're looking?

Certainly there is a variable with the detected max line length of
a message being handled; you can code ACL to check it and reject
the message if you regard it as overlarge.

> but for outgoing mails it must be forced in the code?

... nor what you mean by "forced in the code", given that
the smtp transport has an option, which you can set in your
configuration, and will be used for outgoing mails sent by that
transport.

Both ACL and transport options are part of your configuration.
--
Cheers,
Jeremy

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: [Transport error]: message has lines too long for transport [ In reply to ]
On 27/01/2022 19:31, Andrew C Aitchison via Exim-users wrote:
>>
>> - What do you think about implementing re-folding headers to make mail RFC-compliant again (ie. fix what outlook spolied)?
>
> Unfortunately that can break DKIM signatures.

Obviously it would have to be optional, and with appropriate
documentation warnings. But it's a Simple Matter Of Proagramming,
right? And documentation, and testcase development.

Volunteers welcome.
--
Cheers,
Jeremy

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: [Transport error]: message has lines too long for transport [ In reply to ]
On 2022-01-27 at 14:31:41 UTC-0500 (Thu, 27 Jan 2022 19:31:41 +0000
(GMT))
Andrew C Aitchison via Exim-users <andrew@aitchison.me.uk>
is rumored to have said:

> On Thu, 27 Jan 2022, Marcin Gryszkalis via Exim-users wrote:
[...]

>> - What do you think about implementing re-folding headers to make
>> mail RFC-compliant again (ie. fix what outlook spolied)?
>
> Unfortunately that can break DKIM signatures.

True, but only if the signer is using 'simple' header canonicalization,
in which case they surely must expect and so deserve to have their
signatures broken.

Yes, I'm MOSTLY serious.

--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: [Transport error]: message has lines too long for transport [ In reply to ]
On Fri, 28 Jan 2022, Bill Cole via Exim-users wrote:

> On 2022-01-27 at 14:31:41 UTC-0500 (Thu, 27 Jan 2022 19:31:41 +0000 (GMT))
> Andrew C Aitchison via Exim-users <andrew@aitchison.me.uk>
> is rumored to have said:
>
>> On Thu, 27 Jan 2022, Marcin Gryszkalis via Exim-users wrote:
> [...]
>
>>> - What do you think about implementing re-folding headers to make mail
>>> RFC-compliant again (ie. fix what outlook spolied)?
>>
>> Unfortunately that can break DKIM signatures.
>
> True, but only if the signer is using 'simple' header canonicalization, in
> which case they surely must expect and so deserve to have their signatures
> broken.
>
> Yes, I'm MOSTLY serious.

Ah.
Now I have my head around header canonicalization I agree and am relieved;
we don't need to worry too much about breaking DKIM.

Would we need a setting to say whether to break or reject messages with
simple header canonicalization and long headers, or is it clear which
we should do ?

--
Andrew C. Aitchison Kendal, UK
andrew@aitchison.me.uk

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: [Transport error]: message has lines too long for transport [ In reply to ]
On 2022-01-29 at 05:26:21 UTC-0500 (Sat, 29 Jan 2022 10:26:21 +0000
(GMT))
Andrew C Aitchison via Exim-users <andrew@aitchison.me.uk>
is rumored to have said:

> On Fri, 28 Jan 2022, Bill Cole via Exim-users wrote:
>
>> On 2022-01-27 at 14:31:41 UTC-0500 (Thu, 27 Jan 2022 19:31:41 +0000
>> (GMT))
>> Andrew C Aitchison via Exim-users <andrew@aitchison.me.uk>
>> is rumored to have said:
>>
>>> On Thu, 27 Jan 2022, Marcin Gryszkalis via Exim-users wrote:
>> [...]
>>
>>>> - What do you think about implementing re-folding headers to make
>>>> mail RFC-compliant again (ie. fix what outlook spolied)?
>>>
>>> Unfortunately that can break DKIM signatures.
>>
>> True, but only if the signer is using 'simple' header
>> canonicalization, in which case they surely must expect and so
>> deserve to have their signatures broken.
>>
>> Yes, I'm MOSTLY serious.
>
> Ah.
> Now I have my head around header canonicalization I agree and am
> relieved;
> we don't need to worry too much about breaking DKIM.
>
> Would we need a setting to say whether to break or reject messages
> with simple header canonicalization and long headers, or is it clear
> which we should do ?

I think the correct approach is to go ahead and fold the header, maybe
add a header noting that 'break' (like Sendmail's X-Autoconverted
header) and expose the fact that DKIM is generally a garbage mechanism.
Rejecting mail simply because it has a bad DKIM signature or an overlong
line is a great way to reject otherwise legitimate mail in a
semi-chaotic pattern.

IMHO, DKIM canonicalizations are primary evidence of the standard being
mis-designed; "simple" is positively ridiculous for mail moving outside
of unified administrative realms (where DKIM serves what purpose?) and
'relaxed' is inadequate to its supposed purpose. It feels like DKIM was
devised by people who didn't actually work with real-world mail systems.
At all. It seems to me that DKIM needs *at least* a new default
canonicalization defined, and the existing canonicalizations should be
deprecated.

--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/