https://bugs.exim.org/show_bug.cgi?id=3075
Bug ID: 3075
Summary: base64 content LF converted to space when starting
line ends with CRLF
Product: Exim
Version: 4.96+
Hardware: x86-64
OS: Linux
Status: NEW
Severity: bug
Priority: medium
Component: Transports
Assignee: unallocated@exim.org
Reporter: support@qthhosting.com
CC: exim-dev@lists.exim.org
Created attachment 1468
--> https://bugs.exim.org/attachment.cgi?id=1468&action=edit
Email containing the long single line base64 attachment with LF replaced with
spaces
Submitted bug to cPanel and they asked that I submit this to Exim. As of Exim
4.96.2, client's camera notification emails are rejected with "lines too long
for transport". The cPanel tech has traced it down to the following.
Exim had a recent CVE published regarding a potential "SMTP Smuggling" security
issue as outlined here: https://nvd.nist.gov/vuln/detail/CVE-2023-51766
This has changed how Exim is parsing messages based on the line endings that
are being used:
https://www.exim.org/exim-html-current/doc/html/spec_html/ch-message_processing.html
* If the first header line received in a message ends with CRLF, a subsequent
bare LF in a header line is treated in the same way as a bare CR in a header
line and a bare LF in a body line is replaced with a space.
This change was introduced in Exim 4.71, however, we backported this specific
fix to 4.96 to patch our offering of Exim against CVE-2023-51766.
Essentially, when the first newline in a message ends with a "CRLF" symbol,
then any lines in the body that end with "LF" are converted into a space
instead of being treated as a newline as well. Checking the message you
provided with the name "broken-with-long-line.eml" (attached here), this
appears to be what is occurring as the base64 content is as such:
----SNIP----
--====my_split_tag====
Content-Type: image/jpg;
name="=?utf-8?b?TW90aW9uX0NIMDJfMjAyNDAyMTRfMTc0NDU0LmpwZw==?="
Content-Transfer-Encoding: base64
/9j/4AAQSkZJRgABAQAAAQABAAD/2wDFAAMCAgMCAgMDAwMEAwMEBQgFBQQEBQoHBwYIDAoM
DAsKCwsNDhIQDQ4RDgsLEBYQERMUFRUVDA8XGBYUGBIUFRQBAwQEBQQFCQUFCRQNCw0UFBQU
FBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFAIDBAQFBAUJ
BQUJFA0LDRQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQU
----SNIP----
If you notice, there are spaces in between the batch of base64 characters in
the above. The message that was received by Microsoft has these separated by
newlines:
----SNIP----
--====my_split_tag====
Content-Type: image/jpg;
name="=?utf-8?b?TW90aW9uX0NIMDJfMjAyNDAyMTRfMjIwMjMxLmpwZw==?="
Content-Transfer-Encoding: base64
/9j/4AAQSkZJRgABAQAAAQABAAD/2wDFAAMCAgMCAgMDAwMEAwMEBQgFBQQEBQoHBwYIDAoMDAsK
CwsNDhIQDQ4RDgsLEBYQERMUFRUVDA8XGBYUGBIUFRQBAwQEBQQFCQUFCRQNCw0UFBQUFBQUFBQU
FBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFAIDBAQFBAUJBQUJFA0LDRQU
FBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQU/8AAEQgEMAeA
----SNIP----
I can confirm that your message does begin with the CRLF line ending:
----SNIP----
# file broken-with-long-line.eml
Downloads/broken-with-long-line.eml: SMTP mail text, ASCII text, with very long
lines (64298), with CRLF line terminators
----SNIP----
This confirms this as well:
----SNIP----
# cat -e valid-sent-by-microsoft.eml| head
Return-Path: <redacted@outlook.com>^M$
Delivered-To: redacted@clientdomain.com^M$
Received: from mail.serverdomain.com^M$
----SNIP----
The ^M$ is the CRLF symbol; while an LF symbol is just $
Based on this, due to a recent change in Exim to patch this CVE-2023-51766,
your messages are starting with the CRLF line ending while the base64 content
(the image) is being ended with an LF symbol for each line of its output. This
is causing Exim to convert each of the LF line endings to just a space, and is
therefore treating the entire base64 output as a single line in the message.
This then causes the "Message has lines too long" error as the full base64
output is being treated as a single line.
Here is the specific commit that is changing this behavior in Exim:
https://github.com/Exim/exim/commit/5bb786d5ad568a88d50d15452aacc8404047e5ca
The following code is what this commit changed:
----SNIP----
if (ch == '\n' && first_line_ended_crlf == TRUE /* and not TRUE_UNSET */ )
/* dot, LF but we are in CRLF mode. Attack? */
ch = ' '; /* replace the LF with a space */
else if (ch == '\r')
----SNIP----
As you can see, the LF characters are being replaced with a space when in
"CRLF" mode. The result is the email can no longer be sent due to 'Lines too
long for transport'.
--
You are receiving this mail because:
You are on the CC list for the bug.
--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Bug ID: 3075
Summary: base64 content LF converted to space when starting
line ends with CRLF
Product: Exim
Version: 4.96+
Hardware: x86-64
OS: Linux
Status: NEW
Severity: bug
Priority: medium
Component: Transports
Assignee: unallocated@exim.org
Reporter: support@qthhosting.com
CC: exim-dev@lists.exim.org
Created attachment 1468
--> https://bugs.exim.org/attachment.cgi?id=1468&action=edit
Email containing the long single line base64 attachment with LF replaced with
spaces
Submitted bug to cPanel and they asked that I submit this to Exim. As of Exim
4.96.2, client's camera notification emails are rejected with "lines too long
for transport". The cPanel tech has traced it down to the following.
Exim had a recent CVE published regarding a potential "SMTP Smuggling" security
issue as outlined here: https://nvd.nist.gov/vuln/detail/CVE-2023-51766
This has changed how Exim is parsing messages based on the line endings that
are being used:
https://www.exim.org/exim-html-current/doc/html/spec_html/ch-message_processing.html
* If the first header line received in a message ends with CRLF, a subsequent
bare LF in a header line is treated in the same way as a bare CR in a header
line and a bare LF in a body line is replaced with a space.
This change was introduced in Exim 4.71, however, we backported this specific
fix to 4.96 to patch our offering of Exim against CVE-2023-51766.
Essentially, when the first newline in a message ends with a "CRLF" symbol,
then any lines in the body that end with "LF" are converted into a space
instead of being treated as a newline as well. Checking the message you
provided with the name "broken-with-long-line.eml" (attached here), this
appears to be what is occurring as the base64 content is as such:
----SNIP----
--====my_split_tag====
Content-Type: image/jpg;
name="=?utf-8?b?TW90aW9uX0NIMDJfMjAyNDAyMTRfMTc0NDU0LmpwZw==?="
Content-Transfer-Encoding: base64
/9j/4AAQSkZJRgABAQAAAQABAAD/2wDFAAMCAgMCAgMDAwMEAwMEBQgFBQQEBQoHBwYIDAoM
DAsKCwsNDhIQDQ4RDgsLEBYQERMUFRUVDA8XGBYUGBIUFRQBAwQEBQQFCQUFCRQNCw0UFBQU
FBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFAIDBAQFBAUJ
BQUJFA0LDRQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQU
----SNIP----
If you notice, there are spaces in between the batch of base64 characters in
the above. The message that was received by Microsoft has these separated by
newlines:
----SNIP----
--====my_split_tag====
Content-Type: image/jpg;
name="=?utf-8?b?TW90aW9uX0NIMDJfMjAyNDAyMTRfMjIwMjMxLmpwZw==?="
Content-Transfer-Encoding: base64
/9j/4AAQSkZJRgABAQAAAQABAAD/2wDFAAMCAgMCAgMDAwMEAwMEBQgFBQQEBQoHBwYIDAoMDAsK
CwsNDhIQDQ4RDgsLEBYQERMUFRUVDA8XGBYUGBIUFRQBAwQEBQQFCQUFCRQNCw0UFBQUFBQUFBQU
FBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFAIDBAQFBAUJBQUJFA0LDRQU
FBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQU/8AAEQgEMAeA
----SNIP----
I can confirm that your message does begin with the CRLF line ending:
----SNIP----
# file broken-with-long-line.eml
Downloads/broken-with-long-line.eml: SMTP mail text, ASCII text, with very long
lines (64298), with CRLF line terminators
----SNIP----
This confirms this as well:
----SNIP----
# cat -e valid-sent-by-microsoft.eml| head
Return-Path: <redacted@outlook.com>^M$
Delivered-To: redacted@clientdomain.com^M$
Received: from mail.serverdomain.com^M$
----SNIP----
The ^M$ is the CRLF symbol; while an LF symbol is just $
Based on this, due to a recent change in Exim to patch this CVE-2023-51766,
your messages are starting with the CRLF line ending while the base64 content
(the image) is being ended with an LF symbol for each line of its output. This
is causing Exim to convert each of the LF line endings to just a space, and is
therefore treating the entire base64 output as a single line in the message.
This then causes the "Message has lines too long" error as the full base64
output is being treated as a single line.
Here is the specific commit that is changing this behavior in Exim:
https://github.com/Exim/exim/commit/5bb786d5ad568a88d50d15452aacc8404047e5ca
The following code is what this commit changed:
----SNIP----
if (ch == '\n' && first_line_ended_crlf == TRUE /* and not TRUE_UNSET */ )
/* dot, LF but we are in CRLF mode. Attack? */
ch = ' '; /* replace the LF with a space */
else if (ch == '\r')
----SNIP----
As you can see, the LF characters are being replaced with a space when in
"CRLF" mode. The result is the email can no longer be sent due to 'Lines too
long for transport'.
--
You are receiving this mail because:
You are on the CC list for the bug.
--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@lists.exim.org
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/