Mailing List Archive

[Bug 2886] Crashes in SMTP delivery attempt following a deferral
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #2 from Gedalya <gedalya@gedalya.net> ---
The deferrals are either in response to RCPT TO (gmail over quota) or post DATA
(suspicious content)

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #1 from Gedalya <gedalya@gedalya.net> ---
Created attachment 1415
--> https://bugs.exim.org/attachment.cgi?id=1415&action=edit
backtrace for crash during ARC signing

Adding a backtrace for crash in arc_sign()

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #3 from Jeremy Harris <jgh146exb@wizmail.org> ---
This needs following up; we can't trust that bt

> warning: core file may not match specified executable file.

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #4 from Gedalya <gedalya@gedalya.net> ---
If I go out of my way to invoke "/usr/sbin/exim4 -q" when causing the crash,
that message is not displayed.
On Debian, /usr/sbin/exim -> exim4. One naturally uses "exim -q", and gdb gives
that message, every single time. The rest of the bt seems unaffected.

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #5 from Gedalya <gedalya@gedalya.net> ---
Either way I think this should be very easily reproducible, I've tried several
boxes, several custom builds, with or without ARC, DMARC, MySQL and so on, exim
4.95 and 4.96, the only thing in common was that I was using Debian's packaging
and stuck with gnutls.

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #6 from Jeremy Harris <jgh146exb@wizmail.org> ---
I wonder if your build is failing to null-fill not-specifically-initialized
file-scope statics?

#0 0x00005620ff359503 in arc_sign (signspec=<optimized out>,
sigheaders=0x5621001fc580, errstr=errstr@entry=0x7ffd081d0980) at
./b-exim4-daemon-custom/build-Linux-x86_64/arc.c:1663

1660 if ((rheaders = arc_sign_scan_headers(&arc_sign_ctx, sigheaders)))
1661 {
1662 hdr_rlist ** rp;
1663 for (rp = &headers_rlist; *rp; ) rp = &(*rp)->prev;
1664 *rp = rheaders;
1665 }


What do "p *rp" and "p headers_rlist" say for that core?
(On that theory, an "=NULL" at line 93 would help. But only for the arcsigning
case).

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #7 from Gedalya <gedalya@gedalya.net> ---
I didn't keep that exact same core, but I have a new one that seems equivalent.

#0 0x0000559e51e99993 in arc_sign (signspec=<optimized out>,
sigheaders=0x559e529bb580, errstr=errstr@entry=0x7ffcc0624e20) at
./b-exim4-daemon-custom/build-Linux-x86_64/arc.c:1663
rp = 0x672e6c69616d3d48
identity = 0x559e529bb8a8 "****.com"
selector = 0x559e529bb8c8 "arc-20220506"
privkey = 0x559e525ad2f8 "-----BEGIN PRIVATE KEY-----\n"...
opts = 0x559e529bb92a ""
s = <optimized out>
options = <optimized out>
sep = 58
headers = <optimized out>
rheaders = 0x559e529bb950
ar = {data = <optimized out>, len = <optimized out>}
instance = <optimized out>
g = 0x0
b = <optimized out>
__FUNCTION__ = "arc_sign"
ret_sigheaders = <optimized out>


(gdb) p *rp
Cannot access memory at address 0x672e6c69616d3d48
(gdb) p headers_rlist
$1 = (hdr_rlist *) 0x559e52b48d78

Does this help?
Please talk to me as you would to a little child, as you find necessary :-)

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #8 from Jeremy Harris <jgh146exb@wizmail.org> ---
See if you can repro with that NULL-init.

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #9 from Gedalya <gedalya@gedalya.net> ---
--- exim4-4.96~RC0.orig/src/arc.c
+++ exim4-4.96~RC0/src/arc.c
@@ -90,7 +90,7 @@ typedef struct arc_ctx {

static time_t now;
static time_t expire;
-static hdr_rlist * headers_rlist;
+static hdr_rlist * headers_rlist = NULL;
static arc_ctx arc_sign_ctx = { NULL };
static arc_ctx arc_verify_ctx = { NULL };


Core was generated by `/usr/sbin/exim4 -q'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00005599b240c993 in arc_sign (signspec=<optimized out>,
sigheaders=0x5599b2fea580, errstr=errstr@entry=0x7ffeafba7af0) at
./b-exim4-daemon-custom/build-Linux-x86_64/arc.c:1663
1663 ./b-exim4-daemon-custom/build-Linux-x86_64/arc.c: No such file or
directory.
(gdb) set pagination off
(gdb) bt full
#0 0x00005599b240c993 in arc_sign (signspec=<optimized out>,
sigheaders=0x5599b2fea580, errstr=errstr@entry=0x7ffeafba7af0) at
./b-exim4-daemon-custom/build-Linux-x86_64/arc.c:1663
rp = 0x672e6c69616d3d48
identity = 0x5599b2fea8a8 "****.com"
selector = 0x5599b2fea8c8 "arc-20220506"
privkey = 0x5599b2bdc2f8 "-----BEGIN PRIVATE KEY-----\n"...
opts = 0x5599b2fea92a ""
s = <optimized out>
options = <optimized out>
sep = 58
headers = <optimized out>
rheaders = 0x5599b2fea950
ar = {data = <optimized out>, len = <optimized out>}
instance = <optimized out>
g = 0x0
b = <optimized out>
__FUNCTION__ = "arc_sign"
ret_sigheaders = <optimized out>

(gdb) p *rp
Cannot access memory at address 0x672e6c69616d3d48
(gdb) p headers_rlist
$1 = (hdr_rlist *) 0x5599b3177d78

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #10 from Gedalya <gedalya@gedalya.net> ---
Some more details on the ARC issue.

If I queue the message with "-oMa 25.25.25.25 -odq", the queue runner crashes
as follows which seems to be the "normal" and "correct" way to crash, as would
real messages coming from the Internet.

#0 0x00007f56f3ac1af9 in gnutls_x509_trust_list_deinit (list=0x55876d7fe0b0,
all=1) at ../../../lib/x509/verify-high.c:213
i = <optimized out>
j = 0
#1 0x00007f56f3a300cb in gnutls_certificate_free_credentials
(sc=0x55876d7ff2b0) at ../../lib/cert-cred.c:403
No locals.
#2 0x000055876bf9c1c7 in tls_close (ct_ctx=0x55876d5701a0,
do_shutdown=do_shutdown@entry=2) at
./b-exim4-daemon-custom/build-Linux-x86_64/tls-gnu.c:3777
state = 0x55876d5701a0
tlsp = 0x55876c02e3e0 <tls_out>
__FUNCTION__ = "tls_close"
#3 0x000055876bfc9c79 in smtp_deliver (addrlist=addrlist@entry=0x55876d55f988,
host=host@entry=0x55876dac5838, host_af=host_af@entry=2, defport=<optimized
out>, interface=<optimized out>, tblock=tblock@entry=0x55876d56f4f8,
message_defer=<optimized out>, suppress_tls=<optimized out>) at
./b-exim4-daemon-custom/build-Linux-x86_64/transports/smtp.c:4850
n = <optimized out>
ob = <optimized out>
yield = <optimized out>
save_errno = 1812207714
rc = <optimized out>
message = 0x0
new_message_id =
"\360+sJ\376\177\000\000\000\000\000\000\000\000\000\000@"
sx = 0x55876da3a130
__FUNCTION__ = "smtp_deliver"
pass_message = 0
dane_held = <optimized out>
tcw_done = 1
tcw = 0
SEND_MESSAGE = <optimized out>

The ARC crash happens because I omit -oMa 25.25.25.25 and the message is thus
locally submitted. Per the configuration, DKIM signing occurs (which is not
what would normally happen). The message injected (locally or with -oMa) does
not have any Authentication-Results header.

2022-05-12 16:49:21 1npC09-0007sc-1D <= ***@***.com U=root P=local S=4573
id=***
2022-05-12 16:49:46 1npC09-0007sc-1D ARC: no Authentication-Results header for
signing
2022-05-12 16:49:46 1npC09-0007sc-1D H=mail.gedalya.net [******]: SMTP error
from remote mail server after pipelined end of data: 451 Temporary local
problem - please try later
2022-05-12 16:49:46 1npC09-0007sc-1D H=mx2.gedalya.net [******] Network is
unreachable
2022-05-12 16:49:46 1npC09-0007sc-1D SIGSEGV (fault address: (nil))
2022-05-12 16:49:46 1npC09-0007sc-1D SIGSEGV (null pointer indirection)
2022-05-12 16:49:46 1npC09-0007sc-1D SIGSEGV (30343 delivering 1npC09-0007sc-1D
to mx2.gedalya.net [****] (gedalya@gedalya.net)
2022-05-12 16:49:46 1npC09-0007sc-1D Delivery status for gedalya@gedalya.net:
got 0 of 7 bytes (pipeheader) from transport process 30343 for transport smtp
2022-05-12 16:49:46 1npC09-0007sc-1D == gedalya@gedalya.net R=dnslookup
T=remote_smtp defer (-1): smtp transport process returned non-zero status
0x008b: terminated by signal 11

Running exim -q -d :

[attempt first server]
...
ARC: requesting bodyhash
DKIM: new bodyhash sha256/simple/-1
dkim signing direct-mode
...
GnuTLS<3>: ASSERT: ../../../lib/nettle/mpi.c[wrap_nettle_mpi_print]:60
DKIM [***.com] b computed: xx....xx
ARC: sign for ****.com
LOG: MAIN
ARC: no Authentication-Results header for signing
SMTP+> BDAT 5360 LAST

[attempt second server]
ARC: requesting bodyhash
DKIM: new bodyhash sha256/simple/-1
dkim signing direct-mode
...
GnuTLS<3>: ASSERT: ../../../lib/nettle/mpi.c[wrap_nettle_mpi_print]:60
DKIM [swdhost.com] b computed: xx....xx
ARC: sign for ****.com
LOG: MAIN PANIC
SIGSEGV (fault address: (nil))
LOG: MAIN PANIC
SIGSEGV (null pointer indirection)
LOG: MAIN PANIC
SIGSEGV (30594 delivering 1npC09-0007sc-1D to mx2.gedalya.net [*****]
(gedalya@gedalya.net)
)
reading pipe for subprocess 30594 (not ended yet)
expect 7 bytes (pipeheader) from tpt process 30594
LOG: MAIN PANIC
Delivery status for gedalya@gedalya.net: got 0 of 7 bytes (pipeheader) from
transport process 30594 for transport smtp

When using -oMa, "ARC: no Authentication-Results header for signing" appears
for both attempts in the message log.

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #11 from Jeremy Harris <jgh146exb@wizmail.org> ---
Please say exactly what the commandline and the message headers submitted
were, for the non-oMa case.

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #12 from Gedalya <gedalya@gedalya.net> ---
# exim -odq -f deferme@example.com gedalya@gedalya.net < 2.msg

# cat 2.msg
Subject: test

this is a test

# exim -q 1npxYE-0005R6-18 1npxYE-0005R6-18
2022-05-14 19:35:58 1npxYE-0005R6-18 SIGSEGV (fault address: 0x402a)
2022-05-14 19:35:58 1npxYE-0005R6-18 SEGV_MAPERR
2022-05-14 19:35:58 1npxYE-0005R6-18 SIGSEGV (maybe attempt to write to
immutable memory)
2022-05-14 19:35:58 1npxYE-0005R6-18 SIGSEGV (20909 delivering 1npxYE-0005R6-18
to mx2.gedalya.net [***] (gedalya@gedalya.net)
)
2022-05-14 19:35:58 1npxYE-0005R6-18 Delivery status for gedalya@gedalya.net:
got 0 of 7 bytes (pipeheader) from transport process 20909 for transport smtp

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #13 from Jeremy Harris <jgh146exb@wizmail.org> ---
Works for me.

Testsuite script:

exim -odq -f deferme@example.com gedalya@gedalya.net
Subject: test

this is a test

****
exim -d+all -q $msg1
****


Debug output section:

21:04:16 20777 will pipeline QUIT
21:04:16 20777 dkim signing direct-mode
21:04:16 20777 DKIM >> Body data for hash, canonicalized
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
21:04:16 20777 this{SP}is{SP}a{SP}test{CR}{LF}
21:04:16 20777 DKIM
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
21:04:16 20777 DKIM: finish bodyhash sha256/simple/-1 len 16
21:04:16 20777 DKIM: no signatures
21:04:16 20777 DKIM: no signatures to use
21:04:16 20777 ARC: sign for test.ex
21:04:16 20777 LOG: MAIN
21:04:16 20777 ARC: no Authentication-Results header for signing
21:04:16 20777 SMTP+> BDAT 332 LAST
21:04:16 20777 cmd buf flush 86 bytes (more expected)
21:04:16 20777 cannot use sendfile for body: spoolfile not wireformat
21:04:16 20777 writing data block fd=8 size=332 timeout=300 (more expected)
21:04:16 20777 SMTP+> QUIT
21:04:16 20777 cmd buf flush 6 bytes (more expected)
21:04:16 20777 SMTP(shutdown)>>
21:04:16 20777 sync_responses expect mail
21:04:16 20777 read response data: size=114
21:04:16 20777 SMTP<< 250 OK
21:04:16 20777 sync_responses expect rcpt for gedalya@gedalya.net
21:04:16 20777 SMTP<< 250 Accepted
21:04:16 20777 SMTP<< 250- 332 byte chunk, total 332
21:04:16 20777 250 OK id=1npxzs-0005P8-27
21:04:16 20777 S:journalling gedalya@gedalya.net
21:04:16 20777 ok=1 send_quit=0 send_rset=0 continue_more=0 yield=0
first_address is NULL
21:04:16 20777 SMTP<< 221 test.ex closing connection
21:04:16 20777 SMTP(close)>>
21:04:16 20777 cmdlog: '220:EHLO:250-:MAIL|:RCPT|:BDAT:QUIT:250:250:250-:221'
21:04:16 20777 set_process_info: 20777 delivering 1npxzr-0005Oz-2L: just tried
127.0.0.1 [127.0.0.1]:1225 for gedalya@gedalya.net: result OK
21:04:16 20777 Leaving tsmtp transport
21:04:16 20777 set_process_info: 20777 delivering 1npxzr-0005Oz-2L (just run
tsmtp for gedalya@gedalya.net in subprocess)

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #14 from Gedalya <gedalya@gedalya.net> ---
It worked for you first of all in the sense that the remote party did not
defer, which makes the test irrelevant.

The remote party which accepted your message does not seem to be my server, nor
was "deferme@example.com" the *exact* sender address which my ACL is configured
to defer, despite your request to be exact, but that has been added now.

In either case I really did mean it when I said this bug is triggered by the
first remote server responding with a deferral and you're now more than welcome
to test against my own servers by sending a message from deferme@example.com to
gedalya@gedalya.net,

$ dig +short gedalya.net mx
10 mail.gedalya.net. <-- will defer
20 mx2.gedalya.net. <-- will accept, but the sending queue runner should
crash

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #15 from Jeremy Harris <jgh146exb@wizmail.org> ---
Still can't duplicate locally.

The first conn goes
cmdlog:
'220:EHLO:250-:STARTTLS:220:EHLO:250-:MAIL|:RCPT|:BDAT:QUIT:250:451:503-:221'

The second:
22:48:10 25726 will pipeline QUIT
22:48:10 25726 dkim signing direct-mode
22:48:10 25726 DKIM >> Body data for hash, canonicalized
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
22:48:10 25726 this{SP}is{SP}a{SP}test{CR}{LF}
22:48:10 25726 DKIM
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
22:48:10 25726 DKIM: finish bodyhash sha256/simple/-1 len 16
22:48:10 25726 DKIM: no signatures
22:48:10 25726 DKIM: no signatures to use
22:48:10 25726 ARC: sign for test.ex
22:48:10 25726 LOG: MAIN
22:48:10 25726 ARC: no Authentication-Results header for signing
22:48:10 25726 SMTP+> BDAT 332 LAST
22:48:10 25726 cmd buf flush 86 bytes (more expected)
22:48:10 25726 gnutls_record_cork(session=0x5590af3a31a0)
22:48:10 25726 tls_write(0x5590af989bb8, 86, more)
22:48:10 25726 gnutls_record_send(session=0x5590af3a31a0,
buffer=0x5590af989bb8, left=86)
22:48:10 25726 outbytes=86
22:48:10 25726 cannot use sendfile for body: spoolfile not wireformat
22:48:10 25726 writing data block fd=6 size=332 timeout=300 (more expected)
22:48:10 25726 tls_write(0x5590af38a048, 332, more)
22:48:10 25726 gnutls_record_send(session=0x5590af3a31a0,
buffer=0x5590af38a048, left=332)
22:48:10 25726 outbytes=332
22:48:10 25726 SMTP>> QUIT
22:48:10 25726 cmd buf flush 6 bytes
22:48:10 25726 tls_write(0x5590af989bb8, 6)
22:48:10 25726 gnutls_record_send(session=0x5590af3a31a0,
buffer=0x5590af989bb8, left=6)
22:48:10 25726 outbytes=6
22:48:10 25726 gnutls_record_uncork(session=0x5590af3a31a0)
22:48:10 25726 GnuTLS<2>: FIPS140-2 context is not set
22:48:10 25726 sync_responses expect mail
22:48:10 25726 Calling gnutls_record_recv(session=0x5590af3a31a0,
buffer=0x5590af988bb8, len=4096)
22:48:10 25726 GnuTLS<2>: FIPS140-2 context is not set
22:48:10 25726 read response data: size=114
22:48:10 25726 SMTP<< 250 OK
22:48:10 25726 sync_responses expect rcpt for gedalya@gedalya.net
22:48:10 25726 SMTP<< 250 Accepted
22:48:10 25726 SMTP<< 250- 332 byte chunk, total 332
22:48:10 25726 250 OK id=1npzcQ-0006h2-32
22:48:10 25726 S:journalling gedalya@gedalya.net
22:48:10 25726 ok=1 send_quit=0 send_rset=0 continue_more=0 yield=0
first_address is NULL
22:48:10 25726 SMTP<< 221 test.ex closing connection
22:48:10 25726 Calling gnutls_record_recv(session=0x5590af3a31a0,
buffer=0x5590af988bb8, len=4096)
22:48:10 25726 GnuTLS<2>: FIPS140-2 context is not set
22:48:10 25726 GnuTLS<3>: ASSERT: record.c[_gnutls_recv_in_buffers]:1589
22:48:10 25726 Got TLS_EOF
22:48:10 25726 tls_close(): shutting down TLS (with response-wait)
22:48:10 25726 tls_write((nil), 0)
22:48:10 25726 GnuTLS<3>: ASSERT: buffers.c[_gnutls_io_write_flush]:696
22:48:10 25726 GnuTLS<2>: FIPS140-2 context is not set
22:48:10 25726 SMTP(close)>>
22:48:10 25726 cmdlog:
'220:EHLO:250-:STARTTLS:220:EHLO:250-:MAIL|:RCPT|:BDAT:QUIT:250:250:250-:221'
22:48:10 25726 set_process_info: 25726 delivering 1npzcP-0006gi-0q: just tried
127.0.0.1 [127.0.0.1]:1225 for gedalya@gedalya.net: result OK
22:48:10 25726 Leaving gsmtp transport

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #16 from Andreas Metzler <eximusers@bebt.de> ---
Managed to reproduce this with very vanilla exim in a Debian sid chroot:
----------------------
ametzler@argenau:/tmp/EXIM-from-source/exim-4.96-RC0$ grep -E -v
'^#|^[[:space:]]*$' Local/Makefile
BIN_DIRECTORY=/usr/exim/bin
CONFIGURE_FILE=/usr/exim/configure
EXIM_USER=mail
SPOOL_DIRECTORY=/var/spool/exim
USE_GNUTLS=yes
USE_GNUTLS_PC=gnutls gnutls-dane
ROUTER_ACCEPT=yes
ROUTER_DNSLOOKUP=yes
ROUTER_IPLITERAL=yes
ROUTER_MANUALROUTE=yes
ROUTER_QUERYPROGRAM=yes
ROUTER_REDIRECT=yes
TRANSPORT_APPENDFILE=yes
TRANSPORT_AUTOREPLY=yes
TRANSPORT_PIPE=yes
TRANSPORT_SMTP=yes
LOOKUP_DBM=yes
LOOKUP_LSEARCH=yes
LOOKUP_DNSDB=yes
PCRE2_CONFIG=yes
SUPPORT_DANE=yes
DISABLE_MAL_AVE=yes
DISABLE_MAL_KAV=yes
DISABLE_MAL_MKS=yes
FIXED_NEVER_USERS=root
AUTH_CRAM_MD5=yes
HEADERS_CHARSET="ISO-8859-1"
SYSLOG_LOG_PID=yes
EXICYCLOG_MAX=10
COMPRESS_COMMAND=/usr/bin/gzip
COMPRESS_SUFFIX=gz
ZCAT_COMMAND=/usr/bin/zcat
SYSTEM_ALIASES_FILE=/etc/aliases
EXIM_TMPDIR="/tmp"
----------------------

/usr/exim/configure is unmodified.

(eximtest)root@argenau:/# /usr/exim/bin/exim -odq -f deferme@example.com
gedalya@gedalya.net < /tmp/2.msg
(eximtest)root@argenau:/# /usr/exim/bin/exim -bp
0m 312 1nqCu1-0003d6-07 <deferme@example.com>
gedalya@gedalya.net
(eximtest)root@argenau:/# /usr/exim/bin/exim -d+all -q 1nqCu1-0003d6-07
2>&1 | tee /tmp/exim.debug
[... - will attach]
(eximtest)root@argenau:/# /usr/exim/bin/exim -bp
1m 312 1nqCu1-0003d6-07 <deferme@example.com> *** frozen ***
D gedalya@gedalya.net

(eximtest)root@argenau:/# cat /var/spool/exim/log/paniclog
2022-05-15 12:00:14 1nqCu1-0003d6-07 SIGSEGV (fault address: 0x1)
2022-05-15 12:00:14 1nqCu1-0003d6-07 SEGV_MAPERR
2022-05-15 12:00:14 1nqCu1-0003d6-07 SIGSEGV (null pointer indirection)
2022-05-15 12:00:14 1nqCu1-0003d6-07 SIGSEGV (13972 delivering 1nqCu1-0003d6-07
to mx2.gedalya.net [104.131.53.251] (gedalya@gedalya.net)
)
2022-05-15 12:00:14 1nqCu1-0003d6-07 Delivery status for gedalya@gedalya.net:
got 0 of 7 bytes (pipeheader) from transport process 13972 for transport smtp

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #17 from Andreas Metzler <eximusers@bebt.de> ---
Created attachment 1416
--> https://bugs.exim.org/attachment.cgi?id=1416&action=edit
(unpatched) exim debug output

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #18 from Andreas Metzler <eximusers@bebt.de> ---
(In reply to Andreas Metzler from comment #16)
> Managed to reproduce this with very vanilla exim in a Debian sid chroot:

Version 4.69RC0, built/installed with

env CFLAGS='-D_LARGEFILE_SOURCE -g -O2 -fstack-protector-strong -Wformat
-Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2' LFLAGS='-Wl,-z,relro
-Wl,-z,now' LDFLAGS='-Wl,-z,relro -Wl,-z,now' make FULLECHO=''
env CFLAGS='-D_LARGEFILE_SOURCE -g -O2 -fstack-protector-strong -Wformat
-Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2' LFLAGS='-Wl,-z,relro
-Wl,-z,now' LDFLAGS='-Wl,-z,relro -Wl,-z,now' make FULLECHO='' install

cu Andreas

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

Jeremy Harris <jgh146exb@wizmail.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |ASSIGNED

--- Comment #19 from Jeremy Harris <jgh146exb@wizmail.org> ---
(In reply to Andreas Metzler from comment #17)
> Created attachment 1416 [details]
> (unpatched) exim debug output

Thanks Andreas. For this variant, the message is properly transferred
(accepted by the destination on the second MX tried) and then we segv
after the peer has indicated a TLS close.

It'd be useful to peek at a core stack to see if the crash was actually in the
GnuTLS library on some subsequent call into it. The debug trace:

12:00:14 13972 Calling gnutls_record_recv(session=0x558e5bd826b0,
buffer=0x558e5bdb60e8, len=4096)
12:00:14 13972 GnuTLS<3>: ASSERT:
../../lib/record.c[_gnutls_recv_in_buffers]:1589
12:00:14 13972 Got TLS_EOF (that read returned empty)

12:00:14 13972 tls_close(): shutting down TLS (with response-wait)
12:00:14 13972 tls_write((nil), 0) (zero bytes to write,
should not call into lib here)

12:00:14 13972 GnuTLS<3>: ASSERT:
../../lib/buffers.c[_gnutls_io_write_flush]:696 (unclear how we
got here)

12:00:14 13972 LOG: MAIN PANIC
12:00:14 13972 SIGSEGV (fault address: 0x1)

is concerning wrt. that _gnutls_io_write_flush location, but not
definitive as to the location trigerring the segv.

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #20 from Andreas Metzler <eximusers@bebt.de> ---
Hello,

given that I had some time but no thought/smartness to spare I ran git bisect
which found
-----
6a9cf7f890226aa085842cd3d94b13e78ea31637 is the first bad commit
commit 6a9cf7f890226aa085842cd3d94b13e78ea31637
Date: Sat Oct 3 20:59:15 2020 +0100

TLS: preload configuration items
-----

the nice thing about Gedalya testcase is that it does not require exim to be
suid root. Out od laziness I used the throwaway chroot with exim-user=mail but
I think no privileges (installing to ~/eximtest with exim-user=ametzler) would
also work.

cu Andreas

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #21 from Gedalya <gedalya@gedalya.net> ---
Did pretty much the same as Andreas, built in sid chroot using the same make
command from git (4.96-RC1).

Unmodified runtime config, same EDITME as Andreas. No ARC or DKIM signing yet.

It crashed in tls_client_start > verify_certificate etc.

Will attach gedalya.vanilla.1.exim.bt and gedalya.vanilla.1.exim.debug

Is there any further testing I could do that would be helpful?

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #22 from Gedalya <gedalya@gedalya.net> ---
Created attachment 1417
--> https://bugs.exim.org/attachment.cgi?id=1417&action=edit
gedalya.vanilla.1.exim.bt

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #23 from Gedalya <gedalya@gedalya.net> ---
Created attachment 1418
--> https://bugs.exim.org/attachment.cgi?id=1418&action=edit
gedalya.vanilla.1.exim.debug

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #24 from Gedalya <gedalya@gedalya.net> ---
Created attachment 1419
--> https://bugs.exim.org/attachment.cgi?id=1419&action=edit
gedalya.vanilla.2.debug_and_bt

Disabling verification lets the message deliver and the crash occurs in
tls_close > gnutls_certificate_free_credentials > gnutls_x509_trust_list_deinit

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 2886] Crashes in SMTP delivery attempt following a deferral [ In reply to ]
https://bugs.exim.org/show_bug.cgi?id=2886

--- Comment #25 from Jeremy Harris <jgh146exb@wizmail.org> ---
This one is different to Andreas'; the crash is during the verify stage of
TLS establishment. The stacktrace is:

_gnutls_trust_list_get_issuer
gnutls_x509_trust_list_get_issuer
gnutls_x509_trust_list_verify_crt2
_gnutls_x509_cert_verify_peers
gnutls_certificate_verify_peers ^^^
gnutls_certificate_verify_peers2 ^^^ GnuTLS library
verify_certificate vvv Exim
tls_client_start vvv
smtp_setup_conn

From looking at the GnuTLS source I'm not able to guess what state it's
missing. It's unfortunate that it follows a null pointer rather than
checking and returning an error from the gnutls_certificate_verify_peers2 API
call; I'd call that a bug in GnuTLS.

It's interesting that we had a good TLS conn for the first MX tried, in the
same process. Presumably that leaves GnuTLS in some awkward state. If the
preload support Andreas identified is also relevant to this variant then
the "client CA bundle" is suspect. We're relying on the bundle loaded
during the parent Exim startup (either daemon or cmdline-send), rather than
(as before that commit) loading it afresh for every TLS connection.

A workaround would be to introduce a '$' into the transport
tls_verify_certificates option. "${expand:}" would suffice, added to the
existing; this is just to make exim think the option value might vary so must
not be cached.

I'd suggest raising a bug against GnuTLS for this.
Testing with a range of different GnuTLS versions might also be useful.

--
You are receiving this mail because:
You are on the CC list for the bug.
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##

1 2  View All