Mailing List Archive

Re: [exim-announce] Exim 4.96-RC0 released
Beware that the just released RC0 for Exim 4.96 may break your Dovecot
LDA delivery. It did break mine, which is similar to what is described
on https://wiki.dovecot.org/LDA/Exim

Here is the relevant ChangeLog entry:

JH/25 Taint-check exec arguments for transport-initiated external processes.
Previously, tainted values could be used. This affects "pipe", "lmtp" and
"queryprogram" transport, transport-filter, and ETRN commands.
The ${run} expansion is also affected: in "preexpand" mode no part of
the command line may be tainted, in default mode the executable name
may not be tainted.

As of now I don't have a personal working solution to get untained data.
I did try a small hack, but Exim was smart enough to see what I was
doing, so I'm rolling back until I better unstand the tainting
facilities and how to untaint the data.

The concept is great, though.

• Jeremy Harris via Exim-announce [2022-04-23 20:23]:
> Hi all,
>
> Enough changes have accumulated since 4.95 for it to be worth
> spinning another release. The first Release Candidate is available
>
> - as tarball: https://ftp.exim.org/pub/exim/exim4/test
> - directly from Git: https://git.exim.org
> tag exim-4.96-RC0
>
> The signatures on the release tarballs (and this mail) should be
> mine, key ID 0xBCE58C8CE41F32DF.
>
> Please test and give feedback on the RC so that we can shake out
> issues before 4.96 goes official.
>
> The sourcebase is still open for feature additions at this point
> in the release process.
>
>
> New stuff we've added since 4.95:
>
> - A new ACL condition: seen. Records/tests a timestamp against a key.
> - A variant of the "mask" expansion operator to give normalised IPv6.
> - UTC output option for exim_dumpdb, exim_fixdb.
> - An event for failing TLS connects to the daemon.
> - The ACL "debug" control gains options "stop", "pretrigger" and "trigger".
> - Query-style lookups are now checked for quoting, if the query string is
> built using untrusted data ("tainted"). For now lack of quoting is merely
> logged; a future release will upgrade this to an error.
> - The expansion conditions match_<list-type> and inlist now set $value for
> the expansion of the "true" result of the ${if}. With a static list, this
> can be used for de-tainting.
>
> Notable removals since 4.95:
>
> - the "allow_insecure_tainted_data" main config option and the
> "taint" log_selector. These were previously deprecated.
>
> As usual, you should read the ChangeLog file for other changes.
> --
> Cheers,
> Jeremy




> --
> ## List details at https://lists.exim.org/mailman/listinfo/exim-announce Exim details at http://www.exim.org/ ##


--
-- Kirill Miazine <km@krot.org>

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
Re: [exim-announce] Exim 4.96-RC0 released [ In reply to ]
On Mon, 25 Apr 2022, Kirill Miazine via Exim-dev wrote:

> Beware that the just released RC0 for Exim 4.96 may break your Dovecot
> LDA delivery. It did break mine, which is similar to what is described
> on https://wiki.dovecot.org/LDA/Exim
>
> Here is the relevant ChangeLog entry:
>
> JH/25 Taint-check exec arguments for transport-initiated external processes.
> Previously, tainted values could be used. This affects "pipe", "lmtp" and
> "queryprogram" transport, transport-filter, and ETRN commands.
> The ${run} expansion is also affected: in "preexpand" mode no part of
> the command line may be tainted, in default mode the executable name
> may not be tainted.

> • Jeremy Harris via Exim-announce [2022-04-23 20:23]:
>> Notable removals since 4.95:
>>
>> - the "allow_insecure_tainted_data" main config option and the
>> "taint" log_selector. These were previously deprecated.

That isn't a good combination. Please could we keep the option to
allow_insecure_tainted_data if there are new taint features ?

That way we can continue to run live systems while we resolve
these sort of problems.

Thanks,

--
Andrew C. Aitchison Kendal, UK
andrew@aitchison.me.uk
--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
Re: [exim-announce] Exim 4.96-RC0 released [ In reply to ]
So my workaround was to create a simple wrapper and call it, instead of
dovecot-lda:

$ cat /local/bin/dovecot-lda-wrapper
#!/bin/sh
exec /usr/local/libexec/dovecot/dovecot-lda \
-d "${LOCAL_PART}@${DOMAIN}" \
-a "${LOCAL_PART}${LOCAL_PART_SUFFIX}@${DOMAIN}" \
-r "${LOCAL_PART}${LOCAL_PART_SUFFIX}@${DOMAIN}" \
-f "${SENDER}"

Here's how it is called from Exim:

dovecot_pipe:
driver = pipe
command = /local/bin/dovecot-lda-wrapper

• Kirill Miazine via Exim-dev [2022-04-25 14:47]:
> Beware that the just released RC0 for Exim 4.96 may break your Dovecot
> LDA delivery. It did break mine, which is similar to what is described
> on https://wiki.dovecot.org/LDA/Exim
>
> Here is the relevant ChangeLog entry:
>
> JH/25 Taint-check exec arguments for transport-initiated external processes.
> Previously, tainted values could be used. This affects "pipe", "lmtp" and
> "queryprogram" transport, transport-filter, and ETRN commands.
> The ${run} expansion is also affected: in "preexpand" mode no part of
> the command line may be tainted, in default mode the executable name
> may not be tainted.
>
> As of now I don't have a personal working solution to get untained data.
> I did try a small hack, but Exim was smart enough to see what I was
> doing, so I'm rolling back until I better unstand the tainting
> facilities and how to untaint the data.
>
> The concept is great, though.
>
> • Jeremy Harris via Exim-announce [2022-04-23 20:23]:
> > Hi all,
> >
> > Enough changes have accumulated since 4.95 for it to be worth
> > spinning another release. The first Release Candidate is available
> >
> > - as tarball: https://ftp.exim.org/pub/exim/exim4/test
> > - directly from Git: https://git.exim.org
> > tag exim-4.96-RC0
> >
> > The signatures on the release tarballs (and this mail) should be
> > mine, key ID 0xBCE58C8CE41F32DF.
> >
> > Please test and give feedback on the RC so that we can shake out
> > issues before 4.96 goes official.
> >
> > The sourcebase is still open for feature additions at this point
> > in the release process.
> >
> >
> > New stuff we've added since 4.95:
> >
> > - A new ACL condition: seen. Records/tests a timestamp against a key.
> > - A variant of the "mask" expansion operator to give normalised IPv6.
> > - UTC output option for exim_dumpdb, exim_fixdb.
> > - An event for failing TLS connects to the daemon.
> > - The ACL "debug" control gains options "stop", "pretrigger" and "trigger".
> > - Query-style lookups are now checked for quoting, if the query string is
> > built using untrusted data ("tainted"). For now lack of quoting is merely
> > logged; a future release will upgrade this to an error.
> > - The expansion conditions match_<list-type> and inlist now set $value for
> > the expansion of the "true" result of the ${if}. With a static list, this
> > can be used for de-tainting.
> >
> > Notable removals since 4.95:
> >
> > - the "allow_insecure_tainted_data" main config option and the
> > "taint" log_selector. These were previously deprecated.
> >
> > As usual, you should read the ChangeLog file for other changes.
> > --
> > Cheers,
> > Jeremy
--
-- Kirill Miazine <km@krot.org>

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
Re: [exim-announce] Exim 4.96-RC0 released [ In reply to ]
On 26/04/2022 08:28, Andrew C Aitchison via Exim-dev wrote:
>> • Jeremy Harris via Exim-announce [2022-04-23 20:23]:
>>> Notable removals since 4.95:
>>>
>>>   - the "allow_insecure_tainted_data" main config option and the
>>>     "taint" log_selector.  These were previously deprecated.
>
> That isn't a good combination. Please could we keep the option to
> allow_insecure_tainted_data if there are new taint features ?
>
> That way we can continue to run live systems while we resolve
> these sort of problems.


The trouble with that is that it means the coverage of tracking
tainted data use can never be extended.

The commit for that removal is fairly extensive:

$ git log --stat 8f2cf8f5ad^1..8f2cf8f5ad
commit 8f2cf8f5adaa08ef84b47bf9bc2f71e39236c22d
Author: Jeremy Harris <jgh146exb@wizmail.org>
Date: Tue Mar 1 23:12:53 2022 +0000

Revert introduction of alloc_insecure_tainted_data

tidy log.c

(cherry picked from commit 0327b6460eec64da6b0c1543c7e9b3d0f8cb9294)
(cherry picked from commit 8021b95c2e266861aba29c97b4bb90dc6f7637a2)

This reverts commit f9a3fcddba223133019368e7cd6d51449fc54e7b.
This reverts commit 8fc13e4adcdf5d0cec382c401ce72592569084fe.
This reverts commit 2382cd3e81838709abd3b0c1b410f65274d90e25.
This reverts commit ace68726852d08deec815a37f369a4e31be77813.
This reverts commit 4a2bd5dc2f157c50b7e4e7491eac1c930efd2100.
This reverts commit e8b8b133c685e7b09f672016d117c7d8b49e70b1.
This reverts commit f7509ba6fb5e1033c1406b87f057c9c48a217d27.
This reverts commit dbac5a049acbe645a816b4a5e895c5be0de53483.
This reverts commit f7da81e789e2f20b00f46f07260488f337984b84.
This reverts commit f7c791b769a3a5395d92d29d27aa58f9d442373c.
This reverts commit 0fa46a83a55054c65ffc539405f62bf86aac5b44.
This reverts commit ec7e44dc9134307d74afa4b07f09afbdd019282c.
This reverts commit ee4924bc711f54a751448fb5cee5ec4fc9c96196.
This reverts commit 951b668a191ef510a4e27d8204c5fa82ca957a07.
This reverts commit 18d243312bf3a23bd0f464fac44797e2720e03ec.
This reverts commit 16c884a4818594069253de460bf9926f69d50fe5.
This reverts commit 368ecb000c58995c5f61443d45d43942f1f431d0.
This reverts commit 331817e995b05793ec840476fac67e8f7c638a47.
This reverts commit 07343a5b1968f4f5e41664c15fd636bdb6a6cc48.
This reverts commit 060cf1e3c9a0a6960b771cdff6f0a5a2ca9b114c.
This reverts commit 28d2eab1414ef8d20ff0fde7026aa52fd01ef795.
This reverts commit b6b4b129892a99747a586e5d4acb68fe7176ab4b.
This reverts commit 77b478579a0029def01d1b3a4ea591eac447832a.
This reverts commit 0103b34645278151851c31cf1c1976150d907bb4.
This reverts commit f94ca3e3ed2bc5a68ac54c5487e0216ea8db8470.
This reverts commit 7a33a3fc2c1ae4df30eaabaf25fc59e2d6ab5d6c.
This reverts commit 305c0579ab4286b464c2cd589843a7e60f59dfaf.

doc/doc-docbook/spec.xfpt | 28 ------------
doc/doc-txt/ChangeLog | 3 ++
src/src/EDITME | 7 ---
src/src/acl.c | 38 +++++++----------
src/src/config.h.defaults | 2 -
src/src/dbstuff.h | 6 ++-
src/src/deliver.c | 16 +++----
src/src/directory.c | 4 +-
src/src/expand.c | 212 +++++++++++++++++++++++++++++++++++++++++++++----------------------------------------------
src/src/functions.h | 54 ++++++-----------------
src/src/globals.c | 10 -----
src/src/globals.h | 4 --
src/src/log.c | 130 +++++++++++++++++++++++---------------------------------
src/src/lookups/lf_sqlperform.c | 14 +++---
src/src/macros.h | 3 --
src/src/parse.c | 6 ++-
src/src/rda.c | 4 +-
src/src/readconf.c | 3 --
src/src/routers/rf_get_transport.c | 4 +-
src/src/search.c | 8 +++-
src/src/smtp_out.c | 7 ++-
src/src/transports/appendfile.c | 11 -----
src/src/transports/autoreply.c | 21 ++++-----
src/src/transports/pipe.c | 9 ++--
src/src/transports/smtp.c | 5 ++-
test/aux-fixed/0990/example.com | 1 -
test/confs/0990 | 2 -
test/log/0608 | 2 +-
test/paniclog/0608 | 2 +-
test/scripts/0990-Allow-Tainted-Data/0990 | 7 ---
test/scripts/0990-Allow-Tainted-Data/REQUIRES | 1 -
test/stderr/0608 | 2 +-
test/stderr/0990 | 3 --
test/stderr/2620 | 24 +++++------
test/stdout/0990 | 4 --
35 files changed, 259 insertions(+), 398 deletions(-)

- and probably cannot be directly reverted, since the support
for taint-tracking was complexified to handle lookup-quote
tracking (4191cb1503).

--
Cheers,
Jeremy

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
Re: Exim 4.96-RC0 released [ In reply to ]
On 2022-04-27 Jeremy Harris via Exim-dev <exim-dev@exim.org> wrote:
> On 26/04/2022 08:28, Andrew C Aitchison via Exim-dev wrote:
>>> • Jeremy Harris via Exim-announce [2022-04-23 20:23]:
>>>> Notable removals since 4.95:

>>>>   - the "allow_insecure_tainted_data" main config option and the
>>>>     "taint" log_selector.  These were previously deprecated.

>> That isn't a good combination. Please could we keep the option to
>> allow_insecure_tainted_data if there are new taint features ?

>> That way we can continue to run live systems while we resolve
>> these sort of problems.

> The trouble with that is that it means the coverage of tracking
> tainted data use can never be extended.
[...]

Hello,

I think it could be less problematic if configurations that already
triggered an error in 4.95 (and needed allow_insecure_tainted_data to
work) stopped working with 4.96 even if allow_insecure_tainted_data was
set.

However users need/want something equivalent to test upgrades to 4.96
for problems with the new taint checks (requirement for quoting in
query-style lookups and taint-check exec arguments for
transport-initiated external processes).

People upgrading directly from < 4.93 to 4.96 would still have to deal
with hard breakage on upgrades, but requirig a two step upgrade might be
considered a fair compromise.

cu Andreas
--
`What a good friend you are to him, Dr. Maturin. His other friends are
so grateful to you.'
`I sew his ears on from time to time, sure'

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
Re: Exim 4.96-RC0 released [ In reply to ]
On 30/04/2022 09:34, Andreas Metzler via Exim-dev wrote:
> (requirement for quoting in
> query-style lookups

Those are not hard-fails as currently implemented.

> and taint-check exec arguments for
> transport-initiated external processes).

These are.
--
Cheers,
Jeremy

--
## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
Re: Exim 4.96-RC0 released [ In reply to ]
Andreas Metzler via Exim-dev <exim-dev@exim.org> (Sa 30 Apr 2022 10:34:23 CEST):
>
> People upgrading directly from < 4.93 to 4.96 would still have to deal
> with hard breakage on upgrades, but requirig a two step upgrade might be
> considered a fair compromise.

Yes, that's something I'm thinking about too.

4.95: "allow_insecure_tainted_data" allows a smooth upgrade from 4.94
4.96: "allow_insecure_tainted_data" doesn't cover all that taint issues
that were "accepted" (turned into warnings) for 4.95, but turns
the new implemented errors into warnings.
… and so on

That is, having always one release as a grace period to sort out the taint
issues.

We have quite complex configurations and there isn't always a way to
test them in advance. Rolling back during an update isn't an option in
many cases, and prevents further checking, as the the previous versions
works, and maybe breaks with config changes I made already to comply
with our new taint checks.

--
Heiko