Mailing List Archive

Andrew C Aitchison via Exim-users <exim-users@exim.org> (Fr 29 Apr 2022 18:16:45 CEST):
> To which Jeremy replied:
> > The trouble with that is that it means the coverage of tracking
> > tainted data use can never be extended.
> >
> > The commit for that removal is fairly extensive:
> - see https://lists.exim.org/lurker/message/20220427.174941.443df2eb.en.html
> for the 27 reverts and 35 files changed.
>
> Given that taint checking appeared in Exim 4.93 and
> allow_insecure_tainted_data in Exim 4.95,
> this (Exim 4.96) would be the first time that allow_insecure_tainted_data
> would actually be helpful.
>
> Is it just me, or are others worried about the new taint checking
> having unexpected consequences and no way to disable it for debugging ?

The "allow_insecure_tainted_data" was introduced to ease the migration
from 4.94 to 4.95, giving you/us a timeframe to upgrade existing
configurations to be taintproof.

Before upgrading to 4.96 you should have a taintproof (secure)
configuration. The deprecation of "allow_insecure_tainted_data" was
announced with the advent of this option already.

Which point did I miss? Do we have *new* taintchecks that break
configurations that were considered secure with 4.95?

Best regards from Dresden/Germany
Viele Grüße aus Dresden
Heiko Schlittermann
--
SCHLITTERMANN.de ---------------------------- internet & unix support -
Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
gnupg encrypted messages are welcome --------------- key ID: F69376CE -

On Fri, Apr 29, 2022 at 05:16:45PM +0100, Andrew C Aitchison via Exim-users wrote:

> Given that taint checking appeared in Exim 4.93 and
> allow_insecure_tainted_data in Exim 4.95,
> this (Exim 4.96) would be the first time that allow_insecure_tainted_data
> would actually be helpful.
>
> Is it just me, or are others worried about the new taint checking
> having unexpected consequences and no way to disable it for debugging ?

I'd prefer the allow_insecure_tainted_data never be removed, now or in the future. At the least, as an experimental feature that requires intentional enabling during a source build. At the worst as a separate community maintained patch against the official source for each new release. Maintaining production mail systems that handle millions of messages a month is no trivial feat, and a single taint failure can turn (and has turned) a routine upgrade plan into a mess.

Thanks,
Dean

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Dean Brooks via Exim-users <exim-users@exim.org> wrote:

> On Fri, Apr 29, 2022 at 05:16:45PM +0100, Andrew C Aitchison via Exim-users wrote:
>
> > Given that taint checking appeared in Exim 4.93 and
> > allow_insecure_tainted_data in Exim 4.95,
> > this (Exim 4.96) would be the first time that allow_insecure_tainted_data
> > would actually be helpful.
> >
> > Is it just me, or are others worried about the new taint checking
> > having unexpected consequences and no way to disable it for debugging ?
>
> I'd prefer the allow_insecure_tainted_data never be removed, now or in the future. At the least, as an experimental feature that requires intentional enabling during a source build. At the worst as a separate community maintained patch against the official source for each new release. Maintaining production mail systems that handle millions of messages a month is no trivial feat, and a single taint failure can turn (and has turned) a routine upgrade plan into a mess.

100% agreement. Having to include it as build option is reasonable.

Michael

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

• Heiko Schlittermann via Exim-users [2022-04-29 21:07]:
[...]
> The "allow_insecure_tainted_data" was introduced to ease the migration
> from 4.94 to 4.95, giving you/us a timeframe to upgrade existing
> configurations to be taintproof.
>
> Before upgrading to 4.96 you should have a taintproof (secure)
> configuration. The deprecation of "allow_insecure_tainted_data" was
> announced with the advent of this option already.
>
> Which point did I miss? Do we have *new* taintchecks that break
> configurations that were considered secure with 4.95?

I had a setup which didn't use allow_insecure_tainted_data, but I was
hit by

JH/25 Taint-check exec arguments for transport-initiated external processes.
Previously, tainted values could be used. This affects "pipe", "lmtp" and
"queryprogram" transport, transport-filter, and ETRN commands.
The ${run} expansion is also affected: in "preexpand" mode no part of
the command line may be tainted, in default mode the executable name
may not be tainted.

I welcome taint checking facilities, but I'm afraid that the
introduction of taint checking for exec arguments will cause lots of
broken configurations. Even the Exim spec has (had) examples which would
be broken by the change. When shell is not used, the only reason for
tain checking argument would be to protect the command being called in
case it's not prepared to deal with malicious arguments.

When using Perl's taint checking facilities (-T) some 20 years ago,
I remember I spent quite a while reading perlsec(1). Memories from that
time helped me when trying to understand taint checking in Exim. The
Exim documentation is still not very detailed about the concept. I'm not
ready to write anything, as I haven't gained enough understanding myself
yet.

I'd welcome some generic way to untaint data. E.g. Perl would allow to
subpatterns from regular expression match to untaint parts of data. For
local parts and domains there's kind of a way, but for local part
affixes or sender addresses, I couldn't find a way.

> Best regards from Dresden/Germany
> Viele Grüße aus Dresden
> Heiko Schlittermann
> --
> SCHLITTERMANN.de ---------------------------- internet & unix support -
> Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
> gnupg encrypted messages are welcome --------------- key ID: F69376CE -

--
-- Kirill Miazine <km@krot.org>

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

On 29/04/2022 22:41, Kirill Miazine via Exim-users wrote:
> I'd welcome some generic way to untaint data.

If you know of one which does not require a list
of known-good values, and is not trivially abusable
by blind copy-pasting of recipes found on random blogs -
I'm all ears.
--
Cheers,
Jeremy

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

D?a 29. apríla 2022 21:41:55 UTC používate? Kirill Miazine via Exim-users <exim-users@exim.org> napísal:

>I'd welcome some generic way to untaint data. E.g. Perl would allow to
>subpatterns from regular expression match to untaint parts of data. For
>local parts and domains there's kind of a way, but for local part
>affixes or sender addresses, I couldn't find a way.

Yes, as i wrote the same already some time ago, some generic
${detaint:...} expansion is missing. All current solutions are based
on "local" DB lookup. But it is not suitable e.g. to my case, where i
verify recipients from my MX to my other MTA (where local DB are
stored) by callout. But that doey not detaint recipient address nor
domain, thus i have to use some tricks to be able to use per
recipient/domain (something as) quarantine on MX...

As redis support is not full (and on Debian is missing at all) i use
${run ...} to communicate with redis and i afraid, that i will have
problems to use it in new version, etc, etc. I have very mixed feel.
And i can only afraid, what will arrive next. Perhaps reject to detaint
by that local DB values (as someone can insert insecure values
there)?

After more than 30 years in computer & network word i learn, that
software can either allow to do flexible configuration (including
mistakes) or do not allow to do mistakes (but without flexibility),
never can be achieved both at once... Until recent, exim was in the
first case and we will see, where it go.

Regards

--
Slavko

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

On 30/04/2022 00:54, Slavko (tblt) via Exim-users wrote:
> Yes, as i wrote the same already some time ago, some generic
> ${detaint:...} expansion is missing.

That would be instantly abused.

> verify recipients from my MX to my other MTA (where local DB are
> stored) by callout. But that doey not detaint recipient address nor
> domain,

That's worthy of consideration; thank you for the idea.
Essentially, it would be treating a backend MTA as a trusted DB
for lookup.

> As redis support is not full (and on Debian is missing at all) i use
> ${run ...} to communicate with redis and i afraid, that i will have
> problems to use it in new version,

Volunteers to work on any aspect, including redis support, are
always welcome. It really needs someone who uses it and finds
a facility lacking (meaning: not me).

In the meantime, the ${run } expansion is not taint-checked
(and therefore still fertile ground for security breaches).

--
Cheers,
Jeremy

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

On 29/04/2022 20:07, Heiko Schlittermann via Exim-users wrote:

> Do we have *new* taintchecks that break
> configurations that were considered secure with 4.95?

I has a hash_32_64 of data, accepted in 4.95, requires quote_pgsql with
4.96.

Does a hash pass a taint? Whatever, easily adjusted in my config.

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

• Jeremy Harris via Exim-users [2022-04-29 23:40]:
> > I'd welcome some generic way to untaint data.
>
> If you know of one which does not require a list
> of known-good values, and is not trivially abusable
> by blind copy-pasting of recipes found on random blogs -
> I'm all ears.

I think that something like ${untaint{$unsafe}{pattern}} could work.

The reason for this is that taint checking is to prevent untrusted
external data from being used in dangerous ways and thus cause troubles
to the system where Exim is running. Pattern would be a regular
expression, which should match the entire $unsafe string, or a *, which
would match anything and which would imply that the user knows what they
are doing. Whether or not to allow * could be a complike time flag.

--
-- Kirill Miazine <km@krot.org>

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Hi,

D?a Sat, 30 Apr 2022 10:10:08 +0100 Jeremy Harris via Exim-users
<exim-users@exim.org> napísal:

> On 30/04/2022 00:54, Slavko (tblt) via Exim-users wrote:
> > Yes, as i wrote the same already some time ago, some generic
> > ${detaint:...} expansion is missing.
>
> That would be instantly abused.

I understand, but IMO exim's dev have not take responsibility behind
stupid admins... But, please, how ${detaint:...} differs eg. from:

${lookup{...} lsearch*,ret=key{file_with_*_only}}

The only differences i see are length of expansion to type and to be
less effective (lookup will be done twice).

> > verify recipients from my MX to my other MTA (where local DB are
> > stored) by callout. But that doey not detaint recipient address nor
> > domain,
>
> That's worthy of consideration; thank you for the idea.
> Essentially, it would be treating a backend MTA as a trusted DB
> for lookup.

Nice, and please, can you consider in that "trusted DB" something,
which can interpret deffer responses?

I mean real 4xx responses, not eg. temporary network problem or so. For
now i do not use this feature, as i cannot distinguish these two
(network problem vs. response) states. But returning deffer from
remote MTA is wanted, eg. for quotas.

> Volunteers to work on any aspect, including redis support, are
> always welcome. It really needs someone who uses it and finds
> a facility lacking (meaning: not me).

I do not afraid to help, but my C knowledge is less even than basic,
and i feel too old (and not healthy) to start learn it now, especially
when i evade C for years ;-)

I do not consider itself as redis expert, but i use redis with MTA/MSA.
I have to build own exim, to i can test these build-in redis lookups,
but i stop to test it, when i realize, that boolean responses are not
usable. There are relative simple workarounds eg. for EXISTS, where one
can try to fetch key's value. But this prevents to test multiple keys
at once and with more "complex" commands, e.g. SISMEMBER this can be
more hard, as redis sets can be large, and fetching whole set (to check
if something is in it) is not ideal and i use these sets eg. for per
user country BL/WL on MSA shared with IMAP. These are not too large,
but anyway.

I feed redis's streams with some logging details, and (while not
directly from exim) i use redis to limit/count access by its HLL
with sliding window and some lua help. And i use its PUBSUB to
distribute fail2ban blocks over multiple machines... Thus i consider
redis as very useful to share state across multiple machines.

Thus, if someone can do things in C, i can provide examples and
test them and we can together get some results, from which can profit
all.

regards

--
Slavko
https://www.slavino.sk

On 2022-04-30 James via Exim-users <exim-users@exim.org> wrote:
> On 29/04/2022 20:07, Heiko Schlittermann via Exim-users wrote:

> > Do we have *new* taintchecks that break
> > configurations that were considered secure with 4.95?

> I has a hash_32_64 of data, accepted in 4.95, requires quote_pgsql with
> 4.96.

> Does a hash pass a taint? Whatever, easily adjusted in my config.

Could you show the complete transport/router that triggered the error and
how you modified the router to fix this?

TIA, cu Andreas

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

On 01/05/2022 10:32, Andreas Metzler via Exim-users wrote:
>>> Do we have *new* taintchecks that break
>>> configurations that were considered secure with 4.95?
>> I has a hash_32_64 of data, accepted in 4.95, requires quote_pgsql with
>> 4.96.
>> Does a hash pass a taint? Whatever, easily adjusted in my config.
> Could you show the complete transport/router that triggered the error and
> how you modified the router to fix this?


4.95 happy:

set acl_m_greyhash =
${hash_32_62:$sender_helo_name$sender_address$local_part$domain}
set acl_m_dontcare = ${lookup pgsql{DELETE FROM acl_data_check
WHERE hash = '$acl_m_greyhash' AND result = 'GREY';}}


4.96 complains:

Apr 30 10:23:44 XXXXXX exim: [ID 197553 mail.alert] tainted search query
is not properly quoted (ACL defer, /etc/opt/XXX/exim/exim.conf 641):
DELETE FROM acl_data_check WHERE hash =
'5nRjzb5pF9dbd6GaIahLWbvrSNx3Hxi9' AND result = 'GREY';


4.96 happy, added "${quote_pgsql:...}":

set acl_m_greyhash =
${hash_32_62:$sender_helo_name$sender_address$local_part$domain}
set acl_m_dontcare = ${lookup pgsql{DELETE FROM acl_data_check
WHERE hash = '${quote_pgsql:$acl_m_greyhash}' AND result = 'GREY';}}




4.96 appears to pass a taint flag though the hash whereas 4.95 does not
or the checking rigour has changed such that it is now reported.


These two lines are just logging the receipt in table acl_data_check;
greylisting is done by calling an external process.




--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

On 01/05/2022 10:58, James via Exim-users wrote:
>     set acl_m_greyhash = ${hash_32_62:$sender_helo_name$sender_address$local_part$domain}

If that subject string for the hash operator was less than
33 chars long, the operator returns it unchanged.
If an attacker slipped some SQL syntax in there, your lookup
would not do what you expected.

So it was already broken, lacking a quoting operation,
and 4.96 discovered this for you.
--
Cheers,
Jeremy

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

On 01/05/2022 11:19, Jeremy Harris via Exim-users wrote:
> If that subject string for the hash operator was less than
> 33 chars long, the operator returns it unchanged.
> If an attacker slipped some SQL syntax in there, your lookup
> would not do what you expected.

The hash did not do what I expect.

$ echo 1 | md5sum
b026324c6904b2a9cb4b88d6d61c81d1 -

> So it was already broken, lacking a quoting operation,
> and 4.96 discovered this for you.

Indeed, most grateful and I changed my config without complaint. All I
was doing was answering the question "Do we have *new* taintchecks..."





--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

On 30/04/2022 12:04, Slavko via Exim-users wrote:
>> That's worthy of consideration; thank you for the idea.
>> Essentially, it would be treating a backend MTA as a trusted DB
>> for lookup.

Committed at 7bdf04110b.

> can you consider in that "trusted DB" something,
> which can interpret deffer responses?

I'm not sure what you're wanting here that isn't already
present; if the callout gets a 4xx then the ACL should
defer. What "interpretation" are you looking for?
--
Cheers,
Jeremy

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Ahoj,

D?a Sun, 1 May 2022 11:54:32 +0100 Jeremy Harris via Exim-users
<exim-users@exim.org> napísal:

> On 30/04/2022 12:04, Slavko via Exim-users wrote:
> >> That's worthy of consideration; thank you for the idea.
> >> Essentially, it would be treating a backend MTA as a trusted DB
> >> for lookup.
>
> Committed at 7bdf04110b.

Thanks, from the docs in that commit it looks as exactly what i want,
only things what is not clean from it is: will be appropriate $*_data
variables filled on defer result with defer_ok option? I mean in case,
when callout do not fail due network errors. I expect that no, but to
be sure.

I do not know if that will be useful, i only want to know it.

> > can you consider in that "trusted DB" something,
> > which can interpret deffer responses?
>
> I'm not sure what you're wanting here that isn't already
> present; if the callout gets a 4xx then the ACL should
> defer. What "interpretation" are you looking for?

Sorry if i was not clean, my English is not best, i will try to be more
descriptive (all is about local recipient callouts to other own MTA)...

Yes, ACL deffer on temporary error, but one cannot distinguish on
that deffer, if it was due network/server error or it is defer response.

Currently there are two (beside other) possibility:

+ without defer_ok, on any defer reason the ACL immediately returns
defer result and one cannot act on it (except warn)

+ with defer_ok, that condition is true on any defer reason and ACL
continues to next condition(s)

The second looks OK at first, but to distinguish if that defer happens
due network error or as real MTA response, one have to parse
$acl_verify_message, if recipient is valid, but defers (for some
reason), or recipient status is unknown because defer happens eg.
because remote MTA is down. While it is OK (for me) to defer in both
cases, i want to send different message.

The $*_verify_failure variables, by docs, are not usable on defer
response (with defer_ok), as its value is set (mostly) at reject
(recipient).

Yes, there are recipes how to distinguish temporary/permanent errors
from that callout by the warn(s) verb, but these are not compatible
with no_cache (in mean, that callout is done multiple times then), and
still cannot distinguish network vs. real temporary errors.

In retry rules i can distinguish connection failures from recipient
failures and i want something as that (while not as precise), i want to
know, if it is network error (configuration error, DNS error, etc), or
it is real result of recipient verification, thus recipient is
verified, but it doesn't accept messages for now.

I do callout twice, once on inbound MX to check recipient address on
remote MSA. That remote MSA know only about aliases, real user DB is
maintained by dovecot, thus i use second callout to dovecot to verify
it. The dovecot itself is responsible for quotas too and it is able to
return temporary error response on quota excess, but i cannot use it,
as i cannot (simple) distinguish network errors from that quota.

IMO, sufficient solution can be, if the $*_verify_failure variables will
at least have their defer counterparts for mail & recipient values, eg.
mail_defer, recipient_defer.

I hope that now it is more clear now...

And BTW, the $acl_verify_message is horrible, and i cannot use it
directly, as i do not want to tell random remote connections, that i am
doing callout and do not want to message have two codes. To extract
message only from it i have to use (but i afraid, that this sg{} will
fail for multiline responses):

${sg {$acl_verify_message}{^Callout verification failed:\\n\\d+ }{}}

regards

--
Slavko
https://www.slavino.sk

On 01/05/2022 15:16, Slavko via Exim-users wrote:
> will be appropriate $*_data
> variables filled on defer result with defer_ok option? I mean in case,
> when callout do not fail due network errors.

No.

> + without defer_ok, on any defer reason the ACL immediately returns
> defer result and one cannot act on it (except warn)

You can, but it takes a little coding complexity:
do the verify in a sub-acl, setting up a variable first.

For instance, I have:

# Technique for explicitly seeing defers; credit andrew@supernews.net
# Call this from a "warn" verb.
sverify_callout:
require set acl_c8 = defer
# Easynet puts a 30s delay on bounce inputs :-(
accept verify = sender/callout=60s,random
set acl_c8 = accept
deny set acl_c8 = deny
acl = acl_delay_escalator "sender verify fail"

> cannot distinguish network vs. real temporary errors

That's true. You might be able to do something using events.
--
Cheers,
Jeremy

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/

Hi,

D?a Sun, 1 May 2022 15:49:01 +0100 Jeremy Harris via Exim-users
<exim-users@exim.org> napísal:

> For instance, I have:

Thanks, seems to be better than the one of which i am aware, as callout
is doing only once.

> > cannot distinguish network vs. real temporary errors
>
> That's true. You might be able to do something using events.

I played with events some time ago, but i do understand how to use them,
or more precise they was not useful for purpose which i willed at that
time. I will rethink it...

regards

--
Slavko
https://www.slavino.sk