Mailing List Archive

Callout verification with SpamAssassin ?
Preface:

- Firsty: yes, I'm fully aware of all issues associated with
https://en.wikipedia.org/wiki/Callout_verification
(and there is a LOT of them!)

- I'm not looking for debate about general usefulness of Callout
verification (and the system for which it is being investigated is
not general-purpose e-mail system).

- I'm also not looking for alternative sender validations and related
schemes which might give similar results (like SPF / DKIM
verifications, SpamAssassin AWL/TxRep/whitelist_* etc.) but only
for checking sender via Callout verification.


The question:

I'm looking for existing solution to check in SpamAssassin (as a part
of custom complex set of meta rules) whether e-mail of the sender[1] has
recently[2] been "callout-verified" [3] by '250 Ok' response to RCPT TO.

The system in question has amavis / postfix beneath, if that helps
(so e.g. re-using postfix verify_cache.db is an option)

Is anyone aware of an existing SpamAssassin plugin or similar which
can do SMTP Callout verification?

Thanks,
Matija

[1] where sender is ideally header "From:" (possibly overriden by
"Reply-To:" header if it exists); but I'd settle for envelope
FROM too if that is the best that can be easily done

[2] caching for callout verification is implied and required; so
e-mail address which have already been queried won't be asked
again for some time.

[3] as noted at start, all caveats with Callout results are known
(e.g. that it does not guarantee that the sender actually exists
or that the e-mail to that address can actually be sent in the
future)

--
Opinions above are GNU-copylefted.
Re: Callout verification with SpamAssassin ? [ In reply to ]
On 2024-02-18 at 18:40:45 UTC-0500 (Mon, 19 Feb 2024 00:40:45 +0100)
Matija Nalis <mnalis-sa-list@voyager.hr>
is rumored to have said:

> Preface:
>
> - Firsty: yes, I'm fully aware of all issues associated with
> https://en.wikipedia.org/wiki/Callout_verification
> (and there is a LOT of them!)

Which is why SA does not support such so-called verification in any way.
It never will as long as I'm a contributor.

> - I'm not looking for debate about general usefulness of Callout
> verification (and the system for which it is being investigated is
> not general-purpose e-mail system).

This is a bit like saying you don't want to debate the general
usefulness of spamming. And then going on to ask about ways to spam. I
do not care about why you are trying to use SMTP callback verification,
because it is a fundamentally broken concept. If it looks like a
solution to you, you are refusing to look at solving your real problem.

All set then. SA is not the right tool for you. Try something like Exim,
MailMunge, or MIMEDefang that let you write arbitrary code for the
mail-handling flow. I suppose you may be able do it in sendmail.cf too,
if you're into self-torture.




--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire
Re: Callout verification with SpamAssassin ? [ In reply to ]
On Mon, Feb 19, 2024 at 02:38:03PM -0500, Bill Cole wrote:
> On 2024-02-18 at 18:40:45 UTC-0500 (Mon, 19 Feb 2024 00:40:45 +0100)
> Matija Nalis <mnalis-sa-list@voyager.hr> is rumored to have said:
> > - Firsty: yes, I'm fully aware of all issues associated with
> > https://en.wikipedia.org/wiki/Callout_verification
>
> Which is why SA does not support such so-called verification in any way. It
> never will as long as I'm a contributor.

It's fine, really. No need to get all upset :)

> > - I'm not looking for debate about general usefulness of Callout
> > verification (and the system for which it is being investigated is
> > not general-purpose e-mail system).
>
> This is a bit like saying you don't want to debate the general
> usefulness of spamming. And then going on to ask about ways to
> spam.

It's more like saying when one has a question about using SA, they
might not want engaging in a debate about why SA is written in perl
instead of in rust (even though the advantages of rust are obvious).

Or if they ask advice how to use SPF plugin, they might not want to
waste time discussing shortcomings of SPF as a technology.

Anyway, idea was to reduce spam, not promote it, so your comparison
sounds a little too negative :(
(Also, there is generally no need to discuss usefullness of spamming:
it is obviously profitable or people wouldn't be doing it)

> I do not care about why you are trying to use SMTP callback
> verification,

Just as well, as I'm not particularly inclined to waste significantly
more time trying to explain the details, especially if they've put
wall before themselves ("never will...")

> because it is a fundamentally broken concept.

Yes, it is broken to a certain extent. But to be fair, whole SMTP
itself is based on fundamentally broken concepts, don't you agree?

And most of SMTP workarounds are fundamentally broken too in many
ways. SPF is broken. RBLs are definitely way too broken. Pattern
matching is quite broken. Bayes is broken. etc.

That is IMHO why SA (still) exists at all -- each of the attempts of
fixing the broken SMTP is also broken (to some higher or lower
extent), so SA requires that *multiple* broken technologies become
*broken at the same time on the same email* before if produces false
positive (yet, both false positives and false negatives still happen
from time to time. Because all of it is broken, and we do not live in
perfect world).

> If it looks like a solution to you, you are refusing to look at
> solving your real problem.

Real problem is of course that e-mail is still the only universal
widespread communication technology which has not been silo-ed away
by megacoorps (yet; although they are working hard on that), and that
people are refusing to *universally* implement DKIM (or some other
sender authentication scheme) worldwide in my lifetime. And yes, in
this one (of hundreds others) use case it does look like improvement
to me (not a solution though - there is no "solution" for e-mail
except perhaps not using it at all).

> All set then. SA is not the right tool for you.

But it is. Although, this question was not for me.

Anyway, unless somebody else has seen something like that, I think
I'll just leave their postfix doing all-or-nothing mail rejection
based on callout verification during SMTP phase; instead of trying to
improve it by just assigning SA score based on it (and doing it on
much reduced number of cases).

(but just because I have other more important things to do at the
moment then dive into writing custom SA plugin; and not because doing
it in SA would not actually improve situation for everyone involved
in this particular case)

--
Opinions above are GNU-copylefted.