Mailing List Archive

Weirdness when Exim calls SpamAssassin - how to debug?
Hi all,

I'm running into an issue with the way Exim 4.94.2 invokes
SpamAssassin (SA).

I have added some custom rules to SA, however when Exim invokes it, my
custom rules are ignored.

However if I use the "spamc" command to manually pass an e-mail into SA
(or if I run SA from the command line) then my custom rules are picked
up correctly.

I originally thought this was a SA problem, however I am unable to
reproduce the issue using SA tools, it is only when Exim calls SA that
I see the problem.

Is there any sort of test mode I can use to see how exactly Exim is
passing the message to SA so I can further debug the problem? I've had
a look through the command line docs and I can only see test modes for
virus scanners, but nothing stands out for the spam checker.

I'm specifying SA in the Exim config as:

spamd_address = 127.0.0.1 783

It runs correctly and the default rules are applied, with the e-mail
being delivered with newly added spam headers, so everything appears to
be working fine. I have added many custom rules in the past (with the
same Exim config) and it has worked fine, it's just the latest custom
rule I added only gets used when *I* call SA, not when Exim calls it,
and I haven't been able to work out why.

Many thanks,
Adam.

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: Weirdness when Exim calls SpamAssassin - how to debug? [ In reply to ]
On 29/11/2022 23:39, Adam Nielsen via Exim-users wrote:
> Is there any sort of test mode I can use to see how exactly Exim is
> passing the message to SA so I can further debug the problem?

Exim takes commandline options to request debug.
--
Cheers,
Jeremy


--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: Weirdness when Exim calls SpamAssassin - how to debug? [ In reply to ]
On Wed, 30 Nov 2022, Adam Nielsen via Exim-users wrote:

> Hi all,
>
> I'm running into an issue with the way Exim 4.94.2 invokes
> SpamAssassin (SA).
>
> I have added some custom rules to SA, however when Exim invokes it, my
> custom rules are ignored.
>
> However if I use the "spamc" command to manually pass an e-mail into SA
> (or if I run SA from the command line) then my custom rules are picked
> up correctly.
>
> I originally thought this was a SA problem, however I am unable to
> reproduce the issue using SA tools, it is only when Exim calls SA that
> I see the problem.
>
> Is there any sort of test mode I can use to see how exactly Exim is
> passing the message to SA so I can further debug the problem? I've had
> a look through the command line docs and I can only see test modes for
> virus scanners, but nothing stands out for the spam checker.
>
> I'm specifying SA in the Exim config as:
>
> spamd_address = 127.0.0.1 783
>
> It runs correctly and the default rules are applied, with the e-mail
> being delivered with newly added spam headers, so everything appears to
> be working fine. I have added many custom rules in the past (with the
> same Exim config) and it has worked fine, it's just the latest custom
> rule I added only gets used when *I* call SA, not when Exim calls it,
> and I haven't been able to work out why.

Could there be something in your environment variables that is not or is
different in those of the exim user (or rather, the user that SA runs as
when started by exim) ?

Exim clears its environment unless you use keep_environment and/or
add_environment:
https://www.exim.org/exim-html-current/doc/html/spec_html/ch-main_configuration.html

--
Andrew C. Aitchison Kendal, UK
andrew@aitchison.me.uk

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: Weirdness when Exim calls SpamAssassin - how to debug? [ In reply to ]
On 2022-11-29 at 18:39:49 UTC-0500 (Wed, 30 Nov 2022 09:39:49 +1000)
Adam Nielsen via Exim-users <a.nielsen@shikadi.net>
is rumored to have said:

> Hi all,
>
> I'm running into an issue with the way Exim 4.94.2 invokes
> SpamAssassin (SA).
>
> I have added some custom rules to SA, however when Exim invokes it, my
> custom rules are ignored.

Where did you add the rules?

SA uses 3 sets of rules:

1. Channel rules. Typically only the default rule channel from the ASF
SA project itself but also any others maintained with sa-update, such as
the popular KAM channel. Location varies b y distribution/packager
choice, usually something like /var/{db,lib,share,}/spamassassin/<SA

2. Site-wide local rules in *.cf text files. Typically
/etc/mail/spamassassin/ but sometimes elsewhere depending on packager
choices.

3. Per-User local rules. These are all the *.cf files in
$HOME/.spamassassin/

1 & 2 should be loaded no matter how you use SA. 3 can be finicky,
especially when using SA via the spamd daemon.

> However if I use the "spamc" command to manually pass an e-mail into
> SA
> (or if I run SA from the command line) then my custom rules are picked
> up correctly.

This looks like SA (specifically in this case: the spamd daemon) is
using different rulesets between Exim and interactive calls. The only
reason that should happen in this pattern is if you've given spamd the
right arguments for per-user rules but Exim and spamc are not running as
the same user OR spamd cannot properly select or access per-user rules
when called from Exim OR spamd has not loaded the current rules.

The last case has the simplest possible solution: just restart spamd.

> I originally thought this was a SA problem, however I am unable to
> reproduce the issue using SA tools, it is only when Exim calls SA that
> I see the problem.

Right, but it is still a SA problem.

The "spamassassin" script always uses the currently effective user's
per-user rules and the current site-wide rules, as it doesn't use spamd.
If you use spamc, you may have environment variables that affect which
per-user rules (if any) it loads and you may be talking to a fresh spamd
child instance which has the latest rules loaded, or you may be talking
to an older one that has a stale ruleset.

It's also possible that you have spamd trying to load per-user rules
that it can find for your command-line invocation but cannot determine
when Exim is calling it.

> Is there any sort of test mode I can use to see how exactly Exim is
> passing the message to SA so I can further debug the problem? I've
> had
> a look through the command line docs and I can only see test modes for
> virus scanners, but nothing stands out for the spam checker.
>
> I'm specifying SA in the Exim config as:
>
> spamd_address = 127.0.0.1 783

That eliminates almost everything that Exim could do "wrong" in calling
SA, given a reasonably normal config (i.e. no embedded Perl.) The spamd
client protocol is very simple and spamd is not tolerant of deviations.

So the problem is in what spamd is doing. It is possible to run spamd
with debug flags (-D <list of diag classes>) to get its activity
details logged but that is not a reasonable thing to do on a high-volume
system.

> It runs correctly and the default rules are applied, with the e-mail
> being delivered with newly added spam headers, so everything appears
> to
> be working fine. I have added many custom rules in the past (with the
> same Exim config) and it has worked fine, it's just the latest custom
> rule I added only gets used when *I* call SA, not when Exim calls it,
> and I haven't been able to work out why.

Simplest possible solution: restart spamd. That would fix the stale
ruleset possibility.

Without knowing your specific spamd config it is impossible to guess
what a fix for a more complex issue with ruleset selection might be...

--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/
Re: Weirdness when Exim calls SpamAssassin - how to debug? [ In reply to ]
On 30/11/2022 00:39, Adam Nielsen via Exim-users wrote:
> Is there any sort of test mode I can use to see how exactly Exim is
> passing the message to SA so I can further debug the problem? I've had
> a look through the command line docs and I can only see test modes for
> virus scanners, but nothing stands out for the spam checker.

You can put
warn control = no_mbox_unspool
at the beginning of your data ACL. See
https://www.exim.org/exim-html-current/doc/html/spec_html/ch-access_control_lists.html
chapter 22 for details.

Be warned! That keeps ALL .eml files in /var/spool/exim/scan until you
remove it again (ACL and files). This can be critical on high volume hosts
and a condition may help keeping things under control.

Then you can "feed" the exact same copy from the scan folder which exim
gave spamd again or use spamassassin -D to see what happens.

Be sure to use the same local user exim uses to "feed" spamd (spamc -u ...).

Greetings, Wolfgang
--
Wolfgang Breyha <wbreyha@gmx.net> | https://www.blafasel.at/
Vienna University Computer Center | Austria

--
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/