Mailing List Archive: sa daemon loads config different to shell ?

sa daemon loads config different to shell ?

frank at powerweb

Jul 27, 2021, 1:44 AM

Post #1 of 9 (1005 views)

Hi,

Im using SA 3.4 on CentOS 7 via milter.
And have a asn.cf in /etc/mail/spamassassin like this:

loadplugin Mail::SpamAssassin::Plugin::ASN
asn_lookup asn.routeviews.org _ASN_ _ASNCIDR_
add_header all ASN _ASN_ _ASNCIDR_

This gets ignored, when sendmail runs mails through spamass-milter and
spamd, there is no X-Spam-ASN nor a X-Spam-Source header inserted.

But it works, when filtering a mail manually via shell like this

# spamassassin < <mailfile>
resulting in the expected headers:
X-Spam-ASN: AS5669 194.77.152.0/23
X-Spam-Source: 194.77.152.124

So I started the spamd in debug-mode to track this down and it logged this:

Jul 26 23:04:40 topixx2 spamd[22655]: config: read file
/etc/mail/spamassassin/asn.cf
Jul 26 23:04:40 topixx2 spamd[22655]: config: fixed relative path:
/var/lib/spamassassin/3.004000/updates_spamassassin_org/25_asn.cf
Jul 26 23:04:40 topixx2 spamd[22655]: config: using
"/var/lib/spamassassin/3.004000/updates_spamassassin_org/25_asn.cf" for
included file
Jul 26 23:04:40 topixx2 spamd[22655]: config: read file
/var/lib/spamassassin/3.004000/updates_spamassassin_org/25_asn.cf
Jul 26 23:04:40 topixx2 spamd[22655]: plugin: loading
Mail::SpamAssassin::Plugin::ASN from @INC
Jul 26 23:04:41 topixx2 spamd[22655]: plugin:
Mail::SpamAssassin::Plugin::ASN=HASH(0x3330530) implements
'parsed_metadata', priority 0
Jul 26 23:04:41 topixx2 spamd[22655]: asn: no first external relay IP
available, skipping ASN check

Im I right, that it looks like, if the setting in my asn.cf could be
overwritten with the default ?

Other .cf-files located in /etc/mail/spamassassin load all right and
show the estimated result, only asn.cf somehow fails.

Where should I look to find the differences in the config of
spamassassin and spamd ?

Mit freundlichen Gruessen, Frank Gadegast
--
PHADE Software - PowerWeb http://www.powerweb.de
Inh. Dipl.-Inform. Frank Gadegast mailto:frank@powerweb.de
Schinkelstrasse 17 fon: +49 33200 52920
14558 Nuthetal OT Rehbruecke, Germany fax: +49 33200 52921
======================================================================

Re: sa daemon loads config different to shell ? [ In reply to ]

frank at powerweb

Jul 27, 2021, 2:52 AM

Post #2 of 9 (1005 views)

On 27.07.21 11:18, Reindl Harald wrote:
>
>

Hi Harald,

> Am 27.07.21 um 10:44 schrieb Dipl-Inform. Frank Gadegast:
>> Im using SA 3.4 on CentOS 7 via milter.
>> And have a asn.cf in /etc/mail/spamassassin like this:
>>
>> loadplugin Mail::SpamAssassin::Plugin::ASN
>> asn_lookup asn.routeviews.org _ASN_ _ASNCIDR_
>> add_header all ASN _ASN_ _ASNCIDR_
>>
>> This gets ignored, when sendmail runs mails through spamass-milter and
>> spamd, there is no X-Spam-ASN nor a X-Spam-Source header inserted
>
> the problem here is that spamass-milter has it's own idea of which
> headers are present - god only knows why
> your asn.cf is running fine ...

Yes, I tweaked the asn.cf to create a test that will be listed in the
X-Spam-Status. The asn.cf really works, even when loading via
spamass-milter ...

> but spamass-milter throws away the header

... also right. spamass-milter is removing the header lines.

Yes, I found this here
https://github.com/andybalholm/spamass-milter/issues/11

Seems to be, that spamass-milter simply strippes out any X-Spam* header
lines, not caring, if the own call to spamd sets them, hm.

Im really not getting, why spamass-milter should strip X-Spam-lines of
the header AFTER SA was running. If Im right, SA is stripping them of
anyway, before running or modifying anything ...

Anybody an idea how to get arround this ?

--

Mit freundlichen Gruessen, Frank Gadegast
--
MOTD: "have you enabled SSL on a website or mailbox today ?"
--
PHADE Software - PowerWeb http://www.powerweb.de
Inh. Dipl.-Inform. Frank Gadegast mailto:frank@powerweb.de
Schinkelstrasse 17 fon: +49 33200 52920
14558 Nuthetal OT Rehbruecke, Germany fax: +49 33200 52921
======================================================================

Re: sa daemon loads config different to shell ? [ In reply to ]

jahlives at gmx

Jul 27, 2021, 4:17 AM

Post #3 of 9 (1005 views)

> Jul 26 23:04:41 topixx2 spamd[22655]: asn: no first external relay IP
available, skipping ASN check

the plugin seems to be loaded but as no external relay IP found there is
nothing to check. Ensure that your trusted_networks and/or
internal_networks is set correctly.

Cheers

tobi

On 7/27/21 10:44 AM, Dipl-Inform. Frank Gadegast wrote:
>
> Hi,
>
> Im using SA 3.4 on CentOS 7 via milter.
> And have a asn.cf in /etc/mail/spamassassin like this:
>
> loadplugin Mail::SpamAssassin::Plugin::ASN
> asn_lookup asn.routeviews.org _ASN_ _ASNCIDR_
> add_header all ASN _ASN_ _ASNCIDR_
>
> This gets ignored, when sendmail runs mails through spamass-milter and
> spamd, there is no X-Spam-ASN nor a X-Spam-Source header inserted.
>
> But it works, when filtering a mail manually via shell like this
>
> # spamassassin < <mailfile>
> resulting in the expected headers:
> X-Spam-ASN: AS5669 194.77.152.0/23
> X-Spam-Source: 194.77.152.124
>
> So I started the spamd in debug-mode to track this down and it logged
> this:
>
> Jul 26 23:04:40 topixx2 spamd[22655]: config: read file
> /etc/mail/spamassassin/asn.cf
> Jul 26 23:04:40 topixx2 spamd[22655]: config: fixed relative path:
> /var/lib/spamassassin/3.004000/updates_spamassassin_org/25_asn.cf
> Jul 26 23:04:40 topixx2 spamd[22655]: config: using
> "/var/lib/spamassassin/3.004000/updates_spamassassin_org/25_asn.cf"
> for included file
> Jul 26 23:04:40 topixx2 spamd[22655]: config: read file
> /var/lib/spamassassin/3.004000/updates_spamassassin_org/25_asn.cf
> Jul 26 23:04:40 topixx2 spamd[22655]: plugin: loading
> Mail::SpamAssassin::Plugin::ASN from @INC
> Jul 26 23:04:41 topixx2 spamd[22655]: plugin:
> Mail::SpamAssassin::Plugin::ASN=HASH(0x3330530) implements
> 'parsed_metadata', priority 0
> Jul 26 23:04:41 topixx2 spamd[22655]: asn: no first external relay IP
> available, skipping ASN check
>
> Im I right, that it looks like, if the setting in my asn.cf could be
> overwritten with the default ?
>
> Other .cf-files located in /etc/mail/spamassassin load all right and
> show the estimated result, only asn.cf somehow fails.
>
>
> Where should I look to find the differences in the config of
> spamassassin and spamd ?
>
>
>
>
> Mit freundlichen Gruessen, Frank Gadegast
> --
> PHADE Software - PowerWeb                       http://www.powerweb.de
> Inh. Dipl.-Inform. Frank Gadegast             mailto:frank@powerweb.de
> Schinkelstrasse 17                                fon: +49 33200 52920
> 14558 Nuthetal OT Rehbruecke, Germany             fax: +49 33200 52921
> ======================================================================
>

Re: sa daemon loads config different to shell ? [ In reply to ]

dbuergin at gluet

Jul 27, 2021, 5:18 AM

Post #4 of 9 (1005 views)

Dipl-Inform. Frank Gadegast:
> Seems to be, that spamass-milter simply strippes out any X-Spam* header lines, not caring, if the own call to spamd sets them, hm.
>
> Im really not getting, why spamass-milter should strip X-Spam-lines of the header AFTER SA was running. If Im right, SA is stripping them of anyway, before running or modifying anything ...
>
>
> Anybody an idea how to get arround this ?

There is an alternative milter (which I maintain) that adds
all X-Spam-* headers received from spamd.

https://crates.io/crates/spamassassin-milter

Re: sa daemon loads config different to shell ? [ In reply to ]

uhlar at fantomas

Jul 27, 2021, 5:51 AM

Post #5 of 9 (1005 views)

>Dipl-Inform. Frank Gadegast:
>>Seems to be, that spamass-milter simply strippes out any X-Spam* header lines, not caring, if the own call to spamd sets them, hm.
>>
>>Im really not getting, why spamass-milter should strip X-Spam-lines of the header AFTER SA was running. If Im right, SA is stripping them of anyway, before running or modifying anything ...
>>
>>
>>Anybody an idea how to get arround this ?

On 27.07.21 14:18, David B?rgin wrote:
>There is an alternative milter (which I maintain) that adds
>all X-Spam-* headers received from spamd.

the original milter does the same. Adds headers from spamd.
However, it does NOT take into account ay X-Spam-* headers received from
remote server.

--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
REALITY.SYS corrupted. Press any key to reboot Universe.

Re: sa daemon loads config different to shell ? [ In reply to ]

dbuergin at gluet

Jul 27, 2021, 6:29 AM

Post #6 of 9 (1005 views)

Matus UHLAR - fantomas:
> On 27.07.21 14:18, David Bürgin wrote:
>> There is an alternative milter (which I maintain) that adds
>> all X-Spam-* headers received from spamd.
>
> the original milter does the same. Adds headers from spamd.
> However, it does NOT take into account ay X-Spam-* headers received from
> remote server.

The point is that spamass-milter does not add custom headers like
‘X-Spam-ASN’ received from spamd, only a few select ones. The
alternative milter on the other hand adds all X-Spam- headers.

This is also what’s discussed in the linked GitHub issue.

Re: sa daemon loads config different to shell ? [ In reply to ]

uhlar at fantomas

Jul 27, 2021, 6:32 AM

Post #7 of 9 (1005 views)

>>On 27.07.21 14:18, David Bürgin wrote:
>>>There is an alternative milter (which I maintain) that adds
>>>all X-Spam-* headers received from spamd.

>Matus UHLAR - fantomas:
>>the original milter does the same. Adds headers from spamd.
>>However, it does NOT take into account ay X-Spam-* headers received from
>>remote server.

On 27.07.21 15:29, David Bürgin wrote:
>The point is that spamass-milter does not add custom headers like
>‘X-Spam-ASN’ received from spamd, only a few select ones. The
>alternative milter on the other hand adds all X-Spam- headers.
>
>This is also what’s discussed in the linked GitHub issue.

sorry then, I think I mistook this with other thread...

--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
WinError #99999: Out of error messages.

Re: sa daemon loads config different to shell ? [ In reply to ]

dbuergin at gluet

Jul 27, 2021, 6:37 AM

Post #8 of 9 (1005 views)

Dipl-Inform. Frank Gadegast:
> On 27.07.21 14:18, David Bürgin wrote:
>> Dipl-Inform. Frank Gadegast:
>>> Seems to be, that spamass-milter simply strippes out any X-Spam* header lines, not caring, if the own call to spamd sets them, hm.
>>>
>>> Im really not getting, why spamass-milter should strip X-Spam-lines of the header AFTER SA was running. If Im right, SA is stripping them of anyway, before running or modifying anything ...
>>>
>>>
>>> Anybody an idea how to get arround this ?
>>
>> There is an alternative milter (which I maintain) that adds
>> all X-Spam-* headers received from spamd.
>>
>> https://crates.io/crates/spamassassin-milter
>
> Looks like your milter needs to fork a spamc, wich then talks to the spamd. This will start lots of spamc processes and is not recommened.
>
> Would then not be any different to call spamc dirctly f.e. via procmail.
>
> You should rewrite your milter to talk directly to the spamd via socket or port.

Yes, it communicates using spamc, just like spamass-milter.

I have been told that it has been working fine in a somewhat larger
deployment. I didn’t mean to derail the thread so will leave it at that.

Ciao,
David

Re: spamass-milter (sa daemon loads config different to shell ?) [ In reply to ]

dbfunk at engineering

Jul 27, 2021, 8:09 AM

Post #9 of 9 (1005 views)

On Tue, 27 Jul 2021, David Bürgin wrote:

> Dipl-Inform. Frank Gadegast:
>> On 27.07.21 14:18, David Bürgin wrote:
>>> Dipl-Inform. Frank Gadegast:
>>>> Seems to be, that spamass-milter simply strippes out any X-Spam* header
>>>> lines, not caring, if the own call to spamd sets them, hm.
>>>>
>>>> Im really not getting, why spamass-milter should strip X-Spam-lines of
>>>> the header AFTER SA was running. If Im right, SA is stripping them of
>>>> anyway, before running or modifying anything ...
>>>>
>>>>
>>>> Anybody an idea how to get arround this ?
>>>
>>> There is an alternative milter (which I maintain) that adds
>>> all X-Spam-* headers received from spamd.
>>>
>>> https://crates.io/crates/spamassassin-milter
>>
>> Looks like your milter needs to fork a spamc, wich then talks to the spamd.
>> This will start lots of spamc processes and is not recommened.
>>
>> Would then not be any different to call spamc dirctly f.e. via procmail.
>>
>> You should rewrite your milter to talk directly to the spamd via socket or
>> port.
>
> Yes, it communicates using spamc, just like spamass-milter.
>
> I have been told that it has been working fine in a somewhat larger
> deployment. I didn’t mean to derail the thread so will leave it at that.

having a spam filtering milter fork off a shell and then run "spamc" to
communicate with spamd does simplify the milter code (and insulates it from
changes in the spamd protocol) but adds risk of shell escape attacks (as well as
additional overhead).
There's already been security related patches needed by spamass-milter
specifically because of this issue.

Writing a milter that directly talks the spamd protocol via a socket (local or
network) is more work but safer and more efficient.
(been there, done that, got the code to prove it).

--
Dave Funk University of Iowa
<dbfunk (at) engineering.uiowa.edu> College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center, 103 S Capitol St.
Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
#include <std_disclaimer.h>
Better is not better, 'standard' is better. B{