Mailing List Archive

On 2024-04-10 at 13:48:47 UTC-0400 (Wed, 10 Apr 2024 12:48:47 -0500)
Darrell Budic <budic@onholyground.com>
is rumored to have said:

> Just checking in here that I’m not doing something wrong with the ASN plugin before I file a bug on this. SpamAssassin 4.0.1 installed from cpan on Alma 9.
>
> I’ve got it configured to use the local maxmind db files, and those show up in logs. Testing in spamassassin itself show that it finds the ASN and includes it in the headers as expected. But when I let spamc/spamd process emails, the X-Spam-ASN headers do not appear. Enabling debug logging on spamd shows it does find the ASN properly, but doesn’t include the header. All my other add_header entries show up as expected.

This smells like a case of not using the config that you think you are.

> Relevant config:

Says you... :)

When you run the spamassassin script from the command line, it loads your user prefs from ~/.spamassassin/user_prefs and uses them. When you use spamc to talk to spamd, which prefs are loaded depends on your configuration of spamd, perhaps using only the global config, possibly using the config of the user running spamd, and possibly (with configuration of spamd that allows it to use per-user configs properly) that of arbitrary users per message.

Differences in how spamc/spamd and spamassassin on the command line behave are almost always due to this.
> report_safe 0
> ifplugin Mail::SpamAssassin::Plugin::ASN
> asn_prefix ''
> asn_lookup asn.routeviews.org _ASN_ _ASNCIDR_
> add_header all ASN _ASN_ _ASNCIDR_
>
> # IPv6 support (Bug 7211)
> asn_lookup_ipv6 origin6.asn.cymru.com _ASN_ _ASNCIDR_
> endif # Mail::SpamAssassin::Plugin::ASN
>
> From the spamd debug log:
>
> Wed Apr 10 17:06:50 2024 [2246409] dbg: geodb: GeoIP2: search found asn /usr/share/GeoIP/GeoLite2-ASN.mmdb
> Wed Apr 10 17:06:50 2024 [2246409] dbg: geodb: GeoIP2: loaded asn from /usr/share/GeoIP/GeoLite2-ASN.mmdb
> Wed Apr 10 17:07:09 2024 [2246418] dbg: asn: using GeoDB ASN for lookups
> Wed Apr 10 17:07:09 2024 [2246418] dbg: asn: using first external relay IP for lookups: 149.72.37.58
> Wed Apr 10 17:07:09 2024 [2246418] dbg: asn: GeoDB found ASN 11377
>
> There are no dgb: markup: entries for the ASN header.
>
> Anything I’m missing?

Look at the debug channel for config and etermine which config files are actually being used by spamd and by spamassassin. (spamc knows nothing of SA configs...)


--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

> On Apr 10, 2024, at 1:30?PM, Bill Cole <sausers-20150205@billmail.scconsult.com> wrote:
>
> On 2024-04-10 at 13:48:47 UTC-0400 (Wed, 10 Apr 2024 12:48:47 -0500)
> Darrell Budic <budic@onholyground.com <mailto:budic@onholyground.com>>
> is rumored to have said:
>
>> Just checking in here that I’m not doing something wrong with the ASN plugin before I file a bug on this. SpamAssassin 4.0.1 installed from cpan on Alma 9.
>>
>> I’ve got it configured to use the local maxmind db files, and those show up in logs. Testing in spamassassin itself show that it finds the ASN and includes it in the headers as expected. But when I let spamc/spamd process emails, the X-Spam-ASN headers do not appear. Enabling debug logging on spamd shows it does find the ASN properly, but doesn’t include the header. All my other add_header entries show up as expected.
>
> This smells like a case of not using the config that you think you are.

I keep thinking that, but the default ruleset's /var/lib/spamassassin/4.000001/updates_spamassassin_org/25_asn.cf should also add headers, and isn’t.

>> Relevant config:
>
> Says you... :)

Sure do :)

> When you run the spamassassin script from the command line, it loads your user prefs from ~/.spamassassin/user_prefs and uses them. When you use spamc to talk to spamd, which prefs are loaded depends on your configuration of spamd, perhaps using only the global config, possibly using the config of the user running spamd, and possibly (with configuration of spamd that allows it to use per-user configs properly) that of arbitrary users per message.
>
> Differences in how spamc/spamd and spamassassin on the command line behave are almost always due to this.

It certainly appears to be reading the right files. From the same debug log snipped earlier:

Wed Apr 10 17:06:48 2024 [2246409] dbg: plugin: loading Mail::SpamAssassin::Plugin::ASN from @INC
Wed Apr 10 17:06:50 2024 [2246409] dbg: plugin: Mail::SpamAssassin::Plugin::ASN=HASH(0x55c6b04063d8) implements 'extract_metadata', priority 0
Wed Apr 10 17:06:48 2024 [2246409] dbg: config: read file /etc/mail/spamassassin/custom.cf
Wed Apr 10 17:06:49 2024 [2246409] dbg: config: parsing file /etc/mail/spamassassin/custom.cf
Wed Apr 10 17:06:49 2024 [2246409] dbg: config: using "/var/lib/spamassassin/4.000001/updates_spamassassin_org/25_asn.cf" for included file
Wed Apr 10 17:06:49 2024 [2246409] dbg: config: read file /var/lib/spamassassin/4.000001/updates_spamassassin_org/25_asn.cf
Wed Apr 10 17:06:49 2024 [2246409] dbg: config: parsing file /var/lib/spamassassin/4.000001/updates_spamassassin_org/25_asn.cf
<previous snips go here>
Wed Apr 10 17:07:09 2024 [2246418] dbg: check: tagrun - tag ASN is now ready, value: 11377 SENDGRID
Wed Apr 10 17:07:09 2024 [2246418] dbg: asn: GeoDB lookup successful, skipping DNS lookup

The asn_prefix and add_header below are in /etc/mail/spamassassin/custom.cf.

>> report_safe 0
>> ifplugin Mail::SpamAssassin::Plugin::ASN
>> asn_prefix ''
>> asn_lookup asn.routeviews.org _ASN_ _ASNCIDR_
>> add_header all ASN _ASN_ _ASNCIDR_
>>
>> # IPv6 support (Bug 7211)
>> asn_lookup_ipv6 origin6.asn.cymru.com _ASN_ _ASNCIDR_
>> endif # Mail::SpamAssassin::Plugin::ASN
>>
>> From the spamd debug log:
>>
>> Wed Apr 10 17:06:50 2024 [2246409] dbg: geodb: GeoIP2: search found asn /usr/share/GeoIP/GeoLite2-ASN.mmdb
>> Wed Apr 10 17:06:50 2024 [2246409] dbg: geodb: GeoIP2: loaded asn from /usr/share/GeoIP/GeoLite2-ASN.mmdb
>> Wed Apr 10 17:07:09 2024 [2246418] dbg: asn: using GeoDB ASN for lookups
>> Wed Apr 10 17:07:09 2024 [2246418] dbg: asn: using first external relay IP for lookups: 149.72.37.58
>> Wed Apr 10 17:07:09 2024 [2246418] dbg: asn: GeoDB found ASN 11377
>>
>> There are no dgb: markup: entries for the ASN header.
>>
>> Anything I’m missing?
>
> Look at the debug channel for config and etermine which config files are actually being used by spamd and by spamassassin. (spamc knows nothing of SA configs…)

Spamassassin reads the same ones.

[root@sa0 spamassassin]# /usr/local/bin/spamassassin -t -D < ~telsin/testemail.eml 2>&1 | egrep -i 'asn'
Apr 10 19:18:24.185 [2249580] dbg: plugin: loading Mail::SpamAssassin::Plugin::ASN from @INC
Apr 10 19:18:24.628 [2249580] dbg: config: fixed relative path: /var/lib/spamassassin/4.000001/updates_spamassassin_org/25_asn.cf
Apr 10 19:18:24.628 [2249580] dbg: config: using "/var/lib/spamassassin/4.000001/updates_spamassassin_org/25_asn.cf" for included file
Apr 10 19:18:24.629 [2249580] dbg: config: read file /var/lib/spamassassin/4.000001/updates_spamassassin_org/25_asn.cf
Apr 10 19:18:24.629 [2249580] dbg: config: parsing file /var/lib/spamassassin/4.000001/updates_spamassassin_org/25_asn.cf
Apr 10 19:18:26.380 [2249580] dbg: geodb: GeoIP2: search found asn /usr/share/GeoIP/GeoLite2-ASN.mmdb
Apr 10 19:18:26.380 [2249580] dbg: geodb: GeoIP2: loaded asn from /usr/share/GeoIP/GeoLite2-ASN.mmdb
Apr 10 19:18:26.384 [2249580] dbg: geodb: database info: GeoIP2 asn: GeoLite2 ASN database / Fri Apr 5 13:11:32 2024
Apr 10 19:18:26.418 [2249580] dbg: plugin: Mail::SpamAssassin::Plugin::ASN=HASH(0x56217c8d3ea0) implements 'extract_metadata', priority 0
Apr 10 19:18:26.419 [2249580] dbg: asn: using GeoDB ASN for lookups
Apr 10 19:18:26.419 [2249580] dbg: asn: using first external relay IP for lookups: 209.85.214.174
Apr 10 19:18:26.419 [2249580] dbg: asn: GeoDB found ASN 15169
Apr 10 19:18:26.419 [2249580] dbg: check: tagrun - tag ASN is now ready, value: 15169 GOOGLE
Apr 10 19:18:26.419 [2249580] dbg: asn: GeoDB lookup successful, skipping DNS lookups
X-Spam-ASN: 15169 GOOGLE

Darrell Budic skrev den 2024-04-10 19:48:

> Anything I’m missing?

using amavisd ?

then try this in amavisd.conf:


@spam_scanners = (
# ['SpamAssassin', 'Amavis::SpamControl::SpamAssassin'],
['SpamdClient', 'Amavis::SpamControl::SpamdClient']
);

1; # insure a defined return value

if this works, its amavisd missing to add that header spamassassin add
in add-header

dont enable both spam_scanners, just one of them, and with the last
start spamd, as you have you already have this

would be nice if its just that

> On Apr 10, 2024, at 2:52?PM, Benny Pedersen <me@junc.eu> wrote:
>
> Darrell Budic skrev den 2024-04-10 19:48:
>
>> Anything I’m missing?
>
> using amavisd ?
>
> then try this in amavisd.conf:
>
>
> @spam_scanners = (
> # ['SpamAssassin', 'Amavis::SpamControl::SpamAssassin'],
> ['SpamdClient', 'Amavis::SpamControl::SpamdClient']
> );
>
> 1; # insure a defined return value
>
> if this works, its amavisd missing to add that header spamassassin add in add-header
>
> dont enable both spam_scanners, just one of them, and with the last start spamd, as you have you already have this
>
> would be nice if its just that
>

No, I”m using spamass-milter to send it over from postfix. Here’s my spamass-milter config in case I missed something there (systemd running it on alma 8 in this case):

EXTRA_FLAGS="-e onholyground.com -u defang -m -r 15 -i 127.0.0.1 -g sa-milt -- --max-size=5120000 --dest=sa0.int.ohgnetworks.com,sa1.int.ohgnetworks.com --randomize"

Both sa0 & sa1 run the same spamassassin/spamd configurations, neither of them add the X-Spam-ASN headers. All other add_header entries work fine.

On 2024-04-10 at 21:19:48 UTC-0400 (Wed, 10 Apr 2024 20:19:48 -0500)
Darrell Budic <budic@onholyground.com>
is rumored to have said:

>> On Apr 10, 2024, at 2:52?PM, Benny Pedersen <me@junc.eu> wrote:
>>
>> Darrell Budic skrev den 2024-04-10 19:48:
>>
>>> Anything I’m missing?
>>
>> using amavisd ?
>>
>> then try this in amavisd.conf:
>>
>>
>> @spam_scanners = (
>> # ['SpamAssassin', 'Amavis::SpamControl::SpamAssassin'],
>> ['SpamdClient', 'Amavis::SpamControl::SpamdClient']
>> );
>>
>> 1; # insure a defined return value
>>
>> if this works, its amavisd missing to add that header spamassassin add in add-header
>>
>> dont enable both spam_scanners, just one of them, and with the last start spamd, as you have you already have this
>>
>> would be nice if its just that
>>
>
> No, I”m using spamass-milter to send it over from postfix. Here’s my spamass-milter config in case I missed something there (systemd running it on alma 8 in this case):
>
> EXTRA_FLAGS="-e onholyground.com -u defang -m -r 15 -i 127.0.0.1 -g sa-milt -- --max-size=5120000 --dest=sa0.int.ohgnetworks.com,sa1.int.ohgnetworks.com --randomize"

That's intriguing because "-u defang" looks like cargo-cult spoor from an installation running MIMEDefang. Does the user 'defang' have appropriate configs?

> Both sa0 & sa1 run the same spamassassin/spamd configurations, neither of them add the X-Spam-ASN headers. All other add_header entries work fine.

Validate that configs on both machines match. In this sort of setup, only the SA config on the spamd hosts of the user spamd is run as makes any difference.

--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

On Apr 11, 2024, at 3:30?PM, Bill Cole <sausers-20150205@billmail.scconsult.com> wrote:
>
> On 2024-04-10 at 21:19:48 UTC-0400 (Wed, 10 Apr 2024 20:19:48 -0500)
> Darrell Budic <budic@onholyground.com <mailto:budic@onholyground.com>>
> is rumored to have said:
>
>>> On Apr 10, 2024, at 2:52?PM, Benny Pedersen <me@junc.eu> wrote:
>>>
>>> Darrell Budic skrev den 2024-04-10 19:48:
>>>
>>>> Anything I’m missing?
>>>
>>> using amavisd ?
>>>
>>> then try this in amavisd.conf:
>>
>> No, I”m using spamass-milter to send it over from postfix. Here’s my spamass-milter config in case I missed something there (systemd running it on alma 8 in this case):
>>
>> EXTRA_FLAGS="-e onholyground.com -u defang -m -r 15 -i 127.0.0.1 -g sa-milt -- --max-size=5120000 --dest=sa0.int.ohgnetworks.com,sa1.int.ohgnetworks.com --randomize"
>
> That's intriguing because "-u defang" looks like cargo-cult spoor from an installation running MIMEDefang. Does the user 'defang' have appropriate configs?

It is indeed, leftover user stuff from before I migrated to postfix and spamass-milter with a database backend for SA prefs. It’s still a valid default user with appropriate configs, but the -e default domain takes precedence so I can have per domain SA policies. Users too, for that matter, but that’s handled by the sql setup.

>> Both sa0 & sa1 run the same spamassassin/spamd configurations, neither of them add the X-Spam-ASN headers. All other add_header entries work fine.
>
> Validate that configs on both machines match. In this sort of setup, only the SA config on the spamd hosts of the user spamd is run as makes any difference.

I push them using ansible, but yeah, a quick audit to double check confirms they are the same.

> On Apr 11, 2024, at 5:51?PM, Darrell Budic <budic@onholyground.com> wrote:
>
> On Apr 11, 2024, at 3:30?PM, Bill Cole <sausers-20150205@billmail.scconsult.com> wrote:
>>
>> On 2024-04-10 at 21:19:48 UTC-0400 (Wed, 10 Apr 2024 20:19:48 -0500)
>> Darrell Budic <budic@onholyground.com <mailto:budic@onholyground.com>>
>> is rumored to have said:
>>
>>>> On Apr 10, 2024, at 2:52?PM, Benny Pedersen <me@junc.eu> wrote:
>>>>
>>>> Darrell Budic skrev den 2024-04-10 19:48:
>>>>
>>>>> Anything I’m missing?
>>>>
>>>> using amavisd ?
>>>>
>>>> then try this in amavisd.conf:
>>>
>>> No, I”m using spamass-milter to send it over from postfix. Here’s my spamass-milter config in case I missed something there (systemd running it on alma 8 in this case):
>>>
>>> EXTRA_FLAGS="-e onholyground.com -u defang -m -r 15 -i 127.0.0.1 -g sa-milt -- --max-size=5120000 --dest=sa0.int.ohgnetworks.com,sa1.int.ohgnetworks.com —randomize"
>>


Found it, even with the -m, spamass-milter only replaces a hardcoded set of X-Spam-* headers, not anything that comes back from spamd. With some more work, I was able to confirm that spamc/spamd were indeed including the headers where they were supposed to be.

Thanks for the help tracking it down, I’m going to reconsider my preference for milters here ;)