Mailing List Archive

Bill Cole wrote:
> I'm not sure how I've not noticed before, but unless I'm missing
> something, there is no way to replicate the [block,welcome]list
> functionalities of the spamassassin script when using the spamc/spamd
> interface.
>
> Does anyone see it hiding somewhere that I don't?
>
> Does anyone have any rationale for this missing functionality?
>
> I don't expect that it would be difficult to add. (Something I've
> believed every time I've taken on a coding task...)

I'm pretty sure you're doing something wrong (maybe a missing/commented
loadplugin entry somewhere? Would expect that to bite the spamassassin
script too tho), because this has been supported for a long time. On a
single standalone system, IME you'd have to go to extra effort to get
spamc/spamd to use a different config than the spamassassin script - if
it works for one, it should work for the other.

I have file-based userprefs on my personal colo server with both[1] in
use and working just fine for a long time (since early SA 2.x at least,
IIRC), spamc called from promail on delivery. Wearing my work hat we
have all three of: local configuration in /etc, local rules channel
configuration, and SQL-based userprefs also using block|welcome entries
of several types, all working just fine[2] with spamc called on delivery
from a custom local delivery agent.

I've just rechecked with SA 4 trunk, and a temporary spamd instance was
quite happy to load an existing .cf containing a loooong list of local
welcomelist entries[3], and correctly hit USER_IN_DKIM_WELCOMELIST more
or less completely fresh out of the box.

The plugins are spread around a bit:

init.pre: SPF supports welcomelist_from_spf
v3.12.pre: DKIM supports welcomelist_from_dkim
v3.20.pre: WLBLEval supports welcomelist_from[_rcvd]

welcomlist_auth seems to be some internal voodoo layered on top of
welcomelist_from_dkim and welcomelist_from_spf, at a quick rummage it
seemed to be quite happy to function and hit with *either* the SPF or
DKIM plugins enabled, modulo having suitable thing for welcomelist_auth
to be looking at.

-kgd

[1] Well, legacy whitelist_*/blacklist_*, on account of having dragged
the config along for so long...

[2] Aside from testing mail well after the fact, where it was sent
through one or another bulk mail platform that sets a ridiculously short
DKIM expiry timestamp. Which isn't SA's fault, although it would be
nice to have a command flag somewhere to force DKIM processing to be "as
of Received: timestamp" or "as of between timestamps in DKIM header" so
as to better confirm if there's even a point in adding the welcomelist_*
entry.

[3] Also technically legacy whitelist_*, but the rule and config
aliasing also worked fine.

On 20.03.24 16:58, Bill Cole wrote:
>I'm not sure how I've not noticed before, but unless I'm missing
>something, there is no way to replicate the [block,welcome]list
>functionalities of the spamassassin script when using the spamc/spamd
>interface.
>
>Does anyone see it hiding somewhere that I don't?
>
>Does anyone have any rationale for this missing functionality?
>
>I don't expect that it would be difficult to add. (Something I've
>believed every time I've taken on a coding task...)

How/where did you try to define it?

"spamc -u" should pass username to spamd which then should use that users'
user_prefs file (if it exists) unless spamd was started with "-x" parameter
or can't access that file.



--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
"Where do you want to go to die?" [Microsoft]

On 3/20/24 21:58, Bill Cole wrote:
> I'm not sure how I've not noticed before, but unless I'm missing something, there is no way to replicate the [block,welcome]list functionalities of the spamassassin script when using the spamc/spamd interface.
>
> Does anyone see it hiding somewhere that I don't?
>
> Does anyone have any rationale for this missing functionality?
>
> I don't expect that it would be difficult to add. (Something I've believed every time I've taken on a coding task...)
>
are you referring to spamassassin -W/-R options that are not present on spamc(1) ?
Giovanni

On 2024-03-21 at 11:57:43 UTC-0400 (Thu, 21 Mar 2024 11:57:43 -0400)
Kris Deugau <kdeugau@vianet.ca>
is rumored to have said:

> Bill Cole wrote:
>> I'm not sure how I've not noticed before, but unless I'm missing
>> something, there is no way to replicate the [block,welcome]list
>> functionalities of the spamassassin script when using the spamc/spamd
>> interface.
>>
>> Does anyone see it hiding somewhere that I don't?
>>
>> Does anyone have any rationale for this missing functionality?
>>
>> I don't expect that it would be difficult to add. (Something I've
>> believed every time I've taken on a coding task...)
>
> I'm pretty sure you're doing something wrong (maybe a
> missing/commented loadplugin entry somewhere?

Maybe you've misunderstood my question. The spamc/spamd system uses
whatever AWL or TxRep DB is configured in evaluating messages and does
the automated part of managing those. The spamassassin-run man page
refers to spamd in the description of -W, -R, and various
--{add,remove}_*list options, but I see no way that they are relevant to
spamd.

> Would expect that to bite the spamassassin script too tho), because
> this has been supported for a long time.

Great. Please enlighten me:

How do I feed spamc a message and tell it to add or remove all
addresses
in it to the global or user welcomelist or blocklist (using +/-
$bignum
in the AWL or TxRep list) or tell it to add or remove individual
addresses
from those lists?

There is no mechanism to do that which I have found in our documentation
or code. I have re-read the PROTOCOL document, which does not seem to
document any mechanism to manipulate the reputation DB.

The spamassassin script does these things, usually by directly accessing
file-based DBs in the local ~/.spamassassin. The strongest reason to use
spamc/spamd is to have spamd on a different machine, so you cannot
depend on the machine where spamc is running having a sane SA config or
even a fully working SA installation.

> On a single standalone system, IME you'd have to go to extra effort to
> get spamc/spamd to use a different config than the spamassassin script
> - if it works for one, it should work for the other.

On one machine, sure: you can just use the spamassassin script to work
with the same AWL or TxRep DB that spamd uses. That's not a relevant
case.

The solution I've used it is to have both the spamc and spamd machines
using the same configuration, with both having identical SA
installations and config and being able to access the same reputation
DB. That is sub-optimal.

>
> I have file-based userprefs on my personal colo server with both[1] in
> use and working just fine for a long time (since early SA 2.x at
> least, IIRC), spamc called from promail on delivery. Wearing my work
> hat we have all three of: local configuration in /etc, local rules
> channel configuration, and SQL-based userprefs also using
> block|welcome entries of several types, all working just fine[2] with
> spamc called on delivery from a custom local delivery agent.
>
> I've just rechecked with SA 4 trunk, and a temporary spamd instance
> was quite happy to load an existing .cf containing a loooong list of
> local welcomelist entries[3], and correctly hit
> USER_IN_DKIM_WELCOMELIST more or less completely fresh out of the box.
>
> The plugins are spread around a bit:
>
> init.pre: SPF supports welcomelist_from_spf
> v3.12.pre: DKIM supports welcomelist_from_dkim
> v3.20.pre: WLBLEval supports welcomelist_from[_rcvd]
>
> welcomlist_auth seems to be some internal voodoo layered on top of
> welcomelist_from_dkim and welcomelist_from_spf, at a quick rummage it
> seemed to be quite happy to function and hit with *either* the SPF or
> DKIM plugins enabled, modulo having suitable thing for
> welcomelist_auth to be looking at.
>
> -kgd
>
> [1] Well, legacy whitelist_*/blacklist_*, on account of having dragged
> the config along for so long...
>
> [2] Aside from testing mail well after the fact, where it was sent
> through one or another bulk mail platform that sets a ridiculously
> short DKIM expiry timestamp. Which isn't SA's fault, although it
> would be nice to have a command flag somewhere to force DKIM
> processing to be "as of Received: timestamp" or "as of between
> timestamps in DKIM header" so as to better confirm if there's even a
> point in adding the welcomelist_* entry.
>
> [3] Also technically legacy whitelist_*, but the rule and config
> aliasing also worked fine.


--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

On 2024-03-21 at 12:08:48 UTC-0400 (Thu, 21 Mar 2024 17:08:48 +0100)
Matus UHLAR - fantomas <uhlar@fantomas.sk>
is rumored to have said:

> On 20.03.24 16:58, Bill Cole wrote:
>> I'm not sure how I've not noticed before, but unless I'm missing
>> something, there is no way to replicate the [block,welcome]list
>> functionalities of the spamassassin script when using the spamc/spamd
>> interface.
>>
>> Does anyone see it hiding somewhere that I don't?
>>
>> Does anyone have any rationale for this missing functionality?
>>
>> I don't expect that it would be difficult to add. (Something I've
>> believed every time I've taken on a coding task...)
>
> How/where did you try to define it?

The *lists are used by spamd just fine, but spamd cannot do the
equivalent of the spamassassin script's -R, -W, and related commands
because spamc has no way to tell it to do those things.

>
> "spamc -u" should pass username to spamd which then should use that
> users' user_prefs file (if it exists) unless spamd was started with
> "-x" parameter or can't access that file.

Imagine a world where spamc and spamd run on different machines, the
ones with spamc may or may not have a working SA installation, and the
spamd is using sitewide {W,B}Lists. Or per-user prefs but in a DB with
virtual users.


--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

On 2024-03-21 at 13:21:54 UTC-0400 (Thu, 21 Mar 2024 18:21:54 +0100)
<giovanni@paclan.it>
is rumored to have said:

> On 3/20/24 21:58, Bill Cole wrote:
>> I'm not sure how I've not noticed before, but unless I'm missing something, there is no way to replicate the [block,welcome]list functionalities of the spamassassin script when using the spamc/spamd interface.
>>
>> Does anyone see it hiding somewhere that I don't?
>>
>> Does anyone have any rationale for this missing functionality?
>>
>> I don't expect that it would be difficult to add. (Something I've believed every time I've taken on a coding task...)
>>
> are you referring to spamassassin -W/-R options that are not present on spamc(1) ?

Yes, (plus all of the related --*-list commands.)

It seems to me that it would require extension of the spamc/spamd protocol and cargo-culting some code from spamassassin to spamd. Looking to provide a more elegant solution to 'strong' feedback than giving scanning-client machines direct access to the reputation DB. For example, the script I have that handles user-identified escaped spam includes this ugly snippet:

spamassassin --remove-from-welcomelist < $spam
spamc -L spam < $spam
spamassassin --add-to-blocklist < $spam

This only works because the local spamassassin and remote spamd share access to the same reputation DB. This is not optimal.



--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

Bill Cole wrote:
> On 2024-03-21 at 11:57:43 UTC-0400 (Thu, 21 Mar 2024 11:57:43 -0400)
> Kris Deugau <kdeugau@vianet.ca>
> is rumored to have said:
>
>> Bill Cole wrote:
>>> I'm not sure how I've not noticed before, but unless I'm missing
>>> something, there is no way to replicate the [block,welcome]list
>>> functionalities of the spamassassin script when using the spamc/spamd
>>> interface.
>>>
>>> Does anyone see it hiding somewhere that I don't?
>>>
>>> Does anyone have any rationale for this missing functionality?
>>>
>>> I don't expect that it would be difficult to add. (Something I've
>>> believed every time I've taken on a coding task...)
>>
>> I'm pretty sure you're doing something wrong (maybe a
>> missing/commented loadplugin entry somewhere?
>
> Maybe you've misunderstood my question. The spamc/spamd system uses
> whatever  AWL or TxRep DB is configured in evaluating messages and does
> the automated part of managing those. The spamassassin-run man page
> refers to spamd in the description of -W, -R, and various
> --{add,remove}_*list options, but I see no way that they are relevant to
> spamd.

Ahhh, OK, completely different subsystem. I misread because I don't use
that functionality at all - on the rare occasions I've wanted to
manipulate AWL data at that fine-grained a level, I "just" dive into the
backing DB and hand-bodge it directly.

I think you're looking for something relating to the -l flag to spamd,
but on reading the man page detail I'm not sure it hooks into AWL,
TXRep, or any similar components - only Bayes and "remote database" (eg
Razor or pyzor).

-l, --allow-tell
Allow learning and forgetting (to a local Bayes database),
reporting and revoking (to a remote database) by spamd. The client
issues a TELL command to tell what type of message is being
processed and whether local (learn/forget) or remote
(report/revoke) databases should be updated.

At a quick first dive, you're probably looking around line 2260 in
spamd, either adding some new logic branches right in spamd or adding
another mode to SpamAssassin/Reporter.pm.

I don't see anything closer.

-kgd

Bill Cole wrote on 22/03/24 8:36 am:
>
> It seems to me that it would require extension of the spamc/spamd protocol and cargo-culting some code from spamassassin to spamd.

Doesn't look like much cargo-culting to do. The spamassassin script just
calls a function in Mail::SpamAssassin.pm for each of the options, where
everything gets done. Unless I missed something it will just be adding
the options to the protocol and having spamd do the right call to a
function in $spamtest passing in $mail instead of where it calls

my $status = Mail::SpamAssassin::PerMsgStatus->new($spamtest, $mail);
$status->check();