Mailing List Archive

Rule tag for _USERNAME_?
All,

For dumb reasons, we at the day job are using spamass-milter, which
doesn't seem to let SpamAssassin add any extra X-Spam-Foo: message tags
beyond stock (I have a github issue open on this, which seems to be
where a fork is being maintained).

However, in order to work around this, I've added more tags to the
X-Spam-Status tag locally. Which is useful because it also lets me grep
my maillogs for those things.

What I can't find, and it feels like it should be a thing, is the
*username* for the tag. That is to say, the username that's being used to
find the user-prefs (in our case, with spamd, it's just %u, we don't have
the user/domain stuff set up).

This *feels* like something a quick .pm file should be able to add rather
than having to modify spamassassin core (and in fact, the tokens for
username, mailbox and domain *are* available for the bayes_sql_query, but
for some reason aren't exposed as tags that can be used in the report
header. Which feels somehow deliberate.)

Would this be easy to do?

It would also mean I could easily glean per-user/per-rule-hit reporting
from my maillogs with a simple grep, rather than having to cross-correlate
the mta logs from the spamd ones. This feels like a win.

-Dan

--


--------Dan Mahoney--------
Techie, Sysadmin, WebGeek
Gushi on efnet/undernet IRC
FB: fb.com/DanielMahoneyIV
LI: linkedin.com/in/gushi
Site: http://www.gushi.org
---------------------------
Re: Rule tag for _USERNAME_? [ In reply to ]
Hi Dan,

Milters are the glue that change the email. SpamAssassin is just giving
data back to the milter.

I believe you will find that X-Spam-Status header is being built by
spamass-milter not by spamassasin. You need to change the milter code to
keep track of the user and add it to the X-Spam-Status line in the
spamass-milter.cpp

Regards,
KAM
--.
Kevin A. McGrail
Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171


On Fri, Jan 15, 2021 at 5:31 PM Dan Mahoney (Gushi) <danm@prime.gushi.org>
wrote:

> All,
>
> For dumb reasons, we at the day job are using spamass-milter, which
> doesn't seem to let SpamAssassin add any extra X-Spam-Foo: message tags
> beyond stock (I have a github issue open on this, which seems to be
> where a fork is being maintained).
>
> However, in order to work around this, I've added more tags to the
> X-Spam-Status tag locally. Which is useful because it also lets me grep
> my maillogs for those things.
>
> What I can't find, and it feels like it should be a thing, is the
> *username* for the tag. That is to say, the username that's being used to
> find the user-prefs (in our case, with spamd, it's just %u, we don't have
> the user/domain stuff set up).
>
> This *feels* like something a quick .pm file should be able to add rather
> than having to modify spamassassin core (and in fact, the tokens for
> username, mailbox and domain *are* available for the bayes_sql_query, but
> for some reason aren't exposed as tags that can be used in the report
> header. Which feels somehow deliberate.)
>
> Would this be easy to do?
>
> It would also mean I could easily glean per-user/per-rule-hit reporting
> from my maillogs with a simple grep, rather than having to cross-correlate
> the mta logs from the spamd ones. This feels like a win.
>
> -Dan
>
> --
>
>
> --------Dan Mahoney--------
> Techie, Sysadmin, WebGeek
> Gushi on efnet/undernet IRC
> FB: fb.com/DanielMahoneyIV
> LI: linkedin.com/in/gushi
> Site: http://www.gushi.org
> ---------------------------
>
>
Re: Rule tag for _USERNAME_? [ In reply to ]
> On Mar 13, 2021, at 7:51 PM, Kevin A. McGrail <kmcgrail@apache.org> wrote:
>
> Hi Dan,
>
> Milters are the glue that change the email. SpamAssassin is just giving data back to the milter.
>
> I believe you will find that X-Spam-Status header is being built by spamass-milter not by spamassasin. You need to change the milter code to keep track of the user and add it to the X-Spam-Status line in the spamass-milter.cpp

That’s not true.

While it’s true that we’ve had issues getting spamass-milter to allow headers OTHER than the standard spamassassin ones through, I can pack the info I want in to the X-Spam-Status header. Here’s an example of a recent mail:

X-Spam-Status: No, score=2.5 required=5.0 tests=DCC_CHECK=1.1,
DCC_REPUT_95_98=0.7,DKIM_INVALID=0.1,DKIM_SIGNED=0.1,
HAS_UNSUBSCRIBE=0.01,HTML_IMAGE_RATIO_04=0.001,HTML_MESSAGE=0.001,
ISC_UNDISCLOSED=0.01,KAM_DMARC_STATUS=0.01,KAM_EU=0.5,
SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=disabled version=3.4.4
Lang=fr ASN=AS3292 USER=_USERNAME_

This comes from SA's local.cf:

local.cf:add_header all Status _YESNO_, score=_SCORE_ required=_REQD_ tests=_TESTSSCORES_ autolearn=_AUTOLEARN_ version=_VERSION_ Lang=_LANGUAGES_ ASN=_ASN_ USER=_USERNAME_
local.cf:add_header all Language _LANGUAGES_ (this one doesn’t show up)

See where I have _USERNAME_ being passed through as a bareword? I would like to have the LHS/RHS of whatever email address was used to pull up userprefs in that field.

-Dan

>
> Regards,
> KAM
> --.
> Kevin A. McGrail
> Member, Apache Software Foundation
> Chair Emeritus Apache SpamAssassin Project
> https://www.linkedin.com/in/kmcgrail <https://www.linkedin.com/in/kmcgrail> - 703.798.0171
>
>
> On Fri, Jan 15, 2021 at 5:31 PM Dan Mahoney (Gushi) <danm@prime.gushi.org <mailto:danm@prime.gushi.org>> wrote:
> All,
>
> For dumb reasons, we at the day job are using spamass-milter, which
> doesn't seem to let SpamAssassin add any extra X-Spam-Foo: message tags
> beyond stock (I have a github issue open on this, which seems to be
> where a fork is being maintained).
>
> However, in order to work around this, I've added more tags to the
> X-Spam-Status tag locally. Which is useful because it also lets me grep
> my maillogs for those things.
>
> What I can't find, and it feels like it should be a thing, is the
> *username* for the tag. That is to say, the username that's being used to
> find the user-prefs (in our case, with spamd, it's just %u, we don't have
> the user/domain stuff set up).
>
> This *feels* like something a quick .pm file should be able to add rather
> than having to modify spamassassin core (and in fact, the tokens for
> username, mailbox and domain *are* available for the bayes_sql_query, but
> for some reason aren't exposed as tags that can be used in the report
> header. Which feels somehow deliberate.)
>
> Would this be easy to do?
>
> It would also mean I could easily glean per-user/per-rule-hit reporting
> from my maillogs with a simple grep, rather than having to cross-correlate
> the mta logs from the spamd ones. This feels like a win.
>
> -Dan
>
> --
>
>
> --------Dan Mahoney--------
> Techie, Sysadmin, WebGeek
> Gushi on efnet/undernet IRC
> FB: fb.com/DanielMahoneyIV <http://fb.com/DanielMahoneyIV>
> LI: linkedin.com/in/gushi <http://linkedin.com/in/gushi>
> Site: http://www.gushi.org <http://www.gushi.org/>
> ---------------------------
>
Re: Rule tag for _USERNAME_? [ In reply to ]
Well, SpamAssMilter *must* be capturing the data from spamc and creating
that header. If you look at the cpp, it's building it. You could change
the Milter to create a header called X-ImaMilter and use any data you want.

But it looks like signal_user_changed sets self->{username} in spamd so if
you want, try this small patch for PerMsgStatus and lmk. I tested it
locally and it works. It would need more documentation and cleanup to add
it but it's safe as a proof of concept:

Index: lib/Mail/SpamAssassin/PerMsgStatus.pm
===================================================================
--- lib/Mail/SpamAssassin/PerMsgStatus.pm (revision 1884910)
+++ lib/Mail/SpamAssassin/PerMsgStatus.pm (working copy)
@@ -257,6 +257,11 @@
my $pms = shift;
$pms->{main}->timer_report();
},
+
+ USERNAME => sub {
+ my $pms = shift;
+ $pms->{main}->{username};
+ },

ADDEDHEADERHAM => sub {
my $pms = shift;


Regards,
KAM
--
Kevin A. McGrail
Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171


On Sun, Mar 14, 2021 at 7:10 AM Dan Mahoney <danm@prime.gushi.org> wrote:

>
>
> On Mar 13, 2021, at 7:51 PM, Kevin A. McGrail <kmcgrail@apache.org> wrote:
>
> Hi Dan,
>
> Milters are the glue that change the email. SpamAssassin is just giving
> data back to the milter.
>
> I believe you will find that X-Spam-Status header is being built by
> spamass-milter not by spamassasin. You need to change the milter code to
> keep track of the user and add it to the X-Spam-Status line in the
> spamass-milter.cpp
>
>
> That’s not true.
>
> While it’s true that we’ve had issues getting spamass-milter to allow
> headers OTHER than the standard spamassassin ones through, I can pack the
> info I want in to the X-Spam-Status header. Here’s an example of a recent
> mail:
>
> X-Spam-Status: No, score=2.5 required=5.0 tests=DCC_CHECK=1.1,
> DCC_REPUT_95_98=0.7,DKIM_INVALID=0.1,DKIM_SIGNED=0.1,
> HAS_UNSUBSCRIBE=0.01,HTML_IMAGE_RATIO_04=0.001,HTML_MESSAGE=0.001,
> ISC_UNDISCLOSED=0.01,KAM_DMARC_STATUS=0.01,KAM_EU=0.5,
> SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=disabled version=3.4.4
> Lang=fr ASN=AS3292 USER=_USERNAME_
>
> This comes from SA's local.cf:
>
> local.cf:add_header all Status _YESNO_, score=_SCORE_ required=_REQD_
> tests=_TESTSSCORES_ autolearn=_AUTOLEARN_ version=_VERSION_
> Lang=_LANGUAGES_ ASN=_ASN_ USER=_USERNAME_
> local.cf:add_header all Language _LANGUAGES_ (this one doesn’t show up)
>
> See where I have _USERNAME_ being passed through as a bareword? I would
> like to have the LHS/RHS of whatever email address was used to pull up
> userprefs in that field.
>
> -Dan
>
>
> Regards,
> KAM
> --.
> Kevin A. McGrail
> Member, Apache Software Foundation
> Chair Emeritus Apache SpamAssassin Project
> https://www.linkedin.com/in/kmcgrail - 703.798.0171
>
>
> On Fri, Jan 15, 2021 at 5:31 PM Dan Mahoney (Gushi) <danm@prime.gushi.org>
> wrote:
>
>> All,
>>
>> For dumb reasons, we at the day job are using spamass-milter, which
>> doesn't seem to let SpamAssassin add any extra X-Spam-Foo: message tags
>> beyond stock (I have a github issue open on this, which seems to be
>> where a fork is being maintained).
>>
>> However, in order to work around this, I've added more tags to the
>> X-Spam-Status tag locally. Which is useful because it also lets me grep
>> my maillogs for those things.
>>
>> What I can't find, and it feels like it should be a thing, is the
>> *username* for the tag. That is to say, the username that's being used
>> to
>> find the user-prefs (in our case, with spamd, it's just %u, we don't have
>> the user/domain stuff set up).
>>
>> This *feels* like something a quick .pm file should be able to add rather
>> than having to modify spamassassin core (and in fact, the tokens for
>> username, mailbox and domain *are* available for the bayes_sql_query, but
>> for some reason aren't exposed as tags that can be used in the report
>> header. Which feels somehow deliberate.)
>>
>> Would this be easy to do?
>>
>> It would also mean I could easily glean per-user/per-rule-hit reporting
>> from my maillogs with a simple grep, rather than having to
>> cross-correlate
>> the mta logs from the spamd ones. This feels like a win.
>>
>> -Dan
>>
>> --
>>
>>
>> --------Dan Mahoney--------
>> Techie, Sysadmin, WebGeek
>> Gushi on efnet/undernet IRC
>> FB: fb.com/DanielMahoneyIV
>> LI: linkedin.com/in/gushi
>> Site: http://www.gushi.org
>> ---------------------------
>>
>>
>
Re: Rule tag for _USERNAME_? [ In reply to ]
> On Mar 14, 2021, at 9:45 PM, Kevin A. McGrail <kmcgrail@apache.org> wrote:
>
> Well, SpamAssMilter *must* be capturing the data from spamc and creating that header. If you look at the cpp, it's building it. You could change the Milter to create a header called X-ImaMilter and use any data you want.

No argument there. My point was that it’s not just checking it for ’normal’ spamassassin output (i.e. sanitizing the header returned to see if it matches any standard. I can put anything i want in that header and (modulo length) it will transit through to my MTA’s logs.

> But it looks like signal_user_changed sets self->{username} in spamd so if you want, try this small patch for PerMsgStatus and lmk. I tested it locally and it works. It would need more documentation and cleanup to add it but it's safe as a proof of concept:
>
> Index: lib/Mail/SpamAssassin/PerMsgStatus.pm
> ===================================================================
> --- lib/Mail/SpamAssassin/PerMsgStatus.pm (revision 1884910)
> +++ lib/Mail/SpamAssassin/PerMsgStatus.pm (working copy)
> @@ -257,6 +257,11 @@
> my $pms = shift;
> $pms->{main}->timer_report();
> },
> +
> + USERNAME => sub {
> + my $pms = shift;
> + $pms->{main}->{username};
> + },
>
> ADDEDHEADERHAM => sub {
> my $pms = shift;

I’ll give that a try in the next day. I’ve been down the rabbit hole on a different project. People in this community will likely notice my efforts tho.

Thanks Kevin

-Dan



>
> On Sun, Mar 14, 2021 at 7:10 AM Dan Mahoney <danm@prime.gushi.org <mailto:danm@prime.gushi.org>> wrote:
>
>
>> On Mar 13, 2021, at 7:51 PM, Kevin A. McGrail <kmcgrail@apache.org <mailto:kmcgrail@apache.org>> wrote:
>>
>> Hi Dan,
>>
>> Milters are the glue that change the email. SpamAssassin is just giving data back to the milter.
>>
>> I believe you will find that X-Spam-Status header is being built by spamass-milter not by spamassasin. You need to change the milter code to keep track of the user and add it to the X-Spam-Status line in the spamass-milter.cpp
>
> That’s not true.
>
> While it’s true that we’ve had issues getting spamass-milter to allow headers OTHER than the standard spamassassin ones through, I can pack the info I want in to the X-Spam-Status header. Here’s an example of a recent mail:
>
> X-Spam-Status: No, score=2.5 required=5.0 tests=DCC_CHECK=1.1,
> DCC_REPUT_95_98=0.7,DKIM_INVALID=0.1,DKIM_SIGNED=0.1,
> HAS_UNSUBSCRIBE=0.01,HTML_IMAGE_RATIO_04=0.001,HTML_MESSAGE=0.001,
> ISC_UNDISCLOSED=0.01,KAM_DMARC_STATUS=0.01,KAM_EU=0.5,
> SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=disabled version=3.4.4
> Lang=fr ASN=AS3292 USER=_USERNAME_
>
> This comes from SA's local.cf <http://local.cf/>:
>
> local.cf:add_header all Status _YESNO_, score=_SCORE_ required=_REQD_ tests=_TESTSSCORES_ autolearn=_AUTOLEARN_ version=_VERSION_ Lang=_LANGUAGES_ ASN=_ASN_ USER=_USERNAME_
> local.cf:add_header all Language _LANGUAGES_ (this one doesn’t show up)
>
> See where I have _USERNAME_ being passed through as a bareword? I would like to have the LHS/RHS of whatever email address was used to pull up userprefs in that field.
>
> -Dan
>
>>
>> Regards,
>> KAM
>> --.
>> Kevin A. McGrail
>> Member, Apache Software Foundation
>> Chair Emeritus Apache SpamAssassin Project
>> https://www.linkedin.com/in/kmcgrail <https://www.linkedin.com/in/kmcgrail> - 703.798.0171
>>
>>
>> On Fri, Jan 15, 2021 at 5:31 PM Dan Mahoney (Gushi) <danm@prime.gushi.org <mailto:danm@prime.gushi.org>> wrote:
>> All,
>>
>> For dumb reasons, we at the day job are using spamass-milter, which
>> doesn't seem to let SpamAssassin add any extra X-Spam-Foo: message tags
>> beyond stock (I have a github issue open on this, which seems to be
>> where a fork is being maintained).
>>
>> However, in order to work around this, I've added more tags to the
>> X-Spam-Status tag locally. Which is useful because it also lets me grep
>> my maillogs for those things.
>>
>> What I can't find, and it feels like it should be a thing, is the
>> *username* for the tag. That is to say, the username that's being used to
>> find the user-prefs (in our case, with spamd, it's just %u, we don't have
>> the user/domain stuff set up).
>>
>> This *feels* like something a quick .pm file should be able to add rather
>> than having to modify spamassassin core (and in fact, the tokens for
>> username, mailbox and domain *are* available for the bayes_sql_query, but
>> for some reason aren't exposed as tags that can be used in the report
>> header. Which feels somehow deliberate.)
>>
>> Would this be easy to do?
>>
>> It would also mean I could easily glean per-user/per-rule-hit reporting
>> from my maillogs with a simple grep, rather than having to cross-correlate
>> the mta logs from the spamd ones. This feels like a win.
>>
>> -Dan
>>
>> --
>>
>>
>> --------Dan Mahoney--------
>> Techie, Sysadmin, WebGeek
>> Gushi on efnet/undernet IRC
>> FB: fb.com/DanielMahoneyIV <http://fb.com/DanielMahoneyIV>
>> LI: linkedin.com/in/gushi <http://linkedin.com/in/gushi>
>> Site: http://www.gushi.org <http://www.gushi.org/>
>> ---------------------------
>>
>
Re: Rule tag for _USERNAME_? [ In reply to ]
Oooh, Intriguing. Keep me in the loop!
--
Kevin A. McGrail
Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171


On Mon, Mar 15, 2021 at 1:37 AM Dan Mahoney <danm@prime.gushi.org> wrote:

>
>
> On Mar 14, 2021, at 9:45 PM, Kevin A. McGrail <kmcgrail@apache.org> wrote:
>
> Well, SpamAssMilter *must* be capturing the data from spamc and creating
> that header. If you look at the cpp, it's building it. You could change
> the Milter to create a header called X-ImaMilter and use any data you want.
>
>
> No argument there. My point was that it’s not just checking it for
> ’normal’ spamassassin output (i.e. sanitizing the header returned to see if
> it matches any standard. I can put anything i want in that header and
> (modulo length) it will transit through to my MTA’s logs.
>
> But it looks like signal_user_changed sets self->{username} in spamd so if
> you want, try this small patch for PerMsgStatus and lmk. I tested it
> locally and it works. It would need more documentation and cleanup to add
> it but it's safe as a proof of concept:
>
> Index: lib/Mail/SpamAssassin/PerMsgStatus.pm
> ===================================================================
> --- lib/Mail/SpamAssassin/PerMsgStatus.pm (revision 1884910)
> +++ lib/Mail/SpamAssassin/PerMsgStatus.pm (working copy)
> @@ -257,6 +257,11 @@
> my $pms = shift;
> $pms->{main}->timer_report();
> },
> +
> + USERNAME => sub {
> + my $pms = shift;
> + $pms->{main}->{username};
> + },
>
> ADDEDHEADERHAM => sub {
> my $pms = shift;
>
>
> I’ll give that a try in the next day. I’ve been down the rabbit hole on a
> different project. People in this community will likely notice my efforts
> tho.
>
> Thanks Kevin
>
> -Dan
>
>
>
>
> On Sun, Mar 14, 2021 at 7:10 AM Dan Mahoney <danm@prime.gushi.org> wrote:
>
>>
>>
>> On Mar 13, 2021, at 7:51 PM, Kevin A. McGrail <kmcgrail@apache.org>
>> wrote:
>>
>> Hi Dan,
>>
>> Milters are the glue that change the email. SpamAssassin is just giving
>> data back to the milter.
>>
>> I believe you will find that X-Spam-Status header is being built by
>> spamass-milter not by spamassasin. You need to change the milter code to
>> keep track of the user and add it to the X-Spam-Status line in the
>> spamass-milter.cpp
>>
>>
>> That’s not true.
>>
>> While it’s true that we’ve had issues getting spamass-milter to allow
>> headers OTHER than the standard spamassassin ones through, I can pack the
>> info I want in to the X-Spam-Status header. Here’s an example of a recent
>> mail:
>>
>> X-Spam-Status: No, score=2.5 required=5.0 tests=DCC_CHECK=1.1,
>> DCC_REPUT_95_98=0.7,DKIM_INVALID=0.1,DKIM_SIGNED=0.1,
>> HAS_UNSUBSCRIBE=0.01,HTML_IMAGE_RATIO_04=0.001,HTML_MESSAGE=0.001,
>> ISC_UNDISCLOSED=0.01,KAM_DMARC_STATUS=0.01,KAM_EU=0.5,
>> SPF_HELO_NONE=0.001,SPF_PASS=-0.001 autolearn=disabled version=3.4.4
>> Lang=fr ASN=AS3292 USER=_USERNAME_
>>
>> This comes from SA's local.cf:
>>
>> local.cf:add_header all Status _YESNO_, score=_SCORE_ required=_REQD_
>> tests=_TESTSSCORES_ autolearn=_AUTOLEARN_ version=_VERSION_
>> Lang=_LANGUAGES_ ASN=_ASN_ USER=_USERNAME_
>> local.cf:add_header all Language _LANGUAGES_ (this one doesn’t show up)
>>
>> See where I have _USERNAME_ being passed through as a bareword? I would
>> like to have the LHS/RHS of whatever email address was used to pull up
>> userprefs in that field.
>>
>> -Dan
>>
>>
>> Regards,
>> KAM
>> --.
>> Kevin A. McGrail
>> Member, Apache Software Foundation
>> Chair Emeritus Apache SpamAssassin Project
>> https://www.linkedin.com/in/kmcgrail - 703.798.0171
>>
>>
>> On Fri, Jan 15, 2021 at 5:31 PM Dan Mahoney (Gushi) <danm@prime.gushi.org>
>> wrote:
>>
>>> All,
>>>
>>> For dumb reasons, we at the day job are using spamass-milter, which
>>> doesn't seem to let SpamAssassin add any extra X-Spam-Foo: message tags
>>> beyond stock (I have a github issue open on this, which seems to be
>>> where a fork is being maintained).
>>>
>>> However, in order to work around this, I've added more tags to the
>>> X-Spam-Status tag locally. Which is useful because it also lets me grep
>>> my maillogs for those things.
>>>
>>> What I can't find, and it feels like it should be a thing, is the
>>> *username* for the tag. That is to say, the username that's being used
>>> to
>>> find the user-prefs (in our case, with spamd, it's just %u, we don't
>>> have
>>> the user/domain stuff set up).
>>>
>>> This *feels* like something a quick .pm file should be able to add
>>> rather
>>> than having to modify spamassassin core (and in fact, the tokens for
>>> username, mailbox and domain *are* available for the bayes_sql_query,
>>> but
>>> for some reason aren't exposed as tags that can be used in the report
>>> header. Which feels somehow deliberate.)
>>>
>>> Would this be easy to do?
>>>
>>> It would also mean I could easily glean per-user/per-rule-hit reporting
>>> from my maillogs with a simple grep, rather than having to
>>> cross-correlate
>>> the mta logs from the spamd ones. This feels like a win.
>>>
>>> -Dan
>>>
>>> --
>>>
>>>
>>> --------Dan Mahoney--------
>>> Techie, Sysadmin, WebGeek
>>> Gushi on efnet/undernet IRC
>>> FB: fb.com/DanielMahoneyIV
>>> LI: linkedin.com/in/gushi
>>> Site: http://www.gushi.org
>>> ---------------------------
>>>
>>>
>>
>