Mailing List Archive

Proposed new "alias" directive (was: Rules referencing WHITELIST or BLACKLIST in process of being Renamed)
On Sun, 19 Jul 2020, Kevin A. McGrail wrote:

>> I was mulling the utility of a new config directive in 4.0 to address
>> backwards compatibility of rule names:
>> ?? alias? RULENAME? RULENAME_2
>>
>> ...which would recognize any config-file directives for RULENAME and
>> apply them as if they'd been written for RULENAME_2.
>>
>> This would discard any existing definition of RULENAME_2 (e.g. header
>> RULENAME_2 ...), and a definition of RULENAME_2 subsequent to that
>> alias would be ignored and a lint warning emitted.
>
> That would work well, yes.? How big a lift for that alias idea do you
> think??

One detail that I'm not in a good position to estimate is the impact of
this on the rules compiler.

The "alias" directive should not affect RE rules at all, other than
perhaps removing one if the alias is defined after a RE rule having the
same name was defined. The big effect of "alias" on rules is on metas,
where it would substitute the root rule name in place of the alias name in
any meta expression where it appeared (this to avoid having the meta rule
logic being modified to be aware of aliases).

I'm assuming that the rules compiler is given the parsed RE rules
collection after the config parsing is complete, and I don't expect the
rules compiler does anything with meta rules, so I *think* that adding an
"alias" directive would not affect the rules compiler at all.

Can someone more familiar with the rules compiler confirm whether my
impression is correct?


>> There is the use case of post-SA message processing looking for
>> specific rule name hits - I can see custom post-SA message processing
>> looking for a hit on "USER_IN_WHITELIST_TO" et al. To avoid breaking
>> that we could add a "report" option to the alias command:
>>
>> ? alias? RULENAME? RULENAME_2 report
>
> this seems over complicated.? Do you know of anyone doing this use case
> that wouldn't just add a meta rule and be done?

As discussed, the "report" option seems needed, or defining "meta XXX"
would remove "alias XXX". I'm still thinking about this one.

I suspect having "meta XXX" remove "alias XXX" will open the risk of
unexpected behaviors especially with respect to scoring, so I'm leaning
towards a "report" option on the alias directive and having "meta XXX"
emit a lint error if "alias XXX" already exists as being more reliable in
use.


--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Venezuela is busy reaping the benefits of Socialism:
in one year 75% of the population has, on average, lost 19 pounds
due to insufficient food, and 82% of households are below the
poverty line. (2016 Venezuelan "Living Conditions Survey")
-----------------------------------------------------------------------
103 days until the Presidential Election
Re: Proposed new "alias" directive (was: Rules referencing WHITELIST or BLACKLIST in process of being Renamed) [ In reply to ]
> The "alias" directive should not affect RE rules at all, other than
> perhaps removing one if the alias is defined after a RE rule having the
> same name was defined.

I would hope another use of ALIAS would be to redirect a subsequent SCORE or
DESCRIPTION directive to the renamed rule rather than producing a warning
for "score for nonexistant rule" or the like.

Perhaps it already does this, I don't know. I throw this out in case it does
not.

Loren
Re: Proposed new "alias" directive (was: Rules referencing WHITELIST or BLACKLIST in process of being Renamed) [ In reply to ]
On 7/23/2020 5:39 PM, Loren Wilton wrote:
>> The "alias" directive should not affect RE rules at all, other than
>> perhaps removing one if the alias is defined after a RE rule having the
>> same name was defined.
>
> I would hope another use of ALIAS would be to redirect a subsequent
> SCORE or DESCRIPTION directive to the renamed rule rather than
> producing a warning for "score for nonexistant rule" or the like.

I'm not sure if the need for alias was overcome by how we solved it (see
below) which allows for rule template processing, local scoring and
local rules to work without any changes.? John, was there other reasons
to add it?

if can(Mail::SpamAssassin::Conf::feature_blocklist_welcomelist)
? #bz7826 renames whitelist to welcomelist
? header USER_IN_WELCOMELIST_TO???????? eval:check_to_in_welcomelist()
? describe USER_IN_WELCOMELIST_TO?????? User is listed in 'welcomelist_to'
? tflags USER_IN_WELCOMELIST_TO???????? userconf nice noautolearn
? score USER_IN_WELCOMELIST_TO????????? -6.0
else
? header USER_IN_WELCOMELIST_TO???????? eval:check_to_in_whitelist()
? describe USER_IN_WELCOMELIST_TO?????? User is listed in 'welcomelist_to'
? tflags USER_IN_WELCOMELIST_TO???????? userconf nice noautolearn
? score USER_IN_WELCOMELIST_TO????????? -0.01

? meta USER_IN_WHITELIST_TO???????????? (USER_IN_WELCOMELIST_TO)
? describe USER_IN_WHITELIST_TO???????? DEPRECATED: See
USER_IN_WELCOMELIST_TO
? tflags USER_IN_WHITELIST_TO?????????? userconf nice noautolearn
? score USER_IN_WHITELIST_TO??????????? -6.0
endif
Re: Proposed new "alias" directive (was: Rules referencing WHITELIST or BLACKLIST in process of being Renamed) [ In reply to ]
On Thu, 23 Jul 2020, Loren Wilton wrote:

>> The "alias" directive should not affect RE rules at all, other than
>> perhaps removing one if the alias is defined after a RE rule having the
>> same name was defined.
>
> I would hope another use of ALIAS would be to redirect a subsequent SCORE or
> DESCRIPTION directive to the renamed rule rather than producing a warning for
> "score for nonexistant rule" or the like.

That's the intent, see the original proposal.

An "alias RULENAME RULENAME_2" directive would cause all subsequent
directives that mention RULENAME to be interpreted as if they had been
written for RULENAME_2 instead.


--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
USMC Rules of Gunfighting #4: If your shooting stance is good,
you're probably not moving fast enough nor using cover correctly.
-----------------------------------------------------------------------
102 days until the Presidential Election
Re: Proposed new "alias" directive (was: Rules referencing WHITELIST or BLACKLIST in process of being Renamed) [ In reply to ]
On Thu, 23 Jul 2020, Kevin A. McGrail wrote:

> On 7/23/2020 5:39 PM, Loren Wilton wrote:
>>> The "alias" directive should not affect RE rules at all, other than
>>> perhaps removing one if the alias is defined after a RE rule having the
>>> same name was defined.
>>
>> I would hope another use of ALIAS would be to redirect a subsequent
>> SCORE or DESCRIPTION directive to the renamed rule rather than
>> producing a warning for "score for nonexistant rule" or the like.
>
> I'm not sure if the need for alias was overcome by how we solved it (see
> below) which allows for rule template processing, local scoring and
> local rules to work without any changes.? John, was there other reasons
> to add it?
>
> if can(Mail::SpamAssassin::Conf::feature_blocklist_welcomelist)
> ? #bz7826 renames whitelist to welcomelist
> ? header USER_IN_WELCOMELIST_TO???????? eval:check_to_in_welcomelist()
> ? describe USER_IN_WELCOMELIST_TO?????? User is listed in 'welcomelist_to'
> ? tflags USER_IN_WELCOMELIST_TO???????? userconf nice noautolearn
> ? score USER_IN_WELCOMELIST_TO????????? -6.0
> else
> ? header USER_IN_WELCOMELIST_TO???????? eval:check_to_in_whitelist()
> ? describe USER_IN_WELCOMELIST_TO?????? User is listed in 'welcomelist_to'
> ? tflags USER_IN_WELCOMELIST_TO???????? userconf nice noautolearn
> ? score USER_IN_WELCOMELIST_TO????????? -0.01
>
> ? meta USER_IN_WHITELIST_TO???????????? (USER_IN_WELCOMELIST_TO)
> ? describe USER_IN_WHITELIST_TO???????? DEPRECATED: See
> USER_IN_WELCOMELIST_TO
> ? tflags USER_IN_WHITELIST_TO?????????? userconf nice noautolearn
> ? score USER_IN_WHITELIST_TO??????????? -6.0
> endif

No, that's incomplete.

The intent was to avoid breaking existing production configurations and
third-party tools when feature_block_welcome becomes available, in order
to complete the coverage of backwards compatibility.

Note in the above rule definition that when that feature becomes
available, the definition for the USER_IN_WHITELIST_TO rule *disappears*,
leading to lint failures and unexpected behavior if local rules reference
it.

Having an "alias" directive lets us avoid that by having local references
to USER_IN_WHITELIST_TO (e.g., notably, scoring) be associated with
USER_IN_WELCOMELIST_TO.

In other words, we'd ship this:

if can(Mail::SpamAssassin::Conf::feature_blocklist_welcomelist)
? #bz7826 renames whitelist to welcomelist
? header USER_IN_WELCOMELIST_TO???????? eval:check_to_in_welcomelist()
? describe USER_IN_WELCOMELIST_TO?????? User is listed in 'welcomelist_to'
? tflags USER_IN_WELCOMELIST_TO???????? userconf nice noautolearn
? score USER_IN_WELCOMELIST_TO????????? -6.0
if can(Mail::SpamAssassin::Conf::feature_alias)
alias USER_IN_WHITELIST_TO USER_IN_WELCOMELIST_TO
endif
else
...


...which avoids "score for nonexistent rule USER_IN_WHITELIST_TO" lint
problems and associated unexpected behavior changes.

And if a particular site is performing post-SA scanning of rule hits, and
that is looking for USER_IN_WHITELIST_TO in particular, then the local
site config would add this:

alias USER_IN_WHITELIST_TO USER_IN_WELCOMELIST_TO publish

...to add the "publish" option to the alias in order to report the
USER_IN_WELCOMELIST_TO hit under USER_IN_WHITELIST_TO as well to support
that post-SA processing until it could be updated to look for
USER_IN_WELCOMELIST_TO instead.

--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
USMC Rules of Gunfighting #4: If your shooting stance is good,
you're probably not moving fast enough nor using cover correctly.
-----------------------------------------------------------------------
102 days until the Presidential Election
Re: Proposed new "alias" directive (was: Rules referencing WHITELIST or BLACKLIST in process of being Renamed) [ In reply to ]
> The intent was to avoid breaking existing production configurations
> and third-party tools when feature_block_welcome becomes available, in
> order to complete the coverage of backwards compatibility.

I was thinking to publicize the one byte change for the can has feature
to disable it to stay with the old rules.? I think that's simpler than
the alias functionality.

--

Kevin A. McGrail
KMcGrail@Apache.org

Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171
Re: Proposed new "alias" directive (was: Rules referencing WHITELIST or BLACKLIST in process of being Renamed) [ In reply to ]
On Fri, 24 Jul 2020, Kevin A. McGrail wrote:

>
>> The intent was to avoid breaking existing production configurations
>> and third-party tools when feature_block_welcome becomes available, in
>> order to complete the coverage of backwards compatibility.
>
> I was thinking to publicize the one byte change for the can has feature
> to disable it to stay with the old rules.  I think that's simpler than
> the alias functionality.

Assuming that the only utility of the "alias" directive is backwards
compatibility in this feature, that's obviously a simpler solution. ?

I haven't been thinking about general utility of "alias" - can anyone
think of a use case that makes it attractive outside backwards
compatibility?

I think it would be better to control feature_block_welcome from the
4.0pre file rather than having to make a code change, regardless of how
small. Is that feasible?


--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
102 days until the Presidential Election
Re: Proposed new "alias" directive (was: Rules referencing WHITELIST or BLACKLIST in process of being Renamed) [ In reply to ]
I think putting it in 4.0 pre makes a lot of sense. I will look at how to
do that.

On Fri, Jul 24, 2020, 17:06 John Hardin <jhardin@impsec.org> wrote:

> On Fri, 24 Jul 2020, Kevin A. McGrail wrote:
>
> >
> >> The intent was to avoid breaking existing production configurations
> >> and third-party tools when feature_block_welcome becomes available, in
> >> order to complete the coverage of backwards compatibility.
> >
> > I was thinking to publicize the one byte change for the can has feature
> > to disable it to stay with the old rules. I think that's simpler than
> > the alias functionality.
>
> Assuming that the only utility of the "alias" directive is backwards
> compatibility in this feature, that's obviously a simpler solution. ?
>
> I haven't been thinking about general utility of "alias" - can anyone
> think of a use case that makes it attractive outside backwards
> compatibility?
>
> I think it would be better to control feature_block_welcome from the
> 4.0pre file rather than having to make a code change, regardless of how
> small. Is that feasible?
>
>
> --
> John Hardin KA7OHZ http://www.impsec.org/~jhardin/
> jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
> key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
> -----------------------------------------------------------------------
> 102 days until the Presidential Election