The de-welcomelisting of MS marketing raises the question: Why do we maintain a "default" welcomelist?
Based on the documentation, the original purpose of the def_welcomelist* (then whitelist) feature set was to give a set of senders of purely legitimate mail from FPs, with a listing having reduced power relative to normal welcomelist entires *because they were widely phished by spammers*. This was before sender authentication (SPF & DKIM) were in broad use and before authentication-empowered welcomelist features existed. Having these senders weakly protected in SA was a preventative measure to keep frustrated admins (or poisoned Bayes DBs) from rashly overprotecting them and all the phishes or blocklisting them. Today there's much less risk in the welcomelist features because we recommend and use the authenticated forms, which means that if you like, you can use SA WL/BL rules to demand only authenticated mail from some senders. In that context, the purpose of the default welcomelist has wandered.
This explains to some degree the lack of clear relevance and meaning of listings. It is a remnant feature that has outlived its original justification. The original list included big names of respected companies that "everyone" (i.e. as warped by SA committer and vocal user non-diversity...) got occasional important mail from and would never want to block mail from. It has drifted into being a list of "good guys" who, based on our committers' experiences, get FPs that they do not deserve. We have drifted perilously close to being a maintainer of a low-visibility free reputation service with lax oversight. We also come near that peril in our explicit lists of TLDs which are objectively dominated by spammers, but in that case I think we have that risk contained because we have a methodology for validating TLD inclusion and removal: testing single-TLD rules in QA. The default welcomelist is unconfined, because we don't have a clear explicit standard or even a formal transparent mechanism for inclusion and removal. My understanding of some listings is that they were based on one mid-sized site's FPs from their wanted mailstream dominated by one-to-few "personal" B2B email. That is very hard to validate, and at this point the list is too big to trim it back to just the original concept, especially since that concept no longer has much real-world use. It needs more structure to keep it from becoming just "friends, employers, and extorters of SA committers" or being perceived as such.
I believe that part of a way to avoid that is an absolute zero-tolerance policy for spam from listees. We cannot support any standard that gets us bogged down into debates with senders over whether their spam is enough spam to justify risking the FPs they would get without a listing, because we cannot measure that. We cannot be subservient to sender business models that require them to take shortcuts in assuring that they do not ever send spam. We must not be telling our users that they should just eat their spam because some sender doesn't want to spend on confirming users and seems to have a working unsub link.
I ultimately want to document how we add and remove listings and what users should expect from the default welcomelist. I think some important elements are:
1. We serve our users: receivers, not senders. Senders claiming FPs need the support of a corroborating would-be receiver.
2. If senders have FPs on objectively legitimate mail, their first and most important step is to identify WHY SpamAssassin thinks it is spam. and address that. Do you need the invisible text? Is the message embedded in a remotely-fetched image? The sea of "&zwnj" entities in your messages' HTML serves what purpose exactly? If there's a real FP problem with some rule that regularly is proved out by RuleQA, open a bug.
3. This is NOT a general-purpose reputation list. It exists to aid SA users who have FPs from SpamAssassin default rules for wanted mail, where we cannot determine any acceptable adjustment to rules which would avoid the problem. It is a "last resort" form of FP mitigation when we cannot find an acceptable general solution that isn't domain-specific to a widely accepted sender domain.
4. We should only add or remove listings based on specific requests backed by transparent evidence. Subversion commit messages are not enough, we need a bug report or a mailing list discussion.
5. Existing entries are presumed valid unless and until they cause a false "ham" classification of spam which can be shared publicly in a useful form.
6. New entries must pass prolonged RuleQA testing of sender-specific rules before being added to the default welcomelist.
As with everything SpamAssassin: input from users and other contributors is eagerly desired...,
--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire
Based on the documentation, the original purpose of the def_welcomelist* (then whitelist) feature set was to give a set of senders of purely legitimate mail from FPs, with a listing having reduced power relative to normal welcomelist entires *because they were widely phished by spammers*. This was before sender authentication (SPF & DKIM) were in broad use and before authentication-empowered welcomelist features existed. Having these senders weakly protected in SA was a preventative measure to keep frustrated admins (or poisoned Bayes DBs) from rashly overprotecting them and all the phishes or blocklisting them. Today there's much less risk in the welcomelist features because we recommend and use the authenticated forms, which means that if you like, you can use SA WL/BL rules to demand only authenticated mail from some senders. In that context, the purpose of the default welcomelist has wandered.
This explains to some degree the lack of clear relevance and meaning of listings. It is a remnant feature that has outlived its original justification. The original list included big names of respected companies that "everyone" (i.e. as warped by SA committer and vocal user non-diversity...) got occasional important mail from and would never want to block mail from. It has drifted into being a list of "good guys" who, based on our committers' experiences, get FPs that they do not deserve. We have drifted perilously close to being a maintainer of a low-visibility free reputation service with lax oversight. We also come near that peril in our explicit lists of TLDs which are objectively dominated by spammers, but in that case I think we have that risk contained because we have a methodology for validating TLD inclusion and removal: testing single-TLD rules in QA. The default welcomelist is unconfined, because we don't have a clear explicit standard or even a formal transparent mechanism for inclusion and removal. My understanding of some listings is that they were based on one mid-sized site's FPs from their wanted mailstream dominated by one-to-few "personal" B2B email. That is very hard to validate, and at this point the list is too big to trim it back to just the original concept, especially since that concept no longer has much real-world use. It needs more structure to keep it from becoming just "friends, employers, and extorters of SA committers" or being perceived as such.
I believe that part of a way to avoid that is an absolute zero-tolerance policy for spam from listees. We cannot support any standard that gets us bogged down into debates with senders over whether their spam is enough spam to justify risking the FPs they would get without a listing, because we cannot measure that. We cannot be subservient to sender business models that require them to take shortcuts in assuring that they do not ever send spam. We must not be telling our users that they should just eat their spam because some sender doesn't want to spend on confirming users and seems to have a working unsub link.
I ultimately want to document how we add and remove listings and what users should expect from the default welcomelist. I think some important elements are:
1. We serve our users: receivers, not senders. Senders claiming FPs need the support of a corroborating would-be receiver.
2. If senders have FPs on objectively legitimate mail, their first and most important step is to identify WHY SpamAssassin thinks it is spam. and address that. Do you need the invisible text? Is the message embedded in a remotely-fetched image? The sea of "&zwnj" entities in your messages' HTML serves what purpose exactly? If there's a real FP problem with some rule that regularly is proved out by RuleQA, open a bug.
3. This is NOT a general-purpose reputation list. It exists to aid SA users who have FPs from SpamAssassin default rules for wanted mail, where we cannot determine any acceptable adjustment to rules which would avoid the problem. It is a "last resort" form of FP mitigation when we cannot find an acceptable general solution that isn't domain-specific to a widely accepted sender domain.
4. We should only add or remove listings based on specific requests backed by transparent evidence. Subversion commit messages are not enough, we need a bug report or a mailing list discussion.
5. Existing entries are presumed valid unless and until they cause a false "ham" classification of spam which can be shared publicly in a useful form.
6. New entries must pass prolonged RuleQA testing of sender-specific rules before being added to the default welcomelist.
As with everything SpamAssassin: input from users and other contributors is eagerly desired...,
--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire