Mailing List Archive: Re: svn commit: rev 20683 - incubator/spamassassin/trunk/rules

Re: svn commit: rev 20683 - incubator/spamassassin/trunk/rules

May 30, 2004, 11:51 PM

Post #1 of 4 (34 views)

I'll comment on this before anyone else does. :-)

> Author: quinlan
> Date: Sun May 30 22:55:49 2004
> New Revision: 20683
>
> Modified:
> incubator/spamassassin/trunk/rules/70_testing.cf
> Log:
> removing T_SPF_PASS_NO_SBL - all of my hits are really spam

I don't think any one rule is sufficient to make a simple "does an SPF
record exist" test worthwhile. The test needs to be paired with *actual
domain names* that are known to be good senders.

I took all SPF_PASS hits (318) in the last 14 days of corpus results and
looked at the other rules hit on those spam messages:

318 SPF_PASS <- spammers with SPF records
247 HTML_MESSAGE
239 SPF_HELO_PASS <- both!
197 T_SPF_PASS_NO_SBL
169 URIBL_WS_SURBL
159 RAZOR2_CF_RANGE_51_100
157 URIBL_SBL
134 RAZOR2_CHECK
130 MIME_HTML_ONLY
121 T_SPF_HELO_PASS_NO_SBL
121 RCVD_IN_SBL
114 BAYES_99
112 T_RCVD_IN_SBL
106 URIBL_BE_SURBL
103 T_RATWARE_RCVD_PF_1
98 CLICK_BELOW
... long tail

I don't think any small set of rules is sufficient. And if you include
too many rules, then the entire point of having a negative rule is
missed. We should be attempting to couple SPF pass with specific names.
For example, it should be required for our default whitelist.

Daniel

--
Daniel Quinlan
http://www.pathname.com/~quinlan/

Re: svn commit: rev 20683 - incubator/spamassassin/trunk/rules [ In reply to ]

felicity at kluge

May 31, 2004, 12:32 PM

Post #2 of 4 (34 views)

Permalink

On Sun, May 30, 2004 at 11:51:25PM -0700, Dan Quinlan wrote:
> I don't think any small set of rules is sufficient. And if you include
> too many rules, then the entire point of having a negative rule is

I agree with this -- SPF_PASS isn't usable as a whitelist since the whole
goal of SPF is to have spammers stop forging, and therefore switch from
SPF_FAIL to SPF_PASS.

> missed. We should be attempting to couple SPF pass with specific names.
> For example, it should be required for our default whitelist.

I don't know about "required" (we can't force these places to use
SPF), but if SPF exists we should definitely use it for the whitelist.
Perhaps we should modify whitelist_from_rcvd that instead of specifying
the Received header we can specify a rulename (ala SPF_PASS)? Well,
it should have a new whitelist name, but it's the idea really ... ;)

Without that, perhaps a meta rule ala:

meta DEF_WL_FORGED USER_IN_DEF_WHITELIST && SPF_FAIL

this is the same idea, but doesn't require SPF records to exist at
the start.

--
Randomly Generated Tagline:
BBSing: Files, folks and fun.

Re: svn commit: rev 20683 - incubator/spamassassin/trunk/rules [ In reply to ]

quinlan at pathname

May 31, 2004, 4:19 PM

Post #3 of 4 (34 views)

Permalink

Daniel Quinlan <quinlan@pathname.com> writes:

>> missed. We should be attempting to couple SPF pass with specific names.
>> For example, it should be required for our default whitelist.

Theo Van Dinter <felicity@kluge.net> writes:

> I don't know about "required" (we can't force these places to use
> SPF), but if SPF exists we should definitely use it for the whitelist.

We can't force them, but they can't force us to include a whitelist
either. We could easily add a requirement: no SPF, no whitelist.

> Perhaps we should modify whitelist_from_rcvd that instead of
> specifying the Received header we can specify a rulename (ala
> SPF_PASS)? Well, it should have a new whitelist name, but it's the
> idea really ... ;)
>
> Without that, perhaps a meta rule ala:
>
> meta DEF_WL_FORGED USER_IN_DEF_WHITELIST && SPF_FAIL
>
> this is the same idea, but doesn't require SPF records to exist at
> the start.

We need testing for both a negative and a positive rule. I think that a
negative rule will be safer since SPF_PASS falses are not dropping off
as fast as I'd like.

0-1 months old:

1.224 1.3110 0.0144 0.989 0.80 1.00 SPF_FAIL:0-1
0.080 0.0375 0.6749 0.053 0.33 -0.00 SPF_PASS:0-1

1-3 months old:

2.473 3.3886 0.0217 0.994 0.79 1.00 SPF_FAIL:1-3
0.969 0.3478 2.6307 0.117 0.39 -0.00 SPF_PASS:1-3

Daniel

--
Daniel Quinlan
http://www.pathname.com/~quinlan/

Re: svn commit: rev 20683 - incubator/spamassassin/trunk/rules [ In reply to ]

sidney at sidney

May 31, 2004, 4:37 PM

Post #4 of 4 (34 views)

Permalink

Daniel Quinlan wrote:
> For example, it should be required for our default whitelist.

If we add something for SPF as a more flexible form of the recvd check
on whitelist entries, we should also expand the forged_whitelist check.
That is the ideal use of SPF, as a negative test for blocking forgery of
a well known address when we are sure that the domain uses SPF.

-- sidney