Mailing List Archive

LuKreme wrote:

> [...]
> Why would no tests be run?

If only defaults rules are used, perhaps simply none matched?

> and autolearn=ham???

Configuration! You might want to lower that threshold.

Bayes would probably have nailed that one.

- Bob

Bob George wrote:

> [...]
> Bayes would probably have nailed that one.

Er, _trained_ bayes anyhow.

- Bob

On 13 Mar 2004, at 14:38, Bob George wrote:
> LuKreme wrote:
>
>> [...]
>> Why would no tests be run?
>
> If only defaults rules are used, perhaps simply none matched?

Here's your post's X-Spam-Status header:

X-Spam-Status: No, hits=-1.7 required=5.0 tests=AWL,BAYES_00,
FROM_ENDS_IN_NUMS,J_CHICKENPOX_44 autolearn=no version=2.63

For what it's worth I have Chickenpox, backhair, and a fairly recent
bigevil installed

>> and autolearn=ham???
>
> Configuration! You might want to lower that threshold.

AFAIK I have made no changes at all to the default levels for anything.
Other then the three above .cf files my local.cf consists only of

rewrite_subject 1
subject_tag (Spam? _HITS_)


$ cat ~kremels/Mail/*/* | grep X-Spam-Status: | grep tests=none | wc -l
2640
$ cat ~kremels/Mail/*/* | grep X-Spam-Status: | grep -v tests=none |
wc -l
13353
$ cat ~kremels/Mail/*/* | grep -A3 X-Spam-Status: | grep BAYES | wc -l
3515


--
I want a refund, I want a light, I want a reason for all this night
after night after night after night

LuKreme wrote:

> [...]
> Here's your post's X-Spam-Status header:
>
> X-Spam-Status: No, hits=-1.7 required=5.0 tests=AWL,BAYES_00,
> FROM_ENDS_IN_NUMS,J_CHICKENPOX_44 autolearn=no version=2.63

Well, it's not showing 'none' at least, so that's encouraging. And it is
hitting the addon rule sets.

> For what it's worth I have Chickenpox, backhair, and a fairly recent
> bigevil installed

Hmm... and --lint shows nothing?

> [...]
> AFAIK I have made no changes at all to the default levels for
> anything. Other then the three above .cf files my local.cf consists
> only of
>
> rewrite_subject 1
> subject_tag (Spam? _HITS_)

I'd be worried about the auto-learning going on with bad scoring. The
default autolearn threshold for ham is .1 IIRC, and you're getting a LOT
of stuff scoring below that for some reason.

Here's what it hit here:

Content analysis details: (9.3 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
1.0 FROM_ENDS_IN_NUMS From: ends in numbers
0.6 J_CHICKENPOX_22 BODY: {2}Letter - punctuation - {2}Letter
0.1 TW_CD BODY: Odd Letter Triples with CD
0.1 TW_MJ BODY: Odd Letter Triples with MJ
5.4 BAYES_99 BODY: Bayesian spam probability is 99 to 100%
[score: 1.0000]
0.3 MIME_HTML_ONLY BODY: Message only has text/html MIME parts
0.6 MIME_HTML_NO_CHARSET RAW: Message text in HTML without charset
-8.0 HABEAS_SWE Has Habeas warrant mark (http://www.habeas.com/)
1.5 RCVD_IN_BL_SPAMCOP_NET RBL: Received via a relay in bl.spamcop.net
[Blocked - see <http://www.spamcop.net/bl.shtml?210.149.124.241>]
1.2 HTML_MIME_NO_HTML_TAG HTML-only message, but there is no HTML tag
0.0 LOCAL_DRUGS_ANXIETY LOCAL_DRUGS_ANXIETY
1.0 LOCAL_SMELLSPAM All non-SpamAssassin bayes says SPAM
0.0 LOCAL_DRUGS_SLEEP LOCAL_DRUGS_SLEEP
0.0 LOCAL_DRUGS_DEPRESSION LOCAL_DRUGS_DEPRESSION
0.0 LOCAL_DRUGS_PAIN LOCAL_DRUGS_PAIN
0.5 LOCAL_DRUGS_MALEDYSFUNCTION_OBFU LOCAL_DRUGS_MALEDYSFUNCTION_OBFU
1.0 LOCAL_DRUGS_MALEDYSFUNCTION LOCAL_DRUGS_MALEDYSFUNCTION
0.0 LOCAL_DRUGS_MUSCLE LOCAL_DRUGS_MUSCLE
1.0 LOCAL_DRUGS_PAIN_MALEDYS LOCAL_DRUGS_PAIN_MALEDYS
1.0 LOCAL_DRUGS_ANXIETY_MALEDYS LOCAL_DRUGS_ANXIETY_MALEDYS
1.0 LOCAL_DRUGS_DEPRESSION_MALEDYS LOCAL_DRUGS_DEPRESSION_MALEDYS
1.0 LOCAL_DRUGS_MANYKINDS LOCAL_DRUGS_MANYKINDS

Note that it IS pretty low scoring, all things considered. It's got the
forged habeas headers, but despite that is still very spammish. Most
pattern rule hits came from Matt Kettler's anti-drug.cf. Bayes has no
problem with random word stuff (as expected). The LOCAL_SMELLSPAM rule
is one I wrote that indicates that FOUR separate bayes tools flagged it
as spam.

Check your rules to make sure everyting's intact, but if it weren't for
bayes and antidrug, this WOULD be a tough one to catch.

- Bob

On 13 Mar 2004, at 19:34, Bob George wrote:
> LuKreme wrote:
>
>> [...]
>> Here's your post's X-Spam-Status header:
>>
>> X-Spam-Status: No, hits=-1.7 required=5.0 tests=AWL,BAYES_00,
>> FROM_ENDS_IN_NUMS,J_CHICKENPOX_44 autolearn=no version=2.63
>
> Well, it's not showing 'none' at least, so that's encouraging. And it
> is hitting the addon rule sets.
>
>> For what it's worth I have Chickenpox, backhair, and a fairly recent
>> bigevil installed
>
> Hmm... and --lint shows nothing?

Nope, nothing at all.

kremels $ spamassassin -V --lint
SpamAssassin version 2.63
kremels $

I'm very puzzled.

I've also very concerned to see I have nearly 20% of my mail getting
tagged with tests=none, but I may be misunderstanding the implications
of that. Seems to me highly implausible that a message wouldn't hit a
AWL or something....

$ sa-learn --dump | head -11
0.000 0 2 0 non-token data: bayes db version
0.000 0 268 0 non-token data: nspam
0.000 0 101077 0 non-token data: nham
0.000 0 129966 0 non-token data: ntokens
0.000 0 1076718523 0 non-token data: oldest atime
0.000 0 1079279775 0 non-token data: newest atime
0.000 0 1079280007 0 non-token data: last journal
sync atime
0.000 0 1078949120 0 non-token data: last expiry
atime
0.000 0 2230640 0 non-token data: last expire
atime delta
0.000 0 33116 0 non-token data: last expire
reduction count
0.049 0 1 1076719407 H*m:1076719206

I notice the nspam count is very low. I generally dump spam messages
into a SPAM box and don't train on them because I thought that messages
tagged as spam where automatically trained. Should I be running
sa-learn against my known spam? I'd only been doing it against spam
that was not correctly tagged by SA. (or spam that was incorrectly
tagged, though that is very very rare).

--
#242755 <jshock221> a freudian slip is when you say one thing but
you're really thinking about a mother.
<Spadgeroonie> no, a freudian slip is sexy underwear your mother wears

LuKreme wrote:

> [...]
> I notice the nspam count is very low. I generally dump spam messages
> into a SPAM box and don't train on them because I thought that
> messages tagged as spam where automatically trained. Should I be
> running sa-learn against my known spam?

Autolearn will only apply if the message scores high enough, assuming
it's turned on. The default is 12 IIRC.

> I'd only been doing it against spam that was not correctly tagged by
> SA. (or spam that was incorrectly tagged, though that is very very rare).

That might explain the low count then. If you're training on the errors,
it SHOULD learn correctly, but I believe a full initial training is
still recommended to speed things up. Have you really only fed it that
little spam though?

Everything may be ok, but like you, I'd be concerned about large numbers
of spam getting "no tests" results. And I didn't see bayes scoring in
your sample.

- Bob

On Mar 14, 2004, at 11:11 AM, Bob George wrote:
> LuKreme wrote:
> That might explain the low count then. If you're training on the
> errors, it SHOULD learn correctly, but I believe a full initial
> training is still recommended to speed things up. Have you really only
> fed it that little spam though?

that's all I've gotten and I don't have a big file-o-spam saved off. I
fed it my mailspool (or a large chunk of it, thus the 100K nham). I
have fed it about 1200 total spam now, so we will see how that goes.

--
"I used to hate the sun, because it'd shone on everything I'd done.
Made me feel that all that I had done was overfill the ashtray of my
life."