Mailing List Archive

Chuck Campbell <campbell@accelinc.com> writes:

> Is there a how-to around, or an implementation of Dspam and/or crm114 into
> the Spamassassin world/way of doing things?
>
> I read the thread here recently, which resulted in an implementation of
> crm114, and I'm looking at it now.
>
> I looked at dspam, which currently appears to sit between sendmail and
> procmail (or maybe replaces procmail) as a mail delivery agent, but it
> seems to me that is an invasive approach.
>
> I like the concept of word sets in a Bayesian approach, and their documentation
> is compelling. I just don't clearly see how to hammer it into the SA
> world (with which I am now familiar).
>
> I was hoping to use procmail to invoke dspam, or actually better yet, have
> SA do it for me.

I wrote basic SA plugin for crm114 and posted on this
list 2 or 3 weeks ago. It basically works, but there's
much place for improvement.

Implementing DSPAM as plugin is a harder task
requiring modification of SA to add support for
rewriting message by plugins because DSPAM signature
(or even attachment with tokens) must somehow be
appended to message.
Eugene

--
Email: jmv@online.ru

On Fri, Mar 12, 2004 at 06:27:57PM +0300, Eugene Morozov wrote:
> Chuck Campbell <campbell@accelinc.com> writes:
>
> > Is there a how-to around, or an implementation of Dspam and/or crm114 into
> > the Spamassassin world/way of doing things?
> >
> > I read the thread here recently, which resulted in an implementation of
> > crm114, and I'm looking at it now.
> >
> > I looked at dspam, which currently appears to sit between sendmail and
> > procmail (or maybe replaces procmail) as a mail delivery agent, but it
> > seems to me that is an invasive approach.
> >
> > I like the concept of word sets in a Bayesian approach, and their documentation
> > is compelling. I just don't clearly see how to hammer it into the SA
> > world (with which I am now familiar).
> >
> > I was hoping to use procmail to invoke dspam, or actually better yet, have
> > SA do it for me.
>
> I wrote basic SA plugin for crm114 and posted on this
> list 2 or 3 weeks ago. It basically works, but there's
> much place for improvement.

This is what I was looking at, and referred to above. Thank you for your
efforts, I applaud them.

It does require SA v3.0 to use plug-ins, right? I'm currently still on 2.63.

> Implementing DSPAM as plugin is a harder task
> requiring modification of SA to add support for
> rewriting message by plugins because DSPAM signature
> (or even attachment with tokens) must somehow be
> appended to message.
> Eugene

Could this be appended as a header? The formail executable (part of the
procmail distribution) can do this easily, but I assume perl could as
well.

Obviously the dspam stuff would then need to be changed, if it is going to
process the header, but maybe it could be added as a header in addition to
the way dspam does it?

This might all be doable, but maybe not in an easily maintained way,
especially for incorporating dspam as a package into the SA world.

I may just have a flawed understanding of the scope of the problem as well.
Right now, I'm trying to see a way forward, without going back three steps
for every one I take.

At the moment it seems that I can only have dspam or spamassassin, and that
they are mutually exclusive.

-chuck

> From: Chuck Campbell [mailto:campbell@accelinc.com]
> Sent: Friday, March 12, 2004 7:10 AM
[...]
>
> I was hoping to use procmail to invoke dspam, or actually better yet, have
> SA do it for me.

(The following suggestion is just a sketch, and has not been tested.)

You might try something along these lines:

1) remove DSPAM from your mail transfer agent's config. file so that
DSPAM is no longer invoked as an LDA.

2) rebuild and install DSPAM with
./configure --enable-delivery-to-stdout --enable-spam-delivery
(these may be enabled by default, not sure)

3) Add a new SA rule, that looks for DSPAM's header:

header DSPAM_SPAM X-DSPAM-Result =~ /^Spam$/ [if-unset: Innocent]
describe DSPAM_SPAM DSPAM says it is spam
score DSPM_SPAM 3.0

## you could also add a negative score if DSPAM says it is innocent.
## Since DSPAM errs on the side of mail being ham, I'd suggest not
## giving this score as much weight.
##
## header DSPAM_HAM X-DSPAM-Result =~ /^Innocent$/ [if-unset: Spam]
## describe DSPAM_HAM DSPAM says it is ham
## score DSPM_HAM -1.0

4) In your .procmailrc file, try the following:

# Run everything through the dspam filter
:0 fw
| dspam --stdout --deliver-spam

# Now run the mail through spamassassin
# (could be combined with the previous pipe to dspam)
:0 fw
| spamassassin

# File the spam
:0
* ^X-Spam-Status: Yes
spam.mbox

---

If you get this working, please post a note back to this list, indicating
what you had to do to make it work.

Chuck Campbell <campbell@accelinc.com> writes:

> This is what I was looking at, and referred to above. Thank you for your
> efforts, I applaud them.
>
> It does require SA v3.0 to use plug-ins, right? I'm
> currently still on 2.63.

Yes, you need development version of SA to use
plug-ins. It's fairly easy to install but when I did so
I've found several bugs thereafter. These bugs were
fixed very soon though.

>
>> Implementing DSPAM as plugin is a harder task
>> requiring modification of SA to add support for
>> rewriting message by plugins because DSPAM signature
>> (or even attachment with tokens) must somehow be
>> appended to message.
>> Eugene
>
> Could this be appended as a header? The formail executable (part of the
> procmail distribution) can do this easily, but I assume perl could as
> well.
>
> Obviously the dspam stuff would then need to be changed, if it is going to
> process the header, but maybe it could be added as a header in addition to
> the way dspam does it?
>
> This might all be doable, but maybe not in an easily maintained way,
> especially for incorporating dspam as a package into
> the SA world.

It isn't necessary to change dspam. Plugin will feed
message to dspam and retrieve message signature and
status from dspam. But for using dspam
training-by-forward feature you have to append dspam
signature to message (or an attachment with tokens used
to train dspam). SA currently doesn't have facilities
for changing message from plugins.

>
> At the moment it seems that I can only have dspam or spamassassin, and that
> they are mutually exclusive.
>

No, you can first pipe message through dspam and then
pipe it through spamassassin.
Eugene

--
Email: jmv@online.ru, eugene@renice.org

Comments/corrections, below.

> 3) Add a new SA rule, that looks for DSPAM's header:
>
> header DSPAM_SPAM X-DSPAM-Result =~ /^Spam$/ [if-unset: Innocent]
> describe DSPAM_SPAM DSPAM says it is spam
> score DSPM_SPAM 3.0
>
> ## you could also add a negative score if DSPAM says it is innocent.
> ## Since DSPAM errs on the side of mail being ham, I'd suggest not
> ## giving this score as much weight.
> ##
> ## header DSPAM_HAM X-DSPAM-Result =~ /^Innocent$/ [if-unset: Spam]
> ## describe DSPAM_HAM DSPAM says it is ham
> ## score DSPM_HAM -1.0
>

Above, I'm not sure if SA removes the white space separating the
header field name from the header value. So, maybe this would be
safer:

header DSPAM_SPAM X-DSPAM-Result =~ /Spam/ [if-unset: Innocent]
## header DSPAM_HAM X-DSPAM-Result =~ /Innocent/ [if-unset: Spam]

> 4) In your .procmailrc file, try the following:
>
> # Run everything through the dspam filter
> :0 fw
> | dspam --stdout --deliver-spam
>
> # Now run the mail through spamassassin
> # (could be combined with the previous pipe to dspam)
> :0 fw
> | spamassassin
>
> # File the spam
> :0
> * ^X-Spam-Status: Yes
> spam.mbox

Above, that mailbox needs to be locked, to prevent race conditions
when delivering (note the ":" after the ":0" below)

# File the spam
:0:
* ^X-Spam-Status: Yes
spam.mbox