Mailing List Archive

Bob George said:
> Since you're already calling a shell script and perl, perhaps this is of
> interest:
>
> http://www.everysoft.com/clamfilter.html
>
> It's a single perl script that I've had good luck with for running inbound
> attachments through clamav.

Nifty, thanks Bob. Wish I had run across this when I first set up ClamAV.

--
Chris Thielen

Easily generate SpamAssassin rules to catch obfuscated spam phrases
(0BFU$C/\TED SPA/\/\ P|-|RA$ES):
http://www.sandgnat.com/cmos/

> http://www.everysoft.com/clamfilter.html
>
> It's a single perl script that I've had good luck with for running inbound
> attachments through clamav.

But it's calling clamscan with the --mbox switch and the manpage says:
DO NOT enable this option in scripts doing things such as
connecting clamscan to your mailer. Or anything accessable to the
public or production boxes. This code is VERY unstable (see the
last weeks Changlog for and example of the bugrate).

So I made a healthy mix of clamstream and clamfilter:
http://andy.spiegl.de/antivirus/clamfilter.pl

It's not perfect, suggestions welcome!
Andy.

--
o _ _ _
------- __o __o /\_ _ \\o (_)\__/o (_) -o)
----- _`\<,_ _`\<,_ _>(_) (_)/<_ \_| \ _|/' \/ /\\
---- (_)/ (_) (_)/ (_) (_) (_) (_) (_)' _\o_ _\_v
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
A small clue and no budget will get you a lot farther
than no clue and a big budget.

On Tue, 10 Feb 2004, Andy Spiegl wrote:

> But it's calling clamscan with the --mbox switch and the manpage says:
> DO NOT enable this option in scripts doing things such as
> connecting clamscan to your mailer. Or anything accessable to the

What version are you looking that? I don't see that in the man page for
0.66 (NOTE ** Anyone running clam should update to 0.66, as it fixes the
DOS bug and catches MyDoom bounces that got through before. Just make sure
you copy etc/freschlam.cf to /usr/local/etc or freshclam won't run. Hey,
it's 0.x software, what do you want ? :-)

I posted my method some time ago, no scripts, 4 lines in procmail.rc, and
I've seen it recomended on the clamav mailing list since I posted it there.


VIRUS=`/usr/local/bin/clamdscan --mbox --disable-summary --stdout -`
:0 Di
* VIRUS ?? FOUND
/dev/null


==========================================================
Chris Candreva -- chris@westnet.com -- (914) 967-7816
WestNet Internet Services of Westchester
http://www.westnet.com/

Christopher X. Candreva <chris@westnet.com> wrote:
> [...]
> I posted my method some time ago, no scripts, 4 lines in
> procmail.rc, and I've seen it recomended on the clamav mailing
> list since I posted it there.
>
>
> VIRUS=`/usr/local/bin/clamdscan --mbox --disable-summary
> --stdout -`
>> 0 Di
> * VIRUS ?? FOUND
> /dev/null

I much prefer this approach, but I seem to be having trouble making it work.
Can you point to a URL/archive with the details?

Thanks!

- Bob

On Wed, 11 Feb 2004, Bob George wrote:

> I much prefer this approach, but I seem to be having trouble making it work.
> Can you point to a URL/archive with the details?

Not offhand, check the settings in the clamav.conf file, make sure you have
SaveStreamToDisk and ScanMail enabled

==========================================================
Chris Candreva -- chris@westnet.com -- (914) 967-7816
WestNet Internet Services of Westchester
http://www.westnet.com/

On Wednesday 11 February 2004 11:55 am, Christopher X. Candreva wrote:
> On Wed, 11 Feb 2004, Bob George wrote:
> > I much prefer this approach, but I seem to be having trouble making it
> > work. Can you point to a URL/archive with the details?
>
> Not offhand, check the settings in the clamav.conf file, make sure you have
> SaveStreamToDisk and ScanMail enabled
>
-I didn't catch this whole thread But I'm using maildrop with a shell script
to pipe the mail through clamd. It works very well. If you need the scripts
let me know.

Brook Humphrey <bah@webmedic.net> wrote:
> [...]
> -I didn't catch this whole thread But I'm using maildrop with
> a shell script to pipe the mail through clamd. It works very
> well. If you need the scripts let me know.

Christopher had a procmail recipe that did it all within procmail. I'm already
doing a lot of stuff with spamassassin, etc. so was trying to avoid calling
shell/perl scripts. Thanks for the offer though.

Thanks to Christopher's hint, I wound up with the following in procmailrc,
which seems to be working:

#======================================================
# clamav virus scan
#======================================================
:0 HB
VIRUS=|/usr/bin/clamscan --mbox --disable-summary --stdout -

:0 Dfw
* VIRUS ?? ^.*: \/.* FOUND
| formail -A "X-Virus-Status: yes, $MATCH"

:0 Efw
| formail -A "X-Virus-Status: no"

Note that I tag infected messages, not dump them immediately, for gathering
stats. It seems to be catching the test infected messages I sent though just
fine. I probably need to dump --stdout as well, since it's not acting as a
filter.

Now -- IF mbox is a problem -- I could just send the BODY through with the
procmail B flag alone, and not use --mbox, right?

I realize we're getting off-topic, but I want to tag spam AND virus messages
with the respective most appropriate tool, then handle final disposition
afterwards (in one place).

- Bob

On Wed, 11 Feb 2004, Bob George wrote:

> Thanks to Christopher's hint, I wound up with the following in procmailrc,
> which seems to be working:
>
> #======================================================
> # clamav virus scan
> #======================================================
> :0 HB
> VIRUS=|/usr/bin/clamscan --mbox --disable-summary --stdout -
>
> :0 Dfw
> * VIRUS ?? ^.*: \/.* FOUND
> | formail -A "X-Virus-Status: yes, $MATCH"
>
> :0 Efw
> | formail -A "X-Virus-Status: no"
>
> Note that I tag infected messages, not dump them immediately, for gathering
> stats. It seems to be catching the test infected messages I sent though just
> fine. I probably need to dump --stdout as well, since it's not acting as a
> filter.
>
> Now -- IF mbox is a problem -- I could just send the BODY through with the
> procmail B flag alone, and not use --mbox, right?

Not quite. You would have to extract the Base64 encoded attachments from
the body, store them in temp files and then feed each of those to clamscan
if you want to omit the '--mbox' switch.

That's what clamscan does for you when you enable the --mbox mode,
and that's the code that has been problimatic in the past.

Another alternative would be to use a different package such as MIMEDefang
or mailscanner to do the extracting.

I note that you're using 'clamscan' rather than 'clamdscan' in that
script. Note that 'clamscan' has significant startup overhead as
opposed to using 'clamdscan' & 'clamd' (similar to spamassassin vs
spamc/spamd).

--
Dave Funk University of Iowa
<dbfunk (at) engineering.uiowa.edu> College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
#include <std_disclaimer.h>
Better is not better, 'standard' is better. B{

David B Funk <dbfunk@engineering.uiowa.edu> wrote:
> [...]
> Not quite. You would have to extract the Base64 encoded
> attachments from the body, store them in temp files and then
> feed each of those to clamscan if you want to omit the
> '--mbox' switch.

Ah, makes sense. I am also checking out bitdefender (freeware vers.) and it
requires similar extraction (as well as unarchiving). That might be an
alternative approach.

> That's what clamscan does for you when you enable the --mbox
> mode, and that's the code that has been problimatic in the past.

I'm using debian sarge/testing, and it doesn't seem to be the latest version.
The --mbox switch isn't causing me any problems that I've noted, but I will
keep my eyes out for an update.

> Another alternative would be to use a different package such
> as MIMEDefang or mailscanner to do the extracting.

I've tested anomy sanitizer, which is similar. I'm looking for something
that'll flag viruses without necessarily stopping all attachments outright.
Sanitizer is very good at defanging HTML in messages though.

> I note that you're using 'clamscan' rather than 'clamdscan' in
> that script. Note that 'clamscan' has significant startup
> overhead as opposed to using 'clamdscan' & 'clamd' (similar to
> spamassassin vs spamc/spamd).

Ah, I missed the daemon version in the deb listing. I've just installed it.
Thanks for the tip.

I also seem to have the procmail recipe for scanning working, but NOT the
procmail E recipe for clean messages. That's a procmail issue though, not SA.
Need to fix that.

- Bob

(I posted David's suggestion to the procmail list, because the topic of
invoking a virus scanner from procmail had come up over there recently as
well. Here is some early feedback. I see a an additional opportunity to
streamline the script My suggestion is prefixed with GF)


From: Scott Wiersdorf <scott@perlcode.org>
Date: Wed, 11 Feb 2004 16:14:16 -0700

On Wed, Feb 11, 2004 at 02:53:25PM -0800, Gary Funck wrote:
>
> Thanks to Christopher's hint, I wound up with the following in procmailrc,
> which seems to be working:
>
> #======================================================
> # clamav virus scan
> #======================================================
> :0 HB
> VIRUS=|/usr/bin/clamscan --mbox --disable-summary --stdout -

The HB flags apply to conditions, of which you have none. By default
procmail pipes both h and b, so you can remove 'HB':


:0
VIRUS=|/usr/bin/clamscan --mbox --disable-summary --stdout -

GF: Since there are no conditions/tests in that rule, this is
simpler:

VIRUS=`/usr/bin/clamscan --mbox --disable-summary --stdout -`



> :0 Dfw
> * VIRUS ?? ^.*: \/.* FOUND
> | formail -A "X-Virus-Status: yes, $MATCH"

This could be made more efficient with an 'h', and 'D' is probably
unnecessary:

:0 hfw
* VIRUS ?? ^.*: \/.* FOUND
| formail -A "X-Virus-Status: yes, $MATCH"


Same with this one:

:0 E hfw
| formail -A "X-Virus-Status: no"

> Note that I tag infected messages, not dump them immediately, for
gathering
> stats. It seems to be catching the test infected messages I sent though
just
> fine. I probably need to dump --stdout as well, since it's not acting as a
> filter.
>
> Now -- IF mbox is a problem -- I could just send the BODY through with the
> procmail B flag alone, and not use --mbox, right?

I'm not sure what heuristics clamav uses to decide mime parts, etc. It
may rely on certain MIME headers (though unlikely) to decide how to
parse the message. The best way is to just try it, I guess. Sandbox applies.

Gary Funck <gary@intrepid.com> wrote:
> (I posted David's suggestion to the procmail list, because the
> topic of invoking a virus scanner from procmail had come up
> over there recently as well. Here is some early feedback. I
> see a an additional opportunity to streamline the script My
> suggestion is prefixed with GF)

I appreciate the assistance!

>>:0 HB
>> VIRUS=|/usr/bin/clamscan --mbox --disable-summary --stdout -
>
> The HB flags apply to conditions, of which you have none. By
> default procmail pipes both h and b, so you can remove 'HB':

I threw that in during an early attempt to get it all working. Without it, it
does work fine.

>> :0
>> VIRUS=|/usr/bin/clamscan --mbox --disable-summary --stdout -
>
> GF: Since there are no conditions/tests in that rule, this is
> simpler:
>
> VIRUS=`/usr/bin/clamscan --mbox --disable-summary --stdout -`

This I does not seem to work for me (procmail v3.22), whereas the pipe version
does. I think the message isn't getting fed in via stdin without the pipe, so I
wind up with just the clamdscan result on an empty file (stream: OK).

>> :0 Dfw
>> * VIRUS ?? ^.*: \/.* FOUND
>> | formail -A "X-Virus-Status: yes, $MATCH"
>
> This could be made more efficient with an 'h', and 'D' is
> probably unnecessary:

I haven't done timing tests, but yes it does work well without. Does D incur a
measurable penalty? I anchored the result to the end of the line, just to play
safe.
:0 hfw
* VIRUS ?? ^.*: \/.* FOUND$
| formail -A "X-Virus-Status: Yes, $MATCH"


NOTE: This (my recipe) did NOT work after all. I have bad luck with E. A quick
google leads me to think that E is quirky. Perhaps only works if previous
recipe is a delivering one? Instead, I just plunked in:

:0 hfw
* VIRUS ?? ^.*OK$
| formail -A "X-Virus-Status: No"

>> [...]
>> Now -- IF mbox is a problem -- I could just send the BODY
>> through with the procmail B flag alone, and not use --mbox,
>> right?
>
> I'm not sure what heuristics clamav uses to decide mime parts,
> etc. It may rely on certain MIME headers (though unlikely) to decide
> how to parse the message. The best way is to just try it, I guess.

It sounds like that's not an easy fix without some additional work. I need to
update to the latest clamav to be safe from --mbox issues.

> Sandbox applies.

Good point!

To tie this all back to SA, I was originally thinking that feeding bayes
(sa-learn) virus-infected messages might be a good thing, hoping that bayes
might catch a Day 0 attack that the virus-scanner might miss. This may be a
case of using the wrong tool for the job, but I'm hoping "smells like a virus"
is useful, while still allowing attachments (realizing this is the root of the
problem.)

- Bob

> From: Bob George [mailto:mailings02@ttlexceeded.com]
> Sent: Wednesday, February 11, 2004 4:43 PM
>
>
> Gary Funck <gary@intrepid.com> wrote:
> > (I posted David's suggestion to the procmail list, because the
> > topic of invoking a virus scanner from procmail had come up
> > over there recently as well. Here is some early feedback. I
> > see a an additional opportunity to streamline the script My
> > suggestion is prefixed with GF)
>
> I appreciate the assistance!
>
> >>:0 HB
> >> VIRUS=|/usr/bin/clamscan --mbox --disable-summary --stdout -
> >
> > The HB flags apply to conditions, of which you have none. By
> > default procmail pipes both h and b, so you can remove 'HB':
>
> I threw that in during an early attempt to get it all working.
> Without it, it
> does work fine.
>
> >> :0
> >> VIRUS=|/usr/bin/clamscan --mbox --disable-summary --stdout -

Above, I'd add a 'w' (wait for result) to be on the safe side;

:0 w
VIRUS=|/usr/bin/clamscan --mbox --disable-summary --stdout -

otherwise, there may be a race condition with the tests that
follow. This might've been a cause of your 'E' rule not working.

> >
> > GF: Since there are no conditions/tests in that rule, this is
> > simpler:
> >
> > VIRUS=`/usr/bin/clamscan --mbox --disable-summary --stdout -`
>
> This I does not seem to work for me (procmail v3.22), whereas the
> pipe version
> does. I think the message isn't getting fed in via stdin without
> the pipe, so I
> wind up with just the clamdscan result on an empty file (stream: OK).

I'm guessing that clamscan returned a non-zero exit code, and in the
backtick context it discards the output, but in the pipe methoed it ignores
the exit code. you might be able to confirm this by placing these lines at
the top of your test recipe,

LOGFILE=`rm -f test.log && echo test.log`
VERBOSE=yes
and then look in test.log (under your $MAILDIR)
for the execution trace of the recipe.

>
> >> :0 Dfw
> >> * VIRUS ?? ^.*: \/.* FOUND
> >> | formail -A "X-Virus-Status: yes, $MATCH"
> >
> > This could be made more efficient with an 'h', and 'D' is
> > probably unnecessary:
>
> I haven't done timing tests, but yes it does work well without.
> Does D incur a
> measurable penalty?

D should be slightly more efficient (it disables the default
'ignore case'). The big savings will come from filtering only
the header.

> I anchored the result to the end of the line,
> just to play
> safe.
> :0 hfw
> * VIRUS ?? ^.*: \/.* FOUND$
> | formail -A "X-Virus-Status: Yes, $MATCH"
>
> >> :0 Efw
> >> | formail -A "X-Virus-Status: no"
>
> NOTE: This (my recipe) did NOT work after all. I have bad luck
> with E. A quick
> google leads me to think that E is quirky.

Really? How so? And how was the recipe breaking when 'E' was
in there? I think VERBOSE=yes and looking at the log would add
some insight.

> Perhaps only works if previous
> recipe is a delivering one? Instead, I just plunked in:
>

Nope. If it delivered (and didn't also do a 'c' carbon copy, the script
would end, like executing a return in the middle of a subroutine. But in
fact, you are not delivering in the first rule, you are 'f' filtering.

> :0 hfw
> * VIRUS ?? ^.*OK$
> | formail -A "X-Virus-Status: No"
>

Gary Funck <gary@intrepid.com> wrote:
> [...]
> Above, I'd add a 'w' (wait for result) to be on the safe side;
>
>> 0 w
> VIRUS=|/usr/bin/clamscan --mbox --disable-summary --stdout -
>
> otherwise, there may be a race condition with the tests that
> follow. This might've been a cause of your 'E' rule not
> working.
> [...]
> I'm guessing that clamscan returned a non-zero exit code, and
> in the backtick context it discards the output, but in the
> pipe methoed it ignores the exit code. you might be able to
> confirm this by placing these lines at the top of your test
> recipe,

That DOES appear to have been the problem. With clamav taking a bit longer to
run than what I'd used before, it seems to have made the difference. Many
thanks.

> [... procmail E quirks ...]
> Really? How so?

That was the impression I was left after skimming:
http://pm-doc.sourceforge.net/pm-tips-body.html Section 5.13. That document is
covering complicated cases, so I probably read too much into it.

>> [...]
>> Perhaps only works if previous
>> recipe is a delivering one? Instead, I just plunked in:
>>
>
> Nope. If it delivered (and didn't also do a 'c' carbon copy,
> the script would end, like executing a return in the middle of
> a subroutine. [...]

Uhm... of course. Hmm. OK, there's a good reason I didn't stay in programming!

- Bob

I've been eagerly following along on this procmail recipe for virus
scanning. And I kindly appreciate the indulgence of the list ...

Based on the messages so far, I reduced the clamav scan to this:

-----------------------------------cut here---------------------------
# Scan for viruses

:0 w
VIRUS=|/usr/local/bin/clamscan --mbox --disable-summary --stdout -

:0 hfw
* VIRUS ?? ^.*: \/.* FOUND
| formail -A "X-Virus-Status: yes, $MATCH"

:0 E hfw
| formail -A "X-Virus-Status: no"

-----------------------------------cut here---------------------------

Gary Funck, you wrote:
>I'm guessing that clamscan returned a non-zero exit code, and in the
>backtick context it discards the output, but in the pipe methoed it ignores
>the exit code. you might be able to confirm this by placing these lines at
>the top of your test recipe,

I sent myself a mail and attached one of the test zip files provided with
clamav, so it should produce a positive result.

With this recipe, procmail produces this log. It's not ignoring the
return code at all:

procmail: Assigning "VIRUS="
procmail: Executing "/usr/local/bin/clamscan,--mbox,--disable-summary,--stdout,-"
procmail: [20344] Thu Feb 12 15:36:29 2004
procmail: Program failure (1) of "/usr/local/bin/clamscan"
procmail: [20344] Thu Feb 12 15:36:29 2004
procmail: No match on "^.*: \/.* FOUND"

Clamscan exits with "1" if a virus is found.

So there are a couple of things I've discovered:

1. It appears that procmail is discarding the output on a non-zero exit
code in pipe context. My procmail version is 3.22. (I'll go check to
see if I'm out of date.)

2. I tested the clam command line from shell with my test email using
both clamscan and clamdscan. Clamscan properly reports the virus.
Clamdscan reports a clean message. I used the same arguments for both.

Mojo
--
Morris Jones <*>
Monrovia, CA
mojo@whiteoaks.com
http://www.whiteoaks.com

> From: Morris Jones [mailto:mojo@whiteoaks.com]
> Sent: Thursday, February 12, 2004 3:57 PM
>
> I've been eagerly following along on this procmail recipe for virus
> scanning. And I kindly appreciate the indulgence of the list ...
>
> Based on the messages so far, I reduced the clamav scan to this:
>
> -----------------------------------cut here---------------------------
> # Scan for viruses
>
> :0 w
> VIRUS=|/usr/local/bin/clamscan --mbox --disable-summary --stdout -
>

Sooner or later, we'll get it right. That needs to be a capital 'W' up
there to tell procmail to ignore non-zero exit codes.

:0 W
VIRUS=|/usr/local/bin/clamscan --mbox --disable-summary --stdout -

On Thu, 12 Feb 2004, Morris Jones wrote:

> 2. I tested the clam command line from shell with my test email using
> both clamscan and clamdscan. Clamscan properly reports the virus.
> Clamdscan reports a clean message. I used the same arguments for both.
>
> Mojo

They -should- return the same result, with the same args and same input.

Are you sure that your clamscan and clamd are built from the same kit
and looking at the same ".cvd" files?

One other possibility, a permissions issue.

Note that "clamscan" runs with the UID of the invoker whereas the
"clamd" runs with the UID configured in the clamav.conf file
(the 'User' parameter, by default "clamav").

Thus the "clamav" user needs read permissions on the file for the
clamdscan to work.

For testing, you can add the "--debug" argument to clamscan
to see what it's doing. It doesn't do anything for clamdscan,
as clamd is doing the work. You will have to turn on "Debug"
in the clamav.conf file (probably also want to turn on "Forground"
and restart clamd in a window to watch the output).

--
Dave Funk University of Iowa
<dbfunk (at) engineering.uiowa.edu> College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
#include <std_disclaimer.h>
Better is not better, 'standard' is better. B{