Unsubscribe
________________________________
From: clamav-users <clamav-users-bounces@lists.clamav.net> on behalf of clamav-users-request@lists.clamav.net <clamav-users-request@lists.clamav.net>
Sent: Saturday, 2 October 2021 12:00 PM
To: clamav-users@lists.clamav.net <clamav-users@lists.clamav.net>
Subject: clamav-users Digest, Vol 203, Issue 2
Send clamav-users mailing list submissions to
clamav-users@lists.clamav.net
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.clamav.net/mailman/listinfo/clamav-users
or, via email, send a message with subject or body 'help' to
clamav-users-request@lists.clamav.net
You can reach the person managing the list at
clamav-users-owner@lists.clamav.net
When replying, please edit your Subject line so it is more specific
than "Re: Contents of clamav-users digest..."
When responding, please don't respond with the entire Digest. Please trim your response.
Today's Topics:
1. Re: Verifying bytecode, phishing and other type of threats
(Indranil)
2. Scanning a zip file fails, extract it, scan with the same
options and it passes (Max Allan)
3. Re: Verifying bytecode, phishing and other type of threats
(G.W. Haywood)
4. Re: Scanning a zip file fails, extract it, scan with the same
options and it passes (Laurent S.)
----------------------------------------------------------------------
Message: 1
Date: Fri, 1 Oct 2021 18:17:03 +0530
From: Indranil <myselfindranil@gmail.com>
To: ClamAV users ML <clamav-users@lists.clamav.net>
Cc: "G.W. Haywood" <clamav@jubileegroup.co.uk>
Subject: Re: [clamav-users] Verifying bytecode, phishing and other
type of threats
Message-ID:
<CAAaur943GFReRR3=_r1-fOLU5jZRT0m-zpbbVsH1SFxcwOOoAQ@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Hello Ged,
Thanks for your response. Here is the list of steps:
1. Download: http://www.clamav.net/downloads/production/ClamAV-0.103.0.exe
2. Installed from this exe.
3. Opened windows powershell in admin mode
4. cd "c:\program files\clamav"
5. copy .\conf_examples\freshclam.conf.sample .\freshclam.conf
6. copy .\conf_examples\clamd.conf.sample .\clamd.conf
7. Commented "Example" in freshclam.conf and clamd.conf
8. Started clamd.exe in one powershell window
9. .\clamdscan -m -i 'C:\Users\indra\AppData\' from another powershell window
10. The result is show below
----------- SCAN SUMMARY -----------
Infected files: 0
Total errors: 800 <<<<<<<<<<<<<<<<<<< Please ignore this.
These are all permission denied errors.
Time: 348.501 sec (5 m 48 s)
Start Date: 2021:10:01 17:35:57
End Date: 2021:10:01 17:41:46
I intend to run clamAv on my windows laptop. There is no mail-server
configured here. However, I plan to scan all new file addition in
C:\Users\<user name>\Downloads, C:\Users\<username>\Desktop,
C:\Users\<username>\Documents, C:\Users\<username>\AppData and new
email for all possible threats (including malware, phishing, bytecode
etc).
In all my test runs I am getting clean results. But at present I am
trying to find examples of various threat detection. So eventually
from the threat message I want to
distinguish a malware from phishing from a bytecode etc. Do you have
any suggestion for me to better visualize (via running test scans)
threat detection?
The output from ./clamconf.exe -n shows that I do not have any
non-default configuration
I have changed anything in clamd.conf. Do you have any suggestion for
any non-default configuration which can lead to better security?
Thank you,
Indranil
On Thu, Sep 30, 2021 at 8:36 PM G.W. Haywood via clamav-users
<clamav-users@lists.clamav.net> wrote:
>
> Hi there,
>
> On Thu, 30 Sep 2021, Indranil via clamav-users wrote:
>
> > I have downloaded ClamAV and verified the eicar test using clamscan
> > on a Windows VM.
>
> Please tell us exactly what you downloaded, what you installed it on,
> and exactly how you installed it. Please also describe how you plan
> to approach scanning in general terms - which will probably make it
> easier to answer a number of questions which you haven't yet asked.
>
> > It appears that the following detection capabilities are also
> > enabled by default: bytecode, scan-mail, phishing-sigs,
> > phishing-scan-urls, scan-pe, scan-elf, scan-pdf, scan-html.
>
> The descriptions of capabilities which you have given resemble some
> configuration and scanning options which I recognize, but they are
> unsubtly different. The relevant options (in the configuration files
> on your machine, and given on the command line) are in the official
> ClamAV documentation:
>
> https://docs.clamav.net/
>
> For example there is a 'Bytecode' option in clamd.conf, but there is
> no 'scan-mail' option that I know of - it is 'ScanMail'. Similarly,
> there are 'PhishingSignatures' and 'PhishingScanURLs' options but not
> those that you give, 'phishing-sigs' and 'phishing-scan-urls'. It
> baffles me that people make such gratuitous changes, it must be very
> confusing to newcomers. There are also quite a few other options
> which you have not mentioned. There's a scanning daemon and a thing
> called a 'milter' (another daemon) which is to scan mail on a mail
> server; these are separate subsystems in their own right which may or
> may not be installed and which will need to be separately configured.
> You have not said whether or not you wish to use a daemon, but that is
> one of the more fundamental decisions - see my first paragraph.
>
> > Out of these options, I am able to test scan-pe, scan-elf, scan-pdf
> > and scan-html using respective files.
>
> Please explain exactly what you mean by 'test'.
>
> > I have not been able to test the rest of the options such as
> > bytecode, scan-mail, phishing-sigs and phishing-scan-urls. Could you
> > please help with the method of verifying individual options.
>
> Please read the official documentation at the link which I have given.
> It is much easier to confirm that your configuration is as you wish
> than to test that the code is doing what you might expect. Simply run
>
> clamconf -n
>
> and you should see the differences between your configuration and the
> documented defaults. In any case the expectation is sometimes based
> on wishful thinking, so please let us know what you expect from your
> tests before you ask us to help you with an exercise which is poorly
> defined and might well be open-ended.
>
> > Also, when a threat is detected, does ClamAV report the type of the
> > threat i.e. does ClamAV report that Threat1 is a ?bytecode? threat,
> > Threat2 is a ?phishing-sigs? threat?
>
> In a way it does, yes. It reports a string like "Something FOUND",
> either at the command line or in some log. It may also report other
> information such as how much data was scanned and the scan time, and
> if you wish you can configure verbose logging, and temorary files to
> be retained for later inspection. Be careful because these can use a
> lot of storage space. The ClamAV 'sigtool' utility can help you to
> investigate what was found.
>
> Here are some examples from a mail server log of things "FOUND":
>
> 258 Porcupine.Junk.36046.UNOFFICIAL FOUND
> 312 YARA.Bank_rule.UNOFFICIAL FOUND
> 360 Win.Packed.Ratx-9895842-0 FOUND
> 366 Sanesecurity.Jurlbl.7e72e8.UNOFFICIAL FOUND
> 17353 YARA.Garbage_Spam_0006_Rule.UNOFFICIAL FOUND
>
> Our mail server uses a milter (not the one available from ClamAV) to
> pass incoming mail streams to the 'clamd' scanning daemon and write to
> the logs. I have used OS tools to trawl the logs for September 2021.
> The counts are the number of times that this particular kind of threat
> was found in the incoming mail stream. As you can see, some lines are
> marked as "UNOFFICIAL". This means that the threat was detected by a
> signature from something other than the official ClamAV database. In
> addition to the official signature databases, we use both our own Yara
> rules and a number of third-party databases. These greatly extend the
> usefulness of ClamAV in our situation. At the time of writing, there
> are about 8.8 million signatures in our ClamAV database. Of those,
> 8.6 million are from the 'official' ClamAV databases and the rest are
> 'third party' and our own. There are 583 of our own Yara signatures.
> As you see from the table, by a very large margin a single one of our
> Yara sigs catches more spam than all the rest put together. That's
> probably because we know a lot more about our spam profile than anyone
> else does. ClamAV is by no means a 'fire and forget' munition, please
> be aware that you are (hopefully) embarking on a journey of discovery.
>
> ClamAV does not attempt to repair anything which it finds. It can be
> instructed to remove, move or copy a suspect file. Please read the
> warnings in the documentation and think *very* carefully before doing
> anything like that, because if you aren't careful you will be a bigger
> threat to your systems than the threats from which you are trying to
> protect them.
>
> > If I am scanning C:\Users\Indranil via clamscan (with recursive option)
> > then does ?C:\Users\Indranil\AppData\Local\Microsoft\Outlook? get tested for
> > virus only if ?scan-mail? option is on?
>
> It isn't like that at all. ClamAV contains code which recognizes
> different types of data. For example, it can tell if a file is an
> archive (like a '.zip' file) or if a data stream appears to be a mail
> message. ClamAV treats files and data streams in much the same ways.
> It can and does scan selectively when it detects such things - certain
> signatures only apply to certain kinds of data - which is mainly why I
> have asked you to define 'test'. Some things happen in ways which you
> might describe as 'behind the scenes', and you need to be very careful
> about how you define tests. This probably means that you will need to
> know a lot more about ClamAV's behaviour than you do now in order to
> be able to test it.
>
> HTH
>
> --
>
> 73,
> Ged.
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
------------------------------
Message: 2
Date: Fri, 1 Oct 2021 17:09:37 +0100
From: Max Allan <max.allan@surevine.com>
To: clamav-users@lists.clamav.net
Subject: [clamav-users] Scanning a zip file fails, extract it, scan
with the same options and it passes
Message-ID:
<CADNp1BYedEHeyPY1QEKThnSb6QFhO8z5i3wky_Bqiz0tVh4FFw@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Hi,
I have a requirement (from the business) to AV scan all docker
containers we create.
I started experimenting with tomcat:latest, which is handy because you
can follow along at home easily!
Someone else has already recommended a scan command :
clamscan <file> \
--infected \
--recursive=yes \
--alert-exceeds-max=yes \
--max-recursion=2000000 \
--max-dir-recursion=2000000 \
--max-files=2000000 \
--max-filesize=2000M \
--max-scansize=2000M \
--max-embeddedpe=2000M \
--max-htmlnormalize=2000M \
--max-htmlnotags=2000M \
--max-scriptnormalize=2000M \
--max-ziptypercg=2000M \
--max-partitions=2000000 \
--max-iconspe=2000000 \
--max-rechwp3=2000000 \
--pcre-match-limit=2000000 \
--pcre-recmatch-limit=2000000 \
--pcre-max-filesize=2000M -a
So, if you run the tomcat:latest container, apt update, apt install
clamav, freshclam and run that scan command against
/usr/local/openjdk-11/lib/src.zip you will probably get a failure :
/usr/local/openjdk-11/lib/src.zip: Heuristics.Limits.Exceeded FOUND
/usr/local/openjdk-11/lib/src.zip!(0)ZIP:jdk.zipfs/jdk/nio/zipfs/ZipInfo.java:
Heuristics.Limits.Exceeded FOUND
---------- SCAN SUMMARY -----------
Known viruses: 8570214
Engine version: 0.103.3
Scanned directories: 0
Scanned files: 1
Infected files: 1
Data scanned: 290.07 MB
Data read: 55.52 MB (ratio 5.22:1)
Time: 260.438 sec (4 m 20 s)
Start Date: 2021:10:01 13:39:47
End Date: 2021:10:01 13:44:07
However, if I extract that zip file to /src and then run clamscan on
/src then it passes without a problem :
----------- SCAN SUMMARY -----------
Known viruses: 8570214
Engine version: 0.103.3
Scanned directories: 2076
Scanned files: 18415
Infected files: 0
Data scanned: 333.04 MB
Data read: 170.92 MB (ratio 1.95:1)
Time: 320.573 sec (5 m 20 s)
Start Date: 2021:10:01 13:23:39
End Date: 2021:10:01 13:29:00
(There are indeed 18415 files in that .zip according to unzip -l)
Or even scan the single file :
clamscan ZipInfo.java --infected --recursive=yes
--alert-exceeds-max=yes --max-recursion=2000000
--max-dir-recursion=2000000 --max-files=2000000
--max-filesize=2000M --max-scansize=2000M --max-embeddedpe=2000M
--max-htmlnormalize=2000M --max-htmlnotags=2000M
--max-scriptnormalize=2000M --max-ziptypercg=2000M
--max-partitions=2000000 --max-iconspe=2000000
--max-rechwp3=2000000 --pcre-match-limit=2000000
--pcre-recmatch-limit=2000000 --pcre-max-filesize=2000M -a
----------- SCAN SUMMARY -----------
Known viruses: 8570214
Engine version: 0.103.3
Scanned directories: 0
Scanned files: 1
Infected files: 0
Data scanned: 0.01 MB
Data read: 0.01 MB (ratio 1.50:1)
Time: 68.326 sec (1 m 8 s)
Start Date: 2021:10:01 16:03:14
End Date: 2021:10:01 16:04:22
Clearly the content of src.zip (ZipInfo.java) IS scannable, when
extracted, but for some reason not scannable when it is in a zip
file... Is this a bug? Or am I specifying some options that are
causing it??
(clamscan -V
ClamAV 0.103.3/26309/Fri Oct 1 09:03:53 2021 )
------------------------------
Message: 3
Date: Fri, 1 Oct 2021 17:24:40 +0100 (BST)
From: "G.W. Haywood" <clamav@jubileegroup.co.uk>
To: Indranil via clamav-users <clamav-users@lists.clamav.net>
Subject: Re: [clamav-users] Verifying bytecode, phishing and other
type of threats
Message-ID: <bcdaf3f5-1948-b3b9-511-c35defe47e6@jubileegroup.co.uk>
Content-Type: text/plain; format=flowed; charset=US-ASCII
Hi there,
On Fri, 1 Oct 2021, Indranil via clamav-users wrote:
> ...
> 9. .\clamdscan -m -i 'C:\Users\indra\AppData\'
The '-i' command-line option is specifically for 'clamscan'. The
'clamdscan' utility will ignore it.
> ... permission denied errors.
Obviously if you want ClamAV to scan something, either it has to have
permission to read it or something else must read it and pass the data
to the ClamAV scanning process. Here, for example, we run a separate
clamd server, which scans everything passed to it over the LAN by the
mail servers. If something manages to compromise the clamd server it
isn't a big deal because it can't access anything else on the network.
By the way in more than eighteen years of using ClamAV, that's never
happened here but it's a possibility you always need to consider.
> ... I plan to scan all new file addition in
> C:\Users\<user name>\Downloads, C:\Users\<username>\Desktop,
> C:\Users\<username>\Documents, C:\Users\<username>\AppData and new
> email for all possible threats ...
> ...
> In all my test runs I am getting clean results. But at present I am
> trying to find examples of various threat detection.
You mean sample of malware etc.?
> So eventually from the threat message I want to distinguish a
> malware from phishing from a bytecode etc. Do you have any
> suggestion for me to better visualize (via running test scans)
> threat detection?
When you see something FOUND, either on the command line or in a log,
you can ask ClamAV's 'sigtool' utility to tell you exactly what in the
data caused the report by passing the FOUND string to it. That will
show you exactly what the signature was looking for. Other than that
I'm not sure exactly what you want to see. Most people I think set up
periodic scans, and look in their logs from time to time. My personal
view is that this is asking for trouble, and I don't scan systems at
all. I only scan incoming data, which seems to be the sort of thing
which you're planning to do. In our case the incoming data is in mail
and it's scanned automatically by mail filters. If anything triggers
detection there then it's stopped in its tracks at that point. Either
it's automatically reported (because the server has seen sufficiently
similar abuse to recognize it without any doubt) and sent to /archive,
or someone (generally me) will see something brightly coloured on the
Web interface and then can look at the situation and take a view. We
use a purpose-built Web interface to the mail systems. This avoids a
huge amount of effort which would otherwise be needed run suspect data
past e.g. Jotti's malware scan, scan logs, report abuse etc. - but (1)
it's only any use if you run mail servers and (2) I don't see how the
effort could be justified for ad-hoc scans of a single Windows box.
> The output from ./clamconf.exe -n shows that I do not have any
> non-default configuration
>
> I have changed anything in clamd.conf. Do you have any suggestion for
> any non-default configuration which can lead to better security?
I take it you mean you have *not* changed anything in clamd.conf - or
at least nothing except commenting the 'Example' line?
To me, the most obvious configuration change to improve security would
be to use any third-party databases that you can get hold of. You can
tweak things in clamd.conf but you need to be careful. It's possible
to cause ClamAV to consume excessive resources and you might crash the
system; that, if you're unlucky, could mean you'd need to spend a lot
of time on recovering it. Running a separate ClamAV server has much
to recommend it.
You haven't asked about other ways of improving security, and to some
extent this list isn't really appropriate for discussing many of them,
but I would strongly advise you not to rely on ClamAV as your only way
of defending against threats of all kinds. If you do, then the game
is most probably already over. In the past I have posted my estimates
of detection rates to this list, you may want to look them over. It's
far more important to maintain good network hygiene than it is to run
a virus scanner. There are vastly more threats against Windows boxes
than there are against other systems - probably more than all threats
of all kinds against all other systems put together. That's one of
the main reasons I don't routinely run any Windows boxes.
--
73,
Ged.
------------------------------
Message: 4
Date: Fri, 01 Oct 2021 17:04:20 +0000
From: "Laurent S." <110ef9e3086d8405c2929e34be5b4340@protonmail.ch>
To: clamav-users@lists.clamav.net
Subject: Re: [clamav-users] Scanning a zip file fails, extract it,
scan with the same options and it passes
Message-ID: <a8352b91-0d45-0ae3-fcb7-2daf14f2ca72@protonmail.ch>
Content-Type: text/plain; charset="utf-8"; Format="flowed"
Dear Max Allan,
Heuristics.Limits.Exceeded doesn't mean the file is infected, but it's
only a warning telling you that something went above the limits you set.
It give the warning this way because of --alert-exceeds-max=yes
ClamAV managed to go search into those files in each case as you can see
from the scan summaries. It will count the zip as a single file.
I would recommend against copy-pasting all those parameters without
having given proper thought into what you are doing.
Best regards,
Laurent
On 01.10.21 18:09, Max Allan via clamav-users wrote:
> Hi,
> I have a requirement (from the business) to AV scan all docker
> containers we create.
> I started experimenting with tomcat:latest, which is handy because you
> can follow along at home easily!
> Someone else has already recommended a scan command :
>
> clamscan <file> \
> --infected \
> --recursive=yes \
> --alert-exceeds-max=yes \
> --max-recursion=2000000 \
> --max-dir-recur
sion=2000000 \
> --max-files=2000000 \
> --max-filesize=2000M \
> --max-scansize=2000M \
> --max-embeddedpe=2000M \
> --max-htmlnormalize=2000M \
> --max-htmlnotags=2000M \
> --max-scriptnormalize=2000M \
> --max-ziptypercg=2000M \
> --max-partitions=2000000 \
> --max-iconspe=2000000 \
> --max-rechwp3=2000000 \
> --pcre-match-limit=2000000 \
> --pcre-recmatch-limit=2000000 \
> --pcre-max-filesize=2000M -a
>
> So, if you run the tomcat:latest container, apt update, apt install
> clamav, freshclam and run that scan command against
> /usr/local/openjdk-11/lib/src.zip you will probably get a failure :
>
> /usr/local/openjdk-11/lib/src.zip: Heuristics.Limits.Exceeded FOUND
> /usr/local/openjdk-11/lib/src.zip!(0)ZIP:jdk.zipfs/jdk/nio/zipfs/ZipInfo.java:
> Heuristics.Limits.Exceeded FOUND
> ---------- SCAN SUMMARY -----------
> Known viruses: 8570214
> Engine version: 0.103.3
> Scanned directories: 0
> Scanned files: 1
>
Infected files: 1
> Data scanned: 290.07 MB
> Data read: 55.52 MB (ratio 5.22:1)
> Time: 260.438 sec (4 m 20 s)
> Start Date: 2021:10:01 13:39:47
> End Date: 2021:10:01 13:44:07
>
>
> However, if I extract that zip file to /src and then run clamscan on
> /src then it passes without a problem :
>
> ----------- SCAN SUMMARY -----------
> Known viruses: 8570214
> Engine version: 0.103.3
> Scanned directories: 2076
> Scanned files: 18415
> Infected files: 0
> Data scanned: 333.04 MB
> Data read: 170.92 MB (ratio 1.95:1)
> Time: 320.573 sec (5 m 20 s)
> Start Date: 2021:10:01 13:23:39
> End Date: 2021:10:01 13:29:00
>
> (There are indeed 18415 files in that .zip according to unzip -l)
>
> Or even scan the single file :
>
> clamscan ZipInfo.java --infected --recursive=yes
> --alert-exceeds-max=yes --max-recursion=2000000
> --max-dir-recursion=2000000 --max-files=2000000
> --max-filesize=2000M --max-scansize=2000M --max-embeddedpe=2000M
> -
-max-htmlnormalize=2000M --max-htmlnotags=2000M
> --max-scriptnormalize=2000M --max-ziptypercg=2000M
> --max-partitions=2000000 --max-iconspe=2000000
> --max-rechwp3=2000000 --pcre-match-limit=2000000
> --pcre-recmatch-limit=2000000 --pcre-max-filesize=2000M -a
>
> ----------- SCAN SUMMARY -----------
> Known viruses: 8570214
> Engine version: 0.103.3
> Scanned directories: 0
> Scanned files: 1
> Infected files: 0
> Data scanned: 0.01 MB
> Data read: 0.01 MB (ratio 1.50:1)
> Time: 68.326 sec (1 m 8 s)
> Start Date: 2021:10:01 16:03:14
> End Date: 2021:10:01 16:04:22
>
>
>
> Clearly the content of src.zip (ZipInfo.java) IS scannable, when
> extracted, but for some reason not scannable when it is in a zip
> file... Is this a bug? Or am I specifying some options that are
> causing it??
>
> (clamscan -V
> ClamAV 0.103.3/26309/Fri Oct 1 09:03:53 2021 )
>
> _______________________________________________
>
> clamav-users mailing list
> clam
av-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: publickey - 110ef9e3086d8405c2929e34be5b4340@protonmail.ch - d23fa0a8.asc
Type: application/pgp-keys
Size: 3204 bytes
Desc: not available
URL: <https://lists.clamav.net/pipermail/clamav-users/attachments/20211001/ff80df95/attachment-0001.key>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 855 bytes
Desc: OpenPGP digital signature
URL: <https://lists.clamav.net/pipermail/clamav-users/attachments/20211001/ff80df95/attachment-0001.sig>
------------------------------
Subject: Digest Footer
_______________________________________________
clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users
https://github.com/vrtadmin/clamav-faq
http://www.clamav.net/contact.html#ml
------------------------------
End of clamav-users Digest, Vol 203, Issue 2
********************************************
________________________________
From: clamav-users <clamav-users-bounces@lists.clamav.net> on behalf of clamav-users-request@lists.clamav.net <clamav-users-request@lists.clamav.net>
Sent: Saturday, 2 October 2021 12:00 PM
To: clamav-users@lists.clamav.net <clamav-users@lists.clamav.net>
Subject: clamav-users Digest, Vol 203, Issue 2
Send clamav-users mailing list submissions to
clamav-users@lists.clamav.net
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.clamav.net/mailman/listinfo/clamav-users
or, via email, send a message with subject or body 'help' to
clamav-users-request@lists.clamav.net
You can reach the person managing the list at
clamav-users-owner@lists.clamav.net
When replying, please edit your Subject line so it is more specific
than "Re: Contents of clamav-users digest..."
When responding, please don't respond with the entire Digest. Please trim your response.
Today's Topics:
1. Re: Verifying bytecode, phishing and other type of threats
(Indranil)
2. Scanning a zip file fails, extract it, scan with the same
options and it passes (Max Allan)
3. Re: Verifying bytecode, phishing and other type of threats
(G.W. Haywood)
4. Re: Scanning a zip file fails, extract it, scan with the same
options and it passes (Laurent S.)
----------------------------------------------------------------------
Message: 1
Date: Fri, 1 Oct 2021 18:17:03 +0530
From: Indranil <myselfindranil@gmail.com>
To: ClamAV users ML <clamav-users@lists.clamav.net>
Cc: "G.W. Haywood" <clamav@jubileegroup.co.uk>
Subject: Re: [clamav-users] Verifying bytecode, phishing and other
type of threats
Message-ID:
<CAAaur943GFReRR3=_r1-fOLU5jZRT0m-zpbbVsH1SFxcwOOoAQ@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Hello Ged,
Thanks for your response. Here is the list of steps:
1. Download: http://www.clamav.net/downloads/production/ClamAV-0.103.0.exe
2. Installed from this exe.
3. Opened windows powershell in admin mode
4. cd "c:\program files\clamav"
5. copy .\conf_examples\freshclam.conf.sample .\freshclam.conf
6. copy .\conf_examples\clamd.conf.sample .\clamd.conf
7. Commented "Example" in freshclam.conf and clamd.conf
8. Started clamd.exe in one powershell window
9. .\clamdscan -m -i 'C:\Users\indra\AppData\' from another powershell window
10. The result is show below
----------- SCAN SUMMARY -----------
Infected files: 0
Total errors: 800 <<<<<<<<<<<<<<<<<<< Please ignore this.
These are all permission denied errors.
Time: 348.501 sec (5 m 48 s)
Start Date: 2021:10:01 17:35:57
End Date: 2021:10:01 17:41:46
I intend to run clamAv on my windows laptop. There is no mail-server
configured here. However, I plan to scan all new file addition in
C:\Users\<user name>\Downloads, C:\Users\<username>\Desktop,
C:\Users\<username>\Documents, C:\Users\<username>\AppData and new
email for all possible threats (including malware, phishing, bytecode
etc).
In all my test runs I am getting clean results. But at present I am
trying to find examples of various threat detection. So eventually
from the threat message I want to
distinguish a malware from phishing from a bytecode etc. Do you have
any suggestion for me to better visualize (via running test scans)
threat detection?
The output from ./clamconf.exe -n shows that I do not have any
non-default configuration
I have changed anything in clamd.conf. Do you have any suggestion for
any non-default configuration which can lead to better security?
Thank you,
Indranil
On Thu, Sep 30, 2021 at 8:36 PM G.W. Haywood via clamav-users
<clamav-users@lists.clamav.net> wrote:
>
> Hi there,
>
> On Thu, 30 Sep 2021, Indranil via clamav-users wrote:
>
> > I have downloaded ClamAV and verified the eicar test using clamscan
> > on a Windows VM.
>
> Please tell us exactly what you downloaded, what you installed it on,
> and exactly how you installed it. Please also describe how you plan
> to approach scanning in general terms - which will probably make it
> easier to answer a number of questions which you haven't yet asked.
>
> > It appears that the following detection capabilities are also
> > enabled by default: bytecode, scan-mail, phishing-sigs,
> > phishing-scan-urls, scan-pe, scan-elf, scan-pdf, scan-html.
>
> The descriptions of capabilities which you have given resemble some
> configuration and scanning options which I recognize, but they are
> unsubtly different. The relevant options (in the configuration files
> on your machine, and given on the command line) are in the official
> ClamAV documentation:
>
> https://docs.clamav.net/
>
> For example there is a 'Bytecode' option in clamd.conf, but there is
> no 'scan-mail' option that I know of - it is 'ScanMail'. Similarly,
> there are 'PhishingSignatures' and 'PhishingScanURLs' options but not
> those that you give, 'phishing-sigs' and 'phishing-scan-urls'. It
> baffles me that people make such gratuitous changes, it must be very
> confusing to newcomers. There are also quite a few other options
> which you have not mentioned. There's a scanning daemon and a thing
> called a 'milter' (another daemon) which is to scan mail on a mail
> server; these are separate subsystems in their own right which may or
> may not be installed and which will need to be separately configured.
> You have not said whether or not you wish to use a daemon, but that is
> one of the more fundamental decisions - see my first paragraph.
>
> > Out of these options, I am able to test scan-pe, scan-elf, scan-pdf
> > and scan-html using respective files.
>
> Please explain exactly what you mean by 'test'.
>
> > I have not been able to test the rest of the options such as
> > bytecode, scan-mail, phishing-sigs and phishing-scan-urls. Could you
> > please help with the method of verifying individual options.
>
> Please read the official documentation at the link which I have given.
> It is much easier to confirm that your configuration is as you wish
> than to test that the code is doing what you might expect. Simply run
>
> clamconf -n
>
> and you should see the differences between your configuration and the
> documented defaults. In any case the expectation is sometimes based
> on wishful thinking, so please let us know what you expect from your
> tests before you ask us to help you with an exercise which is poorly
> defined and might well be open-ended.
>
> > Also, when a threat is detected, does ClamAV report the type of the
> > threat i.e. does ClamAV report that Threat1 is a ?bytecode? threat,
> > Threat2 is a ?phishing-sigs? threat?
>
> In a way it does, yes. It reports a string like "Something FOUND",
> either at the command line or in some log. It may also report other
> information such as how much data was scanned and the scan time, and
> if you wish you can configure verbose logging, and temorary files to
> be retained for later inspection. Be careful because these can use a
> lot of storage space. The ClamAV 'sigtool' utility can help you to
> investigate what was found.
>
> Here are some examples from a mail server log of things "FOUND":
>
> 258 Porcupine.Junk.36046.UNOFFICIAL FOUND
> 312 YARA.Bank_rule.UNOFFICIAL FOUND
> 360 Win.Packed.Ratx-9895842-0 FOUND
> 366 Sanesecurity.Jurlbl.7e72e8.UNOFFICIAL FOUND
> 17353 YARA.Garbage_Spam_0006_Rule.UNOFFICIAL FOUND
>
> Our mail server uses a milter (not the one available from ClamAV) to
> pass incoming mail streams to the 'clamd' scanning daemon and write to
> the logs. I have used OS tools to trawl the logs for September 2021.
> The counts are the number of times that this particular kind of threat
> was found in the incoming mail stream. As you can see, some lines are
> marked as "UNOFFICIAL". This means that the threat was detected by a
> signature from something other than the official ClamAV database. In
> addition to the official signature databases, we use both our own Yara
> rules and a number of third-party databases. These greatly extend the
> usefulness of ClamAV in our situation. At the time of writing, there
> are about 8.8 million signatures in our ClamAV database. Of those,
> 8.6 million are from the 'official' ClamAV databases and the rest are
> 'third party' and our own. There are 583 of our own Yara signatures.
> As you see from the table, by a very large margin a single one of our
> Yara sigs catches more spam than all the rest put together. That's
> probably because we know a lot more about our spam profile than anyone
> else does. ClamAV is by no means a 'fire and forget' munition, please
> be aware that you are (hopefully) embarking on a journey of discovery.
>
> ClamAV does not attempt to repair anything which it finds. It can be
> instructed to remove, move or copy a suspect file. Please read the
> warnings in the documentation and think *very* carefully before doing
> anything like that, because if you aren't careful you will be a bigger
> threat to your systems than the threats from which you are trying to
> protect them.
>
> > If I am scanning C:\Users\Indranil via clamscan (with recursive option)
> > then does ?C:\Users\Indranil\AppData\Local\Microsoft\Outlook? get tested for
> > virus only if ?scan-mail? option is on?
>
> It isn't like that at all. ClamAV contains code which recognizes
> different types of data. For example, it can tell if a file is an
> archive (like a '.zip' file) or if a data stream appears to be a mail
> message. ClamAV treats files and data streams in much the same ways.
> It can and does scan selectively when it detects such things - certain
> signatures only apply to certain kinds of data - which is mainly why I
> have asked you to define 'test'. Some things happen in ways which you
> might describe as 'behind the scenes', and you need to be very careful
> about how you define tests. This probably means that you will need to
> know a lot more about ClamAV's behaviour than you do now in order to
> be able to test it.
>
> HTH
>
> --
>
> 73,
> Ged.
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
------------------------------
Message: 2
Date: Fri, 1 Oct 2021 17:09:37 +0100
From: Max Allan <max.allan@surevine.com>
To: clamav-users@lists.clamav.net
Subject: [clamav-users] Scanning a zip file fails, extract it, scan
with the same options and it passes
Message-ID:
<CADNp1BYedEHeyPY1QEKThnSb6QFhO8z5i3wky_Bqiz0tVh4FFw@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Hi,
I have a requirement (from the business) to AV scan all docker
containers we create.
I started experimenting with tomcat:latest, which is handy because you
can follow along at home easily!
Someone else has already recommended a scan command :
clamscan <file> \
--infected \
--recursive=yes \
--alert-exceeds-max=yes \
--max-recursion=2000000 \
--max-dir-recursion=2000000 \
--max-files=2000000 \
--max-filesize=2000M \
--max-scansize=2000M \
--max-embeddedpe=2000M \
--max-htmlnormalize=2000M \
--max-htmlnotags=2000M \
--max-scriptnormalize=2000M \
--max-ziptypercg=2000M \
--max-partitions=2000000 \
--max-iconspe=2000000 \
--max-rechwp3=2000000 \
--pcre-match-limit=2000000 \
--pcre-recmatch-limit=2000000 \
--pcre-max-filesize=2000M -a
So, if you run the tomcat:latest container, apt update, apt install
clamav, freshclam and run that scan command against
/usr/local/openjdk-11/lib/src.zip you will probably get a failure :
/usr/local/openjdk-11/lib/src.zip: Heuristics.Limits.Exceeded FOUND
/usr/local/openjdk-11/lib/src.zip!(0)ZIP:jdk.zipfs/jdk/nio/zipfs/ZipInfo.java:
Heuristics.Limits.Exceeded FOUND
---------- SCAN SUMMARY -----------
Known viruses: 8570214
Engine version: 0.103.3
Scanned directories: 0
Scanned files: 1
Infected files: 1
Data scanned: 290.07 MB
Data read: 55.52 MB (ratio 5.22:1)
Time: 260.438 sec (4 m 20 s)
Start Date: 2021:10:01 13:39:47
End Date: 2021:10:01 13:44:07
However, if I extract that zip file to /src and then run clamscan on
/src then it passes without a problem :
----------- SCAN SUMMARY -----------
Known viruses: 8570214
Engine version: 0.103.3
Scanned directories: 2076
Scanned files: 18415
Infected files: 0
Data scanned: 333.04 MB
Data read: 170.92 MB (ratio 1.95:1)
Time: 320.573 sec (5 m 20 s)
Start Date: 2021:10:01 13:23:39
End Date: 2021:10:01 13:29:00
(There are indeed 18415 files in that .zip according to unzip -l)
Or even scan the single file :
clamscan ZipInfo.java --infected --recursive=yes
--alert-exceeds-max=yes --max-recursion=2000000
--max-dir-recursion=2000000 --max-files=2000000
--max-filesize=2000M --max-scansize=2000M --max-embeddedpe=2000M
--max-htmlnormalize=2000M --max-htmlnotags=2000M
--max-scriptnormalize=2000M --max-ziptypercg=2000M
--max-partitions=2000000 --max-iconspe=2000000
--max-rechwp3=2000000 --pcre-match-limit=2000000
--pcre-recmatch-limit=2000000 --pcre-max-filesize=2000M -a
----------- SCAN SUMMARY -----------
Known viruses: 8570214
Engine version: 0.103.3
Scanned directories: 0
Scanned files: 1
Infected files: 0
Data scanned: 0.01 MB
Data read: 0.01 MB (ratio 1.50:1)
Time: 68.326 sec (1 m 8 s)
Start Date: 2021:10:01 16:03:14
End Date: 2021:10:01 16:04:22
Clearly the content of src.zip (ZipInfo.java) IS scannable, when
extracted, but for some reason not scannable when it is in a zip
file... Is this a bug? Or am I specifying some options that are
causing it??
(clamscan -V
ClamAV 0.103.3/26309/Fri Oct 1 09:03:53 2021 )
------------------------------
Message: 3
Date: Fri, 1 Oct 2021 17:24:40 +0100 (BST)
From: "G.W. Haywood" <clamav@jubileegroup.co.uk>
To: Indranil via clamav-users <clamav-users@lists.clamav.net>
Subject: Re: [clamav-users] Verifying bytecode, phishing and other
type of threats
Message-ID: <bcdaf3f5-1948-b3b9-511-c35defe47e6@jubileegroup.co.uk>
Content-Type: text/plain; format=flowed; charset=US-ASCII
Hi there,
On Fri, 1 Oct 2021, Indranil via clamav-users wrote:
> ...
> 9. .\clamdscan -m -i 'C:\Users\indra\AppData\'
The '-i' command-line option is specifically for 'clamscan'. The
'clamdscan' utility will ignore it.
> ... permission denied errors.
Obviously if you want ClamAV to scan something, either it has to have
permission to read it or something else must read it and pass the data
to the ClamAV scanning process. Here, for example, we run a separate
clamd server, which scans everything passed to it over the LAN by the
mail servers. If something manages to compromise the clamd server it
isn't a big deal because it can't access anything else on the network.
By the way in more than eighteen years of using ClamAV, that's never
happened here but it's a possibility you always need to consider.
> ... I plan to scan all new file addition in
> C:\Users\<user name>\Downloads, C:\Users\<username>\Desktop,
> C:\Users\<username>\Documents, C:\Users\<username>\AppData and new
> email for all possible threats ...
> ...
> In all my test runs I am getting clean results. But at present I am
> trying to find examples of various threat detection.
You mean sample of malware etc.?
> So eventually from the threat message I want to distinguish a
> malware from phishing from a bytecode etc. Do you have any
> suggestion for me to better visualize (via running test scans)
> threat detection?
When you see something FOUND, either on the command line or in a log,
you can ask ClamAV's 'sigtool' utility to tell you exactly what in the
data caused the report by passing the FOUND string to it. That will
show you exactly what the signature was looking for. Other than that
I'm not sure exactly what you want to see. Most people I think set up
periodic scans, and look in their logs from time to time. My personal
view is that this is asking for trouble, and I don't scan systems at
all. I only scan incoming data, which seems to be the sort of thing
which you're planning to do. In our case the incoming data is in mail
and it's scanned automatically by mail filters. If anything triggers
detection there then it's stopped in its tracks at that point. Either
it's automatically reported (because the server has seen sufficiently
similar abuse to recognize it without any doubt) and sent to /archive,
or someone (generally me) will see something brightly coloured on the
Web interface and then can look at the situation and take a view. We
use a purpose-built Web interface to the mail systems. This avoids a
huge amount of effort which would otherwise be needed run suspect data
past e.g. Jotti's malware scan, scan logs, report abuse etc. - but (1)
it's only any use if you run mail servers and (2) I don't see how the
effort could be justified for ad-hoc scans of a single Windows box.
> The output from ./clamconf.exe -n shows that I do not have any
> non-default configuration
>
> I have changed anything in clamd.conf. Do you have any suggestion for
> any non-default configuration which can lead to better security?
I take it you mean you have *not* changed anything in clamd.conf - or
at least nothing except commenting the 'Example' line?
To me, the most obvious configuration change to improve security would
be to use any third-party databases that you can get hold of. You can
tweak things in clamd.conf but you need to be careful. It's possible
to cause ClamAV to consume excessive resources and you might crash the
system; that, if you're unlucky, could mean you'd need to spend a lot
of time on recovering it. Running a separate ClamAV server has much
to recommend it.
You haven't asked about other ways of improving security, and to some
extent this list isn't really appropriate for discussing many of them,
but I would strongly advise you not to rely on ClamAV as your only way
of defending against threats of all kinds. If you do, then the game
is most probably already over. In the past I have posted my estimates
of detection rates to this list, you may want to look them over. It's
far more important to maintain good network hygiene than it is to run
a virus scanner. There are vastly more threats against Windows boxes
than there are against other systems - probably more than all threats
of all kinds against all other systems put together. That's one of
the main reasons I don't routinely run any Windows boxes.
--
73,
Ged.
------------------------------
Message: 4
Date: Fri, 01 Oct 2021 17:04:20 +0000
From: "Laurent S." <110ef9e3086d8405c2929e34be5b4340@protonmail.ch>
To: clamav-users@lists.clamav.net
Subject: Re: [clamav-users] Scanning a zip file fails, extract it,
scan with the same options and it passes
Message-ID: <a8352b91-0d45-0ae3-fcb7-2daf14f2ca72@protonmail.ch>
Content-Type: text/plain; charset="utf-8"; Format="flowed"
Dear Max Allan,
Heuristics.Limits.Exceeded doesn't mean the file is infected, but it's
only a warning telling you that something went above the limits you set.
It give the warning this way because of --alert-exceeds-max=yes
ClamAV managed to go search into those files in each case as you can see
from the scan summaries. It will count the zip as a single file.
I would recommend against copy-pasting all those parameters without
having given proper thought into what you are doing.
Best regards,
Laurent
On 01.10.21 18:09, Max Allan via clamav-users wrote:
> Hi,
> I have a requirement (from the business) to AV scan all docker
> containers we create.
> I started experimenting with tomcat:latest, which is handy because you
> can follow along at home easily!
> Someone else has already recommended a scan command :
>
> clamscan <file> \
> --infected \
> --recursive=yes \
> --alert-exceeds-max=yes \
> --max-recursion=2000000 \
> --max-dir-recur
sion=2000000 \
> --max-files=2000000 \
> --max-filesize=2000M \
> --max-scansize=2000M \
> --max-embeddedpe=2000M \
> --max-htmlnormalize=2000M \
> --max-htmlnotags=2000M \
> --max-scriptnormalize=2000M \
> --max-ziptypercg=2000M \
> --max-partitions=2000000 \
> --max-iconspe=2000000 \
> --max-rechwp3=2000000 \
> --pcre-match-limit=2000000 \
> --pcre-recmatch-limit=2000000 \
> --pcre-max-filesize=2000M -a
>
> So, if you run the tomcat:latest container, apt update, apt install
> clamav, freshclam and run that scan command against
> /usr/local/openjdk-11/lib/src.zip you will probably get a failure :
>
> /usr/local/openjdk-11/lib/src.zip: Heuristics.Limits.Exceeded FOUND
> /usr/local/openjdk-11/lib/src.zip!(0)ZIP:jdk.zipfs/jdk/nio/zipfs/ZipInfo.java:
> Heuristics.Limits.Exceeded FOUND
> ---------- SCAN SUMMARY -----------
> Known viruses: 8570214
> Engine version: 0.103.3
> Scanned directories: 0
> Scanned files: 1
>
Infected files: 1
> Data scanned: 290.07 MB
> Data read: 55.52 MB (ratio 5.22:1)
> Time: 260.438 sec (4 m 20 s)
> Start Date: 2021:10:01 13:39:47
> End Date: 2021:10:01 13:44:07
>
>
> However, if I extract that zip file to /src and then run clamscan on
> /src then it passes without a problem :
>
> ----------- SCAN SUMMARY -----------
> Known viruses: 8570214
> Engine version: 0.103.3
> Scanned directories: 2076
> Scanned files: 18415
> Infected files: 0
> Data scanned: 333.04 MB
> Data read: 170.92 MB (ratio 1.95:1)
> Time: 320.573 sec (5 m 20 s)
> Start Date: 2021:10:01 13:23:39
> End Date: 2021:10:01 13:29:00
>
> (There are indeed 18415 files in that .zip according to unzip -l)
>
> Or even scan the single file :
>
> clamscan ZipInfo.java --infected --recursive=yes
> --alert-exceeds-max=yes --max-recursion=2000000
> --max-dir-recursion=2000000 --max-files=2000000
> --max-filesize=2000M --max-scansize=2000M --max-embeddedpe=2000M
> -
-max-htmlnormalize=2000M --max-htmlnotags=2000M
> --max-scriptnormalize=2000M --max-ziptypercg=2000M
> --max-partitions=2000000 --max-iconspe=2000000
> --max-rechwp3=2000000 --pcre-match-limit=2000000
> --pcre-recmatch-limit=2000000 --pcre-max-filesize=2000M -a
>
> ----------- SCAN SUMMARY -----------
> Known viruses: 8570214
> Engine version: 0.103.3
> Scanned directories: 0
> Scanned files: 1
> Infected files: 0
> Data scanned: 0.01 MB
> Data read: 0.01 MB (ratio 1.50:1)
> Time: 68.326 sec (1 m 8 s)
> Start Date: 2021:10:01 16:03:14
> End Date: 2021:10:01 16:04:22
>
>
>
> Clearly the content of src.zip (ZipInfo.java) IS scannable, when
> extracted, but for some reason not scannable when it is in a zip
> file... Is this a bug? Or am I specifying some options that are
> causing it??
>
> (clamscan -V
> ClamAV 0.103.3/26309/Fri Oct 1 09:03:53 2021 )
>
> _______________________________________________
>
> clamav-users mailing list
> clam
av-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: publickey - 110ef9e3086d8405c2929e34be5b4340@protonmail.ch - d23fa0a8.asc
Type: application/pgp-keys
Size: 3204 bytes
Desc: not available
URL: <https://lists.clamav.net/pipermail/clamav-users/attachments/20211001/ff80df95/attachment-0001.key>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 855 bytes
Desc: OpenPGP digital signature
URL: <https://lists.clamav.net/pipermail/clamav-users/attachments/20211001/ff80df95/attachment-0001.sig>
------------------------------
Subject: Digest Footer
_______________________________________________
clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users
https://github.com/vrtadmin/clamav-faq
http://www.clamav.net/contact.html#ml
------------------------------
End of clamav-users Digest, Vol 203, Issue 2
********************************************