Mailing List Archive

[clamav-users] Verifying bytecode, phishing and other type of threats
Hello ClamAV folks,

I have downloaded ClamAV and verified
the eicar test using clamscan on a Windows VM. It appears that the
following detection capabilities are also enabled by default:
bytecode, scan-mail, phishing-sigs, phishing-scan-urls, scan-pe,
scan-elf, scan-pdf, scan-html. Out of these options, I am able to test
scan-pe, scan-elf, scan-pdf and scan-html using respective files. I
have not been able to test the rest of the options such as bytecode,
scan-mail, phishing-sigs and phishing-scan-urls. Could you please help
with the method of verifying individual options. Also, when a threat
is detected, does ClamAV report the type of the threat i.e. does
ClamAV report that Threat1 is a ‘bytecode’ threat, Threat2 is a
‘phishing-sigs’ threat?

If I am scanning C:\Users\Indranil via
clamscan (with recursive option) then does
‘C:\Users\Indranil\AppData\Local\Microsoft\Outlook’ get tested for
virus only if ‘scan-mail’ option is on?

Thank you,
Indranil

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Verifying bytecode, phishing and other type of threats [ In reply to ]
Hi there,

On Thu, 30 Sep 2021, Indranil via clamav-users wrote:

> I have downloaded ClamAV and verified the eicar test using clamscan
> on a Windows VM.

Please tell us exactly what you downloaded, what you installed it on,
and exactly how you installed it. Please also describe how you plan
to approach scanning in general terms - which will probably make it
easier to answer a number of questions which you haven't yet asked.

> It appears that the following detection capabilities are also
> enabled by default: bytecode, scan-mail, phishing-sigs,
> phishing-scan-urls, scan-pe, scan-elf, scan-pdf, scan-html.

The descriptions of capabilities which you have given resemble some
configuration and scanning options which I recognize, but they are
unsubtly different. The relevant options (in the configuration files
on your machine, and given on the command line) are in the official
ClamAV documentation:

https://docs.clamav.net/

For example there is a 'Bytecode' option in clamd.conf, but there is
no 'scan-mail' option that I know of - it is 'ScanMail'. Similarly,
there are 'PhishingSignatures' and 'PhishingScanURLs' options but not
those that you give, 'phishing-sigs' and 'phishing-scan-urls'. It
baffles me that people make such gratuitous changes, it must be very
confusing to newcomers. There are also quite a few other options
which you have not mentioned. There's a scanning daemon and a thing
called a 'milter' (another daemon) which is to scan mail on a mail
server; these are separate subsystems in their own right which may or
may not be installed and which will need to be separately configured.
You have not said whether or not you wish to use a daemon, but that is
one of the more fundamental decisions - see my first paragraph.

> Out of these options, I am able to test scan-pe, scan-elf, scan-pdf
> and scan-html using respective files.

Please explain exactly what you mean by 'test'.

> I have not been able to test the rest of the options such as
> bytecode, scan-mail, phishing-sigs and phishing-scan-urls. Could you
> please help with the method of verifying individual options.

Please read the official documentation at the link which I have given.
It is much easier to confirm that your configuration is as you wish
than to test that the code is doing what you might expect. Simply run

clamconf -n

and you should see the differences between your configuration and the
documented defaults. In any case the expectation is sometimes based
on wishful thinking, so please let us know what you expect from your
tests before you ask us to help you with an exercise which is poorly
defined and might well be open-ended.

> Also, when a threat is detected, does ClamAV report the type of the
> threat i.e. does ClamAV report that Threat1 is a ?bytecode? threat,
> Threat2 is a ?phishing-sigs? threat?

In a way it does, yes. It reports a string like "Something FOUND",
either at the command line or in some log. It may also report other
information such as how much data was scanned and the scan time, and
if you wish you can configure verbose logging, and temorary files to
be retained for later inspection. Be careful because these can use a
lot of storage space. The ClamAV 'sigtool' utility can help you to
investigate what was found.

Here are some examples from a mail server log of things "FOUND":

258 Porcupine.Junk.36046.UNOFFICIAL FOUND
312 YARA.Bank_rule.UNOFFICIAL FOUND
360 Win.Packed.Ratx-9895842-0 FOUND
366 Sanesecurity.Jurlbl.7e72e8.UNOFFICIAL FOUND
17353 YARA.Garbage_Spam_0006_Rule.UNOFFICIAL FOUND

Our mail server uses a milter (not the one available from ClamAV) to
pass incoming mail streams to the 'clamd' scanning daemon and write to
the logs. I have used OS tools to trawl the logs for September 2021.
The counts are the number of times that this particular kind of threat
was found in the incoming mail stream. As you can see, some lines are
marked as "UNOFFICIAL". This means that the threat was detected by a
signature from something other than the official ClamAV database. In
addition to the official signature databases, we use both our own Yara
rules and a number of third-party databases. These greatly extend the
usefulness of ClamAV in our situation. At the time of writing, there
are about 8.8 million signatures in our ClamAV database. Of those,
8.6 million are from the 'official' ClamAV databases and the rest are
'third party' and our own. There are 583 of our own Yara signatures.
As you see from the table, by a very large margin a single one of our
Yara sigs catches more spam than all the rest put together. That's
probably because we know a lot more about our spam profile than anyone
else does. ClamAV is by no means a 'fire and forget' munition, please
be aware that you are (hopefully) embarking on a journey of discovery.

ClamAV does not attempt to repair anything which it finds. It can be
instructed to remove, move or copy a suspect file. Please read the
warnings in the documentation and think *very* carefully before doing
anything like that, because if you aren't careful you will be a bigger
threat to your systems than the threats from which you are trying to
protect them.

> If I am scanning C:\Users\Indranil via clamscan (with recursive option)
> then does ?C:\Users\Indranil\AppData\Local\Microsoft\Outlook? get tested for
> virus only if ?scan-mail? option is on?

It isn't like that at all. ClamAV contains code which recognizes
different types of data. For example, it can tell if a file is an
archive (like a '.zip' file) or if a data stream appears to be a mail
message. ClamAV treats files and data streams in much the same ways.
It can and does scan selectively when it detects such things - certain
signatures only apply to certain kinds of data - which is mainly why I
have asked you to define 'test'. Some things happen in ways which you
might describe as 'behind the scenes', and you need to be very careful
about how you define tests. This probably means that you will need to
know a lot more about ClamAV's behaviour than you do now in order to
be able to test it.

HTH

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Verifying bytecode, phishing and other type of threats [ In reply to ]
Hello Ged,

Thanks for your response. Here is the list of steps:

1. Download: http://www.clamav.net/downloads/production/ClamAV-0.103.0.exe
2. Installed from this exe.
3. Opened windows powershell in admin mode
4. cd "c:\program files\clamav"
5. copy .\conf_examples\freshclam.conf.sample .\freshclam.conf
6. copy .\conf_examples\clamd.conf.sample .\clamd.conf
7. Commented "Example" in freshclam.conf and clamd.conf
8. Started clamd.exe in one powershell window
9. .\clamdscan -m -i 'C:\Users\indra\AppData\' from another powershell window
10. The result is show below
----------- SCAN SUMMARY -----------
Infected files: 0
Total errors: 800 <<<<<<<<<<<<<<<<<<< Please ignore this.
These are all permission denied errors.
Time: 348.501 sec (5 m 48 s)
Start Date: 2021:10:01 17:35:57
End Date: 2021:10:01 17:41:46

I intend to run clamAv on my windows laptop. There is no mail-server
configured here. However, I plan to scan all new file addition in
C:\Users\<user name>\Downloads, C:\Users\<username>\Desktop,
C:\Users\<username>\Documents, C:\Users\<username>\AppData and new
email for all possible threats (including malware, phishing, bytecode
etc).

In all my test runs I am getting clean results. But at present I am
trying to find examples of various threat detection. So eventually
from the threat message I want to
distinguish a malware from phishing from a bytecode etc. Do you have
any suggestion for me to better visualize (via running test scans)
threat detection?

The output from ./clamconf.exe -n shows that I do not have any
non-default configuration

I have changed anything in clamd.conf. Do you have any suggestion for
any non-default configuration which can lead to better security?

Thank you,
Indranil


On Thu, Sep 30, 2021 at 8:36 PM G.W. Haywood via clamav-users
<clamav-users@lists.clamav.net> wrote:
>
> Hi there,
>
> On Thu, 30 Sep 2021, Indranil via clamav-users wrote:
>
> > I have downloaded ClamAV and verified the eicar test using clamscan
> > on a Windows VM.
>
> Please tell us exactly what you downloaded, what you installed it on,
> and exactly how you installed it. Please also describe how you plan
> to approach scanning in general terms - which will probably make it
> easier to answer a number of questions which you haven't yet asked.
>
> > It appears that the following detection capabilities are also
> > enabled by default: bytecode, scan-mail, phishing-sigs,
> > phishing-scan-urls, scan-pe, scan-elf, scan-pdf, scan-html.
>
> The descriptions of capabilities which you have given resemble some
> configuration and scanning options which I recognize, but they are
> unsubtly different. The relevant options (in the configuration files
> on your machine, and given on the command line) are in the official
> ClamAV documentation:
>
> https://docs.clamav.net/
>
> For example there is a 'Bytecode' option in clamd.conf, but there is
> no 'scan-mail' option that I know of - it is 'ScanMail'. Similarly,
> there are 'PhishingSignatures' and 'PhishingScanURLs' options but not
> those that you give, 'phishing-sigs' and 'phishing-scan-urls'. It
> baffles me that people make such gratuitous changes, it must be very
> confusing to newcomers. There are also quite a few other options
> which you have not mentioned. There's a scanning daemon and a thing
> called a 'milter' (another daemon) which is to scan mail on a mail
> server; these are separate subsystems in their own right which may or
> may not be installed and which will need to be separately configured.
> You have not said whether or not you wish to use a daemon, but that is
> one of the more fundamental decisions - see my first paragraph.
>
> > Out of these options, I am able to test scan-pe, scan-elf, scan-pdf
> > and scan-html using respective files.
>
> Please explain exactly what you mean by 'test'.
>
> > I have not been able to test the rest of the options such as
> > bytecode, scan-mail, phishing-sigs and phishing-scan-urls. Could you
> > please help with the method of verifying individual options.
>
> Please read the official documentation at the link which I have given.
> It is much easier to confirm that your configuration is as you wish
> than to test that the code is doing what you might expect. Simply run
>
> clamconf -n
>
> and you should see the differences between your configuration and the
> documented defaults. In any case the expectation is sometimes based
> on wishful thinking, so please let us know what you expect from your
> tests before you ask us to help you with an exercise which is poorly
> defined and might well be open-ended.
>
> > Also, when a threat is detected, does ClamAV report the type of the
> > threat i.e. does ClamAV report that Threat1 is a ‘bytecode’ threat,
> > Threat2 is a ‘phishing-sigs’ threat?
>
> In a way it does, yes. It reports a string like "Something FOUND",
> either at the command line or in some log. It may also report other
> information such as how much data was scanned and the scan time, and
> if you wish you can configure verbose logging, and temorary files to
> be retained for later inspection. Be careful because these can use a
> lot of storage space. The ClamAV 'sigtool' utility can help you to
> investigate what was found.
>
> Here are some examples from a mail server log of things "FOUND":
>
> 258 Porcupine.Junk.36046.UNOFFICIAL FOUND
> 312 YARA.Bank_rule.UNOFFICIAL FOUND
> 360 Win.Packed.Ratx-9895842-0 FOUND
> 366 Sanesecurity.Jurlbl.7e72e8.UNOFFICIAL FOUND
> 17353 YARA.Garbage_Spam_0006_Rule.UNOFFICIAL FOUND
>
> Our mail server uses a milter (not the one available from ClamAV) to
> pass incoming mail streams to the 'clamd' scanning daemon and write to
> the logs. I have used OS tools to trawl the logs for September 2021.
> The counts are the number of times that this particular kind of threat
> was found in the incoming mail stream. As you can see, some lines are
> marked as "UNOFFICIAL". This means that the threat was detected by a
> signature from something other than the official ClamAV database. In
> addition to the official signature databases, we use both our own Yara
> rules and a number of third-party databases. These greatly extend the
> usefulness of ClamAV in our situation. At the time of writing, there
> are about 8.8 million signatures in our ClamAV database. Of those,
> 8.6 million are from the 'official' ClamAV databases and the rest are
> 'third party' and our own. There are 583 of our own Yara signatures.
> As you see from the table, by a very large margin a single one of our
> Yara sigs catches more spam than all the rest put together. That's
> probably because we know a lot more about our spam profile than anyone
> else does. ClamAV is by no means a 'fire and forget' munition, please
> be aware that you are (hopefully) embarking on a journey of discovery.
>
> ClamAV does not attempt to repair anything which it finds. It can be
> instructed to remove, move or copy a suspect file. Please read the
> warnings in the documentation and think *very* carefully before doing
> anything like that, because if you aren't careful you will be a bigger
> threat to your systems than the threats from which you are trying to
> protect them.
>
> > If I am scanning C:\Users\Indranil via clamscan (with recursive option)
> > then does ‘C:\Users\Indranil\AppData\Local\Microsoft\Outlook’ get tested for
> > virus only if ‘scan-mail’ option is on?
>
> It isn't like that at all. ClamAV contains code which recognizes
> different types of data. For example, it can tell if a file is an
> archive (like a '.zip' file) or if a data stream appears to be a mail
> message. ClamAV treats files and data streams in much the same ways.
> It can and does scan selectively when it detects such things - certain
> signatures only apply to certain kinds of data - which is mainly why I
> have asked you to define 'test'. Some things happen in ways which you
> might describe as 'behind the scenes', and you need to be very careful
> about how you define tests. This probably means that you will need to
> know a lot more about ClamAV's behaviour than you do now in order to
> be able to test it.
>
> HTH
>
> --
>
> 73,
> Ged.
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Verifying bytecode, phishing and other type of threats [ In reply to ]
Hi there,

On Fri, 1 Oct 2021, Indranil via clamav-users wrote:

> ...
> 9. .\clamdscan -m -i 'C:\Users\indra\AppData\'

The '-i' command-line option is specifically for 'clamscan'. The
'clamdscan' utility will ignore it.

> ... permission denied errors.

Obviously if you want ClamAV to scan something, either it has to have
permission to read it or something else must read it and pass the data
to the ClamAV scanning process. Here, for example, we run a separate
clamd server, which scans everything passed to it over the LAN by the
mail servers. If something manages to compromise the clamd server it
isn't a big deal because it can't access anything else on the network.
By the way in more than eighteen years of using ClamAV, that's never
happened here but it's a possibility you always need to consider.

> ... I plan to scan all new file addition in
> C:\Users\<user name>\Downloads, C:\Users\<username>\Desktop,
> C:\Users\<username>\Documents, C:\Users\<username>\AppData and new
> email for all possible threats ...
> ...
> In all my test runs I am getting clean results. But at present I am
> trying to find examples of various threat detection.

You mean sample of malware etc.?

> So eventually from the threat message I want to distinguish a
> malware from phishing from a bytecode etc. Do you have any
> suggestion for me to better visualize (via running test scans)
> threat detection?

When you see something FOUND, either on the command line or in a log,
you can ask ClamAV's 'sigtool' utility to tell you exactly what in the
data caused the report by passing the FOUND string to it. That will
show you exactly what the signature was looking for. Other than that
I'm not sure exactly what you want to see. Most people I think set up
periodic scans, and look in their logs from time to time. My personal
view is that this is asking for trouble, and I don't scan systems at
all. I only scan incoming data, which seems to be the sort of thing
which you're planning to do. In our case the incoming data is in mail
and it's scanned automatically by mail filters. If anything triggers
detection there then it's stopped in its tracks at that point. Either
it's automatically reported (because the server has seen sufficiently
similar abuse to recognize it without any doubt) and sent to /archive,
or someone (generally me) will see something brightly coloured on the
Web interface and then can look at the situation and take a view. We
use a purpose-built Web interface to the mail systems. This avoids a
huge amount of effort which would otherwise be needed run suspect data
past e.g. Jotti's malware scan, scan logs, report abuse etc. - but (1)
it's only any use if you run mail servers and (2) I don't see how the
effort could be justified for ad-hoc scans of a single Windows box.

> The output from ./clamconf.exe -n shows that I do not have any
> non-default configuration
>
> I have changed anything in clamd.conf. Do you have any suggestion for
> any non-default configuration which can lead to better security?

I take it you mean you have *not* changed anything in clamd.conf - or
at least nothing except commenting the 'Example' line?

To me, the most obvious configuration change to improve security would
be to use any third-party databases that you can get hold of. You can
tweak things in clamd.conf but you need to be careful. It's possible
to cause ClamAV to consume excessive resources and you might crash the
system; that, if you're unlucky, could mean you'd need to spend a lot
of time on recovering it. Running a separate ClamAV server has much
to recommend it.

You haven't asked about other ways of improving security, and to some
extent this list isn't really appropriate for discussing many of them,
but I would strongly advise you not to rely on ClamAV as your only way
of defending against threats of all kinds. If you do, then the game
is most probably already over. In the past I have posted my estimates
of detection rates to this list, you may want to look them over. It's
far more important to maintain good network hygiene than it is to run
a virus scanner. There are vastly more threats against Windows boxes
than there are against other systems - probably more than all threats
of all kinds against all other systems put together. That's one of
the main reasons I don't routinely run any Windows boxes.

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml