Mailing List Archive

[clamav-users] Clamd non recursive MULTISCAN?
Hi,

I'm trying to scan the files under c:\windows\system32 but excluding
folders.
I made a list of files and sent it to clamd using a session and SCAN
command for every file, the whole process takes about 4 minutes (4200
files).

If I send the command "MULTISCAN c:\windows\system32", the recursive
scanning will scan everything including folders, which is not what I want.

This scanning must be processed using multi threads, but I cannot use a
session + MULTISCAN + each individual file, MULTISCAN command is not
allowed under clamd session.

Adding explicit exclusion for the folders inside c:\windows\system32 in
clamd.conf is not an option.

Is there a way to perform a non recursive MULTISCAN, or is there a way to
perform a multithread SCAN command?

Thank you very much!
Re: [clamav-users] Clamd non recursive MULTISCAN? [ In reply to ]
Hi there,

On Fri, 4 Mar 2022, Jorge Elissalde via clamav-users wrote:

> ...
> If I send the command "MULTISCAN c:\windows\system32", the recursive
> scanning will scan everything including folders, which is not what I want.
>
> This scanning must be processed using multi threads ...

Can you be more forthcoming about your requirements?

> Is there a way to perform a non recursive MULTISCAN, or is there a
> way to perform a multithread SCAN command?

To do exactly what you want I think you'll need to create your own
scanning daemon using libclamav to do the actual scanning:

.../docs/html/UserManual/libclamav.html

But before diving into something like that I would suggest first of
all that you make sure the scanner will perform as you expect.

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Clamd non recursive MULTISCAN? [ In reply to ]
Thank you for your answer.
Trying to be more forthcoming I can explain the code I'm making.

- I get the full list of files under c:\windows\system32 folder, just
files, not folders (4913 files in my case).
- I send every file name to clamd using the SCAN command.

The whole process takes almost 5 minutes, and I'm trying to drastically
reduce that time.
The problems I have are:

- I cannot use MULTISCAN command, if I send the command MULTISCAN
c:\windows\system32 everything will be scanned including folders.
- I cannot use MULTISCAN command under a clamd session, clamd does not
allow it (with this option I could send every filename using MULTISCAN
command under a session)
- SCAN is not multithread and this is the reason for the 5 required minutes

Possible solutions are:

- Non recursive MULTISCAN ( seems that this is not available )
- Multithread SCAN command ( seems that is not available )

Is there another solution here?
Thank you very much!






El vie, 4 mar 2022 a las 15:53, G.W. Haywood via clamav-users (<
clamav-users@lists.clamav.net>) escribió:

> Hi there,
>
> On Fri, 4 Mar 2022, Jorge Elissalde via clamav-users wrote:
>
> > ...
> > If I send the command "MULTISCAN c:\windows\system32", the recursive
> > scanning will scan everything including folders, which is not what I
> want.
> >
> > This scanning must be processed using multi threads ...
>
> Can you be more forthcoming about your requirements?
>
> > Is there a way to perform a non recursive MULTISCAN, or is there a
> > way to perform a multithread SCAN command?
>
> To do exactly what you want I think you'll need to create your own
> scanning daemon using libclamav to do the actual scanning:
>
> .../docs/html/UserManual/libclamav.html
>
> But before diving into something like that I would suggest first of
> all that you make sure the scanner will perform as you expect.
>
> --
>
> 73,
> Ged.
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>
Re: [clamav-users] Clamd non recursive MULTISCAN? [ In reply to ]
Thank you for your answer.
Trying to be more forthcoming I can explain the code I'm making.

- I get the full list of files under c:\windows\system32 folder, just
files, not folders (4913 files in my case).
- I send every file name to clamd using the SCAN command.

The whole process takes almost 5 minutes, and I'm trying to drastically
reduce that time.
The problems I have are:

- I cannot use MULTISCAN command, if I send the command MULTISCAN
c:\windows\system32 everything will be scanned including folders.
- I cannot use MULTISCAN command under a clamd session, clamd does not
allow it (with this option I could send every filename using MULTISCAN
command under a session)
- SCAN is not multithread and this is the reason for the 5 required minutes

Possible solutions are:

- Non recursive MULTISCAN ( seems that this is not available )
- Multithread SCAN command ( seems that is not available )

Is there another solution here?
Thank you very much!


El vie, 4 mar 2022 a las 15:53, G.W. Haywood via clamav-users (<
clamav-users@lists.clamav.net>) escribió:

> Hi there,
>
> On Fri, 4 Mar 2022, Jorge Elissalde via clamav-users wrote:
>
> > ...
> > If I send the command "MULTISCAN c:\windows\system32", the recursive
> > scanning will scan everything including folders, which is not what I
> want.
> >
> > This scanning must be processed using multi threads ...
>
> Can you be more forthcoming about your requirements?
>
> > Is there a way to perform a non recursive MULTISCAN, or is there a
> > way to perform a multithread SCAN command?
>
> To do exactly what you want I think you'll need to create your own
> scanning daemon using libclamav to do the actual scanning:
>
> .../docs/html/UserManual/libclamav.html
>
> But before diving into something like that I would suggest first of
> all that you make sure the scanner will perform as you expect.
>
> --
>
> 73,
> Ged.
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>
Re: [clamav-users] Clamd non recursive MULTISCAN? [ In reply to ]
Hi there,

On Fri, 4 Mar 2022, Jorge Elissalde via clamav-users wrote:

> ...
> Trying to be more forthcoming I can explain the code I'm making.
>
> - I get the full list of files under c:\windows\system32 folder, just
> files, not folders (4913 files in my case).
> - I send every file name to clamd using the SCAN command.
>
> The whole process takes almost 5 minutes, and I'm trying to drastically
> reduce that time.

Firstly, I'm not sure that you aren't just running around in circles.

I've seen scans which took hours fail to find threats which were known
to be present. Please keep that in mind. You're scanning for threats
using code which is running on the threatened machine. If I wanted to
attack that machine, the first thing I'd do when I gained access would
be to nobble any virus scanner so that it claimed to find nothing even
when it found something. If some malicious actor was able to modify
files in the ...\system32 directory then that same actor probably also
had the ability to nobble the scanner. Please keep that in mind too.

The reason it takes five minutes to scan your files is basically that
you are scanning something like five thousand files (note: you haven't
mentioned the average size of those files) for something in the region
of ten million threats, likely using modest, general purpose hardware.

If I may take liberties with units, the number of file.threats is of
the order of fifty billion. The average time per file.threat scan is
300/50000000000 or approximately six nanoseconds. That's quite a bit
less than the access time for your RAM (never mind the mass storage!)
and while this is like measuring a piece of string with a theodolite,
typical 21st century CPUs won't be able to execute more than a very
few instructions in that time even if they take a running jump at it.
I shouldn't be surprised if millions of instructions would need to be
executed to scan a single file for a single threat. So I think that,
under the circumstances, ClamAV doesn't do too bad a job.

What are the specifications for your harware?

Why are you scanning the things you're scanning?

Why are you using the scanned device to scan itself?

Why are you not scanning the things you aren't scanning?

How long would you expect your scan to take on your hardware?

What is your target scan time?

Why?

What do you expect the results to tell you?

What do you plan to do when you have them?

How long will *that* take?

How long does your coffee machine take to make a brew?

Can you compare/contrast the various times in your answers?

Please be much more forthcoming.

> Possible solutions are:
>
> - Non recursive MULTISCAN ( seems that this is not available )
> - Multithread SCAN command ( seems that is not available )
>
> Is there another solution here?

According to the (old) ClamAV Bugzilla there was an issue with
MULTISCAN which was fixed years ago:

https://bugzilla.clamav.net/show_bug.cgi?id=1869

I didn't find the issue mentioned here:

https://github.com/Cisco-Talos/clamav/issues/

but although that's where things are being done now, many old Bugzilla
issues haven't been moved to Github.

If you can produce a test case to confirm that Micah's 2018 comment on
Bugzilla is wrong, you should file a report on Github.

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Clamd non recursive MULTISCAN? [ In reply to ]
Ok, thank you.

El sáb, 5 mar 2022 a las 8:48, G.W. Haywood via clamav-users (<
clamav-users@lists.clamav.net>) escribió:

> Hi there,
>
> On Fri, 4 Mar 2022, Jorge Elissalde via clamav-users wrote:
>
> > ...
> > Trying to be more forthcoming I can explain the code I'm making.
> >
> > - I get the full list of files under c:\windows\system32 folder, just
> > files, not folders (4913 files in my case).
> > - I send every file name to clamd using the SCAN command.
> >
> > The whole process takes almost 5 minutes, and I'm trying to drastically
> > reduce that time.
>
> Firstly, I'm not sure that you aren't just running around in circles.
>
> I've seen scans which took hours fail to find threats which were known
> to be present. Please keep that in mind. You're scanning for threats
> using code which is running on the threatened machine. If I wanted to
> attack that machine, the first thing I'd do when I gained access would
> be to nobble any virus scanner so that it claimed to find nothing even
> when it found something. If some malicious actor was able to modify
> files in the ...\system32 directory then that same actor probably also
> had the ability to nobble the scanner. Please keep that in mind too.
>
> The reason it takes five minutes to scan your files is basically that
> you are scanning something like five thousand files (note: you haven't
> mentioned the average size of those files) for something in the region
> of ten million threats, likely using modest, general purpose hardware.
>
> If I may take liberties with units, the number of file.threats is of
> the order of fifty billion. The average time per file.threat scan is
> 300/50000000000 or approximately six nanoseconds. That's quite a bit
> less than the access time for your RAM (never mind the mass storage!)
> and while this is like measuring a piece of string with a theodolite,
> typical 21st century CPUs won't be able to execute more than a very
> few instructions in that time even if they take a running jump at it.
> I shouldn't be surprised if millions of instructions would need to be
> executed to scan a single file for a single threat. So I think that,
> under the circumstances, ClamAV doesn't do too bad a job.
>
> What are the specifications for your harware?
>
> Why are you scanning the things you're scanning?
>
> Why are you using the scanned device to scan itself?
>
> Why are you not scanning the things you aren't scanning?
>
> How long would you expect your scan to take on your hardware?
>
> What is your target scan time?
>
> Why?
>
> What do you expect the results to tell you?
>
> What do you plan to do when you have them?
>
> How long will *that* take?
>
> How long does your coffee machine take to make a brew?
>
> Can you compare/contrast the various times in your answers?
>
> Please be much more forthcoming.
>
> > Possible solutions are:
> >
> > - Non recursive MULTISCAN ( seems that this is not available )
> > - Multithread SCAN command ( seems that is not available )
> >
> > Is there another solution here?
>
> According to the (old) ClamAV Bugzilla there was an issue with
> MULTISCAN which was fixed years ago:
>
> https://bugzilla.clamav.net/show_bug.cgi?id=1869
>
> I didn't find the issue mentioned here:
>
> https://github.com/Cisco-Talos/clamav/issues/
>
> but although that's where things are being done now, many old Bugzilla
> issues haven't been moved to Github.
>
> If you can produce a test case to confirm that Micah's 2018 comment on
> Bugzilla is wrong, you should file a report on Github.
>
> --
>
> 73,
> Ged.
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>