Mailing List Archive

On 05.03.20 15:38, Ashish Poddar via clamav-users wrote:
>We have a situation where we run a clamav daemon to scan files on a system.

how?

>However, in the process, we only use about 10% CPU in the system. We would
>naturally like to increase this number. We were thus trying to come up with
>a way to scan multiple files in parallel on the same system. Is there a way
>we can spawn multiple clam daemons to do this?

afaik single clamav daemon is able to scan multiple files in parallel.

>I am aware of the multiscan mode in clamdscan but I want each scan to be a
>separate process so as to not increase the overall scan time of any one
>file.

I don't understand. Why do you think that scanning in multiple threads
increases scan time?

> Is there a way to do this? Or are there any other alternatives to
>tackle this situation?


>This is my first time posting to this mailing list. I apologize if I made
>any violations while using the mailing list. Thanks for you cooperation.

I don't see any violations here, although I prefer single plaintext
(you used multipart text+html) :)


--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Honk if you love peace and quiet.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

On 05/03/2020 12:26, Matus UHLAR - fantomas wrote:
> On 05.03.20 15:38, Ashish Poddar via clamav-users wrote:
>> We have a situation where we run a clamav daemon to scan files on a
>> system.
>
> how?
>
>> However, in the process, we only use about 10% CPU in the system. We
>> would
>> naturally like to increase this number. We were thus trying to come up
>> with
>> a way to scan multiple files in parallel on the same system. Is there
>> a way
>> we can spawn multiple clam daemons to do this?
>
> afaik single clamav daemon is able to scan multiple files in parallel.
>
>> I am aware of the multiscan mode in clamdscan but I want each scan to
>> be a
>> separate process so as to not increase the overall scan time of any one
>> file.
>
> I don't understand. Why do you think that scanning in multiple threads
> increases scan time?

I don't see hosw muti-thread or multi-process would be different.
But I do not recommend scanning in parallel, the disk I/O is IMHO the
bottleneck, why you use only 10% of CPU, and running multiple scans in
parallel will make things even worse.



--
Best Regards
Vladislav Kurz


_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

> On Mar 5, 2020, at 05:09, Ashish Poddar via clamav-users <clamav-users@lists.clamav.net> wrote:
>
> ?
> Hi all,
>
> We have a situation where we run a clamav daemon to scan files on a system. However, in the process, we only use about 10% CPU in the system. We would naturally like to increase this number. We were thus trying to come up with a way to scan multiple files in parallel on the same system. Is there a way we can spawn multiple clam daemons to do this?

In short, yes. You can run multiple ClamAV daemons. Simply launch each daemon with a different config file for each daemon and then you can have clamdscan processes talking to each of those daemons. At a minimum, you would want to use different sockets for the daemons to listen on. You could set up each daemon to use smaller virus database sets, but that really only impacts the speed at which the daemon reloads when there are database changes.

Are you trying to reduce the time that it takes to scan your entire system? Or is there some other reason that you are looking to run multiple daemons? “Increasing CPU usage” isn’t generally one of the things people look to do just because it’s low.

> I am aware of the multiscan mode in clamdscan but I want each scan to be a separate process so as to not increase the overall scan time of any one file. Is there a way to do this? Or are there any other alternatives to tackle this situation?

Running multiple clamdscan processes doesn’t require multiple clamd daemons to be running. You can run multiple clamdscan processes against a single clamd daemon and it will just spawn more threads to do the scanning. So, for example, you can have a separate clamdscan process working each user’s home directory in parallel and they should all be able to talk to a single clamd daemon.

—Maarten

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Hi there,

On Thu, 5 Mar 2020, Ashish Poddar via clamav-users wrote:

> We have a situation where we run a clamav daemon to scan files on a system.
> However, in the process, we only use about 10% CPU in the system. We would
> naturally like to increase this number. We were thus trying to come up with
> a way to scan multiple files in parallel on the same system.

My crystal ball is a little bit foggy this afternoon, you might want
to be more forthcoming about your system. Sometimes that way you will
get more useful ideas in the replies to your questions.

I suspect that the simplest way to increase your 10% CPU number is to
find _another_ number (probably one which you have not yet measured)
and then find ways to reduce that. Suggestions might be file I/O
transfer rates, latencies/access times/fragmentation, cache hit rates,
memory bandwith, swap (if any) stats, interrupt service stats. I'm
quite sure there will be many others. None of them is particularly
relevant to clamd processes specifically. I do not understand why you
would want to increase CPU utilization, most of the time I'm trying to
find ways to reduce it. If you just mean that you want to make your
filesystem scans complete more quickly, then perhaps, instead of what
you have actually said, you should say that.

> Is there a way we can spawn multiple clam daemons to do this?

It is not clear to me that your suggested solution will have the
desired effect, but yes, you can run multiple clamd daemons, and
indeed I do routinely do that - but not for reasons like yours.

> I am aware of the multiscan mode in clamdscan but I want each scan to be a
> separate process so as to not increase the overall scan time of any one
> file. Is there a way to do this?

Perhaps you can explain this to me more fully so that I can be sure
that I understand what you want to do. If I do understand, it is not
clear to me that causing each scan to be in a separate _process_ will
have the desired effect (it might do the opposite of what you hope).
It will likely be in a separate thread in any case. Have you read
through the example clamd configuration file in the distribution?

> Or are there any other alternatives to tackle this situation?

I think you will first need to do some instrumentation on your systems
to establish what they are doing, and what they are capable of doing.

> This is my first time posting to this mailing list. I apologize if I
> made any violations while using the mailing list.

I wouldn't claim any violation of etiquette, but if you can give us
more information and clearer, reasoned statements of your requirements
that would help us to help you.

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml