Mailing List Archive

[clamav-users] Fwd: Clamd performance issues in docker image
Hi,



We observed performance issues using clamd docker image (
https://hub.docker.com/r/mkodockx/docker-clamav/dockerfile) for virus
scanning.



Our application and clamd (side car deployment) resides on the same pod.
The application sends .zip files (200 KB - 300 KB) to clamd over tcp to
scan using INSTREAM command. The client library that is used is
https://github.com/cdarras/clamav-client library. The .zip file usually
contains multiple xml files and txt files.



We were able to achieve around 4 request per second with 10 vcpu limit for
clamd docker image.



By turning off ScanArchive (false) , the request per second increases to
25. When the scanarchive is true, there are too many requests waiting in
the queue (QUEUEDSINCE) , as observed in clamdtop. But it doesn’t detect
the virus file inside the zip. Is it possible to detect virus inside the
zip with ScanArchive turned off?



Attached is the strace, clamd.conf, pidstat, iostat, htop data during
performance testing.

We are suspecting some issues while writing to the temp directory and
scanning the individual files inside the zip file. We are not able to root
cause the same.



Please let me know your thoughts. Your help is highly appreciated.



Thanks

Zayan
Re: [clamav-users] Fwd: Clamd performance issues in docker image [ In reply to ]
Hi there,

On Fri, 11 Sep 2020, Zayan abdul shukoor via clamav-users wrote:

> ... Is it possible to detect virus inside the zip with ScanArchive
> turned off? ...

Tell the client to unpack zip, send the unpacked content to clamd.

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: [clamav-users] Fwd: Clamd performance issues in docker image [ In reply to ]
The bundle of files need to be together to track it. So the client can't
send it after unzipping.

The issue happens when clamd is run as a docker image.

We have tried mapping the clamd temp directory to tmpfs ram memory, to see
if it improves the scan time.

Thanks
Zayan




On Fri, Sep 11, 2020 at 4:28 PM G.W. Haywood via clamav-users <
clamav-users@lists.clamav.net> wrote:

> Hi there,
>
> On Fri, 11 Sep 2020, Zayan abdul shukoor via clamav-users wrote:
>
> > ... Is it possible to detect virus inside the zip with ScanArchive
> > turned off? ...
>
> Tell the client to unpack zip, send the unpacked content to clamd.
>
> --
>
> 73,
> Ged.
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>
Re: [clamav-users] Fwd: Clamd performance issues in docker image [ In reply to ]
We have also uploaded the strace, clamd.conf, pidstat, iostat, htop data
during performance testing to https://github.com/zaabskr/clamav-traces

On Sun, Sep 13, 2020 at 8:37 AM Zayan abdul shukoor <zayan.as.plc@gmail.com>
wrote:

> The bundle of files need to be together to track it. So the client can't
> send it after unzipping.
>
> The issue happens when clamd is run as a docker image.
>
> We have tried mapping the clamd temp directory to tmpfs ram memory, to see
> if it improves the scan time.
>
> Thanks
> Zayan
>
>
>
>
> On Fri, Sep 11, 2020 at 4:28 PM G.W. Haywood via clamav-users <
> clamav-users@lists.clamav.net> wrote:
>
>> Hi there,
>>
>> On Fri, 11 Sep 2020, Zayan abdul shukoor via clamav-users wrote:
>>
>> > ... Is it possible to detect virus inside the zip with ScanArchive
>> > turned off? ...
>>
>> Tell the client to unpack zip, send the unpacked content to clamd.
>>
>> --
>>
>> 73,
>> Ged.
>>
>> _______________________________________________
>>
>> clamav-users mailing list
>> clamav-users@lists.clamav.net
>> https://lists.clamav.net/mailman/listinfo/clamav-users
>>
>>
>> Help us build a comprehensive ClamAV guide:
>> https://github.com/vrtadmin/clamav-faq
>>
>> http://www.clamav.net/contact.html#ml
>>
>