Mailing List Archive

Thank you Yasuhiro KIMURA ,
Isn't it the second scan result ? The second analyse on same file is faster.
Could tou try to restart clamav-daemon and re-do the analyse with clamdscan.
I've tried it on 3 computers, all are above 40seconds



-----Message d'origine-----
De?: jmedard@amv-sa.fr <jmedard@amv-sa.fr>
Envoy??: lundi 18 mars 2019 11:38
??: clamav-users@lists.clamav.net
Objet?: [clamav-users] Scan very slow

Hi,

Since some weeks, my clamav (0.100.2+dfsg-0+deb9u1) on Debian 9 64b is very
slow for scanning some files.
It seems that this concerns PDF files. We have this problem with a lot of
files.

I'm using daemon version with default clamd.conf

# clamdscan -v /tmp/esploso_A3TH.pdf
/tmp/esploso_A3TH.pdf: OK

----------- SCAN SUMMARY -----------
Infected files: 0
Time: 185.457 sec (3 m 5 s)

For information, we use a good server (2x Xeon E5-2630v3 with 128Go). I
tried on several servers, the scan is very slow too.

I'll attach one example PDF file.

Did you have the same problem ?
Is there not a bug ?

Regards,
JME



_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Hello,

> Did you have the same problem ?
> Is there not a bug ?

Seems same here :

clamdscan -m --fdpass *
/tmp/esploso_A3TH.pdf: OK

----------- SCAN SUMMARY -----------
Infected files: 0
Time: 59.406 sec (0 m 59 s)

Using clamscan --debug shows there is a LOT of embedded items in this
PDF file (2886 files extracted and scanned from this PDF).

--
Cordialement / Best regards,

Arnaud Jacques
G?rant de SecuriteInfo.com

T?l?phone : +33-(0)3.44.39.76.46
E-mail : aj@securiteinfo.com
Site web : https://www.securiteinfo.com
Facebook : https://www.facebook.com/pages/SecuriteInfocom/132872523492286
Twitter : @SecuriteInfoCom

Securiteinfo.com
La S?curit? Informatique - La S?curit? des Informations.
266, rue de Villers
60123 Bonneuil en Valois


_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Hello,
Thanks for your feedback.
Indeed, there are many elements in this file.
It is surprising that the analysis is so long. I tested the file with 6
other antivirus (paid), the analysis lasts less than one second.
Do you think that it is possible for example to limit the number "embedded
items in this PDF file" in order to reduce the analysis time?
Regards,
JME

-----Message d'origine-----
De?: Arnaud Jacques <webmaster@securiteinfo.com>
Envoy??: lundi 18 mars 2019 13:08
??: ClamAV users ML <clamav-users@lists.clamav.net>
Cc?: jmedard@amv-sa.fr
Objet?: Re: [clamav-users] Scan very slow

Hello,

> Did you have the same problem ?
> Is there not a bug ?

Seems same here :

clamdscan -m --fdpass *
/tmp/esploso_A3TH.pdf: OK

----------- SCAN SUMMARY -----------
Infected files: 0
Time: 59.406 sec (0 m 59 s)

Using clamscan --debug shows there is a LOT of embedded items in this PDF
file (2886 files extracted and scanned from this PDF).

--
Cordialement / Best regards,

Arnaud Jacques
G?rant de SecuriteInfo.com

T?l?phone : +33-(0)3.44.39.76.46
E-mail : aj@securiteinfo.com
Site web : https://www.securiteinfo.com
Facebook : https://www.facebook.com/pages/SecuriteInfocom/132872523492286
Twitter : @SecuriteInfoCom

Securiteinfo.com
La S?curit? Informatique - La S?curit? des Informations.
266, rue de Villers
60123 Bonneuil en Valois



_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

From: Jean-Michel via clamav-users <clamav-users@lists.clamav.net>
Subject: Re: [clamav-users] Scan very slow
Date: Mon, 18 Mar 2019 12:30:49 +0100

> Isn't it the second scan result ? The second analyse on same file is faster.
> Could tou try to restart clamav-daemon and re-do the analyse with clamdscan.
> I've tried it on 3 computers, all are above 40seconds

It was first trial. But after restarting clamav-daemon result changed
as following.

yasu@kusanagi[1716]% clamdscan esploso_A3TH.pdf
/home/yasu/tmp/esploso_A3TH.pdf: OK

----------- SCAN SUMMARY -----------
Infected files: 0
Time: 60.551 sec (1 m 0 s)
yasu@kusanagi[1717]%

---
Yasuhiro KIMURA

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Jean-Michel,

Le 18/03/2019 à 14:55, Jean-Michel via clamav-users a écrit :
> Hello,
> Thanks for your feedback.
> Indeed, there are many elements in this file.
> It is surprising that the analysis is so long. I tested the file with 6
> other antivirus (paid), the analysis lasts less than one second.
That does not mean scanning is deep and detection is maximum :)


> Do you think that it is possible for example to limit the number "embedded
> items in this PDF file" in order to reduce the analysis time?

I think so. Please see --max-files option of clamscan (clamdscan has
same option in clamd.conf)
Maybe more usefull options using :
clamscan --help|grep max
I guess you can play with such options to optimize your scan.

--
Cordialement / Best regards,

Arnaud Jacques
Gérant de SecuriteInfo.com

Téléphone : +33-(0)3.44.39.76.46
E-mail : aj@securiteinfo.com
Site web : https://www.securiteinfo.com
Facebook : https://www.facebook.com/pages/SecuriteInfocom/132872523492286
Twitter : @SecuriteInfoCom

Securiteinfo.com
La Sécurité Informatique - La Sécurité des Informations.
266, rue de Villers
60123 Bonneuil en Valois


_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

We've noticed a marked increase in scan times over the last couple of weeks
as well. From the look of it, there's something in the daily file that's
causing it. Whether this is similar to the safebrowsing issue (where the
ordering of entries in the file caused a 3000% increase in time) is unclear.

--Maarten Broekman

Full scans without the daily cvd/cld: Scan time ~60seconds
Full scans with the daily from March 11th: Scan time: 84seconds
Full scans with the daily from March 17th: Scan time: 109seconds

~/clamav# ls -larth /tmp/clamdtest*/daily.cld
-rw-r--r-- 1 clamav clamav 110M Mar 11 04:15 /tmp/clamdtest2/daily.cld
-rw-r--r-- 1 clamav clamav 113M Mar 17 04:15 /tmp/clamdtest/daily.cld

~/clamav# wc /tmp/clamdtest*/daily.cld
1514589 1517471 115031552 /tmp/clamdtest2/daily.cld
1524782 1527664 118202368 /tmp/clamdtest/daily.cld

Single file scans with JUST the daily.cld:
~/clamav# time /opt/clamav/clamav/bin/clamscan -d /tmp/clamdtest2/daily.cld
test42.js
test42.js: OK

----------- SCAN SUMMARY -----------
Known viruses: 1504423
Engine version: 0.100.2
Scanned directories: 1
Scanned files: 1
Infected files: 0
Data scanned: 0.00 MB
Data read: 0.00 MB (ratio 0.00:1)
Time: 5.255 sec (0 m 5 s)

real 0m5.260s
user 0m5.044s
sys 0m0.192s
~/clamav# time /opt/clamav/clamav/bin/clamscan -d /tmp/clamdtest/daily.cld
test42.js
test42.js: OK

----------- SCAN SUMMARY -----------
Known viruses: 1514543
Engine version: 0.100.2
Scanned directories: 1
Scanned files: 1
Infected files: 0
Data scanned: 0.00 MB
Data read: 0.00 MB (ratio 0.00:1)
Time: 9.300 sec (0 m 9 s)

real 0m9.329s
user 0m9.100s
sys 0m0.204s






On Mon, Mar 18, 2019 at 10:02 AM Yasuhiro KIMURA <yasu@utahime.org> wrote:

> From: Jean-Michel via clamav-users <clamav-users@lists.clamav.net>
> Subject: Re: [clamav-users] Scan very slow
> Date: Mon, 18 Mar 2019 12:30:49 +0100
>
> > Isn't it the second scan result ? The second analyse on same file is
> faster.
> > Could tou try to restart clamav-daemon and re-do the analyse with
> clamdscan.
> > I've tried it on 3 computers, all are above 40seconds
>
> It was first trial. But after restarting clamav-daemon result changed
> as following.
>
> yasu@kusanagi[1716]% clamdscan esploso_A3TH.pdf
> /home/yasu/tmp/esploso_A3TH.pdf: OK
>
> ----------- SCAN SUMMARY -----------
> Infected files: 0
> Time: 60.551 sec (1 m 0 s)
> yasu@kusanagi[1717]%
>
> ---
> Yasuhiro KIMURA
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>

Very interesting.
Do you think we should make a bug-report?
Regards





De : Maarten Broekman <maarten.broekman@gmail.com>
Envoyé : lundi 18 mars 2019 15:14
À : ClamAV users ML <clamav-users@lists.clamav.net>
Objet : Re: [clamav-users] Scan very slow



We've noticed a marked increase in scan times over the last couple of weeks as well. From the look of it, there's something in the daily file that's causing it. Whether this is similar to the safebrowsing issue (where the ordering of entries in the file caused a 3000% increase in time) is unclear.



--Maarten Broekman



Full scans without the daily cvd/cld: Scan time ~60seconds

Full scans with the daily from March 11th: Scan time: 84seconds

Full scans with the daily from March 17th: Scan time: 109seconds



~/clamav# ls -larth /tmp/clamdtest*/daily.cld

-rw-r--r-- 1 clamav clamav 110M Mar 11 04:15 /tmp/clamdtest2/daily.cld

-rw-r--r-- 1 clamav clamav 113M Mar 17 04:15 /tmp/clamdtest/daily.cld



~/clamav# wc /tmp/clamdtest*/daily.cld

1514589 1517471 115031552 /tmp/clamdtest2/daily.cld

1524782 1527664 118202368 /tmp/clamdtest/daily.cld



Single file scans with JUST the daily.cld:

~/clamav# time /opt/clamav/clamav/bin/clamscan -d /tmp/clamdtest2/daily.cld test42.js

test42.js: OK



----------- SCAN SUMMARY -----------

Known viruses: 1504423

Engine version: 0.100.2

Scanned directories: 1

Scanned files: 1

Infected files: 0

Data scanned: 0.00 MB

Data read: 0.00 MB (ratio 0.00:1)

Time: 5.255 sec (0 m 5 s)



real 0m5.260s

user 0m5.044s

sys 0m0.192s

~/clamav# time /opt/clamav/clamav/bin/clamscan -d /tmp/clamdtest/daily.cld test42.js

test42.js: OK



----------- SCAN SUMMARY -----------

Known viruses: 1514543

Engine version: 0.100.2

Scanned directories: 1

Scanned files: 1

Infected files: 0

Data scanned: 0.00 MB

Data read: 0.00 MB (ratio 0.00:1)

Time: 9.300 sec (0 m 9 s)



real 0m9.329s

user 0m9.100s

sys 0m0.204s













On Mon, Mar 18, 2019 at 10:02 AM Yasuhiro KIMURA <yasu@utahime.org <mailto:yasu@utahime.org> > wrote:

From: Jean-Michel via clamav-users <clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net> >
Subject: Re: [clamav-users] Scan very slow
Date: Mon, 18 Mar 2019 12:30:49 +0100

> Isn't it the second scan result ? The second analyse on same file is faster.
> Could tou try to restart clamav-daemon and re-do the analyse with clamdscan.
> I've tried it on 3 computers, all are above 40seconds

It was first trial. But after restarting clamav-daemon result changed
as following.

yasu@kusanagi[1716]% clamdscan esploso_A3TH.pdf
/home/yasu/tmp/esploso_A3TH.pdf: OK

----------- SCAN SUMMARY -----------
Infected files: 0
Time: 60.551 sec (1 m 0 s)
yasu@kusanagi[1717]%

---
Yasuhiro KIMURA

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Maarten,

This is very concerning, and the details you’ve provide are quite helpful. Thank you for investigating.
Hopefully we can figure out why the newer daily.cld/cvd is scanning significantly slower than before. Any other details you can provide would probably be helpful. If you’re aware if any specific file types are causing the issue, or if all files appear to scanning slower that will also help.

-Micah


Micah Snyder
ClamAV Development
Talos
Cisco Systems, Inc.



From: clamav-users <clamav-users-bounces@lists.clamav.net> on behalf of Maarten Broekman via clamav-users <clamav-users@lists.clamav.net>
Reply-To: ClamAV users ML <clamav-users@lists.clamav.net>
Date: Monday, March 18, 2019 at 10:37 AM
To: ClamAV users ML <clamav-users@lists.clamav.net>
Cc: Maarten Broekman <maarten.broekman@gmail.com>
Subject: Re: [clamav-users] Scan very slow

We've noticed a marked increase in scan times over the last couple of weeks as well. From the look of it, there's something in the daily file that's causing it. Whether this is similar to the safebrowsing issue (where the ordering of entries in the file caused a 3000% increase in time) is unclear.

--Maarten Broekman

Full scans without the daily cvd/cld: Scan time ~60seconds
Full scans with the daily from March 11th: Scan time: 84seconds
Full scans with the daily from March 17th: Scan time: 109seconds

~/clamav# ls -larth /tmp/clamdtest*/daily.cld
-rw-r--r-- 1 clamav clamav 110M Mar 11 04:15 /tmp/clamdtest2/daily.cld
-rw-r--r-- 1 clamav clamav 113M Mar 17 04:15 /tmp/clamdtest/daily.cld

~/clamav# wc /tmp/clamdtest*/daily.cld
1514589 1517471 115031552 /tmp/clamdtest2/daily.cld
1524782 1527664 118202368 /tmp/clamdtest/daily.cld

Single file scans with JUST the daily.cld:
~/clamav# time /opt/clamav/clamav/bin/clamscan -d /tmp/clamdtest2/daily.cld test42.js
test42.js: OK

----------- SCAN SUMMARY -----------
Known viruses: 1504423
Engine version: 0.100.2
Scanned directories: 1
Scanned files: 1
Infected files: 0
Data scanned: 0.00 MB
Data read: 0.00 MB (ratio 0.00:1)
Time: 5.255 sec (0 m 5 s)

real 0m5.260s
user 0m5.044s
sys 0m0.192s
~/clamav# time /opt/clamav/clamav/bin/clamscan -d /tmp/clamdtest/daily.cld test42.js
test42.js: OK

----------- SCAN SUMMARY -----------
Known viruses: 1514543
Engine version: 0.100.2
Scanned directories: 1
Scanned files: 1
Infected files: 0
Data scanned: 0.00 MB
Data read: 0.00 MB (ratio 0.00:1)
Time: 9.300 sec (0 m 9 s)

real 0m9.329s
user 0m9.100s
sys 0m0.204s






On Mon, Mar 18, 2019 at 10:02 AM Yasuhiro KIMURA <yasu@utahime.org<mailto:yasu@utahime.org>> wrote:
From: Jean-Michel via clamav-users <clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>>
Subject: Re: [clamav-users] Scan very slow
Date: Mon, 18 Mar 2019 12:30:49 +0100

> Isn't it the second scan result ? The second analyse on same file is faster.
> Could tou try to restart clamav-daemon and re-do the analyse with clamdscan.
> I've tried it on 3 computers, all are above 40seconds

It was first trial. But after restarting clamav-daemon result changed
as following.

yasu@kusanagi[1716]% clamdscan esploso_A3TH.pdf
/home/yasu/tmp/esploso_A3TH.pdf: OK

----------- SCAN SUMMARY -----------
Infected files: 0
Time: 60.551 sec (1 m 0 s)
yasu@kusanagi[1717]%

---
Yasuhiro KIMURA

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Hi,

Micah Snyder, Do you know if Clamav was able to trace the orgine of getting crawled in the database "daily.cld" and was able to fix the problem?
Regards



De : Micah Snyder (micasnyd) <micasnyd@cisco.com>
Envoyé : lundi 18 mars 2019 18:09
À : ClamAV users ML <clamav-users@lists.clamav.net>
Objet : Re: [clamav-users] Scan very slow



Maarten,



This is very concerning, and the details you’ve provide are quite helpful. Thank you for investigating.

Hopefully we can figure out why the newer daily.cld/cvd is scanning significantly slower than before. Any other details you can provide would probably be helpful. If you’re aware if any specific file types are causing the issue, or if all files appear to scanning slower that will also help.



-Micah




Micah Snyder
ClamAV Development
Talos
Cisco Systems, Inc.







From: clamav-users <clamav-users-bounces@lists.clamav.net <mailto:clamav-users-bounces@lists.clamav.net> > on behalf of Maarten Broekman via clamav-users <clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net> >
Reply-To: ClamAV users ML <clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net> >
Date: Monday, March 18, 2019 at 10:37 AM
To: ClamAV users ML <clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net> >
Cc: Maarten Broekman <maarten.broekman@gmail.com <mailto:maarten.broekman@gmail.com> >
Subject: Re: [clamav-users] Scan very slow



We've noticed a marked increase in scan times over the last couple of weeks as well. From the look of it, there's something in the daily file that's causing it. Whether this is similar to the safebrowsing issue (where the ordering of entries in the file caused a 3000% increase in time) is unclear.



--Maarten Broekman



Full scans without the daily cvd/cld: Scan time ~60seconds

Full scans with the daily from March 11th: Scan time: 84seconds

Full scans with the daily from March 17th: Scan time: 109seconds



~/clamav# ls -larth /tmp/clamdtest*/daily.cld

-rw-r--r-- 1 clamav clamav 110M Mar 11 04:15 /tmp/clamdtest2/daily.cld

-rw-r--r-- 1 clamav clamav 113M Mar 17 04:15 /tmp/clamdtest/daily.cld



~/clamav# wc /tmp/clamdtest*/daily.cld

1514589 1517471 115031552 /tmp/clamdtest2/daily.cld

1524782 1527664 118202368 /tmp/clamdtest/daily.cld



Single file scans with JUST the daily.cld:

~/clamav# time /opt/clamav/clamav/bin/clamscan -d /tmp/clamdtest2/daily.cld test42.js

test42.js: OK



----------- SCAN SUMMARY -----------

Known viruses: 1504423

Engine version: 0.100.2

Scanned directories: 1

Scanned files: 1

Infected files: 0

Data scanned: 0.00 MB

Data read: 0.00 MB (ratio 0.00:1)

Time: 5.255 sec (0 m 5 s)



real 0m5.260s

user 0m5.044s

sys 0m0.192s

~/clamav# time /opt/clamav/clamav/bin/clamscan -d /tmp/clamdtest/daily.cld test42.js

test42.js: OK



----------- SCAN SUMMARY -----------

Known viruses: 1514543

Engine version: 0.100.2

Scanned directories: 1

Scanned files: 1

Infected files: 0

Data scanned: 0.00 MB

Data read: 0.00 MB (ratio 0.00:1)

Time: 9.300 sec (0 m 9 s)



real 0m9.329s

user 0m9.100s

sys 0m0.204s













On Mon, Mar 18, 2019 at 10:02 AM Yasuhiro KIMURA <yasu@utahime.org <mailto:yasu@utahime.org> > wrote:

From: Jean-Michel via clamav-users <clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net> >
Subject: Re: [clamav-users] Scan very slow
Date: Mon, 18 Mar 2019 12:30:49 +0100

> Isn't it the second scan result ? The second analyse on same file is faster.
> Could tou try to restart clamav-daemon and re-do the analyse with clamdscan.
> I've tried it on 3 computers, all are above 40seconds

It was first trial. But after restarting clamav-daemon result changed
as following.

yasu@kusanagi[1716]% clamdscan esploso_A3TH.pdf
/home/yasu/tmp/esploso_A3TH.pdf: OK

----------- SCAN SUMMARY -----------
Infected files: 0
Time: 60.551 sec (1 m 0 s)
yasu@kusanagi[1717]%

---
Yasuhiro KIMURA

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Reference? First I'm hearing of any such thing.

-Al-

> On Mar 23, 2019, at 02:26, Jean-Michel via clamav-users <clamav-users@lists.clamav.net> wrote:
>
> Hi,
> Micah Snyder, Do you know if Clamav was able to trace the orgine of getting crawled in the database "daily.cld" and was able to fix the problem?
> Regards
>
> De : Micah Snyder (micasnyd) <micasnyd@cisco.com <mailto:micasnyd@cisco.com>>
> Envoyé : lundi 18 mars 2019 18:09
> À : ClamAV users ML <clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>>
> Objet : Re: [clamav-users] Scan very slow
>
> Maarten,
>
> This is very concerning, and the details you’ve provide are quite helpful. Thank you for investigating.
> Hopefully we can figure out why the newer daily.cld/cvd is scanning significantly slower than before. Any other details you can provide would probably be helpful. If you’re aware if any specific file types are causing the issue, or if all files appear to scanning slower that will also help.
>
> -Micah
>
>
> Micah Snyder
> ClamAV Development
> Talos
> Cisco Systems, Inc.
>
>
>
> From: clamav-users <clamav-users-bounces@lists.clamav.net <mailto:clamav-users-bounces@lists.clamav.net>> on behalf of Maarten Broekman via clamav-users <clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>>
> Reply-To: ClamAV users ML <clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>>
> Date: Monday, March 18, 2019 at 10:37 AM
> To: ClamAV users ML <clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>>
> Cc: Maarten Broekman <maarten.broekman@gmail.com <mailto:maarten.broekman@gmail.com>>
> Subject: Re: [clamav-users] Scan very slow
>
> We've noticed a marked increase in scan times over the last couple of weeks as well. From the look of it, there's something in the daily file that's causing it. Whether this is similar to the safebrowsing issue (where the ordering of entries in the file caused a 3000% increase in time) is unclear.
>
> --Maarten Broekman
>
> Full scans without the daily cvd/cld: Scan time ~60seconds
> Full scans with the daily from March 11th: Scan time: 84seconds
> Full scans with the daily from March 17th: Scan time: 109seconds
>
> ~/clamav# ls -larth /tmp/clamdtest*/daily.cld
> -rw-r--r-- 1 clamav clamav 110M Mar 11 04:15 /tmp/clamdtest2/daily.cld
> -rw-r--r-- 1 clamav clamav 113M Mar 17 04:15 /tmp/clamdtest/daily.cld
>
> ~/clamav# wc /tmp/clamdtest*/daily.cld
> 1514589 1517471 115031552 /tmp/clamdtest2/daily.cld
> 1524782 1527664 118202368 /tmp/clamdtest/daily.cld
>
> Single file scans with JUST the daily.cld:
> ~/clamav# time /opt/clamav/clamav/bin/clamscan -d /tmp/clamdtest2/daily.cld test42.js
> test42.js: OK
>
> ----------- SCAN SUMMARY -----------
> Known viruses: 1504423
> Engine version: 0.100.2
> Scanned directories: 1
> Scanned files: 1
> Infected files: 0
> Data scanned: 0.00 MB
> Data read: 0.00 MB (ratio 0.00:1)
> Time: 5.255 sec (0 m 5 s)
>
> real 0m5.260s
> user 0m5.044s
> sys 0m0.192s
> ~/clamav# time /opt/clamav/clamav/bin/clamscan -d /tmp/clamdtest/daily.cld test42.js
> test42.js: OK
>
> ----------- SCAN SUMMARY -----------
> Known viruses: 1514543
> Engine version: 0.100.2
> Scanned directories: 1
> Scanned files: 1
> Infected files: 0
> Data scanned: 0.00 MB
> Data read: 0.00 MB (ratio 0.00:1)
> Time: 9.300 sec (0 m 9 s)
>
> real 0m9.329s
> user 0m9.100s
> sys 0m0.204s
>
>
>
>
>
>
> On Mon, Mar 18, 2019 at 10:02 AM Yasuhiro KIMURA <yasu@utahime.org <mailto:yasu@utahime.org>> wrote:
>> From: Jean-Michel via clamav-users <clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>>
>> Subject: Re: [clamav-users] Scan very slow
>> Date: Mon, 18 Mar 2019 12:30:49 +0100
>>
>> > Isn't it the second scan result ? The second analyse on same file is faster.
>> > Could tou try to restart clamav-daemon and re-do the analyse with clamdscan.
>> > I've tried it on 3 computers, all are above 40seconds
>>
>> It was first trial. But after restarting clamav-daemon result changed
>> as following.
>>
>> yasu@kusanagi[1716]% clamdscan esploso_A3TH.pdf
>> /home/yasu/tmp/esploso_A3TH.pdf: OK
>>
>> ----------- SCAN SUMMARY -----------
>> Infected files: 0
>> Time: 60.551 sec (1 m 0 s)
>> yasu@kusanagi[1717]%
>>
>> ---
>> Yasuhiro KIMURA
>>
>> _______________________________________________
>>
>> clamav-users mailing list
>> clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>
>> https://lists.clamav.net/mailman/listinfo/clamav-users <https://lists.clamav.net/mailman/listinfo/clamav-users>
>>
>>
>> Help us build a comprehensive ClamAV guide:
>> https://github.com/vrtadmin/clamav-faq <https://github.com/vrtadmin/clamav-faq>
>>
>> http://www.clamav.net/contact.html#ml <http://www.clamav.net/contact.html#ml>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>
> https://lists.clamav.net/mailman/listinfo/clamav-users <https://lists.clamav.net/mailman/listinfo/clamav-users>
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq <https://github.com/vrtadmin/clamav-faq>
>
> http://www.clamav.net/contact.html#ml <http://www.clamav.net/contact.html#ml>

See Maarten Broekman tests above

https://lists.clamav.net/pipermail/clamav-users/2019-March/007737.html



De : Al Varnell <alvarnell@mac.com>
Envoyé : samedi 23 mars 2019 10:55
À : ClamAV users ML <clamav-users@lists.clamav.net>
Objet : Re: [clamav-users] Scan very slow



Reference? First I'm hearing of any such thing.



-Al-





On Mar 23, 2019, at 02:26, Jean-Michel via clamav-users <clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net> > wrote:



Hi,

Micah Snyder, Do you know if Clamav was able to trace the orgine of getting crawled in the database "daily.cld" and was able to fix the problem?
Regards



De : Micah Snyder (micasnyd) < <mailto:micasnyd@cisco.com> micasnyd@cisco.com>
Envoyé : lundi 18 mars 2019 18:09
À : ClamAV users ML < <mailto:clamav-users@lists.clamav.net> clamav-users@lists.clamav.net>
Objet : Re: [clamav-users] Scan very slow



Maarten,



This is very concerning, and the details you’ve provide are quite helpful. Thank you for investigating.

Hopefully we can figure out why the newer daily.cld/cvd is scanning significantly slower than before. Any other details you can provide would probably be helpful. If you’re aware if any specific file types are causing the issue, or if all files appear to scanning slower that will also help.



-Micah




Micah Snyder
ClamAV Development
Talos
Cisco Systems, Inc.







From: clamav-users < <mailto:clamav-users-bounces@lists.clamav.net> clamav-users-bounces@lists.clamav.net> on behalf of Maarten Broekman via clamav-users < <mailto:clamav-users@lists.clamav.net> clamav-users@lists.clamav.net>
Reply-To: ClamAV users ML < <mailto:clamav-users@lists.clamav.net> clamav-users@lists.clamav.net>
Date: Monday, March 18, 2019 at 10:37 AM
To: ClamAV users ML < <mailto:clamav-users@lists.clamav.net> clamav-users@lists.clamav.net>
Cc: Maarten Broekman < <mailto:maarten.broekman@gmail.com> maarten.broekman@gmail.com>
Subject: Re: [clamav-users] Scan very slow



We've noticed a marked increase in scan times over the last couple of weeks as well. From the look of it, there's something in the daily file that's causing it. Whether this is similar to the safebrowsing issue (where the ordering of entries in the file caused a 3000% increase in time) is unclear.



--Maarten Broekman



Full scans without the daily cvd/cld: Scan time ~60seconds

Full scans with the daily from March 11th: Scan time: 84seconds

Full scans with the daily from March 17th: Scan time: 109seconds



~/clamav# ls -larth /tmp/clamdtest*/daily.cld

-rw-r--r-- 1 clamav clamav 110M Mar 11 04:15 /tmp/clamdtest2/daily.cld

-rw-r--r-- 1 clamav clamav 113M Mar 17 04:15 /tmp/clamdtest/daily.cld



~/clamav# wc /tmp/clamdtest*/daily.cld

1514589 1517471 115031552 /tmp/clamdtest2/daily.cld

1524782 1527664 118202368 /tmp/clamdtest/daily.cld



Single file scans with JUST the daily.cld:

~/clamav# time /opt/clamav/clamav/bin/clamscan -d /tmp/clamdtest2/daily.cld test42.js

test42.js: OK



----------- SCAN SUMMARY -----------

Known viruses: 1504423

Engine version: 0.100.2

Scanned directories: 1

Scanned files: 1

Infected files: 0

Data scanned: 0.00 MB

Data read: 0.00 MB (ratio 0.00:1)

Time: 5.255 sec (0 m 5 s)



real 0m5.260s

user 0m5.044s

sys 0m0.192s

~/clamav# time /opt/clamav/clamav/bin/clamscan -d /tmp/clamdtest/daily.cld test42.js

test42.js: OK



----------- SCAN SUMMARY -----------

Known viruses: 1514543

Engine version: 0.100.2

Scanned directories: 1

Scanned files: 1

Infected files: 0

Data scanned: 0.00 MB

Data read: 0.00 MB (ratio 0.00:1)

Time: 9.300 sec (0 m 9 s)



real 0m9.329s

user 0m9.100s

sys 0m0.204s













On Mon, Mar 18, 2019 at 10:02 AM Yasuhiro KIMURA < <mailto:yasu@utahime.org> yasu@utahime.org> wrote:

From: Jean-Michel via clamav-users < <mailto:clamav-users@lists.clamav.net> clamav-users@lists.clamav.net>
Subject: Re: [clamav-users] Scan very slow
Date: Mon, 18 Mar 2019 12:30:49 +0100

> Isn't it the second scan result ? The second analyse on same file is faster.
> Could tou try to restart clamav-daemon and re-do the analyse with clamdscan.
> I've tried it on 3 computers, all are above 40seconds

It was first trial. But after restarting clamav-daemon result changed
as following.

yasu@kusanagi[1716]% clamdscan esploso_A3TH.pdf
/home/yasu/tmp/esploso_A3TH.pdf: OK

----------- SCAN SUMMARY -----------
Infected files: 0
Time: 60.551 sec (1 m 0 s)
yasu@kusanagi[1717]%

---
Yasuhiro KIMURA

_______________________________________________

clamav-users mailing list
<mailto:clamav-users@lists.clamav.net> clamav-users@lists.clamav.net
<https://lists.clamav.net/mailman/listinfo/clamav-users> https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
<https://github.com/vrtadmin/clamav-faq> https://github.com/vrtadmin/clamav-faq

<http://www.clamav.net/contact.html#ml> http://www.clamav.net/contact.html#ml


_______________________________________________

clamav-users mailing list
<mailto:clamav-users@lists.clamav.net> clamav-users@lists.clamav.net
<https://lists.clamav.net/mailman/listinfo/clamav-users> https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
<https://github.com/vrtadmin/clamav-faq> https://github.com/vrtadmin/clamav-faq

<http://www.clamav.net/contact.html#ml> http://www.clamav.net/contact.html#ml

Sorry, I misinterpreted the meaning of "crawled" thinking it referred to some sort of compromise of the data.

-Al-

> On Mar 23, 2019, at 09:42, Jean-Michel via clamav-users <clamav-users@lists.clamav.net> wrote:
>
> See Maarten Broekman tests above
> https://lists.clamav.net/pipermail/clamav-users/2019-March/007737.html <https://lists.clamav.net/pipermail/clamav-users/2019-March/007737.html>
>
> De : Al Varnell <alvarnell@mac.com>
> Envoyé : samedi 23 mars 2019 10:55
> À : ClamAV users ML <clamav-users@lists.clamav.net>
> Objet : Re: [clamav-users] Scan very slow
>
> Reference? First I'm hearing of any such thing.
>
> -Al-
>
>
>> On Mar 23, 2019, at 02:26, Jean-Michel via clamav-users <clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>> wrote:
>>
>> Hi,
>> Micah Snyder, Do you know if Clamav was able to trace the orgine of getting crawled in the database "daily.cld" and was able to fix the problem?
>> Regards
>>
>> De : Micah Snyder (micasnyd) <micasnyd@cisco.com <mailto:micasnyd@cisco.com>>
>> Envoyé : lundi 18 mars 2019 18:09
>> À : ClamAV users ML <clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>>
>> Objet : Re: [clamav-users] Scan very slow
>>
>> Maarten,
>>
>> This is very concerning, and the details you’ve provide are quite helpful. Thank you for investigating.
>> Hopefully we can figure out why the newer daily.cld/cvd is scanning significantly slower than before. Any other details you can provide would probably be helpful. If you’re aware if any specific file types are causing the issue, or if all files appear to scanning slower that will also help.
>>
>> -Micah
>>
>>
>> Micah Snyder
>> ClamAV Development
>> Talos
>> Cisco Systems, Inc.
>>
>>
>>
>> From: clamav-users <clamav-users-bounces@lists.clamav.net <mailto:clamav-users-bounces@lists.clamav.net>> on behalf of Maarten Broekman via clamav-users <clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>>
>> Reply-To: ClamAV users ML <clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>>
>> Date: Monday, March 18, 2019 at 10:37 AM
>> To: ClamAV users ML <clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>>
>> Cc: Maarten Broekman <maarten.broekman@gmail.com <mailto:maarten.broekman@gmail.com>>
>> Subject: Re: [clamav-users] Scan very slow
>>
>> We've noticed a marked increase in scan times over the last couple of weeks as well. From the look of it, there's something in the daily file that's causing it. Whether this is similar to the safebrowsing issue (where the ordering of entries in the file caused a 3000% increase in time) is unclear.
>>
>> --Maarten Broekman
>>
>> Full scans without the daily cvd/cld: Scan time ~60seconds
>> Full scans with the daily from March 11th: Scan time: 84seconds
>> Full scans with the daily from March 17th: Scan time: 109seconds
>>
>> ~/clamav# ls -larth /tmp/clamdtest*/daily.cld
>> -rw-r--r-- 1 clamav clamav 110M Mar 11 04:15 /tmp/clamdtest2/daily.cld
>> -rw-r--r-- 1 clamav clamav 113M Mar 17 04:15 /tmp/clamdtest/daily.cld
>>
>> ~/clamav# wc /tmp/clamdtest*/daily.cld
>> 1514589 1517471 115031552 /tmp/clamdtest2/daily.cld
>> 1524782 1527664 118202368 /tmp/clamdtest/daily.cld
>>
>> Single file scans with JUST the daily.cld:
>> ~/clamav# time /opt/clamav/clamav/bin/clamscan -d /tmp/clamdtest2/daily.cld test42.js
>> test42.js: OK
>>
>> ----------- SCAN SUMMARY -----------
>> Known viruses: 1504423
>> Engine version: 0.100.2
>> Scanned directories: 1
>> Scanned files: 1
>> Infected files: 0
>> Data scanned: 0.00 MB
>> Data read: 0.00 MB (ratio 0.00:1)
>> Time: 5.255 sec (0 m 5 s)
>>
>> real 0m5.260s
>> user 0m5.044s
>> sys 0m0.192s
>> ~/clamav# time /opt/clamav/clamav/bin/clamscan -d /tmp/clamdtest/daily.cld test42.js
>> test42.js: OK
>>
>> ----------- SCAN SUMMARY -----------
>> Known viruses: 1514543
>> Engine version: 0.100.2
>> Scanned directories: 1
>> Scanned files: 1
>> Infected files: 0
>> Data scanned: 0.00 MB
>> Data read: 0.00 MB (ratio 0.00:1)
>> Time: 9.300 sec (0 m 9 s)
>>
>> real 0m9.329s
>> user 0m9.100s
>> sys 0m0.204s
>>
>>
>>
>>
>>
>>
>> On Mon, Mar 18, 2019 at 10:02 AM Yasuhiro KIMURA <yasu@utahime.org <mailto:yasu@utahime.org>> wrote:
>>> From: Jean-Michel via clamav-users <clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>>
>>> Subject: Re: [clamav-users] Scan very slow
>>> Date: Mon, 18 Mar 2019 12:30:49 +0100
>>>
>>> > Isn't it the second scan result ? The second analyse on same file is faster.
>>> > Could tou try to restart clamav-daemon and re-do the analyse with clamdscan.
>>> > I've tried it on 3 computers, all are above 40seconds
>>>
>>> It was first trial. But after restarting clamav-daemon result changed
>>> as following.
>>>
>>> yasu@kusanagi[1716]% clamdscan esploso_A3TH.pdf
>>> /home/yasu/tmp/esploso_A3TH.pdf: OK
>>>
>>> ----------- SCAN SUMMARY -----------
>>> Infected files: 0
>>> Time: 60.551 sec (1 m 0 s)
>>> yasu@kusanagi[1717]%
>>>
>>> ---
>>> Yasuhiro KIMURA
>>>
>>> _______________________________________________
>>>
>>> clamav-users mailing list
>>> clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>
>>> https://lists.clamav.net/mailman/listinfo/clamav-users <https://lists.clamav.net/mailman/listinfo/clamav-users>
>>>
>>>
>>> Help us build a comprehensive ClamAV guide:
>>> https://github.com/vrtadmin/clamav-faq <https://github.com/vrtadmin/clamav-faq>
>>>
>>> http://www.clamav.net/contact.html#ml <http://www.clamav.net/contact.html#ml>
>>
>> _______________________________________________
>>
>> clamav-users mailing list
>> clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>
>> https://lists.clamav.net/mailman/listinfo/clamav-users <https://lists.clamav.net/mailman/listinfo/clamav-users>
>>
>>
>> Help us build a comprehensive ClamAV guide:
>> https://github.com/vrtadmin/clamav-faq <https://github.com/vrtadmin/clamav-faq>
>>
>> http://www.clamav.net/contact.html#ml <http://www.clamav.net/contact.html#ml>
>
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net <mailto:clamav-users@lists.clamav.net>
> https://lists.clamav.net/mailman/listinfo/clamav-users <https://lists.clamav.net/mailman/listinfo/clamav-users>
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq <https://github.com/vrtadmin/clamav-faq>
>
> http://www.clamav.net/contact.html#ml <http://www.clamav.net/contact.html#ml>

Hi all,

We've been experiencing this slowdown too. We run every DB update through
an extra FP test against a number of recent Mac OS installs (OS X 10.6 -
10.14, as well as some well-known 3rd party apps) just to weed out any
potentially overzealous signatures. On the new Mac Minis this FP test used
to take around 45 minutes to complete, it now takes almost 3 hours (176
minutes).

From our logs, looking only at the smallest disk we check (Mac OS X 10.6.8)
I've managed to compile the following list of dates, scan times and ClamAV
DB version numbers. For months, the 10.6 disk used to take around 3m 20s to
scan. It always jumped up and down a bit, but really hasn't been right
since around the middle of February.

The list is best viewed using a mono-spaced font. I've marked (with 3
asterisks) scans where the time seems to indicate an issue with the DB
update.

Hopefully this helps someone to narrow things down a bit.
Mark

dd/mm/yy duration DNS Txt
5/2/19 3m 14s TXT from DNS: 0.101.1:58:25351:1549376940:1:63:48440:328
6/2/19 3m 20s TXT from DNS: 0.101.1:58:25352:1549466941:1:63:48444:328
11/2/19 3m 20s TXT from DNS: 0.101.1:58:25356:1549837740:1:63:48460:328
11/2/19 3m 25s TXT from DNS: 0.101.1:58:25356:1549877342:1:63:48462:328
11/2/19 3m 19s TXT from DNS: 0.101.1:58:25357:1549881900:1:63:48462:328
12/2/19 3m 22s TXT from DNS: 0.101.1:58:25357:1549963741:1:63:48466:328
13/2/19 3m 22s TXT from DNS: 0.101.1:58:25358:1550050141:1:63:48470:328
14/2/19 3m 22s TXT from DNS: 0.101.1:58:25359:1550140140:1:63:48472:328
16/2/19 6m 38s TXT from DNS: 0.101.1:58:25361:1550269740:1:63:48472:328 ***
17/2/19 7m 35s TXT from DNS: 0.101.1:58:25362:1550348940:1:63:48472:328
18/2/19 7m 41s TXT from DNS: 0.101.1:58:25363:1550442540:1:63:48472:328
18/2/19 4m 22s TXT from DNS: 0.101.1:58:25364:1550492940:1:63:48472:328
19/2/19 4m 28s TXT from DNS: 0.101.1:58:25365:1550579340:1:63:48472:328
20/2/19 4m 30s TXT from DNS: 0.101.1:58:25365:1550658540:1:63:48472:328
21/2/19 4m 28s TXT from DNS: 0.101.1:58:25366:1550744940:1:63:48472:328
22/2/19 4m 36s TXT from DNS: 0.101.1:58:25368:1550842141:1:63:48472:328
24/2/19 7m 51s TXT from DNS: 0.101.1:58:25370:1551040140:1:63:48472:328 ***
25/2/19 4m 31s TXT from DNS: 0.101.1:58:25371:1551092103:1:63:48472:328
26/2/19 4m 41s TXT from DNS: 0.101.1:58:25372:1551177619:1:63:48472:328
27/2/19 4m 29s TXT from DNS: 0.101.1:58:25373:1551277740:1:63:48472:328
28/2/19 4m 28s TXT from DNS: 0.101.1:58:25373:1551349740:1:63:48472:328
1/3/19 4m 39s TXT from DNS: 0.101.1:58:25374:1551443340:1:63:48472:328
3/3/19 8m 14s TXT from DNS: 0.101.1:58:25376:1551558540:1:63:48472:328 ***
3/3/19 8m 45s TXT from DNS: 0.101.1:58:25377:1551644940:1:63:48472:328
4/3/19 4m 51s TXT from DNS: 0.101.1:58:25377:1551691742:1:63:48472:328
4/3/19 4m 52s TXT from DNS: 0.101.1:58:25378:1551709740:1:63:48472:328
5/3/19 5m 6s TXT from DNS: 0.101.1:58:25379:1551796140:1:63:48472:328
6/3/19 5m 7s TXT from DNS: 0.101.1:58:25380:1551868140:1:63:48473:328
7/3/19 5m 15s TXT from DNS: 0.101.1:58:25381:1551953509:1:63:48474:328
8/3/19 5m 14s TXT from DNS: 0.101.1:58:25382:1552048140:1:63:48478:328
9/3/19 5m 7s TXT from DNS: 0.101.1:58:25383:1552163340:1:63:48482:328
11/3/19 5m 14s TXT from DNS: 0.101.1:58:25384:1552253340:1:63:48485:328
11/3/19 5m 24s TXT from DNS: 0.101.1:58:25385:1552302125:1:63:48487:328
12/3/19 5m 42s TXT from DNS: 0.101.1:58:25386:1552388890:1:63:48490:328
13/3/19 5m 44s TXT from DNS: 0.101.1:58:25386:1552465741:1:63:48492:328
14/3/19 7m 24s TXT from DNS: 0.101.1:58:25388:1552559341:1:63:48495:328 ***
15/3/19 8m 56s TXT from DNS: 0.101.1:58:25389:1552645741:1:63:48498:328 ***
18/3/19 10m 49s TXT from DNS: 0.101.1:58:25392:1552904941:1:63:48507:328 ***
19/3/19 10m 19s TXT from DNS: 0.101.1:58:25393:1552991341:1:63:48510:328
20/3/19 10m 43s TXT from DNS: 0.101.1:58:25394:1553074140:1:63:48513:328
22/3/19 10m 58s TXT from DNS: 0.101.1:58:25395:1553180408:1:63:48517:328
22/3/19 10m 58s TXT from DNS: 0.101.1:58:25396:1553246940:1:63:48519:328




On Sat, 23 Mar 2019 at 23:26, Al Varnell via clamav-users <
clamav-users@lists.clamav.net> wrote:

> Sorry, I misinterpreted the meaning of "crawled" thinking it referred to
> some sort of compromise of the data.
>
> -Al-
>
> On Mar 23, 2019, at 09:42, Jean-Michel via clamav-users <
> clamav-users@lists.clamav.net> wrote:
>
> See Maarten Broekman tests above
> https://lists.clamav.net/pipermail/clamav-users/2019-March/007737.html
>
> *De :* Al Varnell <alvarnell@mac.com>
> *Envoyé :* samedi 23 mars 2019 10:55
> *À :* ClamAV users ML <clamav-users@lists.clamav.net>
> *Objet :* Re: [clamav-users] Scan very slow
>
> Reference? First I'm hearing of any such thing.
>
> -Al-
>
>
> On Mar 23, 2019, at 02:26, Jean-Michel via clamav-users <
> clamav-users@lists.clamav.net> wrote:
>
> Hi,
> Micah Snyder, Do you know if Clamav was able to trace the orgine of
> getting crawled in the database "daily.cld" and was able to fix the problem?
> Regards
>
> *De :* Micah Snyder (micasnyd) <micasnyd@cisco.com>
> *Envoyé :* lundi 18 mars 2019 18:09
> *À :* ClamAV users ML <clamav-users@lists.clamav.net>
> *Objet :* Re: [clamav-users] Scan very slow
>
> Maarten,
>
> This is very concerning, and the details you’ve provide are quite
> helpful. Thank you for investigating.
> Hopefully we can figure out why the newer daily.cld/cvd is scanning
> significantly slower than before. Any other details you can provide would
> probably be helpful. If you’re aware if any specific file types are
> causing the issue, or if all files appear to scanning slower that will also
> help.
>
> -Micah
>
>
> Micah Snyder
> ClamAV Development
> Talos
> Cisco Systems, Inc.
>
>
>
> *From: *clamav-users <clamav-users-bounces@lists.clamav.net> on behalf of
> Maarten Broekman via clamav-users <clamav-users@lists.clamav.net>
> *Reply-To: *ClamAV users ML <clamav-users@lists.clamav.net>
> *Date: *Monday, March 18, 2019 at 10:37 AM
> *To: *ClamAV users ML <clamav-users@lists.clamav.net>
> *Cc: *Maarten Broekman <maarten.broekman@gmail.com>
> *Subject: *Re: [clamav-users] Scan very slow
>
> We've noticed a marked increase in scan times over the last couple of
> weeks as well. From the look of it, there's something in the daily file
> that's causing it. Whether this is similar to the safebrowsing issue (where
> the ordering of entries in the file caused a 3000% increase in time) is
> unclear.
>
> --Maarten Broekman
>
> Full scans without the daily cvd/cld: Scan time ~60seconds
> Full scans with the daily from March 11th: Scan time: 84seconds
> Full scans with the daily from March 17th: Scan time: 109seconds
>
> ~/clamav# ls -larth /tmp/clamdtest*/daily.cld
> -rw-r--r-- 1 clamav clamav 110M Mar 11 04:15 /tmp/clamdtest2/daily.cld
> -rw-r--r-- 1 clamav clamav 113M Mar 17 04:15 /tmp/clamdtest/daily.cld
>
> ~/clamav# wc /tmp/clamdtest*/daily.cld
> 1514589 1517471 115031552 /tmp/clamdtest2/daily.cld
> 1524782 1527664 118202368 /tmp/clamdtest/daily.cld
>
> Single file scans with JUST the daily.cld:
> ~/clamav# time /opt/clamav/clamav/bin/clamscan -d
> /tmp/clamdtest2/daily.cld test42.js
> test42.js: OK
>
> ----------- SCAN SUMMARY -----------
> Known viruses: 1504423
> Engine version: 0.100.2
> Scanned directories: 1
> Scanned files: 1
> Infected files: 0
> Data scanned: 0.00 MB
> Data read: 0.00 MB (ratio 0.00:1)
> Time: 5.255 sec (0 m 5 s)
>
> real 0m5.260s
> user 0m5.044s
> sys 0m0.192s
> ~/clamav# time /opt/clamav/clamav/bin/clamscan -d /tmp/clamdtest/daily.cld
> test42.js
> test42.js: OK
>
> ----------- SCAN SUMMARY -----------
> Known viruses: 1514543
> Engine version: 0.100.2
> Scanned directories: 1
> Scanned files: 1
> Infected files: 0
> Data scanned: 0.00 MB
> Data read: 0.00 MB (ratio 0.00:1)
> Time: 9.300 sec (0 m 9 s)
>
> real 0m9.329s
> user 0m9.100s
> sys 0m0.204s
>
>
>
>
>
>
> On Mon, Mar 18, 2019 at 10:02 AM Yasuhiro KIMURA <yasu@utahime.org> wrote:
>
> From: Jean-Michel via clamav-users <clamav-users@lists.clamav.net>
> Subject: Re: [clamav-users] Scan very slow
> Date: Mon, 18 Mar 2019 12:30:49 +0100
>
> > Isn't it the second scan result ? The second analyse on same file is
> faster.
> > Could tou try to restart clamav-daemon and re-do the analyse with
> clamdscan.
> > I've tried it on 3 computers, all are above 40seconds
>
> It was first trial. But after restarting clamav-daemon result changed
> as following.
>
> yasu@kusanagi[1716]% clamdscan esploso_A3TH.pdf
> /home/yasu/tmp/esploso_A3TH.pdf: OK
>
> ----------- SCAN SUMMARY -----------
> Infected files: 0
> Time: 60.551 sec (1 m 0 s)
> yasu@kusanagi[1717]%
>
> ---
> Yasuhiro KIMURA
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>
>
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>
>
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>

On 2019-03-25 10:52, Mark Allan via clamav-users wrote:
> Hi all,
>
te.
>
> Hopefully this helps someone to narrow things down a bit.
>
> Mark
>

18/3/19 10m 49s TXT from DNS:
0.101.1:58:25392:1552904941:1:63:48507:328 ***

Here's the changes for the above update:

https://lists.gt.net/clamav/virusdb/75154

You can also check sigs quickly per update:

https://lists.gt.net/clamav/virusdb/



--
Cheers,

Steve
Twitter: @sanesecurity

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Cheers Steve,

In the interest of completeness, here's the scan from today (TXT from DNS:
0.101.1:58:25399:1553509741:1:63:48528:328) showing a marked improvement in
scan time, although at 6m 7s it's still almost twice what it used to be.

Mark

On Mon, 25 Mar 2019 at 12:56, Steve Basford <steveb_clamav@sanesecurity.com>
wrote:

> On 2019-03-25 10:52, Mark Allan via clamav-users wrote:
> > Hi all,
> >
> te.
> >
> > Hopefully this helps someone to narrow things down a bit.
> >
> > Mark
> >
>
> 18/3/19 10m 49s TXT from DNS:
> 0.101.1:58:25392:1552904941:1:63:48507:328 ***
>
> Here's the changes for the above update:
>
> https://lists.gt.net/clamav/virusdb/75154
>
> You can also check sigs quickly per update:
>
> https://lists.gt.net/clamav/virusdb/
>
>
>
> --
> Cheers,
>
> Steve
> Twitter: @sanesecurity
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>

Hi Mark, all:

I’m disappointed to hear that it is still slow for you.

We found that the target-type of signatures used for PhishTank.Phishing signatures were causing a significant slowdown. We have dropped them as of this past Saturday ( https://lists.gt.net/clamav/virusdb/75279 ) and in the last two updates have been re-adding them with more specific scan target types. We’re now investigating some other optimizations we can make for the next major ClamAV release to improve scan times but at present we don’t have any other leads for signatures that may be slowing down scans.

Regards,
Micah


From: clamav-users <clamav-users-bounces@lists.clamav.net> on behalf of Mark Allan via clamav-users <clamav-users@lists.clamav.net>
Reply-To: ClamAV users ML <clamav-users@lists.clamav.net>
Date: Monday, March 25, 2019 at 9:37 AM
To: ClamAV users ML <clamav-users@lists.clamav.net>
Cc: Mark Allan <markjallan@gmail.com>
Subject: Re: [clamav-users] Scan very slow

Cheers Steve,

In the interest of completeness, here's the scan from today (TXT from DNS: 0.101.1:58:25399:1553509741:1:63:48528:328) showing a marked improvement in scan time, although at 6m 7s it's still almost twice what it used to be.

Mark

On Mon, 25 Mar 2019 at 12:56, Steve Basford <steveb_clamav@sanesecurity.com<mailto:steveb_clamav@sanesecurity.com>> wrote:
On 2019-03-25 10:52, Mark Allan via clamav-users wrote:
> Hi all,
>
te.
>
> Hopefully this helps someone to narrow things down a bit.
>
> Mark
>

18/3/19 10m 49s TXT from DNS:
0.101.1:58:25392:1552904941:1:63:48507:328 ***

Here's the changes for the above update:

https://lists.gt.net/clamav/virusdb/75154

You can also check sigs quickly per update:

https://lists.gt.net/clamav/virusdb/



--
Cheers,

Steve
Twitter: @sanesecurity

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Hi Micah

It seems that the scanning slow down issue of this time has been solved
at some level with CVD Update of the other day.
However, there is still big discrepancy in between the current condition and
the last condition in one month ago.

Date Files Scan time
2019/02/15 2550338 08:53:57
2019/03/15 2612792 19:22:54
2019/03/26 2634489 18:13:56
2019/03/27 2637201 18:10:05

We know the improvement of this time is due to the details of CVD, because
we did not make any change on the user's system.
We are going to try some tuning for scanning.

We like to know if you still have some room to make further improvement
for this slow down issue.
Thank you for your help, in advance.

Best regards,
Oya

On Mon, 25 Mar 2019 15:45:02 +0000
"Micah Snyder \(micasnyd\) via clamav-users" <clamav-users@lists.clamav.net> wrote:

> Hi Mark, all:
>
> I$B!G(Bm disappointed to hear that it is still slow for you.
>
> We found that the target-type of signatures used for PhishTank.Phishing signatures were causing a significant slowdown. We have dropped them as of this past Saturday ( https://lists.gt.net/clamav/virusdb/75279 ) and in the last two updates have been re-adding them with more specific scan target types. We$B!G(Bre now investigating some other optimizations we can make for the next major ClamAV release to improve scan times but at present we don$B!G(Bt have any other leads for signatures that may be slowing down scans.
>
> Regards,
> Micah
>
>
> From: clamav-users <clamav-users-bounces@lists.clamav.net> on behalf of Mark Allan via clamav-users <clamav-users@lists.clamav.net>
> Reply-To: ClamAV users ML <clamav-users@lists.clamav.net>
> Date: Monday, March 25, 2019 at 9:37 AM
> To: ClamAV users ML <clamav-users@lists.clamav.net>
> Cc: Mark Allan <markjallan@gmail.com>
> Subject: Re: [clamav-users] Scan very slow
>
> Cheers Steve,
>
> In the interest of completeness, here's the scan from today (TXT from DNS: 0.101.1:58:25399:1553509741:1:63:48528:328) showing a marked improvement in scan time, although at 6m 7s it's still almost twice what it used to be.
>
> Mark
>
> On Mon, 25 Mar 2019 at 12:56, Steve Basford <steveb_clamav@sanesecurity.com<mailto:steveb_clamav@sanesecurity.com>> wrote:
> On 2019-03-25 10:52, Mark Allan via clamav-users wrote:
> > Hi all,
> >
> te.
> >
> > Hopefully this helps someone to narrow things down a bit.
> >
> > Mark
> >
>
> 18/3/19 10m 49s TXT from DNS:
> 0.101.1:58:25392:1552904941:1:63:48507:328 ***
>
> Here's the changes for the above update:
>
> https://lists.gt.net/clamav/virusdb/75154
>
> You can also check sigs quickly per update:
>
> https://lists.gt.net/clamav/virusdb/
>
>
>
> --
> Cheers,
>
> Steve
> Twitter: @sanesecurity
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml



_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Thanks Oya for the update. We will continue to investigate the signature performance issue.

Regards,
Micah

?On 3/28/19, 9:50 AM, "clamav-users on behalf of Tsutomu Oyamada" <clamav-users-bounces@lists.clamav.net on behalf of oyamada@promark-inc.com> wrote:

Hi Micah

It seems that the scanning slow down issue of this time has been solved
at some level with CVD Update of the other day.
However, there is still big discrepancy in between the current condition and
the last condition in one month ago.

Date Files Scan time
2019/02/15 2550338 08:53:57
2019/03/15 2612792 19:22:54
2019/03/26 2634489 18:13:56
2019/03/27 2637201 18:10:05

We know the improvement of this time is due to the details of CVD, because
we did not make any change on the user's system.
We are going to try some tuning for scanning.

We like to know if you still have some room to make further improvement
for this slow down issue.
Thank you for your help, in advance.

Best regards,
Oya

On Mon, 25 Mar 2019 15:45:02 +0000
"Micah Snyder \(micasnyd\) via clamav-users" <clamav-users@lists.clamav.net> wrote:

> Hi Mark, all:
>
> I’m disappointed to hear that it is still slow for you.
>
> We found that the target-type of signatures used for PhishTank.Phishing signatures were causing a significant slowdown. We have dropped them as of this past Saturday ( https://lists.gt.net/clamav/virusdb/75279 ) and in the last two updates have been re-adding them with more specific scan target types. We’re now investigating some other optimizations we can make for the next major ClamAV release to improve scan times but at present we don’t have any other leads for signatures that may be slowing down scans.
>
> Regards,
> Micah
>
>
> From: clamav-users <clamav-users-bounces@lists.clamav.net> on behalf of Mark Allan via clamav-users <clamav-users@lists.clamav.net>
> Reply-To: ClamAV users ML <clamav-users@lists.clamav.net>
> Date: Monday, March 25, 2019 at 9:37 AM
> To: ClamAV users ML <clamav-users@lists.clamav.net>
> Cc: Mark Allan <markjallan@gmail.com>
> Subject: Re: [clamav-users] Scan very slow
>
> Cheers Steve,
>
> In the interest of completeness, here's the scan from today (TXT from DNS: 0.101.1:58:25399:1553509741:1:63:48528:328) showing a marked improvement in scan time, although at 6m 7s it's still almost twice what it used to be.
>
> Mark
>
> On Mon, 25 Mar 2019 at 12:56, Steve Basford <steveb_clamav@sanesecurity.com<mailto:steveb_clamav@sanesecurity.com>> wrote:
> On 2019-03-25 10:52, Mark Allan via clamav-users wrote:
> > Hi all,
> >
> te.
> >
> > Hopefully this helps someone to narrow things down a bit.
> >
> > Mark
> >
>
> 18/3/19 10m 49s TXT from DNS:
> 0.101.1:58:25392:1552904941:1:63:48507:328 ***
>
> Here's the changes for the above update:
>
> https://lists.gt.net/clamav/virusdb/75154
>
> You can also check sigs quickly per update:
>
> https://lists.gt.net/clamav/virusdb/
>
>
>
> --
> Cheers,
>
> Steve
> Twitter: @sanesecurity
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml



_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Also CC'ing Micah directly as the mailing list would appear to be offline
(at least lists.clamav.net isn't responding to http requests anyway)

It looks like scan times have gone through the roof. As Oya said, they're
still considerably higher than they were a couple of months ago, but
today's scan time is insane.

Yesterday's scan using
0.101.2:58:25409:1554370140:1:63:48554:328
took 7m 3s

On the same hardware, scanning the same read-only disk image, with today's
scan using
0.101.2:58:25410:1554452941:1:63:48557:328
the scan time has jumped to 26m 15s

This is the longest it has ever taken to scan this volume (cf my previous
email of 25th March)

Is there anything that can be excluded?

Best regards
Mark

On Mon, 1 Apr 2019 at 17:11, Micah Snyder (micasnyd) via clamav-users <
clamav-users@lists.clamav.net> wrote:

> Thanks Oya for the update. We will continue to investigate the signature
> performance issue.
>
> Regards,
> Micah
>
> ?On 3/28/19, 9:50 AM, "clamav-users on behalf of Tsutomu Oyamada" <
> clamav-users-bounces@lists.clamav.net on behalf of oyamada@promark-inc.com>
> wrote:
>
> Hi Micah
>
> It seems that the scanning slow down issue of this time has been
> solved
> at some level with CVD Update of the other day.
> However, there is still big discrepancy in between the current
> condition and
> the last condition in one month ago.
>
> Date Files Scan time
> 2019/02/15 2550338 08:53:57
> 2019/03/15 2612792 19:22:54
> 2019/03/26 2634489 18:13:56
> 2019/03/27 2637201 18:10:05
>
> We know the improvement of this time is due to the details of CVD,
> because
> we did not make any change on the user's system.
> We are going to try some tuning for scanning.
>
> We like to know if you still have some room to make further improvement
> for this slow down issue.
> Thank you for your help, in advance.
>
> Best regards,
> Oya
>
> On Mon, 25 Mar 2019 15:45:02 +0000
> "Micah Snyder \(micasnyd\) via clamav-users" <
> clamav-users@lists.clamav.net> wrote:
>
> > Hi Mark, all:
> >
> > I’m disappointed to hear that it is still slow for you.
> >
> > We found that the target-type of signatures used for
> PhishTank.Phishing signatures were causing a significant slowdown. We
> have dropped them as of this past Saturday (
> https://lists.gt.net/clamav/virusdb/75279 ) and in the last two updates
> have been re-adding them with more specific scan target types. We’re now
> investigating some other optimizations we can make for the next major
> ClamAV release to improve scan times but at present we don’t have any other
> leads for signatures that may be slowing down scans.
> >
> > Regards,
> > Micah
> >
> >
> > From: clamav-users <clamav-users-bounces@lists.clamav.net> on
> behalf of Mark Allan via clamav-users <clamav-users@lists.clamav.net>
> > Reply-To: ClamAV users ML <clamav-users@lists.clamav.net>
> > Date: Monday, March 25, 2019 at 9:37 AM
> > To: ClamAV users ML <clamav-users@lists.clamav.net>
> > Cc: Mark Allan <markjallan@gmail.com>
> > Subject: Re: [clamav-users] Scan very slow
> >
> > Cheers Steve,
> >
> > In the interest of completeness, here's the scan from today (TXT
> from DNS: 0.101.1:58:25399:1553509741:1:63:48528:328) showing a marked
> improvement in scan time, although at 6m 7s it's still almost twice what it
> used to be.
> >
> > Mark
> >
> > On Mon, 25 Mar 2019 at 12:56, Steve Basford <
> steveb_clamav@sanesecurity.com<mailto:steveb_clamav@sanesecurity.com>>
> wrote:
> > On 2019-03-25 10:52, Mark Allan via clamav-users wrote:
> > > Hi all,
> > >
> > te.
> > >
> > > Hopefully this helps someone to narrow things down a bit.
> > >
> > > Mark
> > >
> >
> > 18/3/19 10m 49s TXT from DNS:
> > 0.101.1:58:25392:1552904941:1:63:48507:328 ***
> >
> > Here's the changes for the above update:
> >
> > https://lists.gt.net/clamav/virusdb/75154
> >
> > You can also check sigs quickly per update:
> >
> > https://lists.gt.net/clamav/virusdb/
> >
> >
> >
> > --
> > Cheers,
> >
> > Steve
> > Twitter: @sanesecurity
> >
> > _______________________________________________
> >
> > clamav-users mailing list
> > clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>
> > https://lists.clamav.net/mailman/listinfo/clamav-users
> >
> >
> > Help us build a comprehensive ClamAV guide:
> > https://github.com/vrtadmin/clamav-faq
> >
> > http://www.clamav.net/contact.html#ml
>
>
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>
>
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>

> On Apr 5, 2019, at 09:13, Mark Allan via clamav-users <clamav-users@lists.clamav.net> wrote:
>
> Also CC'ing Micah directly as the mailing list would appear to be offline (at least lists.clamav.net isn't responding to http requests anyway

May want to try https.

Already tried that.



Mark

On Fri, 5 Apr 2019 at 14:20, Joel Esler (jesler) <jesler@cisco.com> wrote:

>
> On Apr 5, 2019, at 09:13, Mark Allan via clamav-users <
> clamav-users@lists.clamav.net> wrote:
>
> Also CC'ing Micah directly as the mailing list would appear to be offline
> (at least lists.clamav.net isn't responding to http requests anyway
>
>
> May want to try https.
>

Works for me.

Sent from my ? iPad

> On Apr 5, 2019, at 09:32, Mark Allan <markjallan@gmail.com> wrote:
>
> Already tried that.
>
>
>
> Mark
>
>> On Fri, 5 Apr 2019 at 14:20, Joel Esler (jesler) <jesler@cisco.com> wrote:
>>
>>> On Apr 5, 2019, at 09:13, Mark Allan via clamav-users <clamav-users@lists.clamav.net> wrote:
>>>
>>> Also CC'ing Micah directly as the mailing list would appear to be offline (at least lists.clamav.net isn't responding to http requests anyway
>>
>> May want to try https.
> <Screenshot 2019-04-05 at 2.31.01 pm.png>

We're getting sidetracked here. The real issue is the scan times with the
latest DB.

On Fri, 5 Apr 2019 at 14:58, Joel Esler (jesler) <jesler@cisco.com> wrote:

> [image: image1.png]
>
> Works for me.
>
> Sent from my ? iPad
>
> On Apr 5, 2019, at 09:32, Mark Allan <markjallan@gmail.com> wrote:
>
> Already tried that.
>
>
>
> Mark
>
> On Fri, 5 Apr 2019 at 14:20, Joel Esler (jesler) <jesler@cisco.com> wrote:
>
>>
>> On Apr 5, 2019, at 09:13, Mark Allan via clamav-users <
>> clamav-users@lists.clamav.net> wrote:
>>
>> Also CC'ing Micah directly as the mailing list would appear to be offline
>> (at least lists.clamav.net isn't responding to http requests anyway
>>
>>
>> May want to try https.
>>
> <Screenshot 2019-04-05 at 2.31.01 pm.png>
>
>

Hi Mark,

Sorry about the delay in responding. I hadn’t looked at my clamav-users filter this morning. Just investigating now. Will respond when I know more.

-Micah

From: Mark Allan <markjallan@gmail.com>
Date: Friday, April 5, 2019 at 9:12 AM
To: ClamAV users ML <clamav-users@lists.clamav.net>, "Micah Snyder (micasnyd)" <micasnyd@cisco.com>
Subject: Re: [clamav-users] Scan very slow

Also CC'ing Micah directly as the mailing list would appear to be offline (at least lists.clamav.net<http://lists.clamav.net> isn't responding to http requests anyway)

It looks like scan times have gone through the roof. As Oya said, they're still considerably higher than they were a couple of months ago, but today's scan time is insane.

Yesterday's scan using
0.101.2:58:25409:1554370140:1:63:48554:328
took 7m 3s

On the same hardware, scanning the same read-only disk image, with today's scan using
0.101.2:58:25410:1554452941:1:63:48557:328
the scan time has jumped to 26m 15s

This is the longest it has ever taken to scan this volume (cf my previous email of 25th March)

Is there anything that can be excluded?

Best regards
Mark

On Mon, 1 Apr 2019 at 17:11, Micah Snyder (micasnyd) via clamav-users <clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>> wrote:
Thanks Oya for the update. We will continue to investigate the signature performance issue.

Regards,
Micah

On 3/28/19, 9:50 AM, "clamav-users on behalf of Tsutomu Oyamada" <clamav-users-bounces@lists.clamav.net<mailto:clamav-users-bounces@lists.clamav.net> on behalf of oyamada@promark-inc.com<mailto:oyamada@promark-inc.com>> wrote:

Hi Micah

It seems that the scanning slow down issue of this time has been solved
at some level with CVD Update of the other day.
However, there is still big discrepancy in between the current condition and
the last condition in one month ago.

Date Files Scan time
2019/02/15 2550338 08:53:57
2019/03/15 2612792 19:22:54
2019/03/26 2634489 18:13:56
2019/03/27 2637201 18:10:05

We know the improvement of this time is due to the details of CVD, because
we did not make any change on the user's system.
We are going to try some tuning for scanning.

We like to know if you still have some room to make further improvement
for this slow down issue.
Thank you for your help, in advance.

Best regards,
Oya

On Mon, 25 Mar 2019 15:45:02 +0000
"Micah Snyder \(micasnyd\) via clamav-users" <clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>> wrote:

> Hi Mark, all:
>
> I’m disappointed to hear that it is still slow for you.
>
> We found that the target-type of signatures used for PhishTank.Phishing signatures were causing a significant slowdown. We have dropped them as of this past Saturday ( https://lists.gt.net/clamav/virusdb/75279 ) and in the last two updates have been re-adding them with more specific scan target types. We’re now investigating some other optimizations we can make for the next major ClamAV release to improve scan times but at present we don’t have any other leads for signatures that may be slowing down scans.
>
> Regards,
> Micah
>
>
> From: clamav-users <clamav-users-bounces@lists.clamav.net<mailto:clamav-users-bounces@lists.clamav.net>> on behalf of Mark Allan via clamav-users <clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>>
> Reply-To: ClamAV users ML <clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>>
> Date: Monday, March 25, 2019 at 9:37 AM
> To: ClamAV users ML <clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>>
> Cc: Mark Allan <markjallan@gmail.com<mailto:markjallan@gmail.com>>
> Subject: Re: [clamav-users] Scan very slow
>
> Cheers Steve,
>
> In the interest of completeness, here's the scan from today (TXT from DNS: 0.101.1:58:25399:1553509741:1:63:48528:328) showing a marked improvement in scan time, although at 6m 7s it's still almost twice what it used to be.
>
> Mark
>
> On Mon, 25 Mar 2019 at 12:56, Steve Basford <steveb_clamav@sanesecurity.com<mailto:steveb_clamav@sanesecurity.com><mailto:steveb_clamav@sanesecurity.com<mailto:steveb_clamav@sanesecurity.com>>> wrote:
> On 2019-03-25 10:52, Mark Allan via clamav-users wrote:
> > Hi all,
> >
> te.
> >
> > Hopefully this helps someone to narrow things down a bit.
> >
> > Mark
> >
>
> 18/3/19 10m 49s TXT from DNS:
> 0.101.1:58:25392:1552904941:1:63:48507:328 ***
>
> Here's the changes for the above update:
>
> https://lists.gt.net/clamav/virusdb/75154
>
> You can also check sigs quickly per update:
>
> https://lists.gt.net/clamav/virusdb/
>
>
>
> --
> Cheers,
>
> Steve
> Twitter: @sanesecurity
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net><mailto:clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>>
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml



_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml



_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml