Mailing List Archive: [clamav-users] OnAccess renders system unusable in ~24h

[clamav-users] OnAccess renders system unusable in ~24h

Sep 24, 2019, 5:37 AM

Post #1 of 5 (206 views)

Hi,
I am running clamd with OnAccess enabled, however its causing the load
on the systems to make them almost unusable within about 24hours.

as you can see sys is at 98%, it seem clamd is stopping other
applications from processing somehow. cannot find anything in the logs.
not sure what debugging would be helpful? any advice would be helpful
here?

# top -bH -d 5 -n 49 -p 134865
top - 12:25:11 up 60 days, 4:29, 1 user, load average: 7.19, 6.92,
6.71
Threads: 3 total, 0 running, 3 sleeping, 0 stopped, 0 zombie
%Cpu(s): 2.7 us, 70.3 sy, 0.0 ni, 27.0 id, 0.0 wa, 0.0 hi, 0.0
si, 0.0 st
KiB Mem : 5908584 total, 251604 free, 4134244 used, 1522736
buff/cache
KiB Swap: 6291452 total, 5947900 free, 343552 used. 1004892 avail
Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+
COMMAND
134865 root 35 15 1114568 745700 612 S 0.0 12.6 3:20.05
clamd
134867 root 20 0 1114568 745700 612 S 0.0 12.6 0:00.00
clamd
134868 root 20 0 1114568 745700 612 S 0.0 12.6 1:23.39
clamd

# clamconf
Checking configuration files in /etc

Config file: clamd.d/scan.conf
------------------------------
AlertExceedsMax disabled
PreludeEnable disabled
PreludeAnalyzerName disabled
LogFile = "/var/log/clamav/clamav.log"
LogFileUnlock = "yes"
LogFileMaxSize = "1048576"
LogTime = "yes"
LogClean disabled
LogSyslog disabled
LogFacility = "LOG_LOCAL6"
LogVerbose disabled
LogRotate disabled
ExtendedDetectionInfo = "yes"
PidFile = "/var/run/clamd.scan/clamd.pid"
TemporaryDirectory disabled
DatabaseDirectory = "/var/lib/clamav"
OfficialDatabaseOnly disabled
LocalSocket = "/var/run/clamd.scan/clamd.sock"
LocalSocketGroup = "virusgroup"
LocalSocketMode = "666"
FixStaleSocket = "yes"
TCPSocket disabled
TCPAddr disabled
MaxConnectionQueueLength = "200"
StreamMaxLength = "26214400"
StreamMinPort = "1024"
StreamMaxPort = "2048"
MaxThreads = "10"
ReadTimeout = "180"
CommandReadTimeout = "30"
SendBufTimeout = "200"
MaxQueue = "100"
IdleTimeout = "30"
ExcludePath disabled
MaxDirectoryRecursion = "20"
FollowDirectorySymlinks disabled
FollowFileSymlinks disabled
CrossFilesystems = "yes"
SelfCheck = "600"
DisableCache disabled
VirusEvent disabled
ExitOnOOM = "yes"
AllowAllMatchScan = "yes"
Foreground disabled
Debug disabled
LeaveTemporaryFiles disabled
User = "root"
Bytecode = "yes"
BytecodeSecurity = "TrustSigned"
BytecodeTimeout = "5000"
BytecodeUnsigned disabled
BytecodeMode = "Auto"
DetectPUA disabled
ExcludePUA disabled
IncludePUA disabled
ScanPE = "yes"
ScanELF = "yes"
ScanMail = "yes"
ScanPartialMessages disabled
PhishingSignatures = "yes"
PhishingScanURLs = "yes"
HeuristicAlerts = "yes"
HeuristicScanPrecedence disabled
StructuredDataDetection disabled
StructuredMinCreditCardCount = "3"
StructuredMinSSNCount = "3"
StructuredSSNFormatNormal = "yes"
StructuredSSNFormatStripped disabled
ScanHTML = "yes"
ScanOLE2 = "yes"
AlertBrokenExecutables disabled
AlertEncrypted disabled
AlertEncryptedArchive disabled
AlertEncryptedDoc disabled
AlertOLE2Macros disabled
AlertPhishingSSLMismatch disabled
AlertPhishingCloak disabled
AlertPartitionIntersection disabled
ScanPDF = "yes"
ScanSWF = "yes"
ScanXMLDOCS = "yes"
ScanHWP3 = "yes"
ScanArchive = "yes"
ForceToDisk disabled
MaxScanTime disabled
MaxScanSize = "104857600"
MaxFileSize = "26214400"
MaxRecursion = "16"
MaxFiles = "10000"
MaxEmbeddedPE = "10485760"
MaxHTMLNormalize = "10485760"
MaxHTMLNoTags = "2097152"
MaxScriptNormalize = "5242880"
MaxZipTypeRcg = "1048576"
MaxPartitions = "50"
MaxIconsPE = "100"
MaxRecHWP3 = "16"
PCREMatchLimit = "100000"
PCRERecMatchLimit = "2000"
PCREMaxFileSize = "26214400"
ScanOnAccess = "yes"
OnAccessMountPath disabled
OnAccessIncludePath = "/home", "/root", "/etc", "/sftp", "/boot",
"/opt", "/media", "/mnt"
OnAccessExcludePath disabled
OnAccessExcludeRootUID = "yes"
OnAccessExcludeUID disabled
OnAccessMaxFileSize = "5242880"
OnAccessDisableDDD disabled
OnAccessPrevention disabled
OnAccessExtraScanning = "yes"
DevACOnly disabled
DevACDepth disabled
DevPerformance disabled
DevLiblog disabled
DisableCertCheck disabled
AlgorithmicDetection = "yes"
BlockMax disabled
PhishingAlwaysBlockSSLMismatch disabled
PhishingAlwaysBlockCloak disabled
PartitionIntersection disabled
OLE2BlockMacros disabled
ArchiveBlockEncrypted disabled

Config file: freshclam.conf
---------------------------
LogFileMaxSize = "2097152"
LogTime = "yes"
LogSyslog = "yes"
LogFacility = "LOG_LOCAL6"
LogVerbose disabled
LogRotate = "yes"
PidFile = "/var/run/freshclam.pid"
DatabaseDirectory = "/var/lib/clamav"
Foreground disabled
Debug disabled
UpdateLogFile = "/var/log/freshclam.log"
DatabaseOwner = "clamupdate"
Checks = "24"
DNSDatabaseInfo = "current.cvd.clamav.net"
DatabaseMirror = "172.16.22.8"
PrivateMirror disabled
MaxAttempts = "5"
ScriptedUpdates = "yes"
TestDatabases = "yes"
CompressLocalDatabase disabled
ExtraDatabase disabled
DatabaseCustomURL disabled
HTTPProxyServer disabled
HTTPProxyPort disabled
HTTPProxyUsername disabled
HTTPProxyPassword disabled
HTTPUserAgent disabled
NotifyClamd = "/etc/clamd.d/scan.conf"
OnUpdateExecute disabled
OnErrorExecute disabled
OnOutdatedExecute disabled
LocalIPAddress disabled
ConnectTimeout = "30"
ReceiveTimeout = "30"
SafeBrowsing disabled
Bytecode = "yes"

mail/clamav-milter.conf not found

Software settings
-----------------
Version: 0.101.4
Optional features supported: MEMPOOL IPv6 AUTOIT_EA06 BZIP2 LIBXML2
PCRE2 ICONV JSON

Database information
--------------------
Database directory: /var/lib/clamav
main.cvd: version 58, sigs: 4566249, built on Wed Jun 7 21:38:10 2017
bytecode.cld: version 331, sigs: 94, built on Thu Sep 19 16:12:33 2019
daily.cld: version 25581, sigs: 1776056, built on Mon Sep 23 08:20:21
2019
Total number of signatures: 6342399

Platform information
--------------------
uname: Linux 3.10.0-957.21.3.el7.x86_64 #1 SMP Tue Jun 18 16:35:19 UTC
2019 x86_64
OS: linux-gnu, ARCH: x86_64, CPU: x86_64
zlib version: 1.2.7 (1.2.7), compile flags: a9
platform id: 0x0a2169690800000002040805

Build information
-----------------
GNU C: 4.8.5 20150623 (Red Hat 4.8.5-39) (4.8.5)
CPPFLAGS:
CFLAGS: -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-
switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64
-mtune=generic -fno-strict-aliasing -D_LARGEFILE_SOURCE
-D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64
CXXFLAGS: -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-
switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64
-mtune=generic
LDFLAGS: -Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld
-Wl,--as-needed
Configure: '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-
linux-gnu' '--program-prefix=' '--disable-dependency-tracking' '
--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '
--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '
--includedir=/usr/include' '--libdir=/usr/lib64' '
--libexecdir=/usr/libexec' '--localstatedir=/var' '
--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '
--infodir=/usr/share/info' '--enable-milter' '--disable-clamav' '
--disable-static' '--disable-zlib-vcheck' '--disable-unrar' '--enable-
id-check' '--enable-dns' '--with-dbdir=/var/lib/clamav' '--with-
group=clamupdate' '--with-user=clamupdate' '--disable-rpath' '
--disable-silent-rules' '--enable-clamdtop' 'build_alias=x86_64-redhat-
linux-gnu' 'host_alias=x86_64-redhat-linux-gnu' 'CXXFLAGS=-O2 -g -pipe
-Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --
param=ssp-buffer-size=4 -grecord-gcc-switches
-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic'
'LDFLAGS=-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld
-Wl,--as-needed' 'CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2
-fexceptions -fstack-protector-strong --param=ssp-buffer-size=4
-grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -
m64 -mtune=generic'
'PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig'
sizeof(void*) = 8
Engine flevel: 105, dconf: 105

uname -a
Linux xxxxxx 3.10.0-957.21.3.el7.x86_64 #1 SMP Tue Jun 18 16:35:19 UTC
2019 x86_64 x86_64 x86_64 GNU/Linux

# cat /etc/centos-release
CentOS Linux release 7.6.1810 (Core)

--
Thank you,
Tim

[Winner of the 2018 Consumer Credit Awards]

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users

Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] OnAccess renders system unusable in ~24h [ In reply to ]

clamav-users at lists

Sep 24, 2019, 7:39 AM

Post #2 of 5 (206 views)

Permalink

Hi there,

On Tue, 24 Sep 2019, Tim Stubbs wrote:

> I am running clamd with OnAccess enabled, however its causing the load
> on the systems to make them almost unusable within about 24hours.

This may be true, but I'd want to know that the suspicion is justified
(and front and centre I personally think scanning most Linux boxes with
ClamAV is a waste of CPU).

> as you can see sys is at 98% ...

No, I see CPU 27% idle and three clamd processes doing nothing. But I
do see a load average of around seven. On my dual CPU 2.7GHz Opterons
I routinely see an average of that sort of figure when they do backups
for a bunch of other machines, and Nagios will whine about it when it
gets over 8, but I don't usually worry about it until it gets into the
double digits.

> it seem clamd is stopping other
> applications from processing somehow. cannot find anything in the logs.
> not sure what debugging would be helpful? any advice would be helpful
> here?

My immediate reaction is - if the suspicion is found to be justified -
that you should try to reduce, initially to a bare minimum, the amount
of work which you're asking the machine to do.

> OnAccessIncludePath = "/home", "/root", "/etc", "/sftp", "/boot", \
> "/opt", "/media", "/mnt"

For example you could remove most of the directories from this list to
see if it helps. There are other things you might try, like limiting
the number of threads. But again, I don't see anything in your 'top'
output which tells me that clamd is heavily loading your machine.

What kinds of threats do you care about? If for example you're not
expecting your Linux boxes to be attacked by Windows malware you could
reduce the size of the ClamAV databases very significantly which might
improve scanning performance.

ClamaV version 0.102 has just been released as a candidate for testing
and I've been running it for some time before the RC was released. It
contains some significant improvements for on-access scanning and, if
you do intend to persevere with on-access scanning, I'd recommend that
you install the latest version from the source.

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users

Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] OnAccess renders system unusable in ~24h [ In reply to ]

tim.stubbs at telrock

Sep 24, 2019, 8:12 AM

Post #3 of 5 (206 views)

Permalink

Hi,
thanks for the quick response. We have been asked to run Realtime scans
as part of our PCI requirement, otherwise I would agree with you 100%.

that wasn't the best worst, example i had a VM this morning 56 49 47,
which went back to 1 when I stopped clamd. I do however have other VMs
where (with the same config) I've never had an issue.

thanks for that, yes we are a linux house, ill try reducing the DB & I
will reduce the paths and test that. failing that I will take a look at
v102

thank you,
I'll update with my findings.
Tim

-----Original Message-----
From: G.W. Haywood via clamav-users <clamav-users@lists.clamav.net>
Reply-To: ClamAV users ML <clamav-users@lists.clamav.net>
To: ClamAV users ML <clamav-users@lists.clamav.net>
Cc: G.W. Haywood <clamav@jubileegroup.co.uk>
Subject: Re: [clamav-users] OnAccess renders system unusable in ~24h
Date: Tue, 24 Sep 2019 15:39:22 +0100

Hi there,

On Tue, 24 Sep 2019, Tim Stubbs wrote:

> I am running clamd with OnAccess enabled, however its causing the
> load
> on the systems to make them almost unusable within about 24hours.

This may be true, but I'd want to know that the suspicion is justified
(and front and centre I personally think scanning most Linux boxes with
ClamAV is a waste of CPU).

> as you can see sys is at 98% ...

No, I see CPU 27% idle and three clamd processes doing nothing. But I
do see a load average of around seven. On my dual CPU 2.7GHz Opterons
I routinely see an average of that sort of figure when they do backups
for a bunch of other machines, and Nagios will whine about it when it
gets over 8, but I don't usually worry about it until it gets into the
double digits.

> it seem clamd is stopping other
> applications from processing somehow. cannot find anything in the
> logs.
> not sure what debugging would be helpful? any advice would be helpful
> here?

My immediate reaction is - if the suspicion is found to be justified -
that you should try to reduce, initially to a bare minimum, the amount
of work which you're asking the machine to do.

> OnAccessIncludePath = "/home", "/root", "/etc", "/sftp", "/boot", \
> "/opt", "/media", "/mnt"

For example you could remove most of the directories from this list to
see if it helps. There are other things you might try, like limiting
the number of threads. But again, I don't see anything in your 'top'
output which tells me that clamd is heavily loading your machine.

What kinds of threats do you care about? If for example you're not
expecting your Linux boxes to be attacked by Windows malware you could
reduce the size of the ClamAV databases very significantly which might
improve scanning performance.

ClamaV version 0.102 has just been released as a candidate for testing
and I've been running it for some time before the RC was released. It
contains some significant improvements for on-access scanning and, if
you do intend to persevere with on-access scanning, I'd recommend that
you install the latest version from the source.

--
Thank you,
Tim

[Winner of the 2018 Consumer Credit Awards]

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users

Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] OnAccess renders system unusable in ~24h [ In reply to ]

tim.stubbs at telrock

Sep 24, 2019, 8:34 AM

Post #4 of 5 (206 views)

Permalink

Hi
> What kinds of threats do you care about? If for example you're not
> expecting your Linux boxes to be attacked by Windows malware you
could
> reduce the size of the ClamAV databases very significantly which
might
> improve scanning performance.

Sorry could you point me in the right direction for how to do this?
good hasn't helped me so far?

thanks
Tim

-----Original Message-----
From: Tim Stubbs <tim.stubbs@telrock.com>
Reply-To: ClamAV users ML <clamav-users@lists.clamav.net>
To: clamav-users@lists.clamav.net <clamav-users@lists.clamav.net>
Subject: Re: [clamav-users] OnAccess renders system unusable in ~24h
Date: Tue, 24 Sep 2019 15:12:16 +0000

Hi,
thanks for the quick response. We have been asked to run Realtime scans
as part of our PCI requirement, otherwise I would agree with you 100%.

that wasn't the best worst, example i had a VM this morning 56 49 47,
which went back to 1 when I stopped clamd. I do however have other VMs
where (with the same config) I've never had an issue.

thanks for that, yes we are a linux house, ill try reducing the DB & I
will reduce the paths and test that. failing that I will take a look at
v102

thank you,
I'll update with my findings.
Tim

-----Original Message-----
From: G.W. Haywood via clamav-users <
clamav-users@lists.clamav.net
>
Reply-To: ClamAV users ML <
clamav-users@lists.clamav.net
>
To: ClamAV users ML <
clamav-users@lists.clamav.net
>
Cc: G.W. Haywood <
clamav@jubileegroup.co.uk
>
Subject: Re: [clamav-users] OnAccess renders system unusable in ~24h
Date: Tue, 24 Sep 2019 15:39:22 +0100

Hi there,

On Tue, 24 Sep 2019, Tim Stubbs wrote:

> I am running clamd with OnAccess enabled, however its causing the
> load
> on the systems to make them almost unusable within about 24hours.

This may be true, but I'd want to know that the suspicion is justified
(and front and centre I personally think scanning most Linux boxes with
ClamAV is a waste of CPU).

> as you can see sys is at 98% ...

No, I see CPU 27% idle and three clamd processes doing nothing. But I
do see a load average of around seven. On my dual CPU 2.7GHz Opterons
I routinely see an average of that sort of figure when they do backups
for a bunch of other machines, and Nagios will whine about it when it
gets over 8, but I don't usually worry about it until it gets into the
double digits.

> it seem clamd is stopping other
> applications from processing somehow. cannot find anything in the
> logs.
> not sure what debugging would be helpful? any advice would be helpful
> here?

My immediate reaction is - if the suspicion is found to be justified -
that you should try to reduce, initially to a bare minimum, the amount
of work which you're asking the machine to do.

> OnAccessIncludePath = "/home", "/root", "/etc", "/sftp", "/boot", \
> "/opt", "/media", "/mnt"

For example you could remove most of the directories from this list to
see if it helps. There are other things you might try, like limiting
the number of threads. But again, I don't see anything in your 'top'
output which tells me that clamd is heavily loading your machine.

What kinds of threats do you care about? If for example you're not
expecting your Linux boxes to be attacked by Windows malware you could
reduce the size of the ClamAV databases very significantly which might
improve scanning performance.

ClamaV version 0.102 has just been released as a candidate for testing
and I've been running it for some time before the RC was released. It
contains some significant improvements for on-access scanning and, if
you do intend to persevere with on-access scanning, I'd recommend that
you install the latest version from the source.

--
Thank you,
Tim

[Winner of the 2018 Consumer Credit Awards]

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net

https://lists.clamav.net/mailman/listinfo/clamav-users

Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

--
Thank you,
Tim

[Winner of the 2018 Consumer Credit Awards]

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users

Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] OnAccess renders system unusable in ~24h [ In reply to ]

clamav-users at lists

Sep 24, 2019, 12:08 PM

Post #5 of 5 (205 views)

Permalink

Hello again,

On Tue, 24 Sep 2019, Tim Stubbs wrote:

>> What kinds of threats do you care about? If for example you're not
>> expecting your Linux boxes to be attacked by Windows malware you
>> could reduce the size of the ClamAV databases very significantly
>> which might improve scanning performance.
>
> Sorry could you point me in the right direction for how to do this?
> good hasn't helped me so far?

Check the archives for this list, it's been discussed recently how
to use an empty database. That might be a good start.

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users

Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml