Hi,
Longtime user, first-time poster ;)
I'm having some issues with trying to get clamscan to skip over some
very large files, without running into memory allocation issues.
A problem directory looks like this:
# ls -alh
total 2.6G
drwxr-xr-x 2 root root 72 May 22 12:17 .
drwxrwxrwt. 12 root root 4.0K May 22 12:16 ..
-rw-r--r-- 1 root root 1.7G May 22 12:13 bigfile1
-rw-r--r-- 1 root root 851M May 22 12:14 bigfile2
-rw-r--r-- 1 root root 10K May 22 12:10 file1
-rw------- 1 root root 94M May 22 12:13 file2
-rw-r--r-- 1 root root 264K May 22 12:14 file3
and scanning it does this:
# clamscan -r .
./file1: OK
LibClamAV Warning: fmap: map allocation failed
LibClamAV Error: CRITICAL: fmap() failed
./bigfile1: Can't allocate memory ERROR
./bigfile2: OK
./file3: OK
./file2: OK
----------- SCAN SUMMARY -----------
Known viruses: 6133971
Engine version: 0.101.2
Scanned directories: 1
Scanned files: 4
Infected files: 0
Total errors: 1
Data scanned: 0.00 MB
Data read: 2624.80 MB (ratio 0.00:1)
Time: 47.989 sec (0 m 47 s)
So, ah-ha, I think, obviously I need to limit the file size being
scanned, so add a --max-filesize flag, but:
# clamscan -r --max-filesize=1024 .
./file1: OK
LibClamAV Warning: fmap: map allocation failed
LibClamAV Error: CRITICAL: fmap() failed
./bigfile1: Can't allocate memory ERROR
./bigfile2: OK
./file3: OK
./file2: OK
----------- SCAN SUMMARY -----------
Known viruses: 6133971
Engine version: 0.101.2
Scanned directories: 1
Scanned files: 4
Infected files: 0
Total errors: 1
Data scanned: 0.00 MB
Data read: 2624.80 MB (ratio 0.00:1)
Time: 49.758 sec (0 m 49 s)
Same outcome. Tried with the debug flag and this was the relevant bit:
LibClamAV debug: in cli_magic_scandesc (reclevel: 0/16)
LibClamAV Warning: fmap: map allocation failed
LibClamAV Error: CRITICAL: fmap() failed
LibClamAV debug: cli_magic_scandesc: returning 20 at line 3891 (no
post, no cache)
./bigfile: Can't allocate memory ERROR
So it looks like clamscan is trying to determine the file type before it
then decides if its too large (in case it's a container?), so fmap()'s
it and then can't allocate memory?
I've got a fair number of servers and VMs being scanned, and I'm make
sure that scan errors (ie exit code 2, unknown error) like this are sent
to the monitoring system for investigation, and this generating noise.
How do I work around this?
(I've considered running a find / -type f -size -50M or similar, dumping
that to a file and feeding that to clamscan via the -f flag, but any
transitory file that's gone away by the time that clamscan gets to it
produces a missing file error, and also exits with code 2, so that's not
great either)
Thanks,
Simon
--
Simon Oxwell | Hosting Team
Funnelback
P: +61 2 6176 3170 | F: +61 2 6230 7313
soxwell@funnelback.com | www.funnelback.com
A: Ground Floor, 51 Allara Street, Civic, Canberra 2601
_______________________________________________
clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq
http://www.clamav.net/contact.html#ml
Longtime user, first-time poster ;)
I'm having some issues with trying to get clamscan to skip over some
very large files, without running into memory allocation issues.
A problem directory looks like this:
# ls -alh
total 2.6G
drwxr-xr-x 2 root root 72 May 22 12:17 .
drwxrwxrwt. 12 root root 4.0K May 22 12:16 ..
-rw-r--r-- 1 root root 1.7G May 22 12:13 bigfile1
-rw-r--r-- 1 root root 851M May 22 12:14 bigfile2
-rw-r--r-- 1 root root 10K May 22 12:10 file1
-rw------- 1 root root 94M May 22 12:13 file2
-rw-r--r-- 1 root root 264K May 22 12:14 file3
and scanning it does this:
# clamscan -r .
./file1: OK
LibClamAV Warning: fmap: map allocation failed
LibClamAV Error: CRITICAL: fmap() failed
./bigfile1: Can't allocate memory ERROR
./bigfile2: OK
./file3: OK
./file2: OK
----------- SCAN SUMMARY -----------
Known viruses: 6133971
Engine version: 0.101.2
Scanned directories: 1
Scanned files: 4
Infected files: 0
Total errors: 1
Data scanned: 0.00 MB
Data read: 2624.80 MB (ratio 0.00:1)
Time: 47.989 sec (0 m 47 s)
So, ah-ha, I think, obviously I need to limit the file size being
scanned, so add a --max-filesize flag, but:
# clamscan -r --max-filesize=1024 .
./file1: OK
LibClamAV Warning: fmap: map allocation failed
LibClamAV Error: CRITICAL: fmap() failed
./bigfile1: Can't allocate memory ERROR
./bigfile2: OK
./file3: OK
./file2: OK
----------- SCAN SUMMARY -----------
Known viruses: 6133971
Engine version: 0.101.2
Scanned directories: 1
Scanned files: 4
Infected files: 0
Total errors: 1
Data scanned: 0.00 MB
Data read: 2624.80 MB (ratio 0.00:1)
Time: 49.758 sec (0 m 49 s)
Same outcome. Tried with the debug flag and this was the relevant bit:
LibClamAV debug: in cli_magic_scandesc (reclevel: 0/16)
LibClamAV Warning: fmap: map allocation failed
LibClamAV Error: CRITICAL: fmap() failed
LibClamAV debug: cli_magic_scandesc: returning 20 at line 3891 (no
post, no cache)
./bigfile: Can't allocate memory ERROR
So it looks like clamscan is trying to determine the file type before it
then decides if its too large (in case it's a container?), so fmap()'s
it and then can't allocate memory?
I've got a fair number of servers and VMs being scanned, and I'm make
sure that scan errors (ie exit code 2, unknown error) like this are sent
to the monitoring system for investigation, and this generating noise.
How do I work around this?
(I've considered running a find / -type f -size -50M or similar, dumping
that to a file and feeding that to clamscan via the -f flag, but any
transitory file that's gone away by the time that clamscan gets to it
produces a missing file error, and also exits with code 2, so that's not
great either)
Thanks,
Simon
--
Simon Oxwell | Hosting Team
Funnelback
P: +61 2 6176 3170 | F: +61 2 6230 7313
soxwell@funnelback.com | www.funnelback.com
A: Ground Floor, 51 Allara Street, Civic, Canberra 2601
_______________________________________________
clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq
http://www.clamav.net/contact.html#ml