Mailing List Archive

I can't comment on this particular error but 2G of RAM is definitely
insufficient and I believe 4G would be as well when freshclam is
applying updates to the database as there would be 2 copies of it in RAM.

Dave.

On 2022-03-17 07:25, Stephen Scotter via clamav-users wrote:
> Hi,
>
> I noticed Clamd has unexpectantly died on two of my virtual machines. Investigating lead me to find similar errors in the logs on both hosts around the times I know clamd died (I'm monitoring for the existence of a clamd process with zabbix but only got around to investigating today due to other commitments)
>
>
> System1
> Virtual Machine
> CPU : 1 socket / 2 cores
> RAM : 2GB Ram (I must have been feeling frugal on the day I created that VM)
> OS : Debian 10 / buster
> Clam :  ClamAV 0.103.5/26484/Thu Mar 17 08:28:38 2022
>
> Mar 13 13:14:27 System1 clamd[9495]: LibClamAV Warning: fmap_readpage: pread fail: asked for 901703 bytes @ offset 4096, got 0
> Mar 13 13:14:27 System1 clamd[9495]: LibClamAV Error: fmap_get_MD5: error reading while generating hash!
> Mar 13 13:14:27 System1 systemd[1]: clamav-daemon.service: Main process exited, code=killed, status=11/SEGV
> Mar 13 13:14:27 System1 systemd[1]: clamav-daemon.service: Failed with result 'signal'.
>
> System2
> Virtual Machine
> 1 socket / 2 cores
> 4GB Ram (This is our standard allocation; This VM isn't in production yet and isn't doing anything)
> OS : Debian 11 / bullseye
> Clam : ClamAV 0.103.5/26484/Thu Mar 17 08:28:38 2022
>
>
> Mar  9 19:56:17 System2 clamd[541]: LibClamAV Warning: fmap_readpage: pread fail: asked for 455239 bytes @ offset 450560, got 0
> Mar  9 19:56:17 System2 clamd[541]: LibClamAV Error: fmap_get_MD5: error reading while generating hash!
> Mar  9 19:56:17 System2 kernel: [19292.532221] clamd[6494]: segfault at 0 ip 00007f73de568a08 sp 00007f73777fd410 error 4 in libclamav.so.9.0.5[7f73de491000+11d000]
> Mar  9 19:56:17 System2 systemd[1]: clamav-daemon.service: Main process exited, code=killed, status=11/SEGV
> Mar  9 19:56:17 System2 systemd[1]: clamav-daemon.service: Failed with result 'signal'.
> Mar  9 19:56:17 System2 systemd[1]: clamav-daemon.service: Consumed 20min 43.611s CPU time.
>
> This feels like it could be related to a lack of RAM? Would anyone be able to confirm I'm on the right track before shutdown the VMs to I allocate Ram?
>
> Cheers
>
> Steve
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml


_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Hi there,

On Thu, 17 Mar 2022, Stephen Scotter via clamav-users wrote:

> I noticed Clamd has unexpectantly died on two [VMs] ...
>
> System1
> Virtual Machine
> CPU : 1 socket / 2 cores
> RAM : 2GB Ram ...
> OS : Debian 10 / buster
> Clam : ClamAV 0.103.5/26484/Thu Mar 17 08:28:38 2022
>
> Mar 13 13:14:27 System1 clamd[9495]: LibClamAV Warning: fmap_readpage: pread fail: asked for 901703 bytes @ offset 4096, got 0

This certainly seems to be saying that ClamAV asked for 900kB of RAM
and that it was disappointed. If there's only 2G in the VM then I'd
expect this to happen more or less on every database reload if you're
(a) using the official signature database and (b) allowing clamd to
scan while reloading signatures - because it uses about 1G of RAM for
the official signatures and twice as much while reloading, unless you
configure it otherwise.

> System2
> Virtual Machine
> 1 socket / 2 cores
> 4GB Ram (This is our standard allocation; This VM isn't in production yet and isn't doing anything)
> OS : Debian 11 / bullseye
> Clam : ClamAV 0.103.5/26484/Thu Mar 17 08:28:38 2022
>
>
> Mar 9 19:56:17 System2 clamd[541]: LibClamAV Warning: fmap_readpage: pread fail: asked for 455239 bytes @ offset 450560, got 0
> ...
> This feels like it could be related to a lack of RAM?

Not so easy to explain if you have 4G of RAM unless it's doing
something else which uses quite a chunk of RAM too.

> Mar 9 19:56:17 System2 systemd[1]: clamav-daemon.service: Consumed 20min 43.611s CPU time.

OTOH if it isn't doing anything, how is it that clamd has managed to
use more than twenty minutes of CPU?

Are you perhaps scanning some large files? What does 'top' say?

We only scan mail, never filesystems, and the mail server limits the
size of anything which might be scanned to a few megabytes. Free RAM
is monitored by Icinga, giving confidence that nothing unexpected is
happening. TTBOMK I've never seen the warnings from LibClamAV that
you're seeing. I've just grepped 18 months worth of clamd logs and
found no example of 'readpage'. We use a Raspberry Pi4B as a clamd
server, it has 4GB of RAM. It's running several other memory-hungry
processes, although clamd tops the list at about 1.6GB resident - we
use dozens of third-party signature databases. Most of the time free
RAM hovers around 2.2GB. During signature database reloads that drops
by around 1G for a minute or so.

There are of course other reasons why clamd might die; I normally only
see that when I'm exprimenting with Yara rules. Generally if I get a
rule horribly wrong[*] then after reloading the signatures clamd will
crash when it next tries to scan something. IMO it's rather too easy
to crash it that way, but I haven't found any other reliable way to
crash it and under 'normal' circumstances. When I'm not messing about
with Yara rules, I find it very stable.

[*] e.g. failing correctly to pair the curly braces...

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Hi Stephen,

Based on this output:
Mar 13 13:14:27 System1 clamd[9495]: LibClamAV Warning: fmap_readpage: pread fail: asked for 901703 bytes @ offset 4096, got 0
Mar 13 13:14:27 System1 clamd[9495]: LibClamAV Error: fmap_get_MD5: error reading while generating hash!
... it looks to me like you're running into this issue: https://github.com/Cisco-Talos/clamav/issues/440

We are working on a fix for this and will include it in the 0.103.6 release planned for late April. Sorry about the frustration.

Regards,
Micah



Micah Snyder
ClamAV Development
Talos
Cisco Systems, Inc.
________________________________
From: clamav-users <clamav-users-bounces@lists.clamav.net> on behalf of Stephen Scotter via clamav-users <clamav-users@lists.clamav.net>
Sent: Thursday, March 17, 2022 4:25 AM
To: ClamAV Users ML <clamav-users@lists.clamav.net>
Cc: Stephen Scotter <sscotter@yahoo.com>
Subject: [clamav-users] LibClamAV Warning: fmap_readpage: pread fail

Hi,

I noticed Clamd has unexpectantly died on two of my virtual machines. Investigating lead me to find similar errors in the logs on both hosts around the times I know clamd died (I'm monitoring for the existence of a clamd process with zabbix but only got around to investigating today due to other commitments)


System1
Virtual Machine
CPU : 1 socket / 2 cores
RAM : 2GB Ram (I must have been feeling frugal on the day I created that VM)
OS : Debian 10 / buster
Clam : ClamAV 0.103.5/26484/Thu Mar 17 08:28:38 2022

Mar 13 13:14:27 System1 clamd[9495]: LibClamAV Warning: fmap_readpage: pread fail: asked for 901703 bytes @ offset 4096, got 0
Mar 13 13:14:27 System1 clamd[9495]: LibClamAV Error: fmap_get_MD5: error reading while generating hash!
Mar 13 13:14:27 System1 systemd[1]: clamav-daemon.service: Main process exited, code=killed, status=11/SEGV
Mar 13 13:14:27 System1 systemd[1]: clamav-daemon.service: Failed with result 'signal'.

System2
Virtual Machine
1 socket / 2 cores
4GB Ram (This is our standard allocation; This VM isn't in production yet and isn't doing anything)
OS : Debian 11 / bullseye
Clam : ClamAV 0.103.5/26484/Thu Mar 17 08:28:38 2022


Mar 9 19:56:17 System2 clamd[541]: LibClamAV Warning: fmap_readpage: pread fail: asked for 455239 bytes @ offset 450560, got 0
Mar 9 19:56:17 System2 clamd[541]: LibClamAV Error: fmap_get_MD5: error reading while generating hash!
Mar 9 19:56:17 System2 kernel: [19292.532221] clamd[6494]: segfault at 0 ip 00007f73de568a08 sp 00007f73777fd410 error 4 in libclamav.so.9.0.5[7f73de491000+11d000]
Mar 9 19:56:17 System2 systemd[1]: clamav-daemon.service: Main process exited, code=killed, status=11/SEGV
Mar 9 19:56:17 System2 systemd[1]: clamav-daemon.service: Failed with result 'signal'.
Mar 9 19:56:17 System2 systemd[1]: clamav-daemon.service: Consumed 20min 43.611s CPU time.

This feels like it could be related to a lack of RAM? Would anyone be able to confirm I'm on the right track before shutdown the VMs to I allocate Ram?

Cheers

Steve

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml