Mailing List Archive

Check 'dmesg' and look for OOM-killer. Clamav needs a descend amount of RAM
- especially when using 3rd-party signatures. As there are random updates
over the day, the reloads could lead to your noticed behavior.

Am Mi., 3. Juni 2020 um 10:00 Uhr schrieb Jesús Arnáiz via clamav-users <
clamav-users@lists.clamav.net>:

> Hi!
>
> I have an Ubuntu server as email server. I use there, MailInABox:
> https://mailinabox.email/
>
> I installed on my own (since it is not part of MailInABox) clamav and I
> configured it to be used with clamsmtp on postfix, so email is scanned
> for virus.
>
> It normally works fine, but, apparently it is crashing everyday more
> than once, meaning a manual restart is needed once or twice per day.
>
> I don't see anything relevant on the logs, more than error of clamsmtp
> complaining it "can't connect to clamd" (expected issue if the service
> is stopped as is the case):
>
> ---
> Jun 3 08:42:54 mailbox clamsmtpd: 10126F: CLAMAV: couldn't connect to:
> /var/run/clamav/clamd.ctl: Connection refused
> Jun 3 08:43:32 mailbox clamsmtpd: 101274: CLAMAV: couldn't connect to:
> /var/run/clamav/clamd.ctl: Connection refused
> ----
>
> A manual restart makes the error disappear and it continue working for a
> while.
>
> For the moment as a temporal workaround, I set some crontab to check the
> "clamd" process and restart it if it is not running. But I would like to
> find the issue and fix it if possible.
>
> Any ideas?
>
> My system is:
>
> ----
> Linux mailbox 5.3.18-3-pve #1 SMP PVE 5.3.18-3 (Tue, 17 Mar 2020
> 16:33:19 +0100) x86_64 x86_64 x86_64 GNU/Linux
>
> Ubuntu 18.04.4 LTS
>
> ClamAV Version:
>
> clamscan -V
> ClamAV 0.102.3/25831/Tue Jun 2 14:41:03 2020
>
> CalmSMTPD:
>
> clamsmtpd (version 1.10)
> (config: /etc/clamsmtpd.conf)
>
>
> Thanks!,
> Jesús.
>
> --
> El software de antivirus Avast ha analizado este correo electrónico en
> busca de virus.
> https://www.avast.com/antivirus
>
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>

Hi there,

On Wed, 3 Jun 2020, Stefan Bauer via clamav-users wrote:
> Am Mi., 3. Juni 2020 schrieb Jesus Arnaiz via clamav-users
>
> > It normally works fine, but, apparently it is crashing everyday more
> > than once, meaning a manual restart is needed once or twice per day.
> >
> > I don't see anything relevant on the logs...
> > Any ideas?
> >
> > My system is:
> > ...
>
> ... Clamav needs a [decent] amount of RAM ...
>
> - especially when using 3rd-party signatures. As there are random updates
> over the day, the reloads could lead to your noticed behavior.

It might help the OP to know that clamd *normally* uses ~1Gbyte of RAM
when it LOADS the 'official' signatures (and more than that of course
if it uses third-party signatures, of which there are many available).

This memory usage is when clamd starts from the 'cold' state, when it
was not previously running, but things are different when it RELOADS
the signature databases while it is running.

In addition to the normal RAM usage, clamd normally uses twice as much
RAM when it RELOADS the signature databases because it has one copy in
its RAM already. This copy may be used to serve scan requests during
the time when it is RELOADING the databases after they have changed.
During the reload, clamd makes a second copy of its data. After this
second copy is loaded and tested, and after any scan request which is
using the first copy has completed, the first copy is deleted and the
RAM which it used is returned to the operating system's free pool. It
usually takes at least some seconds to reload the signature database,
but if the machine is slow it may take several minutes. You really do
not want to drive the machine into using swap space, performance then
will be very poor indeed.

This means that at present - and this will change as more signatures
are added to the databases - you need a minimum of at least 2Gbytes of
RAM to run clamd with automatic database updates, and probably more
than that if the box is to do anything useful. At present I recommend
a minimum of 4Gbytes RAM for a system which runs clamd. In addition,
if you use clamscan, while it scans you will need an *extra* 1 Gbyte
of RAM over and above the RAM used by clamd. Note that clamscan and
clamdscan behave very differently in that clamdscan uses clamd, but
clamscan does not. Of course you can use alternate ways of reloading
the databases, but then you will lose the ability to scan while clamd
is loading signatures. That may be acceptable to you, to others it is
not and the issue has caused some discussion on this list in the past.

Signature database reloads may be triggered in more than one way. If
you use freshclam to update the signatures automatically, it can send
a signal to clamd to cause a reload; if clamd is itself configured to
check, and notices that the a database has changed, it will start the
reload itself; and you can send a 'reload' signal to clamd using its
listening socket, the API is documented in e.g. the clamd man page.

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Hi,

Thanks for your kind message.

Indeed it looks as an issue related to RAM. Reading Freshclam logs all
issues came exactly at that moment when a new DB started to be processed.

Very well explained.

*** I sent this message days ago, but I used the wrong account and it
was returned recently, sorry if I was not quicker with the feedback.

Best Regards,
Jesús.


El 03/06/2020 a las 14:53, G.W. Haywood escribió:
> Hi there,
>
> On Wed, 3 Jun 2020, Stefan Bauer via clamav-users wrote:
>> Am Mi., 3. Juni 2020 schrieb Jesus Arnaiz via clamav-users
>>
>> > It normally works fine, but, apparently it is crashing everyday more
>> > than once, meaning a manual restart is needed once or twice per day.
>> >
>> > I don't see anything relevant on the logs...
>> > Any ideas?
>> >
>> > My system is:
>> > ...
>>
>> ... Clamav needs a [decent] amount of RAM ...
>>
>> - especially when using 3rd-party signatures. As there are random updates
>> over the day, the reloads could lead to your noticed behavior.
>
> It might help the OP to know that clamd *normally* uses ~1Gbyte of RAM
> when it LOADS the 'official' signatures (and more than that of course
> if it uses third-party signatures, of which there are many available).
>
> This memory usage is when clamd starts from the 'cold' state, when it
> was not previously running, but things are different when it RELOADS
> the signature databases while it is running.
>
> In addition to the normal RAM usage, clamd normally uses twice as much
> RAM when it RELOADS the signature databases because it has one copy in
> its RAM already.  This copy may be used to serve scan requests during
> the time when it is RELOADING the databases after they have changed.
> During the reload, clamd makes a second copy of its data.  After this
> second copy is loaded and tested, and after any scan request which is
> using the first copy has completed, the first copy is deleted and the
> RAM which it used is returned to the operating system's free pool.  It
> usually takes at least some seconds to reload the signature database,
> but if the machine is slow it may take several minutes.  You really do
> not want to drive the machine into using swap space, performance then
> will be very poor indeed.
>
> This means that at present - and this will change as more signatures
> are added to the databases - you need a minimum of at least 2Gbytes of
> RAM to run clamd with automatic database updates, and probably more
> than that if the box is to do anything useful.  At present I recommend
> a minimum of 4Gbytes RAM for a system which runs clamd.  In addition,
> if you use clamscan, while it scans you will need an *extra* 1 Gbyte
> of RAM over and above the RAM used by clamd.  Note that clamscan and
> clamdscan behave very differently in that clamdscan uses clamd, but
> clamscan does not.  Of course you can use alternate ways of reloading
> the databases, but then you will lose the ability to scan while clamd
> is loading signatures.  That may be acceptable to you, to others it is
> not and the issue has caused some discussion on this list in the past.
>
> Signature database reloads may be triggered in more than one way.  If
> you use freshclam to update the signatures automatically, it can send
> a signal to clamd to cause a reload; if clamd is itself configured to
> check, and notices that the a database has changed, it will start the
> reload itself; and you can send a 'reload' signal to clamd using its
> listening socket, the API is documented in e.g. the clamd man page.
>


--
El software de antivirus Avast ha analizado este correo electrónico en busca de virus.
https://www.avast.com/antivirus


_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Hi,

Thanks for your kind message.

Indeed it looks as an issue related to RAM. Reading Freshclam logs all
issues came exactly at that moment when a new DB started to be processed.

Very well explained.

*** I sent this message days ago, but I used the wrong account and it
was returned recently, sorry if I was not quicker with the feedback.

Best Regards,
Jesús.

El 03/06/2020 a las 10:43, Stefan Bauer escribió:
> Check 'dmesg' and look for OOM-killer. Clamav needs a descend amount of RAM
> - especially when using 3rd-party signatures. As there are random updates
> over the day, the reloads could lead to your noticed behavior.
>
> Am Mi., 3. Juni 2020 um 10:00 Uhr schrieb Jesús Arnáiz via clamav-users <
> clamav-users@lists.clamav.net>:
>
>> Hi!
>>
>> I have an Ubuntu server as email server. I use there, MailInABox:
>> https://mailinabox.email/
>>
>> I installed on my own (since it is not part of MailInABox) clamav and I
>> configured it to be used with clamsmtp on postfix, so email is scanned
>> for virus.
>>
>> It normally works fine, but, apparently it is crashing everyday more
>> than once, meaning a manual restart is needed once or twice per day.
>>
>> I don't see anything relevant on the logs, more than error of clamsmtp
>> complaining it "can't connect to clamd" (expected issue if the service
>> is stopped as is the case):
>>
>> ---
>> Jun 3 08:42:54 mailbox clamsmtpd: 10126F: CLAMAV: couldn't connect to:
>> /var/run/clamav/clamd.ctl: Connection refused
>> Jun 3 08:43:32 mailbox clamsmtpd: 101274: CLAMAV: couldn't connect to:
>> /var/run/clamav/clamd.ctl: Connection refused
>> ----
>>
>> A manual restart makes the error disappear and it continue working for a
>> while.
>>
>> For the moment as a temporal workaround, I set some crontab to check the
>> "clamd" process and restart it if it is not running. But I would like to
>> find the issue and fix it if possible.
>>
>> Any ideas?
>>
>> My system is:
>>
>> ----
>> Linux mailbox 5.3.18-3-pve #1 SMP PVE 5.3.18-3 (Tue, 17 Mar 2020
>> 16:33:19 +0100) x86_64 x86_64 x86_64 GNU/Linux
>>
>> Ubuntu 18.04.4 LTS
>>
>> ClamAV Version:
>>
>> clamscan -V
>> ClamAV 0.102.3/25831/Tue Jun 2 14:41:03 2020
>>
>> CalmSMTPD:
>>
>> clamsmtpd (version 1.10)
>> (config: /etc/clamsmtpd.conf)
>>
>>
>> Thanks!,
>> Jesús.
>>
>> --
>> El software de antivirus Avast ha analizado este correo electrónico en
>> busca de virus.
>> https://www.avast.com/antivirus
>>
>>
>> _______________________________________________
>>
>> clamav-users mailing list
>> clamav-users@lists.clamav.net
>> https://lists.clamav.net/mailman/listinfo/clamav-users
>>
>>
>> Help us build a comprehensive ClamAV guide:
>> https://github.com/vrtadmin/clamav-faq
>>
>> http://www.clamav.net/contact.html#ml
>>
>


--
El software de antivirus Avast ha analizado este correo electrónico en busca de virus.
https://www.avast.com/antivirus


_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml