Mailing List Archive

0 length bytecode.cvd causing problems with clamav daemon
I am having an issue with 0 length bytecode.cvd files on my scanner
instances. This seems to have started sometime on 22 Feb, I'm afraid I
don't have an exact time. The clamav daemon produces logs like the
following:

Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error: cli_cvdverify:
Can't read CVD header
Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error: Can't load
/var/lib/clamav/bytecode.cld: Broken or not a CVD file
Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error:
cli_loaddbdir(): error loading database /var/lib/clamav/bytecode.cld
Feb 27 14:39:11 av-scan-wrhn clamd[163614]: Mon Feb 27 14:39:11 2023 ->
!Broken or not a CVD file
Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service: Main
process exited, code=exited, status=1/FAILURE
Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service: Failed with
result 'exit-code'.
Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service: Consumed
8.679s CPU time.


I feel like I have narrowed the problem down to a 0 length 'bytecode.cvd'
file. Here is a listing of the definitions directory:

$ ls -l /var/lib/clamav
total 226168
-rw-r--r-- 1 clamav clamav 314802 Feb 27 14:06 bytecode.cld
-rw-r--r-- 1 clamav clamav 0 Feb 27 02:00 bytecode.cvd
-rw-r--r-- 1 clamav clamav 60787973 Feb 27 10:01 daily.cld
-rw-r--r-- 1 clamav clamav 69 Feb 23 15:33 freshclam.dat
-rw-r--r-- 1 clamav clamav 170479789 Feb 27 02:00 main.cvd


My initial fix (before narrowing the problem down to bytecode.cvd) was to

1. stop freshclam
2. clean this directory
3. restart freshclam
4. give it time to get the definitions (from a private mirror)
5. start clamav daemon

This would work for maybe 1/2 day then the empty bytecode.cvd file would
reappear and the daemon would fail.

This morning I was able to spend some more time and find that it was just
the one file that needed to be removed.

I have a local mirror because there are several instances of this scanner
in use (at least 2 instances for several environments). I have checked the
mirror and it appears to be working fine and keeping the definitions up to
date inside our environment. In addition, the scanner instances appear to
be keeping the local set of definitions up to date with the mirror.

The mirror does not have a bytecode.cvd file on it (here is a listing of
its definitions directory)

$ ls -l /var/lib/clamav
total 226172
-rw-r--r-- 1 clamav clamav 314802 Feb 22 22:02 bytecode.cld
-rw-r--r-- 1 clamav clamav 60787973 Feb 27 09:06 daily.cld
-rw-r--r-- 1 clamav clamav 69 Jan 29 2022 freshclam.dat
-rw-r--r-- 1 clamav clamav 170479789 Jan 29 2022 main.cvd
-rw-r--r-- 1 clamav clamav 87 Jan 29 2022 test.html


To the best of my knowledge, the software is up to date:

$ sudo freshclam -V
ClamAV 0.103.8/26825/Mon Feb 27 08:24:38 2023


Here is the freshclam.conf used on all the local sanner instances

$ cat /etc/clamav/freshclam.conf
# Automatically created by the clamav-freshclam postinst
# Comments will get lost when you reconfigure the clamav-freshclam package

DatabaseOwner clamav
UpdateLogFile /var/log/clamav/freshclam.log
LogVerbose false
LogSyslog false
LogFacility LOG_LOCAL6
LogFileMaxSize 0
LogRotate true
LogTime true
Foreground false
Debug false
MaxAttempts 5
DatabaseDirectory /var/lib/clamav
DNSDatabaseInfo current.cvd.clamav.net
ConnectTimeout 30
ReceiveTimeout 0
TestDatabases yes
CompressLocalDatabase no
Bytecode true
NotifyClamd /etc/clamav/clamd.conf
# Check for new database 24 times a day
Checks 24
PrivateMirror http://10.50.0.2
ScriptedUpdates no
PrivateMirror http://10.50.0.2


The scanner has been working fine for about 12 months, keeping the software
and the definitions up to date. The only configuration item that seems to
relate is "Bytecode true", but the description seems to discuss just the
downloading of the file, not whether it is created on the local instance.

Does anyone have any pointers?

Thanks
Kevin
--

*Kevin O'Connor*
Principal DevOps Engineer
M: 617-834-1291

[image: email-footer-logos.jpg (1000×120)]

STATEMENT OF CONFIDENTIALITY: The information contained in this message and
any attachments are intended solely for the addressee(s) and may contain
confidential or privileged information. If you are not the intended
recipient, or responsible for delivering the e-mail to the intended
recipient, you have received this message in error. Any use, dissemination,
forwarding, printing, or copying is strictly prohibited. Please notify
Ampion immediately at security@ampion.net and destroy all copies of this
message and any attachments.
Re: 0 length bytecode.cvd causing problems with clamav daemon [ In reply to ]
why you have set two times the "PrivateMirror" with identically IP's?
Can't believe that this happens with the automated PostInst ????


Von / From: Clamav User Mailinglist <mailto:clamav-users@lists.clamav.net>
An / To: Newcomer01 <mailto:newcomer01@posteo.de>
CC / CC: Kevin O'connor <mailto:koconnor@ampion.net>
Gesendet / Sent: Montag, Februar 27, 2023 um 16:58 (at 04:58 PM) +0100
Betreff / Subject: [clamav-users] 0 length bytecode.cvd causing problems with clamav daemon
> I am having an issue with 0 length bytecode.cvd files on my scanner instances.  This seems to have started sometime on 22 Feb, I'm afraid I don't have an exact time.  The clamav daemon produces logs like the following:
>
> Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error: cli_cvdverify: Can't read CVD header
> Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error: Can't load /var/lib/clamav/bytecode.cld: Broken or not a CVD file
> Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error: cli_loaddbdir(): error loading database /var/lib/clamav/bytecode.cld
> Feb 27 14:39:11 av-scan-wrhn clamd[163614]: Mon Feb 27 14:39:11 2023 -> !Broken or not a CVD file
> Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service: Main process exited, code=exited, status=1/FAILURE
> Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service: Failed with result 'exit-code'.
> Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service: Consumed 8.679s CPU time.
>
>
> I feel like I have narrowed the problem down to a 0 length 'bytecode.cvd' file.  Here is a listing of the definitions directory:
>
> $ ls -l /var/lib/clamav
> total 226168
> -rw-r--r-- 1 clamav clamav    314802 Feb 27 14:06 bytecode.cld
> -rw-r--r-- 1 clamav clamav         0 Feb 27 02:00 bytecode.cvd
> -rw-r--r-- 1 clamav clamav  60787973 Feb 27 10:01 daily.cld
> -rw-r--r-- 1 clamav clamav        69 Feb 23 15:33 freshclam.dat
> -rw-r--r-- 1 clamav clamav 170479789 Feb 27 02:00 main.cvd
>
>
> My initial fix (before narrowing the problem down to bytecode.cvd) was to
>
> 1. stop freshclam
> 2. clean this directory
> 3. restart freshclam
> 4. give it time to get the definitions (from a private mirror)
> 5. start clamav daemon
>
> This would work for maybe 1/2 day then the empty bytecode.cvd file would reappear and the daemon would fail.
>
> This morning I was able to spend some more time and find that it was just the one file that needed to be removed.
>
> I have a local mirror because there are several instances of this scanner in use (at least 2 instances for several environments).  I have checked the mirror and it appears to be working fine and keeping the definitions up to date inside our environment.  In addition, the scanner instances appear to be keeping the local set of definitions up to date with the mirror.
>
> The mirror does not have a bytecode.cvd file on it (here is a listing of its definitions directory)
>
> $ ls -l /var/lib/clamav
> total 226172
> -rw-r--r-- 1 clamav clamav    314802 Feb 22 22:02 bytecode.cld
> -rw-r--r-- 1 clamav clamav  60787973 Feb 27 09:06 daily.cld
> -rw-r--r-- 1 clamav clamav        69 Jan 29  2022 freshclam.dat
> -rw-r--r-- 1 clamav clamav 170479789 Jan 29  2022 main.cvd
> -rw-r--r-- 1 clamav clamav        87 Jan 29  2022 test.html
>
>
> To the best of my knowledge, the software is up to date:
>
> $ sudo freshclam -V
> ClamAV 0.103.8/26825/Mon Feb 27 08:24:38 2023
>
>
> Here is the freshclam.conf used on all the local sanner instances
>
> $ cat /etc/clamav/freshclam.conf
> # Automatically created by the clamav-freshclam postinst
> # Comments will get lost when you reconfigure the clamav-freshclam package
>
> DatabaseOwner clamav
> UpdateLogFile /var/log/clamav/freshclam.log
> LogVerbose false
> LogSyslog false
> LogFacility LOG_LOCAL6
> LogFileMaxSize 0
> LogRotate true
> LogTime true
> Foreground false
> Debug false
> MaxAttempts 5
> DatabaseDirectory /var/lib/clamav
> DNSDatabaseInfo current.cvd.clamav.net <http://current.cvd.clamav.net>
> ConnectTimeout 30
> ReceiveTimeout 0
> TestDatabases yes
> CompressLocalDatabase no
> Bytecode true
> NotifyClamd /etc/clamav/clamd.conf
> # Check for new database 24 times a day
> Checks 24
> PrivateMirror http://10.50.0.2
> ScriptedUpdates no
> PrivateMirror http://10.50.0.2
>
>
> The scanner has been working fine for about 12 months, keeping the software and the definitions up to date.   The only configuration item that seems to relate is "Bytecode true", but the description seems to discuss just the downloading of the file, not whether it is created on the local instance.
>
> Does anyone have any pointers?
>
> Thanks
> Kevin
> --
>
> *Kevin O'Connor*
> Principal DevOps Engineer
> M: 617-834-1291
>
> email-footer-logos.jpg (1000×120)
>
> STATEMENT OF CONFIDENTIALITY: The information contained in this message and any attachments are intended solely for the addressee(s) and may contain confidential or privileged information. If you are not the intended recipient, or responsible for delivering the e-mail to the intended recipient, you have received this message in error. Any use, dissemination, forwarding, printing, or copying is strictly prohibited. Please notify Ampion immediately at security@ampion.net and destroy all copies of this message and any attachments.
>
>
> _______________________________________________
>
> Manage your clamav-users mailing list subscription / unsubscribe:
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/Cisco-Talos/clamav-documentation
>
> https://docs.clamav.net/#mailing-lists-and-chat

_______________________________________________

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat
Re: 0 length bytecode.cvd causing problems with clamav daemon [ In reply to ]
Heh, good question. Just checked again, and it looks like that was a
copy-paste error. There is only one PrivateMirror line.
Kevin

On Mon, Feb 27, 2023 at 12:02 PM newcomer01 via clamav-users <
clamav-users@lists.clamav.net> wrote:

> why you have set two times the "PrivateMirror" with identically IP's?
> Can't believe that this happens with the automated PostInst ????
>
>
> Von / From: Clamav User Mailinglist <mailto:clamav-users@lists.clamav.net>
> An / To: Newcomer01 <mailto:newcomer01@posteo.de>
> CC / CC: Kevin O'connor <mailto:koconnor@ampion.net>
> Gesendet / Sent: Montag, Februar 27, 2023 um 16:58 (at 04:58 PM) +0100
> Betreff / Subject: [clamav-users] 0 length bytecode.cvd causing problems
> with clamav daemon
> > I am having an issue with 0 length bytecode.cvd files on my scanner
> instances. This seems to have started sometime on 22 Feb, I'm afraid I
> don't have an exact time. The clamav daemon produces logs like the
> following:
> >
> > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error:
> cli_cvdverify: Can't read CVD header
> > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error: Can't load
> /var/lib/clamav/bytecode.cld: Broken or not a CVD file
> > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error:
> cli_loaddbdir(): error loading database /var/lib/clamav/bytecode.cld
> > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: Mon Feb 27 14:39:11 2023 ->
> !Broken or not a CVD file
> > Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service: Main
> process exited, code=exited, status=1/FAILURE
> > Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service: Failed
> with result 'exit-code'.
> > Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service: Consumed
> 8.679s CPU time.
> >
> >
> > I feel like I have narrowed the problem down to a 0 length
> 'bytecode.cvd' file. Here is a listing of the definitions directory:
> >
> > $ ls -l /var/lib/clamav
> > total 226168
> > -rw-r--r-- 1 clamav clamav 314802 Feb 27 14:06 bytecode.cld
> > -rw-r--r-- 1 clamav clamav 0 Feb 27 02:00 bytecode.cvd
> > -rw-r--r-- 1 clamav clamav 60787973 Feb 27 10:01 daily.cld
> > -rw-r--r-- 1 clamav clamav 69 Feb 23 15:33 freshclam.dat
> > -rw-r--r-- 1 clamav clamav 170479789 Feb 27 02:00 main.cvd
> >
> >
> > My initial fix (before narrowing the problem down to bytecode.cvd) was to
> >
> > 1. stop freshclam
> > 2. clean this directory
> > 3. restart freshclam
> > 4. give it time to get the definitions (from a private mirror)
> > 5. start clamav daemon
> >
> > This would work for maybe 1/2 day then the empty bytecode.cvd file would
> reappear and the daemon would fail.
> >
> > This morning I was able to spend some more time and find that it was
> just the one file that needed to be removed.
> >
> > I have a local mirror because there are several instances of this
> scanner in use (at least 2 instances for several environments). I have
> checked the mirror and it appears to be working fine and keeping the
> definitions up to date inside our environment. In addition, the scanner
> instances appear to be keeping the local set of definitions up to date with
> the mirror.
> >
> > The mirror does not have a bytecode.cvd file on it (here is a listing of
> its definitions directory)
> >
> > $ ls -l /var/lib/clamav
> > total 226172
> > -rw-r--r-- 1 clamav clamav 314802 Feb 22 22:02 bytecode.cld
> > -rw-r--r-- 1 clamav clamav 60787973 Feb 27 09:06 daily.cld
> > -rw-r--r-- 1 clamav clamav 69 Jan 29 2022 freshclam.dat
> > -rw-r--r-- 1 clamav clamav 170479789 Jan 29 2022 main.cvd
> > -rw-r--r-- 1 clamav clamav 87 Jan 29 2022 test.html
> >
> >
> > To the best of my knowledge, the software is up to date:
> >
> > $ sudo freshclam -V
> > ClamAV 0.103.8/26825/Mon Feb 27 08:24:38 2023
> >
> >
> > Here is the freshclam.conf used on all the local sanner instances
> >
> > $ cat /etc/clamav/freshclam.conf
> > # Automatically created by the clamav-freshclam postinst
> > # Comments will get lost when you reconfigure the clamav-freshclam
> package
> >
> > DatabaseOwner clamav
> > UpdateLogFile /var/log/clamav/freshclam.log
> > LogVerbose false
> > LogSyslog false
> > LogFacility LOG_LOCAL6
> > LogFileMaxSize 0
> > LogRotate true
> > LogTime true
> > Foreground false
> > Debug false
> > MaxAttempts 5
> > DatabaseDirectory /var/lib/clamav
> > DNSDatabaseInfo current.cvd.clamav.net <http://current.cvd.clamav.net
> <http://current.cvd.clamav.net>
> >
> > ConnectTimeout 30
> > ReceiveTimeout 0
> > TestDatabases yes
> > CompressLocalDatabase no
> > Bytecode true
> > NotifyClamd /etc/clamav/clamd.conf
> > # Check for new database 24 times a day
> > Checks 24
> > PrivateMirror http://10.50.0.2
> <http://10.50.0.2>
> > ScriptedUpdates no
> > PrivateMirror http://10.50.0.2
> <http://10.50.0.2>
> >
> >
> > The scanner has been working fine for about 12 months, keeping the
> software and the definitions up to date. The only configuration item that
> seems to relate is "Bytecode true", but the description seems to discuss
> just the downloading of the file, not whether it is created on the local
> instance.
> >
> > Does anyone have any pointers?
> >
> > Thanks
> > Kevin
> > --
> >
> > *Kevin O'Connor*
> > Principal DevOps Engineer
> > M: 617-834-1291
> >
> > email-footer-logos.jpg (1000×120)
> >
> > STATEMENT OF CONFIDENTIALITY: The information contained in this message
> and any attachments are intended solely for the addressee(s) and may
> contain confidential or privileged information. If you are not the intended
> recipient, or responsible for delivering the e-mail to the intended
> recipient, you have received this message in error. Any use, dissemination,
> forwarding, printing, or copying is strictly prohibited. Please notify
> Ampion immediately at security@ampion.net and destroy all copies of this
> message and any attachments.
> >
> >
> > _______________________________________________
> >
> > Manage your clamav-users mailing list subscription / unsubscribe:
> > https://lists.clamav.net/mailman/listinfo/clamav-users
> <https://lists.clamav.net/mailman/listinfo/clamav-users>
> >
> >
> > Help us build a comprehensive ClamAV guide:
> > https://github.com/Cisco-Talos/clamav-documentation
> <https://github.com/Cisco-Talos/clamav-documentation>
> >
> > https://docs.clamav.net/#mailing-lists-and-chat
> <https://docs.clamav.net/#mailing-lists-and-chat>
>
> _______________________________________________
>
> Manage your clamav-users mailing list subscription / unsubscribe:
> https://lists.clamav.net/mailman/listinfo/clamav-users
> <https://lists.clamav.net/mailman/listinfo/clamav-users>
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/Cisco-Talos/clamav-documentation
> <https://github.com/Cisco-Talos/clamav-documentation>
>
> https://docs.clamav.net/#mailing-lists-and-chat
> <https://docs.clamav.net/#mailing-lists-and-chat>
>
Re: 0 length bytecode.cvd causing problems with clamav daemon [ In reply to ]
i would suggest, to delete alle libraries in /var/lib/clamav and download all complete new.
CLD Files comes not regularly, normally we have CVD only.

If i understand this well, CLD Files comes only when error occures while updating.
https:/blog.clamav.net/2021/03/clamav-cvds-cdiffs-and-magic-behind.html


Von / From: Kevin O'connor <mailto:koconnor@ampion.net>
An / To: Newcomer01 <mailto:newcomer01@posteo.de>
Gesendet / Sent: Montag, Februar 27, 2023 um 18:38 (at 06:38 PM) +0100
Betreff / Subject: Re: [clamav-users] 0 length bytecode.cvd causing problems with clamav daemon
> Heh, good question.  Just checked again, and it looks like that was a copy-paste error.  There is only one PrivateMirror line.
> Kevin
>
> On Mon, Feb 27, 2023 at 12:02 PM newcomer01 via clamav-users <clamav-users@lists.clamav.net> wrote:
>
> why you have set two times the "PrivateMirror" with identically IP's?
> Can't believe that this happens with the automated PostInst ????
>
>
> Von / From: Clamav User Mailinglist <mailto:clamav-users@lists.clamav.net>
> An / To: Newcomer01 <mailto:newcomer01@posteo.de>
> CC / CC: Kevin O'connor <mailto:koconnor@ampion.net>
> Gesendet / Sent: Montag, Februar 27, 2023 um 16:58 (at 04:58 PM) +0100
> Betreff / Subject: [clamav-users] 0 length bytecode.cvd causing problems with clamav daemon
> > I am having an issue with 0 length bytecode.cvd files on my scanner instances.  This seems to have started sometime on 22 Feb, I'm afraid I don't have an exact time. The clamav daemon produces logs like the following:
> >
> > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error: cli_cvdverify: Can't read CVD header
> > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error: Can't load /var/lib/clamav/bytecode.cld: Broken or not a CVD file
> > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error: cli_loaddbdir(): error loading database /var/lib/clamav/bytecode.cld
> > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: Mon Feb 27 14:39:11 2023 -> !Broken or not a CVD file
> > Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service: Main process exited, code=exited, status=1/FAILURE
> > Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service: Failed with result 'exit-code'.
> > Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service: Consumed 8.679s CPU time.
> >
> >
> > I feel like I have narrowed the problem down to a 0 length 'bytecode.cvd' file.  Here is a listing of the definitions directory:
> >
> > $ ls -l /var/lib/clamav
> > total 226168
> > -rw-r--r-- 1 clamav clamav    314802 Feb 27 14:06 bytecode.cld
> > -rw-r--r-- 1 clamav clamav         0 Feb 27 02:00 bytecode.cvd
> > -rw-r--r-- 1 clamav clamav  60787973 Feb 27 10:01 daily.cld
> > -rw-r--r-- 1 clamav clamav        69 Feb 23 15:33 freshclam.dat
> > -rw-r--r-- 1 clamav clamav 170479789 Feb 27 02:00 main.cvd
> >
> >
> > My initial fix (before narrowing the problem down to bytecode.cvd) was to
> >
> > 1. stop freshclam
> > 2. clean this directory
> > 3. restart freshclam
> > 4. give it time to get the definitions (from a private mirror)
> > 5. start clamav daemon
> >
> > This would work for maybe 1/2 day then the empty bytecode.cvd file would reappear and the daemon would fail.
> >
> > This morning I was able to spend some more time and find that it was just the one file that needed to be removed.
> >
> > I have a local mirror because there are several instances of this scanner in use (at least 2 instances for several environments).  I have checked the mirror and it appears to be working fine and keeping the definitions up to date inside our environment.  In addition, the scanner instances appear to be keeping the local set of definitions up to date with the mirror.
> >
> > The mirror does not have a bytecode.cvd file on it (here is a listing of its definitions directory)
> >
> > $ ls -l /var/lib/clamav
> > total 226172
> > -rw-r--r-- 1 clamav clamav    314802 Feb 22 22:02 bytecode.cld
> > -rw-r--r-- 1 clamav clamav  60787973 Feb 27 09:06 daily.cld
> > -rw-r--r-- 1 clamav clamav        69 Jan 29  2022 freshclam.dat
> > -rw-r--r-- 1 clamav clamav 170479789 Jan 29  2022 main.cvd
> > -rw-r--r-- 1 clamav clamav        87 Jan 29  2022 test.html
> >
> >
> > To the best of my knowledge, the software is up to date:
> >
> > $ sudo freshclam -V
> > ClamAV 0.103.8/26825/Mon Feb 27 08:24:38 2023
> >
> >
> > Here is the freshclam.conf used on all the local sanner instances
> >
> > $ cat /etc/clamav/freshclam.conf
> > # Automatically created by the clamav-freshclam postinst
> > # Comments will get lost when you reconfigure the clamav-freshclam package
> >
> > DatabaseOwner clamav
> > UpdateLogFile /var/log/clamav/freshclam.log
> > LogVerbose false
> > LogSyslog false
> > LogFacility LOG_LOCAL6
> > LogFileMaxSize 0
> > LogRotate true
> > LogTime true
> > Foreground false
> > Debug false
> > MaxAttempts 5
> > DatabaseDirectory /var/lib/clamav
> > DNSDatabaseInfo current.cvd.clamav.net <http://current.cvd.clamav.net> <http://current.cvd.clamav.net>
> > ConnectTimeout 30
> > ReceiveTimeout 0
> > TestDatabases yes
> > CompressLocalDatabase no
> > Bytecode true
> > NotifyClamd /etc/clamav/clamd.conf
> > # Check for new database 24 times a day
> > Checks 24
> > PrivateMirror http://10.50.0.2
> > ScriptedUpdates no
> > PrivateMirror http://10.50.0.2
> >
> >
> > The scanner has been working fine for about 12 months, keeping the software and the definitions up to date.   The only configuration item that seems to relate is "Bytecode true", but the description seems to discuss just the downloading of the file, not whether it is created on the local instance.
> >
> > Does anyone have any pointers?
> >
> > Thanks
> > Kevin
> > --
> >
> > *Kevin O'Connor*
> > Principal DevOps Engineer
> > M: 617-834-1291
> >
> > email-footer-logos.jpg (1000×120)
> >
> > STATEMENT OF CONFIDENTIALITY: The information contained in this message and any attachments are intended solely for the addressee(s) and may contain confidential or privileged information. If you are not the intended recipient, or responsible for delivering the e-mail to the intended recipient, you have received this message in error. Any use, dissemination, forwarding, printing, or copying is strictly prohibited. Please notify Ampion immediately at security@ampion.net and destroy all copies of this message and any attachments.
> >
> >
> > _______________________________________________
> >
> > Manage your clamav-users mailing list subscription / unsubscribe:
> > https://lists.clamav.net/mailman/listinfo/clamav-users
> >
> >
> > Help us build a comprehensive ClamAV guide:
> > https://github.com/Cisco-Talos/clamav-documentation
> >
> > https://docs.clamav.net/#mailing-lists-and-chat
>
> _______________________________________________
>
> Manage your clamav-users mailing list subscription / unsubscribe:
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/Cisco-Talos/clamav-documentation
>
> https://docs.clamav.net/#mailing-lists-and-chat
>

_______________________________________________

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat
Re: 0 length bytecode.cvd causing problems with clamav daemon [ In reply to ]
Marc,

I had a similar understanding of that document. That is; if there is no
bytecode.cvd pushed by the ClamAV team, it should not exist on my local
scanners. When I checked the mirror and there was no bytecode.cvd file, yet
it appeared on my scanner machines with 0 length, I figured that the new
release had highlighted a misconfiguration in my freshclam.conf that the
earlier version was more forgiving of. However I have not found what that
might be.

Your idea of removing all the files in the /var/lib/clamav directory is
what I found worked initially, but that seems like a poor workaround as I
need this running all the time. I don't know when our clients will drop
files on us that need a scan.

Thanks for looking at it.

Kevin

On Mon, Feb 27, 2023 at 1:11 PM Marc via clamav-users <
clamav-users@lists.clamav.net> wrote:

> i would suggest, to delete alle libraries in /var/lib/clamav and download
> all complete new.
> CLD Files comes not regularly, normally we have CVD only.
>
> If i understand this well, CLD Files comes only when error occures while
> updating.
> https://blog.clamav.net/2021/03/clamav-cvds-cdiffs-and-magic-behind.html
> <https://blog.clamav.net/2021/03/clamav-cvds-cdiffs-and-magic-behind.html>
>
>
> Von / From: Kevin O'connor <mailto:koconnor@ampion.net>
> An / To: Newcomer01 <mailto:newcomer01@posteo.de>
> Gesendet / Sent: Montag, Februar 27, 2023 um 18:38 (at 06:38 PM) +0100
> Betreff / Subject: Re: [clamav-users] 0 length bytecode.cvd causing
> problems with clamav daemon
> > Heh, good question. Just checked again, and it looks like that was a
> copy-paste error. There is only one PrivateMirror line.
> > Kevin
> >
> > On Mon, Feb 27, 2023 at 12:02 PM newcomer01 via clamav-users <
> clamav-users@lists.clamav.net> wrote:
> >
> > why you have set two times the "PrivateMirror" with identically IP's?
> > Can't believe that this happens with the automated PostInst ????
> >
> >
> > Von / From: Clamav User Mailinglist <mailto:
> clamav-users@lists.clamav.net>
> > An / To: Newcomer01 <mailto:newcomer01@posteo.de>
> > CC / CC: Kevin O'connor <mailto:koconnor@ampion.net>
> > Gesendet / Sent: Montag, Februar 27, 2023 um 16:58 (at 04:58 PM) +0100
> > Betreff / Subject: [clamav-users] 0 length bytecode.cvd causing problems
> with clamav daemon
> > > I am having an issue with 0 length bytecode.cvd files on my scanner
> instances. This seems to have started sometime on 22 Feb, I'm afraid I
> don't have an exact time. The clamav daemon produces logs like the
> following:
> > >
> > > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error:
> cli_cvdverify: Can't read CVD header
> > > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error: Can't
> load /var/lib/clamav/bytecode.cld: Broken or not a CVD file
> > > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error:
> cli_loaddbdir(): error loading database /var/lib/clamav/bytecode.cld
> > > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: Mon Feb 27 14:39:11 2023
> -> !Broken or not a CVD file
> > > Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service: Main
> process exited, code=exited, status=1/FAILURE
> > > Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service: Failed
> with result 'exit-code'.
> > > Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service:
> Consumed 8.679s CPU time.
> > >
> > >
> > > I feel like I have narrowed the problem down to a 0 length
> 'bytecode.cvd' file. Here is a listing of the definitions directory:
> > >
> > > $ ls -l /var/lib/clamav
> > > total 226168
> > > -rw-r--r-- 1 clamav clamav 314802 Feb 27 14:06 bytecode.cld
> > > -rw-r--r-- 1 clamav clamav 0 Feb 27 02:00 bytecode.cvd
> > > -rw-r--r-- 1 clamav clamav 60787973 Feb 27 10:01 daily.cld
> > > -rw-r--r-- 1 clamav clamav 69 Feb 23 15:33 freshclam.dat
> > > -rw-r--r-- 1 clamav clamav 170479789 Feb 27 02:00 main.cvd
> > >
> > >
> > > My initial fix (before narrowing the problem down to bytecode.cvd) was
> to
> > >
> > > 1. stop freshclam
> > > 2. clean this directory
> > > 3. restart freshclam
> > > 4. give it time to get the definitions (from a private mirror)
> > > 5. start clamav daemon
> > >
> > > This would work for maybe 1/2 day then the empty bytecode.cvd file
> would reappear and the daemon would fail.
> > >
> > > This morning I was able to spend some more time and find that it was
> just the one file that needed to be removed.
> > >
> > > I have a local mirror because there are several instances of this
> scanner in use (at least 2 instances for several environments). I have
> checked the mirror and it appears to be working fine and keeping the
> definitions up to date inside our environment. In addition, the scanner
> instances appear to be keeping the local set of definitions up to date with
> the mirror.
> > >
> > > The mirror does not have a bytecode.cvd file on it (here is a listing
> of its definitions directory)
> > >
> > > $ ls -l /var/lib/clamav
> > > total 226172
> > > -rw-r--r-- 1 clamav clamav 314802 Feb 22 22:02 bytecode.cld
> > > -rw-r--r-- 1 clamav clamav 60787973 Feb 27 09:06 daily.cld
> > > -rw-r--r-- 1 clamav clamav 69 Jan 29 2022 freshclam.dat
> > > -rw-r--r-- 1 clamav clamav 170479789 Jan 29 2022 main.cvd
> > > -rw-r--r-- 1 clamav clamav 87 Jan 29 2022 test.html
> > >
> > >
> > > To the best of my knowledge, the software is up to date:
> > >
> > > $ sudo freshclam -V
> > > ClamAV 0.103.8/26825/Mon Feb 27 08:24:38 2023
> > >
> > >
> > > Here is the freshclam.conf used on all the local sanner instances
> > >
> > > $ cat /etc/clamav/freshclam.conf
> > > # Automatically created by the clamav-freshclam postinst
> > > # Comments will get lost when you reconfigure the clamav-freshclam
> package
> > >
> > > DatabaseOwner clamav
> > > UpdateLogFile /var/log/clamav/freshclam.log
> > > LogVerbose false
> > > LogSyslog false
> > > LogFacility LOG_LOCAL6
> > > LogFileMaxSize 0
> > > LogRotate true
> > > LogTime true
> > > Foreground false
> > > Debug false
> > > MaxAttempts 5
> > > DatabaseDirectory /var/lib/clamav
> > > DNSDatabaseInfo current.cvd.clamav.net <http://current.cvd.clamav.net
> <http://current.cvd.clamav.net>>
> <http://current.cvd.clamav.net
> <http://current.cvd.clamav.net>
> >
> > > ConnectTimeout 30
> > > ReceiveTimeout 0
> > > TestDatabases yes
> > > CompressLocalDatabase no
> > > Bytecode true
> > > NotifyClamd /etc/clamav/clamd.conf
> > > # Check for new database 24 times a day
> > > Checks 24
> > > PrivateMirror http://10.50.0.2
> <http://10.50.0.2>
> > > ScriptedUpdates no
> > > PrivateMirror http://10.50.0.2
> <http://10.50.0.2>
> > >
> > >
> > > The scanner has been working fine for about 12 months, keeping the
> software and the definitions up to date. The only configuration item that
> seems to relate is "Bytecode true", but the description seems to discuss
> just the downloading of the file, not whether it is created on the local
> instance.
> > >
> > > Does anyone have any pointers?
> > >
> > > Thanks
> > > Kevin
> > > --
> > >
> > > *Kevin O'Connor*
> > > Principal DevOps Engineer
> > > M: 617-834-1291
> > >
> > > email-footer-logos.jpg (1000×120)
> > >
> > > STATEMENT OF CONFIDENTIALITY: The information contained in this
> message and any attachments are intended solely for the addressee(s) and
> may contain confidential or privileged information. If you are not the
> intended recipient, or responsible for delivering the e-mail to the
> intended recipient, you have received this message in error. Any use,
> dissemination, forwarding, printing, or copying is strictly prohibited.
> Please notify Ampion immediately at security@ampion.net and destroy all
> copies of this message and any attachments.
> > >
> > >
> > > _______________________________________________
> > >
> > > Manage your clamav-users mailing list subscription / unsubscribe:
> > > https://lists.clamav.net/mailman/listinfo/clamav-users
> <https://lists.clamav.net/mailman/listinfo/clamav-users>
> > >
> > >
> > > Help us build a comprehensive ClamAV guide:
> > > https://github.com/Cisco-Talos/clamav-documentation
> <https://github.com/Cisco-Talos/clamav-documentation>
> > >
> > > https://docs.clamav.net/#mailing-lists-and-chat
> <https://docs.clamav.net/#mailing-lists-and-chat>
> >
> > _______________________________________________
> >
> > Manage your clamav-users mailing list subscription / unsubscribe:
> > https://lists.clamav.net/mailman/listinfo/clamav-users
> <https://lists.clamav.net/mailman/listinfo/clamav-users>
> >
> >
> > Help us build a comprehensive ClamAV guide:
> > https://github.com/Cisco-Talos/clamav-documentation
> <https://github.com/Cisco-Talos/clamav-documentation>
> >
> > https://docs.clamav.net/#mailing-lists-and-chat
> <https://docs.clamav.net/#mailing-lists-and-chat>
> >
>
> _______________________________________________
>
> Manage your clamav-users mailing list subscription / unsubscribe:
> https://lists.clamav.net/mailman/listinfo/clamav-users
> <https://lists.clamav.net/mailman/listinfo/clamav-users>
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/Cisco-Talos/clamav-documentation
> <https://github.com/Cisco-Talos/clamav-documentation>
>
> https://docs.clamav.net/#mailing-lists-and-chat
> <https://docs.clamav.net/#mailing-lists-and-chat>
>
Re: 0 length bytecode.cvd causing problems with clamav daemon [ In reply to ]
The bytecode.cvd? file is the original.
When there is an update, we publish two things:

1. a bytecode.cdiff? patch file that will update the older bytecode.cvd? to the newest version. This is the "scripted update" mechanism.

If using the .cdiff? patch file to update, it should replace the old bytecode.cvd? with a new bytecode.cld?. We may issue an empty patch file (zero-bytes) to tell freshclam? to download the whole bytecode.cvd? instead. We do this if the patch is so big it is better to just download the whole file, or if is a bug preventing the patch file from working correctly, which there presently is for bytecode signatures (sad!).

This .cdiff? update mechanism would not be used in your situation because ScriptedUpdates is disabled.

2. a new bytecode.cvd?.

This should only be downloaded in two cases: A) If you do not have the old bytecode.cvd? (or cld?) and thus cannot use the patch file to update. And B) If the bytecode.cdiff? patch file is empty.

The issue you're facing feels to me like an issue with what the private mirror is serving. Can you please check if it is serving an empty bytecode.cvd?? It feels like it may be serving both the empty bytecode.cvd? and a bytecode.cld?.

If that's not the case, then we may have a bug in freshclam? and I would love some more information on what freshclam? is downloading when it runs in order to get into this strange state.

Best,
Micah

Micah Snyder
ClamAV Development
Talos
Cisco Systems, Inc.

________________________________
From: clamav-users <clamav-users-bounces@lists.clamav.net> on behalf of Kevin O'Connor via clamav-users <clamav-users@lists.clamav.net>
Sent: Monday, February 27, 2023 11:12 AM
To: ClamAV users ML <clamav-users@lists.clamav.net>
Cc: Kevin O'Connor <koconnor@ampion.net>
Subject: Re: [clamav-users] 0 length bytecode.cvd causing problems with clamav daemon

Marc,

I had a similar understanding of that document. That is; if there is no bytecode.cvd pushed by the ClamAV team, it should not exist on my local scanners. When I checked the mirror and there was no bytecode.cvd file, yet it appeared on my scanner machines with 0 length, I figured that the new release had highlighted a misconfiguration in my freshclam.conf that the earlier version was more forgiving of. However I have not found what that might be.

Your idea of removing all the files in the /var/lib/clamav directory is what I found worked initially, but that seems like a poor workaround as I need this running all the time. I don't know when our clients will drop files on us that need a scan.

Thanks for looking at it.

Kevin

On Mon, Feb 27, 2023 at 1:11 PM Marc via clamav-users <clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>> wrote:
i would suggest, to delete alle libraries in /var/lib/clamav and download all complete new.
CLD Files comes not regularly, normally we have CVD only.

If i understand this well, CLD Files comes only when error occures while updating.
https://blog.clamav.net/2021/03/clamav-cvds-cdiffs-and-magic-behind.html


Von / From: Kevin O'connor <mailto:koconnor@ampion.net<mailto:koconnor@ampion.net>>
An / To: Newcomer01 <mailto:newcomer01@posteo.de<mailto:newcomer01@posteo.de>>
Gesendet / Sent: Montag, Februar 27, 2023 um 18:38 (at 06:38 PM) +0100
Betreff / Subject: Re: [clamav-users] 0 length bytecode.cvd causing problems with clamav daemon
> Heh, good question. Just checked again, and it looks like that was a copy-paste error. There is only one PrivateMirror line.
> Kevin
>
> On Mon, Feb 27, 2023 at 12:02 PM newcomer01 via clamav-users <clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>> wrote:
>
> why you have set two times the "PrivateMirror" with identically IP's?
> Can't believe that this happens with the automated PostInst ????
>
>
> Von / From: Clamav User Mailinglist <mailto:clamav-users@lists.clamav.net<mailto:clamav-users@lists.clamav.net>>
> An / To: Newcomer01 <mailto:newcomer01@posteo.de<mailto:newcomer01@posteo.de>>
> CC / CC: Kevin O'connor <mailto:koconnor@ampion.net<mailto:koconnor@ampion.net>>
> Gesendet / Sent: Montag, Februar 27, 2023 um 16:58 (at 04:58 PM) +0100
> Betreff / Subject: [clamav-users] 0 length bytecode.cvd causing problems with clamav daemon
> > I am having an issue with 0 length bytecode.cvd files on my scanner instances. This seems to have started sometime on 22 Feb, I'm afraid I don't have an exact time. The clamav daemon produces logs like the following:
> >
> > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error: cli_cvdverify: Can't read CVD header
> > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error: Can't load /var/lib/clamav/bytecode.cld: Broken or not a CVD file
> > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error: cli_loaddbdir(): error loading database /var/lib/clamav/bytecode.cld
> > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: Mon Feb 27 14:39:11 2023 -> !Broken or not a CVD file
> > Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service: Main process exited, code=exited, status=1/FAILURE
> > Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service: Failed with result 'exit-code'.
> > Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service: Consumed 8.679s CPU time.
> >
> >
> > I feel like I have narrowed the problem down to a 0 length 'bytecode.cvd' file. Here is a listing of the definitions directory:
> >
> > $ ls -l /var/lib/clamav
> > total 226168
> > -rw-r--r-- 1 clamav clamav 314802 Feb 27 14:06 bytecode.cld
> > -rw-r--r-- 1 clamav clamav 0 Feb 27 02:00 bytecode.cvd
> > -rw-r--r-- 1 clamav clamav 60787973 Feb 27 10:01 daily.cld
> > -rw-r--r-- 1 clamav clamav 69 Feb 23 15:33 freshclam.dat
> > -rw-r--r-- 1 clamav clamav 170479789 Feb 27 02:00 main.cvd
> >
> >
> > My initial fix (before narrowing the problem down to bytecode.cvd) was to
> >
> > 1. stop freshclam
> > 2. clean this directory
> > 3. restart freshclam
> > 4. give it time to get the definitions (from a private mirror)
> > 5. start clamav daemon
> >
> > This would work for maybe 1/2 day then the empty bytecode.cvd file would reappear and the daemon would fail.
> >
> > This morning I was able to spend some more time and find that it was just the one file that needed to be removed.
> >
> > I have a local mirror because there are several instances of this scanner in use (at least 2 instances for several environments). I have checked the mirror and it appears to be working fine and keeping the definitions up to date inside our environment. In addition, the scanner instances appear to be keeping the local set of definitions up to date with the mirror.
> >
> > The mirror does not have a bytecode.cvd file on it (here is a listing of its definitions directory)
> >
> > $ ls -l /var/lib/clamav
> > total 226172
> > -rw-r--r-- 1 clamav clamav 314802 Feb 22 22:02 bytecode.cld
> > -rw-r--r-- 1 clamav clamav 60787973 Feb 27 09:06 daily.cld
> > -rw-r--r-- 1 clamav clamav 69 Jan 29 2022 freshclam.dat
> > -rw-r--r-- 1 clamav clamav 170479789 Jan 29 2022 main.cvd
> > -rw-r--r-- 1 clamav clamav 87 Jan 29 2022 test.html
> >
> >
> > To the best of my knowledge, the software is up to date:
> >
> > $ sudo freshclam -V
> > ClamAV 0.103.8/26825/Mon Feb 27 08:24:38 2023
> >
> >
> > Here is the freshclam.conf used on all the local sanner instances
> >
> > $ cat /etc/clamav/freshclam.conf
> > # Automatically created by the clamav-freshclam postinst
> > # Comments will get lost when you reconfigure the clamav-freshclam package
> >
> > DatabaseOwner clamav
> > UpdateLogFile /var/log/clamav/freshclam.log
> > LogVerbose false
> > LogSyslog false
> > LogFacility LOG_LOCAL6
> > LogFileMaxSize 0
> > LogRotate true
> > LogTime true
> > Foreground false
> > Debug false
> > MaxAttempts 5
> > DatabaseDirectory /var/lib/clamav
> > DNSDatabaseInfo current.cvd.clamav.net<http://current.cvd.clamav.net> <http://current.cvd.clamav.net> <http://current.cvd.clamav.net>
> > ConnectTimeout 30
> > ReceiveTimeout 0
> > TestDatabases yes
> > CompressLocalDatabase no
> > Bytecode true
> > NotifyClamd /etc/clamav/clamd.conf
> > # Check for new database 24 times a day
> > Checks 24
> > PrivateMirror http://10.50.0.2
> > ScriptedUpdates no
> > PrivateMirror http://10.50.0.2
> >
> >
> > The scanner has been working fine for about 12 months, keeping the software and the definitions up to date. The only configuration item that seems to relate is "Bytecode true", but the description seems to discuss just the downloading of the file, not whether it is created on the local instance.
> >
> > Does anyone have any pointers?
> >
> > Thanks
> > Kevin
> > --
> >
> > *Kevin O'Connor*
> > Principal DevOps Engineer
> > M: 617-834-1291
> >
> > email-footer-logos.jpg (1000×120)
> >
> > STATEMENT OF CONFIDENTIALITY: The information contained in this message and any attachments are intended solely for the addressee(s) and may contain confidential or privileged information. If you are not the intended recipient, or responsible for delivering the e-mail to the intended recipient, you have received this message in error. Any use, dissemination, forwarding, printing, or copying is strictly prohibited. Please notify Ampion immediately at security@ampion.net<mailto:security@ampion.net> and destroy all copies of this message and any attachments.
> >
> >
> > _______________________________________________
> >
> > Manage your clamav-users mailing list subscription / unsubscribe:
> > https://lists.clamav.net/mailman/listinfo/clamav-users
> >
> >
> > Help us build a comprehensive ClamAV guide:
> > https://github.com/Cisco-Talos/clamav-documentation
> >
> > https://docs.clamav.net/#mailing-lists-and-chat
>
> _______________________________________________
>
> Manage your clamav-users mailing list subscription / unsubscribe:
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/Cisco-Talos/clamav-documentation
>
> https://docs.clamav.net/#mailing-lists-and-chat
>

_______________________________________________

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat
Re: 0 length bytecode.cvd causing problems with clamav daemon [ In reply to ]
Hi Micah,

I appreciate your response. It has been driving me nuts since it started
about a week ago. Things had been humming along nicely for over a year
until ~22 Feb.

So, as best I can tell at this point, the mirror does not have a
bytecode.cvd to serve up (0 length or otherwise). Here is a listing of
/var/lib/clamav on the mirror.

koconnor@ampion-clamav-mirror:~$ ls -l /var/lib/clamav
total 226196
-rw-r--r-- 1 clamav clamav 314802 Feb 22 22:02 bytecode.cld
-rw-r--r-- 1 clamav clamav 60814501 Mar 1 09:07 daily.cld
-rw-r--r-- 1 clamav clamav 69 Jan 29 2022 freshclam.dat
-rw-r--r-- 1 clamav clamav 170479789 Jan 29 2022 main.cvd
-rw-r--r-- 1 clamav clamav 87 Jan 29 2022 test.html


This is the version of freshclam on the mirror:

koconnor@ampion-clamav-mirror:~$ freshclam -V
ClamAV 0.103.8/26827/Wed Mar 1 08:28:49 2023


And the freshclam.conf on the mirror too.

koconnor@ampion-clamav-mirror:~$ cat /etc/clamav/freshclam.conf
# Automatically created by the clamav-freshclam postinst
# Comments will get lost when you reconfigure the clamav-freshclam package

DatabaseOwner clamav
UpdateLogFile /var/log/clamav/freshclam.log
LogVerbose false
LogSyslog false
LogFacility LOG_LOCAL6
LogFileMaxSize 0
LogRotate true
LogTime true
Foreground false
Debug false
MaxAttempts 5
DatabaseDirectory /var/lib/clamav
DNSDatabaseInfo current.cvd.clamav.net
ConnectTimeout 30
ReceiveTimeout 0
TestDatabases yes
ScriptedUpdates yes
CompressLocalDatabase yes
Bytecode true
NotifyClamd /etc/clamav/clamd.conf
# Check for new database 24 times a day
Checks 24
DatabaseMirror db.local.clamav.net
DatabaseMirror database.clamav.net


I did find something interesting in the logs on the mirror server. First a
listing of the log directory:

koconnor@ampion-clamav-mirror:~$ ls -l /var/log/clamav/*
-rw-r----- 1 clamav clamav 57381 Mar 1 13:07 /var/log/clamav/freshclam.log
-rw-r----- 1 clamav adm 142086 Feb 26 00:00
/var/log/clamav/freshclam.log.1
-rw-r----- 1 clamav clamav 5142 Dec 25 00:00
/var/log/clamav/freshclam.log.10.gz
-rw-r----- 1 clamav adm 5002 Dec 18 00:00
/var/log/clamav/freshclam.log.11.gz
-rw-r----- 1 clamav adm 5008 Dec 11 00:00
/var/log/clamav/freshclam.log.12.gz
-rw-r----- 1 clamav adm 6158 Feb 19 00:00
/var/log/clamav/freshclam.log.2.gz
-rw-r----- 1 clamav adm 4997 Feb 12 00:00
/var/log/clamav/freshclam.log.3.gz
-rw-r----- 1 clamav clamav 5148 Feb 5 00:00
/var/log/clamav/freshclam.log.4.gz
-rw-r----- 1 clamav adm 5023 Jan 29 00:00
/var/log/clamav/freshclam.log.5.gz
-rw-r----- 1 clamav adm 5008 Jan 22 00:00
/var/log/clamav/freshclam.log.6.gz
-rw-r----- 1 clamav adm 4990 Jan 15 00:00
/var/log/clamav/freshclam.log.7.gz
-rw-r----- 1 clamav adm 5009 Jan 8 00:00
/var/log/clamav/freshclam.log.8.gz
-rw-r----- 1 clamav clamav 5174 Jan 1 00:00
/var/log/clamav/freshclam.log.9.gz


Then a search for bytecode.cvd in the most recent log file:

koconnor@ampion-clamav-mirror:~$ sudo grep bytecode.cvd
/var/log/clamav/freshclam.log
koconnor@ampion-clamav-mirror:~$


Followed by a search for that string in the next most recent file:

koconnor@ampion-clamav-mirror:~$ sudo grep bytecode.cvd
/var/log/clamav/freshclam.log.1
Sun Feb 19 00:00:35 2023 -> bytecode.cvd database is up-to-date (version:
333, sigs: 92, f-level: 63, builder: awillia2)
Sun Feb 19 01:00:35 2023 -> bytecode.cvd database is up-to-date (version:
333, sigs: 92, f-level: 63, builder: awillia2)
Sun Feb 19 02:00:35 2023 -> bytecode.cvd database is up-to-date (version:
333, sigs: 92, f-level: 63, builder: awillia2)
Sun Feb 19 03:00:35 2023 -> bytecode.cvd database is up-to-date (version:
333, sigs: 92, f-level: 63, builder: awillia2)

<snip> this is repeated every hour <snip>
Wed Feb 22 18:02:30 2023 -> bytecode.cvd database is up-to-date (version:
333, sigs: 92, f-level: 63, builder: awillia2)
Wed Feb 22 19:02:31 2023 -> bytecode.cvd database is up-to-date (version:
333, sigs: 92, f-level: 63, builder: awillia2)
Wed Feb 22 20:02:31 2023 -> bytecode.cvd database is up-to-date (version:
333, sigs: 92, f-level: 63, builder: awillia2)
Wed Feb 22 21:02:31 2023 -> bytecode.cvd database is up-to-date (version:
333, sigs: 92, f-level: 63, builder: awillia2)
koconnor@ampion-clamav-mirror:~$


This is particularly interesting as the end of that output is approximately
the time when the problem started. Let me know if I should send you a copy
of any of the log files from the mirror. I wasn't sure if that was
appropriate for the listserv.

Thanks again

Kevin

On Tue, Feb 28, 2023 at 1:31 PM Micah Snyder (micasnyd) <micasnyd@cisco.com>
wrote:

> The bytecode.cvd? file is the original.
> When there is an update, we publish two things:
>
> 1. a bytecode.cdiff? patch file that will update the older bytecode.cvd?
> to the newest version. This is the "scripted update" mechanism.
>
> If using the .cdiff? patch file to update, it should replace the old
> bytecode.cvd? with a new bytecode.cld?. We may issue an empty patch
> file (zero-bytes) to tell freshclam? to download the whole bytecode.cvd?
> instead. We do this if the patch is so big it is better to just download
> the whole file, or if is a bug preventing the patch file from working
> correctly, which there presently is for bytecode signatures (sad!).
>
> This .cdiff? update mechanism would not be used in your situation
> because ScriptedUpdates is disabled.
>
> 2. a new bytecode.cvd?.
>
> This should only be downloaded in two cases: A) If you do not have the
> old bytecode.cvd? (or cld?) and thus cannot use the patch file to
> update. And B) If the bytecode.cdiff? patch file is empty.
>
> The issue you're facing feels to me like an issue with what the private
> mirror is serving. Can you please check if it is serving an empty
> bytecode.cvd?? It feels like it may be serving both the empty
> bytecode.cvd? and a bytecode.cld?.
>
> If that's not the case, then we may have a bug in freshclam? and I would
> love some more information on what freshclam? is downloading when it runs
> in order to get into this strange state.
>
> Best,
> Micah
>
> Micah Snyder
> ClamAV Development
> Talos
> Cisco Systems, Inc.
>
> ------------------------------
> *From:* clamav-users <clamav-users-bounces@lists.clamav.net> on behalf of
> Kevin O'Connor via clamav-users <clamav-users@lists.clamav.net>
> *Sent:* Monday, February 27, 2023 11:12 AM
> *To:* ClamAV users ML <clamav-users@lists.clamav.net>
> *Cc:* Kevin O'Connor <koconnor@ampion.net>
> *Subject:* Re: [clamav-users] 0 length bytecode.cvd causing problems with
> clamav daemon
>
> Marc,
>
> I had a similar understanding of that document. That is; if there is no
> bytecode.cvd pushed by the ClamAV team, it should not exist on my local
> scanners. When I checked the mirror and there was no bytecode.cvd file, yet
> it appeared on my scanner machines with 0 length, I figured that the new
> release had highlighted a misconfiguration in my freshclam.conf that the
> earlier version was more forgiving of. However I have not found what that
> might be.
>
> Your idea of removing all the files in the /var/lib/clamav directory is
> what I found worked initially, but that seems like a poor workaround as I
> need this running all the time. I don't know when our clients will drop
> files on us that need a scan.
>
> Thanks for looking at it.
>
> Kevin
>
> On Mon, Feb 27, 2023 at 1:11 PM Marc via clamav-users <
> clamav-users@lists.clamav.net> wrote:
>
> i would suggest, to delete alle libraries in /var/lib/clamav and download
> all complete new.
> CLD Files comes not regularly, normally we have CVD only.
>
> If i understand this well, CLD Files comes only when error occures while
> updating.
> https://blog.clamav.net/2021/03/clamav-cvds-cdiffs-and-magic-behind.html
> <https://blog.clamav.net/2021/03/clamav-cvds-cdiffs-and-magic-behind.html>
>
>
> Von / From: Kevin O'connor <mailto:koconnor@ampion.net>
> An / To: Newcomer01 <mailto:newcomer01@posteo.de>
> Gesendet / Sent: Montag, Februar 27, 2023 um 18:38 (at 06:38 PM) +0100
> Betreff / Subject: Re: [clamav-users] 0 length bytecode.cvd causing
> problems with clamav daemon
> > Heh, good question. Just checked again, and it looks like that was a
> copy-paste error. There is only one PrivateMirror line.
> > Kevin
> >
> > On Mon, Feb 27, 2023 at 12:02 PM newcomer01 via clamav-users <
> clamav-users@lists.clamav.net> wrote:
> >
> > why you have set two times the "PrivateMirror" with identically IP's?
> > Can't believe that this happens with the automated PostInst ????
> >
> >
> > Von / From: Clamav User Mailinglist <mailto:
> clamav-users@lists.clamav.net>
> > An / To: Newcomer01 <mailto:newcomer01@posteo.de>
> > CC / CC: Kevin O'connor <mailto:koconnor@ampion.net>
> > Gesendet / Sent: Montag, Februar 27, 2023 um 16:58 (at 04:58 PM) +0100
> > Betreff / Subject: [clamav-users] 0 length bytecode.cvd causing problems
> with clamav daemon
> > > I am having an issue with 0 length bytecode.cvd files on my scanner
> instances. This seems to have started sometime on 22 Feb, I'm afraid I
> don't have an exact time. The clamav daemon produces logs like the
> following:
> > >
> > > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error:
> cli_cvdverify: Can't read CVD header
> > > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error: Can't
> load /var/lib/clamav/bytecode.cld: Broken or not a CVD file
> > > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: LibClamAV Error:
> cli_loaddbdir(): error loading database /var/lib/clamav/bytecode.cld
> > > Feb 27 14:39:11 av-scan-wrhn clamd[163614]: Mon Feb 27 14:39:11 2023
> -> !Broken or not a CVD file
> > > Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service: Main
> process exited, code=exited, status=1/FAILURE
> > > Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service: Failed
> with result 'exit-code'.
> > > Feb 27 14:39:11 av-scan-wrhn systemd[1]: clamav-daemon.service:
> Consumed 8.679s CPU time.
> > >
> > >
> > > I feel like I have narrowed the problem down to a 0 length
> 'bytecode.cvd' file. Here is a listing of the definitions directory:
> > >
> > > $ ls -l /var/lib/clamav
> > > total 226168
> > > -rw-r--r-- 1 clamav clamav 314802 Feb 27 14:06 bytecode.cld
> > > -rw-r--r-- 1 clamav clamav 0 Feb 27 02:00 bytecode.cvd
> > > -rw-r--r-- 1 clamav clamav 60787973 Feb 27 10:01 daily.cld
> > > -rw-r--r-- 1 clamav clamav 69 Feb 23 15:33 freshclam.dat
> > > -rw-r--r-- 1 clamav clamav 170479789 Feb 27 02:00 main.cvd
> > >
> > >
> > > My initial fix (before narrowing the problem down to bytecode.cvd) was
> to
> > >
> > > 1. stop freshclam
> > > 2. clean this directory
> > > 3. restart freshclam
> > > 4. give it time to get the definitions (from a private mirror)
> > > 5. start clamav daemon
> > >
> > > This would work for maybe 1/2 day then the empty bytecode.cvd file
> would reappear and the daemon would fail.
> > >
> > > This morning I was able to spend some more time and find that it was
> just the one file that needed to be removed.
> > >
> > > I have a local mirror because there are several instances of this
> scanner in use (at least 2 instances for several environments). I have
> checked the mirror and it appears to be working fine and keeping the
> definitions up to date inside our environment. In addition, the scanner
> instances appear to be keeping the local set of definitions up to date with
> the mirror.
> > >
> > > The mirror does not have a bytecode.cvd file on it (here is a listing
> of its definitions directory)
> > >
> > > $ ls -l /var/lib/clamav
> > > total 226172
> > > -rw-r--r-- 1 clamav clamav 314802 Feb 22 22:02 bytecode.cld
> > > -rw-r--r-- 1 clamav clamav 60787973 Feb 27 09:06 daily.cld
> > > -rw-r--r-- 1 clamav clamav 69 Jan 29 2022 freshclam.dat
> > > -rw-r--r-- 1 clamav clamav 170479789 Jan 29 2022 main.cvd
> > > -rw-r--r-- 1 clamav clamav 87 Jan 29 2022 test.html
> > >
> > >
> > > To the best of my knowledge, the software is up to date:
> > >
> > > $ sudo freshclam -V
> > > ClamAV 0.103.8/26825/Mon Feb 27 08:24:38 2023
> > >
> > >
> > > Here is the freshclam.conf used on all the local sanner instances
> > >
> > > $ cat /etc/clamav/freshclam.conf
> > > # Automatically created by the clamav-freshclam postinst
> > > # Comments will get lost when you reconfigure the clamav-freshclam
> package
> > >
> > > DatabaseOwner clamav
> > > UpdateLogFile /var/log/clamav/freshclam.log
> > > LogVerbose false
> > > LogSyslog false
> > > LogFacility LOG_LOCAL6
> > > LogFileMaxSize 0
> > > LogRotate true
> > > LogTime true
> > > Foreground false
> > > Debug false
> > > MaxAttempts 5
> > > DatabaseDirectory /var/lib/clamav
> > > DNSDatabaseInfo current.cvd.clamav.net
> <http://current.cvd.clamav.net>
> <http://current.cvd.clamav.net
> <http://current.cvd.clamav.net>>
> <http://current.cvd.clamav.net
> <http://current.cvd.clamav.net>
> >
> > > ConnectTimeout 30
> > > ReceiveTimeout 0
> > > TestDatabases yes
> > > CompressLocalDatabase no
> > > Bytecode true
> > > NotifyClamd /etc/clamav/clamd.conf
> > > # Check for new database 24 times a day
> > > Checks 24
> > > PrivateMirror http://10.50.0.2
> <http://10.50.0.2>
> > > ScriptedUpdates no
> > > PrivateMirror http://10.50.0.2
> <http://10.50.0.2>
> > >
> > >
> > > The scanner has been working fine for about 12 months, keeping the
> software and the definitions up to date. The only configuration item that
> seems to relate is "Bytecode true", but the description seems to discuss
> just the downloading of the file, not whether it is created on the local
> instance.
> > >
> > > Does anyone have any pointers?
> > >
> > > Thanks
> > > Kevin
> > > --
> > >
> > > *Kevin O'Connor*
> > > Principal DevOps Engineer
> > > M: 617-834-1291
> > >
> > > email-footer-logos.jpg (1000×120)
> > >
> > > STATEMENT OF CONFIDENTIALITY: The information contained in this
> message and any attachments are intended solely for the addressee(s) and
> may contain confidential or privileged information. If you are not the
> intended recipient, or responsible for delivering the e-mail to the
> intended recipient, you have received this message in error. Any use,
> dissemination, forwarding, printing, or copying is strictly prohibited.
> Please notify Ampion immediately at security@ampion.net and destroy all
> copies of this message and any attachments.
> > >
> > >
> > > _______________________________________________
> > >
> > > Manage your clamav-users mailing list subscription / unsubscribe:
> > > https://lists.clamav.net/mailman/listinfo/clamav-users
> <https://lists.clamav.net/mailman/listinfo/clamav-users>
> > >
> > >
> > > Help us build a comprehensive ClamAV guide:
> > > https://github.com/Cisco-Talos/clamav-documentation
> <https://github.com/Cisco-Talos/clamav-documentation>
> > >
> > > https://docs.clamav.net/#mailing-lists-and-chat
> <https://docs.clamav.net/#mailing-lists-and-chat>
> >
> > _______________________________________________
> >
> > Manage your clamav-users mailing list subscription / unsubscribe:
> > https://lists.clamav.net/mailman/listinfo/clamav-users
> <https://lists.clamav.net/mailman/listinfo/clamav-users>
> >
> >
> > Help us build a comprehensive ClamAV guide:
> > https://github.com/Cisco-Talos/clamav-documentation
> <https://github.com/Cisco-Talos/clamav-documentation>
> >
> > https://docs.clamav.net/#mailing-lists-and-chat
> <https://docs.clamav.net/#mailing-lists-and-chat>
> >
>
> _______________________________________________
>
> Manage your clamav-users mailing list subscription / unsubscribe:
> https://lists.clamav.net/mailman/listinfo/clamav-users
> <https://lists.clamav.net/mailman/listinfo/clamav-users>
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/Cisco-Talos/clamav-documentation
> <https://github.com/Cisco-Talos/clamav-documentation>
>
> https://docs.clamav.net/#mailing-lists-and-chat
> <https://docs.clamav.net/#mailing-lists-and-chat>
>
>