Mailing List Archive

Re: [ext] ClamAV 1.0.0 release candidate now available
* Micah Snyder (micasnyd) via clamav-users <clamav-users@lists.clamav.net>:

> We are excited to announce the ClamAV 1.0.0 release candidate!

I'm seeing log entries like this for the machines with 1.0.0-rc
indicating the daily.cld update failed:

Oct 28 00:06:46 de freshclam[1878609]: Fri Oct 28 00:06:46 2022 -> daily database available for update (local version: 26700, remote version: 26701)
Oct 28 00:06:48 de freshclam[1878609]: WARNING: Fri Oct 28 00:06:48 2022 -> [LibClamAV] CVD verification failed for: daily.cld
Oct 28 00:06:48 de freshclam[1878609]: ERROR: Fri Oct 28 00:06:48 2022 -> mkdir_and_chdir_for_cdiff_tmp: Can't unpack daily.cld into /var/lib/clamav/tmp.1e2a6b8a16/clamav-09a73c546a48c9737e48f49fcc7d4195.tmp
Oct 28 00:06:48 de freshclam[1878609]: Fri Oct 28 00:06:48 2022 -> The database server doesn't have the latest patch for the daily database (version 26701). The server will likely have updated if you check again in a few hours.

Checking the permissions on /var/lib/clamav/:

# ls -ld /var/lib/clamav/
drwxr-xr-x 3 clamav clamav 4096 Okt 28 08:49 /var/lib/clamav/

Checking the current state of affairs (it's 09:00am here):
==========================================================

# clamd --version
ClamAV 1.0.0-rc/26700/Wed Oct 26 09:55:46 2022

checked apparmor (removed the profile to be on the safe side for the
tests):

Oct 28 09:06:15 de kernel: [1525842.556230] audit: type=1400 audit(1666940775.160:86): apparmor="STATUS" operation="profile_remove" profile="unconfined" name="/usr/bin/freshclam" pid=2535488 comm="apparmor_parser"

I restarted freshclam to see what happens:

Fri Oct 28 09:07:10 2022 -> --------------------------------------
Fri Oct 28 09:07:10 2022 -> freshclam daemon 1.0.0-rc (OS: Linux, ARCH: x86_64, CPU: x86_64)
Fri Oct 28 09:07:10 2022 -> ClamAV update process started at Fri Oct 28 09:07:10 2022
Fri Oct 28 09:07:10 2022 -> daily database available for update (local version: 26700, remote version: 26701)
Fri Oct 28 09:07:10 2022 -> WARNING: [LibClamAV] CVD verification failed for: daily.cld
Fri Oct 28 09:07:10 2022 -> ERROR: mkdir_and_chdir_for_cdiff_tmp: Can't unpack daily.cld into /var/lib/clamav/tmp.3bbb7ed4d7/clamav-bfba84844f1170e4c4210f03d1759097.tmp
Fri Oct 28 09:07:10 2022 -> The database server doesn't have the latest patch for the daily database (version 26701). The server will likely have updated if you check again in a few hours.
Fri Oct 28 09:07:10 2022 -> main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
Fri Oct 28 09:07:10 2022 -> bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2)
Fri Oct 28 09:07:10 2022 -> --------------------------------------

Still failing.

--

Ralf Hildebrandt
Charité - Universitätsmedizin Berlin
Geschäftsbereich IT | Abteilung Netzwerk

Campus Benjamin Franklin (CBF)
Haus I | 1. OG | Raum 105
Hindenburgdamm 30 | D-12203 Berlin

Tel. +49 30 450 570 155
ralf.hildebrandt@charite.de
https://www.charite.de
_______________________________________________

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat
Re: [ext] ClamAV 1.0.0 release candidate now available [ In reply to ]
> Fri Oct 28 09:07:10 2022 -> --------------------------------------
> Fri Oct 28 09:07:10 2022 -> freshclam daemon 1.0.0-rc (OS: Linux, ARCH: x86_64, CPU: x86_64)
> Fri Oct 28 09:07:10 2022 -> ClamAV update process started at Fri Oct 28 09:07:10 2022
> Fri Oct 28 09:07:10 2022 -> daily database available for update (local version: 26700, remote version: 26701)
> Fri Oct 28 09:07:10 2022 -> WARNING: [LibClamAV] CVD verification failed for: daily.cld
> Fri Oct 28 09:07:10 2022 -> ERROR: mkdir_and_chdir_for_cdiff_tmp: Can't unpack daily.cld into /var/lib/clamav/tmp.3bbb7ed4d7/clamav-bfba84844f1170e4c4210f03d1759097.tmp
> Fri Oct 28 09:07:10 2022 -> The database server doesn't have the latest patch for the daily database (version 26701). The server will likely have updated if you check again in a few hours.
> Fri Oct 28 09:07:10 2022 -> main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
> Fri Oct 28 09:07:10 2022 -> bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2)
> Fri Oct 28 09:07:10 2022 -> --------------------------------------

Another data point - I checked another machine which successfully
updated to 26701 (yesterday already!):

Thu Oct 27 10:00:06 2022 -> --------------------------------------
Thu Oct 27 11:00:06 2022 -> Received signal: wake up
Thu Oct 27 11:00:06 2022 -> ClamAV update process started at Thu Oct 27 11:00:06 2022
Thu Oct 27 11:00:06 2022 -> daily database available for update (local version: 26699, remote version: 26701)
Thu Oct 27 11:00:06 2022 -> WARNING: [LibClamAV] CVD verification failed for: daily.cld
Thu Oct 27 11:00:06 2022 -> ERROR: mkdir_and_chdir_for_cdiff_tmp: Can't unpack daily.cld into /var/lib/clamav/tmp.bfd8f6c0fe/clamav-91f69d4433a1975076fd9905e1f5ca06.tmp
Thu Oct 27 11:00:06 2022 -> WARNING: Incremental update failed, trying to download daily.cvd
Thu Oct 27 11:00:09 2022 -> Testing database: '/var/lib/clamav/tmp.bfd8f6c0fe/clamav-4ad0a44cd8a0ebe2bf630a0b92819105.tmp-daily.cvd'...
Thu Oct 27 11:00:19 2022 -> Database test passed.
Thu Oct 27 11:00:19 2022 -> daily.cvd updated (version: 26701, sigs: 2009238, f-level: 90, builder: raynman)
Thu Oct 27 11:00:19 2022 -> main.cld database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
Thu Oct 27 11:00:19 2022 -> bytecode.cld database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2)
Thu Oct 27 11:00:19 2022 -> --------------------------------------

So the issue is with the incremenatal update daily.cld only, once it
falls back to daily.cvd it's working as it should

--
Ralf Hildebrandt
Charité - Universitätsmedizin Berlin
Geschäftsbereich IT | Abteilung Netzwerk

Campus Benjamin Franklin (CBF)
Haus I | 1. OG | Raum 105
Hindenburgdamm 30 | D-12203 Berlin

Tel. +49 30 450 570 155
ralf.hildebrandt@charite.de
https://www.charite.de
_______________________________________________

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat
Re: [ext] ClamAV 1.0.0 release candidate now available [ In reply to ]
From: Ralf Hildebrandt via clamav-users <clamav-users@lists.clamav.net>
Subject: Re: [clamav-users] [ext] ClamAV 1.0.0 release candidate now available
Date: Fri, 28 Oct 2022 09:10:46 +0200

> * Micah Snyder (micasnyd) via clamav-users <clamav-users@lists.clamav.net>:
>
>> We are excited to announce the ClamAV 1.0.0 release candidate!
>
> I'm seeing log entries like this for the machines with 1.0.0-rc
> indicating the daily.cld update failed:

I experienced same problem while I'm working to update FreeBSD ClamAV
port to 1.0.0-rc. It happens if ClamAV is built with external
TomsFastMath library (that is, ENABLE_EXTERNAL_TOMSFASTMATH option is
ON).

See issue #736 for more detail.

https://github.com/Cisco-Talos/clamav/issues/736

HTH.

---
Yasuhiro Kimura
_______________________________________________

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat
Re: [ext] ClamAV 1.0.0 release candidate now available [ In reply to ]
On Fri, 28 Oct 2022, Yasuhiro Kimura wrote:

> From: Ralf Hildebrandt via clamav-users <clamav-users@lists.clamav.net>
> Subject: Re: [clamav-users] [ext] ClamAV 1.0.0 release candidate now available
> Date: Fri, 28 Oct 2022 09:10:46 +0200
>
>> * Micah Snyder (micasnyd) via clamav-users <clamav-users@lists.clamav.net>:
>>
>>> We are excited to announce the ClamAV 1.0.0 release candidate!
>>
>> I'm seeing log entries like this for the machines with 1.0.0-rc
>> indicating the daily.cld update failed:
>
> I experienced same problem while I'm working to update FreeBSD ClamAV
> port to 1.0.0-rc. It happens if ClamAV is built with external
> TomsFastMath library (that is, ENABLE_EXTERNAL_TOMSFASTMATH option is
> ON).
>
> See issue #736 for more detail.
>
> https://github.com/Cisco-Talos/clamav/issues/736

I am building from the tarball at
https://www.clamav.net/downloads/production/clamav-1.0.0-rc.tar.gz
on Ubuntu kinetic 22.10 (released this month) and have the same problem,
but have not (yet?) managed to resolve it with the internal TomsFastMath
library.

I started by copying the database directory from one built by freshclam v103.7
which has had no problems with recent updates, including yesterday and today.
The freshclam.conf files have different UpdateLogFile and DatabaseDirectory but
are otherwise identical, including
DatabaseMirror db.local.clamav.net
DatabaseMirror database.clamav.net

Yesterday freshclam suggested that I checked again later, but today
it is downloading the .cvd after failing to patch the .cld

Should I, or you, be concerned that after just two days, freshclam
switches to a large download ?

Joel asked:
> You wouldn?t download the cld from the server.
> Or am I reading this thread wrong.

No, but the database directory has an existing .cld to update.

In mkdir_and_chdir_for_cdiff_tmp() libfreshclam_internal.c I see:

/*
* 3) Unpack the existing CVD/CLD database to this directory.
*/
if (CL_SUCCESS != cl_cvdunpack(cvdfile, tmpdir, false)) {
logg(LOGG_ERROR, "mkdir_and_chdir_for_cdiff_tmp: Can't unpack %s into %s\n", cvdfile, tmpdir);
cli_rmdirs(tmpdir);
goto done;
}
but chasing cl_cvdunpack, the verify routines only mention cvd.
Do they verify .cld files too, or is that the real problem ?

------------------------------------

(Whilst the .cvd downloads, the line
Time: 21.7s, ETA: 0.0s [========================>] 57.34MiB/57.34MiB
flickers a lot; does it need to redraw every 0.1 seconds ?)

------------------------------------

# host db.local.clamav.net ; host database.clamav.net
db.local.clamav.net is an alias for db.local.clamav.net.cdn.cloudflare.net.
db.local.clamav.net.cdn.cloudflare.net has address 104.16.219.84
db.local.clamav.net.cdn.cloudflare.net has address 104.16.218.84
db.local.clamav.net.cdn.cloudflare.net has IPv6 address 2606:4700::6810:db54
db.local.clamav.net.cdn.cloudflare.net has IPv6 address 2606:4700::6810:da54
database.clamav.net is an alias for database.clamav.net.cdn.cloudflare.net.
database.clamav.net.cdn.cloudflare.net has address 104.16.219.84
database.clamav.net.cdn.cloudflare.net has address 104.16.218.84
database.clamav.net.cdn.cloudflare.net has IPv6 address 2606:4700::6810:da54
database.clamav.net.cdn.cloudflare.net has IPv6 address 2606:4700::6810:db54


Sat Oct 29 12:30:06 2022 -> --------------------------------------
Sat Oct 29 12:30:06 2022 -> ClamAV update process started at Sat Oct 29 12:30:06 2022
Sat Oct 29 12:30:06 2022 -> daily database available for update (local version: 26702, remote version: 26703)
Sat Oct 29 12:30:06 2022 -> WARNING: [LibClamAV] CVD verification failed for: daily.cld
Sat Oct 29 12:30:06 2022 -> ERROR: mkdir_and_chdir_for_cdiff_tmp: Can't unpack daily.cld into /var/lib/clamav-1.0.0-rc/tmp.b1a2031575/clamav-a369f6069be4efb91a43123096659109.tmp
Sat Oct 29 12:30:06 2022 -> The database server doesn't have the latest patch for the daily database (version 26703). The server will likely have updated if you check again in a few hours.
Sat Oct 29 12:30:06 2022 -> main.cld database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
Sat Oct 29 12:30:06 2022 -> bytecode.cld database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2)
Sun Oct 30 09:22:40 2022 -> --------------------------------------
Sun Oct 30 09:22:40 2022 -> ClamAV update process started at Sun Oct 30 09:22:40 2022
Sun Oct 30 09:22:40 2022 -> daily database available for update (local version: 26702, remote version: 26704)
Sun Oct 30 09:22:41 2022 -> WARNING: [LibClamAV] CVD verification failed for: daily.cld
Sun Oct 30 09:22:41 2022 -> ERROR: mkdir_and_chdir_for_cdiff_tmp: Can't unpack daily.cld into /var/lib/clamav-1.0.0-rc/tmp.1e7a2b62db/clamav-13ead5841234f30e4eb51b6c88c30635.tmp
Sun Oct 30 09:22:41 2022 -> WARNING: Incremental update failed, trying to download daily.cvd
Sun Oct 30 09:23:04 2022 -> Testing database: '/var/lib/clamav-1.0.0-rc/tmp.1e7a2b62db/clamav-1c8714a1574f2f81c3846f238d297cb8.tmp-daily.cvd' ...
Sun Oct 30 09:23:10 2022 -> Database test passed.
Sun Oct 30 09:23:10 2022 -> daily.cvd updated (version: 26704, sigs: 2009539, f-level: 90, builder: raynman)
Sun Oct 30 09:23:10 2022 -> main.cld database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
Sun Oct 30 09:23:10 2022 -> bytecode.cld database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2)
Sun Oct 30 09:23:10 2022 -> Clamd successfully notified about the update.
Sun Oct 30 09:28:04 2022 -> --------------------------------------

---------------------------

Thanks,

--
Andrew C. Aitchison Kendal, UK
andrew@aitchison.me.uk
_______________________________________________

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat
Re: [ext] ClamAV 1.0.0 release candidate now available [ In reply to ]
Hi Ralf,

I spent some time playing with this yesterday but far I have been unable to reproduce this issue. I will continue to investigate. If you find any other clues as to what the trigger may be, please let me know.

Regards,
Micah


Micah Snyder
ClamAV Development
Talos
Cisco Systems, Inc.
________________________________
From: Ralf Hildebrandt <Ralf.Hildebrandt@charite.de>
Sent: Friday, October 28, 2022 12:15 AM
To: Micah Snyder (micasnyd) via clamav-users <clamav-users@lists.clamav.net>; Micah Snyder (micasnyd) <micasnyd@cisco.com>
Subject: Re: [clamav-users] [ext] ClamAV 1.0.0 release candidate now available

> Fri Oct 28 09:07:10 2022 -> --------------------------------------
> Fri Oct 28 09:07:10 2022 -> freshclam daemon 1.0.0-rc (OS: Linux, ARCH: x86_64, CPU: x86_64)
> Fri Oct 28 09:07:10 2022 -> ClamAV update process started at Fri Oct 28 09:07:10 2022
> Fri Oct 28 09:07:10 2022 -> daily database available for update (local version: 26700, remote version: 26701)
> Fri Oct 28 09:07:10 2022 -> WARNING: [LibClamAV] CVD verification failed for: daily.cld
> Fri Oct 28 09:07:10 2022 -> ERROR: mkdir_and_chdir_for_cdiff_tmp: Can't unpack daily.cld into /var/lib/clamav/tmp.3bbb7ed4d7/clamav-bfba84844f1170e4c4210f03d1759097.tmp
> Fri Oct 28 09:07:10 2022 -> The database server doesn't have the latest patch for the daily database (version 26701). The server will likely have updated if you check again in a few hours.
> Fri Oct 28 09:07:10 2022 -> main.cvd database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
> Fri Oct 28 09:07:10 2022 -> bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2)
> Fri Oct 28 09:07:10 2022 -> --------------------------------------

Another data point - I checked another machine which successfully
updated to 26701 (yesterday already!):

Thu Oct 27 10:00:06 2022 -> --------------------------------------
Thu Oct 27 11:00:06 2022 -> Received signal: wake up
Thu Oct 27 11:00:06 2022 -> ClamAV update process started at Thu Oct 27 11:00:06 2022
Thu Oct 27 11:00:06 2022 -> daily database available for update (local version: 26699, remote version: 26701)
Thu Oct 27 11:00:06 2022 -> WARNING: [LibClamAV] CVD verification failed for: daily.cld
Thu Oct 27 11:00:06 2022 -> ERROR: mkdir_and_chdir_for_cdiff_tmp: Can't unpack daily.cld into /var/lib/clamav/tmp.bfd8f6c0fe/clamav-91f69d4433a1975076fd9905e1f5ca06.tmp
Thu Oct 27 11:00:06 2022 -> WARNING: Incremental update failed, trying to download daily.cvd
Thu Oct 27 11:00:09 2022 -> Testing database: '/var/lib/clamav/tmp.bfd8f6c0fe/clamav-4ad0a44cd8a0ebe2bf630a0b92819105.tmp-daily.cvd'...
Thu Oct 27 11:00:19 2022 -> Database test passed.
Thu Oct 27 11:00:19 2022 -> daily.cvd updated (version: 26701, sigs: 2009238, f-level: 90, builder: raynman)
Thu Oct 27 11:00:19 2022 -> main.cld database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
Thu Oct 27 11:00:19 2022 -> bytecode.cld database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2)
Thu Oct 27 11:00:19 2022 -> --------------------------------------

So the issue is with the incremenatal update daily.cld only, once it
falls back to daily.cvd it's working as it should

--
Ralf Hildebrandt
Charit? - Universit?tsmedizin Berlin
Gesch?ftsbereich IT | Abteilung Netzwerk

Campus Benjamin Franklin (CBF)
Haus I | 1. OG | Raum 105
Hindenburgdamm 30 | D-12203 Berlin

Tel. +49 30 450 570 155
ralf.hildebrandt@charite.de
https://www.charite.de
Re: [ext] ClamAV 1.0.0 release candidate now available [ In reply to ]
Oh I see! It is on the second incremental update that the failure occurs -- when the CLD is unpacked to be updated. That should be a very easy fix.

If you can help test it, I will share something as soon as it is ready.

Regards,
Micah


Micah Snyder
ClamAV Development
Talos
Cisco Systems, Inc.
________________________________
From: clamav-users <clamav-users-bounces@lists.clamav.net> on behalf of Andrew C Aitchison via clamav-users <clamav-users@lists.clamav.net>
Sent: Sunday, October 30, 2022 3:34 AM
To: ClamAV users ML <clamav-users@lists.clamav.net>
Cc: Andrew C Aitchison <clamav@aitchison.me.uk>
Subject: Re: [clamav-users] [ext] ClamAV 1.0.0 release candidate now available

On Fri, 28 Oct 2022, Yasuhiro Kimura wrote:

> From: Ralf Hildebrandt via clamav-users <clamav-users@lists.clamav.net>
> Subject: Re: [clamav-users] [ext] ClamAV 1.0.0 release candidate now available
> Date: Fri, 28 Oct 2022 09:10:46 +0200
>
>> * Micah Snyder (micasnyd) via clamav-users <clamav-users@lists.clamav.net>:
>>
>>> We are excited to announce the ClamAV 1.0.0 release candidate!
>>
>> I'm seeing log entries like this for the machines with 1.0.0-rc
>> indicating the daily.cld update failed:
>
> I experienced same problem while I'm working to update FreeBSD ClamAV
> port to 1.0.0-rc. It happens if ClamAV is built with external
> TomsFastMath library (that is, ENABLE_EXTERNAL_TOMSFASTMATH option is
> ON).
>
> See issue #736 for more detail.
>
> https://github.com/Cisco-Talos/clamav/issues/736

I am building from the tarball at
https://www.clamav.net/downloads/production/clamav-1.0.0-rc.tar.gz
on Ubuntu kinetic 22.10 (released this month) and have the same problem,
but have not (yet?) managed to resolve it with the internal TomsFastMath
library.

I started by copying the database directory from one built by freshclam v103.7
which has had no problems with recent updates, including yesterday and today.
The freshclam.conf files have different UpdateLogFile and DatabaseDirectory but
are otherwise identical, including
DatabaseMirror db.local.clamav.net
DatabaseMirror database.clamav.net

Yesterday freshclam suggested that I checked again later, but today
it is downloading the .cvd after failing to patch the .cld

Should I, or you, be concerned that after just two days, freshclam
switches to a large download ?

Joel asked:
> You wouldn?t download the cld from the server.
> Or am I reading this thread wrong.

No, but the database directory has an existing .cld to update.

In mkdir_and_chdir_for_cdiff_tmp() libfreshclam_internal.c I see:

/*
* 3) Unpack the existing CVD/CLD database to this directory.
*/
if (CL_SUCCESS != cl_cvdunpack(cvdfile, tmpdir, false)) {
logg(LOGG_ERROR, "mkdir_and_chdir_for_cdiff_tmp: Can't unpack %s into %s\n", cvdfile, tmpdir);
cli_rmdirs(tmpdir);
goto done;
}
but chasing cl_cvdunpack, the verify routines only mention cvd.
Do they verify .cld files too, or is that the real problem ?

------------------------------------

(Whilst the .cvd downloads, the line
Time: 21.7s, ETA: 0.0s [========================>] 57.34MiB/57.34MiB
flickers a lot; does it need to redraw every 0.1 seconds ?)

------------------------------------

# host db.local.clamav.net ; host database.clamav.net
db.local.clamav.net is an alias for db.local.clamav.net.cdn.cloudflare.net.
db.local.clamav.net.cdn.cloudflare.net has address 104.16.219.84
db.local.clamav.net.cdn.cloudflare.net has address 104.16.218.84
db.local.clamav.net.cdn.cloudflare.net has IPv6 address 2606:4700::6810:db54
db.local.clamav.net.cdn.cloudflare.net has IPv6 address 2606:4700::6810:da54
database.clamav.net is an alias for database.clamav.net.cdn.cloudflare.net.
database.clamav.net.cdn.cloudflare.net has address 104.16.219.84
database.clamav.net.cdn.cloudflare.net has address 104.16.218.84
database.clamav.net.cdn.cloudflare.net has IPv6 address 2606:4700::6810:da54
database.clamav.net.cdn.cloudflare.net has IPv6 address 2606:4700::6810:db54


Sat Oct 29 12:30:06 2022 -> --------------------------------------
Sat Oct 29 12:30:06 2022 -> ClamAV update process started at Sat Oct 29 12:30:06 2022
Sat Oct 29 12:30:06 2022 -> daily database available for update (local version: 26702, remote version: 26703)
Sat Oct 29 12:30:06 2022 -> WARNING: [LibClamAV] CVD verification failed for: daily.cld
Sat Oct 29 12:30:06 2022 -> ERROR: mkdir_and_chdir_for_cdiff_tmp: Can't unpack daily.cld into /var/lib/clamav-1.0.0-rc/tmp.b1a2031575/clamav-a369f6069be4efb91a43123096659109.tmp
Sat Oct 29 12:30:06 2022 -> The database server doesn't have the latest patch for the daily database (version 26703). The server will likely have updated if you check again in a few hours.
Sat Oct 29 12:30:06 2022 -> main.cld database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
Sat Oct 29 12:30:06 2022 -> bytecode.cld database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2)
Sun Oct 30 09:22:40 2022 -> --------------------------------------
Sun Oct 30 09:22:40 2022 -> ClamAV update process started at Sun Oct 30 09:22:40 2022
Sun Oct 30 09:22:40 2022 -> daily database available for update (local version: 26702, remote version: 26704)
Sun Oct 30 09:22:41 2022 -> WARNING: [LibClamAV] CVD verification failed for: daily.cld
Sun Oct 30 09:22:41 2022 -> ERROR: mkdir_and_chdir_for_cdiff_tmp: Can't unpack daily.cld into /var/lib/clamav-1.0.0-rc/tmp.1e7a2b62db/clamav-13ead5841234f30e4eb51b6c88c30635.tmp
Sun Oct 30 09:22:41 2022 -> WARNING: Incremental update failed, trying to download daily.cvd
Sun Oct 30 09:23:04 2022 -> Testing database: '/var/lib/clamav-1.0.0-rc/tmp.1e7a2b62db/clamav-1c8714a1574f2f81c3846f238d297cb8.tmp-daily.cvd' ...
Sun Oct 30 09:23:10 2022 -> Database test passed.
Sun Oct 30 09:23:10 2022 -> daily.cvd updated (version: 26704, sigs: 2009539, f-level: 90, builder: raynman)
Sun Oct 30 09:23:10 2022 -> main.cld database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
Sun Oct 30 09:23:10 2022 -> bytecode.cld database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2)
Sun Oct 30 09:23:10 2022 -> Clamd successfully notified about the update.
Sun Oct 30 09:28:04 2022 -> --------------------------------------

---------------------------

Thanks,

--
Andrew C. Aitchison Kendal, UK
andrew@aitchison.me.uk
_______________________________________________

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat
Re: [ext] ClamAV 1.0.0 release candidate now available [ In reply to ]
Hi Andrew,

> Should cli_cvdverify() even be used to verify .cld files ?

Indeed, it should not.

Here is my PR to fix the issue. Are you able to try it out to help verify it resolves the issue on your end?
https://github.com/Cisco-Talos/clamav/pull/740
[https://opengraph.githubassets.com/fe53b48c8ddd353921519a3075391788df3c30af039e250ba6728bbf35776e86/Cisco-Talos/clamav/pull/740]<https://github.com/Cisco-Talos/clamav/pull/740>
Clam 2167 freshclam cld incremental update by micahsnyder ? Pull Request #740 ? Cisco-Talos/clamav<https://github.com/Cisco-Talos/clamav/pull/740>
Freshclam: fix incremental update on CLD database When adding the cl_cvdunpack() API that (optionally) verifies the database signature, we used it in libfreshclam in a place where it may also unpac...
github.com

Regards,
Micah


Micah Snyder
ClamAV Development
Talos
Cisco Systems, Inc.

________________________________
From: Andrew C Aitchison <andrew@aitchison.me.uk>
Sent: Wednesday, November 2, 2022 8:40 AM
To: Micah Snyder (micasnyd) <micasnyd@cisco.com>
Cc: ClamAV users ML <clamav-users@lists.clamav.net>; Andrew C Aitchison <clamav@aitchison.me.uk>
Subject: Re: [clamav-users] [ext] ClamAV 1.0.0 release candidate now available

On Tue, 1 Nov 2022, Micah Snyder (micasnyd) wrote:

> Oh I see! It is on the second incremental update that the failure occurs -- when the CLD is unpacked to be updated. That should be a very easy fix.
>
> If you can help test it, I will share something as soon as it is ready.

I think I have found the problem.

These .cld files have headers like

ClamAV-VDB:01 Nov 2022 03-52 -0400:26706:2009713:90:X:X:raynman:1667289154

with X in place of both the MD5 and the Digital signature
so cli_cvdverify() has nothing to match and thus fails.

Do *downloaded* .cld files (as opposed to updated and repacked files)
have MD5 and the Digital signature ?

Should cli_cvdverify() even be used to verify .cld files ?

--
Andrew C. Aitchison Kendal, UK
andrew@aitchison.me.uk
Re: [ext] ClamAV 1.0.0 release candidate now available [ In reply to ]
On Wed, 2 Nov 2022, Micah Snyder (micasnyd) wrote:

> Hi Andrew,
>
>> Should cli_cvdverify() even be used to verify .cld files ?
>
> Indeed, it should not.
>
> Here is my PR to fix the issue. Are you able to try it out to help verify it resolves the issue on your end?
> https://github.com/Cisco-Talos/clamav/pull/740
> [https://opengraph.githubassets.com/fe53b48c8ddd353921519a3075391788df3c30af039e250ba6728bbf35776e86/Cisco-Talos/clamav/pull/740]<https://github.com/Cisco-Talos/clamav/pull/740>
> Clam 2167 freshclam cld incremental update by micahsnyder ? Pull Request #740 ? Cisco-Talos/clamav<https://github.com/Cisco-Talos/clamav/pull/740>
> Freshclam: fix incremental update on CLD database When adding the cl_cvdunpack() API that (optionally) verifies the database signature, we used it in libfreshclam in a place where it may also unpac...
> github.com

That patch looks good and my tests are looking good, but I managed to
fall foul of the rate limit so cannot confirm for 24 hours :-(


> ________________________________
> From: Andrew C Aitchison <andrew@aitchison.me.uk>
> Sent: Wednesday, November 2, 2022 8:40 AM
> To: Micah Snyder (micasnyd) <micasnyd@cisco.com>
> Cc: ClamAV users ML <clamav-users@lists.clamav.net>; Andrew C Aitchison <clamav@aitchison.me.uk>
> Subject: Re: [clamav-users] [ext] ClamAV 1.0.0 release candidate now available
>
> On Tue, 1 Nov 2022, Micah Snyder (micasnyd) wrote:
>
>> Oh I see! It is on the second incremental update that the failure occurs -- when the CLD is unpacked to be updated. That should be a very easy fix.
>>
>> If you can help test it, I will share something as soon as it is ready.
>
> I think I have found the problem.
>
> These .cld files have headers like
>
> ClamAV-VDB:01 Nov 2022 03-52 -0400:26706:2009713:90:X:X:raynman:1667289154
>
> with X in place of both the MD5 and the Digital signature
> so cli_cvdverify() has nothing to match and thus fails.
>
> Do *downloaded* .cld files (as opposed to updated and repacked files)
> have MD5 and the Digital signature ?
>
> Should cli_cvdverify() even be used to verify .cld files ?
>
> --
> Andrew C. Aitchison Kendal, UK
> andrew@aitchison.me.uk
>

--
Andrew C. Aitchison Kendal, UK
andrew@aitchison.me.uk
_______________________________________________

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat