Mailing List Archive

On 10/10/2020 01:10, Robert Kudyba wrote:
> Running ClamAV 103.0-1 on Fedora, I have freshclam
> and clamav-unofficial-sigs.sh from
> https://github.com/extremeshok/clamav-unofficial-sigs
> <https://github.com/extremeshok/clamav-unofficial-sigs>
>
> Every few weeks I'll start seeing this error:
>
> ERROR: clam database directory (clam_dbs) not writable /var/lib/clamav
>
> Running this fixes it:
> su clamav -s '/usr/local/sbin/clamav-unofficial-sigs.sh'
>
> Here are the files not owned by clamav:
> -rw-r--r--  1 clamupdate clamupdate    296388 Sep 19  2019 bytecode.cvd
> -rw-r--r--  1 clamupdate clamupdate 112832258 Sep 17 09:53 daily.cvd
> -rw-r--r--  1 clamupdate clamupdate 117859675 Nov 25  2019 main.cvd
>
At first glance it appears someone is running "freshclam" manually as
clamupdate/clamupdate.

Is there only one "freshclam" binary on the system?

Is it running as a daemon or being invoked by some other method(s)?

Is there another that is set{g,u}id clamupdate?

Oh, what binaries *are* set{g,u}id clamupdate?

And who/what regularly uses the "clamupdate" id?

Cheers,
Gary B-)


_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

>
> > Every few weeks I'll start seeing this error:
> >
> > ERROR: clam database directory (clam_dbs) not writable /var/lib/clamav
> >
> > Running this fixes it:
> > su clamav -s '/usr/local/sbin/clamav-unofficial-sigs.sh'
> >
> > Here are the files not owned by clamav:
> > -rw-r--r-- 1 clamupdate clamupdate 296388 Sep 19 2019 bytecode.cvd
> > -rw-r--r-- 1 clamupdate clamupdate 112832258 Sep 17 09:53 daily.cvd
> > -rw-r--r-- 1 clamupdate clamupdate 117859675 Nov 25 2019 main.cvd
> >
> At first glance it appears someone is running "freshclam" manually as
> clamupdate/clamupdate.
>
> Is there only one "freshclam" binary on the system?
>

Yes:
ls -l /usr/bin/freshclam*
-rwxr-xr-x 1 root root 45816 Oct 5 14:05 /usr/bin/freshclam

Is it running as a daemon or being invoked by some other method(s)?
>
Via systemctl:
clamav 937912 0.0 0.0 102816 15860 ? Ss 04:46 0:04
/usr/bin/freshclam -d --foreground=true

systemctl status clamav-freshclam.service
? clamav-freshclam.service - ClamAV virus database updater
Loaded: loaded (/usr/lib/systemd/system/clamav-freshclam.service;
enabled; vendor preset: disabled)
Active: active (running) since Fri 2020-10-09 04:46:04 EDT; 6h ago
Docs: man:freshclam(1)
man:freshclam.conf(5)
https://www.clamav.net/documents
Main PID: 937912 (freshclam)
Tasks: 1 (limit: 154197)
Memory: 337.2M
CGroup: /system.slice/clamav-freshclam.service
??937912 /usr/bin/freshclam -d --foreground=true

And the other one is disabled:
systemctl status clam-freshclam.service
? clam-freshclam.service - freshclam scanner
Loaded: loaded (/usr/lib/systemd/system/clam-freshclam.service;
disabled; vendor preset: disabled)
Active: inactive (dead)


> Is there another that is set{g,u}id clamupdate?
>
> Oh, what binaries *are* set{g,u}id clamupdate?
>
> And who/what regularly uses the "clamupdate" id?
>

Note that I know of. The only reference to clamupdate I see are in the
various config files, e.g., clamav.conf and the 3rd party conf files in
/etc/clamav-unofficial-sigs/

I can track down that this started early this morning:
Oct 09 05:14:02 ERROR: clam database directory (clam_dbs) not writable
/var/lib/clamav

But the only thing in the cron log file at that time is this 3rd
party update:

Oct 9 05:01:01 ourserver CROND[948241]: (root) CMD (run-parts
/etc/cron.hourly)
Oct 9 05:01:01 ourserver run-parts[948241]: (/etc/cron.hourly) starting
0anacron
Oct 9 05:01:01 ourserver run-parts[948241]: (/etc/cron.hourly) finished
0anacron
Oct 9 05:14:01 ourserver CROND[956493]: (clamav) CMD ([ -x
/usr/local/sbin/clamav-unofficial-sigs.sh ] && /usr/bin/bash
/usr/local/sbin/clamav-unofficial-sigs.sh)

I also see this:
cat /etc/cron.d/clamav-unofficial-sigs
14 * * * * clamav [ -x /usr/local/sbin/clamav-unofficial-sigs.sh ] &&
/usr/bin/bash /usr/local/sbin/clamav-unofficial-sigs.sh

and I added a while back clamav to the clamupdate group to try to work
around this:

grep clamupdate /etc/passwd
clamupdate:x:983:979:Clamav database update
user:/var/lib/clamav:/sbin/nologin

grep 979 /etc/group
clamupdate:x:979:clamav

Hi there,

On Fri, 9 Oct 2020, Robert Kudyba wrote:

> Running ClamAV 103.0-1 on Fedora, I have freshclam
> and clamav-unofficial-sigs.sh from
> https://github.com/extremeshok/clamav-unofficial-sigs
> ...
> Every few weeks I'll start seeing this error:
>
> ERROR: clam database directory (clam_dbs) not writable /var/lib/clamav
> ...
> -rw-r--r-- 1 clamupdate clamupdate 296388 Sep 19 2019 bytecode.cvd
> -rw-r--r-- 1 clamupdate clamupdate 112832258 Sep 17 09:53 daily.cvd
> -rw-r--r-- 1 clamupdate clamupdate 117859675 Nov 25 2019 main.cvd
> ...
> I've tried grepping for the clamupdate user in all the .conf files and
> anywhere it appears it's commented out. Any other places to look?

It's a little bit concerning because if something is changing ownership
of the files then (a) it looks like it's running with root permissions
and (b) you don't know what it is.

Are you sure that you don't have something else running which sets the
permissions? Are there logs going back far enough to give you a good
feel for exactly when it happens? If it were my problem I'd probably
start with some simple logging so it was more clear what happened when;
something like a cron job which just makes a listing of the permissions
every minute, appending it to a file in /var/log. Something like this
in a crontab:

* * * * * /bin/echo -n "$(/bin/date) " >> /var/log/clam_perms.log ; \
/bin/ls -l /var/lib/clamav >> /var/log/clam_perms.log

If you just want to paper over the cracks you could for example make a
wrapper for the update script which sets permissions before running it,
or run another script before invocations of the update script so that
the permissions are set first, or hack the update script itself. You
could even use 'chattr' to make the permissions unchangeable.

Later on Fri, 9 Oct 2020, Robert Kudyba wrote:

> The only reference to clamupdate I see are in the various config
> files, e.g., clamav.conf ...

I'm puzzled. Why is there a reference to the 'clamupdate' user in a
file called 'clamav.conf' (which I take to be a bowdlerized version of
something like clamd.conf) if you don't use the 'clamupdate' user ID?
It makes me wonder if there have been changes from some original setup
which did employ that user and which haven't all been flushed through,
or if something else has modified the ClamAV configuration files that
you don't know about.

Years ago I had trouble with the forerunner to the extremeshock script
which resulted in execute bits from scripts getting lost, but that's a
bit different from what you're seeing and it was over a decade ago. I
spent some time with Bill Landry who wrote the original and eventually
we got it fixed. I only mention it because this is eerily similar.

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

>
> Every few weeks I'll start seeing this error:
> >
> > ERROR: clam database directory (clam_dbs) not writable /var/lib/clamav
> > ...
> > -rw-r--r-- 1 clamupdate clamupdate 296388 Sep 19 2019 bytecode.cvd
> > -rw-r--r-- 1 clamupdate clamupdate 112832258 Sep 17 09:53 daily.cvd
> > -rw-r--r-- 1 clamupdate clamupdate 117859675 Nov 25 2019 main.cvd
> > ...
> > I've tried grepping for the clamupdate user in all the .conf files and
> > anywhere it appears it's commented out. Any other places to look?
>
> It's a little bit concerning because if something is changing ownership
> of the files then (a) it looks like it's running with root permissions
> and (b) you don't know what it is.
>
> Are you sure that you don't have something else running which sets the
> permissions?


That's what I'm trying to figure out. I've looked through the crontab
files, e.g., in /etc/conf*, bubcus



> Are there logs going back far enough to give you a good
> feel for exactly when it happens?


I believe so and I have access to the backups which go back at least a
year. That's why I pointed to this log:
/var/log/clamav-unofficial-sigs/clamav-unofficial-sigs.log

And today when it started:
Oct 09 04:15:56 Checking for urlhaus updates...
Oct 09 04:15:56 Checking for updated urlhaus database file: urlhaus.ndb
Oct 09 04:15:56 Testing updated urlhaus database file: urlhaus.ndb
Oct 09 04:15:56 Clamscan reports urlhaus urlhaus.ndb database integrity
tested good
Oct 09 04:15:56 Successfully updated urlhaus production database file:
urlhaus.ndb
Oct 09 04:15:56 Update(s) detected, reloading ClamAV databases
Oct 09 04:15:56 ClamAV databases reloading
Oct 09 04:15:56 Issue tracker :
https://github.com/extremeshok/clamav-unofficial-sigs/issues
Oct 09 04:15:56 Powered By https://eXtremeSHOK.com
*Oct 09 05:14:02 ERROR: clam database directory (clam_dbs) not writable
/var/lib/clamav*

So between 4:15 and 5:15 AM today (EDT).

If it were my problem I'd probably
> start with some simple logging so it was more clear what happened when;
> something like a cron job which just makes a listing of the permissions
> every minute, appending it to a file in /var/log. Something like this
> in a crontab:
>
> * * * * * /bin/echo -n "$(/bin/date) " >> /var/log/clam_perms.log ; \
> /bin/ls -l /var/lib/clamav >> /var/log/clam_perms.log
>

I'll consider this too, perhaps I should contact the ExtremeSHOK
contributors at https://github.com/extremeshok/clamav-unofficial-sigs? Or
perhaps there's some debug option that I'm not aware of? In
/etc/clamav-unofficial-sigs/master.conf
I have:
logging_enabled="yes"
log_file_path="/var/log/clamav-unofficial-sigs"
log_file_name="clamav-unofficial-sigs.log"


> If you just want to paper over the cracks you could for example make a
> wrapper for the update script which sets permissions before running it,
> or run another script before invocations of the update script so that
> the permissions are set first, or hack the update script itself. You
> could even use 'chattr' to make the permissions unchangeable.
>

Yeah I've used the chattr option in other areas, perhaps some logging would
appear if I take this approach.

Later on Fri, 9 Oct 2020, Robert Kudyba wrote:
>
> > The only reference to clamupdate I see are in the various config
> > files, e.g., clamav.conf ...
>
> I'm puzzled. Why is there a reference to the 'clamupdate' user in a
> file called 'clamav.conf' (which I take to be a bowdlerized version of
> something like clamd.conf) if you don't use the 'clamupdate' user ID?
>

Sure looks like earlier versions of Fedora did this according to this bug
report <https://bugzilla.redhat.com/show_bug.cgi?id=963920> and this
discussion
<https://lists.fedoraproject.org/pipermail/test/2013-May/115552.html> on
Fedora Project.

Ha bowdlerized
<https://www.google.com/search?q=bowdlerized&rlz=1C1CHBF_enUS796US796&oq=bow&aqs=chrome.0.69i59j69i57j0l5j69i60.952j1j4&sourceid=chrome&ie=UTF-8>:
(of a text or account) having had material considered improper or offensive
removed.


> It makes me wonder if there have been changes from some original setup
> which did employ that user and which haven't all been flushed through,
> or if something else has modified the ClamAV configuration files that
> you don't know about.
>

I believe I configured and installed it myself less than 2 years ago but
perhaps when I restored some files from a backup I added some old config
files and or/services? I do see:
systemctl status clam
clamav-clamonacc.service clamav-unofficial-sigs.service
clamd.service
clamav-freshclam.service clamav-unofficial-sigs.timer
clam-freshclam.service
clamav-milter.service clamd@scan.service
clamonacc.service

Only clamav-milter, clamd@scan.service and clamav-freshclam.service are
active.

Years ago I had trouble with the forerunner to the extremeshock script
> which resulted in execute bits from scripts getting lost, but that's a
> bit different from what you're seeing and it was over a decade ago. I
> spent some time with Bill Landry who wrote the original and eventually
> we got it fixed. I only mention it because this is eerily similar.


Yes very well could be as the extra config files (that start with os.*)
from ExtremeSHOK don't mention Fedora, just CentOS and perhaps there is
something unique to Fedora. I believe when I was testing I may have tried
to use the clamupdate user as I see these are commented out:
grep clamupdate /etc/clamav-unofficial-sigs/*
/etc/clamav-unofficial-sigs/os.conf:#clam_user="clamupdate"
/etc/clamav-unofficial-sigs/os.conf:#clam_group="clamupdate"

I see Fangfrisch <https://rseichter.github.io/fangfrisch/>is being
maintained as an alternative. Haven't tried it yet.

Hello again,

On Fri, 9 Oct 2020, Robert Kudyba wrote:

> ... today when it started:
> Oct 09 04:15:56 Checking for urlhaus updates...
> Oct 09 04:15:56 Checking for updated urlhaus database file: urlhaus.ndb
> Oct 09 04:15:56 Testing updated urlhaus database file: urlhaus.ndb
> Oct 09 04:15:56 Clamscan reports urlhaus urlhaus.ndb database integrity tested good
> Oct 09 04:15:56 Successfully updated urlhaus production database file: urlhaus.ndb
> Oct 09 04:15:56 Update(s) detected, reloading ClamAV databases
> Oct 09 04:15:56 ClamAV databases reloading
> Oct 09 04:15:56 Issue tracker : https://github.com/extremeshok/clamav-unofficial-sigs/issues
> Oct 09 04:15:56 Powered By https://eXtremeSHOK.com
>*Oct 09 05:14:02 ERROR: clam database directory (clam_dbs) not writable /var/lib/clamav*

Looks clear that the urlhaus db was updated OK. Does the unofficial
update script normally take an hour to run on your system?! The one
we use usually takes just a few minutes.

> ... perhaps I should contact the ExtremeSHOK contributors ...

I'd have said so, yes.

> perhaps there's some debug option that I'm not aware of?

It's just a shell script, you could edit it to put debugging things in
there if you're comfortable with hacking shell scripts. Does it give
usage help if run with no arguments? Does it have the '-i' option?

> ... I do see:
> systemctl status clam
> clamav-clamonacc.service clamav-unofficial-sigs.service
> clamd.service
> clamav-freshclam.service clamav-unofficial-sigs.timer
> clam-freshclam.service
> clamav-milter.service clamd@scan.service
> clamonacc.service

I don't use any of that stuff, I like to know what's going on. It
might be worth disabling all the service frippery and starting the
daemons from the command line to see if it behaves any differently.

> I see Fangfrisch <https://rseichter.github.io/fangfrisch/>is being
> maintained as an alternative. Haven't tried it yet.

It might not be time to throw out the baby just yet, before swapping
one lot of unknowns for another lot of unknowns I'd definitely try a
bit of investigative work. After all other people use this stuff. If
extra logging, disabling services etc don't lead you anywhere it might
be worth purging and reinstalling all the implicated packages.

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

>
> > Oct 09 04:15:56 Checking for urlhaus updates...
> > Oct 09 04:15:56 Checking for updated urlhaus database file: urlhaus.ndb
> > Oct 09 04:15:56 Testing updated urlhaus database file: urlhaus.ndb
> > Oct 09 04:15:56 Clamscan reports urlhaus urlhaus.ndb database integrity
> tested good
> > Oct 09 04:15:56 Successfully updated urlhaus production database file:
> urlhaus.ndb
> > Oct 09 04:15:56 Update(s) detected, reloading ClamAV databases
> > Oct 09 04:15:56 ClamAV databases reloading
> > Oct 09 04:15:56 Issue tracker :
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_extremeshok_clamav-2Dunofficial-2Dsigs_issues&d=DwICAg&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=WaUuzrJtD_PKZ2pBpU-pfAEoxGBj-_rNdSJwvcK9NiI&s=mMxE841bG6uyKmN8KcULOvoeE948yxFA9Mo2udC0y_U&e=
> > Oct 09 04:15:56 Powered By
> https://urldefense.proofpoint.com/v2/url?u=https-3A__eXtremeSHOK.com&d=DwICAg&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=WaUuzrJtD_PKZ2pBpU-pfAEoxGBj-_rNdSJwvcK9NiI&s=7LlLO6tKn_1eYqKp_e8nViWQ6BAjCFkMgYzNFvigtfs&e=
> >*Oct 09 05:14:02 ERROR: clam database directory (clam_dbs) not writable
> /var/lib/clamav*
>
> Looks clear that the urlhaus db was updated OK. Does the unofficial
> update script normally take an hour to run on your system?! The one
> we use usually takes just a few minutes.
>

My bad in trying to economize my post here's the entire update-related
entry:
Oct 09 04:14:01 Preparing Databases
Oct 09 04:14:01 Fri 09 Oct 2020 04:14:01 AM EDT - Pausing database file
updates for 114 seconds...
Oct 09 04:15:55 Fri 09 Oct 2020 04:15:55 AM EDT - Pause complete, checking
for new database files...
Oct 09 04:15:55 Sanesecurity Database File Updates
Oct 09 04:15:55 2 hours have not yet elapsed since the last Sanesecurity
update check
Oct 09 04:15:55 No update check was performed at this time
Oct 09 04:15:55 Next check will be performed in approximately 1 hour(s), 6
minute(s)
Oct 09 04:15:55 SecuriteInfo Database File Updates
Oct 09 04:15:55 4 hours have not yet elapsed since the last SecuriteInfo
update check
Oct 09 04:15:55 No update check was performed at this time
Oct 09 04:15:55 Next check will be performed in approximately 3 hour(s), 6
minute(s)
Oct 09 04:15:55 LinuxMalwareDetect Database File Updates
Oct 09 04:15:55 Checking for LinuxMalwareDetect updates...
Oct 09 04:15:56 No LinuxMalwareDetect database file updates found
Oct 09 04:15:56 MalwarePatrol Database File Updates
Oct 09 04:15:56 24 hours have not yet elapsed since the last malwarepatrol
update check
Oct 09 04:15:56 No update check was performed at this time
Oct 09 04:15:56 Next check will be performed in approximately 7 hour(s), 0
minute(s)
Oct 09 04:15:56 Yara-Rules Database File Updates
Oct 09 04:15:56 Checking for urlhaus updates...
Oct 09 04:15:56 Checking for updated urlhaus database file: urlhaus.ndb
Oct 09 04:15:56 Testing updated urlhaus database file: urlhaus.ndb
Oct 09 04:15:56 Clamscan reports urlhaus urlhaus.ndb database integrity
tested good
Oct 09 04:15:56 Successfully updated urlhaus production database file:
urlhaus.ndb
Oct 09 04:15:56 Update(s) detected, reloading ClamAV databases
Oct 09 04:15:56 ClamAV databases reloading


> > ... perhaps I should contact the ExtremeSHOK contributors ...
>
> I'd have said so, yes.
>

well they may have an idea but I'm starting to think it's not related to
their script. After all the username clamupdate does not come from their
script.

>
> > perhaps there's some debug option that I'm not aware of?
>
> It's just a shell script, you could edit it to put debugging things in
> there if you're comfortable with hacking shell scripts. Does it give
> usage help if run with no arguments? Does it have the '-i' option?
>

Indeed I see some options here:
https://github.com/extremeshok/clamav-unofficial-sigs

So next time it happens I can try some of these:
-v, --verbose Be verbose, enabled when not run under cron
-i, --information Output system and configuration information for viewing
or possible debugging purposes
-t, --test-database Clamscan integrity test a specific database file eg:
'-t filename.ext' (do not include file path)
--check-clamav If ClamD status check is enabled and the socket path is
correctly specifiedthen (sic) test to see if clamd is running or not

Here's what the -i option returns:
su - clamav -s /bin/bash -c '/usr/local/sbin/clamav-unofficial-sigs.sh -i'
################################################################################
eXtremeSHOK.com ClamAV Unofficial Signature Updater
Version: v7.0.1 (2020-01-25)
Required Configuration Version: v91
Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com
################################################################################
Loading config: /etc/clamav-unofficial-sigs/master.conf
Loading config: /etc/clamav-unofficial-sigs/os.conf
Loading config: /etc/clamav-unofficial-sigs/user.conf

*** SCRIPT INFORMATION ***
clamav-unofficial-sigs.sh 7.0.1 (2020-01-25)
Master.conf Version: 91
Minimum required config: 91
*** SYSTEM INFORMATION ***
Linux ourserver 5.7.15-200.fc32.x86_64 #1 SMP Tue Aug 11 16:36:14 UTC 2020
x86_64 x86_64 x86_64 GNU/Linux
*** CLAMSCAN LOCATION & VERSION ***
/usr/bin/clamscan
ClamAV 0.103.0/25952/Fri Oct 9 09:52:40 2020
*** RSYNC LOCATION & VERSION ***
/usr/bin/rsync
rsync version 3.2.3 protocol version 31
*** CURL LOCATION & VERSION ***
/usr/bin/curl
curl 7.69.1 (x86_64-redhat-linux-gnu) libcurl/7.69.1 OpenSSL/1.1.1g-fips
zlib/1.2.11 brotli/1.0.7 libidn2/2.3.0 libpsl/0.21.0 (+libidn2/2.3.0)
libssh/0.9.5/openssl/zlib nghttp2/1.41.0
*** GPG LOCATION & VERSION ***
/usr/bin/gpg
gpg (GnuPG) 2.2.20
*** DIRECTORY INFORMATION ***
Working Directory: /var/lib/clamav-unofficial-sigs
Clam Database Directory: /var/lib/clamav
Configuration Directory: /etc/clamav-unofficial-sigs


> > ... I do see:
> > systemctl status clam
> > clamav-clamonacc.service clamav-unofficial-sigs.service
> > clamd.service
> > clamav-freshclam.service clamav-unofficial-sigs.timer
> > clam-freshclam.service
> > clamav-milter.service clamd@scan.service
> > clamonacc.service
>
> I don't use any of that stuff, I like to know what's going on. It
> might be worth disabling all the service frippery and starting the
> daemons from the command line to see if it behaves any differently.
>

Well systemd is so ingrained in most Linux distributions and the
convenience of starting on reboot is helpful, as all's I need is for our
long-time professor who still has his non-Gmail related email address on
various lists, have a problem getting to his email box, and contacting me
on Xmas eve (like he did last year) as emails are held back as ClamAV isn't
running properly.

Frippery! Ha another one you made me look up.


>
> > I see Fangfrisch <
> https://urldefense.proofpoint.com/v2/url?u=https-3A__rseichter.github.io_fangfrisch_&d=DwICAg&c=aqMfXOEvEJQh2iQMCb7Wy8l0sPnURkcqADc2guUW8IM&r=X0jL9y0sL4r4iU_qVtR3lLNo4tOL1ry_m7-psV3GejY&m=WaUuzrJtD_PKZ2pBpU-pfAEoxGBj-_rNdSJwvcK9NiI&s=7eiHTwe_wlDQm90JBW-6Fudyd4iyBYqMk6hAJzxCDtM&e=
> >is being
> > maintained as an alternative. Haven't tried it yet.
>
> It might not be time to throw out the baby just yet, before swapping
> one lot of unknowns for another lot of unknowns I'd definitely try a
> bit of investigative work. After all other people use this stuff. If
> extra logging, disabling services etc don't lead you anywhere it might
> be worth purging and reinstalling all the implicated packages.


Might be a good idea to purge if I can't figure this out.

Thanks for all you do in this list!

Hi there,

On Sat, 10 Oct 2020, Robert Kudyba wrote:

> ... next time it happens I can try some of these:
> ...

But put some logging in place before it does, so you get as precise a
timeline as you can.

> Here's what the -i option returns:
> ...
> Loading config: /etc/clamav-unofficial-sigs/master.conf
> Loading config: /etc/clamav-unofficial-sigs/os.conf
> Loading config: /etc/clamav-unofficial-sigs/user.conf

I take it you've examined these files for clues? And the systemd unit
files etc.?

> ... the convenience of starting on reboot ...

People used to do that before systemd. :) You can e.g. just put a
command line in /etc/rc.local to get the same effect. The trouble
with systemd is it breaks such a lot of things. The final straw for
me was when it killed an fsck at boot time because it thought it had
been running for too long (90 seconds) leaving me with a trashed 3TB
drive and quite a bit of work to put it back together. One of the
many claims people make about it is dependable naming of the Ethernet
interfaces. Unfortunately they're dependably named something else, so
your remote systems suddenly go dark and you have to get in the Jeep.
Oh, sorry, wrong list.

> ... professor ... Xmas eve ... emails are held back ...

You can probably set it up so that mail flows if ClamAV isn't running,
although most people would probably say that's not a great idea.

> Frippery! Ha another one you made me look up.
> ... Thanks ...

Just returning some favours. It's how it all works. :)

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

>
> On Sat, 10 Oct 2020, Robert Kudyba wrote:
>
> > ... next time it happens I can try some of these:
> > ...
>
> But put some logging in place before it does, so you get as precise a
> timeline as you can.
>
> > Here's what the -i option returns:
> > ...
> > Loading config: /etc/clamav-unofficial-sigs/master.conf
> > Loading config: /etc/clamav-unofficial-sigs/os.conf
> > Loading config: /etc/clamav-unofficial-sigs/user.conf
>
> I take it you've examined these files for clues? And the systemd unit
> files etc.?
>

Indeed and here we are 9 months later and the problem is back. I can see
this happened after Jul 3 at 4:22 AM:
Jul 03 04:22:22 Checking for updated interServer database file:
interservertopline.db

Jul 03 04:22:22 No updated interServer interservertopline.db database file

Jul 03 04:22:22 No interServer database file updates

Jul 03 04:22:22 MalwarePatrol Database File Updates

Jul 03 04:22:22 24 hours have not yet elapsed since the last malwarepatrol
update check

Jul 03 04:22:22 No update check was performed at this time

Jul 03 04:22:22 Next check will be performed in approximately 6 hour(s), 53
minute(s)

Jul 03 04:22:22 URLhaus Database File Updates

Jul 03 04:22:22 Checking for urlhaus updates...

Jul 03 04:22:22 Checking for updated urlhaus database file: urlhaus.ndb

Jul 03 04:22:22 WARNING: Failed connection to
https://urlhaus.abuse.ch/downloads - SKIPPED urlhaus urlhaus.ndb update

Jul 03 04:22:22 No updated urlhaus urlhaus.ndb database file

Jul 03 04:22:22 No urlhaus database file updates

Jul 03 04:22:22 Yara-Rules Database File Updates

Jul 03 04:22:22 24 hours have not yet elapsed since the last
yararulesproject update check

Jul 03 04:22:22 No update check was performed at this time

Jul 03 04:22:22 Next check will be performed in approximately 6 hour(s), 53
minute(s)

Jul 03 04:22:22 Update(s) detected, reloading ClamAV databases

Jul 03 04:22:22 ClamAV databases reloading

Jul 03 04:22:22 Issue tracker :
https://github.com/extremeshok/clamav-unofficial-sigs/issues

Jul 03 04:22:22 Powered By https://eXtremeSHOK.com

Jul 03 05:14:01 ERROR: clam database directory (clam_dbs) not writable
/var/lib/clamav


ps -auwx|grep clam

*clam*av 1533123 0.0 1.2 2783400 1678272 ? Ssl Jul03 7:13
/usr/sbin/*clam*d -c /etc/*clam*d.d/scan.conf

*clam*ilt 1533191 0.0 0.0 1053352 3616 ? Ssl Jul03 0:05
/usr/sbin/*clam*av-milter -c /etc/mail/*clam*av-milter.conf

*clam*av 1533209 0.0 0.0 28268 12480 ? Ss Jul03 0:00
/usr/bin/fresh*clam* -d --foreground=true


ls -ld /var/lib/clamav

drwxr-xr-x. 4 clamupdate clamupdate 8192 Jul 3 04:46 */var/lib/clamav*


and these 3 files have their owner changed but note the old date timestamp:

-rw-r--r-- 1 clamupdate clamupdate 293670 Apr 8 06:32 bytecode.cvd

-rw-r--r-- 1 clamupdate clamupdate 107169718 Jun 22 18:06 daily.cvd

-rw-r--r-- 1 clamupdate clamupdate 117859675 Nov 25 2019 main.cvd


grep clamupdate /etc/clam*/*

/etc/clamav-unofficial-sigs/os.conf:#clam_user="*clamupdate*"

/etc/clamav-unofficial-sigs/os.conf:#clam_group="*clamupdate*"


status clamav-freshclam.service

*?* clamav-freshclam.service - ClamAV virus database updater

Loaded: loaded (/usr/lib/systemd/system/clamav-freshclam.service;
enabled; vendor preset: disabled)

Active: *active (running)* since Sat 2021-07-03 04:46:13 EDT; 1 weeks
1 days ago

Docs: man:freshclam(1)

man:freshclam.conf(5)

https://www.clamav.net/documents

Main PID: 1533209 (freshclam)

Tasks: 1 (limit: 154192)

Memory: 1.7M

CGroup: /system.slice/clamav-freshclam.service

??1533209 /usr/bin/freshclam -d --foreground=true


Jul 11 20:46:13 ourserver.edu freshclam[1533209]: ERROR: Can't create
temporary directory /var/lib/clamav/tmp.92f6163053

Jul 11 20:46:13 ourserver.edu freshclam[1533209]: Hint: The database
directory must be writable for UID 985 or GID 981

Jul 11 20:46:13 ourserver.edu freshclam[1533209]: ERROR: Update failed.

Jul 11 20:46:13 ourserver.edu freshclam[1533209]: Received signal: wake up

Jul 11 20:46:13 ourserver.edu freshclam[1533209]: ClamAV update process
started at Sun Jul 11 20:46:13 2021

Jul 11 20:46:13 ourserver.edu freshclam[1533209]: *DNS record is older than
3 hours.*

Jul 11 20:46:13 ourserver.edu freshclam[1533209]: *Can't create temporary
directory /var/lib/clamav/tmp.92f6163053*

Jul 11 20:46:13 ourserver.edu freshclam[1533209]: Hint: The database
directory must be writable for UID 985 or GID 981

Jul 11 20:46:13 ourserver.edu freshclam[1533209]: *Update failed.*

Jul 11 20:46:13 ourserver.edu freshclam[1533209]:
--------------------------------------


cat /usr/lib/systemd/system/clamav-freshclam.service

[Unit]

Description=ClamAV virus database updater

Documentation=man:freshclam(1) man:freshclam.conf(5)
https://www.clamav.net/documents

# If user wants it run from cron, don't start the daemon.

ConditionPathExists=!/etc/cron.d/clamav-update

Wants=network-online.target

After=network-online.target


[Service]

ExecStart=/usr/bin/freshclam -d --foreground=true


[Install]

WantedBy=multi-user.target


systemctl status clamav-unofficial-sigs.service

? clamav-unofficial-sigs.service - Clamav Unofficial Sigs Update service

Loaded: loaded (/etc/systemd/system/clamav-unofficial-sigs.service;
static)

Active: inactive (dead)

Docs: man:clamav-unofficial-sigs(8)

(base) [root@ourserver ~]# systemctl status clamav-unofficial-sigs.timer

? clamav-unofficial-sigs.timer - Clamav Unofficial Sigs Update timer

Loaded: loaded (/etc/systemd/system/clamav-unofficial-sigs.timer;
disabled; vendor preset: disabled)

Active: inactive (dead)

Trigger: n/a

Triggers: ? clamav-unofficial-sigs.service

Docs: man:clamav-unofficial-sigs(8)


in /etc/cron.d/clamav-unofficial-sigs we have:


14 * * * * clamav [ -x /usr/local/sbin/clamav-unofficial-sigs.sh ] &&
/usr/bin/bash /usr/local/sbin/clamav-unofficial-sigs.sh


Is this a clue in the system logs? UID 985 = clamav


Jul 3 04:22:32 ourserver systemd[1]: Stopping User Manager for UID 985...

Jul 3 04:22:32 ourserver systemd[1519673]: Stopped target Main User Target.

Jul 3 04:22:32 ourserver systemd[1519673]: Stopped target Basic System.

Jul 3 04:22:32 ourserver systemd[1519673]: Stopped target Paths.


grep 985 /etc/passwd

clamav:x:*985*:981::/var/run/clamav:/sbin/nologin

Hi there,

On Sun, 11 Jul 2021, Robert Kudyba wrote:
> On Sat, 10 Oct 2020, G.W. Haywood wrote:
>> On Sat, 10 Oct 2020, Robert Kudyba wrote:
>>
>>> ... next time it happens I can try some of these:
>>> ...
>>
>> ... put some logging in place before it does, so you get as precise a
>> timeline as you can.
>
> Indeed and here we are 9 months later and the problem is back. I can see
> this happened after Jul 3 at 4:22 AM:
> ...
> Jul 03 05:14:01 ERROR: clam database directory (clam_dbs) not writable /var/lib/clamav

Where's the log of the permissions, listed every minute, which I
suggested to you back in October?!

On Fri, 9 Oct 2020, G.W. Haywood wrote:

On Sun, 11 Jul 2021, Robert Kudyba wrote:
> ls -ld /var/lib/clamav
>
> drwxr-xr-x. 4 clamupdate clamupdate 8192 Jul 3 04:46 */var/lib/clamav*

The 'dot' after the directory permissions probably means that SELinux
or similar is involved. If so, it might have been good to mention it
earlier. Have you made sure that there's no other access control than
the file and directory permissions which you've been showing us?

If you made the permissions

drwxrwxr-x

instead, you could probably forget about it - but again it might be to
paper over a crack. Another thought, do you have the 'setgid' bit set
on one of the parent directories?

> ... these 3 files have their owner changed but note the old date timestamp:
>
> -rw-r--r-- 1 clamupdate clamupdate 293670 Apr 8 06:32 bytecode.cvd
>
> -rw-r--r-- 1 clamupdate clamupdate 107169718 Jun 22 18:06 daily.cvd
>
> -rw-r--r-- 1 clamupdate clamupdate 117859675 Nov 25 2019 main.cvd

If it's only these files which are getting the wrong UID/GID then it
sort of implicates whatever is running freshclam, since that's likely
to be the thing which modifies only those files. But I'd still want
to see that log.

> grep 985 /etc/passwd
>
> clamav:x:*985*:981::/var/run/clamav:/sbin/nologin

I guess that group 981 is the GID of the 'clamupdate' group?

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

>
> >> ... next time it happens I can try some of these:
> >>> ...
> >>
> >> ... put some logging in place before it does, so you get as precise a
> >> timeline as you can.
> >
> > Indeed and here we are 9 months later and the problem is back. I can see
> > this happened after Jul 3 at 4:22 AM:
> > ...
> > Jul 03 05:14:01 ERROR: clam database directory (clam_dbs) not writable
> /var/lib/clamav
>
> Where's the log of the permissions, listed every minute, which I
> suggested to you back in October?!
>

I did proffer the -i option:
su - clamav -s /bin/bash -c '/usr/local/sbin/clamav-unofficial-sigs.sh -i'
################################################################################
eXtremeSHOK.com ClamAV Unofficial Signature Updater
Version: v7.2.5 (2021-03-20)
Required Configuration Version: v96
Copyright (c) Adrian Jon Kriel :: admin@extremeshok.com
################################################################################
Loading config: /etc/clamav-unofficial-sigs/master.conf
Loading config: /etc/clamav-unofficial-sigs/os.conf
Loading config: /etc/clamav-unofficial-sigs/user.conf

*** SCRIPT INFORMATION ***
clamav-unofficial-sigs.sh 7.2.5 (2021-03-20)
Master.conf Version: 97
Minimum required config: 96
*** SYSTEM INFORMATION ***
Linux storm.cis.fordham.edu 5.12.12-200.fc33.x86_64 #1 SMP Fri Jun 18
14:28:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
*** CLAMSCAN LOCATION & VERSION ***
/usr/bin/clamscan
ClamAV 0.103.3/26228/Sun Jul 11 07:05:30 2021
*** RSYNC LOCATION & VERSION ***
/usr/bin/rsync
rsync version 3.2.3 protocol version 31
*** CURL LOCATION & VERSION ***
/usr/bin/curl
curl 7.71.1 (x86_64-redhat-linux-gnu) libcurl/7.71.1 OpenSSL/1.1.1k-fips
zlib/1.2.11 brotli/1.0.9 libidn2/2.3.1 libpsl/0.21.1 (+libidn2/2.3.0)
libssh/0.9.5/openssl/zlib nghttp2/1.43.0
*** GPG LOCATION & VERSION ***
/usr/bin/gpg
gpg (GnuPG) 2.2.25
*** DIRECTORY INFORMATION ***
Working Directory: /var/lib/clamav-unofficial-sigs
Clam Database Directory: /var/lib/clamav
Configuration Directory: /etc/clamav-unofficial-sigs

The suggestion you gave me previously:

>* It's just a shell script, you could edit it to put debugging things in
*>* there if you're comfortable with hacking shell scripts. Does it give
*>* usage help if run with no arguments?*

I guess the answer is I'm not comfortable with hacking the shell script.


>
> On Fri, 9 Oct 2020, G.W. Haywood wrote:
> |> ...start with some simple logging [...] Something like this
> |> in a crontab:
> |>
> |> * * * * * /bin/echo -n "$(/bin/date) " >> /var/log/clam_perms.log ; \
> |> /bin/ls -l /var/lib/clamav >> /var/log/clam_perms.log
>

OK just set this in cron but I suppose it isn't useful until the problem
happens again.

On Sun, 11 Jul 2021, Robert Kudyba wrote:
> > ls -ld /var/lib/clamav
> >
> > drwxr-xr-x. 4 clamupdate clamupdate 8192 Jul 3 04:46 */var/lib/clamav*
>
> The 'dot' after the directory permissions probably means that SELinux
> or similar is involved. If so, it might have been good to mention it
> earlier. Have you made sure that there's no other access control than
> the file and directory permissions which you've been showing us?
>

SELinux definitely disabled this entire time.
sestatus
SELinux status: disabled

ls -ald /var/lib/clamav
drwxrwxr-x. 4 clamav clamav 8192 Jul 12 08:23 /var/lib/clamav

ls -Zd /var/lib/clamav
system_u:object_r:antivirus_db_t:s0 /var/lib/clamav


> If you made the permissions
>
> drwxrwxr-x
>
> instead, you could probably forget about it - but again it might be to
> paper over a crack.


OK so some variation of setfattr -h -x security.selinux


> Another thought, do you have the 'setgid' bit set on one of the parent
> directories?
>

Running find /var/lib/ -perm /6000 -type f results in only some Docker
containers


>
> > ... these 3 files have their owner changed but note the old date
> timestamp:
> >
> > -rw-r--r-- 1 clamupdate clamupdate 293670 Apr 8 06:32 bytecode.cvd
> >
> > -rw-r--r-- 1 clamupdate clamupdate 107169718 Jun 22 18:06 daily.cvd
> >
> > -rw-r--r-- 1 clamupdate clamupdate 117859675 Nov 25 2019 main.cvd
>
> If it's only these files which are getting the wrong UID/GID then it
> sort of implicates whatever is running freshclam, since that's likely
> to be the thing which modifies only those files.


ps -auwx|grep fresh
clamav 3930506 0.0 0.0 103116 16108 ? Ss Jul11 0:05
/usr/bin/freshclam -d --foreground=true


> But I'd still want to see that log.
>

The log from the cronjob, freshclam or eXtremeSHOK.com ClamAV Unofficial
Signature Updater?


> > grep 985 /etc/passwd
> >
> > clamav:x:*985*:981::/var/run/clamav:/sbin/nologin
>
> I guess that group 981 is the GID of the 'clamupdate' group?
>

grep 981 /etc/group
clamav:x:981:clamscan,clamilt,clamupdate

Hello again,

On Mon, 12 Jul 2021, Robert Kudyba wrote:

> ... I'm not comfortable with hacking the shell script.

Fair enough. In any case now it looks to me less likely that it's the
shell script that's causing the issue (because you said in your last
mail that just three files showed incorrect ownership).

>> On Fri, 9 Oct 2020, G.W. Haywood wrote:
>> [...] start with some simple logging [...]
>
> OK just set this in cron but I suppose it isn't useful until the
> problem happens again.

Quite so. You set it up and wait. If you'd set it up in October your
wait would now be over. :) Talk to you again around Christmas time?

>> If you made the permissions
>>
>> drwxrwxr-x
>>
>> instead, you could probably forget about it - but again it might be to
>> paper over a crack.
>
> OK so some variation of setfattr -h -x security.selinux

No, I'd just have typed (at a root shell prompt)

chmod g+w /var/lib/clamav

>> Another thought, do you have the 'setgid' bit set on one of the parent
>> directories?
>
> Running find /var/lib/ -perm /6000 -type f results in only some Docker
> containers

I asked about the permissions on the directories, not on files. In
your 'find' command there you specifically limit the search to files
and not directories with "-type f". See 'man find' for more (but IMO
'find' is a bit like a cornered rat and I'm starting to think it might
not be the best tool in the box for you to be playing with). Just use

ls -l / | grep var

to see the permissions on /var and

ls -l /var | grep lib

to see the permissions on /var/lib.

>> But I'd still want to see that log.
>
> The log from the cronjob, freshclam or eXtremeSHOK.com ClamAV Unofficial
> Signature Updater?

The cron job which I suggested. From a root shell prompt, to edit the
crontab give the command

crontab -e

which will fire up the default editor or the one you've configured.
Just paste these two lines (I tweaked it a bit from last October's
version) right at at the bottom:

FILE=/var/log/clam_perms.log
* * * * * /bin/date >> $FILE ; /bin/ls -l /var/lib/clamav >> $FILE

That will write a time/date stamp and a directory listing to the file
every minute until further notice. Yes, there will be quite a lot of
output, but (by the standards of the 21st century) it won't be a huge
file, and you'll get what I'm looking for which is when (to about the
nearest minute) the permissions were changed. If you know to within
the same sort of precision when things are run, that should give you
some clue to what changed the permissions.

> grep 981 /etc/group
> clamav:x:981:clamscan,clamilt,clamupdate

Hmmm. So group ID 981 is 'clamav'. What's the numeric ID for the
'clamupdate' group (and 'clamilt' for completeness)? To me it seems
just a little excessive to have separate users (and maybe groups) for
clamd, clamav-milter and freshclam. I think somebody (probably this
was somebody at Red Hat) lost the plot there, but I suppose you're
stuck with that unless you junk the ClamAV packages and build it all
from source. IMO there's a lot to recommend that.

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

>
> I asked about the permissions on the directories, not on files. In
> your 'find' command there you specifically limit the search to files
> and not directories with "-type f". See 'man find' for more (but IMO
> 'find' is a bit like a cornered rat and I'm starting to think it might
> not be the best tool in the box for you to be playing with). Just use
>
> ls -l / | grep var
>

> to see the permissions on /var and
>

ls -l / | grep var
lrwxrwxrwx 1 root root 19 Aug 31 2020 snap -> /var/lib/snapd/snap
drwxr-xr-x. 23 root root 4096 Jan 11 14:49 var


> ls -l /var | grep lib
>
> to see the permissions on /var/lib.
>

ls -l /var | grep lib
drwxr-xr-x. 95 root root 4096 Mar 20 08:00 lib

>> But I'd still want to see that log.
> >
> > The log from the cronjob, freshclam or eXtremeSHOK.com ClamAV Unofficial
> > Signature Updater?
>
> The cron job which I suggested. From a root shell prompt, to edit the
> crontab give the command
>
> crontab -e
>
> which will fire up the default editor or the one you've configured.
> Just paste these two lines (I tweaked it a bit from last October's
> version) right at at the bottom:
>
> FILE=/var/log/clam_perms.log
> * * * * * /bin/date >> $FILE ; /bin/ls -l /var/lib/clamav >> $FILE
>
> That will write a time/date stamp and a directory listing to the file
> every minute until further notice. Yes, there will be quite a lot of
> output, but (by the standards of the 21st century) it won't be a huge
> file, and you'll get what I'm looking for which is when (to about the
> nearest minute) the permissions were changed. If you know to within
> the same sort of precision when things are run, that should give you
> some clue to what changed the permissions.
>

I had * * * * * /bin/echo -n "$(/bin/date) " >> /var/log/clam_perms.log &&
/bin/ls -l /var/lib/clamav >> /var/log/clam_perms.log so it's been
populating for a couple of hours.

> grep 981 /etc/group
> > clamav:x:981:clamscan,clamilt,clamupdate
>
> Hmmm. So group ID 981 is 'clamav'. What's the numeric ID for the
> 'clamupdate' group (and 'clamilt' for completeness)? To me it seems
> just a little excessive to have separate users (and maybe groups) for
> clamd, clamav-milter and freshclam. I think somebody (probably this
> was somebody at Red Hat) lost the plot there, but I suppose you're
> stuck with that unless you junk the ClamAV packages and build it all
> from source. IMO there's a lot to recommend that.
>

grep clam /etc/passwd
clamilt:x:989:985:Clamav Milter User:/var/run/clamav-milter:/sbin/nologin
clamav:x:985:981::/var/run/clamav:/sbin/nologin
clamupdate:x:983:979:Clamav database update
user:/var/lib/clamav:/sbin/nologin
clamscan:x:982:978:Clamav scanner user:/:/sbin/nologin

Hi there,

On Mon, 12 Jul 2021, Robert Kudyba wrote:

> ls -l / | grep var
> ...
> drwxr-xr-x. 23 root root 4096 Jan 11 14:49 var
> ...
> ls -l /var | grep lib
> drwxr-xr-x. 95 root root 4096 Mar 20 08:00 lib

OK (assuming that you're *really* not using SELinux nor anything like it).

> ... /var/log/clam_perms.log ... populating for a couple of hours.

OK, we wait. :)

> grep clam /etc/passwd
> clamilt:x:989:985:Clamav Milter User:/var/run/clamav-milter:/sbin/nologin
> clamav:x:985:981::/var/run/clamav:/sbin/nologin
> clamupdate:x:983:979:Clamav database update user:/var/lib/clamav:/sbin/nologin
> clamscan:x:982:978:Clamav scanner user:/:/sbin/nologin

Interesting. The 'clamav' user seems not to have been created by the
same setup process which created the other three, since it didn't get
a text description. There's a suspicious gap in the numeric IDs from
985:981 to 989:985 like the milter IDs were added later. Make sense?

What does

grep clam /etc/group

give you?

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

>
>
>
> > grep clam /etc/passwd
> > clamilt:x:989:985:Clamav Milter User:/var/run/clamav-milter:/sbin/nologin
> > clamav:x:985:981::/var/run/clamav:/sbin/nologin
> > clamupdate:x:983:979:Clamav database update
> user:/var/lib/clamav:/sbin/nologin
> > clamscan:x:982:978:Clamav scanner user:/:/sbin/nologin
>
> Interesting. The 'clamav' user seems not to have been created by the
> same setup process which created the other three, since it didn't get
> a text description. There's a suspicious gap in the numeric IDs from
> 985:981 to 989:985 like the milter IDs were added later. Make sense?
>
> What does
>
> grep clam /etc/group
>
> give you?
>
grep clam /etc/group
clamilt:x:985:clamav,clamscan
clamav:x:981:clamscan,clamilt,clamupdate
clamupdate:x:979:clamav
clamscan:x:978:clamilt,clamav
virusgroup:x:949:clamupdate,clamscan,clamilt

After an upgrade of Fedora and subsequent reboot the permission problem
returned. Same the files:
-rw-r--r-- 1 clamupdate clamupdate 293670 Apr 8 06:32 bytecode.cvd
-rw-r--r-- 1 clamupdate clamupdate 107169718 Jun 22 18:06 daily.cvd
-rw-r--r-- 1 clamupdate clamupdate 117859675 Nov 25 2019 main.cvd

as well as the directory:
ls -dl /var/lib/clamav
drwxr-xr-x 4 clamupdate clamupdate 8192 Jul 13 11:39 /var/lib/clamav

Also in the clamav-unofficial-sigs.log file
Jul 13 12:14:01 ERROR: clam database directory (clam_dbs) not writable
/var/lib/clamav

Permission log file is available at
https://storm.cis.fordham.edu/~rkudyba/clam_perms.log

From the cron log file:
Jul 13 12:14:01 ourserver CROND[22349]: (clamav) CMD ([ -x
/usr/local/sbin/clamav-unofficial-sigs.sh ] && /usr/bin/bash
/usr/local/sbin/clamav-unofficial-sigs.sh)
Jul 13 12:14:03 ourserver CROND[22318]: (clamav) CMDEND ([ -x
/usr/local/sbin/clamav-unofficial-sigs.sh ] && /usr/bin/bash
/usr/local/sbin/clamav-unofficial-sigs.sh)

On Mon, Jul 12, 2021 at 12:31 PM Robert Kudyba <rkudyba@fordham.edu> wrote:

>
>>
>> > grep clam /etc/passwd
>> > clamilt:x:989:985:Clamav Milter
>> User:/var/run/clamav-milter:/sbin/nologin
>> > clamav:x:985:981::/var/run/clamav:/sbin/nologin
>> > clamupdate:x:983:979:Clamav database update
>> user:/var/lib/clamav:/sbin/nologin
>> > clamscan:x:982:978:Clamav scanner user:/:/sbin/nologin
>>
>> Interesting. The 'clamav' user seems not to have been created by the
>> same setup process which created the other three, since it didn't get
>> a text description. There's a suspicious gap in the numeric IDs from
>> 985:981 to 989:985 like the milter IDs were added later. Make sense?
>>
>> What does
>>
>> grep clam /etc/group
>>
>> give you?
>>
> grep clam /etc/group
> clamilt:x:985:clamav,clamscan
> clamav:x:981:clamscan,clamilt,clamupdate
> clamupdate:x:979:clamav
> clamscan:x:978:clamilt,clamav
> virusgroup:x:949:clamupdate,clamscan,clamilt
>
>

Hello again,

On Tue, 13 Jul 2021, Robert Kudyba wrote:

> After an upgrade of Fedora and subsequent reboot the permission problem
> returned. Same the files:
> -rw-r--r-- 1 clamupdate clamupdate 293670 Apr 8 06:32 bytecode.cvd
> -rw-r--r-- 1 clamupdate clamupdate 107169718 Jun 22 18:06 daily.cvd
> -rw-r--r-- 1 clamupdate clamupdate 117859675 Nov 25 2019 main.cvd
>
> as well as the directory:
> ls -dl /var/lib/clamav
> drwxr-xr-x 4 clamupdate clamupdate 8192 Jul 13 11:39 /var/lib/clamav
>
> Also in the clamav-unofficial-sigs.log file
> Jul 13 12:14:01 ERROR: clam database directory (clam_dbs) not writable
> /var/lib/clamav
>
> Permission log file is available at
> https://storm.cis.fordham.edu/~rkudyba/clam_perms.log

Now we're gettting somewhere. :)

The log starts with

Mon Jul 12 09:59:01 AM EDT 2021

and the first timestamp for daily.cld is

-rw-r--r-- 1 clamav clamav 327757824 Jul 12 09:59 daily.cld

It is perhaps a little unfortunate that the log starts at the exact
time of the last modification of daily.cld - we might need to come
back to that but I hope not. Also there are three timestamps where
I'd expect only one so I suspect something is a little bit squiffy in
the crontab, but that probably doesn't matter.
In the database directory at 09:59 you have the four files

-rw-r--r-- 1 clamav clamav 1438720 Mar 17 10:47 bytecode.cld
-rw-r--r-- 1 clamav clamav 293670 Apr 8 06:32 bytecode.cvd
-rw-r--r-- 1 clamav clamav 327757824 Jul 12 09:59 daily.cld
-rw-r--r-- 1 clamav clamav 117859675 Nov 25 2019 main.cvd

and a bunch of others which we're not concerned with. Firstly, you
really don't want both a bytecode.cld *and* a bytecode.cvd, so you
should probably just delete the older one. To cut down on the amount
of text I used this shell command to view the log:

$ grep '\(bytecode\|main\.\|daily\|clamupdate\|\(Mon\|Tue\) Jul 1\)' clam_perms.log | less

Then I just searched for interesting things (I've had a lot of
practice at trawling through logs...)

Here's what happens just after 10AM on the 13th:

Tue Jul 13 10:01:01 AM EDT 2021
-rw-r--r-- 1 clamav clamav 1438720 Mar 17 10:47 bytecode.cld
-rw-r--r-- 1 clamav clamav 293670 Apr 8 06:32 bytecode.cvd
-rw-r--r-- 1 clamav clamav 327757824 Jul 12 09:59 daily.cld
-rw-r--r-- 1 clamav clamav 117859675 Nov 25 2019 main.cvd
Tue Jul 13 10:02:01 AM EDT 2021
-rw-r--r-- 1 clamav clamav 1438720 Mar 17 10:47 bytecode.cld
-rw-r--r-- 1 clamav clamav 293670 Apr 8 06:32 bytecode.cvd
-rw-r--r-- 1 clamav clamav 327797248 Jul 13 10:00 daily.cld
-rw-r--r-- 1 clamav clamav 117859675 Nov 25 2019 main.cvd

So daily.cld was updated, presumably by freshclam. That's good, as
nothing seems to have broken. Can you confirm that happened from the
freshclam log? Is freshclam running from cron or as a daemon?

----------------------------------------------------------------------

The next thing that I see of interest is

Tue Jul 13 11:10:02 AM EDT 2021
-rw-r--r-- 1 clamav clamav 1438720 Mar 17 10:47 bytecode.cld
-rw-r--r-- 1 clamav clamav 293670 Apr 8 06:32 bytecode.cvd
-rw-r--r-- 1 clamav clamav 327797248 Jul 13 10:00 daily.cld
-rw-r--r-- 1 clamav clamav 117859675 Nov 25 2019 main.cvd
Tue Jul 13 12:02:01 PM EDT 2021
-rw-r--r-- 1 clamav clamav 1438720 Mar 17 10:47 bytecode.cld
-rw-r--r-- 1 clamupdate clamupdate 293670 Apr 8 06:32 bytecode.cvd
-rw-r--r-- 1 clamav clamav 327797248 Jul 13 10:00 daily.cld
-rw-r--r-- 1 clamupdate clamupdate 107169718 Jun 22 18:06 daily.cvd
-rw-r--r-- 1 clamupdate clamupdate 117859675 Nov 25 2019 main.cvd

There's a fifty minute gap in the log. Why is that? Presumably this
is about the time you updated and rebooted the system. Are you sure
that the system time gets set correctly at boot? We need to know that
we can rely on the timestamps in the logs. All the logs.

Anyway, suddenly the owner/group IDs have changed and you have both a
daily.cld and a daily.cvd - which isn't good news, especially as one
of them is over three weeks old. Where did it come from?

> From the cron log file:
> Jul 13 12:14:01 ourserver CROND[22349]: (clamav) CMD ([ -x
> /usr/local/sbin/clamav-unofficial-sigs.sh ] && /usr/bin/bash
> /usr/local/sbin/clamav-unofficial-sigs.sh)
> Jul 13 12:14:03 ourserver CROND[22318]: (clamav) CMDEND ([ -x
> /usr/local/sbin/clamav-unofficial-sigs.sh ] && /usr/bin/bash
> /usr/local/sbin/clamav-unofficial-sigs.sh)

Assuming that we can believe the timestamps, then any problems that
arose from ownership by the clamupdate user/group had already happened
at 12:02 so it was *not* the run of clamav-unofficial-sigs.sh at 12:14
which caused them.

Is this the first time that clamav-unofficial-sigs.sh ran?

What's in the freshclam log about these times?

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

> -rw-r--r-- 1 clamav clamav 1438720 Mar 17 10:47 bytecode.cld
> -rw-r--r-- 1 clamav clamav 293670 Apr 8 06:32 bytecode.cvd
> -rw-r--r-- 1 clamav clamav 327757824 Jul 12 09:59 daily.cld
> -rw-r--r-- 1 clamav clamav 117859675 Nov 25 2019 main.cvd
>
> and a bunch of others which we're not concerned with. Firstly, you
> really don't want both a bytecode.cld *and* a bytecode.cvd, so you
> should probably just delete the older one.


Done.


> Here's what happens just after 10AM on the 13th:
>
> Tue Jul 13 10:01:01 AM EDT 2021
> -rw-r--r-- 1 clamav clamav 1438720 Mar 17 10:47 bytecode.cld
> -rw-r--r-- 1 clamav clamav 293670 Apr 8 06:32 bytecode.cvd
> -rw-r--r-- 1 clamav clamav 327757824 Jul 12 09:59 daily.cld
> -rw-r--r-- 1 clamav clamav 117859675 Nov 25 2019 main.cvd
> Tue Jul 13 10:02:01 AM EDT 2021
> -rw-r--r-- 1 clamav clamav 1438720 Mar 17 10:47 bytecode.cld
> -rw-r--r-- 1 clamav clamav 293670 Apr 8 06:32 bytecode.cvd
> -rw-r--r-- 1 clamav clamav 327797248 Jul 13 10:00 daily.cld
> -rw-r--r-- 1 clamav clamav 117859675 Nov 25 2019 main.cvd
>
> So daily.cld was updated, presumably by freshclam. That's good, as
> nothing seems to have broken. Can you confirm that happened from the
> freshclam log?


here are the logs from 10:01 AM Jul 13:
Jul 13 10:01:02 storm freshclam[3930506]: Database test passed.
Jul 13 10:01:02 storm freshclam[3930506]: daily.cld updated (version:
26230, sigs: 3995778, f-level: 63, builder: raynman)
Jul 13 10:01:02 storm freshclam[3930506]: daily.cld updated (version:
26230, sigs: 3995778, f-level: 63, builder: raynman)
Jul 13 10:01:02 storm freshclam[3930506]: main.cvd database is up-to-date
(version: 59, sigs: 4564902, f-level: 60, builder: sigmgr)
Jul 13 10:01:02 storm freshclam[3930506]: main.cvd database is up-to-date
(version: 59, sigs: 4564902, f-level: 60, builder: sigmgr)
Jul 13 10:01:02 storm freshclam[3930506]: bytecode.cvd database is
up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2)
Jul 13 10:01:02 storm freshclam[3930506]: bytecode.cvd database is
up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2)
Jul 13 10:01:03 storm freshclam[3930506]: securiteinfo.hdb is up-to-date
(version: custom database)
Jul 13 10:01:03 storm freshclam[3930506]: securiteinfo.hdb is up-to-date
(version: custom database)
Jul 13 10:01:03 storm freshclam[3930506]: securiteinfo.ign2 is up-to-date
(version: custom database)
Jul 13 10:01:03 storm freshclam[3930506]: securiteinfo.ign2 is up-to-date
(version: custom database)
Jul 13 10:01:03 storm freshclam[3930506]: javascript.ndb is up-to-date
(version: custom database)
Jul 13 10:01:03 storm freshclam[3930506]: javascript.ndb is up-to-date
(version: custom database)
Jul 13 10:01:10 storm freshclam[3930506]: Testing database:
'/var/lib/clamav/tmp.f9e1fecbc3/clamav-7b04ccc60e7adc16d356b3b689db8e0f.tmp-spam_marketing.ndb'
...
Jul 13 10:01:10 ourserver freshclam[3930506]: Testing database:
'/var/lib/clamav/tmp.f9e1fecbc3/clamav-7b04ccc60e7adc16d356b3b689db8e0f.tmp-spam_marketing.ndb'
...
Jul 13 10:01:10 ourserver freshclam[3930506]: Database test passed.
Jul 13 10:01:10 ourserver freshclam[3930506]: Database test passed.
Jul 13 10:01:10 ourserver freshclam[3930506]: spam_marketing.ndb updated
(version: custom database, sigs: 31016)
Jul 13 10:01:10 ourserver freshclam[3930506]: spam_marketing.ndb updated
(version: custom database, sigs: 31016)
Jul 13 10:01:10 ourserver freshclam[3930506]: securiteinfohtml.hdb is
up-to-date (version: custom database)
Jul 13 10:01:10 ourserver freshclam[3930506]: securiteinfohtml.hdb is
up-to-date (version: custom database)
Jul 13 10:01:10 ourserver freshclam[3930506]: securiteinfoascii.hdb is
up-to-date (version: custom database)
Jul 13 10:01:10 ourserver freshclam[3930506]: securiteinfoascii.hdb is
up-to-date (version: custom database)
Jul 13 10:01:11 ourserver freshclam[3930506]: securiteinfoandroid.hdb is
up-to-date (version: custom database)
Jul 13 10:01:11 ourserver freshclam[3930506]: securiteinfoandroid.hdb is
up-to-date (version: custom database)
Jul 13 10:01:11 ourserver freshclam[3930506]: securiteinfoold.hdb is
up-to-date (version: custom database)
Jul 13 10:01:11 ourserver freshclam[3930506]: securiteinfoold.hdb is
up-to-date (version: custom database)
Jul 13 10:01:11 ourserver freshclam[3930506]: securiteinfopdf.hdb is
up-to-date (version: custom database)
Jul 13 10:01:11 ourserver freshclam[3930506]: securiteinfopdf.hdb is
up-to-date (version: custom database)
Jul 13 10:01:11 ourserver freshclam[3930506]: safebrowsing.gdb is
up-to-date (version: custom database)
Jul 13 10:01:11 ourserver freshclam[3930506]: safebrowsing.gdb is
up-to-date (version: custom database)
Jul 13 10:01:11 ourserver freshclam[3930506]:
--------------------------------------


> Is freshclam running from cron or as a daemon?
>

Daemon
ps -auwx|grep freshclam
clamav 3818 0.0 0.0 28952 12864 ? Ss 12:00 0:00
/usr/bin/freshclam -d --foreground=true


>
> ----------------------------------------------------------------------
>
> The next thing that I see of interest is
>
> Tue Jul 13 11:10:02 AM EDT 2021
> -rw-r--r-- 1 clamav clamav 1438720 Mar 17 10:47 bytecode.cld
> -rw-r--r-- 1 clamav clamav 293670 Apr 8 06:32 bytecode.cvd
> -rw-r--r-- 1 clamav clamav 327797248 Jul 13 10:00 daily.cld
> -rw-r--r-- 1 clamav clamav 117859675 Nov 25 2019 main.cvd
> Tue Jul 13 12:02:01 PM EDT 2021
> -rw-r--r-- 1 clamav clamav 1438720 Mar 17 10:47 bytecode.cld
> -rw-r--r-- 1 clamupdate clamupdate 293670 Apr 8 06:32 bytecode.cvd
> -rw-r--r-- 1 clamav clamav 327797248 Jul 13 10:00 daily.cld
> -rw-r--r-- 1 clamupdate clamupdate 107169718 Jun 22 18:06 daily.cvd
> -rw-r--r-- 1 clamupdate clamupdate 117859675 Nov 25 2019 main.cvd
>
> There's a fifty minute gap in the log. Why is that? Presumably this
> is about the time you updated and rebooted the system.

correct


> Are you sure
> that the system time gets set correctly at boot? We need to know that
> we can rely on the timestamps in the logs. All the logs.
>

systemctl status chronyd
? chronyd.service - NTP client/server
Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled;
vendor preset: enabled)
Active: active (running) since Tue 2021-07-13 12:00:50 EDT; 2h 46min
ago
Docs: man:chronyd(8)
man:chrony.conf(5)
Process: 3171 ExecStart=/usr/sbin/chronyd $OPTIONS (code=exited,
status=0/SUCCESS)
Main PID: 3232 (chronyd)
Tasks: 1 (limit: 154189)
Memory: 4.6M
CGroup: /system.slice/chronyd.service
??3232 /usr/sbin/chronyd

Jul 13 12:00:50 ourserver.edu systemd[1]: Starting NTP client/server...
Jul 13 12:00:50 ourserver.edu chronyd[3232]: chronyd version 4.1 starting
(+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +NTS
+SECHASH +IPV6 +DEBUG)
Jul 13 12:00:50 ourserver.edu chronyd[3232]: Frequency -34.655 +/- 0.141
ppm read from /var/lib/chrony/drift
Jul 13 12:00:50 ourserver.edu chronyd[3232]: Using right/UTC timezone to
obtain leap second data
Jul 13 12:00:50 ourserver.edu systemd[1]: Started NTP client/server.
Jul 13 12:01:34 ourserver.edu chronyd[3232]: Selected source 50.205.57.38 (
2.fedora.pool.ntp.org)
Jul 13 12:01:34 ourserver.edu chronyd[3232]: System clock TAI offset set to
37 seconds


> Anyway, suddenly the owner/group IDs have changed and you have both a
> daily.cld and a daily.cvd - which isn't good news, especially as one
> of them is over three weeks old. Where did it come from?
>

Right, that's the question.


> > From the cron log file:
> > Jul 13 12:14:01 ourserver CROND[22349]: (clamav) CMD ([ -x
> > /usr/local/sbin/clamav-unofficial-sigs.sh ] && /usr/bin/bash
> > /usr/local/sbin/clamav-unofficial-sigs.sh)
> > Jul 13 12:14:03 ourserver CROND[22318]: (clamav) CMDEND ([ -x
> > /usr/local/sbin/clamav-unofficial-sigs.sh ] && /usr/bin/bash
> > /usr/local/sbin/clamav-unofficial-sigs.sh)
>
> Assuming that we can believe the timestamps, then any problems that
> arose from ownership by the clamupdate user/group had already happened
> at 12:02 so it was *not* the run of clamav-unofficial-sigs.sh at 12:14
> which caused them.
>
> Is this the first time that clamav-unofficial-sigs.sh ran?
>

No it's been running all the time. So are freshclam and
clamav-unofficial-sigs.sh not supposed to run as separate processes?

>
> What's in the freshclam log about these times?
>

Nothing as the upgrade/reboot was still happening. The next freshclam is:
Jul 13 14:00:58 ourserver freshclam[3818]: Received signal: wake up
Jul 13 14:00:58 ourserver freshclam[3818]: ClamAV update process started
at Tue Jul 13 14:00:58 2021
Jul 13 14:00:58 ourserver freshclam[3818]: Received signal: wake up
Jul 13 14:00:58 ourserver freshclam[3818]: ClamAV update process started
at Tue Jul 13 14:00:58 2021
Jul 13 14:00:58 ourserver freshclam[3818]: ERROR: Can't create temporary
directory /var/lib/clamav/tmp.21024dac47
Jul 13 14:00:58 ourserver freshclam[3818]: Hint: The database directory
must be writable for UID 985 or GID 981
Jul 13 14:00:58 ourserver freshclam[3818]: ERROR: Update failed.
Jul 13 14:00:58 ourserver freshclam[3818]: Can't create temporary
directory /var/lib/clamav/tmp.21024dac47
Jul 13 14:00:58 ourserver freshclam[3818]: Hint: The database directory
must be writable for UID 985 or GID 981
Jul 13 14:00:58 ourserver freshclam[3818]: Update failed.
Jul 13 14:00:58 ourserver freshclam[3818]:
--------------------------------------

Hi there,

On Tue, 13 Jul 2021, Robert Kudyba wrote:

>> ... daily.cld was updated, presumably by freshclam. That's good, as
>> nothing seems to have broken. Can you confirm that happened from the
>> freshclam log?
>
> here are the logs from 10:01 AM Jul 13:
> Jul 13 10:01:02 storm freshclam[3930506]: Database test passed.
> Jul 13 10:01:02 storm freshclam[3930506]: daily.cld updated (version: 26230, sigs: 3995778, f-level: 63, builder: raynman)
> Jul 13 10:01:02 storm freshclam[3930506]: daily.cld updated (version: 26230, sigs: 3995778, f-level: 63, builder: raynman)
> ...
> ps -auwx|grep freshclam
> clamav 3818 0.0 0.0 28952 12864 ? Ss 12:00 0:00 /usr/bin/freshclam -d --foreground=true

The logs contain a lot of duplicated lines. Maybe you have both a
line like

StandardOutput=syslog

in your freshclam.service and *also* a line like

LogSyslog yes

in your freshclam.conf (or whatever passes for freshclam.conf in these
screwy RedHat systems). Well, you want one or the other but not both.
I'd suggest commenting out the "LogSyslog yes" line and restarting the
freshclam daemon.

>> Are you sure that the system time gets set correctly at boot? We
>> need to know that we can rely on the timestamps in the logs. ...
>
> ...
> Jul 13 12:00:50 ourserver.edu systemd[1]: Starting NTP client/server...
> Jul 13 12:01:34 ourserver.edu chronyd[3232]: Selected source 50.205.57.38 (2.fedora.pool.ntp.org)

Looks good.

>> Anyway, suddenly the owner/group IDs have changed and you have both a
>> daily.cld and a daily.cvd - which isn't good news, especially as one
>> of them is over three weeks old. Where did it come from?
>>
>
> Right, that's the question.

Looks like it was either the update or the reboot. An easy way to
find out would be to just reboot.

>> Assuming that we can believe the timestamps, then any problems that
>> arose from ownership by the clamupdate user/group had already happened
>> at 12:02 so it was *not* the run of clamav-unofficial-sigs.sh at 12:14
>> which caused them.
>>
>> Is this the first time that clamav-unofficial-sigs.sh ran?
>
> No it's been running all the time.

I think we're confusing each other. The clamav-unofficial-sigs.sh
script doesn't run like a daemon runs. The script is started by
something like a cron entry; it updates the configured databases if
needed, then stops. The unofficial update script only updates (or
should only update) the third-party signature database files, that is
everything except 'main', 'daily' and 'bytecode'. I meant was it at
12:14 that the clamav-unofficial-sigs.sh script ran? Presumably it's
logging its activities somewhere, do you have that log? The log
location will be set in the configuration for the unofficial script.

> So are freshclam and clamav-unofficial-sigs.sh not supposed to run
> as separate processes?

They are completely separate, they know nothing about each other and it
would be bad if they both tried to update the same files. The ClamaV
team provide freshclam as part of the ClamAV 'official' distribution.
The clamav-unofficial-sigs.sh is optional, and is provided separately.
There are completely separate configuration files for both utilities.

I'm beginning to wonder if this is all down to installation of the
unofficial update script with a configuration which assumes that the
user:group should be the same user:group as the clamd daemon and not
the same user:group as the freshclam daemon. The clamd daemon only
needs to read the dababase files but freshclam needs to write them.
I can sort of imagine somebody at RedHat thinking they'd make the
update process and the scanning processes use different UIDs & GIDs
for extra security, but it's not a lot of extra security, and it's
just asking for this kind of trouble when somebody installs another
utility which updates files in the same directory.

>> What's in the freshclam log about these times?
>
> Nothing as the upgrade/reboot was still happening. The next freshclam is:
> Jul 13 14:00:58 ourserver freshclam[3818]: Received signal: wake up
> Jul 13 14:00:58 ourserver freshclam[3818]: ClamAV update process started at Tue Jul 13 14:00:58 2021
> Jul 13 14:00:58 ourserver freshclam[3818]: Received signal: wake up
> Jul 13 14:00:58 ourserver freshclam[3818]: ClamAV update process started at Tue Jul 13 14:00:58 2021
> Jul 13 14:00:58 ourserver freshclam[3818]: ERROR: Can't create temporary directory /var/lib/clamav/tmp.21024dac47
> Jul 13 14:00:58 ourserver freshclam[3818]: Hint: The database directory must be writable for UID 985 or GID 981
> Jul 13 14:00:58 ourserver freshclam[3818]: ERROR: Update failed.
> Jul 13 14:00:58 ourserver freshclam[3818]: Can't create temporary directory /var/lib/clamav/tmp.21024dac47
> Jul 13 14:00:58 ourserver freshclam[3818]: Hint: The database directory must be writable for UID 985 or GID 981
> Jul 13 14:00:58 ourserver freshclam[3818]: Update failed.
> Jul 13 14:00:58 ourserver freshclam[3818]:

The "Hint:" part tells you what the problem is. Something changed the
permissions on the directory /var/lib/clamav/, freshclam can no longer
create the temporary directory it needs for the update. I guess the
unofficial update script disagrees with freshclam about the user and
group for the database directory. I believe freshclam is running as
user clamupdate, group clamupdate (983:979) but you have the ownership
on the directory set to user clamav, group clamav (985:981). Note
that the only things which need to be able to write to the directory
are freshclam and the unofficial update script. These two must be
configured to use the same user:group (or at least so that they can
both write into the directory). Find out where these things are
configured and set them the same. If RedHat updates are setting the
configuration of the freshclam daemon to use clamupdate:clamupdate it
would explain why this happens after an update. In that case it may
be better to settle on clamupdate:clamupdate for both freshclam and
the unofficial script. It's also possible that you have something in
the startup which is setting the directory user:group too, we'll take
a look at that later if need be.

After all this you may have to manually set the user/group IDs of
/var/lib/clamav/ and the files in there (you can do that with the
'chown' and 'chgrp' commands) and you'll need to restart the freshclam
daemon (again).

Finally when the dust has settled and you're happy that it isn't going
to break again you can stop the cron job logging the permissions. But
leave it running until you're sure. :)

HTH

--

73,
Ged.


_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

> > here are the logs from 10:01 AM Jul 13:
> > Jul 13 10:01:02 storm freshclam[3930506]: Database test passed.
> > Jul 13 10:01:02 storm freshclam[3930506]: daily.cld updated (version:
> 26230, sigs: 3995778, f-level: 63, builder: raynman)
> > Jul 13 10:01:02 storm freshclam[3930506]: daily.cld updated (version:
> 26230, sigs: 3995778, f-level: 63, builder: raynman)
> > ...
> > ps -auwx|grep freshclam
> > clamav 3818 0.0 0.0 28952 12864 ? Ss 12:00 0:00
> /usr/bin/freshclam -d --foreground=true
>
> The logs contain a lot of duplicated lines. Maybe you have both a
> line like
>
> StandardOutput=syslog
>
> in your freshclam.service and *also* a line like
>
> LogSyslog yes
>
> in your freshclam.conf (or whatever passes for freshclam.conf in these
> screwy RedHat systems). Well, you want one or the other but not both.
> I'd suggest commenting out the "LogSyslog yes" line and restarting the
> freshclam daemon.
>

I didn't have both set to yes but now I commented out LogSyslog and set
freshclam to log to its own log:
grep -v ^\# /etc/freshclam.conf | grep .
DatabaseDirectory /var/lib/clamav
UpdateLogFile /var/log/freshclam.log
DatabaseOwner clamav
DatabaseMirror database.clamav.net
ConnectTimeout 60
ReceiveTimeout 60
SafeBrowsing no

Side note I did have af few of these enabled but I believe the unofficial
sigs program has them.
#DatabaseCustomURL
http://www.securiteinfo.com/get/signatures/xxx/securiteinfo.hdb
#DatabaseCustomURL
http://www.securiteinfo.com/get/signatures/xxx/securiteinfo.ign2


> Looks like it was either the update or the reboot. An easy way to
> find out would be to just reboot.
>

Perhaps next week as we have summer session and some researchers logged in

>> Assuming that we can believe the timestamps, then any problems that
> >> arose from ownership by the clamupdate user/group had already happened
> >> at 12:02 so it was *not* the run of clamav-unofficial-sigs.sh at 12:14
> >> which caused them.
> >>
> >> Is this the first time that clamav-unofficial-sigs.sh ran?
> >
> > No it's been running all the time.
>
> I think we're confusing each other. The clamav-unofficial-sigs.sh
> script doesn't run like a daemon runs. The script is started by
> something like a cron entry; it updates the configured databases if
> needed, then stops. The unofficial update script only updates (or
> should only update) the third-party signature database files, that is
> everything except 'main', 'daily' and 'bytecode'. I meant was it at
> 12:14 that the clamav-unofficial-sigs.sh script ran? Presumably it's
> logging its activities somewhere, do you have that log? The log
> location will be set in the configuration for the unofficial script.
>

Uploaded at
https://storm.cis.fordham.edu/~rkudyba/clamav-unofficial-sigs.log


> Something changed the
>
permissions on the directory /var/lib/clamav/, freshclam can no longer
> create the temporary directory it needs for the update. I guess the
> unofficial update script disagrees with freshclam about the user and
> group for the database directory. I believe freshclam is running as
> user clamupdate, group clamupdate (983:979) but you have the ownership
> on the directory set to user clamav, group clamav (985:981).


Right which is why I also tried added user clamav to the clamupdate group.


> Note
> that the only things which need to be able to write to the directory
> are freshclam and the unofficial update script. These two must be
> configured to use the same user:group (or at least so that they can
> both write into the directory). Find out where these things are
> configured and set them the same.


grep -i owner /etc/freshclam.conf (comments removed)
DatabaseOwner clamav

grep -i user /etc/clamav-unofficial-sigs/user.conf
user_configuration_complete="yes"
clam_user="clamav"

grep -i user /etc/clamav-unofficial-sigs/user.conf
user_configuration_complete="yes"
clam_user="clamav"


> If RedHat updates are setting the
> configuration of the freshclam daemon to use clamupdate:clamupdate it
> would explain why this happens after an update.


AFAIK, you have to manually run something like rpmconf -a to do force a new
config file.


> In that case it may
> be better to settle on clamupdate:clamupdate for both freshclam and
> the unofficial script.


I'm starting to believe the same. This is how it's set on another server I
oversee and no issues.


> It's also possible that you have something in
> the startup which is setting the directory user:group too, we'll take
> a look at that later if need be.
>

I believe it's this file:
cat /usr/lib/systemd/system/clamav-freshclam.service
[Unit]
Description=ClamAV virus database updater
Documentation=man:freshclam(1) man:freshclam.conf(5)
https://www.clamav.net/documents
# If user wants it run from cron, don't start the daemon.
ConditionPathExists=!/etc/cron.d/clamav-update
Wants=network-online.target
After=network-online.target

[Service]
ExecStart=/usr/bin/freshclam -d --foreground=true

[Install]
WantedBy=multi-user.target


> After all this you may have to manually set the user/group IDs of
> /var/lib/clamav/ and the files in there (you can do that with the
> 'chown' and 'chgrp' commands) and you'll need to restart the freshclam
> daemon (again).
>

If/when this happens again I may just do that.

Finally when the dust has settled and you're happy that it isn't going
> to break again you can stop the cron job logging the permissions. But
> leave it running until you're sure.
>

noted.

Hi there,

On Thu, 15 Jul 2021, Robert Kudyba wrote:

> I didn't have both set to yes but now I commented out LogSyslog and set
> freshclam to log to its own log:
> grep -v ^\# /etc/freshclam.conf | grep .
> DatabaseDirectory /var/lib/clamav
> UpdateLogFile /var/log/freshclam.log
> DatabaseOwner clamav
> DatabaseMirror database.clamav.net
> ConnectTimeout 60
> ReceiveTimeout 60

See my earlier mail to the list today about the freshclam timeout,
60 seconds might be too short for the full file downloads. Well,
it is on our pathetic British Telecom connection.

>> ... do you have that log?
>
> Uploaded at ...

Nothing remarkable there. Presumably you're aware of this warning
in that log?

8<----------------------------------------------------------------------
WARNING: Failed connection to https://urlhaus.abuse.ch/downloads ...
8<----------------------------------------------------------------------

>> Something changed the permissions on the directory /var/lib/clamav/...
>
> Right which is why I also tried added user clamav to the clamupdate group.

If you rely on group write permission you'll need to make the
directory writeable by members of the group:

chmod g+w /var/lib/clamav/

>> ... it may be better to settle on clamupdate:clamupdate for both
>> freshclam and the unofficial script.
>
> I'm starting to believe the same. This is how it's set on another server I
> oversee and no issues.

If it's the same OS distribution you should be able to compare the
configurations, see what they both put in the logs etc. The command

clamconf -n

would be very useful for that but there are other configs as well.

>> It's also possible that you have something in the startup which is
>> setting the directory user:group too, we'll take a look at that
>> later if need be.
>
> I believe it's this file:
> cat /usr/lib/systemd/system/clamav-freshclam.service
> [Unit]
> Description=ClamAV virus database updater
> Documentation=man:freshclam(1) man:freshclam.conf(5)
> https://www.clamav.net/documents
> # If user wants it run from cron, don't start the daemon.
> ConditionPathExists=!/etc/cron.d/clamav-update
> Wants=network-online.target
> After=network-online.target
>
> [Service]
> ExecStart=/usr/bin/freshclam -d --foreground=true
>
> [Install]
> WantedBy=multi-user.target

Nothing there sets the user:group, I guess it's relying on the
freshclam configuration file.

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

>
>
> On Thu, 15 Jul 2021, Robert Kudyba wrote:
>

Here we are Aug 24


> >> ... do you have that log?
> >
> > Uploaded at ...
>
> Nothing remarkable there. Presumably you're aware of this warning
> in that log?
>

See https://storm.cis.fordham.edu/~rkudyba/aug24

At 5:14 AM the problem started happening and cron has:

Aug 24 05:14:01 storm CROND[537748]: (clamav) CMD ([ -x
/usr/local/sbin/clamav-unofficial-sigs.sh ] && /usr/bin/bash
/usr/local/sbin/clamav-unofficial-sigs.sh)

Aug 24 05:14:03 storm CROND[537718]: (clamav) CMDEND ([ -x
/usr/local/sbin/clamav-unofficial-sigs.sh ] && /usr/bin/bash
/usr/local/sbin/clamav-unofficial-sigs.sh)
Aug 24 05:15:01 storm CROND[538116]: (root) CMD (/bin/date >> $FILE ;
/bin/ls -l /var/lib/clamav >> $FILE)

>
> If it's the same OS distribution you should be able to compare the
> configurations, see what they both put in the logs etc. The command
>
> clamconf -n
>
> would be very useful for that but there are other configs as well.
>

clamconf -n

Checking configuration files in /etc


Config file: clamd.d/scan.conf

------------------------------

LogFile = "/var/log/clamd.log"

TCPSocket = "3310"

TCPAddr = "127.0.0.1"

User = "clamav"

PhishingScanURLs disabled

HeuristicScanPrecedence = "yes"

AlertBrokenExecutables = "yes"

AlertBrokenMedia = "yes"

AlertEncrypted = "yes"

AlertEncryptedArchive = "yes"

AlertEncryptedDoc = "yes"

AlertOLE2Macros = "yes"

AlertPhishingSSLMismatch = "yes"

AlertPartitionIntersection = "yes"

MaxScanTime = "350000"

MaxScanSize = "157286400"

MaxFileSize = "31457280"


Config file: freshclam.conf

---------------------------

LogFileMaxSize = "262144000"

LogRotate = "yes"

UpdateLogFile = "/var/log/freshclam.log"

DatabaseOwner = "clamav"

DatabaseMirror = "database.clamav.net"

ConnectTimeout = "60"

ReceiveTimeout = "60"


Config file: mail/clamav-milter.conf

------------------------------------

LogFile = "/var/log/clamav-milter.log"

LogTime = "yes"

LogVerbose = "yes"

User = "clamilt"

ClamdSocket = "tcp:127.0.0.1:3310"

MilterSocket = "inet:6666"

AddHeader = "Add"

Whitelist = "/etc/mail/clamav-milter-whitelist.conf"


Software settings

-----------------

Version: 0.103.3

Optional features supported: MEMPOOL IPv6 AUTOIT_EA06 BZIP2 LIBXML2 PCRE2
ICONV JSON


Database information

--------------------

Database directory: /var/lib/clamav

[3rd Party] badmacro.ndb: 621 sigs

[3rd Party] shelter.ldb: 49 sigs

[3rd Party] CVE-2013-0074.yar: 22 sigs

[3rd Party] foxhole_js.cdb: 48 sigs

[3rd Party] rfxn.yara: 11527 sigs

[3rd Party] urlhaus.ndb: 5445 sigs

[3rd Party] malware.expert.ndb: 1 sig

[3rd Party] sanesecurity.ftm: 170 sigs

[3rd Party] CVE-2013-0422.yar: 25 sigs

[3rd Party] sigwhitelist.ign2: 12 sigs

[3rd Party] junk.ndb: 55801 sigs

[3rd Party] jurlbl.ndb: 5650 sigs

[3rd Party] phish.ndb: 28047 sigs

[3rd Party] rogue.hdb: 1005 sigs

[3rd Party] scam.ndb: 12747 sigs

[3rd Party] spamimg.hdb: 200 sigs

[3rd Party] CVE-2015-1701.yar: 30 sigs

[3rd Party] spamattach.hdb: 14 sigs

[3rd Party] blurl.ndb: 2194 sigs

[3rd Party] CVE-2015-2426.yar: 49 sigs

[3rd Party] malwarehash.hsb: 771 sigs

[3rd Party] CVE-2015-2545.yar: 76 sigs

[3rd Party] foxhole_generic.cdb: 212 sigs

[3rd Party] CVE-2015-5119.yar: 22 sigs

[3rd Party] foxhole_filename.cdb: 2612 sigs

[3rd Party] CVE-2016-5195.yar: 40 sigs

[3rd Party] winnow_malware.hdb: 293 sigs

[3rd Party] winnow_extended_malware_links.ndb: 1 sig

[3rd Party] winnow_malware_links.ndb: 133 sigs

[3rd Party] MiscreantPunch099-Low.ldb: 1199 sigs

[3rd Party] winnow_extended_malware.hdb: 245 sigs

[3rd Party] safebrowsing.gdb: 49126 sigs

[3rd Party] winnow.attachments.hdb: 182 sigs

[3rd Party] CVE-2017-11882.yar: 66 sigs

[3rd Party] winnow_bad_cw.hdb: 1 sig

[3rd Party] EK_BleedingLife.yar: 112 sigs

[3rd Party] bofhland_cracked_URL.ndb: 40 sigs

[3rd Party] WShell_ASPXSpy.yar: 21 sigs

[3rd Party] bofhland_malware_URL.ndb: 4 sigs

[3rd Party] WShell_Drupalgeddon2_icos.yar: 26 sigs

[3rd Party] bofhland_phishing_URL.ndb: 72 sigs

[3rd Party] CVE-2010-0805.yar: 19 sigs

[3rd Party] bofhland_malware_attach.hdb: 1836 sigs

[3rd Party] CVE-2018-20250.yar: 22 sigs

[3rd Party] hackingteam.hsb: 435 sigs

[3rd Party] CVE-2018-4878.yar: 39 sigs

[3rd Party] porcupine.ndb: 6622 sigs

[3rd Party] bank_rule.yar: 11 sigs

[3rd Party] phishtank.ndb: 9388 sigs

[3rd Party] EMAIL_Cryptowall.yar: 52 sigs

[3rd Party] porcupine.hsb: 208 sigs

[3rd Party] scam.yar: 35 sigs

[3rd Party] securiteinfo.ign2: 86 sigs

[3rd Party] JJencode.yar: 19 sigs

[3rd Party] securiteinfo.hdb: 159918 sigs

[3rd Party] interserver256.hdb: 3626 sigs

[3rd Party] securiteinfoold.hdb: 3525608 sigs

[3rd Party] interservertopline.db: 161 sigs

[3rd Party] javascript.ndb: 43708 sigs

main.cvd: version 61, sigs: 6607162, built on Wed Jul 14 22:39:10 2021

[3rd Party] securiteinfohtml.hdb: 55106 sigs

[3rd Party] CVE-2010-0887.yar: 22 sigs

[3rd Party] securiteinfoascii.hdb: 98410 sigs

daily.cld: version 26272, sigs: 1968128, built on Mon Aug 23 04:21:13 2021

[3rd Party] securiteinfopdf.hdb: 3408 sigs

[3rd Party] CVE-2010-1297.yar: 20 sigs

[3rd Party] securiteinfoandroid.hdb: 84401 sigs

[3rd Party] rfxn.ndb: 2039 sigs

[3rd Party] rfxn.hdb: 12932 sigs

daily.cvd: version 26209, sigs: 3992031, built on Tue Jun 22 07:07:55 2021

[3rd Party] malware.expert.hdb: 1 sig

[3rd Party] malware.expert.ldb: 1 sig

[3rd Party] foxhole_js.ndb: 4 sigs

[3rd Party] CVE-2012-0158.yar: 27 sigs

[3rd Party] winnow_spam_complete.ndb: 26 sigs

[3rd Party] whitelist.fp: 3081 sigs

[3rd Party] winnow.complex.patterns.ldb: 3 sigs

[3rd Party] Sanesecurity_spam.yara: 46 sigs

[3rd Party] jurlbla.ndb: 1388 sigs

[3rd Party] lott.ndb: 2335 sigs

[3rd Party] spam.ldb: 2 sigs

[3rd Party] spear.ndb: 1 sig

[3rd Party] spearl.ndb: 1 sig

[3rd Party] malware.expert.fp: 1 sig

[3rd Party] scamnailer.ndb: 1 sig

bytecode.cvd: version 333, sigs: 92, built on Mon Mar 8 10:21:51 2021

[3rd Party] winnow_phish_complete_url.ndb: 54 sigs

[3rd Party] malwarepatrol.db: 9180 sigs

[3rd Party] Sanesecurity_sigtest.yara: 54 sigs

[3rd Party] email_Ukraine_BE_powerattack.yar: 33 sigs

[3rd Party] Email_fake_it_maintenance_bulletin.yar: 29 sigs

[3rd Party] Email_quota_limit_warning.yar: 31 sigs

Total number of signatures: 16770754


Platform information

--------------------

uname: Linux 5.12.14-300.fc34.x86_64 #1 SMP Wed Jun 30 18:30:21 UTC 2021
x86_64

OS: linux-gnu, ARCH: x86_64, CPU: x86_64

zlib version: 1.2.11 (1.2.11), compile flags: a9

platform id: 0x0a217c7c08000000020b0201


Build information

-----------------

GNU C: 11.2.1 20210728 (Red Hat 11.2.1-1) (11.2.1)

CPPFLAGS: -I/usr/include/libprelude

CFLAGS: -O2 -flto=auto -ffat-lto-objects -fexceptions -g
-grecord-gcc-switches -pipe -Wall -Werror=format-security
-Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS
-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
-fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
-D_LARGEFILE_SOURCE
-D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64

CXXFLAGS: -O2 -flto=auto -ffat-lto-objects -fexceptions -g
-grecord-gcc-switches -pipe -Wall -Werror=format-security
-Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS
-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
-fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection

LDFLAGS: -Wl,-z,relro -Wl,--as-needed -Wl,-z,now
-specs=/usr/lib/rpm/redhat/redhat-hardened-ld -lprelude

Configure: '--build=x86_64-redhat-linux-gnu'
'--host=x86_64-redhat-linux-gnu' '--program-prefix='
'--disable-dependency-tracking' '--prefix=/usr' '--exec-prefix=/usr'
'--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc'
'--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64'
'--libexecdir=/usr/libexec' '--localstatedir=/var'
'--sharedstatedir=/var/lib' '--mandir=/usr/share/man'
'--infodir=/usr/share/info' '--enable-milter' '--disable-clamav'
'--disable-static' '--disable-zlib-vcheck' '--disable-unrar'
'--enable-id-check' '--enable-dns' '--with-dbdir=/var/lib/clamav'
'--with-group=clamupdate' '--with-user=clamupdate' '--disable-rpath'
'--disable-silent-rules' '--enable-clamdtop' '--enable-prelude'
'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu'
'CXX=g++' 'CXXFLAGS=-O2 -flto=auto -ffat-lto-objects -fexceptions -g
-grecord-gcc-switches -pipe -Wall -Werror=format-security
-Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS
-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
-fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection'
'LDFLAGS=-Wl,-z,relro -Wl,--as-needed -Wl,-z,now
-specs=/usr/lib/rpm/redhat/redhat-hardened-ld' 'CC=gcc' 'CFLAGS=-O2
-flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe
-Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2
-Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
-fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64
-mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection
-fcf-protection' 'LT_SYS_LIBRARY_PATH=/usr/lib64:'
'PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig'

sizeof(void*) = 8
Engine flevel: 124, dconf: 124

>
>