Mailing List Archive

Use "-o" instead of "--infected" and then you'll proably want to filter out
any messages that end in ": Empty file" or ": Symbolic link".

Jim

On Thu, Mar 11, 2021 at 12:28:49PM +0100, Ben Stuyts wrote:
> Hello,
>
> I’m running a daily clamscan on my home directories, and lately I get this error message:
>
> LibClamAV Warning: fmap: failed to get MD5
> LibClamAV Error: CRITICAL: fmap() failed
>
> I can’t figure out what file clamav is having trouble with. The command I use is:
>
> /usr/bin/find [...] -print0 | xargs -0 /usr/local/bin/clamscan --infected --no-summary --official-db-only=yes
>
> Versions:
>
> Clam AntiVirus: Scanner 0.103.1
> Running on FreeBSD 12.2-RELEASE-p3 GENERIC amd64
>
> Any ideas?
>
> Ben
>
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Thanks Jim, I’ll try this.

Ben


> On 11 Mar 2021, at 13:10, Jim Simmons via clamav-users <clamav-users@lists.clamav.net> wrote:
>
> Use "-o" instead of "--infected" and then you'll proably want to filter out
> any messages that end in ": Empty file" or ": Symbolic link".
>
> Jim
>
> On Thu, Mar 11, 2021 at 12:28:49PM +0100, Ben Stuyts wrote:
>> Hello,
>>
>> I’m running a daily clamscan on my home directories, and lately I get this error message:
>>
>> LibClamAV Warning: fmap: failed to get MD5
>> LibClamAV Error: CRITICAL: fmap() failed
>>
>> I can’t figure out what file clamav is having trouble with. The command I use is:
>>
>> /usr/bin/find [...] -print0 | xargs -0 /usr/local/bin/clamscan --infected --no-summary --official-db-only=yes
>>
>> Versions:
>>
>> Clam AntiVirus: Scanner 0.103.1
>> Running on FreeBSD 12.2-RELEASE-p3 GENERIC amd64
>>
>> Any ideas?
>>
>> Ben
>>
>>
>> _______________________________________________
>>
>> clamav-users mailing list
>> clamav-users@lists.clamav.net
>> https://lists.clamav.net/mailman/listinfo/clamav-users
>>
>>
>> Help us build a comprehensive ClamAV guide:
>> https://github.com/vrtadmin/clamav-faq
>>
>> http://www.clamav.net/contact.html#ml
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>


_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Hello,

I found a file that is causing this:

# clamscan a
LibClamAV Warning: fmap: failed to get MD5
LibClamAV Error: CRITICAL: fmap() failed
/tmp/a: Can't allocate memory ERROR

top says:
Mem: 2393M Active, 4926M Inact, 152K Laundry, 8602M Wired, 1990M Free
Swap: 16G Total, 1892M Used, 14G Free, 11% Inuse

File ‘a’ is a 4.1 GB mbox file. This is a 64-bit FreeBSD system. So is ClamAV trying to mmap that whole file into memory? I can’t find any options to change this behaviour.

There are some bugzilla reports here, but it’s still open:
https://bugzilla.clamav.net/show_bug.cgi?id=10971 <https://bugzilla.clamav.net/show_bug.cgi?id=10971>
https://bugzilla.clamav.net/show_bug.cgi?id=12661 <https://bugzilla.clamav.net/show_bug.cgi?id=12661>

Ben


> On 11 Mar 2021, at 13:48, Ben Stuyts <ben@altesco.nl> wrote:
>
> Thanks Jim, I’ll try this.
>
> Ben
>
>
>> On 11 Mar 2021, at 13:10, Jim Simmons via clamav-users <clamav-users@lists.clamav.net> wrote:
>>
>> Use "-o" instead of "--infected" and then you'll proably want to filter out
>> any messages that end in ": Empty file" or ": Symbolic link".
>>
>> Jim
>>
>> On Thu, Mar 11, 2021 at 12:28:49PM +0100, Ben Stuyts wrote:
>>> Hello,
>>>
>>> I’m running a daily clamscan on my home directories, and lately I get this error message:
>>>
>>> LibClamAV Warning: fmap: failed to get MD5
>>> LibClamAV Error: CRITICAL: fmap() failed
>>>
>>> I can’t figure out what file clamav is having trouble with. The command I use is:
>>>
>>> /usr/bin/find [...] -print0 | xargs -0 /usr/local/bin/clamscan --infected --no-summary --official-db-only=yes
>>>
>>> Versions:
>>>
>>> Clam AntiVirus: Scanner 0.103.1
>>> Running on FreeBSD 12.2-RELEASE-p3 GENERIC amd64
>>>
>>> Any ideas?
>>>
>>> Ben
>>>
>>>
>>> _______________________________________________
>>>
>>> clamav-users mailing list
>>> clamav-users@lists.clamav.net
>>> https://lists.clamav.net/mailman/listinfo/clamav-users
>>>
>>>
>>> Help us build a comprehensive ClamAV guide:
>>> https://github.com/vrtadmin/clamav-faq
>>>
>>> http://www.clamav.net/contact.html#ml
>>
>> _______________________________________________
>>
>> clamav-users mailing list
>> clamav-users@lists.clamav.net
>> https://lists.clamav.net/mailman/listinfo/clamav-users
>>
>>
>> Help us build a comprehensive ClamAV guide:
>> https://github.com/vrtadmin/clamav-faq
>>
>> http://www.clamav.net/contact.html#ml
>>
>
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>

Hi there,

On Wed, 24 Mar 2021, Ben Stuyts wrote:

> ... File ‘a’ is a 4.1 GB mbox file. ...

Then don't scan it:

https://marc.info/?l=clamav-users&m=157549176313237&w=2

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

> On 24 Mar 2021, at 12:14, G.W. Haywood via clamav-users <clamav-users@lists.clamav.net> wrote:
>
> Hi there,
>
> On Wed, 24 Mar 2021, Ben Stuyts wrote:
>
>> ... File ‘a’ is a 4.1 GB mbox file. ...
>
> Then don't scan it:
>
> https://marc.info/?l=clamav-users&m=157549176313237&w=2 <https://marc.info/?l=clamav-users&m=157549176313237&w=2>

Possible, but not the first thing that comes to my mind with email folders.

But thanks for the link, it sheds some light on the underlying problem.

Ben

Hi there,

On Wed, 24 Mar 2021, Ben Stuyts wrote:
> On 24 Mar 2021, at 12:14, G.W. Haywood via clamav-users <clamav-users@lists.clamav.net> wrote:
>> On Wed, 24 Mar 2021, Ben Stuyts wrote:
>>
>>> ... File ‘a’ is a 4.1 GB mbox file. ...
>>
>> Then don't scan it...
>
> Possible, but not the first thing that comes to my mind with email folders.

You could scan what you put into it.

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

> On 25 Mar 2021, at 00:32, G.W. Haywood via clamav-users <clamav-users@lists.clamav.net> wrote:
>
> Hi there,
>
> On Wed, 24 Mar 2021, Ben Stuyts wrote:
>> On 24 Mar 2021, at 12:14, G.W. Haywood via clamav-users <clamav-users@lists.clamav.net> wrote:
>>> On Wed, 24 Mar 2021, Ben Stuyts wrote:
>>>
>>>> ... File ‘a’ is a 4.1 GB mbox file. ...
>>>
>>> Then don't scan it...
>>
>> Possible, but not the first thing that comes to my mind with email folders.
>
> You could scan what you put into it.

We do of course, using clamav-milter. But if there’s a missing virus definition during mail transfer it will be hopefully detected at a later stage.

Ben


_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Hi there,

On Thu, 25 Mar 2021, Ben Stuyts wrote:
>> On 25 Mar 2021, at 00:32, G.W. Haywood via clamav-users <clamav-users@lists.clamav.net> wrote:
>> On Wed, 24 Mar 2021, Ben Stuyts wrote:
>>> On 24 Mar 2021, at 12:14, G.W. Haywood via clamav-users <clamav-users@lists.clamav.net> wrote:
>>>> On Wed, 24 Mar 2021, Ben Stuyts wrote:
>>>>
>>>>> ... File ?a? is a 4.1 GB mbox file. ...
>>>>
>>>> Then don't scan it...
>>>
>>> Possible, but not the first thing that comes to my mind with email folders.
>>
>> You could scan what you put into it.
>
> We do of course, using clamav-milter. But if there?s a missing
> virus definition during mail transfer it will be hopefully detected
> at a later stage.

Emphasis on hopefully, and something about a stable door, and do you
have working estimates of the probabilities? And what do you do if,
when you scan the huge mbox file, ClamAV says it's found something?
It won't tell you which message is the suspicious one, so you'll be
playing about with 'formail' or binary searches or whatever, all the
while wondering who's at risk from this potentially troublesome but
unidentified message. And the detection might not even be triggered
when you split up the messages. You could waste a *lot* of time and
energy that way.

Of course you know that there are alternatives to mbox format, which
not only won't involve you in scanning outlandishly big files but also
give you substantial improvements in performance elsewhere, not to
mention giving clamd's cache of MD5 digests a chance to do something
useful (instead of what it's doing for you now - burning CPU cycles to
no purpose whatsoever). Another added benefit might be that you could
get the scanner to actually identify the suspicious message...

Not that any of this is a recommendation, other than that people think
real hard about what they're doing.

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

On 25 Mar 2021, at 19:38, G.W. Haywood via clamav-users <clamav-users@lists.clamav.net> wrote:
>
> Hi there,
>
> On Thu, 25 Mar 2021, Ben Stuyts wrote:
>>> On 25 Mar 2021, at 00:32, G.W. Haywood via clamav-users <clamav-users@lists.clamav.net> wrote:
>>> On Wed, 24 Mar 2021, Ben Stuyts wrote:
>>>> On 24 Mar 2021, at 12:14, G.W. Haywood via clamav-users <clamav-users@lists.clamav.net> wrote:
>>>>> On Wed, 24 Mar 2021, Ben Stuyts wrote:
>>>>>
>>>>>> ... File ‘a’ is a 4.1 GB mbox file. ...
>>>>>
>>>>> Then don't scan it...
>>>>
>>>> Possible, but not the first thing that comes to my mind with email folders.
>>>
>>> You could scan what you put into it.
>> We do of course, using clamav-milter. But if there’s a missing
>> virus definition during mail transfer it will be hopefully detected
>> at a later stage.
>
> Emphasis on hopefully, and something about a stable door, and do you
> have working estimates of the probabilities?

It happens in around 1 in 100000-200000 delivered msgs.

> And what do you do if,
> when you scan the huge mbox file, ClamAV says it's found something?

I never said that the virus detected is always in a huge mbox file.

> It won't tell you which message is the suspicious one, so you'll be
> playing about with 'formail' or binary searches or whatever, all the
> while wondering who's at risk from this potentially troublesome but
> unidentified message. And the detection might not even be triggered
> when you split up the messages. You could waste a *lot* of time and
> energy that way.

Indeed, and this is all known here. So there’s a script for that. I have not seen the problems you mention.

> Of course you know that there are alternatives to mbox format, which
> not only won't involve you in scanning outlandishly big files but also
> give you substantial improvements in performance elsewhere, not to
> mention giving clamd's cache of MD5 digests a chance to do something
> useful (instead of what it's doing for you now - burning CPU cycles to
> no purpose whatsoever). Another added benefit might be that you could
> get the scanner to actually identify the suspicious message…

Agreed that it is not optimal in that respect, but changing the mail config is not really an option.

> Not that any of this is a recommendation, other than that people think
> real hard about what they're doing.

We try to, thank you.

Ben


_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml