Mailing List Archive

On Wed, 24 May 2023, Tachibanaki Nozomi ($B5LLZ(B $B4uH~(B) wrote:

> Dear Sir or Madam,
>
> Thank you for your help always.
> I am contacting you to ask about MaxFileSize in clamd.conf.
>
> The following description is found in the configuration of
> /usr/local/etc/clamd.conf.
>
> MaxFileSize
> # Technical design limitations prevent ClamAV from scanning files greater than
> # 2 GB at this time.
>
> Is there any plan or possibility to change the technical design
> limitation that prevents scanning files larger than 2 GB in the
> future?

I believe that the intention is to remove this limit at some point.

I wonder whether the technical limitations are less severe for
archive formats such as tar and zip.
Could "small" files inside "large" archives be scanned
without the work necessary for full "large" file support ?

Apart from vulnerabilities caused by 2GB and 4GB limits themselves,
I think scanning inside large archives might solve many of the
reasons for scanning large files.

--
Andrew C. Aitchison Kendal, UK
andrew@aitchison.me.uk
_______________________________________________

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

I agree with you. I suspect the majority of cases today is when people have a large archive of files to scan.

I think best case scenario for people with a need to scan files larger than the present internal 2GB limit is that archives larger than 2GB are decompressed and then the files inside are scanned, but without actually scanning the very large outer archive.

The way to do this as things work today is to script something around clamscan or clamdscan that if the file is too large, handle some assorted file types:

1. if file is a tar.gz, un-tar.gz it and then scan the files within.
2. if file is a zip, un-zip it and then scan the files within.
3. etc.

I think everyone would like if clamav could do this automatically for select archive types. And I think the advantage would be that we would perhaps keep the extracted files in memory, or else at least delete the temp files as we go without extracting all of it to disk before starting to scan.

However, it would be far easier to make a shell script or a python script that wraps clamscan/clamdscan and uses native tools like "tar", "unzip", etc.

Regards,
Micah


Micah Snyder
ClamAV Development
Talos
Cisco Systems, Inc.
________________________________
From: clamav-users <clamav-users-bounces@lists.clamav.net> on behalf of Andrew C Aitchison via clamav-users <clamav-users@lists.clamav.net>
Sent: Wednesday, May 24, 2023 1:34 AM
To: ClamAV users ML <clamav-users@lists.clamav.net>
Cc: Andrew C Aitchison <clamav@aitchison.me.uk>
Subject: Re: [clamav-users] Question About MaxFileSize

On Wed, 24 May 2023, Tachibanaki Nozomi ($B5LLZ(B $B4uH~(B) wrote:

> Dear Sir or Madam,
>
> Thank you for your help always.
> I am contacting you to ask about MaxFileSize in clamd.conf.
>
> The following description is found in the configuration of
> /usr/local/etc/clamd.conf.
>
> MaxFileSize
> # Technical design limitations prevent ClamAV from scanning files greater than
> # 2 GB at this time.
>
> Is there any plan or possibility to change the technical design
> limitation that prevents scanning files larger than 2 GB in the
> future?

I believe that the intention is to remove this limit at some point.

I wonder whether the technical limitations are less severe for
archive formats such as tar and zip.
Could "small" files inside "large" archives be scanned
without the work necessary for full "large" file support ?

Apart from vulnerabilities caused by 2GB and 4GB limits themselves,
I think scanning inside large archives might solve many of the
reasons for scanning large files.

--
Andrew C. Aitchison Kendal, UK
andrew@aitchison.me.uk
_______________________________________________

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

On Thu, 8 Jun 2023, Micah Snyder (micasnyd) wrote:

> I agree with you. I suspect the majority of cases today is when
> people have a large archive of files to scan.
>
> I think best case scenario for people with a need to scan files
> larger than the present internal 2GB limit is that archives larger
> than 2GB are decompressed and then the files inside are scanned, but
> without actually scanning the very large outer archive.
>
> The way to do this as things work today is to script something
> around clamscan or clamdscan that if the file is too large, handle
> some assorted file types:
>
> 1. if file is a tar.gz, un-tar.gz it and then scan the files within.
> 2. if file is a zip, un-zip it and then scan the files within.
> 3. etc.
>
> I think everyone would like if clamav could do this automatically
> for select archive types. And I think the advantage would be that we
> would perhaps keep the extracted files in memory, or else at least
> delete the temp files as we go without extracting all of it to disk
> before starting to scan.
>
> However, it would be far easier to make a shell script or a python
> script that wraps clamscan/clamdscan and uses native tools like
> "tar", "unzip", etc.

Good idea.

Simply untarring or unzipping into a pipe does not separate the packed files.
However at least tar does have an option which allow us to write a one-liner:
(tar xf ~/viruses.tar --to-command='clamdscan -v - || echo " found in $TAR_REALNAME\n\n---"' ) |& egrep -i found
stream: Eicar-Signature FOUND
found in viruses/EICAR.COM.TAR
stream: Eicar-Signature FOUND
found in viruses/eicar.com.txt
stream: Eicar-Signature FOUND
found in viruses/URLEICAR.COM.TAR
stream: Eicar-Signature FOUND
found in viruses/4DOSBOX/EICAR.COM
stream: Eicar-Signature FOUND
found in viruses/EICAR.COM

The echo is needed to show the name of the file inside the archive.

This appears not to write the unpacked files to disk.

--
Andrew C. Aitchison Kendal, UK
andrew@aitchison.me.uk
_______________________________________________

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

I must say I strongly disagree with the approach of feeding files contained in a big archive file one at a time to ClamAV. That's because an archive is *itself* a file.

I have on occasion heard of vulnerabilities in some archiving software, where the mere act of decompressing and extracting an archive can result in malicious code execution due to a bug in the archiving software. After all, such software can itself have the all too common lack of bounds checking (etc.) that could be exploited by a maliciously malformed archive.

It could also be that lower level archive-like files such as ISOs and disk images could, by means of malicious structuring, trigger a total system compromise, because it might well involve the kernel. The way an ISO or disk image is typically used (on Linux, at least) is to create a "loop" device from the file, and then *mount* it as block device -- a clear kernel involvement.

Of course, scanning any file might conceivably trigger a ClamAV bug, and thus a compromise, but that is no reason to add another layer of vulnerability to things. (But it is a good reason not to run ClamAV as root.)

Paul Kosinski



On Thu, 8 Jun 2023 20:55:25 +0000
"Micah Snyder \(micasnyd\) via clamav-users" <clamav-users@lists.clamav.net> wrote:

> I agree with you. I suspect the majority of cases today is when people have a large archive of files to scan.
>
> I think best case scenario for people with a need to scan files larger than the present internal 2GB limit is that archives larger than 2GB are decompressed and then the files inside are scanned, but without actually scanning the very large outer archive.
>
> The way to do this as things work today is to script something around clamscan or clamdscan that if the file is too large, handle some assorted file types:
>
> 1. if file is a tar.gz, un-tar.gz it and then scan the files within.
> 2. if file is a zip, un-zip it and then scan the files within.
> 3. etc.
>
> I think everyone would like if clamav could do this automatically for select archive types. And I think the advantage would be that we would perhaps keep the extracted files in memory, or else at least delete the temp files as we go without extracting all of it to disk before starting to scan.
>
> However, it would be far easier to make a shell script or a python script that wraps clamscan/clamdscan and uses native tools like "tar", "unzip", etc.
>
> Regards,
> Micah
>
>
> Micah Snyder
> ClamAV Development
> Talos
> Cisco Systems, Inc.
_______________________________________________

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

--On Friday, June 09, 2023 6:40 PM -0400 Paul Kosinski via clamav-users
<clamav-users@lists.clamav.net> wrote:

> I have on occasion heard of vulnerabilities in some archiving software,
> where the mere act of decompressing and extracting an archive can result
> in malicious code execution due to a bug in the archiving software. After
> all, such software can itself have the all too common lack of bounds
> checking (etc.) that could be exploited by a maliciously malformed
> archive.
>
> It could also be that lower level archive-like files such as ISOs and
> disk images could, by means of malicious structuring, trigger a total
> system compromise, because it might well involve the kernel. The way an
> ISO or disk image is typically used (on Linux, at least) is to create a
> "loop" device from the file, and then *mount* it as block device -- a
> clear kernel involvement.

Filesystems are also files, interpreted by kernel-level filesystem drivers.
Some filesystems have a compression feature. Scanning ANY file exercises
such code.

> Of course, scanning any file might conceivably trigger a ClamAV bug, and
> thus a compromise, but that is no reason to add another layer of
> vulnerability to things. (But it is a good reason not to run ClamAV as
> root.)

This is also a good reason to run it as a service in a sandbox with minimal
capabilities. The client application (like a mail server) can feed the file
to scan through a socket and rely on the service's sandbox to protect the
client application.

_______________________________________________

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat

You are right. But more than that, merely *reading* a file will exercise such code. I wonder if anybody has devised a file which exploits such a kernel bug? (Shudder.)

After I wrote my objection, I realized that to be even more safe, one should scan removable disks at the block level before mounting them. But given the capacity these days of even USB thumb drives, this approach is pretty much impractical. Beside, what looks like a USB thumb drive might actually act as a USB keyboard! (In fact, I think somebody built a prototype.)


On Fri, 09 Jun 2023 18:15:39 -0700
Kenneth Porter <shiva@sewingwitch.com> wrote:

> Filesystems are also files, interpreted by kernel-level filesystem drivers.
> Some filesystems have a compression feature. Scanning ANY file exercises
> such code.
_______________________________________________

Manage your clamav-users mailing list subscription / unsubscribe:
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/Cisco-Talos/clamav-documentation

https://docs.clamav.net/#mailing-lists-and-chat