Mailing List Archive

CalmAV In-Memory Scan
Hi,
I noticed that when using the INSTREAM command and sending it a memory
buffer of a file, clamd takes the memory buffer and saves it to
the TemporaryDirectory (as defined in the config file).

This is an unnecessary overhead as it requires disk IO in order to scan the
file which is already loaded in-memory.
Is there any way to command clamd to scan the buffer completely in-memory
without writing anything to the disk?

Thanks,
Michael.
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

http://www.clamav.net/contact.html#ml
Re: CalmAV In-Memory Scan [ In reply to ]
> On Apr 5, 2017, at 2:27 PM, Michael Engstler <michael@cybellum.com> wrote:
>
> Hi,
> I noticed that when using the INSTREAM command and sending it a memory
> buffer of a file, clamd takes the memory buffer and saves it to
> the TemporaryDirectory (as defined in the config file).
>
> This is an unnecessary overhead as it requires disk IO in order to scan the
> file which is already loaded in-memory.
> Is there any way to command clamd to scan the buffer completely in-memory
> without writing anything to the disk?

My understanding is that ClamAV requires a rewindable file stream in order to perform scans.

>
> Thanks,
> Michael.
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>
> http://www.clamav.net/contact.html#ml
Re: CalmAV In-Memory Scan [ In reply to ]
On 04/05/2017 09:27 PM, Michael Engstler wrote:
> Hi,
> I noticed that when using the INSTREAM command and sending it a memory
> buffer of a file, clamd takes the memory buffer and saves it to
> the TemporaryDirectory (as defined in the config file).
>
> This is an unnecessary overhead as it requires disk IO in order to scan the
> file which is already loaded in-memory.
> Is there any way to command clamd to scan the buffer completely in-memory
> without writing anything to the disk?

That's a feature that I requested many years ago because I had a non-copying
MIME parser that worked on mmap()ed files, and I wanted to virus-scan some parts
of the message. It was not possible, and I think it's not possible now.

I suggest you to mmap() your memory region to a temp. file, ideally on tmpfs,
and pass this file to clamav. That ought to minimize the overhead.

Eugene
Re: CalmAV In-Memory Scan [ In reply to ]
Hi,
Thanks for the quick response.
Your suggestion sounds interesting, but from what I've seen if you give
clamd a file path, it would copy the file to the temporary directory and
perform its tests on the copied file.
This means that even if i memory map my file, the test would still be done
against the copy file.

Any suggestions how to prevent clamd from copying the file to a temp dir?

Thanks again,
Michael.

On 5 Apr 2017 23:09, "Eugene Crosser" <crosser@average.org> wrote:

On 04/05/2017 09:27 PM, Michael Engstler wrote:
> Hi,
> I noticed that when using the INSTREAM command and sending it a memory
> buffer of a file, clamd takes the memory buffer and saves it to
> the TemporaryDirectory (as defined in the config file).
>
> This is an unnecessary overhead as it requires disk IO in order to scan
the
> file which is already loaded in-memory.
> Is there any way to command clamd to scan the buffer completely in-memory
> without writing anything to the disk?

That's a feature that I requested many years ago because I had a non-copying
MIME parser that worked on mmap()ed files, and I wanted to virus-scan some
parts
of the message. It was not possible, and I think it's not possible now.

I suggest you to mmap() your memory region to a temp. file, ideally on
tmpfs,
and pass this file to clamav. That ought to minimize the overhead.

Eugene


_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

http://www.clamav.net/contact.html#ml
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

http://www.clamav.net/contact.html#ml