Mailing List Archive

libclamav and INSTREAM
Helo!

The clamav daemon has an INSTREAM feature for scanning a stream of data.
I’m developing Go bindings for libclamav (https://github.com/eaigner/clam) and was
wondering why there isn’t such a feature in libclamav?

I searched the libclamav headers for something equal but didn’t find anything similar.
It seems I can only scan by file handle. If I use a pipe handle, it will fail.

Is that correct? Do I really have to write (potentially huge) files to disk to scan for clamav?

Cheers,

--
Erik Aigner


_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net
Re: libclamav and INSTREAM [ In reply to ]
Why would you *have* to write to the disk? No difference between a file handle to something in memory and to a file on the FS.

That being said, i actually used a ramdisk when building my clamav bindings (https://github.com/brandonprry/clam-sharp/).

Sent from a computer

> On Nov 20, 2013, at 12:42, Erik Aigner <aigner.erik@gmail.com> wrote:
>
> Helo!
>
> The clamav daemon has an INSTREAM feature for scanning a stream of data.
> I’m developing Go bindings for libclamav (https://github.com/eaigner/clam) and was
> wondering why there isn’t such a feature in libclamav?
>
> I searched the libclamav headers for something equal but didn’t find anything similar.
> It seems I can only scan by file handle. If I use a pipe handle, it will fail.
>
> Is that correct? Do I really have to write (potentially huge) files to disk to scan for clamav?
>
> Cheers,
>
> --
> Erik Aigner
>
>
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net
Re: libclamav and INSTREAM [ In reply to ]
On 11/20/2013 10:42 PM, Erik Aigner wrote:

> The clamav daemon has an INSTREAM feature for scanning a stream of data.
> I’m developing Go bindings for libclamav (https://github.com/eaigner/clam) and was
> wondering why there isn’t such a feature in libclamav?
>
> I searched the libclamav headers for something equal but didn’t find anything similar.
> It seems I can only scan by file handle. If I use a pipe handle, it will fail.
>
> Is that correct? Do I really have to write (potentially huge) files to disk to scan for clamav?

I've been advocating for a function in libclamav that would scan a memory region
instead of a file years ago. But this idea was never picked up, as far as I
know. I think that the best you can do is mmap() your memory buffer into a
temporary file and give the handle to scan_file().

Scanning a stream would probably be much harder if not impossible. If libclamav
was designed with abstract I/O layer, a la OpenSSL's BIO, it would have been
marvellous, but it is not.

Disclaimer: I am only a user, without the knowledge of clamav's internals.

Eugene
Re: libclamav and INSTREAM [ In reply to ]
"No difference between a file handle to something in memory and to a file on the FS.”

Apparently there is, otherwise it would work with a memory file handle.

--
Erik Aigner


On Wednesday 20 November 2013 at 20:15, Brandon Perry wrote:

> Why would you *have* to write to the disk? No difference between a file handle to something in memory and to a file on the FS.
>
> That being said, i actually used a ramdisk when building my clamav bindings (https://github.com/brandonprry/clam-sharp/).
>
> Sent from a computer
>
> > On Nov 20, 2013, at 12:42, Erik Aigner <aigner.erik@gmail.com (mailto:aigner.erik@gmail.com)> wrote:
> >
> > Helo!
> >
> > The clamav daemon has an INSTREAM feature for scanning a stream of data.
> > I’m developing Go bindings for libclamav (https://github.com/eaigner/clam) and was
> > wondering why there isn’t such a feature in libclamav?
> >
> > I searched the libclamav headers for something equal but didn’t find anything similar.
> > It seems I can only scan by file handle. If I use a pipe handle, it will fail.
> >
> > Is that correct? Do I really have to write (potentially huge) files to disk to scan for clamav?
> >
> > Cheers,
> >
> > --
> > Erik Aigner
> >
> >
> > _______________________________________________
> > http://lurker.clamav.net/list/clamav-devel.html
> > Please submit your patches to our Bugzilla: http://bugs.clamav.net
>
>
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net



_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net
Re: libclamav and INSTREAM [ In reply to ]
On Wed, Nov 20, 2013 at 2:45 PM, Erik Aigner <aigner.erik@gmail.com> wrote:

> "No difference between a file handle to something in memory and to a file
> on the FS.”
>
> Apparently there is, otherwise it would work with a memory file handle.
>
> --
> Erik Aigner
>
>
> On Wednesday 20 November 2013 at 20:15, Brandon Perry wrote:
>
> > Why would you *have* to write to the disk? No difference between a file
> handle to something in memory and to a file on the FS.
> >
> > That being said, i actually used a ramdisk when building my clamav
> bindings (https://github.com/brandonprry/clam-sharp/).
> >
> > Sent from a computer
> >
> > > On Nov 20, 2013, at 12:42, Erik Aigner <aigner.erik@gmail.com (mailto:
> aigner.erik@gmail.com)> wrote:
> > >
> > > Helo!
> > >
> > > The clamav daemon has an INSTREAM feature for scanning a stream of
> data.
> > > I’m developing Go bindings for libclamav (
> https://github.com/eaigner/clam) and was
> > > wondering why there isn’t such a feature in libclamav?
> > >
> > > I searched the libclamav headers for something equal but didn’t find
> anything similar.
> > > It seems I can only scan by file handle. If I use a pipe handle, it
> will fail.
> > >
> > > Is that correct? Do I really have to write (potentially huge) files to
> disk to scan for clamav?
> > >
> > > Cheers,
> > >
> > > --
> > > Erik Aigner
> > >
> > >
> > > _______________________________________________
> > > http://lurker.clamav.net/list/clamav-devel.html
> > > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >
> >
> > _______________________________________________
> > http://lurker.clamav.net/list/clamav-devel.html
> > Please submit your patches to our Bugzilla: http://bugs.clamav.net
>
>
>
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>

Based on the operations ClamAV performs and the way it does them, ClamAV
needs to be able to seek and rewind. Not every "stream" supports that.
Sockets cannot. There are also certain features, like some of the
callbacks, that were designed for file-based access and pass descriptors.

So right now libclamav does not expose functions for scanning blocks of
memory. ClamAV only uses maps that it created and makes sure it releases
them. Under the hood, a lot of the code has been switched over to using
maps ... so perhaps with the right setup call and symbols ... you might be
able to write code to what you are looking to do. Just be aware that you
may want to avoid or turn off certain features. Things like filetyping will
give much different results when dealing with memory blocks instead of
discrete files.

Good luck,

Dave R.

--
---
Dave Raynor
Sourcefire Vulnerability Research Team
draynor@sourcefire.com
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net
Re: libclamav and INSTREAM [ In reply to ]
Have you tried cl_scanmap_callback? It seems like it would work for what
you want. It crashes when scanning rar-sfx files, so I made a temporary
hack-fix for it here,
http://lurker.clamav.net/message/20130813.214249.1113932e.en.html

> Helo!
>
> The clamav daemon has an INSTREAM feature for scanning a stream of data.
> I’m developing Go bindings for libclamav
> (https://github.com/eaigner/clam) and was
> wondering why there isn’t such a feature in libclamav?
>
> I searched the libclamav headers for something equal but didn’t find
> anything similar.
> It seems I can only scan by file handle. If I use a pipe handle, it will
> fail.
>
> Is that correct? Do I really have to write (potentially huge) files to
> disk to scan for clamav?
>
> Cheers,
>
> --
> Erik Aigner
>
>
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net
Re: libclamav and INSTREAM [ In reply to ]
I managed to get it to work using `shm_open` with `ftruncate` and writing it to the shared memory region


--
Erik Aigner


On Thursday 21 November 2013 at 17:28, Andy Singer wrote:

> Have you tried cl_scanmap_callback? It seems like it would work for what
> you want. It crashes when scanning rar-sfx files, so I made a temporary
> hack-fix for it here,
> http://lurker.clamav.net/message/20130813.214249.1113932e.en.html
>
> > Helo!
> >
> > The clamav daemon has an INSTREAM feature for scanning a stream of data.
> > I’m developing Go bindings for libclamav
> > (https://github.com/eaigner/clam) and was
> > wondering why there isn’t such a feature in libclamav?
> >
> > I searched the libclamav headers for something equal but didn’t find
> > anything similar.
> > It seems I can only scan by file handle. If I use a pipe handle, it will
> > fail.
> >
> > Is that correct? Do I really have to write (potentially huge) files to
> > disk to scan for clamav?
> >
> > Cheers,
> >
> > --
> > Erik Aigner
> >
> >
> > _______________________________________________
> > http://lurker.clamav.net/list/clamav-devel.html
> > Please submit your patches to our Bugzilla: http://bugs.clamav.net
>
>
>
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net



_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net