Mailing List Archive: Improvement suggestion : Scan & Hash to optmize re-scan

Improvement suggestion : Scan & Hash to optmize re-scan

Jul 12, 2011, 2:18 AM

Post #1 of 4 (1956 views)

Hello,

I propose a possible speed-up scan (as I think that hashing is faster than
scanning)

The idea comes from security software like WinSonar (
http://www.fewbyte.com/winsonar.html) and file synchronisation.

I propose that when a media has been fully scanned, that for each large file
(maybe define by a threshold) its hash signature (SHA256, MD5, ...) is
stored so that when one re-scan the same media, ClamAV would skip the scan
of file if its current signature matches the scanned one.

One could even image a online central database with hashes of common files.

Yours,

Philippe
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

Re: Improvement suggestion : Scan & Hash to optmize re-scan [ In reply to ]

tkojm at clamav

Jul 12, 2011, 2:27 AM

Post #2 of 4 (1894 views)

Permalink

On Tue, 12 Jul 2011 11:18:20 +0200 sub phil <phil4000n@gmail.com> wrote:
> Hello,
>
> I propose a possible speed-up scan (as I think that hashing is faster than
> scanning)
>
> The idea comes from security software like WinSonar (
> http://www.fewbyte.com/winsonar.html) and file synchronisation.
>
> I propose that when a media has been fully scanned, that for each large file
> (maybe define by a threshold) its hash signature (SHA256, MD5, ...) is
> stored so that when one re-scan the same media, ClamAV would skip the scan
> of file if its current signature matches the scanned one.

Hi Philippe,

we already do this (see libclamav/cache.c). The cache gets purged after
a database update.

Regards,

--
oo ..... Tomasz Kojm <tkojm@clamav.net>
(\/)\......... http://www.ClamAV.net/gpg/tkojm.gpg
\..........._ 0DCA5A08407D5288279DB43454822DC8985A444B
//\ /\ Tue Jul 12 11:24:18 CEST 2011
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

Re: Improvement suggestion : Scan & Hash to optmize re-scan [ In reply to ]

mrflash818 at geophile

Jul 14, 2011, 11:06 AM

Post #3 of 4 (1881 views)

Permalink

Wouldn't that be best done by a separate product, like Tripwire?

Then only scan those files with changed sigs.

...Just my two cents.

> Hello,
>
> I propose a possible speed-up scan (as I think that hashing is faster than
> scanning)
>
> The idea comes from security software like WinSonar (
> http://www.fewbyte.com/winsonar.html) and file synchronisation.
>
> I propose that when a media has been fully scanned, that for each large
> file
> (maybe define by a threshold) its hash signature (SHA256, MD5, ...) is
> stored so that when one re-scan the same media, ClamAV would skip the scan
> of file if its current signature matches the scanned one.
>
> One could even image a online central database with hashes of common
> files.
>
> Yours,
>
> Philippe
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>

--
"Knowledge is Power" -- Francis Bacon

Robert Leyva
mrflash818@geophile.net

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

Re: Improvement suggestion : Scan & Hash to optmize re-scan [ In reply to ]

gim at skrzynka

Jul 26, 2011, 11:52 AM

Post #4 of 4 (1842 views)

Permalink

Robert Leyva in message 'Re: [Clamav-devel] Improvement suggestion : Scan & Hash to optmize re-scan' wrote:
> Wouldn't that be best done by a separate product, like Tripwire?
>
> Then only scan those files with changed sigs.
>

...and require every user of clamav to install tripwire...

doesn't seem like a good idea to me

cheers,
--
main(int a[puts("Michal 'GiM' Spadlinski")]){}
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net