Mailing List Archive

ClamAV scanning
I've noticed that if I use clamscan to scan a file, it always takes around 3.7 seconds no matter the file content, type, or even size (unless it is really very large file). Does any body know what is the feature in the ClamAV to make it takes the same time almost always?

Thanks,

~Moe

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net
Re: ClamAV scanning [ In reply to ]
On 2010-04-18 20:30, Mohammed Al-Saleh wrote:
> I've noticed that if I use clamscan to scan a file, it always takes around 3.7 seconds no matter the file content, type, or even size (unless it is really very large file).
> Does any body know what is the feature in the ClamAV to make it takes
the same time almost always?

clamscan loads the database each time, and loading that takes a few
seconds. The actual scanning time is much smaller.
Compare it to the time needed by clamdscan: that is the scantime, and it
does change according to filesize.

Best regards,
--Edwin
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net
Re: ClamAV scanning [ In reply to ]
Hi Edwin,

Thanks much for your useful replies.
Could you please point to me to where, in the source code, ClamAV does
scanning?
What are the string matching algorithms it uses?

Thanks again,

~Moe



2010/4/18 Török Edwin <edwintorok@gmail.com>

> On 2010-04-18 20:30, Mohammed Al-Saleh wrote:
> > I've noticed that if I use clamscan to scan a file, it always takes
> around 3.7 seconds no matter the file content, type, or even size (unless it
> is really very large file).
> > Does any body know what is the feature in the ClamAV to make it takes
> the same time almost always?
>
> clamscan loads the database each time, and loading that takes a few
> seconds. The actual scanning time is much smaller.
> Compare it to the time needed by clamdscan: that is the scantime, and it
> does change according to filesize.
>
> Best regards,
> --Edwin
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net
Re: ClamAV scanning [ In reply to ]
On 2010-04-18 22:50, Mohammed Al-Saleh wrote:
> Hi Edwin,
>
> Thanks much for your useful replies.
> Could you please point to me to where, in the source code, ClamAV does
> scanning?
> What are the string matching algorithms it uses?

Search the archives of this mailing list, this question has been
answered already.

Best regards,
--Edwin
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net
Re: ClamAV scanning [ In reply to ]
EICAR should only ever be detected as is. It is specially made for testing
AV, and AV has no use for detecting variations of it.

On Fri, Nov 7, 2014 at 11:02 AM, Andrew Camilleri <
andrew.camilleri@gmail.com> wrote:

> Hi!
>
> I am totally new to ClamAV, so please excuse my ignorance.
> I am looking at how AV scanning is done in general, but also specifically
> in ClamAV. I came across this
> <https://www.mail-archive.com/clamav-devel@lists.clamav.net/msg03096.html>
> post, so I got that bit covered and won't repeat questions.
> I am working on a WAF and we will use ClamAV for scanning traffic. I am
> investigating the tolerance in correct classification with respect to
> changes in malware binaries. To conduct my experiments I picked up the
> EICAR "virus" and an actual virus, Zeus, from here
> <https://github.com/Visgean/Zeus>. I noticed that if I change a single
> character in EICAR, ClamAV will fail to detect it; I assume that this is
> due to a static signature (correct me if I am wrong) associated with this
> test virus; this seems like a perfectly good result to me. Next thing was
> to scan Zeus (after a simple git clone) and it picks up a few trojans from
> the ready built binaries. I then changed the first byte of client32.bin
> (one of the files that was marked as a trojan) and scanned it. The result
> was the ClamAV did not recognize the trojan from this simple change. I then
> changed another byte, the 32nd one to be precise, and scanned it. The
> result was that ClamAV correctly classifies the binary as a Trojan. I was a
> little surprised that a change in the first byte would "hide" the trojan
> from scanning, especially since the first two bytes are completely useless
> <http://en.wikipedia.org/wiki/Mark_Zbikowski> in terms of running a
> windows
> binary. My only explanation is that with the change, the file fails some
> integrity check that ClamAV does, to make sure that the binary is runnable;
> I am assuming that there isnt a static signature here, otherwise it would
> not have been picked up with any change. I also did this test with zsb.exe
> in the repo and I got the same results. Finally I performed the same tests
> against McAffee and all these changes had no effect i.e. the trojans where
> always correctly classified. In the case of deltas to EICAR however, McAfee
> did not recognize the "virus".
> Could you please help me to understand the meaning of these results? Also,
> is it possible to view the signature of a virus in the signature database?
> I looked at the doc, but I couldn't find how to do this; but I may have
> missed it and in that case sorry to ask this!
>
> Andrew
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>
> http://www.clamav.net/contact.html#ml
>



--
http://volatile-minds.blogspot.com -- blog
http://www.volatileminds.net -- website
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

http://www.clamav.net/contact.html#ml
Re: ClamAV scanning [ In reply to ]
Hi Brandon,

Many thanks for your reply. I totally agree with you on EICAR, but this
should not happen with Zeus. EICAR was only included as a test case i.e. to
make sure that static signatures are being checked...

Andrew

On 7 November 2014 17:06, Brandon Perry <bperry.volatile@gmail.com> wrote:

> EICAR should only ever be detected as is. It is specially made for testing
> AV, and AV has no use for detecting variations of it.
>
> On Fri, Nov 7, 2014 at 11:02 AM, Andrew Camilleri <
> andrew.camilleri@gmail.com> wrote:
>
> > Hi!
> >
> > I am totally new to ClamAV, so please excuse my ignorance.
> > I am looking at how AV scanning is done in general, but also specifically
> > in ClamAV. I came across this
> > <
> https://www.mail-archive.com/clamav-devel@lists.clamav.net/msg03096.html>
> > post, so I got that bit covered and won't repeat questions.
> > I am working on a WAF and we will use ClamAV for scanning traffic. I am
> > investigating the tolerance in correct classification with respect to
> > changes in malware binaries. To conduct my experiments I picked up the
> > EICAR "virus" and an actual virus, Zeus, from here
> > <https://github.com/Visgean/Zeus>. I noticed that if I change a single
> > character in EICAR, ClamAV will fail to detect it; I assume that this is
> > due to a static signature (correct me if I am wrong) associated with this
> > test virus; this seems like a perfectly good result to me. Next thing was
> > to scan Zeus (after a simple git clone) and it picks up a few trojans
> from
> > the ready built binaries. I then changed the first byte of client32.bin
> > (one of the files that was marked as a trojan) and scanned it. The result
> > was the ClamAV did not recognize the trojan from this simple change. I
> then
> > changed another byte, the 32nd one to be precise, and scanned it. The
> > result was that ClamAV correctly classifies the binary as a Trojan. I
> was a
> > little surprised that a change in the first byte would "hide" the trojan
> > from scanning, especially since the first two bytes are completely
> useless
> > <http://en.wikipedia.org/wiki/Mark_Zbikowski> in terms of running a
> > windows
> > binary. My only explanation is that with the change, the file fails some
> > integrity check that ClamAV does, to make sure that the binary is
> runnable;
> > I am assuming that there isnt a static signature here, otherwise it would
> > not have been picked up with any change. I also did this test with
> zsb.exe
> > in the repo and I got the same results. Finally I performed the same
> tests
> > against McAffee and all these changes had no effect i.e. the trojans
> where
> > always correctly classified. In the case of deltas to EICAR however,
> McAfee
> > did not recognize the "virus".
> > Could you please help me to understand the meaning of these results?
> Also,
> > is it possible to view the signature of a virus in the signature
> database?
> > I looked at the doc, but I couldn't find how to do this; but I may have
> > missed it and in that case sorry to ask this!
> >
> > Andrew
> > _______________________________________________
> > http://lurker.clamav.net/list/clamav-devel.html
> > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >
> > http://www.clamav.net/contact.html#ml
> >
>
>
>
> --
> http://volatile-minds.blogspot.com -- blog
> http://www.volatileminds.net -- website
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>
> http://www.clamav.net/contact.html#ml
>
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

http://www.clamav.net/contact.html#ml
Re: ClamAV scanning [ In reply to ]
Hey Andrew,

The reason why ClamAV failed to detect Zeus when you changed the first byte
is because of file magic. When you changed the first byte, you changed the
type of file ClamAV recognizes it as. In fact, if it's a PE file, then
changing that first byte will cause Windows to not even execute the file.
It could be that the 32nd byte carried no significance and therefore ClamAV
was still able to detect the file as Zeus. So, when you change random data,
you're changing the behavior of the applications that utilize that data,
which is completely expected.

Thanks,

Shawn

On Fri, Nov 7, 2014 at 12:11 PM, Andrew Camilleri <
andrew.camilleri@gmail.com> wrote:

> Hi Brandon,
>
> Many thanks for your reply. I totally agree with you on EICAR, but this
> should not happen with Zeus. EICAR was only included as a test case i.e. to
> make sure that static signatures are being checked...
>
> Andrew
>
> On 7 November 2014 17:06, Brandon Perry <bperry.volatile@gmail.com> wrote:
>
> > EICAR should only ever be detected as is. It is specially made for
> testing
> > AV, and AV has no use for detecting variations of it.
> >
> > On Fri, Nov 7, 2014 at 11:02 AM, Andrew Camilleri <
> > andrew.camilleri@gmail.com> wrote:
> >
> > > Hi!
> > >
> > > I am totally new to ClamAV, so please excuse my ignorance.
> > > I am looking at how AV scanning is done in general, but also
> specifically
> > > in ClamAV. I came across this
> > > <
> > https://www.mail-archive.com/clamav-devel@lists.clamav.net/msg03096.html
> >
> > > post, so I got that bit covered and won't repeat questions.
> > > I am working on a WAF and we will use ClamAV for scanning traffic. I am
> > > investigating the tolerance in correct classification with respect to
> > > changes in malware binaries. To conduct my experiments I picked up the
> > > EICAR "virus" and an actual virus, Zeus, from here
> > > <https://github.com/Visgean/Zeus>. I noticed that if I change a single
> > > character in EICAR, ClamAV will fail to detect it; I assume that this
> is
> > > due to a static signature (correct me if I am wrong) associated with
> this
> > > test virus; this seems like a perfectly good result to me. Next thing
> was
> > > to scan Zeus (after a simple git clone) and it picks up a few trojans
> > from
> > > the ready built binaries. I then changed the first byte of client32.bin
> > > (one of the files that was marked as a trojan) and scanned it. The
> result
> > > was the ClamAV did not recognize the trojan from this simple change. I
> > then
> > > changed another byte, the 32nd one to be precise, and scanned it. The
> > > result was that ClamAV correctly classifies the binary as a Trojan. I
> > was a
> > > little surprised that a change in the first byte would "hide" the
> trojan
> > > from scanning, especially since the first two bytes are completely
> > useless
> > > <http://en.wikipedia.org/wiki/Mark_Zbikowski> in terms of running a
> > > windows
> > > binary. My only explanation is that with the change, the file fails
> some
> > > integrity check that ClamAV does, to make sure that the binary is
> > runnable;
> > > I am assuming that there isnt a static signature here, otherwise it
> would
> > > not have been picked up with any change. I also did this test with
> > zsb.exe
> > > in the repo and I got the same results. Finally I performed the same
> > tests
> > > against McAffee and all these changes had no effect i.e. the trojans
> > where
> > > always correctly classified. In the case of deltas to EICAR however,
> > McAfee
> > > did not recognize the "virus".
> > > Could you please help me to understand the meaning of these results?
> > Also,
> > > is it possible to view the signature of a virus in the signature
> > database?
> > > I looked at the doc, but I couldn't find how to do this; but I may have
> > > missed it and in that case sorry to ask this!
> > >
> > > Andrew
> > > _______________________________________________
> > > http://lurker.clamav.net/list/clamav-devel.html
> > > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> > >
> > > http://www.clamav.net/contact.html#ml
> > >
> >
> >
> >
> > --
> > http://volatile-minds.blogspot.com -- blog
> > http://www.volatileminds.net -- website
> > _______________________________________________
> > http://lurker.clamav.net/list/clamav-devel.html
> > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >
> > http://www.clamav.net/contact.html#ml
> >
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>
> http://www.clamav.net/contact.html#ml
>
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

http://www.clamav.net/contact.html#ml
Re: ClamAV scanning [ In reply to ]
Hi Shawn!

Many thanks for replying. I see what you are saying and I agree with you to
a certain extent. Here is my thinking process. My understanding is that the
job of an AV is to detect malware, and this usually involves pattern
matching. Yes you are right, the malware needs to run, but really that is a
different concern, the OS concern, whereas the concern of an AV is
detection/matching. It is a little alarming to me the fact that I can
easily hide a virus by simply changing a single byte; with the right
context (in our case perhaps through a trivial transformation down the
stack) a virus can come back to life. Do you see my point? Again I am no
expert in this, but McAffee seems to agree with what I am saying. Besides
separation of concerns, you can also argue for robustness; virus writers
have tried all sort of complex schemes to hide their artefacts and this
feels like a legitimate way of achieving that.
So if I understand correctly, ClamAV tries to guess the type of the file?
What happens if it can't tell what it is?
Sorry to drive this point further, I really appreciate all the hard work
people put in this project! But I really need to understand how this works
in detail....

regards,

Andrew



On 7 November 2014 18:41, Shawn Webb <swebb@sourcefire.com> wrote:

> Hey Andrew,
>
> The reason why ClamAV failed to detect Zeus when you changed the first byte
> is because of file magic. When you changed the first byte, you changed the
> type of file ClamAV recognizes it as. In fact, if it's a PE file, then
> changing that first byte will cause Windows to not even execute the file.
> It could be that the 32nd byte carried no significance and therefore ClamAV
> was still able to detect the file as Zeus. So, when you change random data,
> you're changing the behavior of the applications that utilize that data,
> which is completely expected.
>
> Thanks,
>
> Shawn
>
> On Fri, Nov 7, 2014 at 12:11 PM, Andrew Camilleri <
> andrew.camilleri@gmail.com> wrote:
>
> > Hi Brandon,
> >
> > Many thanks for your reply. I totally agree with you on EICAR, but this
> > should not happen with Zeus. EICAR was only included as a test case i.e.
> to
> > make sure that static signatures are being checked...
> >
> > Andrew
> >
> > On 7 November 2014 17:06, Brandon Perry <bperry.volatile@gmail.com>
> wrote:
> >
> > > EICAR should only ever be detected as is. It is specially made for
> > testing
> > > AV, and AV has no use for detecting variations of it.
> > >
> > > On Fri, Nov 7, 2014 at 11:02 AM, Andrew Camilleri <
> > > andrew.camilleri@gmail.com> wrote:
> > >
> > > > Hi!
> > > >
> > > > I am totally new to ClamAV, so please excuse my ignorance.
> > > > I am looking at how AV scanning is done in general, but also
> > specifically
> > > > in ClamAV. I came across this
> > > > <
> > >
> https://www.mail-archive.com/clamav-devel@lists.clamav.net/msg03096.html
> > >
> > > > post, so I got that bit covered and won't repeat questions.
> > > > I am working on a WAF and we will use ClamAV for scanning traffic. I
> am
> > > > investigating the tolerance in correct classification with respect to
> > > > changes in malware binaries. To conduct my experiments I picked up
> the
> > > > EICAR "virus" and an actual virus, Zeus, from here
> > > > <https://github.com/Visgean/Zeus>. I noticed that if I change a
> single
> > > > character in EICAR, ClamAV will fail to detect it; I assume that this
> > is
> > > > due to a static signature (correct me if I am wrong) associated with
> > this
> > > > test virus; this seems like a perfectly good result to me. Next thing
> > was
> > > > to scan Zeus (after a simple git clone) and it picks up a few trojans
> > > from
> > > > the ready built binaries. I then changed the first byte of
> client32.bin
> > > > (one of the files that was marked as a trojan) and scanned it. The
> > result
> > > > was the ClamAV did not recognize the trojan from this simple change.
> I
> > > then
> > > > changed another byte, the 32nd one to be precise, and scanned it. The
> > > > result was that ClamAV correctly classifies the binary as a Trojan. I
> > > was a
> > > > little surprised that a change in the first byte would "hide" the
> > trojan
> > > > from scanning, especially since the first two bytes are completely
> > > useless
> > > > <http://en.wikipedia.org/wiki/Mark_Zbikowski> in terms of running a
> > > > windows
> > > > binary. My only explanation is that with the change, the file fails
> > some
> > > > integrity check that ClamAV does, to make sure that the binary is
> > > runnable;
> > > > I am assuming that there isnt a static signature here, otherwise it
> > would
> > > > not have been picked up with any change. I also did this test with
> > > zsb.exe
> > > > in the repo and I got the same results. Finally I performed the same
> > > tests
> > > > against McAffee and all these changes had no effect i.e. the trojans
> > > where
> > > > always correctly classified. In the case of deltas to EICAR however,
> > > McAfee
> > > > did not recognize the "virus".
> > > > Could you please help me to understand the meaning of these results?
> > > Also,
> > > > is it possible to view the signature of a virus in the signature
> > > database?
> > > > I looked at the doc, but I couldn't find how to do this; but I may
> have
> > > > missed it and in that case sorry to ask this!
> > > >
> > > > Andrew
> > > > _______________________________________________
> > > > http://lurker.clamav.net/list/clamav-devel.html
> > > > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> > > >
> > > > http://www.clamav.net/contact.html#ml
> > > >
> > >
> > >
> > >
> > > --
> > > http://volatile-minds.blogspot.com -- blog
> > > http://www.volatileminds.net -- website
> > > _______________________________________________
> > > http://lurker.clamav.net/list/clamav-devel.html
> > > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> > >
> > > http://www.clamav.net/contact.html#ml
> > >
> > _______________________________________________
> > http://lurker.clamav.net/list/clamav-devel.html
> > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >
> > http://www.clamav.net/contact.html#ml
> >
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>
> http://www.clamav.net/contact.html#ml
>
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

http://www.clamav.net/contact.html#ml
Re: ClamAV scanning [ In reply to ]
Changing the first byte of an MZ header for a PE executable is not a
legitimate way of hiding your artifacts because no one will be able to run
your hidden artifact for the exploit/malware to do anything.

On Sat, Nov 8, 2014 at 11:41 AM, Andrew Camilleri <
andrew.camilleri@gmail.com> wrote:

> Hi Shawn!
>
> Many thanks for replying. I see what you are saying and I agree with you to
> a certain extent. Here is my thinking process. My understanding is that the
> job of an AV is to detect malware, and this usually involves pattern
> matching. Yes you are right, the malware needs to run, but really that is a
> different concern, the OS concern, whereas the concern of an AV is
> detection/matching. It is a little alarming to me the fact that I can
> easily hide a virus by simply changing a single byte; with the right
> context (in our case perhaps through a trivial transformation down the
> stack) a virus can come back to life. Do you see my point? Again I am no
> expert in this, but McAffee seems to agree with what I am saying. Besides
> separation of concerns, you can also argue for robustness; virus writers
> have tried all sort of complex schemes to hide their artefacts and this
> feels like a legitimate way of achieving that.
> So if I understand correctly, ClamAV tries to guess the type of the file?
> What happens if it can't tell what it is?
> Sorry to drive this point further, I really appreciate all the hard work
> people put in this project! But I really need to understand how this works
> in detail....
>
> regards,
>
> Andrew
>
>
>
> On 7 November 2014 18:41, Shawn Webb <swebb@sourcefire.com> wrote:
>
> > Hey Andrew,
> >
> > The reason why ClamAV failed to detect Zeus when you changed the first
> byte
> > is because of file magic. When you changed the first byte, you changed
> the
> > type of file ClamAV recognizes it as. In fact, if it's a PE file, then
> > changing that first byte will cause Windows to not even execute the file.
> > It could be that the 32nd byte carried no significance and therefore
> ClamAV
> > was still able to detect the file as Zeus. So, when you change random
> data,
> > you're changing the behavior of the applications that utilize that data,
> > which is completely expected.
> >
> > Thanks,
> >
> > Shawn
> >
> > On Fri, Nov 7, 2014 at 12:11 PM, Andrew Camilleri <
> > andrew.camilleri@gmail.com> wrote:
> >
> > > Hi Brandon,
> > >
> > > Many thanks for your reply. I totally agree with you on EICAR, but this
> > > should not happen with Zeus. EICAR was only included as a test case
> i.e.
> > to
> > > make sure that static signatures are being checked...
> > >
> > > Andrew
> > >
> > > On 7 November 2014 17:06, Brandon Perry <bperry.volatile@gmail.com>
> > wrote:
> > >
> > > > EICAR should only ever be detected as is. It is specially made for
> > > testing
> > > > AV, and AV has no use for detecting variations of it.
> > > >
> > > > On Fri, Nov 7, 2014 at 11:02 AM, Andrew Camilleri <
> > > > andrew.camilleri@gmail.com> wrote:
> > > >
> > > > > Hi!
> > > > >
> > > > > I am totally new to ClamAV, so please excuse my ignorance.
> > > > > I am looking at how AV scanning is done in general, but also
> > > specifically
> > > > > in ClamAV. I came across this
> > > > > <
> > > >
> > https://www.mail-archive.com/clamav-devel@lists.clamav.net/msg03096.html
> > > >
> > > > > post, so I got that bit covered and won't repeat questions.
> > > > > I am working on a WAF and we will use ClamAV for scanning traffic.
> I
> > am
> > > > > investigating the tolerance in correct classification with respect
> to
> > > > > changes in malware binaries. To conduct my experiments I picked up
> > the
> > > > > EICAR "virus" and an actual virus, Zeus, from here
> > > > > <https://github.com/Visgean/Zeus>. I noticed that if I change a
> > single
> > > > > character in EICAR, ClamAV will fail to detect it; I assume that
> this
> > > is
> > > > > due to a static signature (correct me if I am wrong) associated
> with
> > > this
> > > > > test virus; this seems like a perfectly good result to me. Next
> thing
> > > was
> > > > > to scan Zeus (after a simple git clone) and it picks up a few
> trojans
> > > > from
> > > > > the ready built binaries. I then changed the first byte of
> > client32.bin
> > > > > (one of the files that was marked as a trojan) and scanned it. The
> > > result
> > > > > was the ClamAV did not recognize the trojan from this simple
> change.
> > I
> > > > then
> > > > > changed another byte, the 32nd one to be precise, and scanned it.
> The
> > > > > result was that ClamAV correctly classifies the binary as a
> Trojan. I
> > > > was a
> > > > > little surprised that a change in the first byte would "hide" the
> > > trojan
> > > > > from scanning, especially since the first two bytes are completely
> > > > useless
> > > > > <http://en.wikipedia.org/wiki/Mark_Zbikowski> in terms of running
> a
> > > > > windows
> > > > > binary. My only explanation is that with the change, the file fails
> > > some
> > > > > integrity check that ClamAV does, to make sure that the binary is
> > > > runnable;
> > > > > I am assuming that there isnt a static signature here, otherwise it
> > > would
> > > > > not have been picked up with any change. I also did this test with
> > > > zsb.exe
> > > > > in the repo and I got the same results. Finally I performed the
> same
> > > > tests
> > > > > against McAffee and all these changes had no effect i.e. the
> trojans
> > > > where
> > > > > always correctly classified. In the case of deltas to EICAR
> however,
> > > > McAfee
> > > > > did not recognize the "virus".
> > > > > Could you please help me to understand the meaning of these
> results?
> > > > Also,
> > > > > is it possible to view the signature of a virus in the signature
> > > > database?
> > > > > I looked at the doc, but I couldn't find how to do this; but I may
> > have
> > > > > missed it and in that case sorry to ask this!
> > > > >
> > > > > Andrew
> > > > > _______________________________________________
> > > > > http://lurker.clamav.net/list/clamav-devel.html
> > > > > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> > > > >
> > > > > http://www.clamav.net/contact.html#ml
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > http://volatile-minds.blogspot.com -- blog
> > > > http://www.volatileminds.net -- website
> > > > _______________________________________________
> > > > http://lurker.clamav.net/list/clamav-devel.html
> > > > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> > > >
> > > > http://www.clamav.net/contact.html#ml
> > > >
> > > _______________________________________________
> > > http://lurker.clamav.net/list/clamav-devel.html
> > > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> > >
> > > http://www.clamav.net/contact.html#ml
> > >
> > _______________________________________________
> > http://lurker.clamav.net/list/clamav-devel.html
> > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >
> > http://www.clamav.net/contact.html#ml
> >
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>
> http://www.clamav.net/contact.html#ml
>



--
http://volatile-minds.blogspot.com -- blog
http://www.volatileminds.net -- website
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

http://www.clamav.net/contact.html#ml
Re: ClamAV scanning [ In reply to ]
Hi Andrew,

In order to run an executable file (PE) with a changed MZ, in most cases a malware needs either a dropper or an injector. The dropper will modify the magic bytes to their original values and then execute the file. An injector can just map the file in memory and execute the code directly. Like Brandon said, the file won't be able to run by itself.
However, if you really want to detect a malware even if the header is modified, I suggest to change the TargetType in the signature, from 1 (which means PE) to 0 (which means any file). Assuming that it's an extended signature that will match specific bytes in the code, then even if the header (MZ) is changed, the file will still be detected.

Hope that helps.

Regards,
Andrei


> Date: Sat, 8 Nov 2014 11:50:18 -0600
> From: bperry.volatile@gmail.com
> To: clamav-devel@lists.clamav.net
> Subject: Re: [Clamav-devel] ClamAV scanning
>
> Changing the first byte of an MZ header for a PE executable is not a
> legitimate way of hiding your artifacts because no one will be able to run
> your hidden artifact for the exploit/malware to do anything.
>
> On Sat, Nov 8, 2014 at 11:41 AM, Andrew Camilleri <
> andrew.camilleri@gmail.com> wrote:
>
> > Hi Shawn!
> >
> > Many thanks for replying. I see what you are saying and I agree with you to
> > a certain extent. Here is my thinking process. My understanding is that the
> > job of an AV is to detect malware, and this usually involves pattern
> > matching. Yes you are right, the malware needs to run, but really that is a
> > different concern, the OS concern, whereas the concern of an AV is
> > detection/matching. It is a little alarming to me the fact that I can
> > easily hide a virus by simply changing a single byte; with the right
> > context (in our case perhaps through a trivial transformation down the
> > stack) a virus can come back to life. Do you see my point? Again I am no
> > expert in this, but McAffee seems to agree with what I am saying. Besides
> > separation of concerns, you can also argue for robustness; virus writers
> > have tried all sort of complex schemes to hide their artefacts and this
> > feels like a legitimate way of achieving that.
> > So if I understand correctly, ClamAV tries to guess the type of the file?
> > What happens if it can't tell what it is?
> > Sorry to drive this point further, I really appreciate all the hard work
> > people put in this project! But I really need to understand how this works
> > in detail....
> >
> > regards,
> >
> > Andrew
> >
> >
> >
> > On 7 November 2014 18:41, Shawn Webb <swebb@sourcefire.com> wrote:
> >
> > > Hey Andrew,
> > >
> > > The reason why ClamAV failed to detect Zeus when you changed the first
> > byte
> > > is because of file magic. When you changed the first byte, you changed
> > the
> > > type of file ClamAV recognizes it as. In fact, if it's a PE file, then
> > > changing that first byte will cause Windows to not even execute the file.
> > > It could be that the 32nd byte carried no significance and therefore
> > ClamAV
> > > was still able to detect the file as Zeus. So, when you change random
> > data,
> > > you're changing the behavior of the applications that utilize that data,
> > > which is completely expected.
> > >
> > > Thanks,
> > >
> > > Shawn
> > >
> > > On Fri, Nov 7, 2014 at 12:11 PM, Andrew Camilleri <
> > > andrew.camilleri@gmail.com> wrote:
> > >
> > > > Hi Brandon,
> > > >
> > > > Many thanks for your reply. I totally agree with you on EICAR, but this
> > > > should not happen with Zeus. EICAR was only included as a test case
> > i.e.
> > > to
> > > > make sure that static signatures are being checked...
> > > >
> > > > Andrew
> > > >
> > > > On 7 November 2014 17:06, Brandon Perry <bperry.volatile@gmail.com>
> > > wrote:
> > > >
> > > > > EICAR should only ever be detected as is. It is specially made for
> > > > testing
> > > > > AV, and AV has no use for detecting variations of it.
> > > > >
> > > > > On Fri, Nov 7, 2014 at 11:02 AM, Andrew Camilleri <
> > > > > andrew.camilleri@gmail.com> wrote:
> > > > >
> > > > > > Hi!
> > > > > >
> > > > > > I am totally new to ClamAV, so please excuse my ignorance.
> > > > > > I am looking at how AV scanning is done in general, but also
> > > > specifically
> > > > > > in ClamAV. I came across this
> > > > > > <
> > > > >
> > > https://www.mail-archive.com/clamav-devel@lists.clamav.net/msg03096.html
> > > > >
> > > > > > post, so I got that bit covered and won't repeat questions.
> > > > > > I am working on a WAF and we will use ClamAV for scanning traffic.
> > I
> > > am
> > > > > > investigating the tolerance in correct classification with respect
> > to
> > > > > > changes in malware binaries. To conduct my experiments I picked up
> > > the
> > > > > > EICAR "virus" and an actual virus, Zeus, from here
> > > > > > <https://github.com/Visgean/Zeus>. I noticed that if I change a
> > > single
> > > > > > character in EICAR, ClamAV will fail to detect it; I assume that
> > this
> > > > is
> > > > > > due to a static signature (correct me if I am wrong) associated
> > with
> > > > this
> > > > > > test virus; this seems like a perfectly good result to me. Next
> > thing
> > > > was
> > > > > > to scan Zeus (after a simple git clone) and it picks up a few
> > trojans
> > > > > from
> > > > > > the ready built binaries. I then changed the first byte of
> > > client32.bin
> > > > > > (one of the files that was marked as a trojan) and scanned it. The
> > > > result
> > > > > > was the ClamAV did not recognize the trojan from this simple
> > change.
> > > I
> > > > > then
> > > > > > changed another byte, the 32nd one to be precise, and scanned it.
> > The
> > > > > > result was that ClamAV correctly classifies the binary as a
> > Trojan. I
> > > > > was a
> > > > > > little surprised that a change in the first byte would "hide" the
> > > > trojan
> > > > > > from scanning, especially since the first two bytes are completely
> > > > > useless
> > > > > > <http://en.wikipedia.org/wiki/Mark_Zbikowski> in terms of running
> > a
> > > > > > windows
> > > > > > binary. My only explanation is that with the change, the file fails
> > > > some
> > > > > > integrity check that ClamAV does, to make sure that the binary is
> > > > > runnable;
> > > > > > I am assuming that there isnt a static signature here, otherwise it
> > > > would
> > > > > > not have been picked up with any change. I also did this test with
> > > > > zsb.exe
> > > > > > in the repo and I got the same results. Finally I performed the
> > same
> > > > > tests
> > > > > > against McAffee and all these changes had no effect i.e. the
> > trojans
> > > > > where
> > > > > > always correctly classified. In the case of deltas to EICAR
> > however,
> > > > > McAfee
> > > > > > did not recognize the "virus".
> > > > > > Could you please help me to understand the meaning of these
> > results?
> > > > > Also,
> > > > > > is it possible to view the signature of a virus in the signature
> > > > > database?
> > > > > > I looked at the doc, but I couldn't find how to do this; but I may
> > > have
> > > > > > missed it and in that case sorry to ask this!
> > > > > >
> > > > > > Andrew
> > > > > > _______________________________________________
> > > > > > http://lurker.clamav.net/list/clamav-devel.html
> > > > > > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> > > > > >
> > > > > > http://www.clamav.net/contact.html#ml
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > http://volatile-minds.blogspot.com -- blog
> > > > > http://www.volatileminds.net -- website
> > > > > _______________________________________________
> > > > > http://lurker.clamav.net/list/clamav-devel.html
> > > > > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> > > > >
> > > > > http://www.clamav.net/contact.html#ml
> > > > >
> > > > _______________________________________________
> > > > http://lurker.clamav.net/list/clamav-devel.html
> > > > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> > > >
> > > > http://www.clamav.net/contact.html#ml
> > > >
> > > _______________________________________________
> > > http://lurker.clamav.net/list/clamav-devel.html
> > > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> > >
> > > http://www.clamav.net/contact.html#ml
> > >
> > _______________________________________________
> > http://lurker.clamav.net/list/clamav-devel.html
> > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >
> > http://www.clamav.net/contact.html#ml
> >
>
>
>
> --
> http://volatile-minds.blogspot.com -- blog
> http://www.volatileminds.net -- website
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>
> http://www.clamav.net/contact.html#ml

_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

http://www.clamav.net/contact.html#ml
Re: ClamAV scanning [ In reply to ]
Hi there,

On Sun, 9 Nov 2014, Andrew Camilleri wrote:

> ... I am no expert in this

You said it.

> ... McAffee seems to agree ...

And he's in jail for murder.

> ... I really need to understand how this works in detail....

Then stop writing and start reading.

--

73,
Ged.
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

http://www.clamav.net/contact.html#ml
Re: ClamAV scanning [ In reply to ]
Thank you Brandon & Andrei! I appreciate your help!

On 9 November 2014 01:58, <andreisaygo@live.ie> wrote:

> Hi Andrew,
>
> In order to run an executable file (PE) with a changed MZ, in most cases a
> malware needs either a dropper or an injector. The dropper will modify the
> magic bytes to their original values and then execute the file. An injector
> can just map the file in memory and execute the code directly. Like Brandon
> said, the file won't be able to run by itself.
> However, if you really want to detect a malware even if the header is
> modified, I suggest to change the TargetType in the signature, from 1
> (which means PE) to 0 (which means any file). Assuming that it's an
> extended signature that will match specific bytes in the code, then even if
> the header (MZ) is changed, the file will still be detected.
>
> Hope that helps.
>
> Regards,
> Andrei
>
>
> > Date: Sat, 8 Nov 2014 11:50:18 -0600
> > From: bperry.volatile@gmail.com
> > To: clamav-devel@lists.clamav.net
> > Subject: Re: [Clamav-devel] ClamAV scanning
> >
> > Changing the first byte of an MZ header for a PE executable is not a
> > legitimate way of hiding your artifacts because no one will be able to
> run
> > your hidden artifact for the exploit/malware to do anything.
> >
> > On Sat, Nov 8, 2014 at 11:41 AM, Andrew Camilleri <
> > andrew.camilleri@gmail.com> wrote:
> >
> > > Hi Shawn!
> > >
> > > Many thanks for replying. I see what you are saying and I agree with
> you to
> > > a certain extent. Here is my thinking process. My understanding is
> that the
> > > job of an AV is to detect malware, and this usually involves pattern
> > > matching. Yes you are right, the malware needs to run, but really that
> is a
> > > different concern, the OS concern, whereas the concern of an AV is
> > > detection/matching. It is a little alarming to me the fact that I can
> > > easily hide a virus by simply changing a single byte; with the right
> > > context (in our case perhaps through a trivial transformation down the
> > > stack) a virus can come back to life. Do you see my point? Again I am
> no
> > > expert in this, but McAffee seems to agree with what I am saying.
> Besides
> > > separation of concerns, you can also argue for robustness; virus
> writers
> > > have tried all sort of complex schemes to hide their artefacts and this
> > > feels like a legitimate way of achieving that.
> > > So if I understand correctly, ClamAV tries to guess the type of the
> file?
> > > What happens if it can't tell what it is?
> > > Sorry to drive this point further, I really appreciate all the hard
> work
> > > people put in this project! But I really need to understand how this
> works
> > > in detail....
> > >
> > > regards,
> > >
> > > Andrew
> > >
> > >
> > >
> > > On 7 November 2014 18:41, Shawn Webb <swebb@sourcefire.com> wrote:
> > >
> > > > Hey Andrew,
> > > >
> > > > The reason why ClamAV failed to detect Zeus when you changed the
> first
> > > byte
> > > > is because of file magic. When you changed the first byte, you
> changed
> > > the
> > > > type of file ClamAV recognizes it as. In fact, if it's a PE file,
> then
> > > > changing that first byte will cause Windows to not even execute the
> file.
> > > > It could be that the 32nd byte carried no significance and therefore
> > > ClamAV
> > > > was still able to detect the file as Zeus. So, when you change random
> > > data,
> > > > you're changing the behavior of the applications that utilize that
> data,
> > > > which is completely expected.
> > > >
> > > > Thanks,
> > > >
> > > > Shawn
> > > >
> > > > On Fri, Nov 7, 2014 at 12:11 PM, Andrew Camilleri <
> > > > andrew.camilleri@gmail.com> wrote:
> > > >
> > > > > Hi Brandon,
> > > > >
> > > > > Many thanks for your reply. I totally agree with you on EICAR, but
> this
> > > > > should not happen with Zeus. EICAR was only included as a test case
> > > i.e.
> > > > to
> > > > > make sure that static signatures are being checked...
> > > > >
> > > > > Andrew
> > > > >
> > > > > On 7 November 2014 17:06, Brandon Perry <bperry.volatile@gmail.com
> >
> > > > wrote:
> > > > >
> > > > > > EICAR should only ever be detected as is. It is specially made
> for
> > > > > testing
> > > > > > AV, and AV has no use for detecting variations of it.
> > > > > >
> > > > > > On Fri, Nov 7, 2014 at 11:02 AM, Andrew Camilleri <
> > > > > > andrew.camilleri@gmail.com> wrote:
> > > > > >
> > > > > > > Hi!
> > > > > > >
> > > > > > > I am totally new to ClamAV, so please excuse my ignorance.
> > > > > > > I am looking at how AV scanning is done in general, but also
> > > > > specifically
> > > > > > > in ClamAV. I came across this
> > > > > > > <
> > > > > >
> > > >
> https://www.mail-archive.com/clamav-devel@lists.clamav.net/msg03096.html
> > > > > >
> > > > > > > post, so I got that bit covered and won't repeat questions.
> > > > > > > I am working on a WAF and we will use ClamAV for scanning
> traffic.
> > > I
> > > > am
> > > > > > > investigating the tolerance in correct classification with
> respect
> > > to
> > > > > > > changes in malware binaries. To conduct my experiments I
> picked up
> > > > the
> > > > > > > EICAR "virus" and an actual virus, Zeus, from here
> > > > > > > <https://github.com/Visgean/Zeus>. I noticed that if I change
> a
> > > > single
> > > > > > > character in EICAR, ClamAV will fail to detect it; I assume
> that
> > > this
> > > > > is
> > > > > > > due to a static signature (correct me if I am wrong) associated
> > > with
> > > > > this
> > > > > > > test virus; this seems like a perfectly good result to me. Next
> > > thing
> > > > > was
> > > > > > > to scan Zeus (after a simple git clone) and it picks up a few
> > > trojans
> > > > > > from
> > > > > > > the ready built binaries. I then changed the first byte of
> > > > client32.bin
> > > > > > > (one of the files that was marked as a trojan) and scanned it.
> The
> > > > > result
> > > > > > > was the ClamAV did not recognize the trojan from this simple
> > > change.
> > > > I
> > > > > > then
> > > > > > > changed another byte, the 32nd one to be precise, and scanned
> it.
> > > The
> > > > > > > result was that ClamAV correctly classifies the binary as a
> > > Trojan. I
> > > > > > was a
> > > > > > > little surprised that a change in the first byte would "hide"
> the
> > > > > trojan
> > > > > > > from scanning, especially since the first two bytes are
> completely
> > > > > > useless
> > > > > > > <http://en.wikipedia.org/wiki/Mark_Zbikowski> in terms of
> running
> > > a
> > > > > > > windows
> > > > > > > binary. My only explanation is that with the change, the file
> fails
> > > > > some
> > > > > > > integrity check that ClamAV does, to make sure that the binary
> is
> > > > > > runnable;
> > > > > > > I am assuming that there isnt a static signature here,
> otherwise it
> > > > > would
> > > > > > > not have been picked up with any change. I also did this test
> with
> > > > > > zsb.exe
> > > > > > > in the repo and I got the same results. Finally I performed the
> > > same
> > > > > > tests
> > > > > > > against McAffee and all these changes had no effect i.e. the
> > > trojans
> > > > > > where
> > > > > > > always correctly classified. In the case of deltas to EICAR
> > > however,
> > > > > > McAfee
> > > > > > > did not recognize the "virus".
> > > > > > > Could you please help me to understand the meaning of these
> > > results?
> > > > > > Also,
> > > > > > > is it possible to view the signature of a virus in the
> signature
> > > > > > database?
> > > > > > > I looked at the doc, but I couldn't find how to do this; but I
> may
> > > > have
> > > > > > > missed it and in that case sorry to ask this!
> > > > > > >
> > > > > > > Andrew
> > > > > > > _______________________________________________
> > > > > > > http://lurker.clamav.net/list/clamav-devel.html
> > > > > > > Please submit your patches to our Bugzilla:
> http://bugs.clamav.net
> > > > > > >
> > > > > > > http://www.clamav.net/contact.html#ml
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > http://volatile-minds.blogspot.com -- blog
> > > > > > http://www.volatileminds.net -- website
> > > > > > _______________________________________________
> > > > > > http://lurker.clamav.net/list/clamav-devel.html
> > > > > > Please submit your patches to our Bugzilla:
> http://bugs.clamav.net
> > > > > >
> > > > > > http://www.clamav.net/contact.html#ml
> > > > > >
> > > > > _______________________________________________
> > > > > http://lurker.clamav.net/list/clamav-devel.html
> > > > > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> > > > >
> > > > > http://www.clamav.net/contact.html#ml
> > > > >
> > > > _______________________________________________
> > > > http://lurker.clamav.net/list/clamav-devel.html
> > > > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> > > >
> > > > http://www.clamav.net/contact.html#ml
> > > >
> > > _______________________________________________
> > > http://lurker.clamav.net/list/clamav-devel.html
> > > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> > >
> > > http://www.clamav.net/contact.html#ml
> > >
> >
> >
> >
> > --
> > http://volatile-minds.blogspot.com -- blog
> > http://www.volatileminds.net -- website
> > _______________________________________________
> > http://lurker.clamav.net/list/clamav-devel.html
> > Please submit your patches to our Bugzilla: http://bugs.clamav.net
> >
> > http://www.clamav.net/contact.html#ml
>
> _______________________________________________
> http://lurker.clamav.net/list/clamav-devel.html
> Please submit your patches to our Bugzilla: http://bugs.clamav.net
>
> http://www.clamav.net/contact.html#ml
>
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net

http://www.clamav.net/contact.html#ml