Mailing List Archive

How can we consume .ldb files in ClamAV Ubuntu?
Hi All,

We have ClamAV installed on Ubuntu. On Ubuntu, the rules can be specified
or modified under the directory */var/lib/clamav/main.cvd*. However, We
are trying to consume ClamAV rules from the FireEye as shown below
link which is* .ldb* file and we are trying to convert to *.cvd* format.

Could you please let us know the steps on how to convert the* .ldb* to
*.cvd?* Or how to consume the* .ldb *file in Ubuntu?


FireEye:
https://github.com/fireeye/red_team_tool_countermeasures/blob/master/all-clam.ldb
<https://github.com/fireeye/red_team_tool_countermeasures/blob/master/all-clam.ldb>

Thank you for your time and consideration.

--
Thanks,
Sandeep
Re: How can we consume .ldb files in ClamAV Ubuntu? [ In reply to ]
Sandeep Talla wrote:
> Hi All,
>
> We have ClamAV installed on Ubuntu. On Ubuntu, the rules can be
> specified or modified under the directory */var/lib/clamav/main.cvd*.
> However,  We are trying to consume ClamAV rules from the FireEye as
> shown below link which is*.ldb* file and we are trying to convert to
> *.cvd* format.
>
> Could you please let us know the steps on how to convert the*.ldb* to
> *.cvd?* Or how to consume the*.ldb *file in Ubuntu?

You shouldn't need to convert the format; just put the file in
/var/lib/clamav and clamd or clamscan should pick it up alongside the
stock .cvd and/or .cld files.

-kgd

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: How can we consume .ldb files in ClamAV Ubuntu? [ In reply to ]
Hi Sandeep,

There's no need to convert them. Just put them straight into the clamav database directory and call them whatever_you_want.ldb eg
/var/lib/clamav/fireeye.ldb

As long as the name you choose doesn't conflict with ClamAV's naming (eg main/daily/bytecode etc), the only bits you need to work about are keeping the 'ldb' extension, and ensuring the files are in the correct location with the correct ownership and permissions.

Mark

> On 14 Dec 2020, at 8:33 pm, Sandeep Talla <sandeep.talla@inadev.com> wrote:
>
> Hi All,
>
> We have ClamAV installed on Ubuntu. On Ubuntu, the rules can be specified or modified under the directory /var/lib/clamav/main.cvd. However, We are trying to consume ClamAV rules from the FireEye as shown below link which is .ldb file and we are trying to convert to .cvd format.
>
> Could you please let us know the steps on how to convert the .ldb to .cvd? Or how to consume the .ldb file in Ubuntu?
>
>
> FireEye: https://github.com/fireeye/red_team_tool_countermeasures/blob/master/all-clam.ldb <https://github.com/fireeye/red_team_tool_countermeasures/blob/master/all-clam.ldb>
>
> Thank you for your time and consideration.
>
> --
> Thanks,
> Sandeep
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
Re: How can we consume .ldb files in ClamAV Ubuntu? [ In reply to ]
Hi Mark/Kris,

Thank you for your responses. I have placed the *fireeye.ldb* file under
the directory /var/lib/clamav/ and modified the permission to 644 and
ownership to clamav. Then we have restarted the service Clamav-Deamon and
then started clamscan. However, Clamscam is not picking up the *fireeye.ldb*
file when we verify the Freshclam.log and clamav.log files.

Are there any configuration settings that need to add for *clamd.conf* or
*freshclam.conf* in order to pick up the fireeye.ldb file during clamscan?

On Mon, Dec 14, 2020 at 4:20 PM Mark Allan via clamav-users <
clamav-users@lists.clamav.net> wrote:

> Hi Sandeep,
>
> There's no need to convert them. Just put them straight into the clamav
> database directory and call them whatever_you_want.ldb eg
> /var/lib/clamav/fireeye.ldb
>
> As long as the name you choose doesn't conflict with ClamAV's naming (eg
> main/daily/bytecode etc), the only bits you need to work about are keeping
> the 'ldb' extension, and ensuring the files are in the correct location
> with the correct ownership and permissions.
>
> Mark
>
> On 14 Dec 2020, at 8:33 pm, Sandeep Talla <sandeep.talla@inadev.com>
> wrote:
>
> Hi All,
>
> We have ClamAV installed on Ubuntu. On Ubuntu, the rules can be specified
> or modified under the directory */var/lib/clamav/main.cvd*. However, We
> are trying to consume ClamAV rules from the FireEye as shown below
> link which is* .ldb* file and we are trying to convert to *.cvd* format.
>
> Could you please let us know the steps on how to convert the* .ldb* to
> *.cvd?* Or how to consume the* .ldb *file in Ubuntu?
>
>
> FireEye:
> https://github.com/fireeye/red_team_tool_countermeasures/blob/master/all-clam.ldb
> <https://github.com/fireeye/red_team_tool_countermeasures/blob/master/all-clam.ldb>
>
> Thank you for your time and consideration.
>
> --
> Thanks,
> Sandeep
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>
>
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>


--
Thanks,
Sandeep Talla
Re: How can we consume .ldb files in ClamAV Ubuntu? [ In reply to ]
Sandeep Talla wrote:
> Hi Mark/Kris,
>
> Thank you for your responses. I have placed the *fireeye.ldb* file under
> the directory /var/lib/clamav/ and modified the permission to 644 and
> ownership to clamav. Then we have restarted the service
> Clamav-Deamon and then started clamscan. However, Clamscam is not
> picking up the *fireeye.ldb* file when we verify the Freshclam.log and
> clamav.log files.
>
> Are there any configuration settings that need to add for *clamd.conf*
> or *freshclam.conf* in order to pick up the fireeye.ldb file during
> clamscan?

The only thing that comes to mind is to check for the
"OfficialDatabaseOnly" option in the configuration; if set to "on" or
"yes" this only loads the official databases.

The output from clamscan -D might tell you more.

I have a couple of system using third party and local signatures without
problem just by dropping the files beside the stock files.

-kgd

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: How can we consume .ldb files in ClamAV Ubuntu? [ In reply to ]
Hi there,

On Mon, 14 Dec 2020, Sandeep Talla wrote:

> ... *fireeye.ldb* file under the directory /var/lib/clamav/ ...
> ... Clamscam is not picking up the *fireeye.ldb* file when

Clamscam. I like that. :)

> we verify the Freshclam.log and clamav.log files.

Freshclam will not update the Fireeye data unless it is both available
from a mirror which freshclam can recognize and the mirror location is
given in freshclam.conf using the 'DatabaseCustomURL' option. See the
man page for freshclam.conf for more information. Freshclam will not
mention the file in its logs unless it updates it. But freshclam only
updates the files, it does not affect whether or not clamd loads them,
and it has no effect on clamscan at all.

I do not know what the 'clamav.log' file contains, perhaps it is only
found in Ubuntu systems.

When clamd has reloaded its databases you will see that it writes in
its log the number of signatures which it has loaded. It's quite a
large number, of the order of ten million, but you should see that
after you have the Fireeye data in the correct location and clamd has
reloaded the data, there are 23 more signatures than the last time
clamd loaded the data. Below is an extract from my clamd server log.
I downloaded the file from the URL you gave, dropped it in the clamd
database directory, and issued a RELOAD command using telnet. As you
can see, there are 23 more signatures after the reload.

pi4b530214:/var/log/clamav# >>> grep -i reload clamd.2.log | tail -n 3
Mon Dec 14 22:42:18 2020 -> Database correctly reloaded (11352914 signatures)
Mon Dec 14 23:12:35 2020 -> got command RELOAD (7, 2), argument:
Mon Dec 14 23:13:39 2020 -> Database correctly reloaded (11352937 signatures)

What is the size of your fireeye.ldb file? Have you checked it with a
pager to make sure that it looks OK? It should be 26 lines of text.
Some of them are very long.

> Are there any configuration settings that need to add for *clamd.conf* or
> *freshclam.conf* in order to pick up the fireeye.ldb file during clamscan?

Freshclam.conf is irrelevant. Do you have in clamd.conf the option

--official-db-only

set to 'yes'? See the clamd man page for more information.

If you run

clamscan --debug some_test_file

and pipe the output to a pager or through grep or something you see
listed in the (long) output all the databases which clamscan loads:

ged@pi4b530214:~ $ clamscan --debug phish-test 2>&1 | grep loaded
LibClamAV debug: unrar support loaded from libclamunrar_iface.so.9
LibClamAV debug: daily.info loaded
LibClamAV debug: daily.cfg loaded
...
...
LibClamAV debug: /EXPORTS/clamav/databases/all-clam.ldb loaded
...
...

HTH

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: How can we consume .ldb files in ClamAV Ubuntu? [ In reply to ]
Also, we have shipped detection which detects the same things Fireeye was detecting and much more, also rewritten to be more efficient in the official ruleset.

Sent from my ? iPhone

> On Dec 14, 2020, at 18:54, G.W. Haywood via clamav-users <clamav-users@lists.clamav.net> wrote:
>
> ?Hi there,
>
>> On Mon, 14 Dec 2020, Sandeep Talla wrote:
>>
>> ... *fireeye.ldb* file under the directory /var/lib/clamav/ ...
>> ... Clamscam is not picking up the *fireeye.ldb* file when
>
> Clamscam. I like that. :)
>
>> we verify the Freshclam.log and clamav.log files.
>
> Freshclam will not update the Fireeye data unless it is both available
> from a mirror which freshclam can recognize and the mirror location is
> given in freshclam.conf using the 'DatabaseCustomURL' option. See the
> man page for freshclam.conf for more information. Freshclam will not
> mention the file in its logs unless it updates it. But freshclam only
> updates the files, it does not affect whether or not clamd loads them,
> and it has no effect on clamscan at all.
>
> I do not know what the 'clamav.log' file contains, perhaps it is only
> found in Ubuntu systems.
>
> When clamd has reloaded its databases you will see that it writes in
> its log the number of signatures which it has loaded. It's quite a
> large number, of the order of ten million, but you should see that
> after you have the Fireeye data in the correct location and clamd has
> reloaded the data, there are 23 more signatures than the last time
> clamd loaded the data. Below is an extract from my clamd server log.
> I downloaded the file from the URL you gave, dropped it in the clamd
> database directory, and issued a RELOAD command using telnet. As you
> can see, there are 23 more signatures after the reload.
>
> pi4b530214:/var/log/clamav# >>> grep -i reload clamd.2.log | tail -n 3
> Mon Dec 14 22:42:18 2020 -> Database correctly reloaded (11352914 signatures)
> Mon Dec 14 23:12:35 2020 -> got command RELOAD (7, 2), argument: Mon Dec 14 23:13:39 2020 -> Database correctly reloaded (11352937 signatures)
>
> What is the size of your fireeye.ldb file? Have you checked it with a
> pager to make sure that it looks OK? It should be 26 lines of text.
> Some of them are very long.
>
>> Are there any configuration settings that need to add for *clamd.conf* or
>> *freshclam.conf* in order to pick up the fireeye.ldb file during clamscan?
>
> Freshclam.conf is irrelevant. Do you have in clamd.conf the option
>
> --official-db-only
>
> set to 'yes'? See the clamd man page for more information.
>
> If you run
>
> clamscan --debug some_test_file
>
> and pipe the output to a pager or through grep or something you see
> listed in the (long) output all the databases which clamscan loads:
>
> ged@pi4b530214:~ $ clamscan --debug phish-test 2>&1 | grep loaded
> LibClamAV debug: unrar support loaded from libclamunrar_iface.so.9
> LibClamAV debug: daily.info loaded
> LibClamAV debug: daily.cfg loaded
> ...
> ...
> LibClamAV debug: /EXPORTS/clamav/databases/all-clam.ldb loaded
> ...
> ...
>
> HTH
>
> --
>
> 73,
> Ged.
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
Re: How can we consume .ldb files in ClamAV Ubuntu? [ In reply to ]
Hello,
are those signatures coming from FireEye github already included on the
regular update ?

regards
Luca
Re: How can we consume .ldb files in ClamAV Ubuntu? [ In reply to ]
On 22 December 2020 07:28:53 Luca Sironi via clamav-users
<clamav-users@lists.clamav.net> wrote:
> Hello,
> are those signatures coming from FireEye github already included on the
> regular update ?

Hi...

Joel indicated the other day sigs to detect the problem files are already
in the official Databases :)

Cheers,

Steve
Twitter: @sanesecurity
Re: How can we consume .ldb files in ClamAV Ubuntu? [ In reply to ]
Yes

Sent from my ? iPhone

> On Dec 22, 2020, at 02:30, Luca Sironi via clamav-users <clamav-users@lists.clamav.net> wrote:
>
> ?
> Hello,
> are those signatures coming from FireEye github already included on the regular update ?
>
> regards
> Luca
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
Re: How can we consume .ldb files in ClamAV Ubuntu? [ In reply to ]
Hello Joel, all

sorry if i insist on this topic, i'm still learning the tool.

How can i crosscheck a .ldb file like the one published from Red Eye
with the content of the cvd files i download from clamav?

I tried to unpack those with sigtool but the syntax of the cvd is much more
clear
a signature, a name.

many thanks
Luca
--
http://www.sironi.tk
Re: How can we consume .ldb files in ClamAV Ubuntu? [ In reply to ]
Hi there,

On Wed, 6 Jan 2021, Luca Sironi via clamav-users wrote:

> How can i crosscheck a .ldb file like the one published from Red Eye
> with the content of the cvd files i download from clamav?

Please define "crosscheck". If you mean that you want to check that
two different types of signature store produced by two (or likely
more) different signature writers contain the same signatures for some
malware or other, then be aware that both the names of the signatures
and the signatures themselves are chosen by the writers. There is no
reason to suppose that two different people will choose the same text
for the things that they put in their signature stores, so no reason
why the signatures themselves should be the same, and no reason why
the names of the signatures should even vaguely resemble each other.
The signatures may not even use the same methods of comparison with
the malware. Some signatures will look for things in mail, some for
things in files. There's more, see the documentation about writing
signatures on the ClamAV Website.

If you want to check whether the same malware is detected by two or
more different sets of signatures, then scan a sample of the malware
with one or other of the signature sets loaded.

> I tried to unpack those with sigtool but the syntax of the cvd is
> much more clear a signature, a name.

Your problem is not clear. What did you do? Please show the exact
commands, the resulting output if it is reasonably concise, and why
you didn't like the result. Did you try simply looking at the files
with a pager?

--

73,
Ged.

_______________________________________________

clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml
Re: How can we consume .ldb files in ClamAV Ubuntu? [ In reply to ]
Hello, thank you for your answer.
I understand your point, i guess i should simply trust the project
repository.

I was asked to check whether i could integrate informations coming from

https://github.com/fireeye/red_team_tool_countermeasures/blob/master/all-clam.ldb

with a pre existing clamav installation but i have limited access to the
internet so i could not easily add another CustomerDatabase entry.
So i asked on the ML if that was gonna became part of the standard
repository.
I thought that Red Eye could provide the best signatures to identify binary
stuff they got leaked.

Yes, i was trying to compare the ldb file content with
sigtool --unpack content of daily.cvd and main.cvd

regards
Luca


Il giorno gio 7 gen 2021 alle ore 14:47 G.W. Haywood via clamav-users <
clamav-users@lists.clamav.net> ha scritto:

> Hi there,
>
> On Wed, 6 Jan 2021, Luca Sironi via clamav-users wrote:
>
> > How can i crosscheck a .ldb file like the one published from Red Eye
> > with the content of the cvd files i download from clamav?
>
> Please define "crosscheck". If you mean that you want to check that
> two different types of signature store produced by two (or likely
> more) different signature writers contain the same signatures for some
> malware or other, then be aware that both the names of the signatures
> and the signatures themselves are chosen by the writers. There is no
> reason to suppose that two different people will choose the same text
> for the things that they put in their signature stores, so no reason
> why the signatures themselves should be the same, and no reason why
> the names of the signatures should even vaguely resemble each other.
> The signatures may not even use the same methods of comparison with
> the malware. Some signatures will look for things in mail, some for
> things in files. There's more, see the documentation about writing
> signatures on the ClamAV Website.
>
> If you want to check whether the same malware is detected by two or
> more different sets of signatures, then scan a sample of the malware
> with one or other of the signature sets loaded.
>
> > I tried to unpack those with sigtool but the syntax of the cvd is
> > much more clear a signature, a name.
>
> Your problem is not clear. What did you do? Please show the exact
> commands, the resulting output if it is reasonably concise, and why
> you didn't like the result. Did you try simply looking at the files
> with a pager?
>
> --
>
> 73,
> Ged.
>
> _______________________________________________
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>


--
http://www.sironi.tk