Mailing List Archive

> 3. I could use the ent command which measure the entropy, high
> entropy is an indication of encryption (but jpg have also high
> entropy). However I should then study the distribution of each
> letter to be sure.

A JPEG *body* has high entropy. The JPEG *header* has very low entropy.
That's a relatively good way to spot container files: you look for a
low-entropy header followed by high-entropy data. Zip files, tar.bz2
files, JPEG files, MPEG, the rest, they're all detectable this way.

However, the output of a straight-up block cipher operating in any
modern mode (no ECB!) is going to be totally indistinguishable from a
random number generator for any reasonably-sized file.


_______________________________________________
Gnupg-users mailing list
Gnupg-users@gnupg.org
https://lists.gnupg.org/mailman/listinfo/gnupg-users

Hi,

I just check for a list of ransomware filename patterns (e.g.
*.cryptotorlocker*).

Best regards,
Jan

On 2022-08-04 18:58, Uwe Brauer via Gnupg-users wrote:
>
>
> Hi
>
> I apologize for this message that can be a bit off topic.
> (I am on Ubuntu 16.04)
>
> How can I find say encrypted files in my home directory? The idea is to
> use some magic command together with the find command.
> I know
>
> 1. The file command will return for example for a gpg encrypted file
> file .authinfo.gpg
> .authinfo.gpg: PGP RSA encrypted
>
> 2. However for X509 file I obtain
> file test.p12
> file.p12: data
>
> 3. I could use the ent command which measure the entropy, high
> entropy is an indication of encryption (but jpg have also high
> entropy). However I should then study the distribution of each
> letter to be sure.
>
> So is there any other way to run find and some other script to find
> suspicious files? Google is not really helpful
>
> Regards
>
> Uwe Brauer
>
>
>
> --
> I strongly condemn Putin's war of aggression against the Ukraine.
> I support to deliver weapons to Ukraine's military.
> I support the ban of Russia from SWIFT.
> I support the EU membership of the Ukraine.
>
>
> _______________________________________________
> Gnupg-users mailing list
> Gnupg-users@gnupg.org
> https://lists.gnupg.org/mailman/listinfo/gnupg-users

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Thu, 4 Aug 2022, Jan Eden via Gnupg-users wrote:

> Hi,
>
> I just check for a list of ransomware filename patterns (e.g.
> *.cryptotorlocker*).
>
> Best regards,
> Jan
>
> On 2022-08-04 18:58, Uwe Brauer via Gnupg-users wrote:
>>
>>
>> Hi
>>
>> I apologize for this message that can be a bit off topic.
>> (I am on Ubuntu 16.04)
>>
>> How can I find say encrypted files in my home directory? The idea is to
>> use some magic command together with the find command.
>> I know
>>
>> 1. The file command will return for example for a gpg encrypted file
>> file .authinfo.gpg
>> .authinfo.gpg: PGP RSA encrypted
>>
>> 2. However for X509 file I obtain
>> file test.p12
>> file.p12: data
>>
>> 3. I could use the ent command which measure the entropy, high
>> entropy is an indication of encryption (but jpg have also high
>> entropy). However I should then study the distribution of each
>> letter to be sure.
>>
>> So is there any other way to run find and some other script to find
>> suspicious files? Google is not really helpful
>>
>> Regards
>>
>> Uwe Brauer

Hi Uwe,

my first thought would be to look for compressability (or entropy, as you
suggested) of files. Encrypted files should look like good randomness,
thus not compressable. I would then eliminate the false positives (which
are most likely compressed) by checking their integrity "by protocol" -
i.e. "convert this jpeg to an bmp -> is the bmp (much) bigger than the
jpeg?"

regards,
Erich

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEE3p92iMrPBP64GmxZCu7JB1Xae1oFAmLsF/8ACgkQCu7JB1Xa
e1pojA//RuWb76drpfvfQbcXnqkLhHRwMNPNbdwfZKmxJ4MJTBA+LX9t2is7+mTw
EY7AwyRIAUjV26eOztarEOC2GngKPoIb39VqEK0ilGTT08es41hbSqueIcP5wrH6
3ALUcBcgcT5Rh3pXQGCRYIxPSYmyVx+KUv2525DXePdPizJFU20KGjc3LXatYz9k
KIVaAJnu4+9PcPjFueP0SxrgYOiFkAGCYDqx/NBoNbhzs+5/4s9dIGytJ2CgwZyr
DB3CnjCejvJ25vD1LtiHOBmy5GCYuJAoxlimf5bRTalUF/bsZRcW2UYWp04jIz7S
HI6fl5ASpwX2jtbZgkwgaqKhVfEU4xw3SrTAx89krZDaKM1MbQD8vE1hAmiBuzeb
VygGG1g/s2FZ9kmgHv8TmTOoc9MzDR/aojHcpc4EgcQkRIhRM9GdNYtnUpw8cr0L
E3itanqf+8ZH92BCjNmm+N9tB9VBmPjvXuhWIO6yQF4jOjFUyNbm5Mi3/LvJObys
46vXJPO5o1a//MxjKAS4ly9013PsXsoSpVmCrkPK8qwdy2cxaEAM+jMEDGrot2Um
kSx0e19BdUgXpvOBPVf0Js+UvN3f6mahCLimyhhBoXgd/ievrWOCTI68GHV+izs6
2dL00klyCzQZl0mveNNhfPJDlCEKh0t5CTpMD+mCd0TnvnKDNxM=
=8XRQ
-----END PGP SIGNATURE-----

_______________________________________________
Gnupg-users mailing list
Gnupg-users@gnupg.org
https://lists.gnupg.org/mailman/listinfo/gnupg-users

On Thu, 2022-08-04 at 18:58 +0200, Uwe Brauer via Gnupg-users wrote:
> How can I find say encrypted files in my home directory?

What an interesting exercise! Got me thinking. I'm a total crypto
ignoramus, so take all this with a grain of salt...

I don't think there is any truly reliable way, but a combination of ent
and a relevant expectation might work. For example, if you run ent on a
.txt file, you do not expect to see high entropy, so you would throw
that file up as suspicious. If you run file on a .jpg file, you expect
to see it identified as a JPEG file, so if it is not, you throw it up
as suspicious. Then you manually check files that your system has
identified as suspicious.

Another way to approach it would be to take hashes of all your files
and store the hashes securely (read-only!). You can then compare a
current hash with the known hash, and if the hash has changed, the file
has changed. This is not that good for frequently changing files, but
frequently changing files that are suddenly encrypted are probably
going to be very obvious.

And a third method would be a "canary" or two. Put some tasty-looking
files in your home directory, and regularly check them for changes. If
they ever unexpectedly change, you know to take action.

Anyway - if you come op with a good method, let us know!

Regards, K.

PS: I remember reading a while ago someone writing that as a
technological society advances, its communications become more and more
like random noise, because they will tend to be encrypted and
compressed. The writer was saying this might be one reason we haven't
found life out there - because we can't tell their transmissions apart
from random noise :-)

--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Karl Auer (kauer@biplane.com.au)
http://www.biplane.com.au/kauer

GPG fingerprint: 61A0 99A9 8823 3A75 871E 5D90 BADB B237 260C 9C58
Old fingerprint: 2561 E9EC D868 E73C 8AF1 49CF EE50 4B1D CCA1 5170




_______________________________________________
Gnupg-users mailing list
Gnupg-users@gnupg.org
https://lists.gnupg.org/mailman/listinfo/gnupg-users

>>> "RJHvG" == Robert J Hansen via Gnupg-users <gnupg-users@gnupg.org> writes:

>> 3. I could use the ent command which measure the entropy, high
>> entropy is an indication of encryption (but jpg have also high
>> entropy). However I should then study the distribution of each
>> letter to be sure.

> A JPEG *body* has high entropy. The JPEG *header* has very low
> entropy. That's a relatively good way to spot container files: you
> look for a low-entropy header followed by high-entropy data. Zip
> files, tar.bz2 files, JPEG files, MPEG, the rest, they're all
> detectable this way.

> However, the output of a straight-up block cipher operating in any
> modern mode (no ECB!) is going to be totally indistinguishable from a
> random number generator for any reasonably-sized file.

I see this can can very sophisticated very quickly, but

1. just for the first very rough analysis what is a convenient command to get a list of files that have high entropy?

For example

find . -iname '*.*' -follow -print -exec ent {} \;

Displays to much information that is hard to follow, so I should filter it somehow like

ent test.tex.gpg

| Entropy = 7.997062 bits per byte. | that line could be candidate |
| Optimum compression would reduce the size of this 64224 byte file by 0 percent | another candidate |
| Monte Carlo value for Pi is 3.142376682 (error 0.02 percent) | last candidate |

I also run

Ent test.tex

| Entropy = 5.133812 bits per byte. | candidate |
| Optimum compression would reduce the size of this 214555 byte file by 35 percent | candidate |
| Monte Carlo value for Pi is 3.999888140 (error 27.32 percent) | candidate |


So I am not sure what is the best line, but the question boils down to this, anybody know enough sed or awk or whatsoever to
tell me how ot filter the ent output?

thanks

Uwe Brauer






--
I strongly condemn Putin's war of aggression against the Ukraine.
I support to deliver weapons to Ukraine's military.
I support the ban of Russia from SWIFT.
I support the EU membership of the Ukraine.

On Fri, Aug 05, 2022 at 05:45:53PM +0200, Uwe Brauer via Gnupg-users wrote:
> 1. just for the first very rough analysis what is a convenient command to get a list of files that have high entropy?

The first step might be to install tripwire and only check files, which
tripwire reports as changed. See "man tripwire" after installing it.

Regarding your attempt to find candidate files:

>find . -iname '*.*' -follow -print -exec ent {} \;

Files don't need to have a dot in their name. But they might have unusual
characters in their names instead. So you might actually want to use

find -type f -print0 | xargs -0 ent

Tip: "man find" and "man xargs" describe what those zeroes mean.

>So I am not sure what is the best line, but the question boils down to
>this, anybody know enough sed or awk or whatsoever to tell me how ot filter the ent output?

Gentle suggestion: you'd need to learn such basic usage yourself, before
you rely on them as a tool. especially when attempting to secure your
systems.

Tips (for example):
https://www.amazon.de/Learning-Perl-Making-Things-Possible/dp/1492094951 or
https://www.amazon.de/Effective-awk-Programming-Universal-Processing/dp/1491904615

Regards, JC

--
Experience is the worst teacher. It always gives the test first and the
instruction afterward.


_______________________________________________
Gnupg-users mailing list
Gnupg-users@gnupg.org
https://lists.gnupg.org/mailman/listinfo/gnupg-users

On 2022-08-04 at 18:58 +0200, Uwe Brauer wrote:
>
> Hi
>
> So is there any other way to run find and some other script to find
> suspicious files? Google is not really helpful
>
> Regards
>
> Uwe Brauer

If you suffer a ransomware attack I would say your problem won't be
*noticing* that. If you didn't, that's a failure by the attackers. They
want you to notice (once they're finished), so that they get paid.
Most often, they will change the extension (.ransom, an email
address...) as well as include a ransom note on every directory.

Once you find what pattern they used, it's simple to find all other
files like that.

Regards


_______________________________________________
Gnupg-users mailing list
Gnupg-users@gnupg.org
https://lists.gnupg.org/mailman/listinfo/gnupg-users

On 2022-08-09 22:23, ?ngel wrote:
> On 2022-08-04 at 18:58 +0200, Uwe Brauer wrote:
> >
> > Hi
> >
> > So is there any other way to run find and some other script to find
> > suspicious files? Google is not really helpful
> >
> > Regards
> >
> > Uwe Brauer
>
> If you suffer a ransomware attack I would say your problem won't be
> *noticing* that. If you didn't, that's a failure by the attackers. They
> want you to notice (once they're finished), so that they get paid.
> Most often, they will change the extension (.ransom, an email
> address...) as well as include a ransom note on every directory.
>
> Once you find what pattern they used, it's simple to find all other
> files like that.

I check for certain filename patterns and/or modified files (comparing
to pre-created hashes) before initiating a backup.

Best regards,
Jan

This whole thread is a bit, well cause to ponder ..., and beef a little ...

On Fri, Aug 5, 2022 at 2:40 AM Uwe Brauer via Gnupg-users
<gnupg-users@gnupg.org> wrote:
>
> Hi
>
> I apologize for this message that can be a bit off topic.
> (I am on Ubuntu 16.04)

(Running off to see how much longer that's going to be supported.)

> How can I find say encrypted files in my home directory?

You have encrypted files you aren't tracking? That's a good way to
lose data or whatever was in them.

> The idea is to
> use some magic command together with the find command.
> I know

Magic seems to me to be opposed to the purpose of encryption, but I
guess if that's what you want that's what you want.

> 1. The file command will return for example for a gpg encrypted file
> file .authinfo.gpg
> .authinfo.gpg: PGP RSA encrypted
>
> 2. However for X509 file I obtain
> file test.p12
> file.p12: data
>
> 3. I could use the ent command which measure the entropy, high
> entropy is an indication of encryption (but jpg have also high
> entropy). However I should then study the distribution of each
> letter to be sure.

As has been pointed out, entropy is orthogonal to the question of encryption.

> So is there any other way to run find and some other script to find
> suspicious files? Google is not really helpful

Suspicious files?

Oh. Okay, you or somebody you know has been sloppy and wants to recover.

As you should note from the responses so far, there is no magic solution.

Figure out what is important on the compromised system and work from there.

It used to be a lot simpler, and I could give you a list of steps to
go through, but these days you have to think about compromised BIOS
and compromised media and I/O controllers and such. And the system
with the symptoms is quite possibly not the only compromised system on
your network.

Which I guess may be why you are hoping for magic.

Still, powering the system down, looking for other compromised systems
on the network, removing the media and taking a raw image, deciding
what's important on the compromised media and what can just be thrown
away, etc.

Deciding what's important is an essential step, because you won't know
how to go looking for it if you don't know what you're looking for.

And everything else just has to be tossed -- physically discarded.

Unless you're willing to play craps, in which case, you might consider
paying the people who (hopefully) know where they hid stuff --
although I'd hope you would first consider contacting your local
police or whoever you trust to be able to help, and volunteer to
cooperate in using your data as a trap to catch the miscreants.

--
Joel Rees

http://reiisi.blogspot.jp/p/novels-i-am-writing.html

_______________________________________________
Gnupg-users mailing list
Gnupg-users@gnupg.org
https://lists.gnupg.org/mailman/listinfo/gnupg-users

>>> "JC" == Juergen Christoffel <jc.gnupg18a@unser.net> writes:

> On Fri, Aug 05, 2022 at 05:45:53PM +0200, Uwe Brauer via Gnupg-users wrote:
>> 1. just for the first very rough analysis what is a convenient command to get a list of files that have high entropy?

> The first step might be to install tripwire and only check files, which
> tripwire reports as changed. See "man tripwire" after installing it.


Thanks very much!
> Regarding your attempt to find candidate files:

>> find . -iname '*.*' -follow -print -exec ent {} \;

> Files don't need to have a dot in their name. But they might have unusual
> characters in their names instead. So you might actually want to use

> find -type f -print0 | xargs -0 ent


Well thanks again, but this does not work as expected.
I obtain

,----
| Duplicate file name.
| ent -- Calculate entropy of file. Call
| with ent [options] [input-file]
|
| Options: -b Treat input as a stream of bits
| -c Print occurrence counts
| -f Fold upper to lower case letters
| -t Terse output in CSV format
| -u Print this message
|
| By John Walker
| http://www.fourmilab.ch/
| January 28th, 2008
`----


And adding and of these suggested options does not help

> Tip: "man find" and "man xargs" describe what those zeroes mean.


I try it.

>> So I am not sure what is the best line, but the question boils down to
>> this, anybody know enough sed or awk or whatsoever to tell me how ot filter the ent output?

> Gentle suggestion: you'd need to learn such basic usage yourself, before
> you rely on them as a tool. especially when attempting to secure your
> systems.

> Tips (for example):
> https://www.amazon.de/Learning-Perl-Making-Things-Possible/dp/1492094951 or
> https://www.amazon.de/Effective-awk-Programming-Universal-Processing/dp/1491904615

Thanks my encounters with perl were well unpleasant.

I might, again, try to understand awk better.

Uwe Brauer

--
I strongly condemn Putin's war of aggression against the Ukraine.
I support to deliver weapons to Ukraine's military.
I support the ban of Russia from SWIFT.
I support the EU membership of the Ukraine.