Mailing List Archive

Werner Koch wrote:

> This function registers FUNC_LOG as `logging handler', which means
> that it will be called in case Libgcrypt wants to log a message.
> This function may and should be used prior to calling
> `gcry_check_version'.
>
> This is so that problems during initializaion can be logged.

Hello!
Done!

>
> The real problem however is the use of gcry_set_allocation_handler.
> This installs a new memory allocator defaulting to to standard
> malloc/free. Well, for an application using just gnutls this might not
> be a problem (unless in FIPS mode). However if an application is using
> gnutls directly or indirectly (e.g. through openldap) and also making
> direct use of libgcrypt this will change the standard Libgcrypt memory
> allocators or those set by the actual application. This is a security
> problem because by using a plain malloc and free it is not anymore
> guaranteed that all sensitive data is zeroes out as soon as needed.
>
> If you really, really want to set other Libgcrypt allocation handlers,
> you need to do it in the above initalization block and before setting
> the finished flag. (I'll add an extra sentence to the manual.)

To be honest I don't remember why is this code there. I recollect that
libgcrypt required to set those allocation functions and didn't work
otherwise but this was literally ages ago :) Can libgcrypt work without
setting the memory allocation functions?

regards,
Nikos

_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

Nikos Mavrogiannopoulos <nmav@gnutls.org> writes:

>> The real problem however is the use of gcry_set_allocation_handler.
>> This installs a new memory allocator defaulting to to standard
>> malloc/free. Well, for an application using just gnutls this might not
>> be a problem (unless in FIPS mode). However if an application is using
>> gnutls directly or indirectly (e.g. through openldap) and also making
>> direct use of libgcrypt this will change the standard Libgcrypt memory
>> allocators or those set by the actual application. This is a security
>> problem because by using a plain malloc and free it is not anymore
>> guaranteed that all sensitive data is zeroes out as soon as needed.
>>
>> If you really, really want to set other Libgcrypt allocation handlers,
>> you need to do it in the above initalization block and before setting
>> the finished flag. (I'll add an extra sentence to the manual.)
>
> To be honest I don't remember why is this code there. I recollect that
> libgcrypt required to set those allocation functions and didn't work
> otherwise but this was literally ages ago :) Can libgcrypt work without
> setting the memory allocation functions?

If I remove that code, using any application that uses the GnuTLS
library with some functions just dies:

jas@mocca:~/src/gnutls/src master$ ./certtool -p
Generating a 2048 bit RSA private key...
Ohhhh jeeee: operation is not possible without initialized secure memory
Aborted
jas@mocca:~/src/gnutls/src master$

Is there a recommended way how to resolve this problem?

/Simon


_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

On Thu, 4 Dec 2008 20:37, simon@josefsson.org said:

> If I remove that code, using any application that uses the GnuTLS
> library with some functions just dies:
>
> jas@mocca:~/src/gnutls/src master$ ./certtool -p
> Generating a 2048 bit RSA private key...
> Ohhhh jeeee: operation is not possible without initialized secure memory

Right, because it is not properly initialized. We need to fix the
application and not try to work around such problems.

If you don't have a need for secure memory, for example if your
application does not use secret keys or other confidential data or it
runs in a controlled environment where key material floating around in
memory is not a problem, you - or weel the application - should
initialize Libgcrypt correctly. I recently updated the manual to make
this more clear. Here is an excerpt:

| If you have to protect your keys or other information in memory
| against being swapped out to disk and to enable an automatic overwrite
| of used and freed memory, you need to initialize Libgcrypt this way:
|
| /* Version check should be the very first call because it
| makes sure that important subsystems are intialized. */
| if (!gcry_check_version (GCRYPT_VERSION))
| {
| fputs ("libgcrypt version mismatch\n", stderr);
| exit (2);
| }
|
| /* Disable secure memory. */
| gcry_control (GCRYCTL_DISABLE_SECMEM, 0);
|
| /* ... If required, other initialization goes here. */
|
| /* Tell Libgcrypt that initialization has completed. */
| gcry_control (GCRYCTL_INITIALIZATION_FINISHED, 0);
|
| If you have to protect your keys or other information in memory
| against being swapped out to disk and to enable an automatic overwrite
| of used and freed memory, you need to initialize Libgcrypt this way:
|
| /* Version check should be the very first call because it
| makes sure that important subsystems are intialized. */
| if (!gcry_check_version (GCRYPT_VERSION))
| {
| fputs ("libgcrypt version mismatch\n", stderr);
| exit (2);
| }
|
| /* We don't want to see any warnings, e.g. because we have not yet
| parsed program options which might be used to suppress such
| warnings. */
| gcry_control (GCRYCTL_SUSPEND_SECMEM_WARN);
|
| /* ... If required, other initialization goes here. Note that the
| process might still be running with increased privileges and that
| the secure memory has not been intialized. */
|
| /* Allocate a pool of 16k secure memory. This make the secure memory
| available and also drops privileges where needed. */
| gcry_control (GCRYCTL_INIT_SECMEM, 16384, 0);
|
| /* It is now okay to let Libgcrypt complain when there was/is
| a problem with the secure memory. */
| gcry_control (GCRYCTL_RESUME_SECMEM_WARN);
|
| /* ... If required, other initialization goes here. */
|
| /* Tell Libgcrypt that initialization has completed. */
| gcry_control (GCRYCTL_INITIALIZATION_FINISHED, 0);
|
| It is important that these initialization steps are not done by a
| library but by the actual application. A library using Libgcrypt might
| want to check for finished initialization using:
|
| if (!gcry_control (GCRYCTL_INITIALIZATION_FINISHED_P))
| {
| fputs ("libgcrypt has not been initialized\n", stderr);
| abort ();
| }
|
| Instead of terminating the process, the library may instead print a
| warning and try to initialize Libgcrypt itself. See also the section on
| multi-threading below for more pitfalls.

I am sorry that these things are a bit complicated. However I believe
that we should better choose the safe side of things. The crypto
library is low-level foundation code and problems there would affect a
lot of applications. Better some applications break than applications
requiring real security are exploitable.


Salam-Shalom,

Werner



--
Die Gedanken sind frei. Auschnahme regelt ein Bundeschgesetz.


_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

Werner Koch wrote:
> On Thu, 4 Dec 2008 20:37, simon@josefsson.org said:
>
>> If I remove that code, using any application that uses the GnuTLS
>> library with some functions just dies:
>>
>> jas@mocca:~/src/gnutls/src master$ ./certtool -p
>> Generating a 2048 bit RSA private key...
>> Ohhhh jeeee: operation is not possible without initialized secure memory
>
> Right, because it is not properly initialized. We need to fix the
> application and not try to work around such problems.

I also disagree here :) Have a "secure" memory is only of use to limited
applications that conform to some different threat models than a typical
PC server is faced with. Those special ones know their requirements and
will set this behavior explicitly. For others there should be some
reasonable defaults.

Also if you disagree with my evaluation that common applications do not
require a secure memory, the same argument applies. A reasonable default
with secure memory initialized should be available.

> If you don't have a need for secure memory, for example if your
> application does not use secret keys or other confidential data or it
> runs in a controlled environment where key material floating around in
> memory is not a problem, you - or weel the application - should
> initialize Libgcrypt correctly.

As far as I know modern operating systems do not leave random memory
floating around and being shared between processes. Also memory is being
zeroed before being given to an application. Thus the threat-model for
having a special memory marked as "secure" is not quite clear for me and
this is the reason it was always by default off in gnutls.

> I am sorry that these things are a bit complicated. However I believe
> that we should better choose the safe side of things. The crypto
> library is low-level foundation code and problems there would affect a
> lot of applications. Better some applications break than applications
> requiring real security are exploitable.

I agree choosing the safe side of things. However it should also be
clear what the threat-model of using secure memory will protect from.
The additional security offered by it might not worth the inconvenience
offered by it -limited secure memory that will cause the application to
fail in cases where it was exceeded.

regards,
Nikos

_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

On Fri, 5 Dec 2008 08:50, nmav@gnutls.org said:

> Also if you disagree with my evaluation that common applications do not
> require a secure memory, the same argument applies. A reasonable default
> with secure memory initialized should be available.

There are good defaults available. With recent Linux versions there is
no more need to setuid the application to again access to mlock. Thus
you get mlock-able memory for free. Just make sure to initialize
Libgcrypt properly! Proper initialization is required anyway.

> zeroed before being given to an application. Thus the threat-model for
> having a special memory marked as "secure" is not quite clear for me and
> this is the reason it was always by default off in gnutls.

The threat model is that keys war swapped to disk. You may mitigate
that by using an encrypted swap space - but how many installations do
that? Even OpenBSD does not use encrypted swap by default. Thus
mlock-ed memory is the best solution we have.

You don't think that is an issue for servers? I tend to agree for SSL
keys, however a lot of security policies require zeroization of keys as
early as possible.

Also, the majority of applications using GNuTLS are client applications
and there you really want to safe your keys: User Certificates and keys
used by the application not related to gnutls. For example gnupg uses
gnutls and has a need to keep its keys safe (granted, gnupg uses an
external process to access ldaps and https but other apps might not want
to go into that trouble).

> The additional security offered by it might not worth the inconvenience
> offered by it -limited secure memory that will cause the application to
> fail in cases where it was exceeded.

You see a problem in the limited amount of secure memory when used with
servers? That is a different problem we can solve. However you need to
initialize Libgcrypt properly: In the server case with disabled secure
memory and let the server application do the libgcrypt initialization
and not gnutls. As it stands now, gnutls just overrides good defaults.


Shalom-Salam,

Werner


--
Die Gedanken sind frei. Auschnahme regelt ein Bundeschgesetz.


_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

Werner Koch <wk@gnupg.org> writes:

> On Thu, 4 Dec 2008 20:37, simon@josefsson.org said:
>
>> If I remove that code, using any application that uses the GnuTLS
>> library with some functions just dies:
>>
>> jas@mocca:~/src/gnutls/src master$ ./certtool -p
>> Generating a 2048 bit RSA private key...
>> Ohhhh jeeee: operation is not possible without initialized secure memory
>
> Right, because it is not properly initialized. We need to fix the
> application and not try to work around such problems.

Application? GnuTLS is a library. We don't want to require all
applications that use GnuTLS to call libgcrypt explicitly.

> If you don't have a need for secure memory, for example if your
> application does not use secret keys or other confidential data or it
> runs in a controlled environment where key material floating around in
> memory is not a problem, you - or weel the application - should
> initialize Libgcrypt correctly. I recently updated the manual to make
> this more clear. Here is an excerpt:
>
> | If you have to protect your keys or other information in memory
> | against being swapped out to disk and to enable an automatic overwrite
> | of used and freed memory, you need to initialize Libgcrypt this way:
> |
> | /* Version check should be the very first call because it
> | makes sure that important subsystems are intialized. */
> | if (!gcry_check_version (GCRYPT_VERSION))
> | {
> | fputs ("libgcrypt version mismatch\n", stderr);
> | exit (2);
> | }
> |
> | /* Disable secure memory. */
> | gcry_control (GCRYCTL_DISABLE_SECMEM, 0);
> |
> | /* ... If required, other initialization goes here. */
> |
> | /* Tell Libgcrypt that initialization has completed. */
> | gcry_control (GCRYCTL_INITIALIZATION_FINISHED, 0);

Ok, this would argue for the following solution:

diff --git a/lib/gnutls_global.c b/lib/gnutls_global.c
index f59a47f..2c2a91f 100644
--- a/lib/gnutls_global.c
+++ b/lib/gnutls_global.c
@@ -222,8 +222,7 @@ gnutls_global_init (void)
}

/* for gcrypt in order to be able to allocate memory */
- gcry_set_allocation_handler (gnutls_malloc, gnutls_secure_malloc,
- _gnutls_is_secure_memory, gnutls_realloc, gnutls_free);
+ gcry_control (GCRYCTL_DISABLE_SECMEM, NULL, 0);

gcry_control (GCRYCTL_INITIALIZATION_FINISHED, NULL, 0);

> |
> | If you have to protect your keys or other information in memory
> | against being swapped out to disk and to enable an automatic overwrite
> | of used and freed memory, you need to initialize Libgcrypt this way:

This is the exact same text as above. What is the difference between
these two modes?

> | /* Version check should be the very first call because it
> | makes sure that important subsystems are intialized. */
> | if (!gcry_check_version (GCRYPT_VERSION))
> | {
> | fputs ("libgcrypt version mismatch\n", stderr);
> | exit (2);
> | }
> |
> | /* We don't want to see any warnings, e.g. because we have not yet
> | parsed program options which might be used to suppress such
> | warnings. */
> | gcry_control (GCRYCTL_SUSPEND_SECMEM_WARN);
> |
> | /* ... If required, other initialization goes here. Note that the
> | process might still be running with increased privileges and that
> | the secure memory has not been intialized. */
> |
> | /* Allocate a pool of 16k secure memory. This make the secure memory
> | available and also drops privileges where needed. */
> | gcry_control (GCRYCTL_INIT_SECMEM, 16384, 0);
> |
> | /* It is now okay to let Libgcrypt complain when there was/is
> | a problem with the secure memory. */
> | gcry_control (GCRYCTL_RESUME_SECMEM_WARN);
> |
> | /* ... If required, other initialization goes here. */
> |
> | /* Tell Libgcrypt that initialization has completed. */
> | gcry_control (GCRYCTL_INITIALIZATION_FINISHED, 0);

What does "increased privileges" mean? Does the application needs to be
setuid for this to work? That would also be a non-starter, we can't
require all applications using GnuTLS to be setuid.

> | It is important that these initialization steps are not done by a
> | library but by the actual application.

That seems like a non-starter for GnuTLS. If it is important for
libgcrypt that GnuTLS doesn't initialize libgcrypt, it seems we can't
really use libgcrypt in GnuTLS.

> | A library using Libgcrypt might want to check for finished
> | initialization using:
> |
> | if (!gcry_control (GCRYCTL_INITIALIZATION_FINISHED_P))
> | {
> | fputs ("libgcrypt has not been initialized\n", stderr);
> | abort ();
> | }
> |
> | Instead of terminating the process, the library may instead print a
> | warning and try to initialize Libgcrypt itself.

This is what GnuTLS is doing now, except it is not printing a warning.
What use is there in printing a warning?

> I am sorry that these things are a bit complicated. However I believe
> that we should better choose the safe side of things. The crypto
> library is low-level foundation code and problems there would affect a
> lot of applications. Better some applications break than applications
> requiring real security are exploitable.

The best would that things just work _and_ be secure. I don't see why
it isn't possible to reach that goal here?

/Simon


_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

On Fri, 5 Dec 2008 12:04, simon@josefsson.org said:

> Application? GnuTLS is a library. We don't want to require all
> applications that use GnuTLS to call libgcrypt explicitly.

Right, this is why we have this GCRYCTL_ANY_INITIALIZATION_P feature.
It allows to do an initialization if the application missed to do it.

> Ok, this would argue for the following solution:
>
> diff --git a/lib/gnutls_global.c b/lib/gnutls_global.c
> index f59a47f..2c2a91f 100644
> --- a/lib/gnutls_global.c
> +++ b/lib/gnutls_global.c
> @@ -222,8 +222,7 @@ gnutls_global_init (void)
> }
>
> /* for gcrypt in order to be able to allocate memory */
> - gcry_set_allocation_handler (gnutls_malloc, gnutls_secure_malloc,
> - _gnutls_is_secure_memory, gnutls_realloc, gnutls_free);
> + gcry_control (GCRYCTL_DISABLE_SECMEM, NULL, 0);
>
> gcry_control (GCRYCTL_INITIALIZATION_FINISHED, NULL, 0);

Fine. This is a proper solution. If an applications needs to use
secure memory it can do so and gnutls uses it too. You might want to
state int the manual that gnutls disables the secure memory unless an
application has already initialzied libgcrypt.

> This is the exact same text as above. What is the difference between
> these two modes?

Sorry, cut and paste error. It should read:

If you don't have a need for secure memory, for example if your
application does not use secret keys or other confidential data or it
runs in a controlled environment where key material floating around in
memory is not a problem, you should initialize Libgcrypt this way:

[... GCRYCTL_DISABLE_SECMEM ...]

If you have to protect your keys or other information in memory against
being swapped out to disk and to enable an automatic overwrite of used
and freed memory, you need to initialize Libgcrypt this way:

[... GCRYCTL_INIT_SECMEM ...]


> What does "increased privileges" mean? Does the application needs to be
> setuid for this to work? That would also be a non-starter, we can't
> require all applications using GnuTLS to be setuid.

Depends on the number of pages allocated for the secure memory and the
OS. Current linux version can controll the mlock-able pages with
ulimit.

Right, the application needs to initialize that. A library can't know
how much secure memory is required. Another library initialized later
might have a different idea of the required amount of secure memory and
thus it would be unpredictable. The only solution is that the
application decides and thus initializes libgcrypt.

> That seems like a non-starter for GnuTLS. If it is important for
> libgcrypt that GnuTLS doesn't initialize libgcrypt, it seems we can't
> really use libgcrypt in GnuTLS.

We discussed that in the past ad nauseam when talking about thread
library initialization. In contrast to W32, the shared libray system
used in GNU/Linux is not up to handle certain tasks. The problem is
elevated by using libraries indirectly
(app->libfoo->libldap->libgnutls->libgcrypt->libc) without APP being
aware that it uses libgnutls. As soon as a library needs a specific
global initialization you need to pass the intialization up to the
application.

For libgcrypt we found a somewhat working solution using
GCRYCTL_INITIALIZATION_FINISHED_P. It allows a default to catch the
naive use of crypto without the need for specific initialization.

> This is what GnuTLS is doing now, except it is not printing a warning.
> What use is there in printing a warning?

Except that gnutls does not intialize the secure memory. Printing a
warning is good to inform the application that it should take care of
initialization. I agree that for gnutls such a waning does not make
sense if you change the code as you proposed.

> The best would that things just work _and_ be secure. I don't see why
> it isn't possible to reach that goal here?

See above. Also think of the case if you do not want to use threads -
all will break if you let the libratry decide which threading model to
use. We need some state in libgcrypt: For selftests, for hardware
crypto, for the thread library and so on.


Shalom-Salam,

Werner



--
Die Gedanken sind frei. Auschnahme regelt ein Bundeschgesetz.


_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

Werner Koch <wk@gnupg.org> writes:

> On Fri, 5 Dec 2008 12:04, simon@josefsson.org said:
>
>> Application? GnuTLS is a library. We don't want to require all
>> applications that use GnuTLS to call libgcrypt explicitly.
>
> Right, this is why we have this GCRYCTL_ANY_INITIALIZATION_P feature.
> It allows to do an initialization if the application missed to do it.

Great. I got the impression from the documentation that this isn't the
recommend approach though.

>> Ok, this would argue for the following solution:
>>
>> diff --git a/lib/gnutls_global.c b/lib/gnutls_global.c
>> index f59a47f..2c2a91f 100644
>> --- a/lib/gnutls_global.c
>> +++ b/lib/gnutls_global.c
>> @@ -222,8 +222,7 @@ gnutls_global_init (void)
>> }
>>
>> /* for gcrypt in order to be able to allocate memory */
>> - gcry_set_allocation_handler (gnutls_malloc, gnutls_secure_malloc,
>> - _gnutls_is_secure_memory, gnutls_realloc, gnutls_free);
>> + gcry_control (GCRYCTL_DISABLE_SECMEM, NULL, 0);
>>
>> gcry_control (GCRYCTL_INITIALIZATION_FINISHED, NULL, 0);
>
> Fine. This is a proper solution. If an applications needs to use
> secure memory it can do so and gnutls uses it too. You might want to
> state int the manual that gnutls disables the secure memory unless an
> application has already initialzied libgcrypt.

Yup. And we could also recommend applications to initialize secure
memory if they want.

I'll test the patch above more.

>> This is the exact same text as above. What is the difference between
>> these two modes?
>
> Sorry, cut and paste error. It should read:
>
> If you don't have a need for secure memory, for example if your
> application does not use secret keys or other confidential data or it
> runs in a controlled environment where key material floating around in
> memory is not a problem, you should initialize Libgcrypt this way:
>
> [... GCRYCTL_DISABLE_SECMEM ...]
>
> If you have to protect your keys or other information in memory against
> being swapped out to disk and to enable an automatic overwrite of used
> and freed memory, you need to initialize Libgcrypt this way:
>
> [... GCRYCTL_INIT_SECMEM ...]

Ah, that makes more sense.

>> What does "increased privileges" mean? Does the application needs to be
>> setuid for this to work? That would also be a non-starter, we can't
>> require all applications using GnuTLS to be setuid.
>
> Depends on the number of pages allocated for the secure memory and the
> OS. Current linux version can controll the mlock-able pages with
> ulimit.
>
> Right, the application needs to initialize that. A library can't know
> how much secure memory is required. Another library initialized later
> might have a different idea of the required amount of secure memory and
> thus it would be unpredictable. The only solution is that the
> application decides and thus initializes libgcrypt.

Ok. Using the patch above, and documenting this, seems to be the best
we can achieve then.

>> That seems like a non-starter for GnuTLS. If it is important for
>> libgcrypt that GnuTLS doesn't initialize libgcrypt, it seems we can't
>> really use libgcrypt in GnuTLS.
>
> We discussed that in the past ad nauseam when talking about thread
> library initialization. In contrast to W32, the shared libray system
> used in GNU/Linux is not up to handle certain tasks. The problem is
> elevated by using libraries indirectly
> (app->libfoo->libldap->libgnutls->libgcrypt->libc) without APP being
> aware that it uses libgnutls. As soon as a library needs a specific
> global initialization you need to pass the intialization up to the
> application.
>
> For libgcrypt we found a somewhat working solution using
> GCRYCTL_INITIALIZATION_FINISHED_P. It allows a default to catch the
> naive use of crypto without the need for specific initialization.

Still, it seems like it would be possible to implement this in libgcrypt
without the requirement to modify applications or workarounds.

>> This is what GnuTLS is doing now, except it is not printing a warning.
>> What use is there in printing a warning?
>
> Except that gnutls does not intialize the secure memory. Printing a
> warning is good to inform the application that it should take care of
> initialization. I agree that for gnutls such a waning does not make
> sense if you change the code as you proposed.

OK.

>> The best would that things just work _and_ be secure. I don't see why
>> it isn't possible to reach that goal here?
>
> See above. Also think of the case if you do not want to use threads -
> all will break if you let the libratry decide which threading model to
> use. We need some state in libgcrypt: For selftests, for hardware
> crypto, for the thread library and so on.

OK.

/Simon


_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel