Mailing List Archive

On Thu, 23 Jan 2003 15:14:17 +0100, Simon Josefsson said:

> gcry_control() prints warnings to stderr if called multiple times,
> which isn't always possible to avoid (consider two shared libraries
> each using libgcrypt without knowing about each other, linked into the
> same application). Would disabling the error message cause any
> problems?

No.

However you should never see this message because libgcrypt should
make sure that it is initialized only once - as early as possible and
definitely before you create another thread. The application must
decide whether secure memory is required or not. If there is no need
for it; the following code should be used:

/* Version check should be the very first gcry call because it
makes sure that constructor functrions are run. */
if (!gcry_check_version (GCRYPT_VERSION))
die ("version mismatch\n");
/* Many applications don't require secure memory, so they should
disable it right away. There won't be a problem unless one makes
use of a feature which requires secure memoery - in that case the
process would abort becuase the secmem is not initialized. */
gcry_control (GCRYCTL_DISABLE_SECMEM, 0);

/* .. add whatever initialization you want, but better don't make calls
to libgcrypt from more than one thread ... */

/* Tell Libgcrypt that initialization has completed. */
gcry_control (GCRYCTL_INITIALIZATION_FINISHED, 0);


If you require secure memory, this code should be used:

if (!gcry_check_version (GCRYPT_VERSION))
die ("version mismatch\n");
/* We don't want to see any warnings, e.g. because we have not yet
parsed options which might be used to suppress such warnings */
gcry_control (GCRYCTL_SUSPEND_SECMEM_WARN);

/* ... */

/* Allocate a pool of 16k secure memory. This also drops priviliges
on some systems. */
gcry_control (GCRYCTL_INIT_SECMEM, 16384, 0);

/* It is now okay to let Libgcrypt complain when there was/is a problem
with the secure memory. */
gcry_control (GCRYCTL_RESUME_SECMEM_WARN);

/* Tell Libgcrypt that initialization has completed. */
gcry_control (GCRYCTL_INITIALIZATION_FINISHED, 0);


This sounds a bit complicated but has the advantage that the caller
must decide whether he wants secure memory or not - there is no
default.

It is important that this initialization is not done by a library but
in the application. The library might want to check for finished
initialization using:

if (!gcry_control (GCRYCTL_INITIALIZATION_FINISHED_P))
return MYLIB_ERROR_LIBGCRYPT_NOT_INITIALIZED;

or as GNUTLS does it (while still single-threaded):

if (gcry_control( GCRYCTL_ANY_INITIALIZATION_P) == 0) {
/* for gcrypt in order to be able to allocate memory */
gcry_set_allocation_handler(gnutls_malloc, gnutls_secure_malloc,
_gnutls_is_secure_memory,
gnutls_realloc,
gnutls_free);
gcry_control (GCRYCTL_INITIALIZATION_FINISHED, NULL,0);
}


So the bottom line is that IMHO the message is useful.


Salam-Shalom,

Werner

Werner Koch <wk@gnupg.org> writes:

> On Thu, 23 Jan 2003 15:14:17 +0100, Simon Josefsson said:
>
>> gcry_control() prints warnings to stderr if called multiple times,
>> which isn't always possible to avoid (consider two shared libraries
>> each using libgcrypt without knowing about each other, linked into the
>> same application). Would disabling the error message cause any
>> problems?
>
> No.
>
> However you should never see this message because libgcrypt should
> make sure that it is initialized only once - as early as possible and
> definitely before you create another thread. The application must
> decide whether secure memory is required or not. If there is no need
> for it; the following code should be used:
>
> /* Version check should be the very first gcry call because it
> makes sure that constructor functrions are run. */
> if (!gcry_check_version (GCRYPT_VERSION))
> die ("version mismatch\n");
> /* Many applications don't require secure memory, so they should
> disable it right away. There won't be a problem unless one makes
> use of a feature which requires secure memoery - in that case the
> process would abort becuase the secmem is not initialized. */
> gcry_control (GCRYCTL_DISABLE_SECMEM, 0);
>
> /* .. add whatever initialization you want, but better don't make calls
> to libgcrypt from more than one thread ... */
>
> /* Tell Libgcrypt that initialization has completed. */
> gcry_control (GCRYCTL_INITIALIZATION_FINISHED, 0);
>
>
> If you require secure memory, this code should be used:
>
> if (!gcry_check_version (GCRYPT_VERSION))
> die ("version mismatch\n");
> /* We don't want to see any warnings, e.g. because we have not yet
> parsed options which might be used to suppress such warnings */
> gcry_control (GCRYCTL_SUSPEND_SECMEM_WARN);
>
> /* ... */
>
> /* Allocate a pool of 16k secure memory. This also drops priviliges
> on some systems. */
> gcry_control (GCRYCTL_INIT_SECMEM, 16384, 0);
>
> /* It is now okay to let Libgcrypt complain when there was/is a problem
> with the secure memory. */
> gcry_control (GCRYCTL_RESUME_SECMEM_WARN);
>
> /* Tell Libgcrypt that initialization has completed. */
> gcry_control (GCRYCTL_INITIALIZATION_FINISHED, 0);
>
>
> This sounds a bit complicated but has the advantage that the caller
> must decide whether he wants secure memory or not - there is no
> default.
>
> It is important that this initialization is not done by a library but
> in the application. The library might want to check for finished
> initialization using:
>
> if (!gcry_control (GCRYCTL_INITIALIZATION_FINISHED_P))
> return MYLIB_ERROR_LIBGCRYPT_NOT_INITIALIZED;
>
> or as GNUTLS does it (while still single-threaded):
>
> if (gcry_control( GCRYCTL_ANY_INITIALIZATION_P) == 0) {
> /* for gcrypt in order to be able to allocate memory */
> gcry_set_allocation_handler(gnutls_malloc, gnutls_secure_malloc,
> _gnutls_is_secure_memory,
> gnutls_realloc,
> gnutls_free);
> gcry_control (GCRYCTL_INITIALIZATION_FINISHED, NULL,0);
> }
>
>
> So the bottom line is that IMHO the message is useful.

It gets complex when 1) the library that uses libgcrypt only
optionally links with libgcrypt (compile time check) and 2) the
library using libgcrypt decides at runtime whether libgcrypt, and in
particular secure memory, is needed at all, and 3) the application
implements a network protocol where the stderr print will confuse the
implementation at the other end and cause it to drop the connection.

I'd prefer to hide these details in the library and not have libgcrypt
print anything to stdout or stderr. The
MYLIB_ERROR_LIBGCRYPT_NOT_INITIALIZED approach is closest to working,
but it requires all applications using my library to set up autoconf
tests and code for libgcrypt, even though it might not be needed. The
application would have to ask my library for how much secure memory it
should request as well.

Maybe introducing a GCRYCTL_DISABLE_ALREADY_INIT_SECMEM_WARN gcryctl()
flag would solve it without putting requirements on applications?

Perhaps there is another easier solution.

(I'm having a deja vu about this, perhaps it has been discussed
already.)

On Thu, 23 Jan 2003 18:31:21 +0100, Simon Josefsson said:

> It gets complex when 1) the library that uses libgcrypt only
> optionally links with libgcrypt (compile time check) and 2) the

That's easy, you can #ifdefs that or use ELF magic.

> library using libgcrypt decides at runtime whether libgcrypt, and in
> particular secure memory, is needed at all, and 3) the application

You can't do that because with the Linux kernel you need to run the
program setuid and drop this privilige as ASAP. There is no way to
enclose the secure memory use in a library.

> implements a network protocol where the stderr print will confuse the
> implementation at the other end and cause it to drop the connection.

I can see that. The way to avoid this is by setting up an error
handler:

static void
my_gcry_logger (void *dummy, int level, const char *fmt, va_list arg_ptr)
{
/* optionally do something with LEVEL. */
vfprintf (whereever, fmt, arg_ptr);
}

gcry_set_log_handler (my_gcry_logger, NULL);

Well, there is again the problem who should set this up: the library
or the application.

> Maybe introducing a GCRYCTL_DISABLE_ALREADY_INIT_SECMEM_WARN gcryctl()
> flag would solve it without putting requirements on applications?

Again, if you get this message, something is going wrong. Libgcrypt
should only be initialzed once. If you do the init ala GNUTLS,
everything should work fine as long as only one library wants to use
Libgcrypt - it won't work if there is another library which does not
care about setting a different log handler. Think of GnuCash with its
58 required libs (even more in newer version) - It is likely that
there are 2 libs which like to make use of Libgcrypt.

So what I propose is to do a default init in your library and document
what an application should better do; i.e. initialize Libgcrypt if
at all possible.


Salam-Shalom,

Werner

On Thu, Jan 23, 2003 at 07:04:55PM +0100, Werner Koch wrote:

> > library using libgcrypt decides at runtime whether libgcrypt, and in
> > particular secure memory, is needed at all, and 3) the application
> You can't do that because with the Linux kernel you need to run the
> program setuid and drop this privilige as ASAP. There is no way to
> enclose the secure memory use in a library.

I believe that secure memory should be something exceptional, and not
to be considered standard use of libgcrypt. The benefits are good, but
the problems it has overcome them. Since libgcrypt is usually used by
higher level libraries and probably they are used by a more high interface
library.

Let's say the gnutls uses libgcrypt, and libkoko uses gnutls to provide a
secure a game connection over the net. We cannot expect the game developer
to properly initialize libgcrypt. The libraries must be able to easily and properly
initialize libgcrypt (without secure memory). If someone wants secure memory
and stuff he should initialize himself libgcrypt. The current situation
is quite complicated, and can be simplified much by disabling secure memory
by default.

[...]
> gcry_set_log_handler (my_gcry_logger, NULL);
>
> Well, there is again the problem who should set this up: the library
> or the application.

Also printing errors to stderr by default is dangerous (stderr is actually
fd 2, and in servers it is not stderr). Libgcrypt might write "please do some
input ..." in someone's database tables. That's why I set the libgcrypt's
verbosity to 0 in gnutls. I believe that a low level library should not expect
input nor output to the user.

> Salam-Shalom,
> Werner

--
Nikos Mavroyanopoulos

On Thu, Jan 23, 2003 at 07:04:55PM +0100, Werner Koch wrote:

> > library using libgcrypt decides at runtime whether libgcrypt, and in
> > particular secure memory, is needed at all, and 3) the application
> You can't do that because with the Linux kernel you need to run the
> program setuid and drop this privilige as ASAP. There is no way to
> enclose the secure memory use in a library.

I believe that secure memory should be something exceptional, and not
to be considered standard use of libgcrypt. The benefits are good, but
the problems it has overcome them. Since libgcrypt is usually used by
higher level libraries and probably they are used by a more high interface
library.

Let's say the gnutls uses libgcrypt, and libkoko uses gnutls to provide a
secure a game connection over the net. We cannot expect the game developer
to properly initialize libgcrypt. The libraries must be able to easily and properly
initialize libgcrypt (without secure memory). If someone wants secure memory
and stuff he should initialize himself libgcrypt. The current situation
is quite complicated, and can be simplified much by disabling secure memory
by default.

[...]
> gcry_set_log_handler (my_gcry_logger, NULL);
>
> Well, there is again the problem who should set this up: the library
> or the application.

Also printing errors to stderr by default is dangerous (stderr is actually
fd 2, and in servers it is not stderr). Libgcrypt might write "please do some
input ..." in someone's database tables. That's why I set the libgcrypt's
verbosity to 0 in gnutls. I believe that a low level library should not expect
input nor output to the user.

> Salam-Shalom,
> Werner

--
Nikos Mavroyanopoulos

On Thu, 23 Jan 2003 22:32:44 +0200, Nikos Mavroyanopoulos said:

> I believe that secure memory should be something exceptional, and not
> to be considered standard use of libgcrypt. The benefits are good, but

I agree with you.

> and stuff he should initialize himself libgcrypt. The current situation
> is quite complicated, and can be simplified much by disabling secure memory
> by default.

However, this does not solve other initialization problems. Well, we
could use constructors to trigger them but they are not available on
every platform. For a single threaded application we don't have any
problems but we can get race conditions in a multi threaded
application.

Okay, I will disable secure memory be default.

> Also printing errors to stderr by default is dangerous (stderr is actually
> fd 2, and in servers it is not stderr). Libgcrypt might write "please do some

I have to disagree here. stderr is the designated stream for error
messages and all well behaving software does it like this.
Applications which take stderr as data input are broken (telnet is not
a problem anymore since ssh gets it right). Even libc writes to
stderr, I don't even need to talk about gnome or kde libs.

OTOH, a library should only print severe error messages and that is
what Libgcrypt does - double initialization is a bug and it is better
to let the programmer/user know about this. Such diagnostics are
essential for bug tracking.

Regarding the "not enough random bytes available (need %d bytes)\n"
message, you are right. It should be replaced by a callback mechanism.

On Fri, Jan 24, 2003 at 10:58:00AM +0100, Werner Koch wrote:

> > and stuff he should initialize himself libgcrypt. The current situation
> > is quite complicated, and can be simplified much by disabling secure memory
> > by default.
> However, this does not solve other initialization problems. Well, we
> could use constructors to trigger them but they are not available on
> every platform. For a single threaded application we don't have any
> problems but we can get race conditions in a multi threaded
> application.

Is calling gcry_control (GCRYCTL_INITIALIZATION_FINISHED, NULL,0)
sufficient for the other initialization stuff? If yes then
the only code that a library has to call is
if (gcry_control( GCRYCTL_ANY_INITIALIZATION_P) == 0)
gcry_control(GCRYCTL_INITIALIZATION_FINISHED, NULL,0)

This can be easily moved to a function on startup that is
not thread safe (ie in gnutls is gnutls_global_init).
Other libraries should handle it similary, and there will be
no race conditions.

> > Also printing errors to stderr by default is dangerous (stderr is actually
> > fd 2, and in servers it is not stderr). Libgcrypt might write "please do some

> I have to disagree here. stderr is the designated stream for error
> messages and all well behaving software does it like this.
> Applications which take stderr as data input are broken (telnet is not
> a problem anymore since ssh gets it right). Even libc writes to
> stderr, I don't even need to talk about gnome or kde libs.
> OTOH, a library should only print severe error messages and that is
> what Libgcrypt does - double initialization is a bug and it is better
> to let the programmer/user know about this. Such diagnostics are
> essential for bug tracking.

Ok, but I think that these severe errors should be followed by an
abort() call or something like that. Other errors should only be
printed by the callback mechanism (which is available).

> _______________________________________________
> Gcrypt-devel mailing list
> Gcrypt-devel@gnupg.org
> http://lists.gnupg.org/mailman/listinfo/gcrypt-devel
>

--
Nikos Mavroyanopoulos

On Fri, 24 Jan 2003 14:10:40 +0200, Nikos Mavroyanopoulos said:

> the only code that a library has to call is
> if (gcry_control( GCRYCTL_ANY_INITIALIZATION_P) == 0)
> gcry_control(GCRYCTL_INITIALIZATION_FINISHED, NULL,0)

That is basically what you do in GNUTLS and it is what I consider a
good fallback action.

> Other libraries should handle it similary, and there will be
> no race conditions.

Right.

> Ok, but I think that these severe errors should be followed by an
> abort() call or something like that. Other errors should only be
> printed by the callback mechanism (which is available).

It is not a really hard error but may indicate other problems. This
is a problem of the caller and not necessary of libgcrypt, so I don't
call BUG () as in other places. Consider you have widely deployed an
application and suddenly (e.g. due to a date problem) it aborts all
the time due to such an abort - you won't have a chance for a good
workaround even if is only a minor problem or no real problem at all.

Ask the Ariane engineers about such stuff ;-)


Shalom-Salam,

Werner

Werner Koch <wk@gnupg.org> writes:

>> library using libgcrypt decides at runtime whether libgcrypt, and in
>> particular secure memory, is needed at all, and 3) the application
>
> You can't do that because with the Linux kernel you need to run the
> program setuid and drop this privilige as ASAP. There is no way to
> enclose the secure memory use in a library.

That seems bad. Is there any work going on to make it possible for
user-level code to allocate secure memory from the kernel (possibly
via libc)?

If not, I fear that making applications setuid in order for secure
memory to be available creates more security problems than it solves.

>> implements a network protocol where the stderr print will confuse the
>> implementation at the other end and cause it to drop the connection.
>
> I can see that. The way to avoid this is by setting up an error
> handler:
>
> static void
> my_gcry_logger (void *dummy, int level, const char *fmt, va_list arg_ptr)
> {
> /* optionally do something with LEVEL. */
> vfprintf (whereever, fmt, arg_ptr);
> }
>
> gcry_set_log_handler (my_gcry_logger, NULL);
>
> Well, there is again the problem who should set this up: the library
> or the application.

Yes, and ideally I'd like to hide these details from the application.

Put concisely, I guess what I want is this: If secure memory cannot be
used without support from the application using my library, I want to
use libgcrypt without secure memory, and for this to happen silently.

> So what I propose is to do a default init in your library and document
> what an application should better do; i.e. initialize Libgcrypt if
> at all possible.

This is what I'll do, with the "default init" being:

if (gcry_control (GCRYCTL_ANY_INITIALIZATION_P) == 0)
{
if (gcry_check_version (GCRYPT_VERSION) == NULL)
return GSASL_GCRYPT_ERROR;
if (gcry_control (GCRYCTL_DISABLE_SECMEM, NULL, 0) != GCRYERR_SUCCESS)
return GSASL_GCRYPT_ERROR;
if (gcry_control(GCRYCTL_INITIALIZATION_FINISHED,
NULL, 0) != GCRYERR_SUCCESS)
return GSASL_GCRYPT_ERROR;
}

Quick testing suggests it does what I want.

Thanks.

On Sat, 25 Jan 2003 17:29:52 +0100, Simon Josefsson said:

> That seems bad. Is there any work going on to make it possible for
> user-level code to allocate secure memory from the kernel (possibly
> via libc)?

A long time ago, a patch was proposed to allow for an ulimit
controlled amount of non-pageable memory but rejected by Linux. I
talked with David Miller about this but he also objected and said that
the (at that time new) capabilities should be used for this - well
this does not solve the library problem.

> If not, I fear that making applications setuid in order for secure
> memory to be available creates more security problems than it solves.

I general I think this secure memory is not really required and
operations with sensitive data should better be handled by a dedicated
process, like gpg-agent.

> used without support from the application using my library, I want to
> use libgcrypt without secure memory, and for this to happen silently.

I am going to change it in this way.

> This is what I'll do, with the "default init" being:

Looks okay.


Salam-Shalom,

Werner

Werner Koch <wk@gnupg.org> writes:

>> If not, I fear that making applications setuid in order for secure
>> memory to be available creates more security problems than it solves.
>
> I general I think this secure memory is not really required

My sentiments exactly. People worried about the attack model can fix
their OS to never leave deallocated memory around. It should be more
reliable than auditing all code to make sure it never stores (possibly
derived) secrets in ordinary memory.

> and operations with sensitive data should better be handled by a
> dedicated process, like gpg-agent.

Can the gpg-agent, despite its name, be used by non-gpg applications
too? Offering to handle all user interactions within my library (via
gpg-agent) instead of bothering the application to implement similar
functionality would be useful.

Werner Koch <wk@gnupg.org> writes:

> On Sat, 25 Jan 2003 21:19:52 +0100, Simon Josefsson said:
>
>> Can the gpg-agent, despite its name, be used by non-gpg applications
>> too? Offering to handle all user interactions within my library (via
>> gpg-agent) instead of bothering the application to implement similar
>> functionality would be useful.
>
> Yes. It is basically protocol neutral and if you miss something, it
> should be easy to add what you require.

After thinking about it, it seems I want to input usernames, realms
and possibly more stuff (e.g. mechanism selection, quality of
protection, maximum encryption buffer size). Would adding support for
such stuff be stretching the goal of gpg-agent too much? A feature to
remember passwords and other details, i.e. making it a "wallet" and
storing passwords for different user application (gpg, ssh, sasl,
kerberos), would be useful too.

What I'd like seems to be a generally useful application, so maybe it
already exists. Pointers appreciated.