Mailing List Archive

On Sat, Mar 06, 2004 at 08:25:29PM +0100, Werner Koch wrote:

> Hi!
> we just released libgcrypt 1.1.93; available at the usual places.

Hello Werner,
I've noticed from the changelog that:
* The automatic thread library detection has finally been removed.
From now on, only linking explicitely to libgcrypt, libgcrypt-pth
or libgcrypt-pthread is supported.

Doesn't this break libraries like gnutls? There is no way for
gnutls to know a priori which libraries an application would
use. Should the applications that use gnutls deal with libgcrypt
linking directly? Is there any easy way to do the linking with
the proper library, using m4 macro or something?


> Salam-Shalom,
> Werner

--
Nikos Mavroyanopoulos

At Sun, 7 Mar 2004 20:17:19 +0200,
Nikos Mavroyanopoulos wrote:
>
> On Sat, Mar 06, 2004 at 08:25:29PM +0100, Werner Koch wrote:
>
> > Hi!
> > we just released libgcrypt 1.1.93; available at the usual places.
>
> Hello Werner,
> I've noticed from the changelog that:
> * The automatic thread library detection has finally been removed.
> From now on, only linking explicitely to libgcrypt, libgcrypt-pth
> or libgcrypt-pthread is supported.
>
> Doesn't this break libraries like gnutls? There is no way for
> gnutls to know a priori which libraries an application would
> use. Should the applications that use gnutls deal with libgcrypt
> linking directly? Is there any easy way to do the linking with
> the proper library, using m4 macro or something?

We considered the situation, and the issue is fundamental. There is
no solution we know of.

First, the automatic thread detection was a fancy idea, but it isn't
very portable and it is not very reliable either. You could get the
linking wrong easily (esp with libtool) and had no way to know - there
was no error detection at all.

Well, the current situation is at least portable. Plus, you can see
which thread library is supported (and thus its support is active) by
looking at the libraries you link with. That's a small plus.

The problem of mis-linking is unfortunately still very real, and seems
not fixable. You can link against libgcrypt and libgcrypt-pth and get
no error, not even a warning. If anybody knows a solution to this,
we'd very much like to know about it. Even a GNU-specific solution
would be welcome (and given the plethora of options in the GNU world,
there might be one). At least you can easily find out that's the case
by looking at the library dependencies with ldd.

Now, for gnutls. The fact that gcrypt is not thread-safe, and that
you need gcrypt-pthread (or gcrypt-pth) means that you have to make a
hard choice when developing your application or library that uses
libgcrypt. Mind you, you had to do the same choice before, however,
before the choice didn't seem to involve gcrypt (in fact it did,
because it affected the linking order, but it was subtle and easy to
get wrong).

This means that if gnutls needs to be thread safe (and in that case
its maintainers have made the decision and gone a long way to ensure
that the gnutls interface and implementation is thread safe), then it
should link against the appropriate thread-safe version of gcrypt.
Yes, this might mean to have to provide gnutls, gnutls-pth and
gnutls-pthread.

There is an unfortunate side-effect. Mixing libraries will get things
very wrong. If you mix a library that links against libgcrypt and a
library that uses libgcrypt-pth in your application, that's a bad
idea! It might work, but it is officially unsupported (and I mean
it!). It's similar to the problem of linking with mixed versions of
gcrypt, except that it is now in an orthogonal problem space (mixing
thread packages [incl no thread package at all]). The chance someone
accidently linking against pthread and pth is pretty slim. The chance
of someone linking against the non-threaded and a threaded version of
gcrypt is unfortunately very real.

Yes, this is not good. In particular, as it is likely that
applications like KDE suck in gcrypt in the dozens via dependencies
like ldap.

Well, here is a bright side: If we (that means you ;) find a way to
make it work, transparently, and without ill side effects, then we can
revert back to the state of a single libgcrypt for all thread packages
on platforms where this is supported. All three versions of the
library provide the same ABI and API. So, with a few symlinks, you
can then unite them back. However, until we have this, and I want to
stress that we have not made it easy fro ourselves on this issue, we
are stuck to expressing clearly the limitations that exist due to the
nature of how shared libraries work.

Somebody (maybe even we) might monitor how gcrypt is used in
applications and _libraries_ (that's the difficult case), and contact
problematic users. The monitoring part can be automated with a
script.

Thanks,
Marcus

On Tue, Mar 09, 2004 at 01:20:06PM +0100, Marcus Brinkmann wrote:

> We considered the situation, and the issue is fundamental. There is
> no solution we know of.
> First, the automatic thread detection was a fancy idea, but it isn't
> very portable and it is not very reliable either. You could get the
> linking wrong easily (esp with libtool) and had no way to know - there
> was no error detection at all.
Hello Marcus,
Why no have a libgcrypt linked with both pthread and pth, but do no
locking, unless explicitely requested. That is an application that
uses pthread calls something like gcry_enable_pthread_locks() etc.
Is this a viable solution?

Since the only non-reentrant part is the random generator, an other
solution would be to add a thread safe random number generator api
(ie return handle of a pool), so the only one bothered with
locking is the application.

Another solution would be for libgcrypt to fork a rng proccess,
and communicate using IPC with it. This would be certainly thread
safe (however this only works in systems with fork() available).

> Thanks,
> Marcus

--
Nikos Mavroyanopoulos

> Why no have a libgcrypt linked with both pthread and pth, but do no
> locking, unless explicitely requested. That is an application that
> uses pthread calls something like gcry_enable_pthread_locks() etc.
> Is this a viable solution?

It may be better to have a gcry_set_locking_function(), because some
multithreaded applications do not use pthread nor pth. For instance,
about a problem I have with gnutls, I had to develop a clone() based
hack. libgcrypt shouldn't have to bother with the specific threading
implemention the application uses, should it?

> Since the only non-reentrant part is the random generator, an other
> solution would be to add a thread safe random number generator api
> (ie return handle of a pool), so the only one bothered with
> locking is the application.

Why not simply read "/dev/urandom" on OSes where it is available?

--
BBP

At Tue, 9 Mar 2004 15:08:19 +0200,
Nikos Mavroyanopoulos wrote:
>
> On Tue, Mar 09, 2004 at 01:20:06PM +0100, Marcus Brinkmann wrote:
>
> > We considered the situation, and the issue is fundamental. There is
> > no solution we know of.
> > First, the automatic thread detection was a fancy idea, but it isn't
> > very portable and it is not very reliable either. You could get the
> > linking wrong easily (esp with libtool) and had no way to know - there
> > was no error detection at all.
> Hello Marcus,
> Why no have a libgcrypt linked with both pthread and pth, but do no
> locking, unless explicitely requested. That is an application that
> uses pthread calls something like gcry_enable_pthread_locks() etc.
> Is this a viable solution?

Not really. There are two reasons, a technical one and a policy one.
The technical reason is that linking with pthreads usually has serious
side effects. You can't just link an application that was compiled
without pthread with pthread and hope it will work (there have been
problems with that for example with apache and some plugins in the
past). The reason is that pthread replaces library calls like
read/write (IIRC) and signal handling. I don't have a complete
picture what it does exactly, but it's not as with any other library
which you just link to your program and nothing will happen.

The policy reason is that as a consequence, you have to export the
interface to the application if you use gcrypt in a library. IE, a
library like gnutls would have to provide an interface for the
application to switch to the thread library it wants, and so on,
through all intermediate libraries. We don't fancy such a solution
very much. If we wanted it, there would be a good solution indeed:

"Don't link to any thread library, but provide a function to set
callbacks for locking and I/O operations."

This would be a clean solution, but the policy reason above makes it
inelegant.

> Since the only non-reentrant part is the random generator, an other
> solution would be to add a thread safe random number generator api
> (ie return handle of a pool), so the only one bothered with
> locking is the application.

I don't know gcrypt internals to say if that is the only problematic
location in the code. It very well might be (I assume you have
checked :), but there might be others in the future. Plus, we also
have this issue in gpgme, which is I/O intensive and calls
sub-processes, and we would like a common solution if possible (for
maintenance reason). But all of that would not be convincing if the
solution you propose wouldn't have the exact problem as I wrote above,
namely that you'd then have to pass on the interface in every
intermediate library to the application level.

> Another solution would be for libgcrypt to fork a rng proccess,
> and communicate using IPC with it. This would be certainly thread
> safe (however this only works in systems with fork() available).

Mmh, it's an interesting idea. However, it appears to me that any
such solution would end up to be specifically tailored around one
particular issue (random pool in gcrypt here). It's much more
preferable (for obvious reasons) to have a common solution.

All in all, good suggestions. Keep it going! In particular, don't be
shy to tell me if you think my above counter-arguments are not sound
or incomplete. We are very much open for a discussion of this. We
only believe to have thought it all through, we have not proven it :)

Thanks,
Marcus

At Tue, 9 Mar 2004 14:21:49 +0100,
Brieuc Jeunhomme wrote:
>
> > Why no have a libgcrypt linked with both pthread and pth, but do no
> > locking, unless explicitely requested. That is an application that
> > uses pthread calls something like gcry_enable_pthread_locks() etc.
> > Is this a viable solution?
>
> It may be better to have a gcry_set_locking_function(), because some

As stated in my other mail, this interface would have to be passed
through all intermediate libraries. IE, libldap would have to provide
an interface to set the locking function for gcrypt, and two or three
libraries inbetween as well.

> multithreaded applications do not use pthread nor pth. For instance,
> about a problem I have with gnutls, I had to develop a clone() based
> hack. libgcrypt shouldn't have to bother with the specific threading
> implemention the application uses, should it?

In general, a library might need to protect itself against concurrent
access from multiple application threads. So, yes, it has to bother
with the threading model of the application in a very direct sense.

In addition, a library that is used in a user-level thread package
like pth might block in undesirable places (read(), write(),
select()...) and should at these places instead yield to allow other
user-level threads a chance to run. Again, it has to cooperate with
the thread package of the application.

Callbacks are a solution that can work, but with the above interface cost.

> > Since the only non-reentrant part is the random generator, an other
> > solution would be to add a thread safe random number generator api
> > (ie return handle of a pool), so the only one bothered with
> > locking is the application.
>
> Why not simply read "/dev/urandom" on OSes where it is available?

The question is really much broader than getting randomness. If it is
not randomness, it is locking of internal data structures or something
else. In GnuPG, we pull all strings to achieve our goal: For example,
to fix ggcrypt, the global errno has been replaced with error return
values for all functions. This is a natural interface change that
goes a long way towards thread-safeness. However, sometimes this is
not possible, or rather only possible at extreme maintenance and/or
interface complexity costs.

Example: getenv() is not thread-safe. I might be able to fork() and
use I/O to determine the environment variable. That is possible
thread-safe, but expensive and a lot of work (= high costs). It is
also error prone.

Example: You might be able to develop reentrant interfaces. gpg-error
has a function gpg_strerror_r, which is thread-safe. However, it is
much more difficult to use correctly that gpg_strerror. Still, in
this case, we opted for this solution rather than to make gpg-error
thread aware. In other cases, it is important to provide simple and
easy to use interfaces, and do the locking internally. I don't know,
but for an entropy pool encapsulation and opaqueness sounds like a
good thing. If I would work on a user allocated and maintained pool,
the user might accidently provide wrong arguments and destroy any
security he expects to get.

So, every case has to be considered individually, and we are certainly
doing this. Maybe werner can tell you why the random pool in gcrypt
exists, and why it is internal arather than provided and managed by
the user. In any case we are open to suggestions how to do it better,
although I doubt that in gcrypt's case just using /dev/urandom hits
the bullet (if it were that easy, I guess we'd be doing it already).

Thanks,
Marcus

On Tue, 9 Mar 2004 15:08:19 +0200, Nikos Mavroyanopoulos said:

> locking, unless explicitely requested. That is an application that
> uses pthread calls something like gcry_enable_pthread_locks() etc.
> Is this a viable solution?

Because you need to propagate such a call up to the actual application
possible through many library layers. OpenLAP is just one example and
I bet OpenLDAP is also used in other libraries without the users
realizing it.

> Since the only non-reentrant part is the random generator, an other
> solution would be to add a thread safe random number generator api

No it is not. There are several places like self tests, initialization
code and first of all module registration.

> Another solution would be for libgcrypt to fork a rng proccess,
> and communicate using IPC with it. This would be certainly thread
> safe (however this only works in systems with fork() available).

Then you would end up with several RNG processes because libgcrypt
won't be able to track whether one has already been
started. Anyway, the RNG is not the only place where we need at
locking and access to system functions.

Werner

On Tue, 9 Mar 2004 14:21:49 +0100, Brieuc Jeunhomme said:

> Why not simply read "/dev/urandom" on OSes where it is available?

It is actual used as part of our CSPRNG. However /dev/urandom is not
a clean solution for a RNG and the system would badly suffer from a
lack of entropy if we are not able to hand out different strengths of
random to the application.


Werner

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I just wanted to throw in another reason why I like Nikos suggestion to add a
handle for the PRNG. I would like to be able to pass my own PRNG to the
(RSA) key generation. Now, I don't know if that's currently possible (didn't
have the time to look into this so far), but _if_ the PRNG is the only thing
that keeps gcrypt from being thread-safe, the ability to easily replace the
PRNG with external code gives yet another motivation to do this using some
handle (or more specifically, some callback construct which can be conceived
as a handle).

just my 2 cents

Christian

On Tuesday 09 March 2004 08:35 am, Marcus Brinkmann wrote:
> At Tue, 9 Mar 2004 15:08:19 +0200,
>
> Nikos Mavroyanopoulos wrote:
> > Since the only non-reentrant part is the random generator, an other
> > solution would be to add a thread safe random number generator api
> > (ie return handle of a pool), so the only one bothered with
> > locking is the application.
>
> I don't know gcrypt internals to say if that is the only problematic
> location in the code. It very well might be (I assume you have
> checked :), but there might be others in the future. Plus, we also
> have this issue in gpgme, which is I/O intensive and calls
> sub-processes, and we would like a common solution if possible (for
> maintenance reason). But all of that would not be convincing if the
> solution you propose wouldn't have the exact problem as I wrote above,
> namely that you'd then have to pass on the interface in every
> intermediate library to the application level.
>
> > Another solution would be for libgcrypt to fork a rng proccess,
> > and communicate using IPC with it. This would be certainly thread
> > safe (however this only works in systems with fork() available).
>
> Mmh, it's an interesting idea. However, it appears to me that any
> such solution would end up to be specifically tailored around one
> particular issue (random pool in gcrypt here). It's much more
> preferable (for obvious reasons) to have a common solution.
>
> All in all, good suggestions. Keep it going! In particular, don't be
> shy to tell me if you think my above counter-arguments are not sound
> or incomplete. We are very much open for a discussion of this. We
> only believe to have thought it all through, we have not proven it :)
>
> Thanks,
> Marcus
>
> _______________________________________________
> Gcrypt-devel mailing list
> Gcrypt-devel@gnupg.org
> http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQFATdp99tNtMeXQLkIRAq9GAJ92RBsqu19r0T3JwV2/2QDiYdeIwgCfWHyv
XPuJnftyw6lvjp2SFSS6hJc=
=SqM6
-----END PGP SIGNATURE-----

On Tue, 9 Mar 2004 09:53:48 -0500, Christian Grothoff said:

> I just wanted to throw in another reason why I like Nikos suggestion to add a
> handle for the PRNG. I would like to be able to pass my own PRNG to the
> (RSA) key generation. Now, I don't know if that's currently possible (didn't

I guess what you really want is a faster key generation and not a
different RNG ;-). This might be useful for throw-away keys and a new
RNG does not help as there are other things you need to speed up
(first of all the prime generation) For now taking the code from
libgcrypt and tweaking it for this very special purpose seems to be
the best solution - once we got some experience with it, we can add a
new API for that to Libgcrypt.

> have the time to look into this so far), but _if_ the PRNG is the only thing
> that keeps gcrypt from being thread-safe, the ability to easily replace the

No it is no.


BTW, any idea how we can track why the performance on sparc64 is that
bad? I have the hunch that this is due to a bad alignment and
emulation in the kernel.


Werner

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tuesday 09 March 2004 10:36 am, you wrote:
> On Tue, 9 Mar 2004 09:53:48 -0500, Christian Grothoff said:
> > I just wanted to throw in another reason why I like Nikos suggestion to
> > add a handle for the PRNG. I would like to be able to pass my own PRNG
> > to the (RSA) key generation. Now, I don't know if that's currently
> > possible (didn't
>
> I guess what you really want is a faster key generation and not a
> different RNG ;-).

Actually, no. I'm not talking about the PRNG sparc bugs here. I need to
generate an RSA key deterministically (!) from a hash-code. Read: the hash
code is the only source of random bits allowed. This is not the usual
SSL-style use of RSA in GNUnet and not even a digital signature. It's more
like a new cryptographic primitive that we've proposed in our research (see
http://www.ovmj.org/GNUnet/ecrs.ps, look for KBlocks). It has nothing to do
with speed.

> BTW, any idea how we can track why the performance on sparc64 is that
> bad? I have the hunch that this is due to a bad alignment and
> emulation in the kernel.

Actually, I have no clue other than that I seriously doubt that it is an
alignment problem since the CPU load is rather low. The code seems to be
blocking for extended periods of time reading from /dev/?random. But again,
the sparc64 problem is entirely orthogonal to the PRNG issue.

Christian
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQFATeth9tNtMeXQLkIRAi7IAJ9Y0EDDoGdC0P2w/F1SNsmdiGJAWACfd+r7
JVaEYzQQt3m4noncPg226OY=
=NM+5
-----END PGP SIGNATURE-----

On Tue, Mar 09, 2004 at 03:15:21PM +0100, Werner Koch wrote:

> > Since the only non-reentrant part is the random generator, an other
> > solution would be to add a thread safe random number generator api
> No it is not. There are several places like self tests, initialization
> code and first of all module registration.
Those are unimportant enough to deserve such fuss. Initialization code,
and module registration happen only once and at the beginning of the
process so one shouldn't worry about reentrancy. As far as I remember
the self tests, also doesn't matter if they are protected by locks, since
in the worst case, they would be called more than once. In any case the
tests could be moved to the initialization phase if anyone is concerned
about these.

> > Another solution would be for libgcrypt to fork a rng proccess,
> > and communicate using IPC with it. This would be certainly thread
> > safe (however this only works in systems with fork() available).
> Then you would end up with several RNG processes because libgcrypt
> won't be able to track whether one has already been
> started.
So that would be one proccess per application. That's quite fair.
You couldn't have a process per system, since then you'd have to deal
with security as well.

The current approach (1.1.93) makes all of the multithreaded applications
using gnutls crash, that's why i'm most concerned about this.
The worst thing, is that I cannot do anything about it. Even if gnutls
used /dev/(u,a)random or other rngs, it would still fail, since libgcrypt
uses the internal rng in several places. I believe that the previous
behaviour should be kept (since it works), unless a better one is found.

> Werner

--
Nikos Mavroyanopoulos

At Tue, 9 Mar 2004 19:23:12 +0200,
Nikos Mavroyanopoulos wrote:
> The current approach (1.1.93) makes all of the multithreaded applications
> using gnutls crash, that's why i'm most concerned about this.
> The worst thing, is that I cannot do anything about it. Even if gnutls
> used /dev/(u,a)random or other rngs, it would still fail, since libgcrypt
> uses the internal rng in several places. I believe that the previous
> behaviour should be kept (since it works), unless a better one is found.

The previous behaviour did not work very well. We made the experience
in GPGME that people routinely linked against libgpgme with libtool
and have spurious failures because libtool got the linking order
wrong. The same would happen with libgcrypt.

You can only rely on the autodetection if you get the order right at
link time, and there is no way to express that in libtool with such
weak dependencies.

I recently removed the old compatibility mode for libgcrypt. You can
recover it by reverting the following change:

2004-03-01 Marcus Brinkmann <marcus@g10code.de>

* Makefile.am (libgcrypt_la_SOURCES): Add ath.c.
* ath.c (ath_init): Add missing function.

* Makefile.am (ath_pth_src): Removed.
(ath_pthread_src): Removed.
(libgcrypt_la_SOURCES): Remove ath-compat, $(ath_pth_src) and
$(ath_pthread_src).
* ath-compat.c, ath-pth-compat.c, ath-pthread-compat.c: Files
removed.

This will build a libgcrypt that has auto-detection, and libgcrypt-pth
and libgcrypt-pthread without auto-detection.

You will have to make a choice if you want to link gnutls against
libgcrypt-pthread, essentially making pthread a requirement for use of
gnutls, or if you provide several libraries gnutls, gnutls-pthread,
etc.

I always found the dynamic thread detection a bit on the hackish side
(and I wrote it!). The current approach is much more "from the book".
It has well defined functionality and limitations.

Thanks,
Marcus

At Tue, 9 Mar 2004 19:23:12 +0200,
Nikos Mavroyanopoulos wrote:
> The current approach (1.1.93) makes all of the multithreaded applications
> using gnutls crash, that's why i'm most concerned about this.

Maybe some additional data will help with the decision here.

On my system, I have 86 binaries using gnutls (directly or indirectly).
Of these, 81 already link to pthread.

Only five do not link to pthread. These are cupsdoprint, dirmngr,
lynx, mailq, and newaliases.

This is like I expected it. Many programs these days require pthread
anyway.

If this is similar to the whole picture, I don't see much difficulties
in making gnutls require pthread by default, and optionally build a
non-threaded version for those who really want it (gnutls-nothread).

The programs that were compiled with gnutls would probably have to be
recompiled though (if you don't have to change the ABI anyway).

We know that this is not the ideal solution. As I said in one of my
last mails, the perfect solution doesn't seem to exist.

Thanks,
Marcus

At Tue, 09 Mar 2004 19:46:30 +0100,
Marcus Brinkmann wrote:
> The programs that were compiled with gnutls would probably have to be
> recompiled though (if you don't have to change the ABI anyway).

The programs not using pthread already of course (the five).

On Tue, 9 Mar 2004 19:23:12 +0200, Nikos Mavroyanopoulos said:

> Those are unimportant enough to deserve such fuss. Initialization code,
> and module registration happen only once and at the beginning of the
> process so one shouldn't worry about reentrancy. As far as I remember

No. You can make this sure but 2 libraries higher this might not be
true anymore. And there is also the issue of applications like
interpreters, loading modules at will.

> So that would be one proccess per application. That's quite fair.

One process per thread.

> The current approach (1.1.93) makes all of the multithreaded applications
> using gnutls crash, that's why i'm most concerned about this.

A more severe problem is that applications may link to multiple
versions of Libgcrypt (or gnutls) and thus yield unexpected behaviour.

> uses the internal rng in several places. I believe that the previous
> behaviour should be kept (since it works), unless a better one is found.

But resulting in hard to track bugs. Hmmm, we might add the
old way as a fourth variant of libgcrypt (--thread=auto)

Werner



>> Werner

> --
> Nikos Mavroyanopoulos

> _______________________________________________
> Gcrypt-devel mailing list
> Gcrypt-devel@gnupg.org
> http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

On Tue, Mar 09, 2004 at 07:46:30PM +0100, Marcus Brinkmann wrote:

> > The current approach (1.1.93) makes all of the multithreaded applications
> > using gnutls crash, that's why i'm most concerned about this.
> Maybe some additional data will help with the decision here.
> On my system, I have 86 binaries using gnutls (directly or indirectly).
> Of these, 81 already link to pthread.
> Only five do not link to pthread. These are cupsdoprint, dirmngr,
> lynx, mailq, and newaliases.
> This is like I expected it. Many programs these days require pthread
> anyway.
> If this is similar to the whole picture, I don't see much difficulties
> in making gnutls require pthread by default, and optionally build a
> non-threaded version for those who really want it (gnutls-nothread).

This would require other libraries that now rely on gnutls to do
the same. That is modify the cupsys libraries to have a cupsys-thread
and nonthread (and possibly one for the pth), the ldap libraries
as well and several others depending on them.
I don't think that they'll ever go in to that trouble. They'll probably
revert to openssl and let us go (I would do that).


> We know that this is not the ideal solution. As I said in one of my
> last mails, the perfect solution doesn't seem to exist.

In my opinion libgcrypt should be reentrant without knowing the
application's threading method. If this is not possible and
if locking in libgcrypt is here to stay, then
callback locking[0] is much better than the current solution
since only a single library is involved. I know that the drawback
is that two applications, that use different threading methods, when
linked together could not insert both callbacks.
But this is a very rare situation, actually i've never seen such thing happen.


[0]. that is what openssl does currently.

> Thanks,
> Marcus

--
Nikos Mavroyanopoulos

On Tue, Mar 09, 2004 at 07:33:09PM +0100, Marcus Brinkmann wrote:

> You will have to make a choice if you want to link gnutls against
> libgcrypt-pthread, essentially making pthread a requirement for use of
> gnutls, or if you provide several libraries gnutls, gnutls-pthread,
> etc.
> I always found the dynamic thread detection a bit on the hackish side
> (and I wrote it!). The current approach is much more "from the book".
> It has well defined functionality and limitations.
Yes, but it makes things so complex, that noone will bother use it.
It is not viable to have three or more versions of every library that
uses libgcrypt, directly or indirectly.

> Thanks,
> Marcus

--
Nikos Mavroyanopoulos

On Wed, Mar 10, 2004 at 09:22:49AM +0200, Nikos Mavroyanopoulos wrote:

> since only a single library is involved. I know that the drawback
> is that two applications, that use different threading methods, when
> linked together could not insert both callbacks.
> But this is a very rare situation, actually i've never seen such thing happen.
and this would fail even with the current solution.


--
Nikos Mavroyanopoulos

At Wed, 10 Mar 2004 09:22:49 +0200,
Nikos Mavroyanopoulos wrote:
> > If this is similar to the whole picture, I don't see much difficulties
> > in making gnutls require pthread by default, and optionally build a
> > non-threaded version for those who really want it (gnutls-nothread).
>
> This would require other libraries that now rely on gnutls to do
> the same. That is modify the cupsys libraries to have a cupsys-thread
> and nonthread (and possibly one for the pth), the ldap libraries
> as well and several others depending on them.
> I don't think that they'll ever go in to that trouble. They'll probably
> revert to openssl and let us go (I would do that).

What about just requiring pthread?

> > We know that this is not the ideal solution. As I said in one of my
> > last mails, the perfect solution doesn't seem to exist.
>
> In my opinion libgcrypt should be reentrant without knowing the
> application's threading method. If this is not possible and
> if locking in libgcrypt is here to stay, then
> callback locking[0] is much better than the current solution
> since only a single library is involved. I know that the drawback
> is that two applications, that use different threading methods, when
> linked together could not insert both callbacks.
> But this is a very rare situation, actually i've never seen such thing happen.
> [0]. that is what openssl does currently.

And who sets the callbacks? I don't really see how you can do that
without adding a set_locking_callback interface to each and every
library that is involved. If that is "standard" in that those
libraries like cupsys and ldap etc already have such locking
interfaces, I'd be more than fine with such a solution.

If all those other libraries are going to simply set these callbacks
to the pthread interface anyway, I don't see a very strong reason why
you shouldn't require pthread in gnutls, too.

OTOH, I do see your point here. It would cost us little to provide
such callback locking (and I/O) on top of the current model, and it is
a sane extension. We can let both solutions stay in parallel and see
what people use. In fact, it even makes a moderate amount of sense to
have such a callback interface.

So, I guess you have convinced me here.

Thanks,
Marcus

On Wed, Mar 10, 2004 at 11:29:20AM +0100, Marcus Brinkmann wrote:

> > > If this is similar to the whole picture, I don't see much difficulties
> > > in making gnutls require pthread by default, and optionally build a
> > > non-threaded version for those who really want it (gnutls-nothread).
> > This would require other libraries that now rely on gnutls to do
> > the same. That is modify the cupsys libraries to have a cupsys-thread
> > and nonthread (and possibly one for the pth), the ldap libraries
> > as well and several others depending on them.
> > I don't think that they'll ever go in to that trouble. They'll probably
> > revert to openssl and let us go (I would do that).
> What about just requiring pthread?

I don't know how applications that do not use the pthread would behave, as you
said this may have side effects.

> > In my opinion libgcrypt should be reentrant without knowing the
> > application's threading method. If this is not possible and
> > if locking in libgcrypt is here to stay, then
> > callback locking[0] is much better than the current solution
> > since only a single library is involved. I know that the drawback
> > is that two applications, that use different threading methods, when
> > linked together could not insert both callbacks.
> > But this is a very rare situation, actually i've never seen such thing happen.
> > [0]. that is what openssl does currently.
> And who sets the callbacks? I don't really see how you can do that
> without adding a set_locking_callback interface to each and every
> library that is involved. If that is "standard" in that those
> libraries like cupsys and ldap etc already have such locking
> interfaces, I'd be more than fine with such a solution.
> If all those other libraries are going to simply set these callbacks
> to the pthread interface anyway, I don't see a very strong reason why
> you shouldn't require pthread in gnutls, too.
I think (correct me if i'm wrong) that those libraries
do not use nor require pthreads, so they wouldn't bother to set any callback
lock. I've rarely seen libraries that use threads, so it wouldn't do any harm[0].
In most cases the callback will be used by the multithreaded application that
uses those libraries.

[0]. even if this was the case the threading model, of the library and
the application, most probably would be the same so it wouldn't matter
who sets the callback locks.

> OTOH, I do see your point here. It would cost us little to provide
> such callback locking (and I/O) on top of the current model, and it is
> a sane extension. We can let both solutions stay in parallel and see
> what people use. In fact, it even makes a moderate amount of sense to
> have such a callback interface.
> So, I guess you have convinced me here.
Thanks!

> Thanks,
> Marcus

--
Nikos Mavroyanopoulos

At Wed, 10 Mar 2004 13:25:08 +0200,
Nikos Mavroyanopoulos wrote:
> > What about just requiring pthread?
>
> I don't know how applications that do not use the pthread would behave, as you
> said this may have side effects.

Maybe. Usually it is only a problem if you did not link some code
with pthread, and add a pthreaded plugin.

> > If all those other libraries are going to simply set these callbacks
> > to the pthread interface anyway, I don't see a very strong reason why
> > you shouldn't require pthread in gnutls, too.
> I think (correct me if i'm wrong) that those libraries
> do not use nor require pthreads, so they wouldn't bother to set any callback
> lock. I've rarely seen libraries that use threads, so it wouldn't do any harm[0].
> In most cases the callback will be used by the multithreaded application that
> uses those libraries.

This is exactly the worrying case. After all, the application might
not know about the need to set callbacks. But let's discuss this in
the other thread I started.

Thanks,
Marcus