Mailing List Archive: Questions about multi-threading

Questions about multi-threading

Jul 5, 2006, 3:19 PM

Post #1 of 6 (2327 views)

Hi,

I'm working on the firewall NuFW [1]. I used gprof to find functions which
takes a lot of time. I'm not sure, but I think that gcrypt really uses too
much mutexes lock() and unlock(). I read gcrypt source code, and used gdb
on server to catch lock() calls.

I didn't understood everything, but I have some questions.

(a) Why do ath_mutex_lock() and ath_mutex_unlock() call mutex_init()? It
looks very weird for me, since this function use another mutex!

(b) Why do ath_mutex_destroy() call mutex_init()??? It's stupid to create
a mutex if it doesn't exist ... to destroy it just after its creation!?

(c) Can a mutex lock() or unlock() fails? (or: it is necessary to check
error code?)

(d) Would it be possible to make random() functions (eg.
gcry_random_bytes() and gcry_create_nonce()) really thread safe? (can be
called in two different threads at the same time and not block one the
call until the first ends)

(e) Why using REGISTER_(...); macro and not simply call them in a function
like "gcrypt_global_init();"? Eg. "REGISTER_DEFAULT_DIGESTS" in
cipher/md.c, it uses a mutex to check if digests are already registred or
not. "REGISTER_DEFAULT_DIGESTS" is called in 7 different functions, but is
it really needed?

[1] Firewall NuFW
http://www.nufw.org/

Victor Stinner

_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

Re: Questions about multi-threading [ In reply to ]

marcus.brinkmann at ruhr-uni-bochum

Jul 5, 2006, 11:58 PM

Post #2 of 6 (2252 views)

Permalink

At Thu, 6 Jul 2006 00:19:52 +0200 (CEST),
haypo@inl.fr wrote:
>
> Hi,
>
> I'm working on the firewall NuFW [1]. I used gprof to find functions which
> takes a lot of time. I'm not sure, but I think that gcrypt really uses too
> much mutexes lock() and unlock(). I read gcrypt source code, and used gdb
> on server to catch lock() calls.
>
> I didn't understood everything, but I have some questions.
>
> (a) Why do ath_mutex_lock() and ath_mutex_unlock() call mutex_init()? It
> looks very weird for me, since this function use another mutex!

For static initialization to work (using ATH_MUTEX_INITIALIZER).

> (b) Why do ath_mutex_destroy() call mutex_init()??? It's stupid to create
> a mutex if it doesn't exist ... to destroy it just after its creation!?

Yeah, that seems to be superfluous. However, I don't think this is a
big performance difference (or do you have data indicating
otherwise?).

We at least need to check if it is initialized or not. I agree that
the initialization could be optimized out in the case the the mutex
wasn't initialized and is going to be destroyed. However, this
increases code complexity a bit for a doubtful performance gain: How
often do you destroy a mutex that's not initialized?

Anyway, if you think that should be changed, feel free to send in a patch.

> (c) Can a mutex lock() or unlock() fails? (or: it is necessary to check
> error code?)

Depends on the thread implementation and configuration of the mutex.
We do not check for the internal check_init_lock. Doing so would make
the code a tiny bit more robust. Want to send in a patch?

> (d) Would it be possible to make random() functions (eg.
> gcry_random_bytes() and gcry_create_nonce()) really thread safe? (can be
> called in two different threads at the same time and not block one the
> call until the first ends)

I think only by either using two entropy pools, thereby underutilizing
a scarce resource, or by using a second process.

> (e) Why using REGISTER_(...); macro and not simply call them in a function
> like "gcrypt_global_init();"? Eg. "REGISTER_DEFAULT_DIGESTS" in
> cipher/md.c, it uses a mutex to check if digests are already registred or
> not. "REGISTER_DEFAULT_DIGESTS" is called in 7 different functions, but is
> it really needed?

Not my code, so I can't comment :)

Thanks,
Marcus

_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

Re: Questions about multi-threading [ In reply to ]

wk at gnupg

Jul 6, 2006, 5:42 AM

Post #3 of 6 (2264 views)

Permalink

On Thu, 6 Jul 2006 00:19, haypo@inl.fr said:

> (d) Would it be possible to make random() functions (eg.
> gcry_random_bytes() and gcry_create_nonce()) really thread safe? (can be
> called in two different threads at the same time and not block one the
> call until the first ends)

That is not possible. Ramndom is a scare resource and trhus we
serialize access. Having separate instances of the random pool makes
things actually slower, because they need to be really separate and
can share nothing, in particular not any entropy used as seed.

The latest libgcrypt from SVN features a random daemon which might
be helpful for you. There is not much documentatiion yet and
definitely not well tested; you might want to try it out. You need to
call gcry_control (GCRYCTL_USE_RANDOM_DAEMON, 1) tight after
gcry_check_version to enable use of the daemon - there is a fallback
to the internal RNG if the daemon is not running.

> (e) Why using REGISTER_(...); macro and not simply call them in a function
> like "gcrypt_global_init();"? Eg. "REGISTER_DEFAULT_DIGESTS" in
> cipher/md.c, it uses a mutex to check if digests are already registred or
> not. "REGISTER_DEFAULT_DIGESTS" is called in 7 different functions, but is
> it really needed?

The idea behind the module system is to allow interpreters to load new
algorithms on demand and thus full fledged locking is required.

I think that it is a bit over designed and we may drop this with the
next version. I doubt that the module system is actually used very
often. However, this will be an ABI break.

Salam-Shalom,

Werner

_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

Re: Questions about multi-threading [ In reply to ]

haypo at inl

Jul 6, 2006, 8:11 AM

Post #4 of 6 (2259 views)

Permalink

Hi,

>> (d) Would it be possible to make random() functions (...)
>> really thread safe? (...)
>
> That is not possible. (...)

Ok, I understand. It was just a question :-)

> The latest libgcrypt from SVN features a random daemon which might be
helpful for you.

Nice, I may try it since I need quickly a lot of entropy (create 100
gnutls connections or more).

>> (e) Why using REGISTER_(...); macro (...)
>
> The idea behind the module system is to allow interpreters to load new
algorithms on demand and thus full fledged locking is required.

Hum, I'm not sure that you understood my question. It's a good idea to be
able to add new modules. But my question was: why try to register default
plugins in all functions since one call will be enough? Check my
proposition (two patches) on mailing list. I replaced all "register
default ..." with *one* code only called once.

> However, this will be an ABI break.

My patches don't break ABI nor API.

Haypo

_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

Re: Questions about multi-threading [ In reply to ]

marcus.brinkmann at ruhr-uni-bochum

Jul 7, 2006, 2:08 AM

Post #5 of 6 (2247 views)

Permalink

At Thu, 6 Jul 2006 17:11:47 +0200 (CEST),
haypo@inl.fr wrote:
> > However, this will be an ABI break.
>
> My patches don't break ABI nor API.

Just a general note (I didn't check it against your patches): The
ABI/API is changed not only if you change the call signature of
functions, or memory layout of exposed data structures, but also if
you change the semantic behaviour of functions. For example, if a
function was guaranteed to call an initializer, and then doesn't
anymore, that's a semantic change.

If that is a "break" depends on what we promise about these functions.
This is where one needs to check existing documentation, but also
existing practice, as the documentation is probably not a 100%
complete specification (some common sense applies).

Thanks,
Marcus

_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel

Re: Questions about multi-threading [ In reply to ]

wk at gnupg

Jul 7, 2006, 8:25 AM

Post #6 of 6 (2250 views)

Permalink

On Thu, 6 Jul 2006 17:11, haypo@inl.fr said:

> Nice, I may try it since I need quickly a lot of entropy (create 100
> gnutls connections or more).

That won't give you more entropy of course. However, the pool is
persistent over invocations of applications using libgcrypt.

There are still problems with it. For example applications requesting
very strong random (e.g. for key generation) may stall other
applications. This is much like /dev/random and /dev/urandom.

> able to add new modules. But my question was: why try to register default
> plugins in all functions since one call will be enough? Check my
> proposition (two patches) on mailing list. I replaced all "register
> default ..." with *one* code only called once.

My fault, sorry.

>> However, this will be an ABI break.
>
> My patches don't break ABI nor API.

This was related to removing the module system. However, see Marcus
comments. I don't think your patch would break a documented
behaviour, though.

BTW, we need copyright assignments for patches.

Salam-Shalom,

Werner

_______________________________________________
Gcrypt-devel mailing list
Gcrypt-devel@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel