Mailing List Archive

v1.1.43 hangs with GLib/GDK threads
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi --

> The CVS version contains the new thread handling code, would be cool,

> if you could give that a try.

Sorry I didn't get back to you sooner. I was trying to fight through
some
packaging issues in order to get a release out for my project.

I tried using v1.1.43 with Ultramagnetic and unfortunately, I still have
the same problem. After GTK/GDK threads are initialized, libgcrypt
hangs randomly inside some pthread functions. I'm absolutely certain
this
is libgcrypt's fault because it does not misbehave if I disable
HAVE_PTHREAD in config.h.

Its worthy to mention that these hangs sometimes occur when calling
g_mutex_lock() from my own code. This suggests that libgcrypt is somehow
handling pthreads in a very instable way, though I'm too pressed for
time
to examine it more fully.

Below is a complete program which reproduces this deadlock issue with
the
latest v1.1.43.

- low halo


- ---- SNIP ----

/* Licensed under the GPL v2.0. */
/* Compile with:
* gcc -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include \
* -I/usr/include/gtk-2.0 -I/usr/lib/gtk-2.0/include -o main main.c
\
* `gtk-config --libs` `glib-config --libs` -pthread -lglib-2.0
* -lgmodule-2.0 -lgobject-2.0 -lgthread-2.0 -lgtk-x11-2.0 -lgcrypt
*/
#include <stdio.h>
#include <glib.h>
#include <gcrypt.h>

#define REQUIRED_LIBGCRYPT_VERSION "1.1.43"

gcry_sexp_t *key = NULL;
GMutex *mutex = NULL;

gpointer thread_function(gpointer nothing);
void init_gcrypt(void);
void perform_gcrypt_function(void);
void generate_key(void);

int main(int ac, char **av) {

int i = 0;
GThread *thread = NULL;

init_gcrypt();
g_thread_init(NULL);
gdk_threads_init();

generate_key();

mutex = g_mutex_new();

thread = g_thread_create(&thread_function, NULL, FALSE, NULL);

while(1) {
g_mutex_lock(mutex);
if (rand() % 13 == 0)
sleep(1);

perform_gcrypt_function();

printf("tick: %d\n", i++);

g_mutex_unlock(mutex);
}

return 0;
}

void init_gcrypt(void) {

if (!gcry_control(GCRYCTL_ANY_INITIALIZATION_P)) {
if (!gcry_check_version(REQUIRED_LIBGCRYPT_VERSION)) {
printf("ERROR: this version of ultramagnetic requires libgcrypt
v%s. "
"You only have v%s.\n", REQUIRED_LIBGCRYPT_VERSION,
gcry_check_version(NULL));
exit(1);
}
gcry_control(GCRYCTL_INIT_SECMEM, 1024 * 1024 * 3);
gcry_control(GCRYCTL_INITIALIZATION_FINISHED);
}
printf("libgcrypt v%s initialized.\n", gcry_check_version(NULL));
}

void generate_key(void) {

gcry_sexp_t params;

key = g_new0(gcry_sexp_t, 1);

printf("Generating key...");
fflush(stdout);

gcry_sexp_build(&params, NULL, "(genkey(elg(nbits %d)))", 512);
if (gcry_pk_genkey(key, params) != 0) {
printf("Error generating key.\n");
exit(1);
}
printf("done.\n");

}


gpointer thread_function(gpointer nothing) {

int j = 0;
while(1) {
g_mutex_lock(mutex);

if (rand() % 7 == 0)
sleep(1);

printf("tock: %d\n", j++);
g_mutex_unlock(mutex);
}

}


void perform_gcrypt_function(void) {

gcry_mpi_t an_mpi;
gcry_sexp_t an_sexp, encrypted_sexp;
int err = 0;
unsigned char buffer[] = "Encrypt this!";
int buffer_len = strlen(buffer);
gcry_error_t ret;

if (gcry_mpi_scan(&an_mpi, GCRYMPI_FMT_USG, buffer, buffer_len, NULL)
printf("gcry_mpi_scan failed.\n");
exit(1);
}

if (gcry_sexp_build(&an_sexp, &err,
"(data(flags)(value %m))", an_mpi) != 0) {
printf("gcry_sexp_build failed.\n");
exit(1);
}


if ((ret = gcry_pk_encrypt(&encrypted_sexp, an_sexp, *key)) != 0) {
printf("encryption failed: %s\n", gcry_strerror(ret));
exit(1);
}

}

- ---- SNIP ----

- --
low halo <lowhalo at-s1gn hush d0t c0m>
Defender of Truth and Liberty
http://ultramagnetic.sourceforge.net/

http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x9BFD99BF
58CE 3215 226A 69ED 4D20 4044 C925 54F9 9BFD 99BF

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)

iD8DBQE/W6K+ySVU+Zv9mb8RAgNQAJwI7aIq3soMMEnW/I5cPG5kxxHPNwCgpvKM
9AkC0OZ5TuQ6qAI/L/gqIhw=
=2u+S
-----END PGP SIGNATURE-----




Concerned about your privacy? Follow this link to get
FREE encrypted email: https://www.hushmail.com/?l=2

Free, ultra-private instant messaging with Hush Messenger
https://www.hushmail.com/services.php?subloc=messenger&l=434

Promote security and make money with the Hushmail Affiliate Program:
https://www.hushmail.com/about.php?subloc=affiliate&l=427
Re: v1.1.43 hangs with GLib/GDK threads [ In reply to ]
On Sun, Sep 07, 2003 at 02:25:13PM -0700, lowhalo@hush.com wrote:
> I tried using v1.1.43 with Ultramagnetic and unfortunately, I still have
> the same problem. After GTK/GDK threads are initialized,

What the h... are glib threads? :) If GTK/GDK threads are anything but a
thin wrapper around pthread, then we might need to write a backend for them
and build libgcrypt-g-thread or whatever. Theoretically it should be
sufficient to link to the gcrypt version linked to the same thread package
as glib.

> hangs randomly inside some pthread functions. I'm absolutely certain
> this
> is libgcrypt's fault because it does not misbehave if I disable
> HAVE_PTHREAD in config.h.

If you report deadlocks, please always include a gdb backtrace (bt, bt full)
and other status information (x/20i, info regs) in your report.

Thanks,
Marcus

--
`Rhubarb is no Egyptian god.' GNU http://www.gnu.org marcus@gnu.org
Marcus Brinkmann The Hurd http://www.gnu.org/software/hurd/
Marcus.Brinkmann@ruhr-uni-bochum.de
http://www.marcus-brinkmann.de/
Re: v1.1.43 hangs with GLib/GDK threads [ In reply to ]
On Mon, Sep 08, 2003 at 12:27:36PM +0200, Marcus Brinkmann wrote:

> On Sun, Sep 07, 2003 at 02:25:13PM -0700, lowhalo@hush.com wrote:
> > I tried using v1.1.43 with Ultramagnetic and unfortunately, I still have
> > the same problem. After GTK/GDK threads are initialized,
> What the h... are glib threads? :) If GTK/GDK threads are anything but a
> thin wrapper around pthread, then we might need to write a backend for them
> and build libgcrypt-g-thread or whatever. Theoretically it should be
> sufficient to link to the gcrypt version linked to the same thread package
> as glib.
Why not allow for an application to register it's own locking functions?
This would solve such problems.

> Thanks,
> Marcus

--
Nikos Mavroyanopoulos
Re: v1.1.43 hangs with GLib/GDK threads [ In reply to ]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



> What the h... are glib threads?

http://developer.gnome.org/doc/API/2.0/glib/glib-Threads.html

> If you report deadlocks, please always include a gdb backtrace
> (bt, bt full) and other status information (x/20i, info regs) in
> your report.

I already provided a backtrace in a previous post. But, here is a
'bt full' and 'info reg' (although you can easily get this from
the program I wrote):

#0 0x40640a35 in __pthread_sigsuspend () from /lib/i686/libpthread.so.0
No symbol table info available.
#1 0x4063fdb8 in __pthread_wait_for_restart_signal ()
from /lib/i686/libpthread.so.0
No symbol table info available.
#2 0x40642190 in __pthread_alt_lock () from /lib/i686/libpthread.so.0
No symbol table info available.
#3 0x4063ed77 in pthread_mutex_lock () from /lib/i686/libpthread.so.0
No symbol table info available.
#4 0x405f540d in _gcry_ath_mutex_lock (lock=0x40637810) at ath-compat.c:87
err = -4
#5 0x405f8f9d in _gcry_secmem_malloc (size=24) at secmem.c:485
p = (void *) 0x18
#6 0x405f871d in _gcry_private_malloc_secure (n=24) at stdmem.c:105
No locals.
#7 0x405f6145 in gcry_malloc_secure (n=0) at global.c:397
No locals.
#8 0x405f642d in gcry_xmalloc_secure (n=24) at global.c:523
p = (void *) 0x6
#9 0x4062709d in _gcry_mpi_alloc_limb_space (nlimbs=6, secure=1)
at mpiutil.c:82
p = (mpi_limb_t *) 0x0
#10 0x40627067 in _gcry_mpi_alloc_secure (nlimbs=6) at mpiutil.c:71
a = (struct gcry_mpi *) 0x804b860
#11 0x40627410 in _gcry_mpi_copy (a=0x804b2a0) at mpiutil.c:223
i = 0
b = (struct gcry_mpi *) 0x42138c90
#12 0x406235b5 in gcry_mpi_gcd (g=0x804b6f8, xa=0x804b2a0, xb=0x804b758)
at mpi-gcd.c:35
a = (struct gcry_mpi *) 0x804b2a0
b = (struct gcry_mpi *) 0x804b758
#13 0x40619876 in gen_k (p=0x804b418) at elgamal.c:209
k = (struct gcry_mpi *) 0x804b2a0
temp = (struct gcry_mpi *) 0x804b6f8
p_1 = (struct gcry_mpi *) 0x804b758
orig_nbits = 512
nbits = 178
nbytes = 23
rndbuf = 0x804b4e0 "G?M?\205$\231\001?gZRh?\202Q?s\025\214(?\030"
#14 0x40619c87 in do_encrypt (a=0x804b2d8, b=0x804b460, input=0x804b3f0,

pkey=0xbffff740) at elgamal.c:349
k = (struct gcry_mpi *) 0x804b460
#15 0x406196dc in test_keys (sk=0xbffff7c0, nbits=448) at elgamal.c:135
pk = {p = 0x804b418, g = 0x804b3d8, y = 0x804bf50}
test = (struct gcry_mpi *) 0x804b3f0
out1_a = (struct gcry_mpi *) 0x804b2d8
out1_b = (struct gcry_mpi *) 0x804b460
out2 = (struct gcry_mpi *) 0x804b430
#16 0x40619b2e in generate (sk=0xbffff7c0, nbits=512, ret_factors=0xbffff898)
at elgamal.c:315
p = (struct gcry_mpi *) 0x804b418
p_min1 = (struct gcry_mpi *) 0x804b318
g = (struct gcry_mpi *) 0x804b3d8
x = (struct gcry_mpi *) 0x804bf38
y = (struct gcry_mpi *) 0x804bf50
temp = (struct gcry_mpi *) 0x804b378
qbits = 134528848
xbits = 180
rndbuf = (byte *) 0x804bf50 " "
#17 0x40619ef9 in _gcry_elg_generate (algo=20, nbits=512, dummy=65537,

skey=0xbffff960, retfactors=0xbffff898) at elgamal.c:507
sk = {p = 0x804b418, g = 0x804b3d8, y = 0x804bf50, x = 0x804bf38}
#18 0x405fba4f in pubkey_generate (algorithm=20, nbits=512, use_e=65537,

skey=0xbffff960, retfactors=0xbffff898) at pubkey.c:473
err = GPG_ERR_PUBKEY_ALGO
pubkey = (struct gcry_module *) 0x804a698
#19 0x405fe23f in gcry_pk_genkey (r_key=0x804b248, s_parms=0x804b258)
at pubkey.c:1889
pubkey = (struct gcry_pk_spec *) 0x6
module = (struct gcry_module *) 0x804a698
list = (struct gcry_sexp *) 0x804b300
l2 = (struct gcry_sexp *) 0x0
name = 0x804b30c "512\004"
n = 3
rc = GPG_ERR_NO_ERROR
i = 0
algo_name = 0x40631798 "elg"
sec_elems = 0x406317b4 "pgyx"
pub_elems = 0x406317b0 "pgy"
skey = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}
factors = (struct gcry_mpi **) 0x804b448
nbits = 512
use_e = 65537
#20 0x08048d72 in generate_key ()
No symbol table info available.
#21 0x08048bd2 in main ()
No symbol table info available.
#22 0x420158f7 in __libc_start_main () from /lib/i686/libc.so.6
No symbol table info available.
(gdb) info reg
eax 0xfffffffc -4
ecx 0x8 8
edx 0x0 0
ebx 0xbffff410 -1073744880
esp 0xbffff3f4 0xbffff3f4
ebp 0xbffff3f8 0xbffff3f8
esi 0x40645078 1080316024
edi 0xbffff410 -1073744880
eip 0x40640a35 0x40640a35
eflags 0x246 582
cs 0x23 35
ss 0x2b 43
ds 0x2b 43
es 0x2b 43
fs 0x0 0
gs 0x7 7
fctrl 0x37f 895
fstat 0x0 0
ftag 0xffff 65535
fiseg 0x0 0
fioff 0x0 0
foseg 0x0 0
fooff 0x0 0
fop 0x0 0
xmm0 {f = {0x0, 0x0, 0x0, 0x0}} {f = {-nan(0x7fffff),

-nan(0x7fffff), -nan(0x7fffff), -nan(0x7fffff)}}
xmm1 {f = {0x0, 0x0, 0x0, 0x0}} {f = {-nan(0x7fffff),

-nan(0x7fffff), -nan(0x7fffff), -nan(0x7fffff)}}
xmm2 {f = {0x0, 0x0, 0x0, 0x0}} {f = {-nan(0x7fffff),

-nan(0x7fffff), -nan(0x7fffff), -nan(0x7fffff)}}
xmm3 {f = {0x0, 0x0, 0x0, 0x0}} {f = {-nan(0x7fffff),

-nan(0x7fffff), -nan(0x7fffff), -nan(0x7fffff)}}
xmm4 {f = {0x0, 0x0, 0x0, 0x0}} {f = {-nan(0x7fffff),

-nan(0x7fffff), -nan(0x7fffff), -nan(0x7fffff)}}
xmm5 {f = {0x0, 0x0, 0x0, 0x0}} {f = {-nan(0x7fffff),

-nan(0x7fffff), -nan(0x7fffff), -nan(0x7fffff)}}
xmm6 {f = {0x0, 0x0, 0x0, 0x0}} {f = {-nan(0x7fffff),

-nan(0x7fffff), -nan(0x7fffff), -nan(0x7fffff)}}
xmm7 {f = {0x0, 0x0, 0x0, 0x0}} {f = {-nan(0x7fffff),

-nan(0x7fffff), -nan(0x7fffff), -nan(0x7fffff)}}
mxcsr 0x1f80 8064
orig_eax 0xb3 179


- low halo

- --
low halo <lowhalo at-s1gn hush d0t c0m>
Defender of Truth and Liberty
http://ultramagnetic.sourceforge.net/

http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x9BFD99BF
58CE 3215 226A 69ED 4D20 4044 C925 54F9 9BFD 99BF

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.2 (GNU/Linux)

iD8DBQE/XPhOySVU+Zv9mb8RAjEgAJ9fa4xkC8jDIoVU73pzq7ipowc8BwCg49Kz
FGZLawWFDvhZLF66+cvm7ZI=
=oZGk
-----END PGP SIGNATURE-----




Concerned about your privacy? Follow this link to get
FREE encrypted email: https://www.hushmail.com/?l=2

Free, ultra-private instant messaging with Hush Messenger
https://www.hushmail.com/services.php?subloc=messenger&l=434

Promote security and make money with the Hushmail Affiliate Program:
https://www.hushmail.com/about.php?subloc=affiliate&l=427
Re: v1.1.43 hangs with GLib/GDK threads [ In reply to ]
On Mon, Sep 08, 2003 at 02:41:18PM -0700, lowhalo@hush.com wrote:
> > What the h... are glib threads?
>
> http://developer.gnome.org/doc/API/2.0/glib/glib-Threads.html

This tells me what interface they implement, but not how they are
implemented.

libglib-2.0 doesn't seem to be linked to pthread. So if glib threads are
entirely user level threads, it would be wrong to use it with
gcrypt-pthread, and that alone might trigger errors, depending on how you
link your program.

With user-level threads, it is probably sufficient to just link with plain
gcrypt. However, I don't know if glib ensures that it is user-level or if
it might us system threads. I don't think libgcrypt uses any of the
blocking system calls (read, write, select, ...) so it would not be
"preempted" by a user-level thread package and thus using the non-threaded
version would be safe.

Depsite all this, there might also be a bug in our thread support, of
course. We can try to find out more about that.

Thanks,
Marcus

--
`Rhubarb is no Egyptian god.' GNU http://www.gnu.org marcus@gnu.org
Marcus Brinkmann The Hurd http://www.gnu.org/software/hurd/
Marcus.Brinkmann@ruhr-uni-bochum.de
http://www.marcus-brinkmann.de/
Re: v1.1.43 hangs with GLib/GDK threads [ In reply to ]
On Tue, 9 Sep 2003 09:08:26 +0200, Marcus Brinkmann said:

> libglib-2.0 doesn't seem to be linked to pthread. So if glib threads are

IIRC, the gtk thread support is dynamically linked using dlopen. The
problem with glib is that they try to reinvent POSIX using their own
interface and only specially targeted for GNOME etc.

--
Werner Koch <wk@gnupg.org>
The GnuPG Experts http://g10code.com
Free Software Foundation Europe http://fsfeurope.org
Re: v1.1.43 hangs with GLib/GDK threads [ In reply to ]
Hi,

your code triggered a forgot-to-unlock-mutex-before-returning bug in
src/secmem.c, for which a fix has just been commited to CVS. Thanks!
I wonder wether this was the only problem or wether there are indeed
GTK/GDK-Threads related problems.

moritz
--
((gpg-key-id . "6F984199")
(email . "moritz@duesseldorf.ccc.de")
(webpage . "http://duesseldorf.ccc.de/~moritz/"))
Re: v1.1.43 hangs with GLib/GDK threads [ In reply to ]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Hi --

Glad to hear that the problem's been fixed! A few points:

1.) I updated the sources from CVS, but the compilation failed with:
'make[2]: *** No rule to make target `@LTLIBOBJS@', needed
by `libgcrypt.la'.' However, I patched v1.1.43's secmem.c, and
it does appear to have fixed the problem.

2.) I was wondering if the libgcrypt team was ever planning on
releasing compiled binaries, like RPMs. I'd like to release
RPMs for Ultramagnetic, but I can't distribute a libgcrypt RPM
on SourceForge because of the stupid export laws (SF is in the
U.S.).
Can anyone donate a couple megs of server space to host
binaries for my project? That would be a huge help!
(see http://ultramagnetic.sourceforge.net)

3.) What is the status of the libgcrypt sources? I know that
it is considered beta, but aren't the individual functions
taken from GnuPG? So, are those functions considered stable even
though the entire package is beta?
What I'm trying to get to is this: my project uses
ElGamal, AES-256, and DSA. Would I need to wait until libgcrypt
v2.0.0 is released before I release v1.0 to the public? If so,
would anyone have an idea as to when v2.0.0 is scheduled?

4.) I see that 1024-bit DSA is the largest I can use. Any plans
on making the limit higher? I'm paranoid. (If not, I think I'm
going to switch to 2048-bit RSA sigs...)


Thanks!
- low halo

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQE/d6ALySVU+Zv9mb8RAtqKAKD0Uohwgf1kz+i7NFD29B/SXzLZ9QCgyHMS
Z2naAO5w4dIghpT7DoUPmQU=
=XK5v
-----END PGP SIGNATURE-----




Concerned about your privacy? Follow this link to get
FREE encrypted email: https://www.hushmail.com/?l=2

Free, ultra-private instant messaging with Hush Messenger
https://www.hushmail.com/services.php?subloc=messenger&l=434

Promote security and make money with the Hushmail Affiliate Program:
https://www.hushmail.com/about.php?subloc=affiliate&l=427
Re: v1.1.43 hangs with GLib/GDK threads [ In reply to ]
On Sun, 28 Sep 2003 20:02:55 -0700, said:

> 1.) I updated the sources from CVS, but the compilation failed with:
> 'make[2]: *** No rule to make target `@LTLIBOBJS@', needed
> by `libgcrypt.la'.' However, I patched v1.1.43's secmem.c, and
> it does appear to have fixed the problem.

Sure that you have the latest autoconf tools and did configure with
--enable-maintainer-mode?

> 2.) I was wondering if the libgcrypt team was ever planning on
> releasing compiled binaries, like RPMs. I'd like to release

No. That is something the distributions should do.

> RPMs for Ultramagnetic, but I can't distribute a libgcrypt RPM
> on SourceForge because of the stupid export laws (SF is in the
> U.S.).

There should be no problem anymore with that. IIRC, you just have to
send a mail to some BXA mail address.

> Can anyone donate a couple megs of server space to host
> binaries for my project? That would be a huge help!

Megs are not a problm, traffic is the real issue ;-). If you feel
safer hosting your stuff in Germany, the GUUG will certainly allow me
to offer you some space on ftp.gnupg.org. According to the GPL, you
need to have the source on the same machine, though.

> 3.) What is the status of the libgcrypt sources? I know that
> it is considered beta, but aren't the individual functions

We have recently done some major changes, so we are still waiting on
feedback. A 1.2 version will probably released this year.

> What I'm trying to get to is this: my project uses
> ElGamal, AES-256, and DSA. Would I need to wait until libgcrypt
> v2.0.0 is released before I release v1.0 to the public? If so,

The API should not change anymore, but to be really sure, you better
wait for 1.2.0

> 4.) I see that 1024-bit DSA is the largest I can use. Any plans
> on making the limit higher? I'm paranoid. (If not, I think I'm
> going to switch to 2048-bit RSA sigs...)

DSA is only defined up to 1024 bits; it should be obviously how to
do larger ones using SHA-256 but that is not defined yet. It is
highly questionable whether RSA2048 using SHA-1 is theoretically
harder to break than DSA 1024 using SHA-1.

Werner

--
Werner Koch <wk@gnupg.org>
The GnuPG Experts http://g10code.com
Free Software Foundation Europe http://fsfeurope.org