Mailing List Archive

assertion in _gcry_ath_mutex_lock
I'm getting assertions w/ openldap 2.1.20 compiled against
gnutls-1.0.16 & libgcrypt-1.2.0.

I've tried several versions of gnutls & libgcrypt but the failure mode
is constant across all the versions I've tried. I've attached a stack
trace. The other active threads differ from crash to crash, but the
asserting thread is constant.

If there's other debugging info that would help, just let me know.

Thanks
job


here's what I'm currently running (debian testing base):

gnutls-1.0.16.tar.bz2
libgcrypt11_1.2.0.orig.tar.gz
libtasn1-0.2.10.tar.gz
opencdk8_0.5.5.orig.tar.gz
openldap2_2.1.30.orig.tar.gz


pella:~# !gd
gdb /usr/sbin/slapd
GNU gdb 6.1-debian
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-linux"...(no debugging symbols found)...Using host libthread_db library "/lib/libthread_db.so.1".

(gdb) run -d0
Starting program: /usr/sbin/slapd -d0
[Thread debugging using libthread_db enabled]
[New Thread 16384 (LWP 11139)]
[New Thread 32769 (LWP 11168)]
[New Thread 16386 (LWP 11169)]
[New Thread 32771 (LWP 11171)]
[New Thread 49156 (LWP 11173)]
[New Thread 65541 (LWP 11176)]
[New Thread 81926 (LWP 11177)]
slapd: ath.c:181: _gcry_ath_mutex_lock: Assertion `*lock == ((ath_mutex_t) 0)' failed.

Program received signal SIGABRT, Aborted.
[Switching to Thread 32771 (LWP 11171)]
0x4037a721 in kill () from /lib/libc.so.6
(gdb) bt
#0 0x4037a721 in kill () from /lib/libc.so.6
#1 0x402f5771 in pthread_kill () from /lib/libpthread.so.0
#2 0x402f5a7b in raise () from /lib/libpthread.so.0
#3 0x4037a4d4 in raise () from /lib/libc.so.6
#4 0x4037b9e8 in abort () from /lib/libc.so.6
#5 0x40373b3f in __assert_fail () from /lib/libc.so.6
#6 0x4024fe9b in _gcry_ath_mutex_lock (lock=0x40293920) at ath.c:177
#7 0x4025cbaf in _gcry_fast_random_poll () at random.c:991
#8 0x402587b5 in md_open (h=0x0, algo=2, secure=0, hmac=2) at md.c:441
#9 0x402588b2 in gcry_md_open (h=0x0, algo=0, flags=1076866676) at md.c:472
#10 0x401e3000 in _gnutls_hmac_init (algorithm=GNUTLS_MAC_SHA, key=0x41809d10,
keylen=20) at gnutls_hash_int.c:155
#11 0x401d8105 in _gnutls_compressed2ciphertext (session=0x41803570,
cipher_data=0x818196d "\023H@Ú\177\037\006o¬æ­uéNABh\rVd\a¸\006ÍQ",
cipher_size=2596, compressed=
{data = 0x8124a38 "ÂZ\236¥$¦æ\002ç'I\231\224D+Ï\220M (Õ¡\226g(Æb\031ÂÂLnÀ¡^Bå\031\212¡¸Ó\031", size = 48}, _type=0, random_pad=0) at gnutls_cipher.c:176
#12 0x401d7e4c in _gnutls_encrypt (session=0x41803570, headers=0x0,
headers_size=5,
data=0x81277d8 "0\202\001&\002\001\003d\202\001\037\0049uid=vba-daemon-start,ou=People,dc=vanburenadvisors,dc=com0\201á0\030\004\002cn1\022\004\020vba-daemon-start0:\004\vobjectClass1+\004\aaccount\004\fposixAccount\004\rshadowAccount\004\003top0\031\004\003uid1\022\004\020vba-daemon-start0\023\004\tuidNumber1"..., data_size=298,
ciphertext=0x8181968 "\024\023H@\024\023H@Ú\177\037\006o¬æ­uéNABh\rVd\a¸\006ÍQ", ciphertext_size=2596, type=0, random_pad=0) at gnutls_cipher.c:90
#13 0x401d686d in _gnutls_send_int (session=0x41803570,
type=GNUTLS_APPLICATION_DATA, htype=4294967295, _data=0x81277d8,
sizeofdata=5) at gnutls_record.c:340
#14 0x401d764f in gnutls_record_send (session=0x0, data=0x0, sizeofdata=0)
at gnutls_record.c:871
#15 0x4004de03 in gnutls_SSL_write (ssl=0x402fae74, buf=0x0, buflen=0)
at gnutls.c:724
#16 0x4004b369 in sb_tls_write (sbiod=0x41802f70, buf=0x0, len=0) at tls.c:678
#17 0x4005fb57 in sb_debug_write (sbiod=0x81d4168, buf=0x81277d8, len=298)
at /root/openldap2-2.1.30/libraries/liblber/sockbuf.c:841
#18 0x4005eec4 in ber_int_sb_write (sb=0x81818c8, buf=0x81277d8, len=298)
at /root/openldap2-2.1.30/libraries/liblber/sockbuf.c:427
#19 0x4005bbe9 in ber_flush (sb=0x81818c8, ber=0xbf57e604, freeit=0)
at /root/openldap2-2.1.30/libraries/liblber/io.c:234
#20 0x08066ee8 in backend_operational ()
#21 0x08068855 in slap_send_search_entry ()
#22 0x40674dde in bdb_search (be=0x8102290, conn=0x4074e958, op=0x8133278,
base=0xbf5fe884, nbase=0x0, scope=2, deref=0, slimit=473, tlimit=3600,
filter=0x8116e78, filterstr=0xbf5fe874, attrs=0x8183708, attrsonly=0)
at /root/openldap2-2.1.30/servers/slapd/back-bdb/search.c:774
#23 0x08058f56 in do_search ()
#24 0x08057688 in connection_done ()
#25 0x4002b754 in ldap_int_thread_pool_wrapper (xpool=0x63)
at /root/openldap2-2.1.30/libraries/libldap_r/tpool.c:432
#26 0x402f2e51 in pthread_start_thread () from /lib/libpthread.so.0
#27 0x402f2ecf in pthread_start_thread_event () from /lib/libpthread.so.0
#28 0x4042869a in clone () from /lib/libc.so.6
(gdb) info threads
7 Thread 81926 (LWP 11177) 0x402f5604 in __pthread_sigsuspend ()
from /lib/libpthread.so.0
6 Thread 65541 (LWP 11176) 0x402f5604 in __pthread_sigsuspend ()
from /lib/libpthread.so.0
5 Thread 49156 (LWP 11173) 0x402f5604 in __pthread_sigsuspend ()
from /lib/libpthread.so.0
* 4 Thread 32771 (LWP 11171) 0x4037a721 in kill () from /lib/libc.so.6
3 Thread 16386 (LWP 11169) 0x403e2ea1 in gettimeofday () from /lib/libc.so.6
2 Thread 32769 (LWP 11168) 0x4041f85a in poll () from /lib/libc.so.6
1 Thread 16384 (LWP 11139) 0x402f5604 in __pthread_sigsuspend ()
from /lib/libpthread.so.0
(gdb) info reg
eax 0x0 0
ecx 0x6 6
edx 0x402fae74 1076866676
ebx 0x2ba3 11171
esp 0xbf57dd80 0xbf57dd80
ebp 0xbf57dd98 0xbf57dd98
esi 0x2ba3 11171
edi 0x402fd170 1076875632
eip 0x4037a721 0x4037a721
eflags 0x246 582
cs 0x23 35
ss 0x2b 43
ds 0x2b 43
es 0x2b 43
fs 0x0 0
gs 0x0 0
(gdb) bt full
#0 0x4037a721 in kill () from /lib/libc.so.6
No symbol table info available.
#1 0x402f5771 in pthread_kill () from /lib/libpthread.so.0
No symbol table info available.
#2 0x402f5a7b in raise () from /lib/libpthread.so.0
No symbol table info available.
#3 0x4037a4d4 in raise () from /lib/libc.so.6
No symbol table info available.
#4 0x4037b9e8 in abort () from /lib/libc.so.6
No symbol table info available.
#5 0x40373b3f in __assert_fail () from /lib/libc.so.6
No symbol table info available.
#6 0x4024fe9b in _gcry_ath_mutex_lock (lock=0x40293920) at ath.c:177
No locals.
#7 0x4025cbaf in _gcry_fast_random_poll () at random.c:991
err = 0
#8 0x402587b5 in md_open (h=0x0, algo=2, secure=0, hmac=2) at md.c:441
err = GPG_ERR_NO_ERROR
bufsize = 0
ctx = (struct gcry_md_context *) 0x812b710
hd = 0x812b300
#9 0x402588b2 in gcry_md_open (h=0x0, algo=0, flags=1076866676) at md.c:472
err = GPG_ERR_BAD_PUBKEY
hd = 0x3
#10 0x401e3000 in _gnutls_hmac_init (algorithm=GNUTLS_MAC_SHA, key=0x41809d10,
keylen=20) at gnutls_hash_int.c:155
ret = 0x8182488
result = 0
#11 0x401d8105 in _gnutls_compressed2ciphertext (session=0x41803570,
cipher_data=0x818196d "\023H@Ú\177\037\006o¬æ­uéNABh\rVd\a¸\006ÍQ",
cipher_size=2596, compressed=
{data = 0x8124a38 "ÂZ\236¥$¦æ\002ç'I\231\224D+Ï\220M (Õ¡\226g(Æb\031ÂÂLnÀ¡^Bå\031\212¡¸Ó\031", size = 48}, _type=0, random_pad=0) at gnutls_cipher.c:176
MAC = "p5\200A\224àW¿\\àW¿ãz\035@°\004\200A"
c_length = 16419
pad = 0 '\0'
length = 2
ret = 0
type = 23 '\027'
major = 3 '\003'
minor = 1 '\001'
hash_size = 20
ver = GNUTLS_TLS1_0
blocksize = 16
block_algo = CIPHER_BLOCK
data_ptr = (opaque *) 0x0
#12 0x401d7e4c in _gnutls_encrypt (session=0x41803570, headers=0x0,
headers_size=5,
data=0x81277d8 "0\202\001&\002\001\003d\202\001\037\0049uid=vba-daemon-start,ou=People,dc=vanburenadvisors,dc=com0\201á0\030\004\002cn1\022\004\020vba-daemon-start0:\004\vobjectClass1+\004\aaccount\004\fposixAccount\004\rshadowAccount\004\003top0\031\004\003uid1\022\004\020vba-daemon-start0\023\004\tuidNumber1"..., data_size=298,
ciphertext=0x8181968 "\024\023H@\024\023H@Ú\177\037\006o¬æ­uéNABh\rVd\a¸\006ÍQ", ciphertext_size=2596, type=0, random_pad=0) at gnutls_cipher.c:90
plain = {data = 0x0, size = 1076866676}
comp = {
data = 0x8124a38 "ÂZ\236¥$¦æ\002ç'I\231\224D+Ï\220M (Õ¡\226g(Æb\031ÂÂLnÀ¡^Bå\031\212¡¸Ó\031", size = 48}
ret = 5
free_comp = 1
#13 0x401d686d in _gnutls_send_int (session=0x41803570,
type=GNUTLS_APPLICATION_DATA, htype=4294967295, _data=0x81277d8,
sizeofdata=5) at gnutls_record.c:340
cipher = (
uint8 *) 0x8181968 "\024\023H@\024\023H@Ú\177\037\006o¬æ­uéNABh\rVd\a¸\006ÍQ"
cipher_size = 5
retval = 0
ret = 298
data2send_size = 298
headers = "\027\003\001\000"
#14 0x401d764f in gnutls_record_send (session=0x0, data=0x0, sizeofdata=0)
at gnutls_record.c:871
No locals.
#15 0x4004de03 in gnutls_SSL_write (ssl=0x402fae74, buf=0x0, buflen=0)
at gnutls.c:724
ret = 0
#16 0x4004b369 in sb_tls_write (sbiod=0x41802f70, buf=0x0, len=0) at tls.c:678
p = (struct tls_data *) 0x41807d68
---Type <return> to continue, or q <return> to quit---
ret = 0
err = 0
#17 0x4005fb57 in sb_debug_write (sbiod=0x81d4168, buf=0x81277d8, len=298)
at /root/openldap2-2.1.30/libraries/liblber/sockbuf.c:841
ret = 135796936
#18 0x4005eec4 in ber_int_sb_write (sb=0x81818c8, buf=0x81277d8, len=298)
at /root/openldap2-2.1.30/libraries/liblber/sockbuf.c:427
ret = 298
#19 0x4005bbe9 in ber_flush (sb=0x81818c8, ber=0xbf57e604, freeit=0)
at /root/openldap2-2.1.30/libraries/liblber/io.c:234
towrite = 298
rc = 0
#20 0x08066ee8 in backend_operational ()
No symbol table info available.
#21 0x08068855 in slap_send_search_entry ()
No symbol table info available.
#22 0x40674dde in bdb_search (be=0x8102290, conn=0x4074e958, op=0x8133278,
base=0xbf5fe884, nbase=0x0, scope=2, deref=0, slimit=473, tlimit=3600,
filter=0x8116e78, filterstr=0xbf5fe874, attrs=0x8183708, attrsonly=0)
at /root/openldap2-2.1.30/servers/slapd/back-bdb/search.c:774
result = 0
dn = {bv_len = 0, bv_val = 0x0}
bdb = (struct bdb_info *) 0x8102348
rc = 1081403736
stoptime = 1090301527
id = 69
cursor = 27
candidates = {58, 16, 18, 20, 22, 25, 27, 29, 31, 33, 35, 37, 39, 41,
43, 45, 47, 49, 51, 53, 55, 57, 59, 61, 63, 65, 67, 69, 71, 83, 84, 85, 86,
87, 88, 89, 90, 91, 92, 93, 94, 96, 108, 109, 110, 111, 113, 114, 115, 116,
117, 118, 119, 120, 121, 122, 124, 125, 126, 0 <repeats 130403 times>,
1076846491, 1078461776, 3210731132, 1076846456, 3210731164, 1077709046, 5,
135014227, 1099291552, 3210733404, 134769920, 3210733404, 3210731196,
134769974, 1099291541, 135014216, 0, 0, 0, 135014536, 3210731228, 134825857,
3210733404, 135014520, 0, 1099291541, 0, 3210733424, 3210731260, 134770067,
135015656, 3210733404, 1076846491, 1078461776, 3210731260, 1076846456,
3210731292, 1077709046, 5, 135014227, 1099291673, 3210733532, 134769920,
3210733532, 3210731324, 134769974, 1099291662, 135014216, 0, 0, 0,
135014536, 3210731356, 134825857, 3210733532, 135014520, 3210731372,
1099291662, 0, 3210733552, 3210731388, 134770067, 135015656, 3210733532,
134769920, 1073921455, 0, 3210733552, 3210733580, 134765207, 3210732484, 0,
1032, 0 <repeats 57 times>, 1077606753, 0, 1078461776, 3210731664, 0,
134867508, 135444952, 3210733200, 1077518235, 3210733236, 134867508, 1, 0,
0, 0, 0, 0, 3210733180, 1077511159, 0 <repeats 47 times>, 544407552, 0,
4294967295, 4294967284, 0 <repeats 14 times>, 3210732108, 0, 2, 3210733520,
3210733180...}
e = (Entry *) 0x81d6ea0
v2refs = 0x0
matched = (Entry *) 0x0
realbase = {bv_len = 26,
bv_val = 0x81824a0 "dc=vanburenadvisors,dc=com"}
nentries = 26
manageDSAit = 0
limit = (struct slap_limits_set *) 0x81022f8
isroot = 0
locker = 22
lock = {off = 280640, ndx = 685, gen = 111, mode = DB_LOCK_READ}
#23 0x08058f56 in do_search ()
No symbol table info available.
#24 0x08057688 in connection_done ()
No symbol table info available.
#25 0x4002b754 in ldap_int_thread_pool_wrapper (xpool=0x63)
at /root/openldap2-2.1.30/libraries/libldap_r/tpool.c:432
ctx = (ldap_int_thread_ctx_t *) 0x8102290
ltc_key = {{ltk_key = 0x8121af8, ltk_data = 0x16,
ltk_free = 0x40681e70 <bdb_locker_id_free>}, {ltk_key = 0x40675920,
ltk_data = 0x4079f008, ltk_free = 0x406758f0 <search_stack_free>}, {
ltk_key = 0x0, ltk_data = 0x0, ltk_free = 0}, {ltk_key = 0x0,
ltk_data = 0x0, ltk_free = 0xffffffff}, {ltk_key = 0x0,
ltk_data = 0xffffffff, ltk_free = 0xffffffff}, {ltk_key = 0x0,
ltk_data = 0xffffffff, ltk_free = 0xffffffff}, {ltk_key = 0x0,
ltk_data = 0xffffffff, ltk_free = 0xffffffff}, {ltk_key = 0x0,
ltk_data = 0xffffffff, ltk_free = 0xffffffff}, {ltk_key = 0x0,
ltk_data = 0xffffffff, ltk_free = 0xffffffff}, {ltk_key = 0x0,
ltk_data = 0xffffffff, ltk_free = 0xffffffff}, {ltk_key = 0x0,
ltk_data = 0xffffffff, ltk_free = 0xffffffff}, {ltk_key = 0x0,
ltk_data = 0xffffffff, ltk_free = 0xffffffff}, {ltk_key = 0x0,
ltk_data = 0xffffffff, ltk_free = 0xffffffff}, {ltk_key = 0x0,
ltk_data = 0xffffffff, ltk_free = 0xffffffff}, {ltk_key = 0x0,
ltk_data = 0x0, ltk_free = 0} <repeats 16 times>, {ltk_key = 0x0,
---Type <return> to continue, or q <return> to quit---
ltk_data = 0x40480550, ltk_free = 0xbf5ffaac}, {ltk_key = 0x0,
ltk_data = 0xbf5ffacc, ltk_free = 0x40373579 <uselocale+169>}}
i = 135275152
#26 0x402f2e51 in pthread_start_thread () from /lib/libpthread.so.0
No symbol table info available.
#27 0x402f2ecf in pthread_start_thread_event () from /lib/libpthread.so.0
No symbol table info available.
#28 0x4042869a in clone () from /lib/libc.so.6
No symbol table info available.
(gdb)
Re: assertion in _gcry_ath_mutex_lock [ In reply to ]
Hi,

job@jobbogan.com:
> I'm getting assertions w/ openldap 2.1.20 compiled against
> gnutls-1.0.16 & libgcrypt-1.2.0.
>
Ouch.

> I've tried several versions of gnutls & libgcrypt but the failure mode
> is constant across all the versions I've tried. I've attached a stack
> trace. The other active threads differ from crash to crash, but the
> asserting thread is constant.
>
Can you try reproducing this under Electric Fence?

(gdb) set env LD_PRELOAD /usr/lib/libefence.so
(gdb) run -d0

--
Matthias Urlichs | {M:U} IT Design @ m-u-it.de | smurf@smurf.noris.de
Re: assertion in _gcry_ath_mutex_lock [ In reply to ]
At Tue, 20 Jul 2004 00:22:17 -0500,
job@jobbogan.com wrote:
>
> I'm getting assertions w/ openldap 2.1.20 compiled against
> gnutls-1.0.16 & libgcrypt-1.2.0.
>
> I've tried several versions of gnutls & libgcrypt but the failure mode
> is constant across all the versions I've tried. I've attached a stack
> trace. The other active threads differ from crash to crash, but the
> asserting thread is constant.

You are using pthread, but you have not initialized gcrypt to use the
pthread mutex stubs. Please refer to the documentation on how to do that.

I am waging a guess that this is already your problem. The stubs used
for the non-threaded case use the assertions to check for locking
issues, but they rely on the fact that there is never contention in a
single-threaded program (otherwise it would dead lock anyway :).

So, if there is contention, and you didn't initialize gcrypt properly,
you will trigger the assertion.

A programming error in gcrypt could also trigger this assertion, but
this would show as a dead lock in the threaded case (if properly
initialized).

Thanks,
Marcus