Mailing List Archive

[Bug 392] BGPD crashes with bgpd in free()
Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug
report.

http://bugzilla.quagga.net/show_bug.cgi?id=392





------- Additional Comments From paul@dishone.st 2007-08-13 13:28 -------
Can you run it under (or attach) a debugger and get a stack trace?




------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
_______________________________________________
Quagga-bugs mailing list
Quagga-bugs@lists.quagga.net
http://lists.quagga.net/mailman/listinfo/quagga-bugs
[Bug 392] BGPD crashes with bgpd in free() [ In reply to ]
Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug
report.

http://bugzilla.quagga.net/show_bug.cgi?id=392





------- Additional Comments From gardiner@purdigital.net 2007-08-13 14:09 -------
Sorry, I'm not really accustomed to the programming internals. Here is the
stack trace

GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...
(gdb) run
Starting program: /usr/local/sbin/bgpd
bgpd in free(): error: junk pointer, too high to make sense

Program received signal SIGABRT, Aborted.
0x2824becb in kill () from /lib/libc.so.6
(gdb) bt
#0 0x2824becb in kill () from /lib/libc.so.6
#1 0x2824be68 in raise () from /lib/libc.so.6
#2 0x2824ab78 in abort () from /lib/libc.so.6
#3 0x281e7fdb in _UTF8_init () from /lib/libc.so.6
#4 0xbfbfedf8 in ?? ()
#5 0x28251dd3 in sys_nsig () from /lib/libc.so.6
#6 0x28251cd3 in sys_nsig () from /lib/libc.so.6
#7 0x28251df0 in sys_nsig () from /lib/libc.so.6
#8 0x00000000 in ?? ()
#9 0x2825cd80 in ?? () from /lib/libc.so.6
#10 0xbfbfe828 in ?? ()
#11 0x281e8009 in _UTF8_init () from /lib/libc.so.6
#12 0x2825cd80 in ?? () from /lib/libc.so.6
#13 0x28272a24 in _nsyyin () from /lib/libc.so.6
#14 0xbfbfe8d8 in ?? ()
#15 0x281e8d69 in _UTF8_init () from /lib/libc.so.6
#16 0x082514d0 in ?? ()
#17 0x00000000 in ?? ()
#18 0x00000010 in ?? ()
#19 0x082514d0 in ?? ()
#20 0x00000001 in ?? ()
#21 0x2816046c in __JCR_LIST__ () from /usr/local/lib/libzebra.so.0
#22 0xbfbfe878 in ?? ()
#23 0x2816046c in __JCR_LIST__ () from /usr/local/lib/libzebra.so.0
---Type <return> to continue, or q <return> to quit---
#24 0xbfbfec10 in ?? ()
#25 0x2812db2c in alloc_inc (type=673566080) at memory.c:224
Previous frame inner to this frame (corrupt stack?)


------- Additional Comments From gert@greenie.muc.de 2007-08-14 08:29 -------
Hi,

I am also observing this bug - FreeBSD 6.2 on Sparc64, Quagga 0.99.8 from the
FreeBSD ports, but also using non-port-patched (pristine) Quagga sources.

There are two different sorts of crashes:

- when loading a config that has neighbour statements, it will crash right
away, and the back trace looks like this:

#0 0x00000000407206a8 in kill () from /lib/libc.so.6
#1 0x000000004071f0d4 in abort () from /lib/libc.so.6
#2 0x00000000406abc24 in _UTF8_init () from /lib/libc.so.6
#3 0x00000000406abc84 in _UTF8_init () from /lib/libc.so.6
#4 0x00000000406ad028 in _UTF8_init () from /lib/libc.so.6
#5 0x0000000000159b94 in stream_free (s=0x492440) at stream.c:125
#6 0x0000000000128518 in bgp_write (thread=0x13) at bgp_packet.c:99
#7 0x00000000001539b0 in thread_call (thread=0x7fdffffe9c0) at thread.c:1051
#8 0x0000000000102d84 in main (argc=5, argv=0x7fdffffec50) at bgp_main.c:323

if I load a configuration that has no neighbour statements, and then add a
single neighbour with an outgoing route-map:

neighbor 193.149.44.249 remote-as 5539
neighbor 193.149.44.249 route-map set-no-export out

where the route-map exists, and looks like this:

route-map set-no-export permit 10
set community no-export

then I get a crash that looks like this:

#2 0x000000000015d5d0 in _zlog_assert_failed (assertion=0x1a5258 "(((s)->endp)
<= (s)->size)", file=0x1a5178 "stream.c", line=191, function=0x1a4ed0
"stream_get_endp") at log.c:605
#3 0x0000000000159fb4 in stream_get_endp (s=0x49e700) stream.c:191
#4 0x0000000000128340 in bgp_write (thread=0x7fdffffe9f0) at bgp_packet.c:633

to track that down, I have instrumented bgp_write_packet() in bgp_packet.c
with a number of printf()s (not changing anything else!) and from that moment
on, bgpd worked like a charm, no more crashes. Which is bad, otherwise I might
have been able to pinpoint this more closely.

I have the feeling that something is overwriting its bounds in
bgp_update_packet() (I had one crash with corrupt stack frame), but can't see
anything obvious.



------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
_______________________________________________
Quagga-bugs mailing list
Quagga-bugs@lists.quagga.net
http://lists.quagga.net/mailman/listinfo/quagga-bugs
[Bug 392] BGPD crashes with bgpd in free() [ In reply to ]
Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug
report.

http://bugzilla.quagga.net/show_bug.cgi?id=392


gert@greenie.muc.de changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |gert@greenie.muc.de






------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
_______________________________________________
Quagga-bugs mailing list
Quagga-bugs@lists.quagga.net
http://lists.quagga.net/mailman/listinfo/quagga-bugs
[Bug 392] BGPD crashes with bgpd in free() [ In reply to ]
Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug
report.

http://bugzilla.quagga.net/show_bug.cgi?id=392


paul@dishone.st changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|UNCONFIRMED |NEW
Ever Confirmed| |1




------- Additional Comments From paul@dishone.st 2007-08-14 11:49 -------
Aha, you got a stream assertion. So something writing into a stream perhaps?

What is the most minimal config you know of that causes problems?



------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
_______________________________________________
Quagga-bugs mailing list
Quagga-bugs@lists.quagga.net
http://lists.quagga.net/mailman/listinfo/quagga-bugs
[Bug 392] BGPD crashes with bgpd in free() [ In reply to ]
Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug
report.

http://bugzilla.quagga.net/show_bug.cgi?id=392





------- Additional Comments From gardiner@purdigital.net 2007-08-14 12:18 -------
Like Gert's excellent response showed, it appears that the problem involves
route-maps. It doesn't seem to matter whether there are any match or set
statements in the route-map; simply having the statement 'neighbor x.x.x.x
route-map x out' in the config causes the crash. Inbound route-maps don't cause
the crash to occur.

The smallest config I've gotten it to is:

router bgp xxxxx
redistribute static (or network 172.25.1.0 mask 255.255.255.0)
neighbor 172.18.1.1 remote-as xxxxx
neighbor 172.18.1.1 route-map blah out







------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
_______________________________________________
Quagga-bugs mailing list
Quagga-bugs@lists.quagga.net
http://lists.quagga.net/mailman/listinfo/quagga-bugs
[Bug 392] BGPD crashes with bgpd in free() [ In reply to ]
Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug
report.

http://bugzilla.quagga.net/show_bug.cgi?id=392





------- Additional Comments From gert@greenie.muc.de 2007-08-14 13:06 -------
Hi,

my test config for "immediate crash" is this one:

--------------------------------------------------------
!
! Zebra configuration saved from vty
! 2007/07/20 14:10:26
!
hostname ns3-base
password XXX
enable password YYY
!
router bgp 65300
bgp router-id 193.149.44.250
network 193.149.44.49/32
neighbor 193.149.44.253 remote-as 5539
neighbor 193.149.44.253 route-map set-no-export out
!
route-map set-no-export permit 10
set community no-export
!
!
-------------------------------------------------------

with this, I get the "assert" type of crash:

(gdb) run -f bgpd.conf
Starting program: /var/tmp/quagga-0.99.8/bgpd/bgpd -f bgpd.conf

Program received signal SIGABRT, Aborted.
0x00000000407206a8 in kill () from /lib/libc.so.6
(gdb) where
#0 0x00000000407206a8 in kill () from /lib/libc.so.6
#1 0x000000004071f0d4 in abort () from /lib/libc.so.6
#2 0x000000000015d5d0 in _zlog_assert_failed (
assertion=0x1a5258 "(((s)->endp) <= (s)->size)", file=0x1a5178 "stream.c",
line=191, function=0x1a4ed0 "stream_get_endp") at log.c:605
#3 0x0000000000159fb4 in stream_get_endp (s=0x46e200) at stream.c:191
#4 0x0000000000128340 in bgp_write (thread=0x7fdffffea00) at bgp_packet.c:633
#5 0x00000000001539b0 in thread_call (thread=0x7fdffffea00) at thread.c:1051
#6 0x0000000000102d84 in main (argc=3, argv=0x7fdffffec90) at bgp_main.c:323

looking at "s", I agree that it looks fishy:

(gdb) up
#3 0x0000000000159fb4 in stream_get_endp (s=0x46e200) at stream.c:191
191 STREAM_VERIFY_SANE(s);
(gdb) print s
$1 = (struct stream *) 0x46e200
(gdb) print *s
$2 = {next = 0x0, getp = 0, endp = 4734976, size = 0, data = 0x46e280 ""}





------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
_______________________________________________
Quagga-bugs mailing list
Quagga-bugs@lists.quagga.net
http://lists.quagga.net/mailman/listinfo/quagga-bugs
[Bug 392] BGPD crashes with bgpd in free() [ In reply to ]
Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug
report.

http://bugzilla.quagga.net/show_bug.cgi?id=392





------- Additional Comments From gert@greenie.muc.de 2007-08-14 13:07 -------
oh, forgot one thing: having "just the configuration" does not seem to be the
element that triggers it - bgpd needs to have an established TCP session to the
configured neighbour. (Which might be obvious from the traces, but I just want
to note it anyway).

------- Additional Comments From paul@dishone.st 2007-08-20 15:19 -------
FWIW, a Valgrind on i386, with a config similar to the suggested one, does not
reveal any problems. Still investigating.




------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
_______________________________________________
Quagga-bugs mailing list
Quagga-bugs@lists.quagga.net
http://lists.quagga.net/mailman/listinfo/quagga-bugs
[Bug 392] BGPD crashes with bgpd in free() [ In reply to ]
Please do not reply directly to this email. All additional
comments should be made in the comments box of this bug
report.

http://bugzilla.quagga.net/show_bug.cgi?id=392


paul@dishone.st changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |ASSIGNED




------- Additional Comments From paul@dishone.st 2007-08-23 19:21 -------
See bug #398 just opened. Seems i made a mistake and tested the wrong thing
(inbound map), reporter of bug #398 has caught it.

Can you confirm these bugs are the same and whether #398 fixes it?




------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.
_______________________________________________
Quagga-bugs mailing list
Quagga-bugs@lists.quagga.net
http://lists.quagga.net/mailman/listinfo/quagga-bugs