Hi,
I recently encountered an odd issue that my disk is filled up and
after I enlarge the disk space and restarted rsyslog, rsyslog daemon
keeps crashing with following error log in signal SIGSEGV:
action-8-builtin:omfwd queue[DA]: qDeqDisk error happened at around
offset 13746086609 [v8.2112.0 try https://www.rsyslog.com/e/2059
action-8-builtin:omfwd queue[DA]: error dequeueing element - ignoring,
but strange things may happen [v8.2112.0 try
https://www.rsyslog.com/e/2059 ]
I saved the coredump and using gdb to check the data, here's the stack
information:
#0 rsCStrDestruct (ppThis=0x7fffdc075550) at ../runtime/stringbuf.c:222
#1 varDestruct (ppThis=0x7fffe38248a8) at ../runtime/var.c:72
#2 0x000055555559e0fb in MsgDeserialize (pMsg=0x7fffdc026f70,
pStrm=pStrm@entry=0x555555655ce0)
at ../runtime/msg.c:1451
#3 0x00005555555ac07f in objDeserializeWithMethods
(ppObj=0x7fffe3824a28, pszTypeExpected=0x5555555f09fe "msg",
lenTypeExpected=3, pStrm=0x555555655ce0, fFixup=0x0, pUsr=0x0,
objConstruct=0x555555599460 <msgConstructForDeserializer>,
objConstructFinalize=0x0,
objDeserialize=0x55555559e080 <MsgDeserialize>) at ../runtime/obj.c:932
#4 0x00005555555b9c0e in qDeqDisk (pThis=0x55555564ecc0,
ppMsg=<optimized out>) at ../runtime/queue.c:1071
And when I examine the data in pMsg from MsgDeserialize() call, it
seems all psz* timestamps data are some garbage characters like
pszTimestamp3164 = "\000Service member=". I checked the source code
and this struct expects users input. So I have some questions:
1. Is that normal for those garbage characters in psz* timestamp values?
2. After I deleted the corrupted queue file it started working but I
am wondering shall rsyslog skip the entry then move forward to process
the rest of entries?
3. What else shall I look into from coredump at this moment?
Thanks.
Eric
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
I recently encountered an odd issue that my disk is filled up and
after I enlarge the disk space and restarted rsyslog, rsyslog daemon
keeps crashing with following error log in signal SIGSEGV:
action-8-builtin:omfwd queue[DA]: qDeqDisk error happened at around
offset 13746086609 [v8.2112.0 try https://www.rsyslog.com/e/2059
action-8-builtin:omfwd queue[DA]: error dequeueing element - ignoring,
but strange things may happen [v8.2112.0 try
https://www.rsyslog.com/e/2059 ]
I saved the coredump and using gdb to check the data, here's the stack
information:
#0 rsCStrDestruct (ppThis=0x7fffdc075550) at ../runtime/stringbuf.c:222
#1 varDestruct (ppThis=0x7fffe38248a8) at ../runtime/var.c:72
#2 0x000055555559e0fb in MsgDeserialize (pMsg=0x7fffdc026f70,
pStrm=pStrm@entry=0x555555655ce0)
at ../runtime/msg.c:1451
#3 0x00005555555ac07f in objDeserializeWithMethods
(ppObj=0x7fffe3824a28, pszTypeExpected=0x5555555f09fe "msg",
lenTypeExpected=3, pStrm=0x555555655ce0, fFixup=0x0, pUsr=0x0,
objConstruct=0x555555599460 <msgConstructForDeserializer>,
objConstructFinalize=0x0,
objDeserialize=0x55555559e080 <MsgDeserialize>) at ../runtime/obj.c:932
#4 0x00005555555b9c0e in qDeqDisk (pThis=0x55555564ecc0,
ppMsg=<optimized out>) at ../runtime/queue.c:1071
And when I examine the data in pMsg from MsgDeserialize() call, it
seems all psz* timestamps data are some garbage characters like
pszTimestamp3164 = "\000Service member=". I checked the source code
and this struct expects users input. So I have some questions:
1. Is that normal for those garbage characters in psz* timestamp values?
2. After I deleted the corrupted queue file it started working but I
am wondering shall rsyslog skip the entry then move forward to process
the rest of entries?
3. What else shall I look into from coredump at this moment?
Thanks.
Eric
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.