Mailing List Archive

Hi Paul

sorry but still unconfirmed

see attached screenshot of debug-interface as well as
"stress-test.txt" which normally should not output
anything because it prcatically makes a lot of noise
while check again and aagin the same messages against
known checksums

the "Ostrava Tuesday morning" and "multipart_message_6"
which is from the dbmail-autotests imported on my IMAP
seems to get most likely broken, but others to randomly

"Ostrava Tuesday morning is the one with context to
"8a042214ae1d120581740020f4e73c3cf8d3a6c0" and was the
first customer complaint

Regards
Harry

Am 20.07.2014 12:30, schrieb Paul J Stevens:
> It was a buffer overrun which has now been fixed to the best of my
> abilities.
>
> I've done another round of coverity analysis on the master branch, and
> this doesn't trigger any new warnings with regard to mentioned fix which
> is also in the dbmail_3_1 branch.
>
> I've just tagged and thereby released 3.1.16 which contains just this fix.
>
> Being on vacation, I don't have full access to my usual release scripts
> (home-network is unreachable), so a tag will have to do.
>
> But I've also uploaded the tar file seperately to www.dbmail.org, so all
> the usual download locations should work.
>
> On 18-07-14 23:05, Reindl Harald (mobile) wrote:
>> Interesting - your attachment showed exactly the same bug I have only randomly for whatever type of messages and only under load
>>
>> I would sell my soul for a predictable reproducer because in that case debug logging / gdb / strace could shed some light and since it's always the same want finally happens it's pretty sure only a few lines of code in dbmail to fix it
>>
>> I am still sure it's exactly the same code path in both of our cases and only the trigger to went that path is different
>>
>> Maybe Paul has some idea how you can provide debug info's since you are able to isolate what happens while my trigger ends in a large mess of mixed debug logs
>>
>>
>> -------- Ursprüngliche Nachricht --------
>> Von: Guilherme Souza <gslsouza@gmail.com>
>> Gesendet: 18. Juli 2014 21:20:35 MESZ
>> An: DBMail mailinglist <dbmail@dbmail.org>
>> Betreff: Re: [Dbmail] corrupted multi-mime messages
>>
>> I was no able to reply to your message.
>> In my case is not random.
>>
>> Every message html formatted with an attached file sent by ensignia webmail
>> shows the same missing boundaries.
>>
>> Every message sent by roundcube or thunderbird shows correctly.
>>
>>
>> On Fri, Jul 18, 2014 at 2:31 PM, Reindl Harald (mobile)
>> <h.reindl@thelounge.net> wrote:
>>> That is exactly what i reported a few days ago - that's for sure a race condition in reconstruction because i get predictable identical broken messages in a loop while mixed load on my testserver - at the same machine 1000 loops receive the same message without other clients no broken one
>> _______________________________________________
>> DBmail mailing list
>> DBmail@dbmail.org
>> http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
>>
>>
>> --
>>
>> Reindl Harald (mobile)
>> the lounge interactive design GmbH
>> A-1060 Vienna, Hofmühlgasse 17
>> CTO / CISO / Software-Development
>> +43 (676) 40 221 40
>> http://www.thelounge.net
>> _______________________________________________
>> DBmail mailing list
>> DBmail@dbmail.org
>> http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
>>
>

--

Reindl Harald
the lounge interactive design GmbH
A-1060 Vienna, Hofmühlgasse 17
CTO / CISO / Software-Development
m: +43 (676) 40 221 40, p: +43 (1) 595 3999 33
icq: 154546673, http://www.thelounge.net/

http://www.thelounge.net/signature.asc.what.htm

*attached* a compressed example *diff* broken / non-broken
copy while compare the same mail IMAP / POP3 in a loop

33 out of 1000 always identically broken

it don't matter on which test message i start the loop
the result is the same in case of multi-mime mails

fact is there are times while running mixed load for a minute
not a single one get broken and in random time windows any
multimime message for some seconds is damaged the same way
_______________________________________________________________

that below is the temp-folder where the loop stores diffs
look at the number at the end, that's the loop-comuter

at the begin of the loop a horrible result with 19 broken
copies in serie and later 118-125 are again 8 in serie and
they all have the same difference as "example-diff.zip"

the mixed load now get "multipart_message_3" instead
"multipart_message_6" most of the time broken
_______________________________________________________________

dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_002.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_003.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_004.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_005.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_006.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_007.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_009.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_010.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_011.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_012.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_013.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_014.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_015.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_016.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_017.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_018.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_019.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_020.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_021.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_084.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_089.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_118.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_119.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_120.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_121.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_122.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_123.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_124.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_125.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_335.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_364.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_380.diff
dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_707.diff
_______________________________________________

Jul 20 13:26:56 POP3 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
Jul 20 13:26:56 IMAP 24: EXPECTED: 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND:
97039dbf12052e527808ca16188971a405fcb529 (multipart_message_3)
Jul 20 13:26:56 IMAP 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
Jul 20 13:27:23 POP3 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
Jul 20 13:27:23 IMAP 24: EXPECTED: 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND:
97039dbf12052e527808ca16188971a405fcb529 (multipart_message_3)
Jul 20 13:27:23 IMAP 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
Jul 20 13:27:52 POP3 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
Jul 20 13:27:52 IMAP 24: EXPECTED: 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND:
97039dbf12052e527808ca16188971a405fcb529 (multipart_message_3)
Jul 20 13:27:52 IMAP 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
Jul 20 13:28:15 POP3 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
Jul 20 13:28:15 IMAP 24: EXPECTED: 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND:
97039dbf12052e527808ca16188971a405fcb529 (multipart_message_3)
Jul 20 13:28:15 IMAP 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
Am 20.07.2014 12:56, schrieb Reindl Harald:
> Hi Paul
>
> sorry but still unconfirmed
>
> see attached screenshot of debug-interface as well as
> "stress-test.txt" which normally should not output
> anything because it prcatically makes a lot of noise
> while check again and aagin the same messages against
> known checksums
>
> the "Ostrava Tuesday morning" and "multipart_message_6"
> which is from the dbmail-autotests imported on my IMAP
> seems to get most likely broken, but others to randomly
>
> "Ostrava Tuesday morning is the one with context to
> "8a042214ae1d120581740020f4e73c3cf8d3a6c0" and was the
> first customer complaint
>
> Regards
> Harry
>
> Am 20.07.2014 12:30, schrieb Paul J Stevens:
>> It was a buffer overrun which has now been fixed to the best of my
>> abilities.
>>
>> I've done another round of coverity analysis on the master branch, and
>> this doesn't trigger any new warnings with regard to mentioned fix which
>> is also in the dbmail_3_1 branch.
>>
>> I've just tagged and thereby released 3.1.16 which contains just this fix.
>>
>> Being on vacation, I don't have full access to my usual release scripts
>> (home-network is unreachable), so a tag will have to do.
>>
>> But I've also uploaded the tar file seperately to www.dbmail.org, so all
>> the usual download locations should work.
>>
>> On 18-07-14 23:05, Reindl Harald (mobile) wrote:
>>> Interesting - your attachment showed exactly the same bug I have only randomly for whatever type of messages and only under load
>>>
>>> I would sell my soul for a predictable reproducer because in that case debug logging / gdb / strace could shed some light and since it's always the same want finally happens it's pretty sure only a few lines of code in dbmail to fix it
>>>
>>> I am still sure it's exactly the same code path in both of our cases and only the trigger to went that path is different
>>>
>>> Maybe Paul has some idea how you can provide debug info's since you are able to isolate what happens while my trigger ends in a large mess of mixed debug logs
>>>
>>>
>>> -------- Ursprüngliche Nachricht --------
>>> Von: Guilherme Souza <gslsouza@gmail.com>
>>> Gesendet: 18. Juli 2014 21:20:35 MESZ
>>> An: DBMail mailinglist <dbmail@dbmail.org>
>>> Betreff: Re: [Dbmail] corrupted multi-mime messages
>>>
>>> I was no able to reply to your message.
>>> In my case is not random.
>>>
>>> Every message html formatted with an attached file sent by ensignia webmail
>>> shows the same missing boundaries.
>>>
>>> Every message sent by roundcube or thunderbird shows correctly.
>>>
>>>
>>> On Fri, Jul 18, 2014 at 2:31 PM, Reindl Harald (mobile)
>>> <h.reindl@thelounge.net> wrote:
>>>> That is exactly what i reported a few days ago - that's for sure a race condition in reconstruction because i get predictable identical broken messages in a loop while mixed load on my testserver - at the same machine 1000 loops receive the same message without other clients no broken one
>>> _______________________________________________
>>> DBmail mailing list
>>> DBmail@dbmail.org
>>> http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
>>>
>>>
>>> --
>>>
>>> Reindl Harald (mobile)
>>> the lounge interactive design GmbH
>>> A-1060 Vienna, Hofmühlgasse 17
>>> CTO / CISO / Software-Development
>>> +43 (676) 40 221 40
>>> http://www.thelounge.net
>>> _______________________________________________
>>> DBmail mailing list
>>> DBmail@dbmail.org
>>> http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
>>>
>>
>
>
>
> _______________________________________________
> DBmail mailing list
> DBmail@dbmail.org
> http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
>

--

Reindl Harald
the lounge interactive design GmbH
A-1060 Vienna, Hofmühlgasse 17
CTO / CISO / Software-Development
m: +43 (676) 40 221 40, p: +43 (1) 595 3999 33
icq: 154546673, http://www.thelounge.net/

http://www.thelounge.net/signature.asc.what.htm