CAUTION
3.1.16 at least has a regression in case of the commit below
that's a different problem and the testmessage with downgrading
is reconstructed proper
http://git.dbmail.eu/paul/dbmail/commit/?h=dbmail_3_1&id=8a042214ae1d120581740020f4e73c3cf8d3a6c0
however, the mime-part-id problem is still the same under load
i sent Paul some debug data offlist because they contain
real messages i am allowed by the mailowner to forward
only that way, not in public
Regards
Harry
Am 20.07.2014 13:38, schrieb Reindl Harald:
> *attached* a compressed example *diff* broken / non-broken
> copy while compare the same mail IMAP / POP3 in a loop
>
> 33 out of 1000 always identically broken
>
> it don't matter on which test message i start the loop
> the result is the same in case of multi-mime mails
>
> fact is there are times while running mixed load for a minute
> not a single one get broken and in random time windows any
> multimime message for some seconds is damaged the same way
> _______________________________________________________________
>
> that below is the temp-folder where the loop stores diffs
> look at the number at the end, that's the loop-comuter
>
> at the begin of the loop a horrible result with 19 broken
> copies in serie and later 118-125 are again 8 in serie and
> they all have the same difference as "example-diff.zip"
>
> the mixed load now get "multipart_message_3" instead
> "multipart_message_6" most of the time broken
> _______________________________________________________________
>
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_002.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_003.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_004.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_005.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_006.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_007.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_009.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_010.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_011.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_012.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_013.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_014.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_015.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_016.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_017.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_018.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_019.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_020.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_021.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_084.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_089.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_118.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_119.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_120.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_121.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_122.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_123.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_124.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_125.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_335.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_364.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_380.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_707.diff
> _______________________________________________
>
> Jul 20 13:26:56 POP3 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
> Jul 20 13:26:56 IMAP 24: EXPECTED: 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND:
> 97039dbf12052e527808ca16188971a405fcb529 (multipart_message_3)
> Jul 20 13:26:56 IMAP 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
> Jul 20 13:27:23 POP3 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
> Jul 20 13:27:23 IMAP 24: EXPECTED: 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND:
> 97039dbf12052e527808ca16188971a405fcb529 (multipart_message_3)
> Jul 20 13:27:23 IMAP 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
> Jul 20 13:27:52 POP3 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
> Jul 20 13:27:52 IMAP 24: EXPECTED: 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND:
> 97039dbf12052e527808ca16188971a405fcb529 (multipart_message_3)
> Jul 20 13:27:52 IMAP 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
> Jul 20 13:28:15 POP3 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
> Jul 20 13:28:15 IMAP 24: EXPECTED: 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND:
> 97039dbf12052e527808ca16188971a405fcb529 (multipart_message_3)
> Jul 20 13:28:15 IMAP 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
> Am 20.07.2014 12:56, schrieb Reindl Harald:
>> Hi Paul
>>
>> sorry but still unconfirmed
>>
>> see attached screenshot of debug-interface as well as
>> "stress-test.txt" which normally should not output
>> anything because it prcatically makes a lot of noise
>> while check again and aagin the same messages against
>> known checksums
>>
>> the "Ostrava Tuesday morning" and "multipart_message_6"
>> which is from the dbmail-autotests imported on my IMAP
>> seems to get most likely broken, but others to randomly
>>
>> "Ostrava Tuesday morning is the one with context to
>> "8a042214ae1d120581740020f4e73c3cf8d3a6c0" and was the
>> first customer complaint
>>
>> Regards
>> Harry
>>
>> Am 20.07.2014 12:30, schrieb Paul J Stevens:
>>> It was a buffer overrun which has now been fixed to the best of my
>>> abilities.
>>>
>>> I've done another round of coverity analysis on the master branch, and
>>> this doesn't trigger any new warnings with regard to mentioned fix which
>>> is also in the dbmail_3_1 branch.
>>>
>>> I've just tagged and thereby released 3.1.16 which contains just this fix.
>>>
>>> Being on vacation, I don't have full access to my usual release scripts
>>> (home-network is unreachable), so a tag will have to do.
>>>
>>> But I've also uploaded the tar file seperately to www.dbmail.org, so all
>>> the usual download locations should work.
>>>
>>> On 18-07-14 23:05, Reindl Harald (mobile) wrote:
>>>> Interesting - your attachment showed exactly the same bug I have only randomly for whatever type of messages and only under load
>>>>
>>>> I would sell my soul for a predictable reproducer because in that case debug logging / gdb / strace could shed some light and since it's always the same want finally happens it's pretty sure only a few lines of code in dbmail to fix it
>>>>
>>>> I am still sure it's exactly the same code path in both of our cases and only the trigger to went that path is different
>>>>
>>>> Maybe Paul has some idea how you can provide debug info's since you are able to isolate what happens while my trigger ends in a large mess of mixed debug logs
>>>>
>>>>
>>>> -------- Ursprüngliche Nachricht --------
>>>> Von: Guilherme Souza <gslsouza@gmail.com>
>>>> Gesendet: 18. Juli 2014 21:20:35 MESZ
>>>> An: DBMail mailinglist <dbmail@dbmail.org>
>>>> Betreff: Re: [Dbmail] corrupted multi-mime messages
>>>>
>>>> I was no able to reply to your message.
>>>> In my case is not random.
>>>>
>>>> Every message html formatted with an attached file sent by ensignia webmail
>>>> shows the same missing boundaries.
>>>>
>>>> Every message sent by roundcube or thunderbird shows correctly.
>>>>
>>>>
>>>> On Fri, Jul 18, 2014 at 2:31 PM, Reindl Harald (mobile)
>>>> <h.reindl@thelounge.net> wrote:
>>>>> That is exactly what i reported a few days ago - that's for sure a race condition in reconstruction because i get predictable identical broken messages in a loop while mixed load on my testserver - at the same machine 1000 loops receive the same message without other clients no broken one
3.1.16 at least has a regression in case of the commit below
that's a different problem and the testmessage with downgrading
is reconstructed proper
http://git.dbmail.eu/paul/dbmail/commit/?h=dbmail_3_1&id=8a042214ae1d120581740020f4e73c3cf8d3a6c0
however, the mime-part-id problem is still the same under load
i sent Paul some debug data offlist because they contain
real messages i am allowed by the mailowner to forward
only that way, not in public
Regards
Harry
Am 20.07.2014 13:38, schrieb Reindl Harald:
> *attached* a compressed example *diff* broken / non-broken
> copy while compare the same mail IMAP / POP3 in a loop
>
> 33 out of 1000 always identically broken
>
> it don't matter on which test message i start the loop
> the result is the same in case of multi-mime mails
>
> fact is there are times while running mixed load for a minute
> not a single one get broken and in random time windows any
> multimime message for some seconds is damaged the same way
> _______________________________________________________________
>
> that below is the temp-folder where the loop stores diffs
> look at the number at the end, that's the loop-comuter
>
> at the begin of the loop a horrible result with 19 broken
> copies in serie and later 118-125 are again 8 in serie and
> they all have the same difference as "example-diff.zip"
>
> the mixed load now get "multipart_message_3" instead
> "multipart_message_6" most of the time broken
> _______________________________________________________________
>
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_002.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_003.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_004.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_005.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_006.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_007.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_009.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_010.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_011.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_012.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_013.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_014.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_015.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_016.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_017.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_018.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_019.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_020.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_021.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_084.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_089.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_118.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_119.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_120.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_121.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_122.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_123.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_124.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_125.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_335.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_364.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_380.diff
> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_707.diff
> _______________________________________________
>
> Jul 20 13:26:56 POP3 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
> Jul 20 13:26:56 IMAP 24: EXPECTED: 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND:
> 97039dbf12052e527808ca16188971a405fcb529 (multipart_message_3)
> Jul 20 13:26:56 IMAP 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
> Jul 20 13:27:23 POP3 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
> Jul 20 13:27:23 IMAP 24: EXPECTED: 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND:
> 97039dbf12052e527808ca16188971a405fcb529 (multipart_message_3)
> Jul 20 13:27:23 IMAP 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
> Jul 20 13:27:52 POP3 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
> Jul 20 13:27:52 IMAP 24: EXPECTED: 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND:
> 97039dbf12052e527808ca16188971a405fcb529 (multipart_message_3)
> Jul 20 13:27:52 IMAP 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
> Jul 20 13:28:15 POP3 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
> Jul 20 13:28:15 IMAP 24: EXPECTED: 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND:
> 97039dbf12052e527808ca16188971a405fcb529 (multipart_message_3)
> Jul 20 13:28:15 IMAP 33: EXPECTED: 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND:
> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday morning)
> Am 20.07.2014 12:56, schrieb Reindl Harald:
>> Hi Paul
>>
>> sorry but still unconfirmed
>>
>> see attached screenshot of debug-interface as well as
>> "stress-test.txt" which normally should not output
>> anything because it prcatically makes a lot of noise
>> while check again and aagin the same messages against
>> known checksums
>>
>> the "Ostrava Tuesday morning" and "multipart_message_6"
>> which is from the dbmail-autotests imported on my IMAP
>> seems to get most likely broken, but others to randomly
>>
>> "Ostrava Tuesday morning is the one with context to
>> "8a042214ae1d120581740020f4e73c3cf8d3a6c0" and was the
>> first customer complaint
>>
>> Regards
>> Harry
>>
>> Am 20.07.2014 12:30, schrieb Paul J Stevens:
>>> It was a buffer overrun which has now been fixed to the best of my
>>> abilities.
>>>
>>> I've done another round of coverity analysis on the master branch, and
>>> this doesn't trigger any new warnings with regard to mentioned fix which
>>> is also in the dbmail_3_1 branch.
>>>
>>> I've just tagged and thereby released 3.1.16 which contains just this fix.
>>>
>>> Being on vacation, I don't have full access to my usual release scripts
>>> (home-network is unreachable), so a tag will have to do.
>>>
>>> But I've also uploaded the tar file seperately to www.dbmail.org, so all
>>> the usual download locations should work.
>>>
>>> On 18-07-14 23:05, Reindl Harald (mobile) wrote:
>>>> Interesting - your attachment showed exactly the same bug I have only randomly for whatever type of messages and only under load
>>>>
>>>> I would sell my soul for a predictable reproducer because in that case debug logging / gdb / strace could shed some light and since it's always the same want finally happens it's pretty sure only a few lines of code in dbmail to fix it
>>>>
>>>> I am still sure it's exactly the same code path in both of our cases and only the trigger to went that path is different
>>>>
>>>> Maybe Paul has some idea how you can provide debug info's since you are able to isolate what happens while my trigger ends in a large mess of mixed debug logs
>>>>
>>>>
>>>> -------- Ursprüngliche Nachricht --------
>>>> Von: Guilherme Souza <gslsouza@gmail.com>
>>>> Gesendet: 18. Juli 2014 21:20:35 MESZ
>>>> An: DBMail mailinglist <dbmail@dbmail.org>
>>>> Betreff: Re: [Dbmail] corrupted multi-mime messages
>>>>
>>>> I was no able to reply to your message.
>>>> In my case is not random.
>>>>
>>>> Every message html formatted with an attached file sent by ensignia webmail
>>>> shows the same missing boundaries.
>>>>
>>>> Every message sent by roundcube or thunderbird shows correctly.
>>>>
>>>>
>>>> On Fri, Jul 18, 2014 at 2:31 PM, Reindl Harald (mobile)
>>>> <h.reindl@thelounge.net> wrote:
>>>>> That is exactly what i reported a few days ago - that's for sure a race condition in reconstruction because i get predictable identical broken messages in a loop while mixed load on my testserver - at the same machine 1000 loops receive the same message without other clients no broken one