Mailing List Archive

Am 15.07.2014 19:36, schrieb Reindl Harald:
> am i the only one receiving randomly broken messages
> with POP3 as well as IMAP under concurrency?
>
> every time when this happens there are lines with only "--"
> instead for example "--------------09080803050403000503070"
>
> see also atteched screenshot of compare
>
> no idea what here happens but it brings me really in large
> troubles because at some days it happens way too often and
> typically your only chance to get the message in a clean
> state is use roundcube - not always possible for POP3 users
> deleting mails from the server directly after receive
>
> i miss feedback from Paul and hope he is well but only busy

that is pretty sure a memory problem due reconstruct and if
it hits the results for specific messages are AFAIK
predictable, hopefully the screenshot from the debuginterface
is not to large for the list

see checksums for the mailbody in both error cases - identical
it *seems* to affect more POP3 than IMAP, but i was also
affected last friday with IMAP, a completly empty message
in thunderbird with a similar raw-source not able to display

12baedbadb4b0ed23dbe167325fbe90a6188eff2 / 12baedbadb4b0ed23dbe167325fbe90a6188eff2
12baedbadb4b0ed23dbe167325fbe90a6188eff2 / 12baedbadb4b0ed23dbe167325fbe90a6188eff2
12baedbadb4b0ed23dbe167325fbe90a6188eff2 / 12baedbadb4b0ed23dbe167325fbe90a6188eff2
8a95d13939171542c06966d1627fdab113f564f0 / 12baedbadb4b0ed23dbe167325fbe90a6188eff2
12baedbadb4b0ed23dbe167325fbe90a6188eff2 / 12baedbadb4b0ed23dbe167325fbe90a6188eff2
12baedbadb4b0ed23dbe167325fbe90a6188eff2 / 12baedbadb4b0ed23dbe167325fbe90a6188eff2
8a95d13939171542c06966d1627fdab113f564f0 / 12baedbadb4b0ed23dbe167325fbe90a6188eff2

and it happens only with concurrency

* 1000 times the same message POP3/IMAP sha1-match

* 100 times the same compare loop with concurrency
POP3/IMAP from other scripts fetching different
messages, change flags and mix several loads leads
in 2 or 3 mismatches on the otherwise clean loop

i wanna die :-(

Am 15.07.2014 19:43, schrieb Reindl Harald:
> Am 15.07.2014 19:36, schrieb Reindl Harald:
>> am i the only one receiving randomly broken messages
>> with POP3 as well as IMAP under concurrency?
>>
>> every time when this happens there are lines with only "--"
>> instead for example "--------------09080803050403000503070"
>>
>> see also atteched screenshot of compare
>>
>> no idea what here happens but it brings me really in large
>> troubles because at some days it happens way too often and
>> typically your only chance to get the message in a clean
>> state is use roundcube - not always possible for POP3 users
>> deleting mails from the server directly after receive
>>
>> i miss feedback from Paul and hope he is well but only busy
>
> that is pretty sure a memory problem due reconstruct and if
> it hits the results for specific messages are AFAIK
> predictable, hopefully the screenshot from the debuginterface
> is not to large for the list
>
> see checksums for the mailbody in both error cases - identical
> it *seems* to affect more POP3 than IMAP, but i was also
> affected last friday with IMAP, a completly empty message
> in thunderbird with a similar raw-source not able to display
>
> 12baedbadb4b0ed23dbe167325fbe90a6188eff2 / 12baedbadb4b0ed23dbe167325fbe90a6188eff2
> 12baedbadb4b0ed23dbe167325fbe90a6188eff2 / 12baedbadb4b0ed23dbe167325fbe90a6188eff2
> 12baedbadb4b0ed23dbe167325fbe90a6188eff2 / 12baedbadb4b0ed23dbe167325fbe90a6188eff2
> 8a95d13939171542c06966d1627fdab113f564f0 / 12baedbadb4b0ed23dbe167325fbe90a6188eff2
> 12baedbadb4b0ed23dbe167325fbe90a6188eff2 / 12baedbadb4b0ed23dbe167325fbe90a6188eff2
> 12baedbadb4b0ed23dbe167325fbe90a6188eff2 / 12baedbadb4b0ed23dbe167325fbe90a6188eff2
> 8a95d13939171542c06966d1627fdab113f564f0 / 12baedbadb4b0ed23dbe167325fbe90a6188eff2

I am using dmail 3.1.14

Sending html formatted email with an attached file results in
missing mime boundaries.

For some reason it happens with ensignia webmail [squirrelmail] with
an html plugin.
It does not happen with roundcube or thunderbird imap client, for instance.

Message sent to other mail servers is OK, like gmail or yahoo is ok.

Attached image shows missing boundaries.

That is exactly what i reported a few days ago - that's for sure a race condition in reconstruction because i get predictable identical broken messages in a loop while mixed load on my testserver - at the same machine 1000 loops receive the same message without other clients no broken one

I managed a peak of 45 identical broken ones out of 1000 and sent the diffs Paul (wrongly mentioned they differ while the file checksum also contains the temp names inside the diff files and their content is otherwise identical)

I am not at home but you should find my thread easily in the archives

All broken messages have lines with only - - while expecting a mime id after that

It happens as said randomly depending on how many other clients at the same time are receiving messages and is completely independent of the client

It affects both, POP3 and IMAP while POP3 seems to be harder affected, at least in my tests - anyways, I had myself broken messages via IMAP this week

One chance is to logout and login in roundcube to force a reload of the message or in case of desktop clients use the Webmail and forward a non broken copy

I am somehow happy about your report because it confirms that it's not my environment and here are fedora 19 machines affected the same way as Fedora 20


-------- Ursprüngliche Nachricht --------
Von: Guilherme Souza <gslsouza@gmail.com>
Gesendet: 18. Juli 2014 19:05:38 MESZ
An: dbmail@dbmail.org
Betreff: [Dbmail] corrupted multi-mime messages

I am using dmail 3.1.14

Sending html formatted email with an attached file results in
missing mime boundaries.

For some reason it happens with ensignia webmail [squirrelmail] with
an html plugin.
It does not happen with roundcube or thunderbird imap client, for instance.

Message sent to other mail servers is OK, like gmail or yahoo is ok.

Attached image shows missing boundaries.


------------------------------------------------------------------------

_______________________________________________
DBmail mailing list
DBmail@dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail


--

Reindl Harald (mobile)
the lounge interactive design GmbH
A-1060 Vienna, Hofmühlgasse 17
CTO / CISO / Software-Development
+43 (676) 40 221 40
http://www.thelounge.net
_______________________________________________
DBmail mailing list
DBmail@dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail

I was no able to reply to your message.
In my case is not random.

Every message html formatted with an attached file sent by ensignia webmail
shows the same missing boundaries.

Every message sent by roundcube or thunderbird shows correctly.


On Fri, Jul 18, 2014 at 2:31 PM, Reindl Harald (mobile)
<h.reindl@thelounge.net> wrote:
> That is exactly what i reported a few days ago - that's for sure a race condition in reconstruction because i get predictable identical broken messages in a loop while mixed load on my testserver - at the same machine 1000 loops receive the same message without other clients no broken one
_______________________________________________
DBmail mailing list
DBmail@dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail

I was able to fix the error modifying the way squirremail
builds boundaries. Now I can see the text of the messages.
Every message I test now seems to be OK.

Instead of:
Content-Type: multipart/mixed; boundary="----=_20140717124156_79350"

I changed to:
Content-Type: multipart/mixed; boundary="=_2014071712415679350"

And

multipart/alternative;boundary="1295fsu_trap_05397_65142171704102_=----"

to:
multipart/alternative;boundary="=_12950539765142171704102"
_______________________________________________
DBmail mailing list
DBmail@dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail

Interesting - your attachment showed exactly the same bug I have only randomly for whatever type of messages and only under load

I would sell my soul for a predictable reproducer because in that case debug logging / gdb / strace could shed some light and since it's always the same want finally happens it's pretty sure only a few lines of code in dbmail to fix it

I am still sure it's exactly the same code path in both of our cases and only the trigger to went that path is different

Maybe Paul has some idea how you can provide debug info's since you are able to isolate what happens while my trigger ends in a large mess of mixed debug logs


-------- Ursprüngliche Nachricht --------
Von: Guilherme Souza <gslsouza@gmail.com>
Gesendet: 18. Juli 2014 21:20:35 MESZ
An: DBMail mailinglist <dbmail@dbmail.org>
Betreff: Re: [Dbmail] corrupted multi-mime messages

I was no able to reply to your message.
In my case is not random.

Every message html formatted with an attached file sent by ensignia webmail
shows the same missing boundaries.

Every message sent by roundcube or thunderbird shows correctly.


On Fri, Jul 18, 2014 at 2:31 PM, Reindl Harald (mobile)
<h.reindl@thelounge.net> wrote:
> That is exactly what i reported a few days ago - that's for sure a race condition in reconstruction because i get predictable identical broken messages in a loop while mixed load on my testserver - at the same machine 1000 loops receive the same message without other clients no broken one
_______________________________________________
DBmail mailing list
DBmail@dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail


--

Reindl Harald (mobile)
the lounge interactive design GmbH
A-1060 Vienna, Hofmühlgasse 17
CTO / CISO / Software-Development
+43 (676) 40 221 40
http://www.thelounge.net
_______________________________________________
DBmail mailing list
DBmail@dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail