Mailing List Archive

In memory decoding?
Been looking over RipMIME, and it looks like a great, and very useful
library.

One problem though. The program I want to integrate it with processes
messages completely in memory.

Is it possible to have ripMIME process a message in memory, and then
return a linked list of binary attachments to my program for virus scanning?

Given that 99.999% of all e-mail messages are less than 10 megs, and
that a large chunk (esp. servers) of computers have a gigabyte, or more
of memory, it doesn't make sense to write a message to disk. If the
computer runs out of room in memory, the kernal will automatically swap
to disk what isn't needed.

I searched through the library and couldn't find such a function but I
may have missed something. Has anyone developed a patch that would give
me that ability?

-- L~
_______________________________________________
Ripmime-general mailing list
Ripmime-general@pldaniels.com
http://www.pldaniels.com/mailman/listinfo/ripmime-general
Re: In memory decoding? [ In reply to ]
Ladar,

No, you've not missed anything. At this point there is no 'in memory' processing option for ripMIME; however, it is
being planned. I simply do not know at what point I'll have it out, possibly version 2.x or 1.4.x.

The problem is that at some mail-size point (which would be user-defined, of course) ripMIME would have to revert back
to being a on-disk decoder, simply to avoid it consuming all the resources. While 99.9% of emails would be decoded in
memory, I know that there will be always a 0.1% which break the rules (ripMIME is perpetually shaped by emails that
break the rules).

So, to summarise, currently there is no memory-based decoding or in-memory file store.

Regards.

--
Paul L Daniels - PLD Software - Xamime
Unix systems Internet Development A.B.N. 19 500 721 806
PGP Public Key at http://www.pldaniels.com/gpg-keys.pld

_______________________________________________
Ripmime-general mailing list
Ripmime-general@pldaniels.com
http://www.pldaniels.com/mailman/listinfo/ripmime-general
Re: In memory decoding? [ In reply to ]
Definately add me to the list of people who would like to see that feature.

In the meantime, I will code the app to use files (mostly because I want to
get this thing done...).

In the future, when I have a bit of spare time, would you be willing to
accept a patch which integrates such a feature?

The only reason I haven't written it already is because the way you
architected your library, every function assumes its working with a file
handle. Correct me if I am wrong, but right now I would have to duplicate
every function you use, and alter the duplicated function so that it
operates on buffers instead of file handles?

I don't have the time at the moment to go through and change every function,
but I may tackle such a chanellege once this project is complete.

Do you have a better plan for implementation, something I might not have
considered? Is there a function I could pass a buffer to, and have it
operate on already?


-- L~

----- Original Message -----
Date: Thu August 05, 2004 08:21 AM
From: Paul L Daniels <pldaniels@pldaniels.com>
To: Ladar Levison <ladar@nerdshack.com>
cc: ripmime-general@pldaniels.com
Subject: Re: [Ripmime-general] In memory decoding?

Ladar,

No, you've not missed anything. At this point there is no 'in memory'
processing option for ripMIME; however, it is
being planned. I simply do not know at what point I'll have it out,
possibly version 2.x or 1.4.x.

The problem is that at some mail-size point (which would be user-defined,
of course) ripMIME would have to revert back
to being a on-disk decoder, simply to avoid it consuming all the resources.
While 99.9% of emails would be decoded in
memory, I know that there will be always a 0.1% which break the rules
(ripMIME is perpetually shaped by emails that
break the rules).

So, to summarise, currently there is no memory-based decoding or in-memory
file store.

Regards.

--
Paul L Daniels - PLD Software - Xamime
Unix systems Internet Development A.B.N. 19 500 721 806
PGP Public Key at http://www.pldaniels.com/gpg-keys.pld


_______________________________________________
Ripmime-general mailing list
Ripmime-general@pldaniels.com
http://www.pldaniels.com/mailman/listinfo/ripmime-general
Re: In memory decoding? [ In reply to ]
Hello,

Ladar Levison wrote:

>The only reason I haven't written it already is because the way you
>architected your library, every function assumes its working with a file
>handle. Correct me if I am wrong, but right now I would have to duplicate
>every function you use, and alter the duplicated function so that it
>operates on buffers instead of file handles?
>
>
I had already done this a couple years back when I first joined the
list. Since then I've coded my own mime decoding. One thing you have to
be careful of is the huge amounts of memory it uses. Our system now uses
temp files instead of files named like attachments. I like our solution
and still use ripmime to compare decoded files, etc. (Great software!)

If anything, I'd just change ripmime to use temp files instead of
in-memory files and save the filenames, etc in memory somewhere.

Cheers,
Ben

_______________________________________________
Ripmime-general mailing list
Ripmime-general@pldaniels.com
http://www.pldaniels.com/mailman/listinfo/ripmime-general
Re: In memory decoding? [ In reply to ]
Ladar Levison wrote:

>Howdy Ben --
>
>Is your code open-source? I'd like to take a look at it.
>
>-- L~


Please see attached. =)

This is what we used in the first version of our parser. Modified
ripmime, etc. with in-memory decoding. I release this to you under the
MIT license. And browsing through the code I noticed some other libs
being used (linkedlist, etc). Please see this page for those libs:
http://www.webgroupmedia.com/software/

Cheers,
Ben

PS: sorry for the delay, we've been busy.