Mailing List Archive

For the uber-large headers problem, would you rather bounce the message or add
the is_header column to the messageblks table and allow for multiple header
blocks? Also, since we currently pass the header around as a single char
array, how would you propose to throttle the memory usage so that someone
cannot DoS the server with garbage jigabyte headers?

Aaron


Ilja Booij <ilja@ic-s.nl> said:

> Hi,
>
> I'd like to release rc4 tomorrow. Is anybody aware of any showstoppers
> we have to fix before releasing rc4 other than the ones I'll mention below?
>
> * Headers > 512kB don't work. Messages get dropped.
>
> * code indentation: run indent -i8 -kr (unless anybody wants something
> completely different :) )
>
> Ilja
>
>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>



--

Having a flag on the messageblks is great for joins (see previous post); but
if there is the possibility of multiple header blocks that may pose a
problem.

However -- you could set the value to "1" for 1 header block, "2" for
multiple header blocks (or qty, etc) -- and then if when you do the simple
join:

E.g. select messages.*, messages.isheader, messageblks.messageblk as
headers from
messages,messageblks where
messages.message_idnr=messageblks.messageblks_idnr and
messageblks.isheader=1 order by <SORT KEY>

If the 'isheader' flag returned 1, then *just for that row* the client
applications could do a second pass where they looked up all the header
fields.

/Mark

PS. Happy with bouncing messages with long headers though myself -- bound to
be spammers...



> -----Original Message-----
> From: dbmail-dev-bounces@dbmail.org
> [mailto:dbmail-dev-bounces@dbmail.org] On Behalf Of Aaron Stone
> Sent: Friday, 19 March 2004 7:53 a.m.
> To: DBMAIL Developers Mailinglist
> Subject: Re: [Dbmail-dev] dbmail 2.0 rc4
>
> For the uber-large headers problem, would you rather bounce
> the message or add
> the is_header column to the messageblks table and allow for
> multiple header
> blocks? Also, since we currently pass the header around as a
> single char
> array, how would you propose to throttle the memory usage so
> that someone
> cannot DoS the server with garbage jigabyte headers?
>
> Aaron
>
>
> Ilja Booij <ilja@ic-s.nl> said:
>
> > Hi,
> >
> > I'd like to release rc4 tomorrow. Is anybody aware of any
> showstoppers
> > we have to fix before releasing rc4 other than the ones
> I'll mention below?
> >
> > * Headers > 512kB don't work. Messages get dropped.
> >
> > * code indentation: run indent -i8 -kr (unless anybody
> wants something
> > completely different :) )
> >
> > Ilja
> >
> >
> > _______________________________________________
> > Dbmail-dev mailing list
> > Dbmail-dev@dbmail.org
> > http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >
>
>
>
> --
>
>
>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>

for now I'll go for bouncing messages with ueberlong headers. Later on,
we'll have to decide on a nice strategy to work with longer headers.

Ilja

Aaron Stone wrote:

> For the uber-large headers problem, would you rather bounce the message or add
> the is_header column to the messageblks table and allow for multiple header
> blocks? Also, since we currently pass the header around as a single char
> array, how would you propose to throttle the memory usage so that someone
> cannot DoS the server with garbage jigabyte headers?
>
> Aaron
>
>
> Ilja Booij <ilja@ic-s.nl> said:
>
>
>>Hi,
>>
>>I'd like to release rc4 tomorrow. Is anybody aware of any showstoppers
>>we have to fix before releasing rc4 other than the ones I'll mention below?
>>
>>* Headers > 512kB don't work. Messages get dropped.
>>
>>* code indentation: run indent -i8 -kr (unless anybody wants something
>>completely different :) )
>>
>>Ilja
>>
>>
>>_______________________________________________
>>Dbmail-dev mailing list
>>Dbmail-dev@dbmail.org
>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>
>
>
>
>

I wonder what message dbmail-smtp should send to the MTA when the header
is to big: EX_DATAERR?

EX_TEMPFAIL does not seem like the right messages, but I wonder if
EX_DATAERR would be understood by the MTA.

Any ideas?

Ilja


Ilja Booij wrote:

> for now I'll go for bouncing messages with ueberlong headers. Later on,
> we'll have to decide on a nice strategy to work with longer headers.
>
> Ilja
>
> Aaron Stone wrote:
>
>> For the uber-large headers problem, would you rather bounce the
>> message or add
>> the is_header column to the messageblks table and allow for multiple
>> header
>> blocks? Also, since we currently pass the header around as a single char
>> array, how would you propose to throttle the memory usage so that someone
>> cannot DoS the server with garbage jigabyte headers?
>>
>> Aaron
>>
>>
>> Ilja Booij <ilja@ic-s.nl> said:
>>
>>
>>> Hi,
>>>
>>> I'd like to release rc4 tomorrow. Is anybody aware of any
>>> showstoppers we have to fix before releasing rc4 other than the ones
>>> I'll mention below?
>>>
>>> * Headers > 512kB don't work. Messages get dropped.
>>>
>>> * code indentation: run indent -i8 -kr (unless anybody wants
>>> something completely different :) )
>>>
>>> Ilja
>>>
>>>
>>> _______________________________________________
>>> Dbmail-dev mailing list
>>> Dbmail-dev@dbmail.org
>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>
>>
>>
>>
>>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev

I think I've detected some bugs in lmtp.c which cause the communication
to go out of sync.

When we are reading the input from the client (after the client has sent
"DATA" and dbmail-lmtp has sent "354 Start mail input") and detect an
error in the headers, we sent a "500 whatever error the client". However
, the client happily keeps on sending the complete message. As it
probably should.

I'll test if I can correct this by sending all further client input to a
blackhole, and returning a "500 error" after receiving the whole message.

Ilja


Ilja Booij wrote:

> I wonder what message dbmail-smtp should send to the MTA when the header
> is to big: EX_DATAERR?
>
> EX_TEMPFAIL does not seem like the right messages, but I wonder if
> EX_DATAERR would be understood by the MTA.
>
> Any ideas?
>
> Ilja
>
>
> Ilja Booij wrote:
>
>> for now I'll go for bouncing messages with ueberlong headers. Later
>> on, we'll have to decide on a nice strategy to work with longer headers.
>>
>> Ilja
>>
>> Aaron Stone wrote:
>>
>>> For the uber-large headers problem, would you rather bounce the
>>> message or add
>>> the is_header column to the messageblks table and allow for multiple
>>> header
>>> blocks? Also, since we currently pass the header around as a single char
>>> array, how would you propose to throttle the memory usage so that
>>> someone
>>> cannot DoS the server with garbage jigabyte headers?
>>>
>>> Aaron
>>>
>>>
>>> Ilja Booij <ilja@ic-s.nl> said:
>>>
>>>
>>>> Hi,
>>>>
>>>> I'd like to release rc4 tomorrow. Is anybody aware of any
>>>> showstoppers we have to fix before releasing rc4 other than the ones
>>>> I'll mention below?
>>>>
>>>> * Headers > 512kB don't work. Messages get dropped.
>>>>
>>>> * code indentation: run indent -i8 -kr (unless anybody wants
>>>> something completely different :) )
>>>>
>>>> Ilja
>>>>
>>>>
>>>> _______________________________________________
>>>> Dbmail-dev mailing list
>>>> Dbmail-dev@dbmail.org
>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>
>>>
>>>
>>>
>>>
>> _______________________________________________
>> Dbmail-dev mailing list
>> Dbmail-dev@dbmail.org
>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev

Ilja Booij wrote:
>
> I'll test if I can correct this by sending all further client input to a
> blackhole, and returning a "500 error" after receiving the whole message.

Tried it, and it works!
I'll check it with dbmail-smtp as well, and add it to all other error
messages regarding headers.

Ilja

>
> Ilja
>
>
> Ilja Booij wrote:
>
>> I wonder what message dbmail-smtp should send to the MTA when the
>> header is to big: EX_DATAERR?
>>
>> EX_TEMPFAIL does not seem like the right messages, but I wonder if
>> EX_DATAERR would be understood by the MTA.
>>
>> Any ideas?
>>
>> Ilja
>>
>>
>> Ilja Booij wrote:
>>
>>> for now I'll go for bouncing messages with ueberlong headers. Later
>>> on, we'll have to decide on a nice strategy to work with longer headers.
>>>
>>> Ilja
>>>
>>> Aaron Stone wrote:
>>>
>>>> For the uber-large headers problem, would you rather bounce the
>>>> message or add
>>>> the is_header column to the messageblks table and allow for multiple
>>>> header
>>>> blocks? Also, since we currently pass the header around as a single
>>>> char
>>>> array, how would you propose to throttle the memory usage so that
>>>> someone
>>>> cannot DoS the server with garbage jigabyte headers?
>>>>
>>>> Aaron
>>>>
>>>>
>>>> Ilja Booij <ilja@ic-s.nl> said:
>>>>
>>>>
>>>>> Hi,
>>>>>
>>>>> I'd like to release rc4 tomorrow. Is anybody aware of any
>>>>> showstoppers we have to fix before releasing rc4 other than the
>>>>> ones I'll mention below?
>>>>>
>>>>> * Headers > 512kB don't work. Messages get dropped.
>>>>>
>>>>> * code indentation: run indent -i8 -kr (unless anybody wants
>>>>> something completely different :) )
>>>>>
>>>>> Ilja
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Dbmail-dev mailing list
>>>>> Dbmail-dev@dbmail.org
>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>
>>>>
>>>>
>>>>
>>>>
>>> _______________________________________________
>>> Dbmail-dev mailing list
>>> Dbmail-dev@dbmail.org
>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>
>>
>> _______________________________________________
>> Dbmail-dev mailing list
>> Dbmail-dev@dbmail.org
>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev

Ilja Booij wrote:

> Ilja Booij wrote:
>
>>
>> I'll test if I can correct this by sending all further client input to
>> a blackhole, and returning a "500 error" after receiving the whole
>> message.
>
>
> Tried it, and it works!
> I'll check it with dbmail-smtp as well, and add it to all other error
> messages regarding headers.

All is well I guess:
dbmail-lmtp and dbmail-smtp both return an error to the MTA, which then
bounces the message. For lack of a better alternative, dbmail-smtp
returns EX_DATAERR to signal an error in the data to the MTA.

The maximum header size that is accepted is READ_BLOCK_SIZE.

Ilja