Mailing List Archive: Proposing a dedicated dbmail user account

Proposing a dedicated dbmail user account

Apr 23, 2003, 11:01 AM

Post #1 of 5 (1163 views)

I've been working on the recoding of the delivery pipeline, and the
particular piece of code I'm focusing on right now is in insert_messages()
in pipe.c, and is responsible for delivery to forwarding addresses. As I
described in an earlier post, the algorithm is that if the message was
delivered to any user in the database, then this message will be read out
of the database for the forwards. If the message was not delivered to the
database, then the message goes directly from the incoming stream to the
sendmail processes.

While it seems like a sensible shortcut to skip the database if not
needed, the result is that every forwarding address needs to be processes
at once. So for 100 forwards, that's 100 instances of sendmail at once!

I think that adding db_* functions to work with a temporary table would be
a good option to use, but then considered that a simpler option, which
would require no additional code, would be adding a dedicated dbmail user
account to the database. For example, reserve useridnr 1 or 0 (although
I'm not sure that an auto_increment column can even hold the number 0...)

With this requirement in place, all incoming mail can be immediately
delivered to the database for the dbmail user, further processing can
occur (for example, body matching filters) and then the message can be
copied to each real user using db_copymsg() and sent to the forwards using
the current mechanism. Look ma, no extra tables needed!

Transition issues are of course the biggest concern, since before this
code could run, somebody's useridnr would have to be renumbered...

Thoughts, comment?

Aaron

RE: Proposing a dedicated dbmail user account [ In reply to ]

jesse at kci

Apr 23, 2003, 11:49 AM

Post #2 of 5 (1145 views)

Permalink

> While it seems like a sensible shortcut to skip the database if not
> needed, the result is that every forwarding address needs to be processes
> at once. So for 100 forwards, that's 100 instances of sendmail at once!

Offhand, can dbmail simply specify multiple recipients for a message
(ie. one sendmail process, 100 recipients)? It'd of course require
some max limit to be set, and wouldn't address the actual issue here,
but overall seems like it'd make more sense, wouldn't it?

> I think that adding db_* functions to work with a temporary table would be
> a good option to use, but then considered that a simpler option, which
> would require no additional code, would be adding a dedicated dbmail user
> account to the database. For example, reserve useridnr 1 or 0 (although

You wouldn't even need a user account, that I can see. All you
need for using messages/messageblks tables is to point to a mailbox_idnr, for which you can use value 0 (unless someone has
added foreign key constraints from messages.mailbox_idnr to
mailboxes table, which is not how dbmail currently "ships").

> I'm not sure that an auto_increment column can even hold the number 0...)

FWIW, I know you can in postgres (eg. we reserve an id of 0 for some
special things, which has to be manually inserted, and everything else
gets an auto increment value).

--
Jesse Norell
jesse (at) kci.net

RE: Proposing a dedicated dbmail user account [ In reply to ]

aaron at engr

Apr 23, 2003, 2:08 PM

Post #3 of 5 (1128 views)

Permalink

On Wed, 23 Apr 2003, Jesse Norell wrote:

>
> > While it seems like a sensible shortcut to skip the database if not
> > needed, the result is that every forwarding address needs to be processes
> > at once. So for 100 forwards, that's 100 instances of sendmail at once!
>
> Offhand, can dbmail simply specify multiple recipients for a message
> (ie. one sendmail process, 100 recipients)? It'd of course require
> some max limit to be set, and wouldn't address the actual issue here,
> but overall seems like it'd make more sense, wouldn't it?
>

I don't think that you can pass multiple destinations... but if you can,
it will cut down only on code needed. A reflow the delivery paths is still
needed to support body filtering and pipes to non-sendmail programs.

>
> > I think that adding db_* functions to work with a temporary table would be
> > a good option to use, but then considered that a simpler option, which
> > would require no additional code, would be adding a dedicated dbmail user
> > account to the database. For example, reserve useridnr 1 or 0 (although
>
> You wouldn't even need a user account, that I can see. All you
> need for using messages/messageblks tables is to point to a mailbox_idnr, for which you can use value 0 (unless someone has
> added foreign key constraints from messages.mailbox_idnr to
> mailboxes table, which is not how dbmail currently "ships").
>

Foreign key constraints could be a worry, although they would also break
non-database authenticators such as LDAP. Currently, db_insert_message()
expects a useridnr and a (char *) mailbox name. This precludes delivery
directly to a mailbox by id number :-\

>
> > I'm not sure that an auto_increment column can even hold the number 0...)
>
> FWIW, I know you can in postgres (eg. we reserve an id of 0 for some
> special things, which has to be manually inserted, and everything else
> gets an auto increment value).
>

Sounds good, I'll write the code with useridnr 0, as both MySQL and
PostgreSQL will have started their auto counters at 1. If the code is
accepted, though, I would want to have an INSERT statement in the
create_tables.sql scripts so that 0 is officially reserved.

>
>
>
> --
> Jesse Norell
> jesse (at) kci.net
>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>

Re: Proposing a dedicated dbmail user account [ In reply to ]

Magnus.Sundberg at dican

Apr 24, 2003, 12:47 AM

Post #4 of 5 (1137 views)

Permalink

Aaron Stone wrote:
> On Wed, 23 Apr 2003, Jesse Norell wrote:
>
>
>>>While it seems like a sensible shortcut to skip the database if not
>>>needed, the result is that every forwarding address needs to be processes
>>>at once. So for 100 forwards, that's 100 instances of sendmail at once!
>>
>> Offhand, can dbmail simply specify multiple recipients for a message
>>(ie. one sendmail process, 100 recipients)? It'd of course require
>>some max limit to be set, and wouldn't address the actual issue here,
>>but overall seems like it'd make more sense, wouldn't it?
>>
Forwarding to 100 mail adresses at the same time requires 100 sendmail
processes?
How is this implemented in Mailman, the list server running this
mailinglist?
With an ordinary unix mail installation, you specify a list to forward
your mail to, for instance userlist@foo.bar in your /etc/aliases file

Why not forward one mail to sendmail to userlist@foo.bar and then let
sendmail do the list expansion, by looking up the alias from the
database instead of /etc/aliases. I beleive this is one of the tasks
sendmail is written for.

>>--
>>Jesse Norell
>>jesse (at) kci.net
>>
>>_______________________________________________
>>Dbmail-dev mailing list
>>Dbmail-dev@dbmail.org
>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>

/Magnus

Re: Proposing a dedicated dbmail user account [ In reply to ]

aaron at engr

Apr 24, 2003, 2:24 AM

Post #5 of 5 (1135 views)

Permalink

There are two reasons for this:

The first is that "sendmail" isn't necessarily Sendmail. It could be any
number of work-alikes, such as Postfix :-) Or it could be a shell script
or your own creation or just about anything that's chmod +x. That said,
while this program *should* be able to accept multiple recipients on the
command line as Sendmail does, it may not.

Note that your aliases table could always have an entry like this:

alias forward
user@domain user@domain2 name@domain3 jim@someplace etc@more.many.and

And in that case, DBMail would simply call your sendmail with that whole
thing as the argument, and it had better support multiple recipients!

The second is that right now, the loop looks like this:

while( we have more recipients )
{
Figure out how this recipient wants to be forwarded
{
Simple piped
Piped with an mbox header
Via sendmail to some external address
}
Execute the program
Put a pointer to its stdin in a pipe list
}

while( we have more data )
{
while( we have more pipes )
{
Send the current chunk on its way
}
}

This is optimized for small sets of forwards, where you want to get
everthing going as quickly as possible with the least internal overhead.
Find yourself with a large number of forwards and you're toast, fast.
The hassle of first storing the message to the database, then reading it
back out to the forwarding addresses a few at a time will be expensive for
these small sets, but should scale way up without killing your server...

Aaron

On Thu, 24 Apr 2003, Magnus Sundberg wrote:

> Aaron Stone wrote:
> > On Wed, 23 Apr 2003, Jesse Norell wrote:
> >
> >
> >>>While it seems like a sensible shortcut to skip the database if not
> >>>needed, the result is that every forwarding address needs to be processes
> >>>at once. So for 100 forwards, that's 100 instances of sendmail at once!
> >>
> >> Offhand, can dbmail simply specify multiple recipients for a message
> >>(ie. one sendmail process, 100 recipients)? It'd of course require
> >>some max limit to be set, and wouldn't address the actual issue here,
> >>but overall seems like it'd make more sense, wouldn't it?
> >>
> Forwarding to 100 mail adresses at the same time requires 100 sendmail
> processes?
> How is this implemented in Mailman, the list server running this
> mailinglist?
> With an ordinary unix mail installation, you specify a list to forward
> your mail to, for instance userlist@foo.bar in your /etc/aliases file
>
> Why not forward one mail to sendmail to userlist@foo.bar and then let
> sendmail do the list expansion, by looking up the alias from the
> database instead of /etc/aliases. I beleive this is one of the tasks
> sendmail is written for.
>
>
> >>--
> >>Jesse Norell
> >>jesse (at) kci.net
> >>
> >>_______________________________________________
> >>Dbmail-dev mailing list
> >>Dbmail-dev@dbmail.org
> >>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>
>
> /Magnus
>
>
>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>