Mailing List Archive

Hi,

Aaron Stone wrote:

> The reason for the double deliveries in the LMTP daemon is because the old
> insert_messages() function used to take lists of users to directly deliver.
> The new insert_messages() function takes a list of dsnuser structures, and
> expects that *either* the useridnr field is filled (for direct-to-useridnr
> delivery, which isn't supported by any of the frontends, but is supported in
> the lower layers ;-) or the address field, which is first checked as a
> username and then as an alias (this allows usernames in the form of
> 'user@server' to operate without requiring an alias that says 'user@server
> delivers to user@server').
>
> Some refactoring might be necessary, because we need to inform the MTA whether
> or not its envelope recipients were valid before it will send us the message.
> This means users need to be validated before read_header(), which is a problem
> because at the moment this vaildation happens in insert_messages().
OK, that's clear
>
> Fine and dandy in pipe land, where the MTA will send the message regardless of
> the disposition of the recipients, and only at the very end are error
> conditions returned as exit codes... in LMTP land we need to know ahead of time.
>
> I think what I'll do is move the user validation code from insert_messages()
> into dsn.c, perhaps calling it dsn_check_users() or dsn_prepare_deliveries().
> Then, we'll have this order:
>
> For command-line and envelope-recipient deliveries:
> - receive addresses and store them into the dsnusers list.
> - dsn_prepare_deliveries()
> - if no deliveries are valid, return an error.
>
> - read_header()
>
> For deliver-to header deliveries:
> - mime_readheader()
> - mail_adr_list()
> - dsn_prepare_deliveries()
> - if no deliveries are valid, return an error.
>
> - insert_messages()

I like the part where you say 'What I'll do' :)
Do you think this work will take long? I think we must really release
rc3 tomorrow (Friday, March 5th). This stuff really has to work if we
want to release.

Ilja

Working on it as we speak! Catch you in about an hour?

Aaron


Ilja Booij <ilja@ic-s.nl> said:

> Hi,
>
> Aaron Stone wrote:
>
> > The reason for the double deliveries in the LMTP daemon is because the old
> > insert_messages() function used to take lists of users to directly deliver.
> > The new insert_messages() function takes a list of dsnuser structures, and
> > expects that *either* the useridnr field is filled (for direct-to-useridnr
> > delivery, which isn't supported by any of the frontends, but is supported in
> > the lower layers ;-) or the address field, which is first checked as a
> > username and then as an alias (this allows usernames in the form of
> > 'user@server' to operate without requiring an alias that says 'user@server
> > delivers to user@server').
> >
> > Some refactoring might be necessary, because we need to inform the MTA whether
> > or not its envelope recipients were valid before it will send us the message.
> > This means users need to be validated before read_header(), which is a problem
> > because at the moment this vaildation happens in insert_messages().
> OK, that's clear
> >
> > Fine and dandy in pipe land, where the MTA will send the message regardless of
> > the disposition of the recipients, and only at the very end are error
> > conditions returned as exit codes... in LMTP land we need to know ahead of
time.
> >
> > I think what I'll do is move the user validation code from insert_messages()
> > into dsn.c, perhaps calling it dsn_check_users() or dsn_prepare_deliveries().
> > Then, we'll have this order:
> >
> > For command-line and envelope-recipient deliveries:
> > - receive addresses and store them into the dsnusers list.
> > - dsn_prepare_deliveries()
> > - if no deliveries are valid, return an error.
> >
> > - read_header()
> >
> > For deliver-to header deliveries:
> > - mime_readheader()
> > - mail_adr_list()
> > - dsn_prepare_deliveries()
> > - if no deliveries are valid, return an error.
> >
> > - insert_messages()
>
> I like the part where you say 'What I'll do' :)
> Do you think this work will take long? I think we must really release
> rc3 tomorrow (Friday, March 5th). This stuff really has to work if we
> want to release.
>
> Ilja
>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>



--

sounds ok to me :)

Ilja

Aaron Stone wrote:

> Working on it as we speak! Catch you in about an hour?
>
> Aaron
>
>
> Ilja Booij <ilja@ic-s.nl> said:
>
>
>>Hi,
>>
>>Aaron Stone wrote:
>>
>>
>>>The reason for the double deliveries in the LMTP daemon is because the old
>>>insert_messages() function used to take lists of users to directly deliver.
>>>The new insert_messages() function takes a list of dsnuser structures, and
>>>expects that *either* the useridnr field is filled (for direct-to-useridnr
>>>delivery, which isn't supported by any of the frontends, but is supported in
>>>the lower layers ;-) or the address field, which is first checked as a
>>>username and then as an alias (this allows usernames in the form of
>>>'user@server' to operate without requiring an alias that says 'user@server
>>>delivers to user@server').
>>>
>>>Some refactoring might be necessary, because we need to inform the MTA whether
>>>or not its envelope recipients were valid before it will send us the message.
>>>This means users need to be validated before read_header(), which is a problem
>>>because at the moment this vaildation happens in insert_messages().
>>
>>OK, that's clear
>>
>>>Fine and dandy in pipe land, where the MTA will send the message regardless of
>>>the disposition of the recipients, and only at the very end are error
>>>conditions returned as exit codes... in LMTP land we need to know ahead of
>
> time.
>
>>>I think what I'll do is move the user validation code from insert_messages()
>>>into dsn.c, perhaps calling it dsn_check_users() or dsn_prepare_deliveries().
>>>Then, we'll have this order:
>>>
>>>For command-line and envelope-recipient deliveries:
>>>- receive addresses and store them into the dsnusers list.
>>>- dsn_prepare_deliveries()
>>>- if no deliveries are valid, return an error.
>>>
>>>- read_header()
>>>
>>>For deliver-to header deliveries:
>>>- mime_readheader()
>>>- mail_adr_list()
>>>- dsn_prepare_deliveries()
>>>- if no deliveries are valid, return an error.
>>>
>>>- insert_messages()
>>
>>I like the part where you say 'What I'll do' :)
>>Do you think this work will take long? I think we must really release
>>rc3 tomorrow (Friday, March 5th). This stuff really has to work if we
>>want to release.
>>
>>Ilja
>>
>>_______________________________________________
>>Dbmail-dev mailing list
>>Dbmail-dev@dbmail.org
>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>
>
>
>
>

New versions checked in, though not extensively tested yet.

Ilja Booij <ilja@ic-s.nl> said:

> sounds ok to me :)
>
> Ilja
>
> Aaron Stone wrote:
>
> > Working on it as we speak! Catch you in about an hour?
> >
> > Aaron
> >
> >
> > Ilja Booij <ilja@ic-s.nl> said:
> >
> >
> >>Hi,
> >>
> >>Aaron Stone wrote:
> >>
> >>
> >>>The reason for the double deliveries in the LMTP daemon is because the old
> >>>insert_messages() function used to take lists of users to directly deliver.
> >>>The new insert_messages() function takes a list of dsnuser structures, and
> >>>expects that *either* the useridnr field is filled (for direct-to-useridnr
> >>>delivery, which isn't supported by any of the frontends, but is supported in
> >>>the lower layers ;-) or the address field, which is first checked as a
> >>>username and then as an alias (this allows usernames in the form of
> >>>'user@server' to operate without requiring an alias that says 'user@server
> >>>delivers to user@server').
> >>>
> >>>Some refactoring might be necessary, because we need to inform the MTA
whether
> >>>or not its envelope recipients were valid before it will send us the message.
> >>>This means users need to be validated before read_header(), which is a
problem
> >>>because at the moment this vaildation happens in insert_messages().
> >>
> >>OK, that's clear
> >>
> >>>Fine and dandy in pipe land, where the MTA will send the message
regardless of
> >>>the disposition of the recipients, and only at the very end are error
> >>>conditions returned as exit codes... in LMTP land we need to know ahead of
> >
> > time.
> >
> >>>I think what I'll do is move the user validation code from insert_messages()
> >>>into dsn.c, perhaps calling it dsn_check_users() or dsn_prepare_deliveries().
> >>>Then, we'll have this order:
> >>>
> >>>For command-line and envelope-recipient deliveries:
> >>>- receive addresses and store them into the dsnusers list.
> >>>- dsn_prepare_deliveries()
> >>>- if no deliveries are valid, return an error.
> >>>
> >>>- read_header()
> >>>
> >>>For deliver-to header deliveries:
> >>>- mime_readheader()
> >>>- mail_adr_list()
> >>>- dsn_prepare_deliveries()
> >>>- if no deliveries are valid, return an error.
> >>>
> >>>- insert_messages()
> >>
> >>I like the part where you say 'What I'll do' :)
> >>Do you think this work will take long? I think we must really release
> >>rc3 tomorrow (Friday, March 5th). This stuff really has to work if we
> >>want to release.
> >>
> >>Ilja
> >>
> >>_______________________________________________
> >>Dbmail-dev mailing list
> >>Dbmail-dev@dbmail.org
> >>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>
> >
> >
> >
> >
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>



--

OK,

this seems to have tackled the problem of the double delivery :)

There still is another problem though:
It still seems to hang for a while on every second delivery attempt to
the LMTP daemon.

Can you try the following scenario:

* configure MTA to use dbmail-lmtp for delivery
* send message to a recipient
* verify that the message is received using dbmail-lmtp
* send a second message.

What I observe here is that the second message takes a lot longer to be
delivered than the first one. The second one is only delivered after
minutes.

I have the feeling that something is not right here..

Ilja

Aaron Stone wrote:

> New versions checked in, though not extensively tested yet.
>
> Ilja Booij <ilja@ic-s.nl> said:
>
>
>>sounds ok to me :)
>>
>>Ilja
>>
>>Aaron Stone wrote:
>>
>>
>>>Working on it as we speak! Catch you in about an hour?
>>>
>>>Aaron
>>>
>>>
>>>Ilja Booij <ilja@ic-s.nl> said:
>>>
>>>
>>>
>>>>Hi,
>>>>
>>>>Aaron Stone wrote:
>>>>
>>>>
>>>>
>>>>>The reason for the double deliveries in the LMTP daemon is because the old
>>>>>insert_messages() function used to take lists of users to directly deliver.
>>>>>The new insert_messages() function takes a list of dsnuser structures, and
>>>>>expects that *either* the useridnr field is filled (for direct-to-useridnr
>>>>>delivery, which isn't supported by any of the frontends, but is supported in
>>>>>the lower layers ;-) or the address field, which is first checked as a
>>>>>username and then as an alias (this allows usernames in the form of
>>>>>'user@server' to operate without requiring an alias that says 'user@server
>>>>>delivers to user@server').
>>>>>
>>>>>Some refactoring might be necessary, because we need to inform the MTA
>
> whether
>
>>>>>or not its envelope recipients were valid before it will send us the message.
>>>>>This means users need to be validated before read_header(), which is a
>
> problem
>
>>>>>because at the moment this vaildation happens in insert_messages().
>>>>
>>>>OK, that's clear
>>>>
>>>>
>>>>>Fine and dandy in pipe land, where the MTA will send the message
>
> regardless of
>
>>>>>the disposition of the recipients, and only at the very end are error
>>>>>conditions returned as exit codes... in LMTP land we need to know ahead of
>>>
>>>time.
>>>
>>>
>>>>>I think what I'll do is move the user validation code from insert_messages()
>>>>>into dsn.c, perhaps calling it dsn_check_users() or dsn_prepare_deliveries().
>>>>>Then, we'll have this order:
>>>>>
>>>>>For command-line and envelope-recipient deliveries:
>>>>>- receive addresses and store them into the dsnusers list.
>>>>>- dsn_prepare_deliveries()
>>>>>- if no deliveries are valid, return an error.
>>>>>
>>>>>- read_header()
>>>>>
>>>>>For deliver-to header deliveries:
>>>>>- mime_readheader()
>>>>>- mail_adr_list()
>>>>>- dsn_prepare_deliveries()
>>>>>- if no deliveries are valid, return an error.
>>>>>
>>>>>- insert_messages()
>>>>
>>>>I like the part where you say 'What I'll do' :)
>>>>Do you think this work will take long? I think we must really release
>>>>rc3 tomorrow (Friday, March 5th). This stuff really has to work if we
>>>>want to release.
>>>>
>>>>Ilja
>>>>
>>>>_______________________________________________
>>>>Dbmail-dev mailing list
>>>>Dbmail-dev@dbmail.org
>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>
>>>
>>>
>>>
>>>
>>_______________________________________________
>>Dbmail-dev mailing list
>>Dbmail-dev@dbmail.org
>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>
>
>
>
>

OK, I've found something:

It seems that Postfix sends a RSET command, when we expect to get the
header.

readheader() then waits for postfix to send more, while postfix waits
for a '250 OK' Message from dbmail-lmtp.

So, there's probably something wrong in our LMTP-logic.
I'll take a look at it, after I've done some small scripting job here
for a customer.

Ilja

Ilja Booij wrote:

> OK,
>
> this seems to have tackled the problem of the double delivery :)
>
> There still is another problem though:
> It still seems to hang for a while on every second delivery attempt to
> the LMTP daemon.
>
> Can you try the following scenario:
>
> * configure MTA to use dbmail-lmtp for delivery
> * send message to a recipient
> * verify that the message is received using dbmail-lmtp
> * send a second message.
>
> What I observe here is that the second message takes a lot longer to be
> delivered than the first one. The second one is only delivered after
> minutes.
>
> I have the feeling that something is not right here..
>
> Ilja
>
> Aaron Stone wrote:
>
>> New versions checked in, though not extensively tested yet.
>>
>> Ilja Booij <ilja@ic-s.nl> said:
>>
>>
>>> sounds ok to me :)
>>>
>>> Ilja
>>>
>>> Aaron Stone wrote:
>>>
>>>
>>>> Working on it as we speak! Catch you in about an hour?
>>>>
>>>> Aaron
>>>>
>>>>
>>>> Ilja Booij <ilja@ic-s.nl> said:
>>>>
>>>>
>>>>
>>>>> Hi,
>>>>>
>>>>> Aaron Stone wrote:
>>>>>
>>>>>
>>>>>
>>>>>> The reason for the double deliveries in the LMTP daemon is because
>>>>>> the old
>>>>>> insert_messages() function used to take lists of users to directly
>>>>>> deliver.
>>>>>> The new insert_messages() function takes a list of dsnuser
>>>>>> structures, and
>>>>>> expects that *either* the useridnr field is filled (for
>>>>>> direct-to-useridnr
>>>>>> delivery, which isn't supported by any of the frontends, but is
>>>>>> supported in
>>>>>> the lower layers ;-) or the address field, which is first checked
>>>>>> as a
>>>>>> username and then as an alias (this allows usernames in the form of
>>>>>> 'user@server' to operate without requiring an alias that says
>>>>>> 'user@server
>>>>>> delivers to user@server').
>>>>>>
>>>>>> Some refactoring might be necessary, because we need to inform the
>>>>>> MTA
>>
>>
>> whether
>>
>>>>>> or not its envelope recipients were valid before it will send us
>>>>>> the message.
>>>>>> This means users need to be validated before read_header(), which
>>>>>> is a
>>
>>
>> problem
>>
>>>>>> because at the moment this vaildation happens in insert_messages().
>>>>>
>>>>>
>>>>> OK, that's clear
>>>>>
>>>>>
>>>>>> Fine and dandy in pipe land, where the MTA will send the message
>>
>>
>> regardless of
>>
>>>>>> the disposition of the recipients, and only at the very end are error
>>>>>> conditions returned as exit codes... in LMTP land we need to know
>>>>>> ahead of
>>>>
>>>>
>>>> time.
>>>>
>>>>
>>>>>> I think what I'll do is move the user validation code from
>>>>>> insert_messages()
>>>>>> into dsn.c, perhaps calling it dsn_check_users() or
>>>>>> dsn_prepare_deliveries().
>>>>>> Then, we'll have this order:
>>>>>>
>>>>>> For command-line and envelope-recipient deliveries:
>>>>>> - receive addresses and store them into the dsnusers list.
>>>>>> - dsn_prepare_deliveries()
>>>>>> - if no deliveries are valid, return an error.
>>>>>>
>>>>>> - read_header()
>>>>>>
>>>>>> For deliver-to header deliveries:
>>>>>> - mime_readheader()
>>>>>> - mail_adr_list()
>>>>>> - dsn_prepare_deliveries()
>>>>>> - if no deliveries are valid, return an error.
>>>>>>
>>>>>> - insert_messages()
>>>>>
>>>>>
>>>>> I like the part where you say 'What I'll do' :)
>>>>> Do you think this work will take long? I think we must really
>>>>> release rc3 tomorrow (Friday, March 5th). This stuff really has to
>>>>> work if we want to release.
>>>>>
>>>>> Ilja
>>>>>
>>>>> _______________________________________________
>>>>> Dbmail-dev mailing list
>>>>> Dbmail-dev@dbmail.org
>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>
>>>>
>>>>
>>>>
>>>>
>>> _______________________________________________
>>> Dbmail-dev mailing list
>>> Dbmail-dev@dbmail.org
>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>
>>
>>
>>
>>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev

Hi,

I haven't been able to find the cause of the problem yet. I found some
more info in the logs though:

Mar 4 16:55:34 test01 postfix/lmtp[21340]: 3378119F30A:
to=<target@test01.office.fastxs.net>,
relay=localhost.fastxs.net[127.0.0.1], delay=0, status=bounced (host
localhost.fastxs.net[127.0.0.1] said: 250 Recipient
<target@test01.office.fastxs.net> OK)

apparently, postfix thinks we want to bounce the message. But why? A
previous message to the same user arrives correctly..

strange, looking further. Aaron, can you also take a look at this?

Ilja

Ilja Booij wrote:

> OK, I've found something:
>
> It seems that Postfix sends a RSET command, when we expect to get the
> header.
>
> readheader() then waits for postfix to send more, while postfix waits
> for a '250 OK' Message from dbmail-lmtp.
>
> So, there's probably something wrong in our LMTP-logic.
> I'll take a look at it, after I've done some small scripting job here
> for a customer.
>
> Ilja
>
> Ilja Booij wrote:
>
>> OK,
>>
>> this seems to have tackled the problem of the double delivery :)
>>
>> There still is another problem though:
>> It still seems to hang for a while on every second delivery attempt to
>> the LMTP daemon.
>>
>> Can you try the following scenario:
>>
>> * configure MTA to use dbmail-lmtp for delivery
>> * send message to a recipient
>> * verify that the message is received using dbmail-lmtp
>> * send a second message.
>>
>> What I observe here is that the second message takes a lot longer to
>> be delivered than the first one. The second one is only delivered
>> after minutes.
>>
>> I have the feeling that something is not right here..
>>
>> Ilja
>>
>> Aaron Stone wrote:
>>
>>> New versions checked in, though not extensively tested yet.
>>>
>>> Ilja Booij <ilja@ic-s.nl> said:
>>>
>>>
>>>> sounds ok to me :)
>>>>
>>>> Ilja
>>>>
>>>> Aaron Stone wrote:
>>>>
>>>>
>>>>> Working on it as we speak! Catch you in about an hour?
>>>>>
>>>>> Aaron
>>>>>
>>>>>
>>>>> Ilja Booij <ilja@ic-s.nl> said:
>>>>>
>>>>>
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Aaron Stone wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> The reason for the double deliveries in the LMTP daemon is
>>>>>>> because the old
>>>>>>> insert_messages() function used to take lists of users to
>>>>>>> directly deliver.
>>>>>>> The new insert_messages() function takes a list of dsnuser
>>>>>>> structures, and
>>>>>>> expects that *either* the useridnr field is filled (for
>>>>>>> direct-to-useridnr
>>>>>>> delivery, which isn't supported by any of the frontends, but is
>>>>>>> supported in
>>>>>>> the lower layers ;-) or the address field, which is first checked
>>>>>>> as a
>>>>>>> username and then as an alias (this allows usernames in the form of
>>>>>>> 'user@server' to operate without requiring an alias that says
>>>>>>> 'user@server
>>>>>>> delivers to user@server').
>>>>>>>
>>>>>>> Some refactoring might be necessary, because we need to inform
>>>>>>> the MTA
>>>
>>>
>>>
>>> whether
>>>
>>>>>>> or not its envelope recipients were valid before it will send us
>>>>>>> the message.
>>>>>>> This means users need to be validated before read_header(), which
>>>>>>> is a
>>>
>>>
>>>
>>> problem
>>>
>>>>>>> because at the moment this vaildation happens in insert_messages().
>>>>>>
>>>>>>
>>>>>>
>>>>>> OK, that's clear
>>>>>>
>>>>>>
>>>>>>> Fine and dandy in pipe land, where the MTA will send the message
>>>
>>>
>>>
>>> regardless of
>>>
>>>>>>> the disposition of the recipients, and only at the very end are
>>>>>>> error
>>>>>>> conditions returned as exit codes... in LMTP land we need to know
>>>>>>> ahead of
>>>>>
>>>>>
>>>>>
>>>>> time.
>>>>>
>>>>>
>>>>>>> I think what I'll do is move the user validation code from
>>>>>>> insert_messages()
>>>>>>> into dsn.c, perhaps calling it dsn_check_users() or
>>>>>>> dsn_prepare_deliveries().
>>>>>>> Then, we'll have this order:
>>>>>>>
>>>>>>> For command-line and envelope-recipient deliveries:
>>>>>>> - receive addresses and store them into the dsnusers list.
>>>>>>> - dsn_prepare_deliveries()
>>>>>>> - if no deliveries are valid, return an error.
>>>>>>>
>>>>>>> - read_header()
>>>>>>>
>>>>>>> For deliver-to header deliveries:
>>>>>>> - mime_readheader()
>>>>>>> - mail_adr_list()
>>>>>>> - dsn_prepare_deliveries()
>>>>>>> - if no deliveries are valid, return an error.
>>>>>>>
>>>>>>> - insert_messages()
>>>>>>
>>>>>>
>>>>>>
>>>>>> I like the part where you say 'What I'll do' :)
>>>>>> Do you think this work will take long? I think we must really
>>>>>> release rc3 tomorrow (Friday, March 5th). This stuff really has to
>>>>>> work if we want to release.
>>>>>>
>>>>>> Ilja
>>>>>>
>>>>>> _______________________________________________
>>>>>> Dbmail-dev mailing list
>>>>>> Dbmail-dev@dbmail.org
>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>> _______________________________________________
>>>> Dbmail-dev mailing list
>>>> Dbmail-dev@dbmail.org
>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>
>>>
>>>
>>>
>>>
>> _______________________________________________
>> Dbmail-dev mailing list
>> Dbmail-dev@dbmail.org
>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev

Sure thing, I've got about an hour of time this morning. Looks like you're
burning the midnight oil over there! Yikes!

Aaron


Ilja Booij <ilja@ic-s.nl> said:

> Hi,
>
> I haven't been able to find the cause of the problem yet. I found some
> more info in the logs though:
>
> Mar 4 16:55:34 test01 postfix/lmtp[21340]: 3378119F30A:
> to=<target@test01.office.fastxs.net>,
> relay=localhost.fastxs.net[127.0.0.1], delay=0, status=bounced (host
> localhost.fastxs.net[127.0.0.1] said: 250 Recipient
> <target@test01.office.fastxs.net> OK)
>
> apparently, postfix thinks we want to bounce the message. But why? A
> previous message to the same user arrives correctly..
>
> strange, looking further. Aaron, can you also take a look at this?
>
> Ilja
>
> Ilja Booij wrote:
>
> > OK, I've found something:
> >
> > It seems that Postfix sends a RSET command, when we expect to get the
> > header.
> >
> > readheader() then waits for postfix to send more, while postfix waits
> > for a '250 OK' Message from dbmail-lmtp.
> >
> > So, there's probably something wrong in our LMTP-logic.
> > I'll take a look at it, after I've done some small scripting job here
> > for a customer.
> >
> > Ilja
> >
> > Ilja Booij wrote:
> >
> >> OK,
> >>
> >> this seems to have tackled the problem of the double delivery :)
> >>
> >> There still is another problem though:
> >> It still seems to hang for a while on every second delivery attempt to
> >> the LMTP daemon.
> >>
> >> Can you try the following scenario:
> >>
> >> * configure MTA to use dbmail-lmtp for delivery
> >> * send message to a recipient
> >> * verify that the message is received using dbmail-lmtp
> >> * send a second message.
> >>
> >> What I observe here is that the second message takes a lot longer to
> >> be delivered than the first one. The second one is only delivered
> >> after minutes.
> >>
> >> I have the feeling that something is not right here..
> >>
> >> Ilja
> >>
> >> Aaron Stone wrote:
> >>
> >>> New versions checked in, though not extensively tested yet.
> >>>
> >>> Ilja Booij <ilja@ic-s.nl> said:
> >>>
> >>>
> >>>> sounds ok to me :)
> >>>>
> >>>> Ilja
> >>>>
> >>>> Aaron Stone wrote:
> >>>>
> >>>>
> >>>>> Working on it as we speak! Catch you in about an hour?
> >>>>>
> >>>>> Aaron
> >>>>>
> >>>>>
> >>>>> Ilja Booij <ilja@ic-s.nl> said:
> >>>>>
> >>>>>
> >>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> Aaron Stone wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> The reason for the double deliveries in the LMTP daemon is
> >>>>>>> because the old
> >>>>>>> insert_messages() function used to take lists of users to
> >>>>>>> directly deliver.
> >>>>>>> The new insert_messages() function takes a list of dsnuser
> >>>>>>> structures, and
> >>>>>>> expects that *either* the useridnr field is filled (for
> >>>>>>> direct-to-useridnr
> >>>>>>> delivery, which isn't supported by any of the frontends, but is
> >>>>>>> supported in
> >>>>>>> the lower layers ;-) or the address field, which is first checked
> >>>>>>> as a
> >>>>>>> username and then as an alias (this allows usernames in the form of
> >>>>>>> 'user@server' to operate without requiring an alias that says
> >>>>>>> 'user@server
> >>>>>>> delivers to user@server').
> >>>>>>>
> >>>>>>> Some refactoring might be necessary, because we need to inform
> >>>>>>> the MTA
> >>>
> >>>
> >>>
> >>> whether
> >>>
> >>>>>>> or not its envelope recipients were valid before it will send us
> >>>>>>> the message.
> >>>>>>> This means users need to be validated before read_header(), which
> >>>>>>> is a
> >>>
> >>>
> >>>
> >>> problem
> >>>
> >>>>>>> because at the moment this vaildation happens in insert_messages().
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> OK, that's clear
> >>>>>>
> >>>>>>
> >>>>>>> Fine and dandy in pipe land, where the MTA will send the message
> >>>
> >>>
> >>>
> >>> regardless of
> >>>
> >>>>>>> the disposition of the recipients, and only at the very end are
> >>>>>>> error
> >>>>>>> conditions returned as exit codes... in LMTP land we need to know
> >>>>>>> ahead of
> >>>>>
> >>>>>
> >>>>>
> >>>>> time.
> >>>>>
> >>>>>
> >>>>>>> I think what I'll do is move the user validation code from
> >>>>>>> insert_messages()
> >>>>>>> into dsn.c, perhaps calling it dsn_check_users() or
> >>>>>>> dsn_prepare_deliveries().
> >>>>>>> Then, we'll have this order:
> >>>>>>>
> >>>>>>> For command-line and envelope-recipient deliveries:
> >>>>>>> - receive addresses and store them into the dsnusers list.
> >>>>>>> - dsn_prepare_deliveries()
> >>>>>>> - if no deliveries are valid, return an error.
> >>>>>>>
> >>>>>>> - read_header()
> >>>>>>>
> >>>>>>> For deliver-to header deliveries:
> >>>>>>> - mime_readheader()
> >>>>>>> - mail_adr_list()
> >>>>>>> - dsn_prepare_deliveries()
> >>>>>>> - if no deliveries are valid, return an error.
> >>>>>>>
> >>>>>>> - insert_messages()
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> I like the part where you say 'What I'll do' :)
> >>>>>> Do you think this work will take long? I think we must really
> >>>>>> release rc3 tomorrow (Friday, March 5th). This stuff really has to
> >>>>>> work if we want to release.
> >>>>>>
> >>>>>> Ilja
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> Dbmail-dev mailing list
> >>>>>> Dbmail-dev@dbmail.org
> >>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>> _______________________________________________
> >>>> Dbmail-dev mailing list
> >>>> Dbmail-dev@dbmail.org
> >>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>>>
> >>>
> >>>
> >>>
> >>>
> >> _______________________________________________
> >> Dbmail-dev mailing list
> >> Dbmail-dev@dbmail.org
> >> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >
> > _______________________________________________
> > Dbmail-dev mailing list
> > Dbmail-dev@dbmail.org
> > http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>



--

New to the list here. But I read over the archives regarding this
problem and I may have a little to add.

I was able to get postfix to consistently deliver via lmtp to dbmail
(even two consecutive messages to the same user) by setting the postfix
directive lmtp_cache_connection to false instead of true. This would
termintate the lmtp connection after each delivery. I assume, with
lmtp_cache_connection turned on, the connection would die after
lmtp_timeout was reached and a new connection would be established and
the mail would get sent. This may have been a "neat feature" that works
with Cyrus (postfix's preferred lmtp client) or it may be a
specification of the lmtp rfc (I have not read it). But it seems like a
simple enough fix to make it work. It would probably just involve
having the conversation status fall back to a state the postfix expects
after message delivery and waiting in that state until either the
connection terminates or the conversation picks up again. I took a
cursory glance at the lmtp.c and lmtpd.c code but did not have enough
time to really dig into it.

Good Luck. I am anxiously awaiting the 2.0 release. :)

Robert Theisen

Aaron Stone wrote:

>Sure thing, I've got about an hour of time this morning. Looks like you're
>burning the midnight oil over there! Yikes!
>
>Aaron
>
>
>Ilja Booij <ilja@ic-s.nl> said:
>
>
>
>>Hi,
>>
>>I haven't been able to find the cause of the problem yet. I found some
>>more info in the logs though:
>>
>>Mar 4 16:55:34 test01 postfix/lmtp[21340]: 3378119F30A:
>>to=<target@test01.office.fastxs.net>,
>>relay=localhost.fastxs.net[127.0.0.1], delay=0, status=bounced (host
>>localhost.fastxs.net[127.0.0.1] said: 250 Recipient
>><target@test01.office.fastxs.net> OK)
>>
>>apparently, postfix thinks we want to bounce the message. But why? A
>>previous message to the same user arrives correctly..
>>
>>strange, looking further. Aaron, can you also take a look at this?
>>
>>Ilja
>>
>>Ilja Booij wrote:
>>
>>
>>
>>>OK, I've found something:
>>>
>>>It seems that Postfix sends a RSET command, when we expect to get the
>>>header.
>>>
>>>readheader() then waits for postfix to send more, while postfix waits
>>>for a '250 OK' Message from dbmail-lmtp.
>>>
>>>So, there's probably something wrong in our LMTP-logic.
>>>I'll take a look at it, after I've done some small scripting job here
>>>for a customer.
>>>
>>>Ilja
>>>
>>>Ilja Booij wrote:
>>>
>>>
>>>
>>>>OK,
>>>>
>>>>this seems to have tackled the problem of the double delivery :)
>>>>
>>>>There still is another problem though:
>>>>It still seems to hang for a while on every second delivery attempt to
>>>>the LMTP daemon.
>>>>
>>>>Can you try the following scenario:
>>>>
>>>>* configure MTA to use dbmail-lmtp for delivery
>>>>* send message to a recipient
>>>>* verify that the message is received using dbmail-lmtp
>>>>* send a second message.
>>>>
>>>>What I observe here is that the second message takes a lot longer to
>>>>be delivered than the first one. The second one is only delivered
>>>>after minutes.
>>>>
>>>>I have the feeling that something is not right here..
>>>>
>>>>Ilja
>>>>
>>>>Aaron Stone wrote:
>>>>
>>>>
>>>>
>>>>>New versions checked in, though not extensively tested yet.
>>>>>
>>>>>Ilja Booij <ilja@ic-s.nl> said:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>sounds ok to me :)
>>>>>>
>>>>>>Ilja
>>>>>>
>>>>>>Aaron Stone wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>Working on it as we speak! Catch you in about an hour?
>>>>>>>
>>>>>>>Aaron
>>>>>>>
>>>>>>>
>>>>>>>Ilja Booij <ilja@ic-s.nl> said:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>Hi,
>>>>>>>>
>>>>>>>>Aaron Stone wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>The reason for the double deliveries in the LMTP daemon is
>>>>>>>>>because the old
>>>>>>>>>insert_messages() function used to take lists of users to
>>>>>>>>>directly deliver.
>>>>>>>>>The new insert_messages() function takes a list of dsnuser
>>>>>>>>>structures, and
>>>>>>>>>expects that *either* the useridnr field is filled (for
>>>>>>>>>direct-to-useridnr
>>>>>>>>>delivery, which isn't supported by any of the frontends, but is
>>>>>>>>>supported in
>>>>>>>>>the lower layers ;-) or the address field, which is first checked
>>>>>>>>>as a
>>>>>>>>>username and then as an alias (this allows usernames in the form of
>>>>>>>>>'user@server' to operate without requiring an alias that says
>>>>>>>>>'user@server
>>>>>>>>>delivers to user@server').
>>>>>>>>>
>>>>>>>>>Some refactoring might be necessary, because we need to inform
>>>>>>>>>the MTA
>>>>>>>>>
>>>>>>>>>
>>>>>
>>>>>whether
>>>>>
>>>>>
>>>>>
>>>>>>>>>or not its envelope recipients were valid before it will send us
>>>>>>>>>the message.
>>>>>>>>>This means users need to be validated before read_header(), which
>>>>>>>>>is a
>>>>>>>>>
>>>>>>>>>
>>>>>
>>>>>problem
>>>>>
>>>>>
>>>>>
>>>>>>>>>because at the moment this vaildation happens in insert_messages().
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>OK, that's clear
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>Fine and dandy in pipe land, where the MTA will send the message
>>>>>>>>>
>>>>>>>>>
>>>>>
>>>>>regardless of
>>>>>
>>>>>
>>>>>
>>>>>>>>>the disposition of the recipients, and only at the very end are
>>>>>>>>>error
>>>>>>>>>conditions returned as exit codes... in LMTP land we need to know
>>>>>>>>>ahead of
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>time.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>>I think what I'll do is move the user validation code from
>>>>>>>>>insert_messages()
>>>>>>>>>into dsn.c, perhaps calling it dsn_check_users() or
>>>>>>>>>dsn_prepare_deliveries().
>>>>>>>>>Then, we'll have this order:
>>>>>>>>>
>>>>>>>>>For command-line and envelope-recipient deliveries:
>>>>>>>>>- receive addresses and store them into the dsnusers list.
>>>>>>>>>- dsn_prepare_deliveries()
>>>>>>>>>- if no deliveries are valid, return an error.
>>>>>>>>>
>>>>>>>>>- read_header()
>>>>>>>>>
>>>>>>>>>For deliver-to header deliveries:
>>>>>>>>>- mime_readheader()
>>>>>>>>>- mail_adr_list()
>>>>>>>>>- dsn_prepare_deliveries()
>>>>>>>>>- if no deliveries are valid, return an error.
>>>>>>>>>
>>>>>>>>>- insert_messages()
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>I like the part where you say 'What I'll do' :)
>>>>>>>>Do you think this work will take long? I think we must really
>>>>>>>>release rc3 tomorrow (Friday, March 5th). This stuff really has to
>>>>>>>>work if we want to release.
>>>>>>>>
>>>>>>>>Ilja
>>>>>>>>
>>>>>>>>_______________________________________________
>>>>>>>>Dbmail-dev mailing list
>>>>>>>>Dbmail-dev@dbmail.org
>>>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>_______________________________________________
>>>>>>Dbmail-dev mailing list
>>>>>>Dbmail-dev@dbmail.org
>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>_______________________________________________
>>>>Dbmail-dev mailing list
>>>>Dbmail-dev@dbmail.org
>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>
>>>>
>>>_______________________________________________
>>>Dbmail-dev mailing list
>>>Dbmail-dev@dbmail.org
>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>
>>>
>>_______________________________________________
>>Dbmail-dev mailing list
>>Dbmail-dev@dbmail.org
>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>
>>
>>
>
>
>
>
>

Hi,

Robert, thanks for your suggestion. It seems to work perfectly here.
Messages now get delivered to users without a problem.

I'll take a look at lmtp.c to see if I can spot the difference between
the server getting a QUIT from the client and a reconnection on a new
message, and a cached connection.

Ilja


Robert Theisen wrote:
> New to the list here. But I read over the archives regarding this
> problem and I may have a little to add.
>
> I was able to get postfix to consistently deliver via lmtp to dbmail
> (even two consecutive messages to the same user) by setting the postfix
> directive lmtp_cache_connection to false instead of true. This would
> termintate the lmtp connection after each delivery. I assume, with
> lmtp_cache_connection turned on, the connection would die after
> lmtp_timeout was reached and a new connection would be established and
> the mail would get sent. This may have been a "neat feature" that works
> with Cyrus (postfix's preferred lmtp client) or it may be a
> specification of the lmtp rfc (I have not read it). But it seems like a
> simple enough fix to make it work. It would probably just involve
> having the conversation status fall back to a state the postfix expects
> after message delivery and waiting in that state until either the
> connection terminates or the conversation picks up again. I took a
> cursory glance at the lmtp.c and lmtpd.c code but did not have enough
> time to really dig into it.
> Good Luck. I am anxiously awaiting the 2.0 release. :)
>
> Robert Theisen
>
> Aaron Stone wrote:
>
>> Sure thing, I've got about an hour of time this morning. Looks like
>> you're
>> burning the midnight oil over there! Yikes!
>>
>> Aaron
>>
>>
>> Ilja Booij <ilja@ic-s.nl> said:
>>
>>
>>
>>> Hi,
>>>
>>> I haven't been able to find the cause of the problem yet. I found
>>> some more info in the logs though:
>>>
>>> Mar 4 16:55:34 test01 postfix/lmtp[21340]: 3378119F30A:
>>> to=<target@test01.office.fastxs.net>,
>>> relay=localhost.fastxs.net[127.0.0.1], delay=0, status=bounced (host
>>> localhost.fastxs.net[127.0.0.1] said: 250 Recipient
>>> <target@test01.office.fastxs.net> OK)
>>>
>>> apparently, postfix thinks we want to bounce the message. But why? A
>>> previous message to the same user arrives correctly..
>>>
>>> strange, looking further. Aaron, can you also take a look at this?
>>>
>>> Ilja
>>>
>>> Ilja Booij wrote:
>>>
>>>
>>>
>>>> OK, I've found something:
>>>>
>>>> It seems that Postfix sends a RSET command, when we expect to get
>>>> the header.
>>>>
>>>> readheader() then waits for postfix to send more, while postfix
>>>> waits for a '250 OK' Message from dbmail-lmtp.
>>>>
>>>> So, there's probably something wrong in our LMTP-logic.
>>>> I'll take a look at it, after I've done some small scripting job
>>>> here for a customer.
>>>>
>>>> Ilja
>>>>
>>>> Ilja Booij wrote:
>>>>
>>>>
>>>>
>>>>> OK,
>>>>>
>>>>> this seems to have tackled the problem of the double delivery :)
>>>>>
>>>>> There still is another problem though:
>>>>> It still seems to hang for a while on every second delivery attempt
>>>>> to the LMTP daemon.
>>>>>
>>>>> Can you try the following scenario:
>>>>>
>>>>> * configure MTA to use dbmail-lmtp for delivery
>>>>> * send message to a recipient
>>>>> * verify that the message is received using dbmail-lmtp
>>>>> * send a second message.
>>>>>
>>>>> What I observe here is that the second message takes a lot longer
>>>>> to be delivered than the first one. The second one is only
>>>>> delivered after minutes.
>>>>>
>>>>> I have the feeling that something is not right here..
>>>>>
>>>>> Ilja
>>>>>
>>>>> Aaron Stone wrote:
>>>>>
>>>>>
>>>>>
>>>>>> New versions checked in, though not extensively tested yet.
>>>>>>
>>>>>> Ilja Booij <ilja@ic-s.nl> said:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> sounds ok to me :)
>>>>>>>
>>>>>>> Ilja
>>>>>>>
>>>>>>> Aaron Stone wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Working on it as we speak! Catch you in about an hour?
>>>>>>>>
>>>>>>>> Aaron
>>>>>>>>
>>>>>>>>
>>>>>>>> Ilja Booij <ilja@ic-s.nl> said:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Aaron Stone wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> The reason for the double deliveries in the LMTP daemon is
>>>>>>>>>> because the old
>>>>>>>>>> insert_messages() function used to take lists of users to
>>>>>>>>>> directly deliver.
>>>>>>>>>> The new insert_messages() function takes a list of dsnuser
>>>>>>>>>> structures, and
>>>>>>>>>> expects that *either* the useridnr field is filled (for
>>>>>>>>>> direct-to-useridnr
>>>>>>>>>> delivery, which isn't supported by any of the frontends, but
>>>>>>>>>> is supported in
>>>>>>>>>> the lower layers ;-) or the address field, which is first
>>>>>>>>>> checked as a
>>>>>>>>>> username and then as an alias (this allows usernames in the
>>>>>>>>>> form of
>>>>>>>>>> 'user@server' to operate without requiring an alias that says
>>>>>>>>>> 'user@server
>>>>>>>>>> delivers to user@server').
>>>>>>>>>>
>>>>>>>>>> Some refactoring might be necessary, because we need to inform
>>>>>>>>>> the MTA
>>>>>>>>>>
>>>>>>
>>>>>>
>>>>>> whether
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>>> or not its envelope recipients were valid before it will send
>>>>>>>>>> us the message.
>>>>>>>>>> This means users need to be validated before read_header(),
>>>>>>>>>> which is a
>>>>>>>>>>
>>>>>>
>>>>>>
>>>>>> problem
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>>> because at the moment this vaildation happens in
>>>>>>>>>> insert_messages().
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> OK, that's clear
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Fine and dandy in pipe land, where the MTA will send the message
>>>>>>>>>>
>>>>>>
>>>>>>
>>>>>> regardless of
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>>> the disposition of the recipients, and only at the very end
>>>>>>>>>> are error
>>>>>>>>>> conditions returned as exit codes... in LMTP land we need to
>>>>>>>>>> know ahead of
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> time.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>> I think what I'll do is move the user validation code from
>>>>>>>>>> insert_messages()
>>>>>>>>>> into dsn.c, perhaps calling it dsn_check_users() or
>>>>>>>>>> dsn_prepare_deliveries().
>>>>>>>>>> Then, we'll have this order:
>>>>>>>>>>
>>>>>>>>>> For command-line and envelope-recipient deliveries:
>>>>>>>>>> - receive addresses and store them into the dsnusers list.
>>>>>>>>>> - dsn_prepare_deliveries()
>>>>>>>>>> - if no deliveries are valid, return an error.
>>>>>>>>>>
>>>>>>>>>> - read_header()
>>>>>>>>>>
>>>>>>>>>> For deliver-to header deliveries:
>>>>>>>>>> - mime_readheader()
>>>>>>>>>> - mail_adr_list()
>>>>>>>>>> - dsn_prepare_deliveries()
>>>>>>>>>> - if no deliveries are valid, return an error.
>>>>>>>>>>
>>>>>>>>>> - insert_messages()
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I like the part where you say 'What I'll do' :)
>>>>>>>>> Do you think this work will take long? I think we must really
>>>>>>>>> release rc3 tomorrow (Friday, March 5th). This stuff really has
>>>>>>>>> to work if we want to release.
>>>>>>>>>
>>>>>>>>> Ilja
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Dbmail-dev mailing list
>>>>>>>>> Dbmail-dev@dbmail.org
>>>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Dbmail-dev mailing list
>>>>>>> Dbmail-dev@dbmail.org
>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Dbmail-dev mailing list
>>>>> Dbmail-dev@dbmail.org
>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>
>>>>
>>>> _______________________________________________
>>>> Dbmail-dev mailing list
>>>> Dbmail-dev@dbmail.org
>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>
>>>
>>> _______________________________________________
>>> Dbmail-dev mailing list
>>> Dbmail-dev@dbmail.org
>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>
>>>
>>
>>
>>
>>
>>
>>
>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev

Hmm, I still don't really understand the problem. I've done some
ngrep-ing, to see what goes over the network.

This is a session that's OK:

T 2004/03/05 13:45:46.324232 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
220 test01 DBMail LMTP service ready to rock..

##
T 2004/03/05 13:45:46.324696 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
LHLO test01.office.fastxs.net..

##
T 2004/03/05 13:45:46.324900 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
250-test01..250-PIPELINING..250-ENHANCEDSTATUSCODES..250 SIZE..

#
T 2004/03/05 13:45:46.325306 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
MAIL FROM:<ilja@earthquake.office.fastxs.net> SIZE=866..RCPT
TO:<target@test01.office.fastxs.net>..DATA..
#
T 2004/03/05 13:45:46.325479 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
250 Sender <ilja@earthquake.office.fastxs.net> OK..

##
T 2004/03/05 13:45:46.365648 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
250 Recipient <target@test01.office.fastxs.net> OK..354 Start mail
input; end with <CRLF>.<CRLF>..
##
T 2004/03/05 13:45:46.365974 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
Received: from earthquake.office.fastxs.net
(earthquake.office.fastxs.net [10.0.1.15])...by test01.office.fastxs.n
et (Postfix) with ESMTP id 394DD19F30A...for
<target@test01.office.fastxs.net>; Fri, 5 Mar 2004 13:45:46 +0100 (C
ET)..Received: from earthquake.office.fastxs.net (localhost
[127.0.0.1])...by earthquake.office.fastxs.net (Postfi
x) with ESMTP id 405D4375FE...for <target@test01.office.fastxs.net>;
Fri, 5 Mar 2004 13:49:12 +0100 (CET)..Receiv
ed: (from ilja@localhost)...by earthquake.office.fastxs.net
(8.12.9/8.12.9/Submit) id i25CnCo0005116...for target@
test01.office.fastxs.net; Fri, 5 Mar 2004 13:49:12 +0100 (CET)..Date:
Fri, 5 Mar 2004 13:49:12 +0100 (CET)..From:
Ilja Booij <ilja@earthquake.office.fastxs.net>..Message-Id:
<200403051249.i25CnCo0005116@earthquake.office.fastxs.
27......27.....
##
T 2004/03/05 13:45:46.568851 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
250 Message received OK..

##
T 2004/03/05 13:45:46.608610 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
250 Recipient <target@test01.office.fastxs.net> OK..



If another message is sent (over the same connection):


##
T 2004/03/05 13:46:01.879964 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
RSET..

##
T 2004/03/05 13:46:01.880127 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
MAIL FROM:<ilja@earthquake.office.fastxs.net> SIZE=858..RCPT
TO:<ilja@test01.office.fastxs.net>..DATA..
##
T 2004/03/05 13:46:01.880368 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
250 OK..

##
T 2004/03/05 13:46:01.880460 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
250 Sender <ilja@earthquake.office.fastxs.net> OK..

##
T 2004/03/05 13:46:01.883495 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
250 Recipient <ilja@test01.office.fastxs.net> OK..

##
T 2004/03/05 13:46:01.883541 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
354 Start mail input; end with <CRLF>.<CRLF>..

##
T 2004/03/05 13:47:41.920055 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
RSET..QUIT..

###
T 2004/03/05 13:52:41.875033 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
500 Error reading header...

#

You can see that the client (postfix) starts with an RSET command, and
starts sending MAIL_FROM, RCPT and DATA.
The server responds with 250 OK to the reset, and 250 Sender OK and 250
Recipient OK, and lets the client start sending (354 Start mail input).

The problem is that Postfix detects that it should not send data,
because the message is supposedly bounced by dbmail.. But why?

Ilja

Ilja Booij wrote:

> Hi,
>
> Robert, thanks for your suggestion. It seems to work perfectly here.
> Messages now get delivered to users without a problem.
>
> I'll take a look at lmtp.c to see if I can spot the difference between
> the server getting a QUIT from the client and a reconnection on a new
> message, and a cached connection.
>
> Ilja
>
>
> Robert Theisen wrote:
>
>> New to the list here. But I read over the archives regarding this
>> problem and I may have a little to add.
>>
>> I was able to get postfix to consistently deliver via lmtp to dbmail
>> (even two consecutive messages to the same user) by setting the
>> postfix directive lmtp_cache_connection to false instead of true.
>> This would termintate the lmtp connection after each delivery. I
>> assume, with lmtp_cache_connection turned on, the connection would die
>> after lmtp_timeout was reached and a new connection would be
>> established and the mail would get sent. This may have been a "neat
>> feature" that works with Cyrus (postfix's preferred lmtp client) or it
>> may be a specification of the lmtp rfc (I have not read it). But it
>> seems like a simple enough fix to make it work. It would probably
>> just involve having the conversation status fall back to a state the
>> postfix expects after message delivery and waiting in that state until
>> either the connection terminates or the conversation picks up again.
>> I took a cursory glance at the lmtp.c and lmtpd.c code but did not
>> have enough time to really dig into it.
>> Good Luck. I am anxiously awaiting the 2.0 release. :)
>>
>> Robert Theisen
>>
>> Aaron Stone wrote:
>>
>>> Sure thing, I've got about an hour of time this morning. Looks like
>>> you're
>>> burning the midnight oil over there! Yikes!
>>>
>>> Aaron
>>>
>>>
>>> Ilja Booij <ilja@ic-s.nl> said:
>>>
>>>
>>>
>>>> Hi,
>>>>
>>>> I haven't been able to find the cause of the problem yet. I found
>>>> some more info in the logs though:
>>>>
>>>> Mar 4 16:55:34 test01 postfix/lmtp[21340]: 3378119F30A:
>>>> to=<target@test01.office.fastxs.net>,
>>>> relay=localhost.fastxs.net[127.0.0.1], delay=0, status=bounced (host
>>>> localhost.fastxs.net[127.0.0.1] said: 250 Recipient
>>>> <target@test01.office.fastxs.net> OK)
>>>>
>>>> apparently, postfix thinks we want to bounce the message. But why? A
>>>> previous message to the same user arrives correctly..
>>>>
>>>> strange, looking further. Aaron, can you also take a look at this?
>>>>
>>>> Ilja
>>>>
>>>> Ilja Booij wrote:
>>>>
>>>>
>>>>
>>>>> OK, I've found something:
>>>>>
>>>>> It seems that Postfix sends a RSET command, when we expect to get
>>>>> the header.
>>>>>
>>>>> readheader() then waits for postfix to send more, while postfix
>>>>> waits for a '250 OK' Message from dbmail-lmtp.
>>>>>
>>>>> So, there's probably something wrong in our LMTP-logic.
>>>>> I'll take a look at it, after I've done some small scripting job
>>>>> here for a customer.
>>>>>
>>>>> Ilja
>>>>>
>>>>> Ilja Booij wrote:
>>>>>
>>>>>
>>>>>
>>>>>> OK,
>>>>>>
>>>>>> this seems to have tackled the problem of the double delivery :)
>>>>>>
>>>>>> There still is another problem though:
>>>>>> It still seems to hang for a while on every second delivery
>>>>>> attempt to the LMTP daemon.
>>>>>>
>>>>>> Can you try the following scenario:
>>>>>>
>>>>>> * configure MTA to use dbmail-lmtp for delivery
>>>>>> * send message to a recipient
>>>>>> * verify that the message is received using dbmail-lmtp
>>>>>> * send a second message.
>>>>>>
>>>>>> What I observe here is that the second message takes a lot longer
>>>>>> to be delivered than the first one. The second one is only
>>>>>> delivered after minutes.
>>>>>>
>>>>>> I have the feeling that something is not right here..
>>>>>>
>>>>>> Ilja
>>>>>>
>>>>>> Aaron Stone wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> New versions checked in, though not extensively tested yet.
>>>>>>>
>>>>>>> Ilja Booij <ilja@ic-s.nl> said:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> sounds ok to me :)
>>>>>>>>
>>>>>>>> Ilja
>>>>>>>>
>>>>>>>> Aaron Stone wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Working on it as we speak! Catch you in about an hour?
>>>>>>>>>
>>>>>>>>> Aaron
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Ilja Booij <ilja@ic-s.nl> said:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> Aaron Stone wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> The reason for the double deliveries in the LMTP daemon is
>>>>>>>>>>> because the old
>>>>>>>>>>> insert_messages() function used to take lists of users to
>>>>>>>>>>> directly deliver.
>>>>>>>>>>> The new insert_messages() function takes a list of dsnuser
>>>>>>>>>>> structures, and
>>>>>>>>>>> expects that *either* the useridnr field is filled (for
>>>>>>>>>>> direct-to-useridnr
>>>>>>>>>>> delivery, which isn't supported by any of the frontends, but
>>>>>>>>>>> is supported in
>>>>>>>>>>> the lower layers ;-) or the address field, which is first
>>>>>>>>>>> checked as a
>>>>>>>>>>> username and then as an alias (this allows usernames in the
>>>>>>>>>>> form of
>>>>>>>>>>> 'user@server' to operate without requiring an alias that says
>>>>>>>>>>> 'user@server
>>>>>>>>>>> delivers to user@server').
>>>>>>>>>>>
>>>>>>>>>>> Some refactoring might be necessary, because we need to
>>>>>>>>>>> inform the MTA
>>>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> whether
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>>>> or not its envelope recipients were valid before it will send
>>>>>>>>>>> us the message.
>>>>>>>>>>> This means users need to be validated before read_header(),
>>>>>>>>>>> which is a
>>>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> problem
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>>>> because at the moment this vaildation happens in
>>>>>>>>>>> insert_messages().
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> OK, that's clear
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Fine and dandy in pipe land, where the MTA will send the message
>>>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> regardless of
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>>>> the disposition of the recipients, and only at the very end
>>>>>>>>>>> are error
>>>>>>>>>>> conditions returned as exit codes... in LMTP land we need to
>>>>>>>>>>> know ahead of
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> time.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> I think what I'll do is move the user validation code from
>>>>>>>>>>> insert_messages()
>>>>>>>>>>> into dsn.c, perhaps calling it dsn_check_users() or
>>>>>>>>>>> dsn_prepare_deliveries().
>>>>>>>>>>> Then, we'll have this order:
>>>>>>>>>>>
>>>>>>>>>>> For command-line and envelope-recipient deliveries:
>>>>>>>>>>> - receive addresses and store them into the dsnusers list.
>>>>>>>>>>> - dsn_prepare_deliveries()
>>>>>>>>>>> - if no deliveries are valid, return an error.
>>>>>>>>>>>
>>>>>>>>>>> - read_header()
>>>>>>>>>>>
>>>>>>>>>>> For deliver-to header deliveries:
>>>>>>>>>>> - mime_readheader()
>>>>>>>>>>> - mail_adr_list()
>>>>>>>>>>> - dsn_prepare_deliveries()
>>>>>>>>>>> - if no deliveries are valid, return an error.
>>>>>>>>>>>
>>>>>>>>>>> - insert_messages()
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I like the part where you say 'What I'll do' :)
>>>>>>>>>> Do you think this work will take long? I think we must really
>>>>>>>>>> release rc3 tomorrow (Friday, March 5th). This stuff really
>>>>>>>>>> has to work if we want to release.
>>>>>>>>>>
>>>>>>>>>> Ilja
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Dbmail-dev mailing list
>>>>>>>>>> Dbmail-dev@dbmail.org
>>>>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Dbmail-dev mailing list
>>>>>>>> Dbmail-dev@dbmail.org
>>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Dbmail-dev mailing list
>>>>>> Dbmail-dev@dbmail.org
>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Dbmail-dev mailing list
>>>>> Dbmail-dev@dbmail.org
>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Dbmail-dev mailing list
>>>> Dbmail-dev@dbmail.org
>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>
>>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>> _______________________________________________
>> Dbmail-dev mailing list
>> Dbmail-dev@dbmail.org
>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev

I'm going to go and get Cyrus and play with their lmtp server and see what
differs. Postfix seems to be strict about a few little things... it's possible
that they'd only tested with Cyrus or a small handful of others and therefore
have some extra assumptions about what an LMTP session is supposed to look
like. RFC's and Real Life Don't Always Match (TM).

So, for example, when I was talking smtp directly with Postfix, the MAIL FROM:
and RCPT TO: commands were only accepted with the colons, ':', and otherwise
generated an error. IMHO, the RFC suggests that little "errors" like that
should be accepted anyhow.

Aaron


Ilja Booij <ilja@ic-s.nl> said:

> Hmm, I still don't really understand the problem. I've done some
> ngrep-ing, to see what goes over the network.
>
> This is a session that's OK:
>
> T 2004/03/05 13:45:46.324232 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> 220 test01 DBMail LMTP service ready to rock..
>
> ##
> T 2004/03/05 13:45:46.324696 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> LHLO test01.office.fastxs.net..
>
> ##
> T 2004/03/05 13:45:46.324900 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> 250-test01..250-PIPELINING..250-ENHANCEDSTATUSCODES..250 SIZE..
>
> #
> T 2004/03/05 13:45:46.325306 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> MAIL FROM:<ilja@earthquake.office.fastxs.net> SIZE=866..RCPT
> TO:<target@test01.office.fastxs.net>..DATA..
> #
> T 2004/03/05 13:45:46.325479 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> 250 Sender <ilja@earthquake.office.fastxs.net> OK..
>
> ##
> T 2004/03/05 13:45:46.365648 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> 250 Recipient <target@test01.office.fastxs.net> OK..354 Start mail
> input; end with <CRLF>.<CRLF>..
> ##
> T 2004/03/05 13:45:46.365974 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> Received: from earthquake.office.fastxs.net
> (earthquake.office.fastxs.net [10.0.1.15])...by test01.office.fastxs.n
> et (Postfix) with ESMTP id 394DD19F30A...for
> <target@test01.office.fastxs.net>; Fri, 5 Mar 2004 13:45:46 +0100 (C
> ET)..Received: from earthquake.office.fastxs.net (localhost
> [127.0.0.1])...by earthquake.office.fastxs.net (Postfi
> x) with ESMTP id 405D4375FE...for <target@test01.office.fastxs.net>;
> Fri, 5 Mar 2004 13:49:12 +0100 (CET)..Receiv
> ed: (from ilja@localhost)...by earthquake.office.fastxs.net
> (8.12.9/8.12.9/Submit) id i25CnCo0005116...for target@
> test01.office.fastxs.net; Fri, 5 Mar 2004 13:49:12 +0100 (CET)..Date:
> Fri, 5 Mar 2004 13:49:12 +0100 (CET)..From:
> Ilja Booij <ilja@earthquake.office.fastxs.net>..Message-Id:
> <200403051249.i25CnCo0005116@earthquake.office.fastxs.
> net>..To: target@test01.office.fastxs.net..Subject: 5-3
> 27......27.....
> ##
> T 2004/03/05 13:45:46.568851 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> 250 Message received OK..
>
> ##
> T 2004/03/05 13:45:46.608610 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> 250 Recipient <target@test01.office.fastxs.net> OK..
>
>
>
> If another message is sent (over the same connection):
>
>
> ##
> T 2004/03/05 13:46:01.879964 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> RSET..
>
> ##
> T 2004/03/05 13:46:01.880127 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> MAIL FROM:<ilja@earthquake.office.fastxs.net> SIZE=858..RCPT
> TO:<ilja@test01.office.fastxs.net>..DATA..
> ##
> T 2004/03/05 13:46:01.880368 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> 250 OK..
>
> ##
> T 2004/03/05 13:46:01.880460 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> 250 Sender <ilja@earthquake.office.fastxs.net> OK..
>
> ##
> T 2004/03/05 13:46:01.883495 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> 250 Recipient <ilja@test01.office.fastxs.net> OK..
>
> ##
> T 2004/03/05 13:46:01.883541 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> 354 Start mail input; end with <CRLF>.<CRLF>..
>
> ##
> T 2004/03/05 13:47:41.920055 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> RSET..QUIT..
>
> ###
> T 2004/03/05 13:52:41.875033 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> 500 Error reading header...
>
> #
>
> You can see that the client (postfix) starts with an RSET command, and
> starts sending MAIL_FROM, RCPT and DATA.
> The server responds with 250 OK to the reset, and 250 Sender OK and 250
> Recipient OK, and lets the client start sending (354 Start mail input).
>
> The problem is that Postfix detects that it should not send data,
> because the message is supposedly bounced by dbmail.. But why?
>
> Ilja
>
> Ilja Booij wrote:
>
> > Hi,
> >
> > Robert, thanks for your suggestion. It seems to work perfectly here.
> > Messages now get delivered to users without a problem.
> >
> > I'll take a look at lmtp.c to see if I can spot the difference between
> > the server getting a QUIT from the client and a reconnection on a new
> > message, and a cached connection.
> >
> > Ilja
> >
> >
> > Robert Theisen wrote:
> >
> >> New to the list here. But I read over the archives regarding this
> >> problem and I may have a little to add.
> >>
> >> I was able to get postfix to consistently deliver via lmtp to dbmail
> >> (even two consecutive messages to the same user) by setting the
> >> postfix directive lmtp_cache_connection to false instead of true.
> >> This would termintate the lmtp connection after each delivery. I
> >> assume, with lmtp_cache_connection turned on, the connection would die
> >> after lmtp_timeout was reached and a new connection would be
> >> established and the mail would get sent. This may have been a "neat
> >> feature" that works with Cyrus (postfix's preferred lmtp client) or it
> >> may be a specification of the lmtp rfc (I have not read it). But it
> >> seems like a simple enough fix to make it work. It would probably
> >> just involve having the conversation status fall back to a state the
> >> postfix expects after message delivery and waiting in that state until
> >> either the connection terminates or the conversation picks up again.
> >> I took a cursory glance at the lmtp.c and lmtpd.c code but did not
> >> have enough time to really dig into it.
> >> Good Luck. I am anxiously awaiting the 2.0 release. :)
> >>
> >> Robert Theisen
> >>
> >> Aaron Stone wrote:
> >>
> >>> Sure thing, I've got about an hour of time this morning. Looks like
> >>> you're
> >>> burning the midnight oil over there! Yikes!
> >>>
> >>> Aaron
> >>>
> >>>
> >>> Ilja Booij <ilja@ic-s.nl> said:
> >>>
> >>>
> >>>
> >>>> Hi,
> >>>>
> >>>> I haven't been able to find the cause of the problem yet. I found
> >>>> some more info in the logs though:
> >>>>
> >>>> Mar 4 16:55:34 test01 postfix/lmtp[21340]: 3378119F30A:
> >>>> to=<target@test01.office.fastxs.net>,
> >>>> relay=localhost.fastxs.net[127.0.0.1], delay=0, status=bounced (host
> >>>> localhost.fastxs.net[127.0.0.1] said: 250 Recipient
> >>>> <target@test01.office.fastxs.net> OK)
> >>>>
> >>>> apparently, postfix thinks we want to bounce the message. But why? A
> >>>> previous message to the same user arrives correctly..
> >>>>
> >>>> strange, looking further. Aaron, can you also take a look at this?
> >>>>
> >>>> Ilja
> >>>>
> >>>> Ilja Booij wrote:
> >>>>
> >>>>
> >>>>
> >>>>> OK, I've found something:
> >>>>>
> >>>>> It seems that Postfix sends a RSET command, when we expect to get
> >>>>> the header.
> >>>>>
> >>>>> readheader() then waits for postfix to send more, while postfix
> >>>>> waits for a '250 OK' Message from dbmail-lmtp.
> >>>>>
> >>>>> So, there's probably something wrong in our LMTP-logic.
> >>>>> I'll take a look at it, after I've done some small scripting job
> >>>>> here for a customer.
> >>>>>
> >>>>> Ilja
> >>>>>
> >>>>> Ilja Booij wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>>> OK,
> >>>>>>
> >>>>>> this seems to have tackled the problem of the double delivery :)
> >>>>>>
> >>>>>> There still is another problem though:
> >>>>>> It still seems to hang for a while on every second delivery
> >>>>>> attempt to the LMTP daemon.
> >>>>>>
> >>>>>> Can you try the following scenario:
> >>>>>>
> >>>>>> * configure MTA to use dbmail-lmtp for delivery
> >>>>>> * send message to a recipient
> >>>>>> * verify that the message is received using dbmail-lmtp
> >>>>>> * send a second message.
> >>>>>>
> >>>>>> What I observe here is that the second message takes a lot longer
> >>>>>> to be delivered than the first one. The second one is only
> >>>>>> delivered after minutes.
> >>>>>>
> >>>>>> I have the feeling that something is not right here..
> >>>>>>
> >>>>>> Ilja
> >>>>>>
> >>>>>> Aaron Stone wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> New versions checked in, though not extensively tested yet.
> >>>>>>>
> >>>>>>> Ilja Booij <ilja@ic-s.nl> said:
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> sounds ok to me :)
> >>>>>>>>
> >>>>>>>> Ilja
> >>>>>>>>
> >>>>>>>> Aaron Stone wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> Working on it as we speak! Catch you in about an hour?
> >>>>>>>>>
> >>>>>>>>> Aaron
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Ilja Booij <ilja@ic-s.nl> said:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> Hi,
> >>>>>>>>>>
> >>>>>>>>>> Aaron Stone wrote:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> The reason for the double deliveries in the LMTP daemon is
> >>>>>>>>>>> because the old
> >>>>>>>>>>> insert_messages() function used to take lists of users to
> >>>>>>>>>>> directly deliver.
> >>>>>>>>>>> The new insert_messages() function takes a list of dsnuser
> >>>>>>>>>>> structures, and
> >>>>>>>>>>> expects that *either* the useridnr field is filled (for
> >>>>>>>>>>> direct-to-useridnr
> >>>>>>>>>>> delivery, which isn't supported by any of the frontends, but
> >>>>>>>>>>> is supported in
> >>>>>>>>>>> the lower layers ;-) or the address field, which is first
> >>>>>>>>>>> checked as a
> >>>>>>>>>>> username and then as an alias (this allows usernames in the
> >>>>>>>>>>> form of
> >>>>>>>>>>> 'user@server' to operate without requiring an alias that says
> >>>>>>>>>>> 'user@server
> >>>>>>>>>>> delivers to user@server').
> >>>>>>>>>>>
> >>>>>>>>>>> Some refactoring might be necessary, because we need to
> >>>>>>>>>>> inform the MTA
> >>>>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> whether
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>>>>> or not its envelope recipients were valid before it will send
> >>>>>>>>>>> us the message.
> >>>>>>>>>>> This means users need to be validated before read_header(),
> >>>>>>>>>>> which is a
> >>>>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> problem
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>>>>> because at the moment this vaildation happens in
> >>>>>>>>>>> insert_messages().
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> OK, that's clear
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> Fine and dandy in pipe land, where the MTA will send the message
> >>>>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> regardless of
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>>>>> the disposition of the recipients, and only at the very end
> >>>>>>>>>>> are error
> >>>>>>>>>>> conditions returned as exit codes... in LMTP land we need to
> >>>>>>>>>>> know ahead of
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> time.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>> I think what I'll do is move the user validation code from
> >>>>>>>>>>> insert_messages()
> >>>>>>>>>>> into dsn.c, perhaps calling it dsn_check_users() or
> >>>>>>>>>>> dsn_prepare_deliveries().
> >>>>>>>>>>> Then, we'll have this order:
> >>>>>>>>>>>
> >>>>>>>>>>> For command-line and envelope-recipient deliveries:
> >>>>>>>>>>> - receive addresses and store them into the dsnusers list.
> >>>>>>>>>>> - dsn_prepare_deliveries()
> >>>>>>>>>>> - if no deliveries are valid, return an error.
> >>>>>>>>>>>
> >>>>>>>>>>> - read_header()
> >>>>>>>>>>>
> >>>>>>>>>>> For deliver-to header deliveries:
> >>>>>>>>>>> - mime_readheader()
> >>>>>>>>>>> - mail_adr_list()
> >>>>>>>>>>> - dsn_prepare_deliveries()
> >>>>>>>>>>> - if no deliveries are valid, return an error.
> >>>>>>>>>>>
> >>>>>>>>>>> - insert_messages()
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> I like the part where you say 'What I'll do' :)
> >>>>>>>>>> Do you think this work will take long? I think we must really
> >>>>>>>>>> release rc3 tomorrow (Friday, March 5th). This stuff really
> >>>>>>>>>> has to work if we want to release.
> >>>>>>>>>>
> >>>>>>>>>> Ilja
> >>>>>>>>>>
> >>>>>>>>>> _______________________________________________
> >>>>>>>>>> Dbmail-dev mailing list
> >>>>>>>>>> Dbmail-dev@dbmail.org
> >>>>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> _______________________________________________
> >>>>>>>> Dbmail-dev mailing list
> >>>>>>>> Dbmail-dev@dbmail.org
> >>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> Dbmail-dev mailing list
> >>>>>> Dbmail-dev@dbmail.org
> >>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>>>>>
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> Dbmail-dev mailing list
> >>>>> Dbmail-dev@dbmail.org
> >>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>>>>
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> Dbmail-dev mailing list
> >>>> Dbmail-dev@dbmail.org
> >>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>>>
> >>>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>
> >> _______________________________________________
> >> Dbmail-dev mailing list
> >> Dbmail-dev@dbmail.org
> >> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >
> > _______________________________________________
> > Dbmail-dev mailing list
> > Dbmail-dev@dbmail.org
> > http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>



--

Hi,

with some help from a guy on postfix-users@postfix.org I found something:

The "250 Recipient <target@test01.office.fastxs.net> OK" message from
dbmail-lmtp is out of sync. The postfix/lmtp program stores this message
until the next message is sent, causing the messages between postfix and
dbmail-smtp to be horribly out-of-sync.

I guess this should be quite easy to fix. I'll see if I have some time
to do it this weekend. Otherwise I'll do it on Monday. Unless anybody
else wants to do it of course ;)

I'll be off in a few minutes. First a beer, then it's off to home :)

Have a nice weekend!

Ilja


Ilja Booij wrote:

> Hmm, I still don't really understand the problem. I've done some
> ngrep-ing, to see what goes over the network.
>
> This is a session that's OK:
>
> T 2004/03/05 13:45:46.324232 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> 220 test01 DBMail LMTP service ready to rock..
> ##
> T 2004/03/05 13:45:46.324696 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> LHLO test01.office.fastxs.net..
> ##
> T 2004/03/05 13:45:46.324900 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> 250-test01..250-PIPELINING..250-ENHANCEDSTATUSCODES..250 SIZE..
> #
> T 2004/03/05 13:45:46.325306 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> MAIL FROM:<ilja@earthquake.office.fastxs.net> SIZE=866..RCPT
> TO:<target@test01.office.fastxs.net>..DATA..
> #
> T 2004/03/05 13:45:46.325479 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> 250 Sender <ilja@earthquake.office.fastxs.net> OK..
> ##
> T 2004/03/05 13:45:46.365648 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> 250 Recipient <target@test01.office.fastxs.net> OK..354 Start mail
> input; end with <CRLF>.<CRLF>..
> ##
> T 2004/03/05 13:45:46.365974 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> Received: from earthquake.office.fastxs.net
> (earthquake.office.fastxs.net [10.0.1.15])...by test01.office.fastxs.n
> et (Postfix) with ESMTP id 394DD19F30A...for
> <target@test01.office.fastxs.net>; Fri, 5 Mar 2004 13:45:46 +0100 (C
> ET)..Received: from earthquake.office.fastxs.net (localhost
> [127.0.0.1])...by earthquake.office.fastxs.net (Postfi
> x) with ESMTP id 405D4375FE...for <target@test01.office.fastxs.net>;
> Fri, 5 Mar 2004 13:49:12 +0100 (CET)..Receiv
> ed: (from ilja@localhost)...by earthquake.office.fastxs.net
> (8.12.9/8.12.9/Submit) id i25CnCo0005116...for target@
> test01.office.fastxs.net; Fri, 5 Mar 2004 13:49:12 +0100 (CET)..Date:
> Fri, 5 Mar 2004 13:49:12 +0100 (CET)..From:
> Ilja Booij <ilja@earthquake.office.fastxs.net>..Message-Id:
> <200403051249.i25CnCo0005116@earthquake.office.fastxs.
> net>..To: target@test01.office.fastxs.net..Subject: 5-3 27......27.....
> ##
> T 2004/03/05 13:45:46.568851 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> 250 Message received OK..
> ##
> T 2004/03/05 13:45:46.608610 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> c..
>
>
>
> If another message is sent (over the same connection):
>
>
> ##
> T 2004/03/05 13:46:01.879964 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> RSET..
> ##
> T 2004/03/05 13:46:01.880127 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> MAIL FROM:<ilja@earthquake.office.fastxs.net> SIZE=858..RCPT
> TO:<ilja@test01.office.fastxs.net>..DATA..
> ##
> T 2004/03/05 13:46:01.880368 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> 250 OK..
> ##
> T 2004/03/05 13:46:01.880460 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> 250 Sender <ilja@earthquake.office.fastxs.net> OK..
> ##
> T 2004/03/05 13:46:01.883495 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> 250 Recipient <ilja@test01.office.fastxs.net> OK..
> ##
> T 2004/03/05 13:46:01.883541 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> 354 Start mail input; end with <CRLF>.<CRLF>..
> ##
> T 2004/03/05 13:47:41.920055 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> RSET..QUIT..
> ###
> T 2004/03/05 13:52:41.875033 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> 500 Error reading header...
> #
>
> You can see that the client (postfix) starts with an RSET command, and
> starts sending MAIL_FROM, RCPT and DATA.
> The server responds with 250 OK to the reset, and 250 Sender OK and 250
> Recipient OK, and lets the client start sending (354 Start mail input).
>
> The problem is that Postfix detects that it should not send data,
> because the message is supposedly bounced by dbmail.. But why?
>
> Ilja
>
> Ilja Booij wrote:
>
>> Hi,
>>
>> Robert, thanks for your suggestion. It seems to work perfectly here.
>> Messages now get delivered to users without a problem.
>>
>> I'll take a look at lmtp.c to see if I can spot the difference between
>> the server getting a QUIT from the client and a reconnection on a new
>> message, and a cached connection.
>>
>> Ilja
>>
>>
>> Robert Theisen wrote:
>>
>>> New to the list here. But I read over the archives regarding this
>>> problem and I may have a little to add.
>>>
>>> I was able to get postfix to consistently deliver via lmtp to dbmail
>>> (even two consecutive messages to the same user) by setting the
>>> postfix directive lmtp_cache_connection to false instead of true.
>>> This would termintate the lmtp connection after each delivery. I
>>> assume, with lmtp_cache_connection turned on, the connection would
>>> die after lmtp_timeout was reached and a new connection would be
>>> established and the mail would get sent. This may have been a "neat
>>> feature" that works with Cyrus (postfix's preferred lmtp client) or
>>> it may be a specification of the lmtp rfc (I have not read it). But
>>> it seems like a simple enough fix to make it work. It would probably
>>> just involve having the conversation status fall back to a state the
>>> postfix expects after message delivery and waiting in that state
>>> until either the connection terminates or the conversation picks up
>>> again. I took a cursory glance at the lmtp.c and lmtpd.c code but
>>> did not have enough time to really dig into it.
>>> Good Luck. I am anxiously awaiting the 2.0 release. :)
>>>
>>> Robert Theisen
>>>
>>> Aaron Stone wrote:
>>>
>>>> Sure thing, I've got about an hour of time this morning. Looks like
>>>> you're
>>>> burning the midnight oil over there! Yikes!
>>>>
>>>> Aaron
>>>>
>>>>
>>>> Ilja Booij <ilja@ic-s.nl> said:
>>>>
>>>>
>>>>
>>>>> Hi,
>>>>>
>>>>> I haven't been able to find the cause of the problem yet. I found
>>>>> some more info in the logs though:
>>>>>
>>>>> Mar 4 16:55:34 test01 postfix/lmtp[21340]: 3378119F30A:
>>>>> to=<target@test01.office.fastxs.net>,
>>>>> relay=localhost.fastxs.net[127.0.0.1], delay=0, status=bounced
>>>>> (host localhost.fastxs.net[127.0.0.1] said: 250 Recipient
>>>>> <target@test01.office.fastxs.net> OK)
>>>>>
>>>>> apparently, postfix thinks we want to bounce the message. But why?
>>>>> A previous message to the same user arrives correctly..
>>>>>
>>>>> strange, looking further. Aaron, can you also take a look at this?
>>>>>
>>>>> Ilja
>>>>>
>>>>> Ilja Booij wrote:
>>>>>
>>>>>
>>>>>
>>>>>> OK, I've found something:
>>>>>>
>>>>>> It seems that Postfix sends a RSET command, when we expect to get
>>>>>> the header.
>>>>>>
>>>>>> readheader() then waits for postfix to send more, while postfix
>>>>>> waits for a '250 OK' Message from dbmail-lmtp.
>>>>>>
>>>>>> So, there's probably something wrong in our LMTP-logic.
>>>>>> I'll take a look at it, after I've done some small scripting job
>>>>>> here for a customer.
>>>>>>
>>>>>> Ilja
>>>>>>
>>>>>> Ilja Booij wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> OK,
>>>>>>>
>>>>>>> this seems to have tackled the problem of the double delivery :)
>>>>>>>
>>>>>>> There still is another problem though:
>>>>>>> It still seems to hang for a while on every second delivery
>>>>>>> attempt to the LMTP daemon.
>>>>>>>
>>>>>>> Can you try the following scenario:
>>>>>>>
>>>>>>> * configure MTA to use dbmail-lmtp for delivery
>>>>>>> * send message to a recipient
>>>>>>> * verify that the message is received using dbmail-lmtp
>>>>>>> * send a second message.
>>>>>>>
>>>>>>> What I observe here is that the second message takes a lot longer
>>>>>>> to be delivered than the first one. The second one is only
>>>>>>> delivered after minutes.
>>>>>>>
>>>>>>> I have the feeling that something is not right here..
>>>>>>>
>>>>>>> Ilja
>>>>>>>
>>>>>>> Aaron Stone wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> New versions checked in, though not extensively tested yet.
>>>>>>>>
>>>>>>>> Ilja Booij <ilja@ic-s.nl> said:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> sounds ok to me :)
>>>>>>>>>
>>>>>>>>> Ilja
>>>>>>>>>
>>>>>>>>> Aaron Stone wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Working on it as we speak! Catch you in about an hour?
>>>>>>>>>>
>>>>>>>>>> Aaron
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Ilja Booij <ilja@ic-s.nl> said:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> Aaron Stone wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> The reason for the double deliveries in the LMTP daemon is
>>>>>>>>>>>> because the old
>>>>>>>>>>>> insert_messages() function used to take lists of users to
>>>>>>>>>>>> directly deliver.
>>>>>>>>>>>> The new insert_messages() function takes a list of dsnuser
>>>>>>>>>>>> structures, and
>>>>>>>>>>>> expects that *either* the useridnr field is filled (for
>>>>>>>>>>>> direct-to-useridnr
>>>>>>>>>>>> delivery, which isn't supported by any of the frontends, but
>>>>>>>>>>>> is supported in
>>>>>>>>>>>> the lower layers ;-) or the address field, which is first
>>>>>>>>>>>> checked as a
>>>>>>>>>>>> username and then as an alias (this allows usernames in the
>>>>>>>>>>>> form of
>>>>>>>>>>>> 'user@server' to operate without requiring an alias that
>>>>>>>>>>>> says 'user@server
>>>>>>>>>>>> delivers to user@server').
>>>>>>>>>>>>
>>>>>>>>>>>> Some refactoring might be necessary, because we need to
>>>>>>>>>>>> inform the MTA
>>>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> whether
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>>>> or not its envelope recipients were valid before it will
>>>>>>>>>>>> send us the message.
>>>>>>>>>>>> This means users need to be validated before read_header(),
>>>>>>>>>>>> which is a
>>>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> problem
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>>>> because at the moment this vaildation happens in
>>>>>>>>>>>> insert_messages().
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> OK, that's clear
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Fine and dandy in pipe land, where the MTA will send the
>>>>>>>>>>>> message
>>>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> regardless of
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>>>> the disposition of the recipients, and only at the very end
>>>>>>>>>>>> are error
>>>>>>>>>>>> conditions returned as exit codes... in LMTP land we need to
>>>>>>>>>>>> know ahead of
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> time.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>> I think what I'll do is move the user validation code from
>>>>>>>>>>>> insert_messages()
>>>>>>>>>>>> into dsn.c, perhaps calling it dsn_check_users() or
>>>>>>>>>>>> dsn_prepare_deliveries().
>>>>>>>>>>>> Then, we'll have this order:
>>>>>>>>>>>>
>>>>>>>>>>>> For command-line and envelope-recipient deliveries:
>>>>>>>>>>>> - receive addresses and store them into the dsnusers list.
>>>>>>>>>>>> - dsn_prepare_deliveries()
>>>>>>>>>>>> - if no deliveries are valid, return an error.
>>>>>>>>>>>>
>>>>>>>>>>>> - read_header()
>>>>>>>>>>>>
>>>>>>>>>>>> For deliver-to header deliveries:
>>>>>>>>>>>> - mime_readheader()
>>>>>>>>>>>> - mail_adr_list()
>>>>>>>>>>>> - dsn_prepare_deliveries()
>>>>>>>>>>>> - if no deliveries are valid, return an error.
>>>>>>>>>>>>
>>>>>>>>>>>> - insert_messages()
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I like the part where you say 'What I'll do' :)
>>>>>>>>>>> Do you think this work will take long? I think we must really
>>>>>>>>>>> release rc3 tomorrow (Friday, March 5th). This stuff really
>>>>>>>>>>> has to work if we want to release.
>>>>>>>>>>>
>>>>>>>>>>> Ilja
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Dbmail-dev mailing list
>>>>>>>>>>> Dbmail-dev@dbmail.org
>>>>>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Dbmail-dev mailing list
>>>>>>>>> Dbmail-dev@dbmail.org
>>>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Dbmail-dev mailing list
>>>>>>> Dbmail-dev@dbmail.org
>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Dbmail-dev mailing list
>>>>>> Dbmail-dev@dbmail.org
>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Dbmail-dev mailing list
>>>>> Dbmail-dev@dbmail.org
>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Dbmail-dev mailing list
>>> Dbmail-dev@dbmail.org
>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>
>>
>> _______________________________________________
>> Dbmail-dev mailing list
>> Dbmail-dev@dbmail.org
>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev

Hi,

Removing the
"250 Message received OK" after having received the message takes care
of the error. I can just send several messages, with
"lmtp_cache_connection = yes" in /etc/postfix/main.cf

I'm starting to wonder whether LMTP requires us to send a message if the
message was received OK by the server.

Ilja

Ilja Booij wrote:

> Hi,
>
> with some help from a guy on postfix-users@postfix.org I found something:
>
> The "250 Recipient <target@test01.office.fastxs.net> OK" message from
> dbmail-lmtp is out of sync. The postfix/lmtp program stores this message
> until the next message is sent, causing the messages between postfix and
> dbmail-smtp to be horribly out-of-sync.
>
> I guess this should be quite easy to fix. I'll see if I have some time
> to do it this weekend. Otherwise I'll do it on Monday. Unless anybody
> else wants to do it of course ;)
>
> I'll be off in a few minutes. First a beer, then it's off to home :)
>
> Have a nice weekend!
>
> Ilja
>
>
> Ilja Booij wrote:
>
>> Hmm, I still don't really understand the problem. I've done some
>> ngrep-ing, to see what goes over the network.
>>
>> This is a session that's OK:
>>
>> T 2004/03/05 13:45:46.324232 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>> 220 test01 DBMail LMTP service ready to rock..
>> ##
>> T 2004/03/05 13:45:46.324696 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>> LHLO test01.office.fastxs.net..
>> ##
>> T 2004/03/05 13:45:46.324900 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>> 250-test01..250-PIPELINING..250-ENHANCEDSTATUSCODES..250 SIZE..
>> #
>> T 2004/03/05 13:45:46.325306 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>> MAIL FROM:<ilja@earthquake.office.fastxs.net> SIZE=866..RCPT
>> TO:<target@test01.office.fastxs.net>..DATA..
>> #
>> T 2004/03/05 13:45:46.325479 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>> 250 Sender <ilja@earthquake.office.fastxs.net> OK..
>> ##
>> T 2004/03/05 13:45:46.365648 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>> 250 Recipient <target@test01.office.fastxs.net> OK..354 Start mail
>> input; end with <CRLF>.<CRLF>..
>> ##
>> T 2004/03/05 13:45:46.365974 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>> Received: from earthquake.office.fastxs.net
>> (earthquake.office.fastxs.net [10.0.1.15])...by test01.office.fastxs.n
>> et (Postfix) with ESMTP id 394DD19F30A...for
>> <target@test01.office.fastxs.net>; Fri, 5 Mar 2004 13:45:46 +0100 (C
>> ET)..Received: from earthquake.office.fastxs.net (localhost
>> [127.0.0.1])...by earthquake.office.fastxs.net (Postfi
>> x) with ESMTP id 405D4375FE...for <target@test01.office.fastxs.net>;
>> Fri, 5 Mar 2004 13:49:12 +0100 (CET)..Receiv
>> ed: (from ilja@localhost)...by earthquake.office.fastxs.net
>> (8.12.9/8.12.9/Submit) id i25CnCo0005116...for target@
>> test01.office.fastxs.net; Fri, 5 Mar 2004 13:49:12 +0100
>> (CET)..Date: Fri, 5 Mar 2004 13:49:12 +0100 (CET)..From:
>> Ilja Booij <ilja@earthquake.office.fastxs.net>..Message-Id:
>> <200403051249.i25CnCo0005116@earthquake.office.fastxs.
>> net>..To: target@test01.office.fastxs.net..Subject: 5-3 27......27.....
>> ##
>> T 2004/03/05 13:45:46.568851 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>> 250 Message received OK..
>> ##
>> T 2004/03/05 13:45:46.608610 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>> c..
>>
>>
>>
>> If another message is sent (over the same connection):
>>
>>
>> ##
>> T 2004/03/05 13:46:01.879964 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>> RSET..
>> ##
>> T 2004/03/05 13:46:01.880127 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>> MAIL FROM:<ilja@earthquake.office.fastxs.net> SIZE=858..RCPT
>> TO:<ilja@test01.office.fastxs.net>..DATA..
>> ##
>> T 2004/03/05 13:46:01.880368 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>> 250 OK..
>> ##
>> T 2004/03/05 13:46:01.880460 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>> 250 Sender <ilja@earthquake.office.fastxs.net> OK..
>> ##
>> T 2004/03/05 13:46:01.883495 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>> 250 Recipient <ilja@test01.office.fastxs.net> OK..
>> ##
>> T 2004/03/05 13:46:01.883541 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>> 354 Start mail input; end with <CRLF>.<CRLF>..
>> ##
>> T 2004/03/05 13:47:41.920055 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>> RSET..QUIT..
>> ###
>> T 2004/03/05 13:52:41.875033 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>> 500 Error reading header...
>> #
>>
>> You can see that the client (postfix) starts with an RSET command, and
>> starts sending MAIL_FROM, RCPT and DATA.
>> The server responds with 250 OK to the reset, and 250 Sender OK and
>> 250 Recipient OK, and lets the client start sending (354 Start mail
>> input).
>>
>> The problem is that Postfix detects that it should not send data,
>> because the message is supposedly bounced by dbmail.. But why?
>>
>> Ilja
>>
>> Ilja Booij wrote:
>>
>>> Hi,
>>>
>>> Robert, thanks for your suggestion. It seems to work perfectly here.
>>> Messages now get delivered to users without a problem.
>>>
>>> I'll take a look at lmtp.c to see if I can spot the difference
>>> between the server getting a QUIT from the client and a reconnection
>>> on a new message, and a cached connection.
>>>
>>> Ilja
>>>
>>>
>>> Robert Theisen wrote:
>>>
>>>> New to the list here. But I read over the archives regarding this
>>>> problem and I may have a little to add.
>>>>
>>>> I was able to get postfix to consistently deliver via lmtp to dbmail
>>>> (even two consecutive messages to the same user) by setting the
>>>> postfix directive lmtp_cache_connection to false instead of true.
>>>> This would termintate the lmtp connection after each delivery. I
>>>> assume, with lmtp_cache_connection turned on, the connection would
>>>> die after lmtp_timeout was reached and a new connection would be
>>>> established and the mail would get sent. This may have been a "neat
>>>> feature" that works with Cyrus (postfix's preferred lmtp client) or
>>>> it may be a specification of the lmtp rfc (I have not read it). But
>>>> it seems like a simple enough fix to make it work. It would
>>>> probably just involve having the conversation status fall back to a
>>>> state the postfix expects after message delivery and waiting in that
>>>> state until either the connection terminates or the conversation
>>>> picks up again. I took a cursory glance at the lmtp.c and lmtpd.c
>>>> code but did not have enough time to really dig into it.
>>>> Good Luck. I am anxiously awaiting the 2.0 release. :)
>>>>
>>>> Robert Theisen
>>>>
>>>> Aaron Stone wrote:
>>>>
>>>>> Sure thing, I've got about an hour of time this morning. Looks like
>>>>> you're
>>>>> burning the midnight oil over there! Yikes!
>>>>>
>>>>> Aaron
>>>>>
>>>>>
>>>>> Ilja Booij <ilja@ic-s.nl> said:
>>>>>
>>>>>
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I haven't been able to find the cause of the problem yet. I found
>>>>>> some more info in the logs though:
>>>>>>
>>>>>> Mar 4 16:55:34 test01 postfix/lmtp[21340]: 3378119F30A:
>>>>>> to=<target@test01.office.fastxs.net>,
>>>>>> relay=localhost.fastxs.net[127.0.0.1], delay=0, status=bounced
>>>>>> (host localhost.fastxs.net[127.0.0.1] said: 250 Recipient
>>>>>> <target@test01.office.fastxs.net> OK)
>>>>>>
>>>>>> apparently, postfix thinks we want to bounce the message. But why?
>>>>>> A previous message to the same user arrives correctly..
>>>>>>
>>>>>> strange, looking further. Aaron, can you also take a look at this?
>>>>>>
>>>>>> Ilja
>>>>>>
>>>>>> Ilja Booij wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> OK, I've found something:
>>>>>>>
>>>>>>> It seems that Postfix sends a RSET command, when we expect to get
>>>>>>> the header.
>>>>>>>
>>>>>>> readheader() then waits for postfix to send more, while postfix
>>>>>>> waits for a '250 OK' Message from dbmail-lmtp.
>>>>>>>
>>>>>>> So, there's probably something wrong in our LMTP-logic.
>>>>>>> I'll take a look at it, after I've done some small scripting job
>>>>>>> here for a customer.
>>>>>>>
>>>>>>> Ilja
>>>>>>>
>>>>>>> Ilja Booij wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> OK,
>>>>>>>>
>>>>>>>> this seems to have tackled the problem of the double delivery :)
>>>>>>>>
>>>>>>>> There still is another problem though:
>>>>>>>> It still seems to hang for a while on every second delivery
>>>>>>>> attempt to the LMTP daemon.
>>>>>>>>
>>>>>>>> Can you try the following scenario:
>>>>>>>>
>>>>>>>> * configure MTA to use dbmail-lmtp for delivery
>>>>>>>> * send message to a recipient
>>>>>>>> * verify that the message is received using dbmail-lmtp
>>>>>>>> * send a second message.
>>>>>>>>
>>>>>>>> What I observe here is that the second message takes a lot
>>>>>>>> longer to be delivered than the first one. The second one is
>>>>>>>> only delivered after minutes.
>>>>>>>>
>>>>>>>> I have the feeling that something is not right here..
>>>>>>>>
>>>>>>>> Ilja
>>>>>>>>
>>>>>>>> Aaron Stone wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> New versions checked in, though not extensively tested yet.
>>>>>>>>>
>>>>>>>>> Ilja Booij <ilja@ic-s.nl> said:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> sounds ok to me :)
>>>>>>>>>>
>>>>>>>>>> Ilja
>>>>>>>>>>
>>>>>>>>>> Aaron Stone wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Working on it as we speak! Catch you in about an hour?
>>>>>>>>>>>
>>>>>>>>>>> Aaron
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Ilja Booij <ilja@ic-s.nl> said:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> Aaron Stone wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> The reason for the double deliveries in the LMTP daemon is
>>>>>>>>>>>>> because the old
>>>>>>>>>>>>> insert_messages() function used to take lists of users to
>>>>>>>>>>>>> directly deliver.
>>>>>>>>>>>>> The new insert_messages() function takes a list of dsnuser
>>>>>>>>>>>>> structures, and
>>>>>>>>>>>>> expects that *either* the useridnr field is filled (for
>>>>>>>>>>>>> direct-to-useridnr
>>>>>>>>>>>>> delivery, which isn't supported by any of the frontends,
>>>>>>>>>>>>> but is supported in
>>>>>>>>>>>>> the lower layers ;-) or the address field, which is first
>>>>>>>>>>>>> checked as a
>>>>>>>>>>>>> username and then as an alias (this allows usernames in the
>>>>>>>>>>>>> form of
>>>>>>>>>>>>> 'user@server' to operate without requiring an alias that
>>>>>>>>>>>>> says 'user@server
>>>>>>>>>>>>> delivers to user@server').
>>>>>>>>>>>>>
>>>>>>>>>>>>> Some refactoring might be necessary, because we need to
>>>>>>>>>>>>> inform the MTA
>>>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> whether
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>>> or not its envelope recipients were valid before it will
>>>>>>>>>>>>> send us the message.
>>>>>>>>>>>>> This means users need to be validated before read_header(),
>>>>>>>>>>>>> which is a
>>>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> problem
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>>> because at the moment this vaildation happens in
>>>>>>>>>>>>> insert_messages().
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> OK, that's clear
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> Fine and dandy in pipe land, where the MTA will send the
>>>>>>>>>>>>> message
>>>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> regardless of
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>>> the disposition of the recipients, and only at the very end
>>>>>>>>>>>>> are error
>>>>>>>>>>>>> conditions returned as exit codes... in LMTP land we need
>>>>>>>>>>>>> to know ahead of
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> time.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>> I think what I'll do is move the user validation code from
>>>>>>>>>>>>> insert_messages()
>>>>>>>>>>>>> into dsn.c, perhaps calling it dsn_check_users() or
>>>>>>>>>>>>> dsn_prepare_deliveries().
>>>>>>>>>>>>> Then, we'll have this order:
>>>>>>>>>>>>>
>>>>>>>>>>>>> For command-line and envelope-recipient deliveries:
>>>>>>>>>>>>> - receive addresses and store them into the dsnusers list.
>>>>>>>>>>>>> - dsn_prepare_deliveries()
>>>>>>>>>>>>> - if no deliveries are valid, return an error.
>>>>>>>>>>>>>
>>>>>>>>>>>>> - read_header()
>>>>>>>>>>>>>
>>>>>>>>>>>>> For deliver-to header deliveries:
>>>>>>>>>>>>> - mime_readheader()
>>>>>>>>>>>>> - mail_adr_list()
>>>>>>>>>>>>> - dsn_prepare_deliveries()
>>>>>>>>>>>>> - if no deliveries are valid, return an error.
>>>>>>>>>>>>>
>>>>>>>>>>>>> - insert_messages()
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I like the part where you say 'What I'll do' :)
>>>>>>>>>>>> Do you think this work will take long? I think we must
>>>>>>>>>>>> really release rc3 tomorrow (Friday, March 5th). This stuff
>>>>>>>>>>>> really has to work if we want to release.
>>>>>>>>>>>>
>>>>>>>>>>>> Ilja
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Dbmail-dev mailing list
>>>>>>>>>>>> Dbmail-dev@dbmail.org
>>>>>>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Dbmail-dev mailing list
>>>>>>>>>> Dbmail-dev@dbmail.org
>>>>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Dbmail-dev mailing list
>>>>>>>> Dbmail-dev@dbmail.org
>>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Dbmail-dev mailing list
>>>>>>> Dbmail-dev@dbmail.org
>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Dbmail-dev mailing list
>>>>>> Dbmail-dev@dbmail.org
>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Dbmail-dev mailing list
>>>> Dbmail-dev@dbmail.org
>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> Dbmail-dev mailing list
>>> Dbmail-dev@dbmail.org
>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>
>>
>> _______________________________________________
>> Dbmail-dev mailing list
>> Dbmail-dev@dbmail.org
>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev

After some more reading of RFC 2033 I'm inclined to believe an LMTP
client does not get a response after sending the DATA, except for the
responses about the delivery of the message to the users.

Example session from the RFC:

S: 220 foo.edu LMTP server ready
C: LHLO foo.edu
S: 250-foo.edu
S: 250-PIPELINING
S: 250 SIZE
C: MAIL FROM:<chris@bar.com>
S: 250 OK

C: RCPT TO:<pat@foo.edu>
S: 250 OK
C: RCPT TO:<jones@foo.edu>
S: 550 No such user here
C: RCPT TO:<green@foo.edu>
S: 250 OK
C: DATA
S: 354 Start mail input; end with <CRLF>.<CRLF>
C: Blah blah blah...
C: ...etc. etc. etc.
C: .
S: 250 OK
S: 452 <green@foo.edu> is temporarily over quota
C: QUIT
S: 221 foo.edu closing connection

The important bit is after the DATA line
there are two responses from the server after the data line:
1. "250 OK": the message to chris@bar.com was delivered
2. "452 <green@foo.edu> is temporarily over quota", the message to
green@foo.edu was not delivered.

There is no line indicating that the message was received by the server.
The server should only indicate when a message was not received correctly.

I'll complete remove the
fprintf((FILE *)stream, "250 Message received OK\r\n"); line


Ilja Booij wrote:

> Hi,
>
> Removing the
> "250 Message received OK" after having received the message takes care
> of the error. I can just send several messages, with
> "lmtp_cache_connection = yes" in /etc/postfix/main.cf
>
> I'm starting to wonder whether LMTP requires us to send a message if the
> message was received OK by the server.
>
> Ilja
>
> Ilja Booij wrote:
>
>> Hi,
>>
>> with some help from a guy on postfix-users@postfix.org I found something:
>>
>> The "250 Recipient <target@test01.office.fastxs.net> OK" message from
>> dbmail-lmtp is out of sync. The postfix/lmtp program stores this
>> message until the next message is sent, causing the messages between
>> postfix and dbmail-smtp to be horribly out-of-sync.
>>
>> I guess this should be quite easy to fix. I'll see if I have some time
>> to do it this weekend. Otherwise I'll do it on Monday. Unless anybody
>> else wants to do it of course ;)
>>
>> I'll be off in a few minutes. First a beer, then it's off to home :)
>>
>> Have a nice weekend!
>>
>> Ilja
>>
>>
>> Ilja Booij wrote:
>>
>>> Hmm, I still don't really understand the problem. I've done some
>>> ngrep-ing, to see what goes over the network.
>>>
>>> This is a session that's OK:
>>>
>>> T 2004/03/05 13:45:46.324232 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>> 220 test01 DBMail LMTP service ready to rock..
>>> ##
>>> T 2004/03/05 13:45:46.324696 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>>> LHLO test01.office.fastxs.net..
>>> ##
>>> T 2004/03/05 13:45:46.324900 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>> 250-test01..250-PIPELINING..250-ENHANCEDSTATUSCODES..250 SIZE..
>>> #
>>> T 2004/03/05 13:45:46.325306 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>>> MAIL FROM:<ilja@earthquake.office.fastxs.net> SIZE=866..RCPT
>>> TO:<target@test01.office.fastxs.net>..DATA..
>>> #
>>> T 2004/03/05 13:45:46.325479 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>> 250 Sender <ilja@earthquake.office.fastxs.net> OK..
>>> ##
>>> T 2004/03/05 13:45:46.365648 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>> 250 Recipient <target@test01.office.fastxs.net> OK..354 Start mail
>>> input; end with <CRLF>.<CRLF>..
>>> ##
>>> T 2004/03/05 13:45:46.365974 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>>> Received: from earthquake.office.fastxs.net
>>> (earthquake.office.fastxs.net [10.0.1.15])...by test01.office.fastxs.n
>>> et (Postfix) with ESMTP id 394DD19F30A...for
>>> <target@test01.office.fastxs.net>; Fri, 5 Mar 2004 13:45:46 +0100 (C
>>> ET)..Received: from earthquake.office.fastxs.net (localhost
>>> [127.0.0.1])...by earthquake.office.fastxs.net (Postfi
>>> x) with ESMTP id 405D4375FE...for
>>> <target@test01.office.fastxs.net>; Fri, 5 Mar 2004 13:49:12 +0100
>>> (CET)..Receiv
>>> ed: (from ilja@localhost)...by earthquake.office.fastxs.net
>>> (8.12.9/8.12.9/Submit) id i25CnCo0005116...for target@
>>> test01.office.fastxs.net; Fri, 5 Mar 2004 13:49:12 +0100
>>> (CET)..Date: Fri, 5 Mar 2004 13:49:12 +0100 (CET)..From:
>>> Ilja Booij <ilja@earthquake.office.fastxs.net>..Message-Id:
>>> <200403051249.i25CnCo0005116@earthquake.office.fastxs.
>>> net>..To: target@test01.office.fastxs.net..Subject: 5-3
>>> 27......27.....
>>> ##
>>> T 2004/03/05 13:45:46.568851 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>> 250 Message received OK..
>>> ##
>>> T 2004/03/05 13:45:46.608610 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>> c..
>>>
>>>
>>>
>>> If another message is sent (over the same connection):
>>>
>>>
>>> ##
>>> T 2004/03/05 13:46:01.879964 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>>> RSET..
>>> ##
>>> T 2004/03/05 13:46:01.880127 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>>> MAIL FROM:<ilja@earthquake.office.fastxs.net> SIZE=858..RCPT
>>> TO:<ilja@test01.office.fastxs.net>..DATA..
>>> ##
>>> T 2004/03/05 13:46:01.880368 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>> 250 OK..
>>> ##
>>> T 2004/03/05 13:46:01.880460 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>> 250 Sender <ilja@earthquake.office.fastxs.net> OK..
>>> ##
>>> T 2004/03/05 13:46:01.883495 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>> 250 Recipient <ilja@test01.office.fastxs.net> OK..
>>> ##
>>> T 2004/03/05 13:46:01.883541 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>> 354 Start mail input; end with <CRLF>.<CRLF>..
>>> ##
>>> T 2004/03/05 13:47:41.920055 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>>> RSET..QUIT..
>>> ###
>>> T 2004/03/05 13:52:41.875033 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>> 500 Error reading header...
>>> #
>>>
>>> You can see that the client (postfix) starts with an RSET command,
>>> and starts sending MAIL_FROM, RCPT and DATA.
>>> The server responds with 250 OK to the reset, and 250 Sender OK and
>>> 250 Recipient OK, and lets the client start sending (354 Start mail
>>> input).
>>>
>>> The problem is that Postfix detects that it should not send data,
>>> because the message is supposedly bounced by dbmail.. But why?
>>>
>>> Ilja
>>>
>>> Ilja Booij wrote:
>>>
>>>> Hi,
>>>>
>>>> Robert, thanks for your suggestion. It seems to work perfectly here.
>>>> Messages now get delivered to users without a problem.
>>>>
>>>> I'll take a look at lmtp.c to see if I can spot the difference
>>>> between the server getting a QUIT from the client and a reconnection
>>>> on a new message, and a cached connection.
>>>>
>>>> Ilja
>>>>
>>>>
>>>> Robert Theisen wrote:
>>>>
>>>>> New to the list here. But I read over the archives regarding this
>>>>> problem and I may have a little to add.
>>>>>
>>>>> I was able to get postfix to consistently deliver via lmtp to
>>>>> dbmail (even two consecutive messages to the same user) by setting
>>>>> the postfix directive lmtp_cache_connection to false instead of
>>>>> true. This would termintate the lmtp connection after each
>>>>> delivery. I assume, with lmtp_cache_connection turned on, the
>>>>> connection would die after lmtp_timeout was reached and a new
>>>>> connection would be established and the mail would get sent. This
>>>>> may have been a "neat feature" that works with Cyrus (postfix's
>>>>> preferred lmtp client) or it may be a specification of the lmtp rfc
>>>>> (I have not read it). But it seems like a simple enough fix to
>>>>> make it work. It would probably just involve having the
>>>>> conversation status fall back to a state the postfix expects after
>>>>> message delivery and waiting in that state until either the
>>>>> connection terminates or the conversation picks up again. I took
>>>>> a cursory glance at the lmtp.c and lmtpd.c code but did not have
>>>>> enough time to really dig into it.
>>>>> Good Luck. I am anxiously awaiting the 2.0 release. :)
>>>>>
>>>>> Robert Theisen
>>>>>
>>>>> Aaron Stone wrote:
>>>>>
>>>>>> Sure thing, I've got about an hour of time this morning. Looks
>>>>>> like you're
>>>>>> burning the midnight oil over there! Yikes!
>>>>>>
>>>>>> Aaron
>>>>>>
>>>>>>
>>>>>> Ilja Booij <ilja@ic-s.nl> said:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I haven't been able to find the cause of the problem yet. I found
>>>>>>> some more info in the logs though:
>>>>>>>
>>>>>>> Mar 4 16:55:34 test01 postfix/lmtp[21340]: 3378119F30A:
>>>>>>> to=<target@test01.office.fastxs.net>,
>>>>>>> relay=localhost.fastxs.net[127.0.0.1], delay=0, status=bounced
>>>>>>> (host localhost.fastxs.net[127.0.0.1] said: 250 Recipient
>>>>>>> <target@test01.office.fastxs.net> OK)
>>>>>>>
>>>>>>> apparently, postfix thinks we want to bounce the message. But
>>>>>>> why? A previous message to the same user arrives correctly..
>>>>>>>
>>>>>>> strange, looking further. Aaron, can you also take a look at this?
>>>>>>>
>>>>>>> Ilja
>>>>>>>
>>>>>>> Ilja Booij wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> OK, I've found something:
>>>>>>>>
>>>>>>>> It seems that Postfix sends a RSET command, when we expect to
>>>>>>>> get the header.
>>>>>>>>
>>>>>>>> readheader() then waits for postfix to send more, while postfix
>>>>>>>> waits for a '250 OK' Message from dbmail-lmtp.
>>>>>>>>
>>>>>>>> So, there's probably something wrong in our LMTP-logic.
>>>>>>>> I'll take a look at it, after I've done some small scripting job
>>>>>>>> here for a customer.
>>>>>>>>
>>>>>>>> Ilja
>>>>>>>>
>>>>>>>> Ilja Booij wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> OK,
>>>>>>>>>
>>>>>>>>> this seems to have tackled the problem of the double delivery :)
>>>>>>>>>
>>>>>>>>> There still is another problem though:
>>>>>>>>> It still seems to hang for a while on every second delivery
>>>>>>>>> attempt to the LMTP daemon.
>>>>>>>>>
>>>>>>>>> Can you try the following scenario:
>>>>>>>>>
>>>>>>>>> * configure MTA to use dbmail-lmtp for delivery
>>>>>>>>> * send message to a recipient
>>>>>>>>> * verify that the message is received using dbmail-lmtp
>>>>>>>>> * send a second message.
>>>>>>>>>
>>>>>>>>> What I observe here is that the second message takes a lot
>>>>>>>>> longer to be delivered than the first one. The second one is
>>>>>>>>> only delivered after minutes.
>>>>>>>>>
>>>>>>>>> I have the feeling that something is not right here..
>>>>>>>>>
>>>>>>>>> Ilja
>>>>>>>>>
>>>>>>>>> Aaron Stone wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> New versions checked in, though not extensively tested yet.
>>>>>>>>>>
>>>>>>>>>> Ilja Booij <ilja@ic-s.nl> said:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> sounds ok to me :)
>>>>>>>>>>>
>>>>>>>>>>> Ilja
>>>>>>>>>>>
>>>>>>>>>>> Aaron Stone wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Working on it as we speak! Catch you in about an hour?
>>>>>>>>>>>>
>>>>>>>>>>>> Aaron
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Ilja Booij <ilja@ic-s.nl> said:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Aaron Stone wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> The reason for the double deliveries in the LMTP daemon is
>>>>>>>>>>>>>> because the old
>>>>>>>>>>>>>> insert_messages() function used to take lists of users to
>>>>>>>>>>>>>> directly deliver.
>>>>>>>>>>>>>> The new insert_messages() function takes a list of dsnuser
>>>>>>>>>>>>>> structures, and
>>>>>>>>>>>>>> expects that *either* the useridnr field is filled (for
>>>>>>>>>>>>>> direct-to-useridnr
>>>>>>>>>>>>>> delivery, which isn't supported by any of the frontends,
>>>>>>>>>>>>>> but is supported in
>>>>>>>>>>>>>> the lower layers ;-) or the address field, which is first
>>>>>>>>>>>>>> checked as a
>>>>>>>>>>>>>> username and then as an alias (this allows usernames in
>>>>>>>>>>>>>> the form of
>>>>>>>>>>>>>> 'user@server' to operate without requiring an alias that
>>>>>>>>>>>>>> says 'user@server
>>>>>>>>>>>>>> delivers to user@server').
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Some refactoring might be necessary, because we need to
>>>>>>>>>>>>>> inform the MTA
>>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> whether
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>>>> or not its envelope recipients were valid before it will
>>>>>>>>>>>>>> send us the message.
>>>>>>>>>>>>>> This means users need to be validated before
>>>>>>>>>>>>>> read_header(), which is a
>>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> problem
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>>>> because at the moment this vaildation happens in
>>>>>>>>>>>>>> insert_messages().
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> OK, that's clear
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Fine and dandy in pipe land, where the MTA will send the
>>>>>>>>>>>>>> message
>>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> regardless of
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>>>> the disposition of the recipients, and only at the very
>>>>>>>>>>>>>> end are error
>>>>>>>>>>>>>> conditions returned as exit codes... in LMTP land we need
>>>>>>>>>>>>>> to know ahead of
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> time.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>> I think what I'll do is move the user validation code from
>>>>>>>>>>>>>> insert_messages()
>>>>>>>>>>>>>> into dsn.c, perhaps calling it dsn_check_users() or
>>>>>>>>>>>>>> dsn_prepare_deliveries().
>>>>>>>>>>>>>> Then, we'll have this order:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> For command-line and envelope-recipient deliveries:
>>>>>>>>>>>>>> - receive addresses and store them into the dsnusers list.
>>>>>>>>>>>>>> - dsn_prepare_deliveries()
>>>>>>>>>>>>>> - if no deliveries are valid, return an error.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - read_header()
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> For deliver-to header deliveries:
>>>>>>>>>>>>>> - mime_readheader()
>>>>>>>>>>>>>> - mail_adr_list()
>>>>>>>>>>>>>> - dsn_prepare_deliveries()
>>>>>>>>>>>>>> - if no deliveries are valid, return an error.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - insert_messages()
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I like the part where you say 'What I'll do' :)
>>>>>>>>>>>>> Do you think this work will take long? I think we must
>>>>>>>>>>>>> really release rc3 tomorrow (Friday, March 5th). This stuff
>>>>>>>>>>>>> really has to work if we want to release.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ilja
>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> Dbmail-dev mailing list
>>>>>>>>>>>>> Dbmail-dev@dbmail.org
>>>>>>>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Dbmail-dev mailing list
>>>>>>>>>>> Dbmail-dev@dbmail.org
>>>>>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Dbmail-dev mailing list
>>>>>>>>> Dbmail-dev@dbmail.org
>>>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Dbmail-dev mailing list
>>>>>>>> Dbmail-dev@dbmail.org
>>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Dbmail-dev mailing list
>>>>>>> Dbmail-dev@dbmail.org
>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Dbmail-dev mailing list
>>>>> Dbmail-dev@dbmail.org
>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Dbmail-dev mailing list
>>>> Dbmail-dev@dbmail.org
>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> Dbmail-dev mailing list
>>> Dbmail-dev@dbmail.org
>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>
>>
>> _______________________________________________
>> Dbmail-dev mailing list
>> Dbmail-dev@dbmail.org
>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev

In this example, there are three recipients, pat, jones and green. Following
the data command, there are only two acknowledgements. This is contrary to my
understanding that each recipient had to be acknowledged:

The LMTP protocol causes the server to return, after the final "." of
the DATA command, one reply for each recipient. Therefore, if the
queue manager is configured to use LMTP instead of SMTP when
transferring messages to the delivery agents, then the delivery
agents may attempt delivery to each recipient after the final "." and
individually report the status for each recipient. Connections which
should use the LMTP protocol are drawn in the diagram above using
asterisks.

So looking at jones, it is clear why: jones failed the RCPT command, and so
was no longer in the recipient list when the DATA command came around. Then
read this:

The additional restriction is that when there have been no successful
RCPT commands in the mail transaction, the DATA command MUST fail
with a 503 reply code.

The change is that after the final ".", the server returns one reply
for each previously successful RCPT command in the mail transaction,
in the order that the RCPT commands were issued. Even if there were
multiple successful RCPT commands giving the same forward-path, there
must be one reply for each successful RCPT command.

I think that what the first paragraph in this quote means is not that DATA is
failed, per se, but that the whole conversation is flagged with a 503.

So here's what I think needs to be fixed:

- If there are no recipients, use 503 AND NOTHING ELSE to report it,
which means changing this section of code to use a 503:

if (!has_recipients)
{
trace(TRACE_DEBUG,"main(): no valid recipients found, cancel message.");
fprintf((FILE *)stream, "554 No valid recipients.\r\n" );
}

- When a recipient is failed, report the error and then *remove that
recipient from the dsnusers list* so that after the message is
received, their failure is not reported a second time.

I'll get on it this afternoon, please test it in the morning, your time!

Aaron


Ilja Booij <ilja@ic-s.nl> said:

> After some more reading of RFC 2033 I'm inclined to believe an LMTP
> client does not get a response after sending the DATA, except for the
> responses about the delivery of the message to the users.
>
> Example session from the RFC:
>
> S: 220 foo.edu LMTP server ready
> C: LHLO foo.edu
> S: 250-foo.edu
> S: 250-PIPELINING
> S: 250 SIZE
> C: MAIL FROM:<chris@bar.com>
> S: 250 OK
>
> C: RCPT TO:<pat@foo.edu>
> S: 250 OK
> C: RCPT TO:<jones@foo.edu>
> S: 550 No such user here
> C: RCPT TO:<green@foo.edu>
> S: 250 OK
> C: DATA
> S: 354 Start mail input; end with <CRLF>.<CRLF>
> C: Blah blah blah...
> C: ...etc. etc. etc.
> C: .
> S: 250 OK
> S: 452 <green@foo.edu> is temporarily over quota
> C: QUIT
> S: 221 foo.edu closing connection
>
> The important bit is after the DATA line
> there are two responses from the server after the data line:
> 1. "250 OK": the message to chris@bar.com was delivered
> 2. "452 <green@foo.edu> is temporarily over quota", the message to
> green@foo.edu was not delivered.
>
> There is no line indicating that the message was received by the server.
> The server should only indicate when a message was not received correctly.
>
> I'll complete remove the
> fprintf((FILE *)stream, "250 Message received OK\r\n"); line
>
>
> Ilja Booij wrote:
>
> > Hi,
> >
> > Removing the
> > "250 Message received OK" after having received the message takes care
> > of the error. I can just send several messages, with
> > "lmtp_cache_connection = yes" in /etc/postfix/main.cf
> >
> > I'm starting to wonder whether LMTP requires us to send a message if the
> > message was received OK by the server.
> >
> > Ilja
> >
> > Ilja Booij wrote:
> >
> >> Hi,
> >>
> >> with some help from a guy on postfix-users@postfix.org I found something:
> >>
> >> The "250 Recipient <target@test01.office.fastxs.net> OK" message from
> >> dbmail-lmtp is out of sync. The postfix/lmtp program stores this
> >> message until the next message is sent, causing the messages between
> >> postfix and dbmail-smtp to be horribly out-of-sync.
> >>
> >> I guess this should be quite easy to fix. I'll see if I have some time
> >> to do it this weekend. Otherwise I'll do it on Monday. Unless anybody
> >> else wants to do it of course ;)
> >>
> >> I'll be off in a few minutes. First a beer, then it's off to home :)
> >>
> >> Have a nice weekend!
> >>
> >> Ilja
> >>
> >>
> >> Ilja Booij wrote:
> >>
> >>> Hmm, I still don't really understand the problem. I've done some
> >>> ngrep-ing, to see what goes over the network.
> >>>
> >>> This is a session that's OK:
> >>>
> >>> T 2004/03/05 13:45:46.324232 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> >>> 220 test01 DBMail LMTP service ready to rock..
> >>> ##
> >>> T 2004/03/05 13:45:46.324696 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> >>> LHLO test01.office.fastxs.net..
> >>> ##
> >>> T 2004/03/05 13:45:46.324900 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> >>> 250-test01..250-PIPELINING..250-ENHANCEDSTATUSCODES..250 SIZE..
> >>> #
> >>> T 2004/03/05 13:45:46.325306 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> >>> MAIL FROM:<ilja@earthquake.office.fastxs.net> SIZE=866..RCPT
> >>> TO:<target@test01.office.fastxs.net>..DATA..
> >>> #
> >>> T 2004/03/05 13:45:46.325479 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> >>> 250 Sender <ilja@earthquake.office.fastxs.net> OK..
> >>> ##
> >>> T 2004/03/05 13:45:46.365648 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> >>> 250 Recipient <target@test01.office.fastxs.net> OK..354 Start mail
> >>> input; end with <CRLF>.<CRLF>..
> >>> ##
> >>> T 2004/03/05 13:45:46.365974 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> >>> Received: from earthquake.office.fastxs.net
> >>> (earthquake.office.fastxs.net [10.0.1.15])...by test01.office.fastxs.n
> >>> et (Postfix) with ESMTP id 394DD19F30A...for
> >>> <target@test01.office.fastxs.net>; Fri, 5 Mar 2004 13:45:46 +0100 (C
> >>> ET)..Received: from earthquake.office.fastxs.net (localhost
> >>> [127.0.0.1])...by earthquake.office.fastxs.net (Postfi
> >>> x) with ESMTP id 405D4375FE...for
> >>> <target@test01.office.fastxs.net>; Fri, 5 Mar 2004 13:49:12 +0100
> >>> (CET)..Receiv
> >>> ed: (from ilja@localhost)...by earthquake.office.fastxs.net
> >>> (8.12.9/8.12.9/Submit) id i25CnCo0005116...for target@
> >>> test01.office.fastxs.net; Fri, 5 Mar 2004 13:49:12 +0100
> >>> (CET)..Date: Fri, 5 Mar 2004 13:49:12 +0100 (CET)..From:
> >>> Ilja Booij <ilja@earthquake.office.fastxs.net>..Message-Id:
> >>> <200403051249.i25CnCo0005116@earthquake.office.fastxs.
> >>> net>..To: target@test01.office.fastxs.net..Subject: 5-3
> >>> 27......27.....
> >>> ##
> >>> T 2004/03/05 13:45:46.568851 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> >>> 250 Message received OK..
> >>> ##
> >>> T 2004/03/05 13:45:46.608610 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> >>> c..
> >>>
> >>>
> >>>
> >>> If another message is sent (over the same connection):
> >>>
> >>>
> >>> ##
> >>> T 2004/03/05 13:46:01.879964 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> >>> RSET..
> >>> ##
> >>> T 2004/03/05 13:46:01.880127 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> >>> MAIL FROM:<ilja@earthquake.office.fastxs.net> SIZE=858..RCPT
> >>> TO:<ilja@test01.office.fastxs.net>..DATA..
> >>> ##
> >>> T 2004/03/05 13:46:01.880368 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> >>> 250 OK..
> >>> ##
> >>> T 2004/03/05 13:46:01.880460 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> >>> 250 Sender <ilja@earthquake.office.fastxs.net> OK..
> >>> ##
> >>> T 2004/03/05 13:46:01.883495 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> >>> 250 Recipient <ilja@test01.office.fastxs.net> OK..
> >>> ##
> >>> T 2004/03/05 13:46:01.883541 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> >>> 354 Start mail input; end with <CRLF>.<CRLF>..
> >>> ##
> >>> T 2004/03/05 13:47:41.920055 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> >>> RSET..QUIT..
> >>> ###
> >>> T 2004/03/05 13:52:41.875033 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> >>> 500 Error reading header...
> >>> #
> >>>
> >>> You can see that the client (postfix) starts with an RSET command,
> >>> and starts sending MAIL_FROM, RCPT and DATA.
> >>> The server responds with 250 OK to the reset, and 250 Sender OK and
> >>> 250 Recipient OK, and lets the client start sending (354 Start mail
> >>> input).
> >>>
> >>> The problem is that Postfix detects that it should not send data,
> >>> because the message is supposedly bounced by dbmail.. But why?
> >>>
> >>> Ilja
> >>>
> >>> Ilja Booij wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> Robert, thanks for your suggestion. It seems to work perfectly here.
> >>>> Messages now get delivered to users without a problem.
> >>>>
> >>>> I'll take a look at lmtp.c to see if I can spot the difference
> >>>> between the server getting a QUIT from the client and a reconnection
> >>>> on a new message, and a cached connection.
> >>>>
> >>>> Ilja
> >>>>
> >>>>
> >>>> Robert Theisen wrote:
> >>>>
> >>>>> New to the list here. But I read over the archives regarding this
> >>>>> problem and I may have a little to add.
> >>>>>
> >>>>> I was able to get postfix to consistently deliver via lmtp to
> >>>>> dbmail (even two consecutive messages to the same user) by setting
> >>>>> the postfix directive lmtp_cache_connection to false instead of
> >>>>> true. This would termintate the lmtp connection after each
> >>>>> delivery. I assume, with lmtp_cache_connection turned on, the
> >>>>> connection would die after lmtp_timeout was reached and a new
> >>>>> connection would be established and the mail would get sent. This
> >>>>> may have been a "neat feature" that works with Cyrus (postfix's
> >>>>> preferred lmtp client) or it may be a specification of the lmtp rfc
> >>>>> (I have not read it). But it seems like a simple enough fix to
> >>>>> make it work. It would probably just involve having the
> >>>>> conversation status fall back to a state the postfix expects after
> >>>>> message delivery and waiting in that state until either the
> >>>>> connection terminates or the conversation picks up again. I took
> >>>>> a cursory glance at the lmtp.c and lmtpd.c code but did not have
> >>>>> enough time to really dig into it.
> >>>>> Good Luck. I am anxiously awaiting the 2.0 release. :)
> >>>>>
> >>>>> Robert Theisen
> >>>>>
> >>>>> Aaron Stone wrote:
> >>>>>
> >>>>>> Sure thing, I've got about an hour of time this morning. Looks
> >>>>>> like you're
> >>>>>> burning the midnight oil over there! Yikes!
> >>>>>>
> >>>>>> Aaron
> >>>>>>
> >>>>>>
> >>>>>> Ilja Booij <ilja@ic-s.nl> said:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> I haven't been able to find the cause of the problem yet. I found
> >>>>>>> some more info in the logs though:
> >>>>>>>
> >>>>>>> Mar 4 16:55:34 test01 postfix/lmtp[21340]: 3378119F30A:
> >>>>>>> to=<target@test01.office.fastxs.net>,
> >>>>>>> relay=localhost.fastxs.net[127.0.0.1], delay=0, status=bounced
> >>>>>>> (host localhost.fastxs.net[127.0.0.1] said: 250 Recipient
> >>>>>>> <target@test01.office.fastxs.net> OK)
> >>>>>>>
> >>>>>>> apparently, postfix thinks we want to bounce the message. But
> >>>>>>> why? A previous message to the same user arrives correctly..
> >>>>>>>
> >>>>>>> strange, looking further. Aaron, can you also take a look at this?
> >>>>>>>
> >>>>>>> Ilja
> >>>>>>>
> >>>>>>> Ilja Booij wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> OK, I've found something:
> >>>>>>>>
> >>>>>>>> It seems that Postfix sends a RSET command, when we expect to
> >>>>>>>> get the header.
> >>>>>>>>
> >>>>>>>> readheader() then waits for postfix to send more, while postfix
> >>>>>>>> waits for a '250 OK' Message from dbmail-lmtp.
> >>>>>>>>
> >>>>>>>> So, there's probably something wrong in our LMTP-logic.
> >>>>>>>> I'll take a look at it, after I've done some small scripting job
> >>>>>>>> here for a customer.
> >>>>>>>>
> >>>>>>>> Ilja
> >>>>>>>>
> >>>>>>>> Ilja Booij wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> OK,
> >>>>>>>>>
> >>>>>>>>> this seems to have tackled the problem of the double delivery :)
> >>>>>>>>>
> >>>>>>>>> There still is another problem though:
> >>>>>>>>> It still seems to hang for a while on every second delivery
> >>>>>>>>> attempt to the LMTP daemon.
> >>>>>>>>>
> >>>>>>>>> Can you try the following scenario:
> >>>>>>>>>
> >>>>>>>>> * configure MTA to use dbmail-lmtp for delivery
> >>>>>>>>> * send message to a recipient
> >>>>>>>>> * verify that the message is received using dbmail-lmtp
> >>>>>>>>> * send a second message.
> >>>>>>>>>
> >>>>>>>>> What I observe here is that the second message takes a lot
> >>>>>>>>> longer to be delivered than the first one. The second one is
> >>>>>>>>> only delivered after minutes.
> >>>>>>>>>
> >>>>>>>>> I have the feeling that something is not right here..
> >>>>>>>>>
> >>>>>>>>> Ilja
> >>>>>>>>>
> >>>>>>>>> Aaron Stone wrote:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> New versions checked in, though not extensively tested yet.
> >>>>>>>>>>
> >>>>>>>>>> Ilja Booij <ilja@ic-s.nl> said:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> sounds ok to me :)
> >>>>>>>>>>>
> >>>>>>>>>>> Ilja
> >>>>>>>>>>>
> >>>>>>>>>>> Aaron Stone wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>> Working on it as we speak! Catch you in about an hour?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Aaron
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Ilja Booij <ilja@ic-s.nl> said:
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Aaron Stone wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> The reason for the double deliveries in the LMTP daemon is
> >>>>>>>>>>>>>> because the old
> >>>>>>>>>>>>>> insert_messages() function used to take lists of users to
> >>>>>>>>>>>>>> directly deliver.
> >>>>>>>>>>>>>> The new insert_messages() function takes a list of dsnuser
> >>>>>>>>>>>>>> structures, and
> >>>>>>>>>>>>>> expects that *either* the useridnr field is filled (for
> >>>>>>>>>>>>>> direct-to-useridnr
> >>>>>>>>>>>>>> delivery, which isn't supported by any of the frontends,
> >>>>>>>>>>>>>> but is supported in
> >>>>>>>>>>>>>> the lower layers ;-) or the address field, which is first
> >>>>>>>>>>>>>> checked as a
> >>>>>>>>>>>>>> username and then as an alias (this allows usernames in
> >>>>>>>>>>>>>> the form of
> >>>>>>>>>>>>>> 'user@server' to operate without requiring an alias that
> >>>>>>>>>>>>>> says 'user@server
> >>>>>>>>>>>>>> delivers to user@server').
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Some refactoring might be necessary, because we need to
> >>>>>>>>>>>>>> inform the MTA
> >>>>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> whether
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>>>> or not its envelope recipients were valid before it will
> >>>>>>>>>>>>>> send us the message.
> >>>>>>>>>>>>>> This means users need to be validated before
> >>>>>>>>>>>>>> read_header(), which is a
> >>>>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> problem
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>>>> because at the moment this vaildation happens in
> >>>>>>>>>>>>>> insert_messages().
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> OK, that's clear
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Fine and dandy in pipe land, where the MTA will send the
> >>>>>>>>>>>>>> message
> >>>>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> regardless of
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>>>> the disposition of the recipients, and only at the very
> >>>>>>>>>>>>>> end are error
> >>>>>>>>>>>>>> conditions returned as exit codes... in LMTP land we need
> >>>>>>>>>>>>>> to know ahead of
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> time.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>> I think what I'll do is move the user validation code from
> >>>>>>>>>>>>>> insert_messages()
> >>>>>>>>>>>>>> into dsn.c, perhaps calling it dsn_check_users() or
> >>>>>>>>>>>>>> dsn_prepare_deliveries().
> >>>>>>>>>>>>>> Then, we'll have this order:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> For command-line and envelope-recipient deliveries:
> >>>>>>>>>>>>>> - receive addresses and store them into the dsnusers list.
> >>>>>>>>>>>>>> - dsn_prepare_deliveries()
> >>>>>>>>>>>>>> - if no deliveries are valid, return an error.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> - read_header()
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> For deliver-to header deliveries:
> >>>>>>>>>>>>>> - mime_readheader()
> >>>>>>>>>>>>>> - mail_adr_list()
> >>>>>>>>>>>>>> - dsn_prepare_deliveries()
> >>>>>>>>>>>>>> - if no deliveries are valid, return an error.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> - insert_messages()
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I like the part where you say 'What I'll do' :)
> >>>>>>>>>>>>> Do you think this work will take long? I think we must
> >>>>>>>>>>>>> really release rc3 tomorrow (Friday, March 5th). This stuff
> >>>>>>>>>>>>> really has to work if we want to release.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Ilja
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> _______________________________________________
> >>>>>>>>>>>>> Dbmail-dev mailing list
> >>>>>>>>>>>>> Dbmail-dev@dbmail.org
> >>>>>>>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> _______________________________________________
> >>>>>>>>>>> Dbmail-dev mailing list
> >>>>>>>>>>> Dbmail-dev@dbmail.org
> >>>>>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> _______________________________________________
> >>>>>>>>> Dbmail-dev mailing list
> >>>>>>>>> Dbmail-dev@dbmail.org
> >>>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> _______________________________________________
> >>>>>>>> Dbmail-dev mailing list
> >>>>>>>> Dbmail-dev@dbmail.org
> >>>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> _______________________________________________
> >>>>>>> Dbmail-dev mailing list
> >>>>>>> Dbmail-dev@dbmail.org
> >>>>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> Dbmail-dev mailing list
> >>>>> Dbmail-dev@dbmail.org
> >>>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> Dbmail-dev mailing list
> >>>> Dbmail-dev@dbmail.org
> >>>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> Dbmail-dev mailing list
> >>> Dbmail-dev@dbmail.org
> >>> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>
> >>
> >> _______________________________________________
> >> Dbmail-dev mailing list
> >> Dbmail-dev@dbmail.org
> >> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >
> > _______________________________________________
> > Dbmail-dev mailing list
> > Dbmail-dev@dbmail.org
> > http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>



--

Hi,

(respone inline)

Aaron Stone wrote:

> In this example, there are three recipients, pat, jones and green. Following
> the data command, there are only two acknowledgements. This is contrary to my
> understanding that each recipient had to be acknowledged:
>
> The LMTP protocol causes the server to return, after the final "." of
> the DATA command, one reply for each recipient. Therefore, if the
> queue manager is configured to use LMTP instead of SMTP when
> transferring messages to the delivery agents, then the delivery
> agents may attempt delivery to each recipient after the final "." and
> individually report the status for each recipient. Connections which
> should use the LMTP protocol are drawn in the diagram above using
> asterisks.
>
> So looking at jones, it is clear why: jones failed the RCPT command, and so
> was no longer in the recipient list when the DATA command came around. Then

Clear :)

> read this:
>
> The additional restriction is that when there have been no successful
> RCPT commands in the mail transaction, the DATA command MUST fail
> with a 503 reply code.
>
> The change is that after the final ".", the server returns one reply
> for each previously successful RCPT command in the mail transaction,
> in the order that the RCPT commands were issued. Even if there were
> multiple successful RCPT commands giving the same forward-path, there
> must be one reply for each successful RCPT command.
>
> I think that what the first paragraph in this quote means is not that DATA is
> failed, per se, but that the whole conversation is flagged with a 503.
>
> So here's what I think needs to be fixed:
>
> - If there are no recipients, use 503 AND NOTHING ELSE to report it,
> which means changing this section of code to use a 503:
>
> if (!has_recipients)
> {
> trace(TRACE_DEBUG,"main(): no valid recipients found, cancel message.");
> fprintf((FILE *)stream, "554 No valid recipients.\r\n" );
> }

Yep, change the 554 to 503

>
> - When a recipient is failed, report the error and then *remove that
> recipient from the dsnusers list* so that after the message is
> received, their failure is not reported a second time.

Sounds good :)
>
> I'll get on it this afternoon, please test it in the morning, your time!
I will.

Ilja

>
> Ilja Booij <ilja@ic-s.nl> said:
>
>
>>After some more reading of RFC 2033 I'm inclined to believe an LMTP
>>client does not get a response after sending the DATA, except for the
>>responses about the delivery of the message to the users.
>>
>>Example session from the RFC:
>>
>> S: 220 foo.edu LMTP server ready
>> C: LHLO foo.edu
>> S: 250-foo.edu
>> S: 250-PIPELINING
>> S: 250 SIZE
>> C: MAIL FROM:<chris@bar.com>
>> S: 250 OK
>>
>> C: RCPT TO:<pat@foo.edu>
>> S: 250 OK
>> C: RCPT TO:<jones@foo.edu>
>> S: 550 No such user here
>> C: RCPT TO:<green@foo.edu>
>> S: 250 OK
>> C: DATA
>> S: 354 Start mail input; end with <CRLF>.<CRLF>
>> C: Blah blah blah...
>> C: ...etc. etc. etc.
>> C: .
>> S: 250 OK
>> S: 452 <green@foo.edu> is temporarily over quota
>> C: QUIT
>> S: 221 foo.edu closing connection
>>
>>The important bit is after the DATA line
>>there are two responses from the server after the data line:
>>1. "250 OK": the message to chris@bar.com was delivered
>>2. "452 <green@foo.edu> is temporarily over quota", the message to
>>green@foo.edu was not delivered.
>>
>>There is no line indicating that the message was received by the server.
>>The server should only indicate when a message was not received correctly.
>>
>>I'll complete remove the
>>fprintf((FILE *)stream, "250 Message received OK\r\n"); line
>>
>>
>>Ilja Booij wrote:
>>
>>
>>>Hi,
>>>
>>>Removing the
>>>"250 Message received OK" after having received the message takes care
>>>of the error. I can just send several messages, with
>>>"lmtp_cache_connection = yes" in /etc/postfix/main.cf
>>>
>>>I'm starting to wonder whether LMTP requires us to send a message if the
>>>message was received OK by the server.
>>>
>>>Ilja
>>>
>>>Ilja Booij wrote:
>>>
>>>
>>>>Hi,
>>>>
>>>>with some help from a guy on postfix-users@postfix.org I found something:
>>>>
>>>>The "250 Recipient <target@test01.office.fastxs.net> OK" message from
>>>>dbmail-lmtp is out of sync. The postfix/lmtp program stores this
>>>>message until the next message is sent, causing the messages between
>>>>postfix and dbmail-smtp to be horribly out-of-sync.
>>>>
>>>>I guess this should be quite easy to fix. I'll see if I have some time
>>>>to do it this weekend. Otherwise I'll do it on Monday. Unless anybody
>>>>else wants to do it of course ;)
>>>>
>>>>I'll be off in a few minutes. First a beer, then it's off to home :)
>>>>
>>>>Have a nice weekend!
>>>>
>>>>Ilja
>>>>
>>>>
>>>>Ilja Booij wrote:
>>>>
>>>>
>>>>>Hmm, I still don't really understand the problem. I've done some
>>>>>ngrep-ing, to see what goes over the network.
>>>>>
>>>>>This is a session that's OK:
>>>>>
>>>>>T 2004/03/05 13:45:46.324232 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>>>> 220 test01 DBMail LMTP service ready to rock..
>>>>>##
>>>>>T 2004/03/05 13:45:46.324696 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>>>>> LHLO test01.office.fastxs.net..
>>>>>##
>>>>>T 2004/03/05 13:45:46.324900 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>>>> 250-test01..250-PIPELINING..250-ENHANCEDSTATUSCODES..250 SIZE..
>>>>>#
>>>>>T 2004/03/05 13:45:46.325306 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>>>>> MAIL FROM:<ilja@earthquake.office.fastxs.net> SIZE=866..RCPT
>>>>>TO:<target@test01.office.fastxs.net>..DATA..
>>>>>#
>>>>>T 2004/03/05 13:45:46.325479 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>>>> 250 Sender <ilja@earthquake.office.fastxs.net> OK..
>>>>>##
>>>>>T 2004/03/05 13:45:46.365648 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>>>> 250 Recipient <target@test01.office.fastxs.net> OK..354 Start mail
>>>>>input; end with <CRLF>.<CRLF>..
>>>>>##
>>>>>T 2004/03/05 13:45:46.365974 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>>>>> Received: from earthquake.office.fastxs.net
>>>>>(earthquake.office.fastxs.net [10.0.1.15])...by test01.office.fastxs.n
>>>>> et (Postfix) with ESMTP id 394DD19F30A...for
>>>>><target@test01.office.fastxs.net>; Fri, 5 Mar 2004 13:45:46 +0100 (C
>>>>> ET)..Received: from earthquake.office.fastxs.net (localhost
>>>>>[127.0.0.1])...by earthquake.office.fastxs.net (Postfi
>>>>> x) with ESMTP id 405D4375FE...for
>>>>><target@test01.office.fastxs.net>; Fri, 5 Mar 2004 13:49:12 +0100
>>>>>(CET)..Receiv
>>>>> ed: (from ilja@localhost)...by earthquake.office.fastxs.net
>>>>>(8.12.9/8.12.9/Submit) id i25CnCo0005116...for target@
>>>>> test01.office.fastxs.net; Fri, 5 Mar 2004 13:49:12 +0100
>>>>>(CET)..Date: Fri, 5 Mar 2004 13:49:12 +0100 (CET)..From:
>>>>> Ilja Booij <ilja@earthquake.office.fastxs.net>..Message-Id:
>>>>><200403051249.i25CnCo0005116@earthquake.office.fastxs.
>>>>> net>..To: target@test01.office.fastxs.net..Subject: 5-3
>>>>>27......27.....
>>>>>##
>>>>>T 2004/03/05 13:45:46.568851 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>>>> 250 Message received OK..
>>>>>##
>>>>>T 2004/03/05 13:45:46.608610 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>>>> c..
>>>>>
>>>>>
>>>>>
>>>>>If another message is sent (over the same connection):
>>>>>
>>>>>
>>>>>##
>>>>>T 2004/03/05 13:46:01.879964 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>>>>> RSET..
>>>>>##
>>>>>T 2004/03/05 13:46:01.880127 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>>>>> MAIL FROM:<ilja@earthquake.office.fastxs.net> SIZE=858..RCPT
>>>>>TO:<ilja@test01.office.fastxs.net>..DATA..
>>>>>##
>>>>>T 2004/03/05 13:46:01.880368 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>>>> 250 OK..
>>>>>##
>>>>>T 2004/03/05 13:46:01.880460 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>>>> 250 Sender <ilja@earthquake.office.fastxs.net> OK..
>>>>>##
>>>>>T 2004/03/05 13:46:01.883495 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>>>> 250 Recipient <ilja@test01.office.fastxs.net> OK..
>>>>>##
>>>>>T 2004/03/05 13:46:01.883541 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>>>> 354 Start mail input; end with <CRLF>.<CRLF>..
>>>>>##
>>>>>T 2004/03/05 13:47:41.920055 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>>>>> RSET..QUIT..
>>>>>###
>>>>>T 2004/03/05 13:52:41.875033 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>>>> 500 Error reading header...
>>>>>#
>>>>>
>>>>>You can see that the client (postfix) starts with an RSET command,
>>>>>and starts sending MAIL_FROM, RCPT and DATA.
>>>>>The server responds with 250 OK to the reset, and 250 Sender OK and
>>>>>250 Recipient OK, and lets the client start sending (354 Start mail
>>>>>input).
>>>>>
>>>>>The problem is that Postfix detects that it should not send data,
>>>>>because the message is supposedly bounced by dbmail.. But why?
>>>>>
>>>>>Ilja
>>>>>
>>>>>Ilja Booij wrote:
>>>>>
>>>>>
>>>>>>Hi,
>>>>>>
>>>>>>Robert, thanks for your suggestion. It seems to work perfectly here.
>>>>>>Messages now get delivered to users without a problem.
>>>>>>
>>>>>>I'll take a look at lmtp.c to see if I can spot the difference
>>>>>>between the server getting a QUIT from the client and a reconnection
>>>>>>on a new message, and a cached connection.
>>>>>>
>>>>>>Ilja
>>>>>>
>>>>>>
>>>>>>Robert Theisen wrote:
>>>>>>
>>>>>>
>>>>>>>New to the list here. But I read over the archives regarding this
>>>>>>>problem and I may have a little to add.
>>>>>>>
>>>>>>>I was able to get postfix to consistently deliver via lmtp to
>>>>>>>dbmail (even two consecutive messages to the same user) by setting
>>>>>>>the postfix directive lmtp_cache_connection to false instead of
>>>>>>>true. This would termintate the lmtp connection after each
>>>>>>>delivery. I assume, with lmtp_cache_connection turned on, the
>>>>>>>connection would die after lmtp_timeout was reached and a new
>>>>>>>connection would be established and the mail would get sent. This
>>>>>>>may have been a "neat feature" that works with Cyrus (postfix's
>>>>>>>preferred lmtp client) or it may be a specification of the lmtp rfc
>>>>>>>(I have not read it). But it seems like a simple enough fix to
>>>>>>>make it work. It would probably just involve having the
>>>>>>>conversation status fall back to a state the postfix expects after
>>>>>>>message delivery and waiting in that state until either the
>>>>>>>connection terminates or the conversation picks up again. I took
>>>>>>>a cursory glance at the lmtp.c and lmtpd.c code but did not have
>>>>>>>enough time to really dig into it.
>>>>>>>Good Luck. I am anxiously awaiting the 2.0 release. :)
>>>>>>>
>>>>>>>Robert Theisen
>>>>>>>
>>>>>>>Aaron Stone wrote:
>>>>>>>
>>>>>>>
>>>>>>>>Sure thing, I've got about an hour of time this morning. Looks
>>>>>>>>like you're
>>>>>>>>burning the midnight oil over there! Yikes!
>>>>>>>>
>>>>>>>>Aaron
>>>>>>>>
>>>>>>>>
>>>>>>>>Ilja Booij <ilja@ic-s.nl> said:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>Hi,
>>>>>>>>>
>>>>>>>>>I haven't been able to find the cause of the problem yet. I found
>>>>>>>>>some more info in the logs though:
>>>>>>>>>
>>>>>>>>>Mar 4 16:55:34 test01 postfix/lmtp[21340]: 3378119F30A:
>>>>>>>>>to=<target@test01.office.fastxs.net>,
>>>>>>>>>relay=localhost.fastxs.net[127.0.0.1], delay=0, status=bounced
>>>>>>>>>(host localhost.fastxs.net[127.0.0.1] said: 250 Recipient
>>>>>>>>><target@test01.office.fastxs.net> OK)
>>>>>>>>>
>>>>>>>>>apparently, postfix thinks we want to bounce the message. But
>>>>>>>>>why? A previous message to the same user arrives correctly..
>>>>>>>>>
>>>>>>>>>strange, looking further. Aaron, can you also take a look at this?
>>>>>>>>>
>>>>>>>>>Ilja
>>>>>>>>>
>>>>>>>>>Ilja Booij wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>OK, I've found something:
>>>>>>>>>>
>>>>>>>>>>It seems that Postfix sends a RSET command, when we expect to
>>>>>>>>>>get the header.
>>>>>>>>>>
>>>>>>>>>>readheader() then waits for postfix to send more, while postfix
>>>>>>>>>>waits for a '250 OK' Message from dbmail-lmtp.
>>>>>>>>>>
>>>>>>>>>>So, there's probably something wrong in our LMTP-logic.
>>>>>>>>>>I'll take a look at it, after I've done some small scripting job
>>>>>>>>>>here for a customer.
>>>>>>>>>>
>>>>>>>>>>Ilja
>>>>>>>>>>
>>>>>>>>>>Ilja Booij wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>OK,
>>>>>>>>>>>
>>>>>>>>>>>this seems to have tackled the problem of the double delivery :)
>>>>>>>>>>>
>>>>>>>>>>>There still is another problem though:
>>>>>>>>>>>It still seems to hang for a while on every second delivery
>>>>>>>>>>>attempt to the LMTP daemon.
>>>>>>>>>>>
>>>>>>>>>>>Can you try the following scenario:
>>>>>>>>>>>
>>>>>>>>>>>* configure MTA to use dbmail-lmtp for delivery
>>>>>>>>>>>* send message to a recipient
>>>>>>>>>>>* verify that the message is received using dbmail-lmtp
>>>>>>>>>>>* send a second message.
>>>>>>>>>>>
>>>>>>>>>>>What I observe here is that the second message takes a lot
>>>>>>>>>>>longer to be delivered than the first one. The second one is
>>>>>>>>>>>only delivered after minutes.
>>>>>>>>>>>
>>>>>>>>>>>I have the feeling that something is not right here..
>>>>>>>>>>>
>>>>>>>>>>>Ilja
>>>>>>>>>>>
>>>>>>>>>>>Aaron Stone wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>New versions checked in, though not extensively tested yet.
>>>>>>>>>>>>
>>>>>>>>>>>>Ilja Booij <ilja@ic-s.nl> said:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>sounds ok to me :)
>>>>>>>>>>>>>
>>>>>>>>>>>>>Ilja
>>>>>>>>>>>>>
>>>>>>>>>>>>>Aaron Stone wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>Working on it as we speak! Catch you in about an hour?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>Aaron
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>Ilja Booij <ilja@ic-s.nl> said:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>Aaron Stone wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>The reason for the double deliveries in the LMTP daemon is
>>>>>>>>>>>>>>>>because the old
>>>>>>>>>>>>>>>>insert_messages() function used to take lists of users to
>>>>>>>>>>>>>>>>directly deliver.
>>>>>>>>>>>>>>>>The new insert_messages() function takes a list of dsnuser
>>>>>>>>>>>>>>>>structures, and
>>>>>>>>>>>>>>>>expects that *either* the useridnr field is filled (for
>>>>>>>>>>>>>>>>direct-to-useridnr
>>>>>>>>>>>>>>>>delivery, which isn't supported by any of the frontends,
>>>>>>>>>>>>>>>>but is supported in
>>>>>>>>>>>>>>>>the lower layers ;-) or the address field, which is first
>>>>>>>>>>>>>>>>checked as a
>>>>>>>>>>>>>>>>username and then as an alias (this allows usernames in
>>>>>>>>>>>>>>>>the form of
>>>>>>>>>>>>>>>>'user@server' to operate without requiring an alias that
>>>>>>>>>>>>>>>>says 'user@server
>>>>>>>>>>>>>>>>delivers to user@server').
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>Some refactoring might be necessary, because we need to
>>>>>>>>>>>>>>>>inform the MTA
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>whether
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>>>>or not its envelope recipients were valid before it will
>>>>>>>>>>>>>>>>send us the message.
>>>>>>>>>>>>>>>>This means users need to be validated before
>>>>>>>>>>>>>>>>read_header(), which is a
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>problem
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>>>>because at the moment this vaildation happens in
>>>>>>>>>>>>>>>>insert_messages().
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>OK, that's clear
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>Fine and dandy in pipe land, where the MTA will send the
>>>>>>>>>>>>>>>>message
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>regardless of
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>>>>the disposition of the recipients, and only at the very
>>>>>>>>>>>>>>>>end are error
>>>>>>>>>>>>>>>>conditions returned as exit codes... in LMTP land we need
>>>>>>>>>>>>>>>>to know ahead of
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>time.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>I think what I'll do is move the user validation code from
>>>>>>>>>>>>>>>>insert_messages()
>>>>>>>>>>>>>>>>into dsn.c, perhaps calling it dsn_check_users() or
>>>>>>>>>>>>>>>>dsn_prepare_deliveries().
>>>>>>>>>>>>>>>>Then, we'll have this order:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>For command-line and envelope-recipient deliveries:
>>>>>>>>>>>>>>>>- receive addresses and store them into the dsnusers list.
>>>>>>>>>>>>>>>>- dsn_prepare_deliveries()
>>>>>>>>>>>>>>>>- if no deliveries are valid, return an error.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>- read_header()
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>For deliver-to header deliveries:
>>>>>>>>>>>>>>>>- mime_readheader()
>>>>>>>>>>>>>>>>- mail_adr_list()
>>>>>>>>>>>>>>>>- dsn_prepare_deliveries()
>>>>>>>>>>>>>>>>- if no deliveries are valid, return an error.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>- insert_messages()
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>I like the part where you say 'What I'll do' :)
>>>>>>>>>>>>>>>Do you think this work will take long? I think we must
>>>>>>>>>>>>>>>really release rc3 tomorrow (Friday, March 5th). This stuff
>>>>>>>>>>>>>>>really has to work if we want to release.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>Ilja
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>_______________________________________________
>>>>>>>>>>>>>>>Dbmail-dev mailing list
>>>>>>>>>>>>>>>Dbmail-dev@dbmail.org
>>>>>>>>>>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>_______________________________________________
>>>>>>>>>>>>>Dbmail-dev mailing list
>>>>>>>>>>>>>Dbmail-dev@dbmail.org
>>>>>>>>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>_______________________________________________
>>>>>>>>>>>Dbmail-dev mailing list
>>>>>>>>>>>Dbmail-dev@dbmail.org
>>>>>>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>_______________________________________________
>>>>>>>>>>Dbmail-dev mailing list
>>>>>>>>>>Dbmail-dev@dbmail.org
>>>>>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>_______________________________________________
>>>>>>>>>Dbmail-dev mailing list
>>>>>>>>>Dbmail-dev@dbmail.org
>>>>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>_______________________________________________
>>>>>>>Dbmail-dev mailing list
>>>>>>>Dbmail-dev@dbmail.org
>>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>_______________________________________________
>>>>>>Dbmail-dev mailing list
>>>>>>Dbmail-dev@dbmail.org
>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>
>>>>>
>>>>>
>>>>>_______________________________________________
>>>>>Dbmail-dev mailing list
>>>>>Dbmail-dev@dbmail.org
>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>
>>>>
>>>>_______________________________________________
>>>>Dbmail-dev mailing list
>>>>Dbmail-dev@dbmail.org
>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>
>>>_______________________________________________
>>>Dbmail-dev mailing list
>>>Dbmail-dev@dbmail.org
>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>
>>_______________________________________________
>>Dbmail-dev mailing list
>>Dbmail-dev@dbmail.org
>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>
>
>
>
>

Ok, then you're still up, and I just fixed the other two sections of code :-)

Aaron


Ilja Booij <ilja@ic-s.nl> said:

> Hi,
>
> (respone inline)
>
> Aaron Stone wrote:
>
> > In this example, there are three recipients, pat, jones and green. Following
> > the data command, there are only two acknowledgements. This is contrary to my
> > understanding that each recipient had to be acknowledged:
> >
> > The LMTP protocol causes the server to return, after the final "." of
> > the DATA command, one reply for each recipient. Therefore, if the
> > queue manager is configured to use LMTP instead of SMTP when
> > transferring messages to the delivery agents, then the delivery
> > agents may attempt delivery to each recipient after the final "." and
> > individually report the status for each recipient. Connections which
> > should use the LMTP protocol are drawn in the diagram above using
> > asterisks.
> >
> > So looking at jones, it is clear why: jones failed the RCPT command, and so
> > was no longer in the recipient list when the DATA command came around. Then
>
> Clear :)
>
> > read this:
> >
> > The additional restriction is that when there have been no successful
> > RCPT commands in the mail transaction, the DATA command MUST fail
> > with a 503 reply code.
> >
> > The change is that after the final ".", the server returns one reply
> > for each previously successful RCPT command in the mail transaction,
> > in the order that the RCPT commands were issued. Even if there were
> > multiple successful RCPT commands giving the same forward-path, there
> > must be one reply for each successful RCPT command.
> >
> > I think that what the first paragraph in this quote means is not that DATA is
> > failed, per se, but that the whole conversation is flagged with a 503.
> >
> > So here's what I think needs to be fixed:
> >
> > - If there are no recipients, use 503 AND NOTHING ELSE to report it,
> > which means changing this section of code to use a 503:
> >
> > if (!has_recipients)
> > {
> > trace(TRACE_DEBUG,"main(): no valid recipients found, cancel message.");
> > fprintf((FILE *)stream, "554 No valid recipients.\r\n" );
> > }
>
> Yep, change the 554 to 503
>
> >
> > - When a recipient is failed, report the error and then *remove that
> > recipient from the dsnusers list* so that after the message is
> > received, their failure is not reported a second time.
>
> Sounds good :)
> >
> > I'll get on it this afternoon, please test it in the morning, your time!
> I will.
>
> Ilja
>
> >
> > Ilja Booij <ilja@ic-s.nl> said:
> >
> >
> >>After some more reading of RFC 2033 I'm inclined to believe an LMTP
> >>client does not get a response after sending the DATA, except for the
> >>responses about the delivery of the message to the users.
> >>
> >>Example session from the RFC:
> >>
> >> S: 220 foo.edu LMTP server ready
> >> C: LHLO foo.edu
> >> S: 250-foo.edu
> >> S: 250-PIPELINING
> >> S: 250 SIZE
> >> C: MAIL FROM:<chris@bar.com>
> >> S: 250 OK
> >>
> >> C: RCPT TO:<pat@foo.edu>
> >> S: 250 OK
> >> C: RCPT TO:<jones@foo.edu>
> >> S: 550 No such user here
> >> C: RCPT TO:<green@foo.edu>
> >> S: 250 OK
> >> C: DATA
> >> S: 354 Start mail input; end with <CRLF>.<CRLF>
> >> C: Blah blah blah...
> >> C: ...etc. etc. etc.
> >> C: .
> >> S: 250 OK
> >> S: 452 <green@foo.edu> is temporarily over quota
> >> C: QUIT
> >> S: 221 foo.edu closing connection
> >>
> >>The important bit is after the DATA line
> >>there are two responses from the server after the data line:
> >>1. "250 OK": the message to chris@bar.com was delivered
> >>2. "452 <green@foo.edu> is temporarily over quota", the message to
> >>green@foo.edu was not delivered.
> >>
> >>There is no line indicating that the message was received by the server.
> >>The server should only indicate when a message was not received correctly.
> >>
> >>I'll complete remove the
> >>fprintf((FILE *)stream, "250 Message received OK\r\n"); line
> >>
> >>
> >>Ilja Booij wrote:
> >>
> >>
> >>>Hi,
> >>>
> >>>Removing the
> >>>"250 Message received OK" after having received the message takes care
> >>>of the error. I can just send several messages, with
> >>>"lmtp_cache_connection = yes" in /etc/postfix/main.cf
> >>>
> >>>I'm starting to wonder whether LMTP requires us to send a message if the
> >>>message was received OK by the server.
> >>>
> >>>Ilja
> >>>
> >>>Ilja Booij wrote:
> >>>
> >>>
> >>>>Hi,
> >>>>
> >>>>with some help from a guy on postfix-users@postfix.org I found something:
> >>>>
> >>>>The "250 Recipient <target@test01.office.fastxs.net> OK" message from
> >>>>dbmail-lmtp is out of sync. The postfix/lmtp program stores this
> >>>>message until the next message is sent, causing the messages between
> >>>>postfix and dbmail-smtp to be horribly out-of-sync.
> >>>>
> >>>>I guess this should be quite easy to fix. I'll see if I have some time
> >>>>to do it this weekend. Otherwise I'll do it on Monday. Unless anybody
> >>>>else wants to do it of course ;)
> >>>>
> >>>>I'll be off in a few minutes. First a beer, then it's off to home :)
> >>>>
> >>>>Have a nice weekend!
> >>>>
> >>>>Ilja
> >>>>
> >>>>
> >>>>Ilja Booij wrote:
> >>>>
> >>>>
> >>>>>Hmm, I still don't really understand the problem. I've done some
> >>>>>ngrep-ing, to see what goes over the network.
> >>>>>
> >>>>>This is a session that's OK:
> >>>>>
> >>>>>T 2004/03/05 13:45:46.324232 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> >>>>> 220 test01 DBMail LMTP service ready to rock..
> >>>>>##
> >>>>>T 2004/03/05 13:45:46.324696 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> >>>>> LHLO test01.office.fastxs.net..
> >>>>>##
> >>>>>T 2004/03/05 13:45:46.324900 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> >>>>> 250-test01..250-PIPELINING..250-ENHANCEDSTATUSCODES..250 SIZE..
> >>>>>#
> >>>>>T 2004/03/05 13:45:46.325306 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> >>>>> MAIL FROM:<ilja@earthquake.office.fastxs.net> SIZE=866..RCPT
> >>>>>TO:<target@test01.office.fastxs.net>..DATA..
> >>>>>#
> >>>>>T 2004/03/05 13:45:46.325479 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> >>>>> 250 Sender <ilja@earthquake.office.fastxs.net> OK..
> >>>>>##
> >>>>>T 2004/03/05 13:45:46.365648 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> >>>>> 250 Recipient <target@test01.office.fastxs.net> OK..354 Start mail
> >>>>>input; end with <CRLF>.<CRLF>..
> >>>>>##
> >>>>>T 2004/03/05 13:45:46.365974 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> >>>>> Received: from earthquake.office.fastxs.net
> >>>>>(earthquake.office.fastxs.net [10.0.1.15])...by test01.office.fastxs.n
> >>>>> et (Postfix) with ESMTP id 394DD19F30A...for
> >>>>><target@test01.office.fastxs.net>; Fri, 5 Mar 2004 13:45:46 +0100 (C
> >>>>> ET)..Received: from earthquake.office.fastxs.net (localhost
> >>>>>[127.0.0.1])...by earthquake.office.fastxs.net (Postfi
> >>>>> x) with ESMTP id 405D4375FE...for
> >>>>><target@test01.office.fastxs.net>; Fri, 5 Mar 2004 13:49:12 +0100
> >>>>>(CET)..Receiv
> >>>>> ed: (from ilja@localhost)...by earthquake.office.fastxs.net
> >>>>>(8.12.9/8.12.9/Submit) id i25CnCo0005116...for target@
> >>>>> test01.office.fastxs.net; Fri, 5 Mar 2004 13:49:12 +0100
> >>>>>(CET)..Date: Fri, 5 Mar 2004 13:49:12 +0100 (CET)..From:
> >>>>> Ilja Booij <ilja@earthquake.office.fastxs.net>..Message-Id:
> >>>>><200403051249.i25CnCo0005116@earthquake.office.fastxs.
> >>>>> net>..To: target@test01.office.fastxs.net..Subject: 5-3
> >>>>>27......27.....
> >>>>>##
> >>>>>T 2004/03/05 13:45:46.568851 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> >>>>> 250 Message received OK..
> >>>>>##
> >>>>>T 2004/03/05 13:45:46.608610 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> >>>>> c..
> >>>>>
> >>>>>
> >>>>>
> >>>>>If another message is sent (over the same connection):
> >>>>>
> >>>>>
> >>>>>##
> >>>>>T 2004/03/05 13:46:01.879964 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> >>>>> RSET..
> >>>>>##
> >>>>>T 2004/03/05 13:46:01.880127 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> >>>>> MAIL FROM:<ilja@earthquake.office.fastxs.net> SIZE=858..RCPT
> >>>>>TO:<ilja@test01.office.fastxs.net>..DATA..
> >>>>>##
> >>>>>T 2004/03/05 13:46:01.880368 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> >>>>> 250 OK..
> >>>>>##
> >>>>>T 2004/03/05 13:46:01.880460 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> >>>>> 250 Sender <ilja@earthquake.office.fastxs.net> OK..
> >>>>>##
> >>>>>T 2004/03/05 13:46:01.883495 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> >>>>> 250 Recipient <ilja@test01.office.fastxs.net> OK..
> >>>>>##
> >>>>>T 2004/03/05 13:46:01.883541 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> >>>>> 354 Start mail input; end with <CRLF>.<CRLF>..
> >>>>>##
> >>>>>T 2004/03/05 13:47:41.920055 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
> >>>>> RSET..QUIT..
> >>>>>###
> >>>>>T 2004/03/05 13:52:41.875033 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
> >>>>> 500 Error reading header...
> >>>>>#
> >>>>>
> >>>>>You can see that the client (postfix) starts with an RSET command,
> >>>>>and starts sending MAIL_FROM, RCPT and DATA.
> >>>>>The server responds with 250 OK to the reset, and 250 Sender OK and
> >>>>>250 Recipient OK, and lets the client start sending (354 Start mail
> >>>>>input).
> >>>>>
> >>>>>The problem is that Postfix detects that it should not send data,
> >>>>>because the message is supposedly bounced by dbmail.. But why?
> >>>>>
> >>>>>Ilja
> >>>>>
> >>>>>Ilja Booij wrote:
> >>>>>
> >>>>>
> >>>>>>Hi,
> >>>>>>
> >>>>>>Robert, thanks for your suggestion. It seems to work perfectly here.
> >>>>>>Messages now get delivered to users without a problem.
> >>>>>>
> >>>>>>I'll take a look at lmtp.c to see if I can spot the difference
> >>>>>>between the server getting a QUIT from the client and a reconnection
> >>>>>>on a new message, and a cached connection.
> >>>>>>
> >>>>>>Ilja
> >>>>>>
> >>>>>>
> >>>>>>Robert Theisen wrote:
> >>>>>>
> >>>>>>
> >>>>>>>New to the list here. But I read over the archives regarding this
> >>>>>>>problem and I may have a little to add.
> >>>>>>>
> >>>>>>>I was able to get postfix to consistently deliver via lmtp to
> >>>>>>>dbmail (even two consecutive messages to the same user) by setting
> >>>>>>>the postfix directive lmtp_cache_connection to false instead of
> >>>>>>>true. This would termintate the lmtp connection after each
> >>>>>>>delivery. I assume, with lmtp_cache_connection turned on, the
> >>>>>>>connection would die after lmtp_timeout was reached and a new
> >>>>>>>connection would be established and the mail would get sent. This
> >>>>>>>may have been a "neat feature" that works with Cyrus (postfix's
> >>>>>>>preferred lmtp client) or it may be a specification of the lmtp rfc
> >>>>>>>(I have not read it). But it seems like a simple enough fix to
> >>>>>>>make it work. It would probably just involve having the
> >>>>>>>conversation status fall back to a state the postfix expects after
> >>>>>>>message delivery and waiting in that state until either the
> >>>>>>>connection terminates or the conversation picks up again. I took
> >>>>>>>a cursory glance at the lmtp.c and lmtpd.c code but did not have
> >>>>>>>enough time to really dig into it.
> >>>>>>>Good Luck. I am anxiously awaiting the 2.0 release. :)
> >>>>>>>
> >>>>>>>Robert Theisen
> >>>>>>>
> >>>>>>>Aaron Stone wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>>Sure thing, I've got about an hour of time this morning. Looks
> >>>>>>>>like you're
> >>>>>>>>burning the midnight oil over there! Yikes!
> >>>>>>>>
> >>>>>>>>Aaron
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>Ilja Booij <ilja@ic-s.nl> said:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>Hi,
> >>>>>>>>>
> >>>>>>>>>I haven't been able to find the cause of the problem yet. I found
> >>>>>>>>>some more info in the logs though:
> >>>>>>>>>
> >>>>>>>>>Mar 4 16:55:34 test01 postfix/lmtp[21340]: 3378119F30A:
> >>>>>>>>>to=<target@test01.office.fastxs.net>,
> >>>>>>>>>relay=localhost.fastxs.net[127.0.0.1], delay=0, status=bounced
> >>>>>>>>>(host localhost.fastxs.net[127.0.0.1] said: 250 Recipient
> >>>>>>>>><target@test01.office.fastxs.net> OK)
> >>>>>>>>>
> >>>>>>>>>apparently, postfix thinks we want to bounce the message. But
> >>>>>>>>>why? A previous message to the same user arrives correctly..
> >>>>>>>>>
> >>>>>>>>>strange, looking further. Aaron, can you also take a look at this?
> >>>>>>>>>
> >>>>>>>>>Ilja
> >>>>>>>>>
> >>>>>>>>>Ilja Booij wrote:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>OK, I've found something:
> >>>>>>>>>>
> >>>>>>>>>>It seems that Postfix sends a RSET command, when we expect to
> >>>>>>>>>>get the header.
> >>>>>>>>>>
> >>>>>>>>>>readheader() then waits for postfix to send more, while postfix
> >>>>>>>>>>waits for a '250 OK' Message from dbmail-lmtp.
> >>>>>>>>>>
> >>>>>>>>>>So, there's probably something wrong in our LMTP-logic.
> >>>>>>>>>>I'll take a look at it, after I've done some small scripting job
> >>>>>>>>>>here for a customer.
> >>>>>>>>>>
> >>>>>>>>>>Ilja
> >>>>>>>>>>
> >>>>>>>>>>Ilja Booij wrote:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>OK,
> >>>>>>>>>>>
> >>>>>>>>>>>this seems to have tackled the problem of the double delivery :)
> >>>>>>>>>>>
> >>>>>>>>>>>There still is another problem though:
> >>>>>>>>>>>It still seems to hang for a while on every second delivery
> >>>>>>>>>>>attempt to the LMTP daemon.
> >>>>>>>>>>>
> >>>>>>>>>>>Can you try the following scenario:
> >>>>>>>>>>>
> >>>>>>>>>>>* configure MTA to use dbmail-lmtp for delivery
> >>>>>>>>>>>* send message to a recipient
> >>>>>>>>>>>* verify that the message is received using dbmail-lmtp
> >>>>>>>>>>>* send a second message.
> >>>>>>>>>>>
> >>>>>>>>>>>What I observe here is that the second message takes a lot
> >>>>>>>>>>>longer to be delivered than the first one. The second one is
> >>>>>>>>>>>only delivered after minutes.
> >>>>>>>>>>>
> >>>>>>>>>>>I have the feeling that something is not right here..
> >>>>>>>>>>>
> >>>>>>>>>>>Ilja
> >>>>>>>>>>>
> >>>>>>>>>>>Aaron Stone wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>New versions checked in, though not extensively tested yet.
> >>>>>>>>>>>>
> >>>>>>>>>>>>Ilja Booij <ilja@ic-s.nl> said:
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>sounds ok to me :)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>Ilja
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>Aaron Stone wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>Working on it as we speak! Catch you in about an hour?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>Aaron
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>Ilja Booij <ilja@ic-s.nl> said:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>Hi,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>Aaron Stone wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>The reason for the double deliveries in the LMTP daemon is
> >>>>>>>>>>>>>>>>because the old
> >>>>>>>>>>>>>>>>insert_messages() function used to take lists of users to
> >>>>>>>>>>>>>>>>directly deliver.
> >>>>>>>>>>>>>>>>The new insert_messages() function takes a list of dsnuser
> >>>>>>>>>>>>>>>>structures, and
> >>>>>>>>>>>>>>>>expects that *either* the useridnr field is filled (for
> >>>>>>>>>>>>>>>>direct-to-useridnr
> >>>>>>>>>>>>>>>>delivery, which isn't supported by any of the frontends,
> >>>>>>>>>>>>>>>>but is supported in
> >>>>>>>>>>>>>>>>the lower layers ;-) or the address field, which is first
> >>>>>>>>>>>>>>>>checked as a
> >>>>>>>>>>>>>>>>username and then as an alias (this allows usernames in
> >>>>>>>>>>>>>>>>the form of
> >>>>>>>>>>>>>>>>'user@server' to operate without requiring an alias that
> >>>>>>>>>>>>>>>>says 'user@server
> >>>>>>>>>>>>>>>>delivers to user@server').
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>Some refactoring might be necessary, because we need to
> >>>>>>>>>>>>>>>>inform the MTA
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>whether
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>>>>or not its envelope recipients were valid before it will
> >>>>>>>>>>>>>>>>send us the message.
> >>>>>>>>>>>>>>>>This means users need to be validated before
> >>>>>>>>>>>>>>>>read_header(), which is a
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>problem
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>>>>because at the moment this vaildation happens in
> >>>>>>>>>>>>>>>>insert_messages().
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>OK, that's clear
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>Fine and dandy in pipe land, where the MTA will send the
> >>>>>>>>>>>>>>>>message
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>regardless of
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>>>>the disposition of the recipients, and only at the very
> >>>>>>>>>>>>>>>>end are error
> >>>>>>>>>>>>>>>>conditions returned as exit codes... in LMTP land we need
> >>>>>>>>>>>>>>>>to know ahead of
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>time.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>I think what I'll do is move the user validation code from
> >>>>>>>>>>>>>>>>insert_messages()
> >>>>>>>>>>>>>>>>into dsn.c, perhaps calling it dsn_check_users() or
> >>>>>>>>>>>>>>>>dsn_prepare_deliveries().
> >>>>>>>>>>>>>>>>Then, we'll have this order:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>For command-line and envelope-recipient deliveries:
> >>>>>>>>>>>>>>>>- receive addresses and store them into the dsnusers list.
> >>>>>>>>>>>>>>>>- dsn_prepare_deliveries()
> >>>>>>>>>>>>>>>>- if no deliveries are valid, return an error.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>- read_header()
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>For deliver-to header deliveries:
> >>>>>>>>>>>>>>>>- mime_readheader()
> >>>>>>>>>>>>>>>>- mail_adr_list()
> >>>>>>>>>>>>>>>>- dsn_prepare_deliveries()
> >>>>>>>>>>>>>>>>- if no deliveries are valid, return an error.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>- insert_messages()
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>I like the part where you say 'What I'll do' :)
> >>>>>>>>>>>>>>>Do you think this work will take long? I think we must
> >>>>>>>>>>>>>>>really release rc3 tomorrow (Friday, March 5th). This stuff
> >>>>>>>>>>>>>>>really has to work if we want to release.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>Ilja
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>_______________________________________________
> >>>>>>>>>>>>>>>Dbmail-dev mailing list
> >>>>>>>>>>>>>>>Dbmail-dev@dbmail.org
> >>>>>>>>>>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>_______________________________________________
> >>>>>>>>>>>>>Dbmail-dev mailing list
> >>>>>>>>>>>>>Dbmail-dev@dbmail.org
> >>>>>>>>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>_______________________________________________
> >>>>>>>>>>>Dbmail-dev mailing list
> >>>>>>>>>>>Dbmail-dev@dbmail.org
> >>>>>>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>_______________________________________________
> >>>>>>>>>>Dbmail-dev mailing list
> >>>>>>>>>>Dbmail-dev@dbmail.org
> >>>>>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>_______________________________________________
> >>>>>>>>>Dbmail-dev mailing list
> >>>>>>>>>Dbmail-dev@dbmail.org
> >>>>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>_______________________________________________
> >>>>>>>Dbmail-dev mailing list
> >>>>>>>Dbmail-dev@dbmail.org
> >>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>_______________________________________________
> >>>>>>Dbmail-dev mailing list
> >>>>>>Dbmail-dev@dbmail.org
> >>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>>>>
> >>>>>
> >>>>>
> >>>>>_______________________________________________
> >>>>>Dbmail-dev mailing list
> >>>>>Dbmail-dev@dbmail.org
> >>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>>>
> >>>>
> >>>>_______________________________________________
> >>>>Dbmail-dev mailing list
> >>>>Dbmail-dev@dbmail.org
> >>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>>
> >>>_______________________________________________
> >>>Dbmail-dev mailing list
> >>>Dbmail-dev@dbmail.org
> >>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>
> >>_______________________________________________
> >>Dbmail-dev mailing list
> >>Dbmail-dev@dbmail.org
> >>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>
> >
> >
> >
> >
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>



--

Hi,

I've found another problem :)

When dbmail-lmtp rejects a message (unknown user for instance), postfix
generates a bounce message, with "MAIL FROM: <>". This is legal (normal
for notification messages from an MTA). DBMail does not handle it,
because it cannot find an address between "<" and ">", which results in
a "500 No address found" from dbmail-lmtp.

This happens when the bounce message is sent to a user which is handled
by the same dbmail instance.

capiche?

If you have no time for this, I'll fix it tomorrow. I'm off to home now!

Ilja


Aaron Stone wrote:

> Ok, then you're still up, and I just fixed the other two sections of code :-)
>
> Aaron
>
>
> Ilja Booij <ilja@ic-s.nl> said:
>
>
>>Hi,
>>
>>(respone inline)
>>
>>Aaron Stone wrote:
>>
>>
>>>In this example, there are three recipients, pat, jones and green. Following
>>>the data command, there are only two acknowledgements. This is contrary to my
>>>understanding that each recipient had to be acknowledged:
>>>
>>> The LMTP protocol causes the server to return, after the final "." of
>>> the DATA command, one reply for each recipient. Therefore, if the
>>> queue manager is configured to use LMTP instead of SMTP when
>>> transferring messages to the delivery agents, then the delivery
>>> agents may attempt delivery to each recipient after the final "." and
>>> individually report the status for each recipient. Connections which
>>> should use the LMTP protocol are drawn in the diagram above using
>>> asterisks.
>>>
>>>So looking at jones, it is clear why: jones failed the RCPT command, and so
>>>was no longer in the recipient list when the DATA command came around. Then
>>
>>Clear :)
>>
>>
>>>read this:
>>>
>>> The additional restriction is that when there have been no successful
>>> RCPT commands in the mail transaction, the DATA command MUST fail
>>> with a 503 reply code.
>>>
>>> The change is that after the final ".", the server returns one reply
>>> for each previously successful RCPT command in the mail transaction,
>>> in the order that the RCPT commands were issued. Even if there were
>>> multiple successful RCPT commands giving the same forward-path, there
>>> must be one reply for each successful RCPT command.
>>>
>>>I think that what the first paragraph in this quote means is not that DATA is
>>>failed, per se, but that the whole conversation is flagged with a 503.
>>>
>>>So here's what I think needs to be fixed:
>>>
>>> - If there are no recipients, use 503 AND NOTHING ELSE to report it,
>>> which means changing this section of code to use a 503:
>>>
>>>if (!has_recipients)
>>> {
>>> trace(TRACE_DEBUG,"main(): no valid recipients found, cancel message.");
>>> fprintf((FILE *)stream, "554 No valid recipients.\r\n" );
>>> }
>>
>>Yep, change the 554 to 503
>>
>>
>>> - When a recipient is failed, report the error and then *remove that
>>> recipient from the dsnusers list* so that after the message is
>>> received, their failure is not reported a second time.
>>
>>Sounds good :)
>>
>>>I'll get on it this afternoon, please test it in the morning, your time!
>>
>>I will.
>>
>>Ilja
>>
>>
>>>Ilja Booij <ilja@ic-s.nl> said:
>>>
>>>
>>>
>>>>After some more reading of RFC 2033 I'm inclined to believe an LMTP
>>>>client does not get a response after sending the DATA, except for the
>>>>responses about the delivery of the message to the users.
>>>>
>>>>Example session from the RFC:
>>>>
>>>> S: 220 foo.edu LMTP server ready
>>>> C: LHLO foo.edu
>>>> S: 250-foo.edu
>>>> S: 250-PIPELINING
>>>> S: 250 SIZE
>>>> C: MAIL FROM:<chris@bar.com>
>>>> S: 250 OK
>>>>
>>>> C: RCPT TO:<pat@foo.edu>
>>>> S: 250 OK
>>>> C: RCPT TO:<jones@foo.edu>
>>>> S: 550 No such user here
>>>> C: RCPT TO:<green@foo.edu>
>>>> S: 250 OK
>>>> C: DATA
>>>> S: 354 Start mail input; end with <CRLF>.<CRLF>
>>>> C: Blah blah blah...
>>>> C: ...etc. etc. etc.
>>>> C: .
>>>> S: 250 OK
>>>> S: 452 <green@foo.edu> is temporarily over quota
>>>> C: QUIT
>>>> S: 221 foo.edu closing connection
>>>>
>>>>The important bit is after the DATA line
>>>>there are two responses from the server after the data line:
>>>>1. "250 OK": the message to chris@bar.com was delivered
>>>>2. "452 <green@foo.edu> is temporarily over quota", the message to
>>>>green@foo.edu was not delivered.
>>>>
>>>>There is no line indicating that the message was received by the server.
>>>>The server should only indicate when a message was not received correctly.
>>>>
>>>>I'll complete remove the
>>>>fprintf((FILE *)stream, "250 Message received OK\r\n"); line
>>>>
>>>>
>>>>Ilja Booij wrote:
>>>>
>>>>
>>>>
>>>>>Hi,
>>>>>
>>>>>Removing the
>>>>>"250 Message received OK" after having received the message takes care
>>>>>of the error. I can just send several messages, with
>>>>>"lmtp_cache_connection = yes" in /etc/postfix/main.cf
>>>>>
>>>>>I'm starting to wonder whether LMTP requires us to send a message if the
>>>>>message was received OK by the server.
>>>>>
>>>>>Ilja
>>>>>
>>>>>Ilja Booij wrote:
>>>>>
>>>>>
>>>>>
>>>>>>Hi,
>>>>>>
>>>>>>with some help from a guy on postfix-users@postfix.org I found something:
>>>>>>
>>>>>>The "250 Recipient <target@test01.office.fastxs.net> OK" message from
>>>>>>dbmail-lmtp is out of sync. The postfix/lmtp program stores this
>>>>>>message until the next message is sent, causing the messages between
>>>>>>postfix and dbmail-smtp to be horribly out-of-sync.
>>>>>>
>>>>>>I guess this should be quite easy to fix. I'll see if I have some time
>>>>>>to do it this weekend. Otherwise I'll do it on Monday. Unless anybody
>>>>>>else wants to do it of course ;)
>>>>>>
>>>>>>I'll be off in a few minutes. First a beer, then it's off to home :)
>>>>>>
>>>>>>Have a nice weekend!
>>>>>>
>>>>>>Ilja
>>>>>>
>>>>>>
>>>>>>Ilja Booij wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>>Hmm, I still don't really understand the problem. I've done some
>>>>>>>ngrep-ing, to see what goes over the network.
>>>>>>>
>>>>>>>This is a session that's OK:
>>>>>>>
>>>>>>>T 2004/03/05 13:45:46.324232 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>>>>>> 220 test01 DBMail LMTP service ready to rock..
>>>>>>>##
>>>>>>>T 2004/03/05 13:45:46.324696 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>>>>>>> LHLO test01.office.fastxs.net..
>>>>>>>##
>>>>>>>T 2004/03/05 13:45:46.324900 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>>>>>> 250-test01..250-PIPELINING..250-ENHANCEDSTATUSCODES..250 SIZE..
>>>>>>>#
>>>>>>>T 2004/03/05 13:45:46.325306 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>>>>>>> MAIL FROM:<ilja@earthquake.office.fastxs.net> SIZE=866..RCPT
>>>>>>>TO:<target@test01.office.fastxs.net>..DATA..
>>>>>>>#
>>>>>>>T 2004/03/05 13:45:46.325479 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>>>>>> 250 Sender <ilja@earthquake.office.fastxs.net> OK..
>>>>>>>##
>>>>>>>T 2004/03/05 13:45:46.365648 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>>>>>> 250 Recipient <target@test01.office.fastxs.net> OK..354 Start mail
>>>>>>>input; end with <CRLF>.<CRLF>..
>>>>>>>##
>>>>>>>T 2004/03/05 13:45:46.365974 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>>>>>>> Received: from earthquake.office.fastxs.net
>>>>>>>(earthquake.office.fastxs.net [10.0.1.15])...by test01.office.fastxs.n
>>>>>>> et (Postfix) with ESMTP id 394DD19F30A...for
>>>>>>><target@test01.office.fastxs.net>; Fri, 5 Mar 2004 13:45:46 +0100 (C
>>>>>>> ET)..Received: from earthquake.office.fastxs.net (localhost
>>>>>>>[127.0.0.1])...by earthquake.office.fastxs.net (Postfi
>>>>>>> x) with ESMTP id 405D4375FE...for
>>>>>>><target@test01.office.fastxs.net>; Fri, 5 Mar 2004 13:49:12 +0100
>>>>>>>(CET)..Receiv
>>>>>>> ed: (from ilja@localhost)...by earthquake.office.fastxs.net
>>>>>>>(8.12.9/8.12.9/Submit) id i25CnCo0005116...for target@
>>>>>>> test01.office.fastxs.net; Fri, 5 Mar 2004 13:49:12 +0100
>>>>>>>(CET)..Date: Fri, 5 Mar 2004 13:49:12 +0100 (CET)..From:
>>>>>>> Ilja Booij <ilja@earthquake.office.fastxs.net>..Message-Id:
>>>>>>><200403051249.i25CnCo0005116@earthquake.office.fastxs.
>>>>>>> net>..To: target@test01.office.fastxs.net..Subject: 5-3
>>>>>>>27......27.....
>>>>>>>##
>>>>>>>T 2004/03/05 13:45:46.568851 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>>>>>> 250 Message received OK..
>>>>>>>##
>>>>>>>T 2004/03/05 13:45:46.608610 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>>>>>> c..
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>If another message is sent (over the same connection):
>>>>>>>
>>>>>>>
>>>>>>>##
>>>>>>>T 2004/03/05 13:46:01.879964 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>>>>>>> RSET..
>>>>>>>##
>>>>>>>T 2004/03/05 13:46:01.880127 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>>>>>>> MAIL FROM:<ilja@earthquake.office.fastxs.net> SIZE=858..RCPT
>>>>>>>TO:<ilja@test01.office.fastxs.net>..DATA..
>>>>>>>##
>>>>>>>T 2004/03/05 13:46:01.880368 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>>>>>> 250 OK..
>>>>>>>##
>>>>>>>T 2004/03/05 13:46:01.880460 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>>>>>> 250 Sender <ilja@earthquake.office.fastxs.net> OK..
>>>>>>>##
>>>>>>>T 2004/03/05 13:46:01.883495 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>>>>>> 250 Recipient <ilja@test01.office.fastxs.net> OK..
>>>>>>>##
>>>>>>>T 2004/03/05 13:46:01.883541 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>>>>>> 354 Start mail input; end with <CRLF>.<CRLF>..
>>>>>>>##
>>>>>>>T 2004/03/05 13:47:41.920055 127.0.0.1:46715 -> 127.0.0.1:10024 [AP]
>>>>>>> RSET..QUIT..
>>>>>>>###
>>>>>>>T 2004/03/05 13:52:41.875033 127.0.0.1:10024 -> 127.0.0.1:46715 [AP]
>>>>>>> 500 Error reading header...
>>>>>>>#
>>>>>>>
>>>>>>>You can see that the client (postfix) starts with an RSET command,
>>>>>>>and starts sending MAIL_FROM, RCPT and DATA.
>>>>>>>The server responds with 250 OK to the reset, and 250 Sender OK and
>>>>>>>250 Recipient OK, and lets the client start sending (354 Start mail
>>>>>>>input).
>>>>>>>
>>>>>>>The problem is that Postfix detects that it should not send data,
>>>>>>>because the message is supposedly bounced by dbmail.. But why?
>>>>>>>
>>>>>>>Ilja
>>>>>>>
>>>>>>>Ilja Booij wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>Hi,
>>>>>>>>
>>>>>>>>Robert, thanks for your suggestion. It seems to work perfectly here.
>>>>>>>>Messages now get delivered to users without a problem.
>>>>>>>>
>>>>>>>>I'll take a look at lmtp.c to see if I can spot the difference
>>>>>>>>between the server getting a QUIT from the client and a reconnection
>>>>>>>>on a new message, and a cached connection.
>>>>>>>>
>>>>>>>>Ilja
>>>>>>>>
>>>>>>>>
>>>>>>>>Robert Theisen wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>New to the list here. But I read over the archives regarding this
>>>>>>>>>problem and I may have a little to add.
>>>>>>>>>
>>>>>>>>>I was able to get postfix to consistently deliver via lmtp to
>>>>>>>>>dbmail (even two consecutive messages to the same user) by setting
>>>>>>>>>the postfix directive lmtp_cache_connection to false instead of
>>>>>>>>>true. This would termintate the lmtp connection after each
>>>>>>>>>delivery. I assume, with lmtp_cache_connection turned on, the
>>>>>>>>>connection would die after lmtp_timeout was reached and a new
>>>>>>>>>connection would be established and the mail would get sent. This
>>>>>>>>>may have been a "neat feature" that works with Cyrus (postfix's
>>>>>>>>>preferred lmtp client) or it may be a specification of the lmtp rfc
>>>>>>>>>(I have not read it). But it seems like a simple enough fix to
>>>>>>>>>make it work. It would probably just involve having the
>>>>>>>>>conversation status fall back to a state the postfix expects after
>>>>>>>>>message delivery and waiting in that state until either the
>>>>>>>>>connection terminates or the conversation picks up again. I took
>>>>>>>>>a cursory glance at the lmtp.c and lmtpd.c code but did not have
>>>>>>>>>enough time to really dig into it.
>>>>>>>>>Good Luck. I am anxiously awaiting the 2.0 release. :)
>>>>>>>>>
>>>>>>>>>Robert Theisen
>>>>>>>>>
>>>>>>>>>Aaron Stone wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>Sure thing, I've got about an hour of time this morning. Looks
>>>>>>>>>>like you're
>>>>>>>>>>burning the midnight oil over there! Yikes!
>>>>>>>>>>
>>>>>>>>>>Aaron
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>Ilja Booij <ilja@ic-s.nl> said:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>Hi,
>>>>>>>>>>>
>>>>>>>>>>>I haven't been able to find the cause of the problem yet. I found
>>>>>>>>>>>some more info in the logs though:
>>>>>>>>>>>
>>>>>>>>>>>Mar 4 16:55:34 test01 postfix/lmtp[21340]: 3378119F30A:
>>>>>>>>>>>to=<target@test01.office.fastxs.net>,
>>>>>>>>>>>relay=localhost.fastxs.net[127.0.0.1], delay=0, status=bounced
>>>>>>>>>>>(host localhost.fastxs.net[127.0.0.1] said: 250 Recipient
>>>>>>>>>>><target@test01.office.fastxs.net> OK)
>>>>>>>>>>>
>>>>>>>>>>>apparently, postfix thinks we want to bounce the message. But
>>>>>>>>>>>why? A previous message to the same user arrives correctly..
>>>>>>>>>>>
>>>>>>>>>>>strange, looking further. Aaron, can you also take a look at this?
>>>>>>>>>>>
>>>>>>>>>>>Ilja
>>>>>>>>>>>
>>>>>>>>>>>Ilja Booij wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>OK, I've found something:
>>>>>>>>>>>>
>>>>>>>>>>>>It seems that Postfix sends a RSET command, when we expect to
>>>>>>>>>>>>get the header.
>>>>>>>>>>>>
>>>>>>>>>>>>readheader() then waits for postfix to send more, while postfix
>>>>>>>>>>>>waits for a '250 OK' Message from dbmail-lmtp.
>>>>>>>>>>>>
>>>>>>>>>>>>So, there's probably something wrong in our LMTP-logic.
>>>>>>>>>>>>I'll take a look at it, after I've done some small scripting job
>>>>>>>>>>>>here for a customer.
>>>>>>>>>>>>
>>>>>>>>>>>>Ilja
>>>>>>>>>>>>
>>>>>>>>>>>>Ilja Booij wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>OK,
>>>>>>>>>>>>>
>>>>>>>>>>>>>this seems to have tackled the problem of the double delivery :)
>>>>>>>>>>>>>
>>>>>>>>>>>>>There still is another problem though:
>>>>>>>>>>>>>It still seems to hang for a while on every second delivery
>>>>>>>>>>>>>attempt to the LMTP daemon.
>>>>>>>>>>>>>
>>>>>>>>>>>>>Can you try the following scenario:
>>>>>>>>>>>>>
>>>>>>>>>>>>>* configure MTA to use dbmail-lmtp for delivery
>>>>>>>>>>>>>* send message to a recipient
>>>>>>>>>>>>>* verify that the message is received using dbmail-lmtp
>>>>>>>>>>>>>* send a second message.
>>>>>>>>>>>>>
>>>>>>>>>>>>>What I observe here is that the second message takes a lot
>>>>>>>>>>>>>longer to be delivered than the first one. The second one is
>>>>>>>>>>>>>only delivered after minutes.
>>>>>>>>>>>>>
>>>>>>>>>>>>>I have the feeling that something is not right here..
>>>>>>>>>>>>>
>>>>>>>>>>>>>Ilja
>>>>>>>>>>>>>
>>>>>>>>>>>>>Aaron Stone wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>New versions checked in, though not extensively tested yet.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>Ilja Booij <ilja@ic-s.nl> said:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>sounds ok to me :)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>Ilja
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>Aaron Stone wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>Working on it as we speak! Catch you in about an hour?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>Aaron
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>Ilja Booij <ilja@ic-s.nl> said:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>Hi,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>Aaron Stone wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>The reason for the double deliveries in the LMTP daemon is
>>>>>>>>>>>>>>>>>>because the old
>>>>>>>>>>>>>>>>>>insert_messages() function used to take lists of users to
>>>>>>>>>>>>>>>>>>directly deliver.
>>>>>>>>>>>>>>>>>>The new insert_messages() function takes a list of dsnuser
>>>>>>>>>>>>>>>>>>structures, and
>>>>>>>>>>>>>>>>>>expects that *either* the useridnr field is filled (for
>>>>>>>>>>>>>>>>>>direct-to-useridnr
>>>>>>>>>>>>>>>>>>delivery, which isn't supported by any of the frontends,
>>>>>>>>>>>>>>>>>>but is supported in
>>>>>>>>>>>>>>>>>>the lower layers ;-) or the address field, which is first
>>>>>>>>>>>>>>>>>>checked as a
>>>>>>>>>>>>>>>>>>username and then as an alias (this allows usernames in
>>>>>>>>>>>>>>>>>>the form of
>>>>>>>>>>>>>>>>>>'user@server' to operate without requiring an alias that
>>>>>>>>>>>>>>>>>>says 'user@server
>>>>>>>>>>>>>>>>>>delivers to user@server').
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>Some refactoring might be necessary, because we need to
>>>>>>>>>>>>>>>>>>inform the MTA
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>whether
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>or not its envelope recipients were valid before it will
>>>>>>>>>>>>>>>>>>send us the message.
>>>>>>>>>>>>>>>>>>This means users need to be validated before
>>>>>>>>>>>>>>>>>>read_header(), which is a
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>problem
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>because at the moment this vaildation happens in
>>>>>>>>>>>>>>>>>>insert_messages().
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>OK, that's clear
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>Fine and dandy in pipe land, where the MTA will send the
>>>>>>>>>>>>>>>>>>message
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>regardless of
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>the disposition of the recipients, and only at the very
>>>>>>>>>>>>>>>>>>end are error
>>>>>>>>>>>>>>>>>>conditions returned as exit codes... in LMTP land we need
>>>>>>>>>>>>>>>>>>to know ahead of
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>time.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>I think what I'll do is move the user validation code from
>>>>>>>>>>>>>>>>>>insert_messages()
>>>>>>>>>>>>>>>>>>into dsn.c, perhaps calling it dsn_check_users() or
>>>>>>>>>>>>>>>>>>dsn_prepare_deliveries().
>>>>>>>>>>>>>>>>>>Then, we'll have this order:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>For command-line and envelope-recipient deliveries:
>>>>>>>>>>>>>>>>>>- receive addresses and store them into the dsnusers list.
>>>>>>>>>>>>>>>>>>- dsn_prepare_deliveries()
>>>>>>>>>>>>>>>>>>- if no deliveries are valid, return an error.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>- read_header()
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>For deliver-to header deliveries:
>>>>>>>>>>>>>>>>>>- mime_readheader()
>>>>>>>>>>>>>>>>>>- mail_adr_list()
>>>>>>>>>>>>>>>>>>- dsn_prepare_deliveries()
>>>>>>>>>>>>>>>>>>- if no deliveries are valid, return an error.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>- insert_messages()
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>I like the part where you say 'What I'll do' :)
>>>>>>>>>>>>>>>>>Do you think this work will take long? I think we must
>>>>>>>>>>>>>>>>>really release rc3 tomorrow (Friday, March 5th). This stuff
>>>>>>>>>>>>>>>>>really has to work if we want to release.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>Ilja
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>_______________________________________________
>>>>>>>>>>>>>>>>>Dbmail-dev mailing list
>>>>>>>>>>>>>>>>>Dbmail-dev@dbmail.org
>>>>>>>>>>>>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>_______________________________________________
>>>>>>>>>>>>>>>Dbmail-dev mailing list
>>>>>>>>>>>>>>>Dbmail-dev@dbmail.org
>>>>>>>>>>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>_______________________________________________
>>>>>>>>>>>>>Dbmail-dev mailing list
>>>>>>>>>>>>>Dbmail-dev@dbmail.org
>>>>>>>>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>_______________________________________________
>>>>>>>>>>>>Dbmail-dev mailing list
>>>>>>>>>>>>Dbmail-dev@dbmail.org
>>>>>>>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>_______________________________________________
>>>>>>>>>>>Dbmail-dev mailing list
>>>>>>>>>>>Dbmail-dev@dbmail.org
>>>>>>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>_______________________________________________
>>>>>>>>>Dbmail-dev mailing list
>>>>>>>>>Dbmail-dev@dbmail.org
>>>>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>_______________________________________________
>>>>>>>>Dbmail-dev mailing list
>>>>>>>>Dbmail-dev@dbmail.org
>>>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>_______________________________________________
>>>>>>>Dbmail-dev mailing list
>>>>>>>Dbmail-dev@dbmail.org
>>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>>
>>>>>>
>>>>>>_______________________________________________
>>>>>>Dbmail-dev mailing list
>>>>>>Dbmail-dev@dbmail.org
>>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>>
>>>>>_______________________________________________
>>>>>Dbmail-dev mailing list
>>>>>Dbmail-dev@dbmail.org
>>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>
>>>>_______________________________________________
>>>>Dbmail-dev mailing list
>>>>Dbmail-dev@dbmail.org
>>>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>>>
>>>
>>>
>>>
>>>
>>_______________________________________________
>>Dbmail-dev mailing list
>>Dbmail-dev@dbmail.org
>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>
>
>
>
>