Mailing List Archive: Problems with multiple smtp recipients

Problems with multiple smtp recipients

Mar 7, 2005, 5:40 AM

Post #1 of 9 (3454 views)

Hi there,

The following is an (interesting) case were the remote server denies
(or rather "postpones") some of the RCPT TO: recipient's delivery.

It appears to be related to a maximum number of recipients problem, but not quite
from the message below , so the question is:

How can I get exim to deliver the email to those it can (which the remote side said is
okay, ie 250) but retry the others (ie. the 4xx) later and report on those not
deliverable (ie. the 5xx etc.)??
I guess this might be a developers issue, as it appears to "go away" once I've dropped the
max_rcpt for the time being, though I would expect exim to notice which ones are
allowed and which ones aren't, ie. should be queued/returned.

GReetz
HEndrik

<exim -v -qff output>

Unfrozen by forced delivery
Connecting to mail3.sun.ac.za [146.232.64.13]:25 ... connected
SMTP<< 220 mail3.sun.ac.za ESMTP Exim 4.34 Sun, 06 Mar 2005 22:11:46 +0200
SMTP>> EHLO hvs.envisage.co.za
SMTP<< 250-mail3.sun.ac.za Hello wblv-165-40-22.telkomadsl.co.za [165.165.40.22]
250-SIZE 52428800
250-PIPELINING
250 HELP
SMTP>> MAIL FROM:<MAILLISTcpt-bounces@envisage.co.za> SIZE=32949
SMTP>> RCPT TO:<hhh@sun.ac.za>
SMTP>> RCPT TO:<ccccccc@ing.sun.ac.za>
SMTP>> RCPT TO:<22222222222@maties.sun.ac.za>
SMTP>> RCPT TO:<333333333@sun.ac.za>
SMTP>> RCPT TO:<44444444@sun.ac.za>
SMTP>> RCPT TO:<jjjjjjjjj@sun.ac.za>
SMTP>> RCPT TO:<55555555555@sun.ac.za>
SMTP>> RCPT TO:<uuuuuuu@sun.ac.za>
SMTP>> RCPT TO:<6666666@sun.ac.za>
SMTP>> RCPT TO:<777777@sun.ac.za>
SMTP>> DATA
SMTP<< 250 OK
SMTP<< 250 Accepted
SMTP<< 250 Accepted
SMTP<< 250 Accepted
SMTP<< 451 Try this address in the next batch, please.
LOG: MAIN
SMTP error from remote mailer after RCPT TO:<333333333@sun.ac.za>: host mail3.sun.ac.za [146.232.64.13]: 451 Try this address in the next batch, please.
2005-03-06 22:11:46 1D69At-0000lL-Gu SMTP error from remote mailer after RCPT TO:<333333333@sun.ac.za>: host mail3.sun.ac.za [146.232.64.13]: 451 Try this address in the next batch, please.
SMTP<< 250 Accepted
SMTP<< 250 Accepted
SMTP<< 250 Accepted
SMTP<< 250 Accepted
SMTP<< 250 Accepted
SMTP<< 451 Try this address in the next batch, please.
LOG: MAIN
SMTP error from remote mailer after RCPT TO:<777777@sun.ac.za>: host mail3.sun.ac.za [146.232.64.13]: 451 Try this address in the next batch, please.
LOG: MAIN
== hhh@sun.ac.za R=lookuphost T=remote_smtp defer (-1): smtp transport process returned non-zero status 0x0100: exit code 1
LOG: MAIN
== ccccccc@ing.sun.ac.za R=lookuphost T=remote_smtp defer (-1): smtp transport process returned non-zero status 0x0100: exit code 1
LOG: MAIN
== 22222222222@maties.sun.ac.za R=lookuphost T=remote_smtp defer (-1): smtp transport process returned non-zero status 0x0100: exit code 1
LOG: MAIN
== 333333333@sun.ac.za R=lookuphost T=remote_smtp defer (-1): smtp transport process returned non-zero status 0x0100: exit code 1
LOG: MAIN
== 44444444@sun.ac.za R=lookuphost T=remote_smtp defer (-1): smtp transport process returned non-zero status 0x0100: exit code 1
LOG: MAIN
== jjjjjjjjj@sun.ac.za R=lookuphost T=remote_smtp defer (-1): smtp transport process returned non-zero status 0x0100: exit code 1
LOG: MAIN
== 55555555555@sun.ac.za R=lookuphost T=remote_smtp defer (-1): smtp transport process returned non-zero status 0x0100: exit code 1
LOG: MAIN
== uuuuuuu@sun.ac.za R=lookuphost T=remote_smtp defer (-1): smtp transport process returned non-zero status 0x0100: exit code 1
LOG: MAIN
== 6666666@sun.ac.za R=lookuphost T=remote_smtp defer (-1): smtp transport process returned non-zero status 0x0100: exit code 1
LOG: MAIN
== 777777@sun.ac.za R=lookuphost T=remote_smtp defer (-1): smtp transport process returned non-zero status 0x0100: exit code 1
LOG: MAIN
Frozen
LOG: queue_run MAIN
End queue run: pid=26742 -qff
-su-2.05b#

----- End forwarded message -----

----- End forwarded message -----

Re: Problems with multiple smtp recipients [ In reply to ]

ph10 at cus

Mar 7, 2005, 7:11 AM

Post #2 of 9 (3375 views)

Permalink

On Mon, 7 Mar 2005, Hendrik Visage wrote:

> How can I get exim to deliver the email to those it can (which the remote side said is
> okay, ie 250) but retry the others (ie. the 4xx) later and report on those not
> deliverable (ie. the 5xx etc.)??

It should do that automatically. However, you do not say which release
of Exim you are talking about.

> LOG: MAIN
> == hhh@sun.ac.za R=lookuphost T=remote_smtp defer (-1): smtp transport
> process returned non-zero status 0x0100: exit code 1
^^^^^^^^^^^^^^^^^^^^^^^^

THAT is the real problem. The transport process has not ended cleanly,
so Exim defers all the addresses, independent of the responses they
received. We need more information to understand why that happened. You
need to try another delivery with debugging turned on to see if that
gives any clue.

--
Philip Hazel University of Cambridge Computing Service,
ph10@cus.cam.ac.uk Cambridge, England. Phone: +44 1223 334714.
Get the Exim 4 book: http://www.uit.co.uk/exim-book

Re: Problems with multiple smtp recipients [ In reply to ]

hvisage at envisage

Mar 7, 2005, 7:49 AM

Post #3 of 9 (3374 views)

Permalink

On Mon, Mar 07, 2005 at 02:11:39PM +0000, Philip Hazel wrote:
> On Mon, 7 Mar 2005, Hendrik Visage wrote:
>
> > How can I get exim to deliver the email to those it can (which the remote side said is
> > okay, ie 250) but retry the others (ie. the 4xx) later and report on those not
> > deliverable (ie. the 5xx etc.)??
>
> It should do that automatically. However, you do not say which release
> of Exim you are talking about.

4.31
Upgraded to 4.50, same trouble, dropped the max_rcpt to three, delivery went through.

> > LOG: MAIN
> > == hhh@sun.ac.za R=lookuphost T=remote_smtp defer (-1): smtp transport
> > process returned non-zero status 0x0100: exit code 1
> ^^^^^^^^^^^^^^^^^^^^^^^^
>
> THAT is the real problem. The transport process has not ended cleanly,
> so Exim defers all the addresses, independent of the responses they
> received. We need more information to understand why that happened. You
> need to try another delivery with debugging turned on to see if that
> gives any clue.

What type of flags are you needing on the command line of the queue run
for the report?

Re: Problems with multiple smtp recipients [ In reply to ]

hvisage at envisage

Mar 7, 2005, 8:34 AM

Post #4 of 9 (3377 views)

Permalink

On Mon, Mar 07, 2005 at 02:11:39PM +0000, Philip Hazel wrote:
> It should do that automatically. However, you do not say which release
> of Exim you are talking about.

As mentioned before currently 4.50

> > LOG: MAIN
> > == hhh@sun.ac.za R=lookuphost T=remote_smtp defer (-1): smtp transport
> > process returned non-zero status 0x0100: exit code 1
> ^^^^^^^^^^^^^^^^^^^^^^^^
>
> THAT is the real problem. The transport process has not ended cleanly,
> so Exim defers all the addresses, independent of the responses they
> received. We need more information to understand why that happened. You
> need to try another delivery with debugging turned on to see if that
> gives any clue.

exim -d -v -qff:
SMTP<< 220 mail1.sun.ac.za ESMTP Exim 4.34 Mon, 07 Mar 2005 17:21:30 +0200
146.232.64.12 in hosts_avoid_esmtp? no (option unset)
SMTP>> EHLO hvs.envisage.co.za
waiting for data on socket
read response data: size=120
SMTP<< 250-mail1.sun.ac.za Hello wblv-165-40-22.telkomadsl.co.za [165.165.40.2
2]
250-SIZE 52428800
250-PIPELINING
250 HELP
using PIPELINING
146.232.64.12 in hosts_require_auth? no (option unset)
SMTP>> MAIL FROM:<MAILINTcpt-bounces@envisage.co.za> SIZE=34560
SMTP>> RCPT TO:<111111@sun.ac.za>
SMTP>> RCPT TO:<jjjjjj@sun.ac.za>
SMTP>> RCPT TO:<22222@sun.ac.za>
SMTP>> RCPT TO:<bbbb@sun.ac.za>
SMTP>> RCPT TO:<333333@sun.ac.za>
SMTP>> RCPT TO:<4444444@sun.ac.za>
SMTP>> DATA
waiting for data on socket
read response data: size=323
SMTP<< 250 OK
SMTP<< 250 Accepted
SMTP<< 451 Try this address in the next batch, please.
LOG: MAIN
SMTP error from remote mailer after RCPT TO:<jjjjjj@sun.ac.za>: host mail1.s
un.ac.za [146.232.64.12]: 451 Try this address in the next batch, please.
2005-03-07 17:21:32 1D8IGG-0005Eg-PF SMTP error from remote mailer after RCPT TO
:<jjjjjj@sun.ac.za>: host mail1.sun.ac.za [146.232.64.12]: 451 Try this addres
s in the next batch, please.
added retry item for R:jjjjjj@sun.ac.za: errno=-44 more_errno=51,M flags=0
SMTP<< 451 Try this address in the next batch, please.
LOG: MAIN
SMTP error from remote mailer after RCPT TO:<22222@sun.ac.za>: host mail1.s
un.ac.za [146.232.64.12]: 451 Try this address in the next batch, please.
reading pipe for subprocess 22154 (not ended)
read() yielded 0
remote delivery process 22154 ended: status=0100
set_process_info: 22153 delivering 1D8IGG-0005Eg-PF
post-process 111111@sun.ac.za (1)
LOG: MAIN
== 111111@sun.ac.za R=lookuphost T=remote_smtp defer (-1): smtp transport pr
ocess returned non-zero status 0x0100: exit code 1
post-process jjjjjj@sun.ac.za (1)
LOG: MAIN

Re: Problems with multiple smtp recipients [ In reply to ]

ph10 at cus

Mar 7, 2005, 9:54 AM

Post #5 of 9 (3370 views)

Permalink

On Mon, 7 Mar 2005, Hendrik Visage wrote:

> exim -d -v -qff:

Please try without -v (which just gives very limited output, and has
overridden -d). Sorry, I should have said to use -d.

--
Philip Hazel University of Cambridge Computing Service,
ph10@cus.cam.ac.uk Cambridge, England. Phone: +44 1223 334714.
Get the Exim 4 book: http://www.uit.co.uk/exim-book

Re: Problems with multiple smtp recipients [ In reply to ]

hvisage at envisage

Mar 7, 2005, 2:04 PM

Post #6 of 9 (3400 views)

Permalink

On Mon, Mar 07, 2005 at 04:54:18PM +0000, Philip Hazel wrote:
> On Mon, 7 Mar 2005, Hendrik Visage wrote:
>
> > exim -d -v -qff:
>
> Please try without -v (which just gives very limited output, and has
> overridden -d). Sorry, I should have said to use -d.

SMTP>> MAIL FROM:<mensacpt-bounces@envisage.co.za> SIZE=34560
SMTP>> RCPT TO:<11111111@sun.ac.za>
SMTP>> RCPT TO:<jjjjjj@sun.ac.za>
SMTP>> RCPT TO:<2222222@sun.ac.za>
SMTP>> RCPT TO:<bbbbbb@sun.ac.za>
SMTP>> RCPT TO:<44444444@sun.ac.za>
SMTP>> RCPT TO:<5555555@sun.ac.za>
SMTP>> DATA
waiting for data on socket
read response data: size=323
SMTP<< 250 OK
SMTP<< 250 Accepted
SMTP<< 451 Try this address in the next batch, please.
LOG: MAIN
SMTP error from remote mailer after RCPT TO:<jjjjjj@sun.ac.za>: host mail1.sun.ac.za [146.232.64.12]: 451 Try this a
ddress in the next batch, please.
2005-03-07 22:57:43 1D8IGG-0005Eg-PF SMTP error from remote mailer after RCPT TO:<jjjjjj@sun.ac.za>: host mail1.sun.ac
.za [146.232.64.12]: 451 Try this address in the next batch, please.
added retry item for R:jjjjjj@sun.ac.za: errno=-44 more_errno=51,M flags=0
SMTP<< 451 Try this address in the next batch, please.
LOG: MAIN
SMTP error from remote mailer after RCPT TO:<2222222@sun.ac.za>: host mail1.sun.ac.za [146.232.64.12]: 451 Try this a
ddress in the next batch, please.
reading pipe for subprocess 24170 (not ended)
read() yielded 0
remote delivery process 24170 ended: status=0100
set_process_info: 24169 delivering 1D8IGG-0005Eg-PF
post-process 11111111@sun.ac.za (1)
LOG: MAIN
== 11111111@sun.ac.za R=lookuphost T=remote_smtp defer (-1): smtp transport process returned non-zero status 0x0100: e
xit code 1
post-process jjjjjj@sun.ac.za (1)
LOG: MAIN
== jjjjjj@sun.ac.za R=lookuphost T=remote_smtp defer (-1): smtp transport process returned non-zero status 0x0100: e
xit code 1

<Rest of addresses the same as the above>

Attached also my compressed config.

sh-2.05b$ exim -bV
Exim version 4.50 #1 built 06-Mar-2005 21:25:41
Copyright (c) University of Cambridge 2004
Berkeley DB: Sleepycat Software: Berkeley DB 3.3.11: (July 12, 2001)
Support for: iconv() PAM TCPwrappers
Lookups: lsearch wildlsearch nwildlsearch iplsearch dbm dbmnz mysql nis nis0
Authenticators: cram_md5 plaintext
Routers: accept dnslookup ipliteral manualroute queryprogram redirect
Transports: appendfile/maildir autoreply pipe smtp
Fixed never_users: 0
Configuration file is /etc/exim.conf

Re: Problems with multiple smtp recipients [ In reply to ]

ph10 at cus

Mar 9, 2005, 3:14 AM

Post #7 of 9 (3380 views)

Permalink

On Mon, 7 Mar 2005, Hendrik Visage wrote:

> > Please try without -v (which just gives very limited output, and has
> > overridden -d). Sorry, I should have said to use -d.

Thanks for the test. Sadly, it doesn't tell me anything.

> LOG: MAIN
> SMTP error from remote mailer after RCPT TO:<2222222@sun.ac.za>: host mail1.sun.ac.za [146.232.64.12]: 451 Try this a
> ddress in the next batch, please.
> reading pipe for subprocess 24170 (not ended)
> read() yielded 0
> remote delivery process 24170 ended: status=0100

I was hoping for some more debugging output from the delivery process.
Also, you didn't send ALL the debugging output. I know it is long, but
while trying to analyse this, I found myself wanting to look further
back. One of the problems is that there are multiple processes involved
here. Next time, please use -d+pid so that each debug line has a process
id added. That will make it easier to see which process is outputting what.

However, looking at the code, I regret that I can't see at the moment
how the subprocess could return that status and that debugging output.

There doesn't seem to be anything odd about your configuration, EXCEPT,
wait! I've seen something odd. You have

user = mail

in your smtp transport. Is "mail" your "exim" user? Ah, no, it isn't! I
see elsewhere in the config that you have

admin_groups = exim
exim_user = root

The last of these is worrying. It is not a good idea to run Exim as
root. And it is also less than usual to set the user at runtime. There
can be problems if you run remote transports as anything other than the
Exim user. The manual says, of the "user" option in transports:

For remote transports, you should leave this option unset unless you
really are sure you know what you are doing. When a remote transport is
running, it needs to be able to access Exim's hints databases, because
each host may have its own retry data.

I'm guessing now that the problem is caused because of user id
conflicts. There might be evidence of this in the complete debugging
output.

--
Philip Hazel University of Cambridge Computing Service,
ph10@cus.cam.ac.uk Cambridge, England. Phone: +44 1223 334714.
Get the Exim 4 book: http://www.uit.co.uk/exim-book

Re: Problems with multiple smtp recipients [ In reply to ]

hvisage at envisage

Mar 9, 2005, 7:26 AM

Post #8 of 9 (3357 views)

Permalink

On Wed, Mar 09, 2005 at 10:14:53AM +0000, Philip Hazel wrote:
> On Mon, 7 Mar 2005, Hendrik Visage wrote:
>
> > > Please try without -v (which just gives very limited output, and has
> > > overridden -d). Sorry, I should have said to use -d.
>
> Thanks for the test. Sadly, it doesn't tell me anything.
>

Because of the email addresses in the logs, I'll send the full log straight to
Philip (Compressed that is ;^)

Re: Problems with multiple smtp recipients [ In reply to ]

hvisage at envisage

Mar 9, 2005, 7:30 AM

Post #9 of 9 (3411 views)

Permalink

On Wed, Mar 09, 2005 at 10:14:53AM +0000, Philip Hazel wrote:
> There doesn't seem to be anything odd about your configuration, EXCEPT,
> wait! I've seen something odd. You have
>
> user = mail
>
> in your smtp transport. Is "mail" your "exim" user? Ah, no, it isn't! I
> see elsewhere in the config that you have
>
> admin_groups = exim
> exim_user = root
>
> The last of these is worrying. It is not a good idea to run Exim as

The problem: exim 4 (coming from exim3) the exim user etc. was quite problematic
to get things going :(

What I don't understand, it's only breaking for this case where the remote
mailer fails (in PIPELINING mode, i believe the remote mailer is also exim)
only on *some* of the rcpts. The moment I add the max_rcpts to 3 or less,
the emails goes through without hassles. I think the remote mailer
(sun.ac.za) might have limits on the number of incoming RCPT for spam
reasons etc.

Mailing List Archive

Mailing List Archive

Attached Files: