Mailing List Archive

Problem with queue policy?
Hi

I have noticed a little problem with qmail-0.96. If I have have
several (like 1.000) mails to the same host (which is not uncommon if
you run stuff like ftpmail), and the remote host is down or
unreachable, qmail tries to deliver every single mail. Qmail should
sense this and defer all other mails to the same system and continue
to process other mails in the queue. Right now, it takes a very long
time to try deliver these 1000 mails until it gets to the bottom of
the list and finally tries to deliver other mails.

Is there any advice in this regard?

Otherwise, qmail is a very nice system and sure beats all other
systems that I have used.

Best,
Karsten.

--
M.Sc.E.E, Sys/Net-administrator, BOFH, Aalborg University, Denmark
http://www.kom.auc.dk/~karthy/ for more information.
SunSITE Denmark Manager (http://sunsite.auc.dk/)
Re: Problem with queue policy [ In reply to ]
Hi,

The problem is very related with a remote deliver policy: it would
be nice to have only one delivery for multiple recipients on a single
remote host instead of trying to deliver a single mail for every
remote user at the same remote host. Certainly, in this way it is
possible to lower network traffic and to speed up delivery proccess
(especially, for a case of many remote recipients on the same remote
host).
> Karsten Thygesen wrote:
>
> I have noticed a little problem with qmail-0.96. If I have have
> several (like 1.000) mails to the same host (which is not uncommon if
> you run stuff like ftpmail), and the remote host is down or
> unreachable, qmail tries to deliver every single mail. Qmail should
> sense this and defer all other mails to the same system and continue
> to process other mails in the queue. Right now, it takes a very long
> time to try deliver these 1000 mails until it gets to the bottom of
> the list and finally tries to deliver other mails.
>
> ... deleted ...

========================================================
Vytautas Vysniauskas e-mail: vytasvy@osf.lt
tel: +370-2-611408
UNIX systems administrator
Open Society Fund of Lithuania,
========================================================
Re: Problem with queue policy? [ In reply to ]
> Right now, it takes a very long
> time to try deliver these 1000 mails until it gets to the bottom of
> the list and finally tries to deliver other mails.

The tcpto mechanism will kick in for each IP address after two minutes:
one minute for the first timeout, one minute for the second timeout.

Note that this applies only to timeouts, not to other connection
failures; a host-unreachable error, for example, is presumed to be
transient.

---Dan
Put an end to unauthorized mail relaying. http://pobox.com/~djb/qmail.html
Re: Problem with queue policy? [ In reply to ]
>>>>> "D" == D J Bernstein <djb@koobera.math.uic.edu> writes:

>> Right now, it takes a very long time to try deliver these 1000
>> mails until it gets to the bottom of the list and finally tries to
>> deliver other mails.

D> The tcpto mechanism will kick in for each IP address after two
D> minutes: one minute for the first timeout, one minute for the
D> second timeout.

D> Note that this applies only to timeouts, not to other connection
D> failures; a host-unreachable error, for example, is presumed to be
D> transient.

in this particular case, it was a (trancient) DNS error:

deferral: Sorry,_I_couldn't_find_any_host_by_that_name._(#4.1.2)/

Anyany - it takes quite a long time to process each message, and when
I have 1000+ for the same host, it takes a very long time to get
around the queue.

Any solutions or ideas?

Best,
Karsten

--
Karsten Thygesen Aalborg, Denmark
karthy@dannug.dk Voice: +45 40713727
SunSITE Denmark Manager (AKA karthy@sunsite.auc.dk)
<URL:http://sunsite.auc.dk/~karthy/> PGP-key: 40F2AA29
Re: Problem with queue policy? [ In reply to ]
> deferral: Sorry,_I_couldn't_find_any_host_by_that_name._(#4.1.2)/

DNS caching is the name server's job. The name server should give a
quick SERVFAIL for any lookup that's been failing repeatedly; it should
also merge lookups for the same address. I don't know what the latest
versions of BIND do.

---Dan
Put an end to unauthorized mail relaying. http://pobox.com/~djb/qmail.html
Re: Problem with queue policy? [ In reply to ]
> > deferral: Sorry,_I_couldn't_find_any_host_by_that_name._(#4.1.2)/
> DNS caching is the name server's job. The name server should give a
> quick SERVFAIL for any lookup that's been failing repeatedly; it should
> also merge lookups for the same address. I don't know what the latest
> versions of BIND do.

BIND does cache negative response if configured to do so.
There are some tricky issues regarding the timeout of negative cache
entries that are not completely worked out yet (not even in the DNS
standard), but at least it works for the case of several identical
queries in a row.

olaf
Re: Problem with queue policy? [ In reply to ]
>>>>> "Olaf" == Olaf Titz <olaf@bigred.inka.de> writes:

>> > deferral: Sorry,_I_couldn't_find_any_host_by_that_name._(#4.1.2)/
>> DNS caching is the name server's job. The name server should give a
>> quick SERVFAIL for any lookup that's been failing repeatedly; it
>> should also merge lookups for the same address. I don't know what
>> the latest versions of BIND do.

Olaf> BIND does cache negative response if configured to do so. There
Olaf> are some tricky issues regarding the timeout of negative cache
Olaf> entries that are not completely worked out yet (not even in the
Olaf> DNS standard), but at least it works for the case of several
Olaf> identical queries in a row.

I tuned nscd on Solaris to cache negative hits for a little longer
(default only 5 sec) and it made the queue-scan run considerable
faster.

Best
Karsten

--
Karsten Thygesen Aalborg, Denmark
karthy@dannug.dk Voice: +45 40713727
SunSITE Denmark Manager (AKA karthy@sunsite.auc.dk)
<URL:http://sunsite.auc.dk/~karthy/> PGP-key: 40F2AA29
Re: Problem with queue policy? [ In reply to ]
In message <y79hgj3wuho.fsf@dannug.dk>, Karsten Thygesen writes:
>I tuned nscd on Solaris to cache negative hits for a little longer
>(default only 5 sec) and it made the queue-scan run considerable
>faster.

If you ran a local nameserver instead of nscd I bet you'd get even
faster queue runs. I've found the performance of nscd to be simply
horrible, and Sun even recommends turning it off if you're on a system
with heavy DNS lookups. nscd is a nice idea for sites with big /etc/passwd
files not wanting to run NIS/NIS+ (or absolutely huge widely disperserd
NIS/NIS+ domains), but I've found in general nscd is more of a pain than
it is worth.

--Dave