Mailing List Archive

QUestion re remote concurrency
I'm running a system where we use qmail for deliveries (all remote). I've
set the concurrencyremote to 220 and increased it in conf_unusual.h, but
when I run the logs through zoverall i get an average concurrency of
between 10 and 20. We are currently handling about 90000 remote
deliveries a day on a dual 150mhz ss20 and I'm sure it could do more if I
could get the concurrency higher. Any ideas why there's such a low
average concurrency. The system is working ALL the time, so its not like
there's times where there 0 concurrency. Doing various spot-checks, i
haven't seen more than about 50 qmail-remote's running at a time. The
machine has the ram/processor to handle it, why isn't it reaching the full
220 or even coming close?

\w0zz
Re: QUestion re remote concurrency [ In reply to ]
On Wed, 19 Feb 1997, w0zz wrote:

>
> I'm running a system where we use qmail for deliveries (all remote). I've
>

In addition, I think i have an example system where the use of SMTP RSET's
to send multiple messages would be a useful feature. Our service is
basically a web based email-forwarding service, and a LOT of our customers
are forwarding to AOL, HotMail, and Juno accounts....so many that many
times we'll have thousands of messages waiting in the queue to be
delivered to these hosts due to connection queue's getting filled up from
all of our connection attempts. Qmail eventually gets them all delivered,
but the delay is quite noticable to the customers. If qmail were using
connection caching via RSET's once it got a connection it could send mail
to its hearts content, making the queue clear out a lot faster.

\w0zz
Re: QUestion re remote concurrency [ In reply to ]
> The machine has the ram/processor to handle it,

Your bottleneck is almost certainly the queue disk. Remember that every
message has to be safely written to disk before it's accepted. How many
messages are you receiving, total? What else is that disk doing? How
many I/O operations is it handling per second?

One way to reduce disk load is to stripe the queue across several disks.
All mess/*/ have to be on the same filesystem as pid/, and intd/ has to
be on the same filesystem as todo/, but bounce/ and info/*/ and local/*/
and remote/*/ can be on 70 different filesystems.

> many times we'll have thousands of messages waiting in the queue to be
> delivered to these hosts due to connection queue's getting filled up from
> all of our connection attempts.

No. A message every few seconds to AOL doesn't fill up anything.
Connection caching would make the situation much worse for them.

Probably most of your thousands of messages haven't even been tried yet.
You can verify this by searching for delivery reports in the log.

---Dan
Put an end to unauthorized mail relaying. http://pobox.com/~djb/qmail.html
Re: QUestion re remote concurrency [ In reply to ]
On 20 Feb 1997, D. J. Bernstein wrote:

> > The machine has the ram/processor to handle it,
>
> Your bottleneck is almost certainly the queue disk. Remember that every
> message has to be safely written to disk before it's accepted. How many
> messages are you receiving, total? What else is that disk doing? How
> many I/O operations is it handling per second?
>

this is definately very possible.....and it was a suspicion of mine. The
problem of course, is that this is a production mailserver, and i can't
put a new disk in and move the queue without shutting it down and waiting
for the queue to clear out.

> One way to reduce disk load is to stripe the queue across several disks.
> All mess/*/ have to be on the same filesystem as pid/, and intd/ has to
> be on the same filesystem as todo/, but bounce/ and info/*/ and local/*/
> and remote/*/ can be on 70 different filesystems.
>

So....let me see if i have this right....can i put /var/qmail on a
disksuite striped metadevice? Or are you saying to put mess/pid on one
disk, intd/todo on another, and the others on one of their own?

> > many times we'll have thousands of messages waiting in the queue to be
> > delivered to these hosts due to connection queue's getting filled up from
> > all of our connection attempts.
>
> No. A message every few seconds to AOL doesn't fill up anything.
> Connection caching would make the situation much worse for them.
>
> Probably most of your thousands of messages haven't even been tried yet.
> You can verify this by searching for delivery reports in the log.
>

No they've all attempted delivery and can't establish a connection.

The first 50 or so will succeed and then the rest fail, and you can't
connect to the remote port 25 when this happens.

\w0zz
Re: QUestion re remote concurrency [ In reply to ]
In message <Pine.SOL.3.95.970219202959.14439A-100000@big>,
w0zz <wozz@wookie.net> writes:

> No they've all attempted delivery and can't establish a connection.
>
> The first 50 or so will succeed and then the rest fail, and you can't
> connect to the remote port 25 when this happens.

Sounds like you are filling the remote SYN queue up on the receiving
end, I've seen this happen on our site PP machine before now (and no,
I have no control over it, SEP). When this happens you are stuck until
the connection requests get serviced by inetd or tcpserver.

Here's a message from Dan about this problem.

Chris
--
Christopher Samuel, IT Vulnerabilities Group, chris@rivers.dra.hmg.gb
N-115, Defence Research Agency, St Andrews Road, Great Malvern, England, UK
DISCLAIMER: I write only for myself, not for DRA. Phone: +44 1684 894644
+MIME+ +PGP+
Re: QUestion re remote concurrency [ In reply to ]
> So....let me see if i have this right....can i put /var/qmail on a
> disksuite striped metadevice?

This is what we've done: /var/qmail/queue is a 6 disk pack, configured
as two 3 disk stripes, mirroring each other. At present, it looks like
this is providing about enough I/O bandwidth to keep up with the CPU.

This is with Solstice (n€ Online) DiskSuite.

> Or are you saying to put mess/pid on one
> disk, intd/todo on another, and the others on one of their own?

I think this is what Dan was suggesting. But if you have DiskSuite
(or other kernel support for striping), that should do a better job of
balancing the load, with less admin overhead.

Tim.
Re: QUestion re remote concurrency [ In reply to ]
On Thu, 20 Feb 1997, Tim Goodwin wrote:

> > So....let me see if i have this right....can i put /var/qmail on a
> > disksuite striped metadevice?
>
> This is what we've done: /var/qmail/queue is a 6 disk pack, configured
> as two 3 disk stripes, mirroring each other. At present, it looks like
> this is providing about enough I/O bandwidth to keep up with the CPU.
>

So, 3 disks for the spool in a stripe, and 3 disks mirroring in a stripe?



\w0zz
Re: QUestion re remote concurrency [ In reply to ]
our listserv delivery machine is a ss20/712 with 384MB of memory and four
drives on the bus. /var/qmail is mounted on a mirrored 4G drive (just what
i had lying about). i have concurrency set at 255 which i frequently reach
(or smack into). overall numbers for a somewhat busier than usual day
below. if i get a >255 patch i'll try that to see if i can push the
latency down but the machine may be saturated.

Messages: 37895
Recipients: 165988
Average message tries: 4.56369
Total delivery attempts: 744661
success: 630475
failure: 8070
deferral: 106116
Message bytes: 209415658
Message bytes weighted by success: 708305748
Time span (days): 1
Average message qtime (s): 1314.47
Average xdelay (s): 18.6404
Average ddelay (s): 1269.5
Average concurrency: 160.657

--
paul
pjg@acsu.Buffalo.EDU |public keys at:
| http://urth.acsu.Buffalo.EDU/~pjg/key.html
if the above contains opinions they are mine unless marked otherwise.