Mailing List Archive: Increasing concurrent deliveries > 121

Increasing concurrent deliveries > 121

evanc at synapse

Feb 18, 1997, 11:37 AM

Post #1 of 7 (1701 views)

I can't seem to get more than 121 concurrent qmail-remote's going. I do
an 'unlimit' before starting qmail, so it should in theory be able to
open over 6600 files and over 2000 processes, but it never seems to go
above 121. I have concurrencyremote set to 200.

The only thing I can think of is that 2*121+5 is getting close to 256
open files, so does qmail itself have some limit of the maximum number it
will open?

Evan
--
Evan Champion * Director, Network Operations
mailto:evanc@synapse.net * Directeur, Exploitation du reseau
http://www.synapse.net/ * Synapse Internet

Re: Increasing concurrent deliveries > 121 [ In reply to ]

Feb 18, 1997, 11:57 AM

Post #2 of 7 (1650 views)

On Tue, 18 Feb 1997, Evan Champion wrote:
> I can't seem to get more than 121 concurrent qmail-remote's going. I do
> an 'unlimit' before starting qmail, so it should in theory be able to
> open over 6600 files and over 2000 processes, but it never seems to go
> above 121. I have concurrencyremote set to 200.

There's a documented, hard coded limit of 121 processes. If you want to
override this, peruse the sources.

-Peter

Re: Increasing concurrent deliveries > 121 [ In reply to ]

evanc at synapse

Feb 18, 1997, 12:19 PM

Post #3 of 7 (1668 views)

On Tue, 18 Feb 1997, Peter C. Norton wrote:

> There's a documented, hard coded limit of 121 processes. If you want to
> override this, peruse the sources.

Ok, it seems the limit is hard coded to 120. In order to increase it,
there's a value in conf-unusual.h that has to be changed, and you also
need to #define FD_SETSIZE to something > 256 before including
<sys/types.h>.

It seems that qmail has a hard limit of 255 processes as well.

I'll try defining FD_SETSIZE to 1024 and see what happens.

Evan
--
Evan Champion * Director, Network Operations
mailto:evanc@synapse.net * Directeur, Exploitation du reseau
http://www.synapse.net/ * Synapse Internet

Re: Increasing concurrent deliveries > 121 [ In reply to ]

evanc at synapse

Feb 18, 1997, 1:02 PM

Post #4 of 7 (1662 views)

On Tue, 18 Feb 1997, Evan Champion wrote:

> I'll try defining FD_SETSIZE to 1024 and see what happens.

I changed conf-cc.sh to have CC="cc -DFD_SETSIZE=1024 ..." and then upped
the conf-unusual.h parameter to 255. It worked OK for a while, but then
qmail-rspawn started sucking all availble CPU and when I finally killed
it it, qmail reported it was waiting on 158 deliveries to finish when
there were only 20 running.

Evan
--
Evan Champion * Director, Network Operations
mailto:evanc@synapse.net * Directeur, Exploitation du reseau
http://www.synapse.net/ * Synapse Internet

Re: Increasing concurrent deliveries > 121 [ In reply to ]

Feb 19, 1997, 1:54 AM

Post #5 of 7 (1652 views)

On Tue, 18 Feb 1997, Evan Champion wrote:

> > I'll try defining FD_SETSIZE to 1024 and see what happens.
>
> I changed conf-cc.sh to have CC="cc -DFD_SETSIZE=1024 ..." and then upped
> the conf-unusual.h parameter to 255. It worked OK for a while, but then
> qmail-rspawn started sucking all availble CPU and when I finally killed
> it it, qmail reported it was waiting on 158 deliveries to finish when
> there were only 20 running.

"now let's see if my VolksWagen bug can pull this semi-trailer. woops!
where did my rear bumper go?"

Re: Increasing concurrent deliveries > 121 [ In reply to ]

brian at hyperreal

Feb 20, 1997, 1:04 AM

Post #6 of 7 (1651 views)

On Wed, 19 Feb 1997, Ira Abramov (at work) wrote:
> On Tue, 18 Feb 1997, Evan Champion wrote:
>
> > > I'll try defining FD_SETSIZE to 1024 and see what happens.
> >
> > I changed conf-cc.sh to have CC="cc -DFD_SETSIZE=1024 ..." and then upped
> > the conf-unusual.h parameter to 255. It worked OK for a while, but then
> > qmail-rspawn started sucking all availble CPU and when I finally killed
> > it it, qmail reported it was waiting on 158 deliveries to finish when
> > there were only 20 running.
>
> "now let's see if my VolksWagen bug can pull this semi-trailer. woops!
> where did my rear bumper go?"

I say there's nothing more admirable than trying to squeeze as much juice
out of a given platform as possible. As I said before, with proper
tuning and software design... you'd be surprised. At the very least,
qmail thinking there were more processes waiting to finish than there
actually would be something I'd call a bug.

Brian

--=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=--
brian@hyperreal.com http://www.apache.org http://www.organic.com/jobs

Re: Increasing concurrent deliveries > 121 [ In reply to ]

Feb 20, 1997, 9:38 AM

Post #7 of 7 (1672 views)

> qmail thinking there were more processes waiting to finish than there
> actually would be something I'd call a bug.

Huh? You can't feed random -D options to the compiler and expect
anything to work. (Hey, let's try cc -Dstruct=union!)

In this case, apparently the kernel's select() implementation can't deal
with descriptors past 255.

---Dan
Put an end to unauthorized mail relaying. http://pobox.com/~djb/qmail.html