Forwarding to the list... Does anybody else have any recommendations for
performance tuning.
(I should have known... open my mouth to say how great of experience I've
had tuning certain configs, and everybody wants to know how :-)
---------- Forwarded message ----------
Date: Fri, 29 Dec 2000 15:41:06 -0600 (CST)
From: Michael E Brown <michael_e_brown@dell.com>
To: Pat O'Rourke <orourke@mclinux.com>
Cc: Michael E Brown <michael_e_brown@exchange.dell.com>
Subject: Re: dual CPU balancer
How much memory do you have? How fast of network links? There are some
kernel parameters you can tune in 2.2 that help out, and there are even
more in 2.4. From the top of my head,
1) /proc/sys/net/core/*mem* <-- tune to your memory spec. The defaults are
not optimized for network throughput on large memory machines.
2.) 2.4 only /proc/sys/net/ipv4/*mem*
3.) For fast links, with multiple adapters (Two gig links, dual CPU) 2.4
has NIC-->CPU IRQ binding. That can really help also on heavily loaded
links.
4.) For 2.2 I think I would go into your BIOS or RCU (if you have one) and
hardcode all NIC adapters (Assuming identical/multiple NICS) to the same
IRQ. You get some gain due to cache affinity, and one interrupt may
service IRQs from multiple adapters in one go, on heavily loaded links.
5.) Think "Interrupt coalescing". Figure out how your adapter driver turns
this on and do it. If you are using Intel Gig links, I can send you some
info on how to tune it. Acenic Gig adapters are pretty well documented.
For a really good tuning guide, go to spec.org, and look up the latest TUX
benchmark results posted by Dell. Each benchmark posting has a full list
of kernel parameters that were tuned. This will give you a good starting
point from which to examine your configuration.
The other obvious tuning recommendation: Pick a stable 2.4 kernel and use
that. Any (untuned) 2.4 kernel will blow away 2.2 in a multiprocessor
configuration. If I remember correctly 2.4.0test 10-11 are pretty stable.
--
Michael Brown
Linux DWS Development
Dell Computer Corp
On Fri, 29 Dec 2000, Pat O'Rourke wrote:
>
> > Make that _reality_. In some tests I've done with FTP, I have seen
> > *significant* improvements using dual and quad processors using 2.4. Under
> > 2.2, there are improvements, but not astonishing ones.
>
> Oddly enough, we've seen a performance degradation on 2.2 when using a dual
> CPU machine for the director. It seems the _same_ machine performs much
> better when booted as a uniprocessor?!?
>
> We are runnig 2.2.17 with the 1.0.2 LVS patch applied. Are there any
> obvious settings we should adjust when running LVS on an SMP machine?
>
> Thanks,
>
> Pat
>
>
performance tuning.
(I should have known... open my mouth to say how great of experience I've
had tuning certain configs, and everybody wants to know how :-)
---------- Forwarded message ----------
Date: Fri, 29 Dec 2000 15:41:06 -0600 (CST)
From: Michael E Brown <michael_e_brown@dell.com>
To: Pat O'Rourke <orourke@mclinux.com>
Cc: Michael E Brown <michael_e_brown@exchange.dell.com>
Subject: Re: dual CPU balancer
How much memory do you have? How fast of network links? There are some
kernel parameters you can tune in 2.2 that help out, and there are even
more in 2.4. From the top of my head,
1) /proc/sys/net/core/*mem* <-- tune to your memory spec. The defaults are
not optimized for network throughput on large memory machines.
2.) 2.4 only /proc/sys/net/ipv4/*mem*
3.) For fast links, with multiple adapters (Two gig links, dual CPU) 2.4
has NIC-->CPU IRQ binding. That can really help also on heavily loaded
links.
4.) For 2.2 I think I would go into your BIOS or RCU (if you have one) and
hardcode all NIC adapters (Assuming identical/multiple NICS) to the same
IRQ. You get some gain due to cache affinity, and one interrupt may
service IRQs from multiple adapters in one go, on heavily loaded links.
5.) Think "Interrupt coalescing". Figure out how your adapter driver turns
this on and do it. If you are using Intel Gig links, I can send you some
info on how to tune it. Acenic Gig adapters are pretty well documented.
For a really good tuning guide, go to spec.org, and look up the latest TUX
benchmark results posted by Dell. Each benchmark posting has a full list
of kernel parameters that were tuned. This will give you a good starting
point from which to examine your configuration.
The other obvious tuning recommendation: Pick a stable 2.4 kernel and use
that. Any (untuned) 2.4 kernel will blow away 2.2 in a multiprocessor
configuration. If I remember correctly 2.4.0test 10-11 are pretty stable.
--
Michael Brown
Linux DWS Development
Dell Computer Corp
On Fri, 29 Dec 2000, Pat O'Rourke wrote:
>
> > Make that _reality_. In some tests I've done with FTP, I have seen
> > *significant* improvements using dual and quad processors using 2.4. Under
> > 2.2, there are improvements, but not astonishing ones.
>
> Oddly enough, we've seen a performance degradation on 2.2 when using a dual
> CPU machine for the director. It seems the _same_ machine performs much
> better when booted as a uniprocessor?!?
>
> We are runnig 2.2.17 with the 1.0.2 LVS patch applied. Are there any
> obvious settings we should adjust when running LVS on an SMP machine?
>
> Thanks,
>
> Pat
>
>