Mailing List Archive: Hyperthreading network benchmark

Hyperthreading network benchmark

Oct 30, 2004, 6:32 PM

Post #1 of 4 (339 views)

Hi all,

I found enabling hyperthreading for Xen (the hypervisor layer) would
degrade network performance. I wonder if the synchronization cost of Xen
(the hypervisor) running on multi cores/multiprocessors environment is
high (around 30 %). The benchmark I did is against Intel Pentium 2.4 MHz
Hyperthreaded CPU and Gigabit network (ethernet and switch). I was using
WebStone 2.5 benchmark against Apache 2.0.50-1.0 (FC1 httpd package). I
was also running 2.4.27 for the domains kernel. The result I have:
1. Native:
a. Hyperthread enabled: 370 Mb/sec
b. Hyperthread disabled: 270-290 Mb/sec
2. Domain 0 has the same performance around: 270 Mb/sec
3. Domain 1:
a, Hyperthreaded enabled, domain1 run on different core: 175-185 Mb/sec
b. Hyperthreaded disabled or even if enabled domain 1 is forced to
be in the same Core with domain 0: 255-265 Mb/sec

Please note, I was only sending request to one domain at a time. It
seems when the two domain ran in the same core we had higher
performance. I wonder if any other people has done almost similar test
and have similiar/opposite behavior. Are these behaviors make sense?

Thanks in advance for any comments.

Kuas.

-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

Re: Hyperthreading network benchmark [ In reply to ]

kmacy at eventdriven

Oct 30, 2004, 6:46 PM

Post #2 of 4 (337 views)

Permalink

They've seen anywhere from a 30% decrease to 30% increase in performance
on SPECint with hyperthreading turned on. Bear in mind you're now
sharing a single cache between two processes. For many workloads the
trace cache on the P4 was too small to begin with.

-Kip

On Sat, 30 Oct 2004, kuas wrote:

> Hi all,
>
> I found enabling hyperthreading for Xen (the hypervisor layer) would
> degrade network performance. I wonder if the synchronization cost of Xen
> (the hypervisor) running on multi cores/multiprocessors environment is
> high (around 30 %). The benchmark I did is against Intel Pentium 2.4 MHz
> Hyperthreaded CPU and Gigabit network (ethernet and switch). I was using
> WebStone 2.5 benchmark against Apache 2.0.50-1.0 (FC1 httpd package). I
> was also running 2.4.27 for the domains kernel. The result I have:
> 1. Native:
> a. Hyperthread enabled: 370 Mb/sec
> b. Hyperthread disabled: 270-290 Mb/sec
> 2. Domain 0 has the same performance around: 270 Mb/sec
> 3. Domain 1:
> a, Hyperthreaded enabled, domain1 run on different core: 175-185 Mb/sec
> b. Hyperthreaded disabled or even if enabled domain 1 is forced to
> be in the same Core with domain 0: 255-265 Mb/sec
>
> Please note, I was only sending request to one domain at a time. It
> seems when the two domain ran in the same core we had higher
> performance. I wonder if any other people has done almost similar test
> and have similiar/opposite behavior. Are these behaviors make sense?
>
> Thanks in advance for any comments.
>
> Kuas.
>
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by:
> Sybase ASE Linux Express Edition - download now for FREE
> LinuxWorld Reader's Choice Award Winner for best database on Linux.
> http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xen-devel
>

-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

Re: Hyperthreading network benchmark [ In reply to ]

mark.williamson at cl

Oct 30, 2004, 8:05 PM

Post #3 of 4 (331 views)

Permalink

> I found enabling hyperthreading for Xen (the hypervisor layer) would
> degrade network performance. I wonder if the synchronization cost of Xen
> (the hypervisor) running on multi cores/multiprocessors environment is
> high (around 30 %).

If the domain controlling the network interface is on a different (logical)
processor to the domain which is currently using it, there is an overhead
because inter-processor interrupts (IPIs) must be sent to notify the other
CPU when data is ready.

Xen's pipelined, asynchronous device channels attempt to avoid this problem
but some workloads (e.g. small packets at a very high rate) will still
unavoidably stress the system.

We found in tests that hyperthreading improved performance over simply using
one core, when running dom0 on one hyperthread and dom1 on another. An IPI
is still sent between the two logical CPUs, which still has overhead, however
the performace seemed to be improved because context switches were avoided,
relative to time-slicing both domains on one CPU.

It's interesting that performance reduced for you when enabling HT. Perhaps
the context-swtich vs IPI tradeoff is different for you, either for hardware
reasons (we were using 3GHz Xeons, so had a different core, larger caches,
etc.) or because of some software change.

My $0.02
Mark

-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

Re: Hyperthreading network benchmark [ In reply to ]

doogie at debian

Nov 1, 2004, 3:56 PM

Post #4 of 4 (330 views)

Permalink

On Sat, 30 Oct 2004, Kip Macy wrote:

> They've seen anywhere from a 30% decrease to 30% increase in performance
> on SPECint with hyperthreading turned on. Bear in mind you're now
> sharing a single cache between two processes. For many workloads the
> trace cache on the P4 was too small to begin with.

Not just 2 processes. 2 whole operating systems. An operating system has a
much higher overhead, and you'll probably get lots of cache trashing.

Note, once xen supports SMP, then it might help if it's scheduler were
HT-aware. Also, why not use normal linux schedulers(don't know if you
currently are)?

-------------------------------------------------------
This SF.Net email is sponsored by:
Sybase ASE Linux Express Edition - download now for FREE
LinuxWorld Reader's Choice Award Winner for best database on Linux.
http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel