Hi all. This is for PCIe IO space multiplexing. Please comment.
It is not uncommon that a big iron for server consolidation has
many (e.g. > 16) PCIe slots. It will hold many domains, and
the administrator wants them to boot from pass through devices.
But currently up to 16 hvm domains can boot from pass through device.
This patch series addresses this issue by multiplexing IO space access.
The patches are composed of
Linux part: IO space ressignment code and multiplexing driver
xen part: udev script for the driver
ioemu part: make use of the PCIe io space multiplexing driver
Usage:
Add dom0 kernel command line specifying the pass through devices by
guestiomuldev=[<segment>:]<bus>:<dev>[,[<segment:><bus>:dev]][,...]
(Don't forget add related options. pciback.hide, reassign_resources,
reassigndev or guestdev.)
Details:
PCI expansion ROM BIOS often uses IO port access to boot from its device
and Linux as dom0 exclusively assigns IO space to downstream PCI bridges
and the assignment unit of PCI bridge IO space is 4K. So the only up to
16 PCIe device can be accessed via IO space within 64K IO ports.
So on virtualized environment, it means only up to 16 guest domains
can boot from such pass-through devices.
Limitation:
PCI devices or root complex integrated endpoints aren't supported.
IO port of IO shared devices can't be accessed from dom0 Linux device driver.
But this wouldn't be a big issue because PCIe specification discourages
the use of IO space and recommends that IO space should be used only
for bootable device with ROM code. OS device driver should work without
IO space access.
Test:
At present I have tested with only single multifunction PCIe because
I don't have a machine with complicated PCIe topology nor many PCIe cards.
thanks
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
It is not uncommon that a big iron for server consolidation has
many (e.g. > 16) PCIe slots. It will hold many domains, and
the administrator wants them to boot from pass through devices.
But currently up to 16 hvm domains can boot from pass through device.
This patch series addresses this issue by multiplexing IO space access.
The patches are composed of
Linux part: IO space ressignment code and multiplexing driver
xen part: udev script for the driver
ioemu part: make use of the PCIe io space multiplexing driver
Usage:
Add dom0 kernel command line specifying the pass through devices by
guestiomuldev=[<segment>:]<bus>:<dev>[,[<segment:><bus>:dev]][,...]
(Don't forget add related options. pciback.hide, reassign_resources,
reassigndev or guestdev.)
Details:
PCI expansion ROM BIOS often uses IO port access to boot from its device
and Linux as dom0 exclusively assigns IO space to downstream PCI bridges
and the assignment unit of PCI bridge IO space is 4K. So the only up to
16 PCIe device can be accessed via IO space within 64K IO ports.
So on virtualized environment, it means only up to 16 guest domains
can boot from such pass-through devices.
Limitation:
PCI devices or root complex integrated endpoints aren't supported.
IO port of IO shared devices can't be accessed from dom0 Linux device driver.
But this wouldn't be a big issue because PCIe specification discourages
the use of IO space and recommends that IO space should be used only
for bootable device with ROM code. OS device driver should work without
IO space access.
Test:
At present I have tested with only single multifunction PCIe because
I don't have a machine with complicated PCIe topology nor many PCIe cards.
thanks
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel