Mailing List Archive

[Bug 1227] Out of SW-IOMMU space, 3ware Raid controller/3w-xxxx module
http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1227





------- Comment #3 from fthain@telegraphics.com.au 2011-09-14 23:46 -------
Here's some more background for this bug...

http://lists.xensource.com/archives/html/xen-users/2008-08/msg00109.html
http://lists.xensource.com/archives/html/xen-users/2010-11/msg00377.html
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=606968
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=603727
https://bugzilla.redhat.com/show_bug.cgi?id=236922

Some of these reports seem to be associated with IBM xSeries RSA2 cards, some
with SAS SCSI DMA. I encountered both manifestations on one particular machine
(two different dom0 kernel builds). Detailed report to follow.

I think this bug report is wrongly categorised as "Hardware support"; the
problem seems to be with the software IOMMU.


--
Configure bugmail: http://bugzilla.xensource.com/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

_______________________________________________
Xen-bugs mailing list
Xen-bugs@lists.xensource.com
http://lists.xensource.com/xen-bugs
[Bug 1227] Out of SW-IOMMU space, 3ware Raid controller/3w-xxxx module [ In reply to ]
http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1227





------- Comment #4 from fthain@telegraphics.com.au 2011-09-15 00:07 -------
Created an attachment (id=1019)
--> (http://bugzilla.xensource.com/bugzilla/attachment.cgi?id=1019&action=view)
dom0 dmesg and xm dmesg output


--
Configure bugmail: http://bugzilla.xensource.com/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

_______________________________________________
Xen-bugs mailing list
Xen-bugs@lists.xensource.com
http://lists.xensource.com/xen-bugs
[Bug 1227] Out of SW-IOMMU space, 3ware Raid controller/3w-xxxx module [ In reply to ]
http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1227





------- Comment #5 from fthain@telegraphics.com.au 2011-09-15 00:30 -------
Created an attachment (id=1020)
--> (http://bugzilla.xensource.com/bugzilla/attachment.cgi?id=1020&action=view)
xen/stable-2.6.32.x branch dom0 aacraid crash (October 2010)


--
Configure bugmail: http://bugzilla.xensource.com/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

_______________________________________________
Xen-bugs mailing list
Xen-bugs@lists.xensource.com
http://lists.xensource.com/xen-bugs
[Bug 1227] Out of SW-IOMMU space, 3ware Raid controller/3w-xxxx module [ In reply to ]
http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1227





------- Comment #6 from fthain@telegraphics.com.au 2011-09-15 00:41 -------

This bug also affects an IBM x3550 server I've been maintaining. dmesg and xm
dmesg attached. Some specs:

Gentoo Hardened amd64 Xen-4.1.1

dom0 linux-2.6.32.45 kernel from Jeremy Fitzhardinge's xen/stable-2.6.32.x
branch commit 20a27c1e25b8550066902c9d6ca91631e656dfa3

There's no problem running vanilla 2.6.32.45 on bare metal.

The machine seems to be running OK with the above configuration (in testing for
14 hours and no crash yet, see first attachment), but when it was running Xen
4.0.1 and an earlier 2.6.32.24 dom0 build from xen-stable/2.6.32.x, the aacraid
driver failed immediately (see second attachment) in the same way reported by
others (apparently DMA buffer exhaustion):

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=596419

I'm a bit concerned that it will fail again. Can anyone recommend a suitable
value for swiotlb argument? Is it likely to help?

Here's some specs, in case it helps...

# lspci
00:00.0 Host bridge: Intel Corporation 5000X Chipset Memory Controller Hub (rev
31)
00:02.0 PCI bridge: Intel Corporation 5000 Series Chipset PCI Express x8 Port
2-3 (rev 31)
00:03.0 PCI bridge: Intel Corporation 5000 Series Chipset PCI Express x4 Port 3
(rev 31)
00:04.0 PCI bridge: Intel Corporation 5000 Series Chipset PCI Express x8 Port
4-5 (rev 31)
00:05.0 PCI bridge: Intel Corporation 5000 Series Chipset PCI Express x4 Port 5
(rev 31)
00:06.0 PCI bridge: Intel Corporation 5000 Series Chipset PCI Express x4 Port 6
(rev 31)
00:07.0 PCI bridge: Intel Corporation 5000 Series Chipset PCI Express x4 Port 7
(rev 31)
00:08.0 System peripheral: Intel Corporation 5000 Series Chipset DMA Engine
(rev 31)
00:10.0 Host bridge: Intel Corporation 5000 Series Chipset FSB Registers (rev
31)
00:10.1 Host bridge: Intel Corporation 5000 Series Chipset FSB Registers (rev
31)
00:10.2 Host bridge: Intel Corporation 5000 Series Chipset FSB Registers (rev
31)
00:11.0 Host bridge: Intel Corporation 5000 Series Chipset Reserved Registers
(rev 31)
00:13.0 Host bridge: Intel Corporation 5000 Series Chipset Reserved Registers
(rev 31)
00:15.0 Host bridge: Intel Corporation 5000 Series Chipset FBD Registers (rev
31)
00:16.0 Host bridge: Intel Corporation 5000 Series Chipset FBD Registers (rev
31)
00:1c.0 PCI bridge: Intel Corporation 631xESB/632xESB/3100 Chipset PCI Express
Root Port 1 (rev 09)
00:1d.0 USB Controller: Intel Corporation 631xESB/632xESB/3100 Chipset UHCI USB
Controller #1 (rev 09)
00:1d.1 USB Controller: Intel Corporation 631xESB/632xESB/3100 Chipset UHCI USB
Controller #2 (rev 09)
00:1d.2 USB Controller: Intel Corporation 631xESB/632xESB/3100 Chipset UHCI USB
Controller #3 (rev 09)
00:1d.7 USB Controller: Intel Corporation 631xESB/632xESB/3100 Chipset EHCI
USB2 Controller (rev 09)
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev d9)
00:1f.0 ISA bridge: Intel Corporation 631xESB/632xESB/3100 Chipset LPC
Interface Controller (rev 09)
00:1f.1 IDE interface: Intel Corporation 631xESB/632xESB IDE Controller (rev
09)
00:1f.3 SMBus: Intel Corporation 631xESB/632xESB/3100 Chipset SMBus Controller
(rev 09)
01:01.0 VGA compatible controller: ATI Technologies Inc ES1000 (rev 02)
02:00.0 RAID bus controller: Adaptec AAC-RAID (Rocket) (rev 02)
03:00.0 PCI bridge: Broadcom EPB PCI-Express to PCI-X Bridge (rev c3)
04:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5708 Gigabit
Ethernet (rev 12)
05:00.0 PCI bridge: Broadcom EPB PCI-Express to PCI-X Bridge (rev c3)
06:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5708 Gigabit
Ethernet (rev 12)
10:00.0 PCI bridge: Intel Corporation 6311ESB/6321ESB PCI Express Upstream Port
(rev 01)
10:00.3 PCI bridge: Intel Corporation 6311ESB/6321ESB PCI Express to PCI-X
Bridge (rev 01)
11:00.0 PCI bridge: Intel Corporation 6311ESB/6321ESB PCI Express Downstream
Port E1 (rev 01)
11:01.0 PCI bridge: Intel Corporation 6311ESB/6321ESB PCI Express Downstream
Port E2 (rev 01)


--
Configure bugmail: http://bugzilla.xensource.com/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

_______________________________________________
Xen-bugs mailing list
Xen-bugs@lists.xensource.com
http://lists.xensource.com/xen-bugs
[Bug 1227] Out of SW-IOMMU space, 3ware Raid controller/3w-xxxx module [ In reply to ]
http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1227





------- Comment #7 from konrad.wilk@oracle.com 2011-09-15 01:32 -------
I am pretty sure I responded to you on the mailing list. Did you read my reply
there.

http://lists.xensource.com/archives/html/xen-devel/2011-09/msg00739.html


--
Configure bugmail: http://bugzilla.xensource.com/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

_______________________________________________
Xen-bugs mailing list
Xen-bugs@lists.xensource.com
http://lists.xensource.com/xen-bugs
[Bug 1227] Out of SW-IOMMU space, 3ware Raid controller/3w-xxxx module [ In reply to ]
http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1227


konrad.wilk@oracle.com changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution| |INVALID




------- Comment #8 from konrad.wilk@oracle.com 2011-09-15 01:37 -------
So.. why are you not using the default values (64MB)

I see you have swiotlb=128 which ends up with:

DMA: Placing 0MB software IO TLB between ffff880020000000 - ffff880020040000
DMA: software IO TLB at phys 0x20000000 - 0x20040000

which of course means that there is not enough space. 0MB will
cause issues like this. The parameter (if you look in the
kernel-parameters.txt) defines it as -pages_, not megabytes.

So either leave it as default, or change it to larger value, like swiotlb=65535


--
Configure bugmail: http://bugzilla.xensource.com/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

_______________________________________________
Xen-bugs mailing list
Xen-bugs@lists.xensource.com
http://lists.xensource.com/xen-bugs