Mailing List Archive

[Bug 1227] Out of SW-IOMMU space, 3ware Raid controller/3w-xxxx module
http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1227


fthain@telegraphics.com.au changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |fthain@telegraphics.com.au




------- Comment #9 from fthain@telegraphics.com.au 2011-10-13 04:44 -------
Hi Konrad,

Thanks for your response.

> I am pretty sure I responded to you on the mailing list. Did you read my reply
> there.
>
> http://lists.xensource.com/archives/html/xen-devel/2011-09/msg00739.html

No, "Fantu" is someone else.

> So.. why are you not using the default values (64MB)

Using the default value crashed out on the x3550 when aacraid died (out of
SW-IOMMU space). The backtrace revealed that the problem was apparently DMA
buffer shortage. (So I don't see why this bug report is marked
"RESOLVED/INVALID".)

> I see you have swiotlb=128 which ends up with:
>
> DMA: Placing 0MB software IO TLB between ffff880020000000 - ffff880020040000
> DMA: software IO TLB at phys 0x20000000 - 0x20040000

I was wondering about that "0MB"...

> which of course means that there is not enough space. 0MB will
> cause issues like this. The parameter (if you look in the
> kernel-parameters.txt) defines it as -pages_, not megabytes.

I know, I already searched the kernel Documentation/ subdir before posting
here. It says,
" swiotlb=<pages>[,force]
<pages> Prereserve that many 128K pages for the software IO
bounce buffering."

Of course, 128 * 128K = 16 MB not 0 MB... I never did figure that out.

> So either leave it as default, or change it to larger value, like swiotlb=65535

Seems that the default isn't documented. Do you happen to know what it is?

I can try swiotlb=65535. I suppose this would yield about 512 MB. Assuming that
pages are in fact 8 K not 128 K. Is this correct?

Why did you subtract one from the power of two?


--
Configure bugmail: http://bugzilla.xensource.com/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

_______________________________________________
Xen-bugs mailing list
Xen-bugs@lists.xensource.com
http://lists.xensource.com/xen-bugs
[Bug 1227] Out of SW-IOMMU space, 3ware Raid controller/3w-xxxx module [ In reply to ]
http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1227


fthain@telegraphics.com.au changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|RESOLVED |REOPENED
Resolution|INVALID |




------- Comment #10 from fthain@telegraphics.com.au 2011-10-13 05:04 -------
It seems that swiotlb=128 works fine after the x3550 was upgraded to 16 GB RAM:

# cat /proc/cmdline
root=/dev/sda3 iommu=soft swiotlb=128 dom0_max_vcpus=1 dom0_vcpus_pin
console=tty0 console=ttyS0,38400 ipv6.disable=1
# cat /proc/version
Linux version 2.6.32.45-core2-dom0 (root@stinky) (gcc version 4.4.4 (Gentoo
Hardened 4.4.4-r2 p1.2, pie-0.4.5) ) #1 SMP Tue Aug 30 17:59:41 EST 2011
# uptime
22:54:37 up 28 days, 22:07, 1 user, load average: 0.00, 0.00, 0.00

The crash in the attachment (xen-stable 2.6.32.24) was back when this machine
only had 4 GB RAM. And since reporting this bug, I've also had an IBM x346 with
4 GB RAM crash in the same way (xen-stable 2.6.32.45).

I'm re-opening this bug because the "pages, not megabytes" argument seems to
hold no water.


--
Configure bugmail: http://bugzilla.xensource.com/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

_______________________________________________
Xen-bugs mailing list
Xen-bugs@lists.xensource.com
http://lists.xensource.com/xen-bugs
[Bug 1227] Out of SW-IOMMU space, 3ware Raid controller/3w-xxxx module [ In reply to ]
http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1227





------- Comment #11 from fthain@telegraphics.com.au 2011-10-13 05:28 -------
Seems I quoted the wrong documention above... the xen-stable tree has this
commit:

commit f4d4fde49bb47b4db49c211c7c74471eedb86425
Author: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Date: Fri Jan 29 10:59:10 2010 -0500

[swiotlb] Make 'setup_io_tlb_npages' accept new 'swiotlb=' syntax.

The old syntax for 'swiotlb' is still in effect, and we extend it
now to include the overflow buffer size. The syntax is now:

swiotlb=[force,][nslabs=<pages>,][overflow=<size>] or more
commonly know as:

swiotlb=[force]
swiotlb=[nslabs=<pages>]
swiotlb=[overflow=<size>]


But the code doesn't match the doco:

iommu options only relevant to the software bounce buffering (SWIOTLB) IOMMU
implementation:
swiotlb=[npages=<pages>]
swiotlb=[force]
swiotlb=[overflow=<size>]

...versus

/* The new syntax: swiotlb=nslabs=16384,overflow=32768,force */
val = get_value("nslabs", str, &str);


--
Configure bugmail: http://bugzilla.xensource.com/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

_______________________________________________
Xen-bugs mailing list
Xen-bugs@lists.xensource.com
http://lists.xensource.com/xen-bugs
[Bug 1227] Out of SW-IOMMU space, 3ware Raid controller/3w-xxxx module [ In reply to ]
http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1227


fthain@telegraphics.com.au changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|REOPENED |RESOLVED
Resolution| |INVALID




------- Comment #12 from fthain@telegraphics.com.au 2011-10-13 08:45 -------
> > So.. why are you not using the default values (64MB)

I mistakenly wrote:

> Using the default value crashed out on the x3550 when aacraid died (out of
> SW-IOMMU space). The backtrace revealed that the problem was apparently DMA
> buffer shortage. (So I don't see why this bug report is marked
> "RESOLVED/INVALID".)

Sorry, I misremembered the sequence of events. I think wasn't using the default
at the time of the aacraid crash a year ago.

So to better answer your question, I wasn't using the default because back in
2007 when this system was running an OpenSUSE dom0 kernel, the swiotlb
parameter was required in order to make aacraid work at all. Everything has
been upgraded since but the kernel arguments were never revisited. The bug may
well be invalid. I need to do some more tests.

Regarding the 64 MB default, is that 512 x 128 K or 8192 x 8K? Is it equivalent
to swiotlb=nslabs=8192? And is that the same as swiotlb=8192?


--
Configure bugmail: http://bugzilla.xensource.com/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

_______________________________________________
Xen-bugs mailing list
Xen-bugs@lists.xensource.com
http://lists.xensource.com/xen-bugs