Mailing List Archive

On Mon, 2011-10-10 at 23:02 +0100, Andy Burns wrote:
> [Resent from subscribed address]
>
> Recently had passthrough of 2xPCI DVB-T cards and 1xPCIe DVB-S2 card working,
> the last know config that was *certainly* working was
>
> dom0
> xen-4.1.1-3.fc16.x86_64
> kernel-3.1.0-0.rc7.git0.0.fc16.x86_64
>
> domU
> kernel-3.1.0-0.rc8.git0.0.fc16.x86_64
>
> Since then I've updated
>
> xen-4.1.1-6.fc16.x86_64 on dom0
>
> kernel-3.1.0-0.rc9.git0.0.fc16.x86_64 on dom0 and domU
>
> and updated all other packages to current F16 updates-testing,
> also lots of fiddling with grub2 and systemd on the domU
>
> Only today did I realise that only the PCIe card is now working, not
> the PCI ones, and have since spent several hours trying to get back to
> a working configuration :-(
>
> I've rolled my xen packages back from 4.1.1-6 to 4.1.1-3
> tried booting various combinations of 3.1.0-rc7/rc8/rc9 as dom0 and domU
>
> made sure I still have "pci=resource_alignment" for all the relevant
> PCI devices on the dom0
> made sure I still have "iommu=soft" on the domU
> made sure pci-back is happy with "xm pci-list-assignable-devices"
> made sure devices really have been assigned with "xm pci-list mythf16"
> made sure the devices and drivers show in the domU with "lspci",
> "lsusb" and "lsmod"
> checked "xm dmesg" on dom0 and "dmesg" on dom0 and domU that drivers
> see the hardware and load firmware OK
> scandvb goes through the motions of tuning, but finds no stations,
> this *feels* as though the issue is lack of DMA transfers.
>
> How can I tell if the iommu=soft is taking effect?
> Anything stupid I sound like I've forgotten?

It looks as if you have been pretty thorough.

One thing which is not clear is if you also downgraded all the Xen
utilities/tools packages when you say "Xen packages" or just the
hypervisor itself. (You probably downgraded everything, but I have to
ask).

Does Fedora (yum?) log which packages which it is upgrading? Is it
possible that e.g. scandvb or the firmware package has also been
upgraded (I admit this is slightly straw-clutching).

If you don't passthrough a device are you able to drive it from dom0?

It would probably be useful to post full dmesg output for hypervisor and
both kernels.

Ian.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> > How can I tell if the iommu=soft is taking effect?
> > Anything stupid I sound like I've forgotten?
>
> It looks as if you have been pretty thorough.
>
> One thing which is not clear is if you also downgraded all the Xen
> utilities/tools packages when you say "Xen packages" or just the
> hypervisor itself. (You probably downgraded everything, but I have to
> ask).
>
> Does Fedora (yum?) log which packages which it is upgrading? Is it
> possible that e.g. scandvb or the firmware package has also been
> upgraded (I admit this is slightly straw-clutching).
>
> If you don't passthrough a device are you able to drive it from dom0?
>
> It would probably be useful to post full dmesg output for hypervisor and
> both kernels.

Andy,

I am going to try to see if I can pass in some of my TV cards in
my guest too. Perhaps that might shed some light.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Tue, Oct 11, 2011 at 10:49:37PM +0100, Andy Burns wrote:
> On 11 October 2011 22:25, Andy Burns <xen.lists@burns.me.uk> wrote:
>
> > Well, I have it partly working again
> > I think it's either ...
> >
> > Using "com1=115200,8n1 console=com1" on the hypervisor command with
> > "console=hvc0" on the dom0 kernel command
> > which I think would be a WTF? if that turns out to be the cause.
>
> And the winner is .......... WTF?
>
> I'd enabled serial console for catching some crashers around the
> 3.1.0-rc5 and -rc6 releases, once Konrad's fixes made it upstream I'd
> left the console serial enabled with -rc7 and -rc8, but I reverted to
> VGA console with -rc9 as I was no longer seeing any crashers!
>
> The motherboard is an X38 chipset, but doesn't have VT-d due to lack
> of BIOS support (thanks ASUS) so there's no VGA passthrough going on.
>
> Any educated guesses as to what's the cause?

So it works now right? Did you turn off the machine in between your
reboots - when you went back from CentOS to F16?

(Yes, I did encounter a bug where it would program an hardware interface
that would retain its state after a reboot and cause the newly built kernel
to go wrongly. Spent couple of good days trying to narrow that one down).

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:

> So it works now right?

No. When I went to bed last night I was convinced it was working,
having seen it work three times in a row where it hadn't at all for
the past 24 hours, albeit not knowing why serial console should make
it work.

I left the machines updating to latest Fedora16 branch, but this
morning it's not working again.

> Did you turn off the machine in between your
> reboots - when you went back from CentOS to F16?

I've not booted CentOS for a few weeks now, machine has certainly been
physically powered off since then,both when working and when faiing.

I do know that for the test above where I posted dmesg results as
requested by Ian, the machine was physically powered off and
passthrough failed.

I didn't turn the machine physically off for every test, sometimes I
rebooted it from within the dom0, sometimes powered it down to S5
state (which doesn't always work) and re-woke with WOL, sometimes it
hung and I used the reset button, sometines it hung and I'd press and
hold the power button.

> I did encounter a bug where it would program an hardware interface
> that would retain its state after a reboot and cause the newly built kernel
> to go wrongly. Spent couple of good days trying to narrow that one down

I'll make sure I always physically turn it off until I find the true
cause, back to older hypervisor and kernel packages again for more
tests ...

I think I remember what was the test I did before the boot that
started working again , but honestly can't remember whether I rebooted
or powered off after that.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 12 October 2011 09:01, Andy Burns <xen.lists@burns.me.uk> wrote:

> Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
>
>> So it works now right?
>
> No.
>
> I think I remember what was the test I did before the boot that
> started working again

I didn't get much time to test this today ... reverted to same Xen and
kernel versions that worked briefly last night.

I discovered that (despite what I answered earlier) the PCI tuners
don't work in dom0 under Xen, they only work if the dom0 is booted as
baremetal.

If I reboot from dom0 from baremetal with the PCI cards working into
Xen without powering off, it doesn't "magically" leave the PCI cards
in a state that allows them to work in the domU.

The thing which *seemed* to put it into a good mood last night was
booting dom0 with serial console and the domU with the PIC cards but
without the PCIe card, but that made no difference today.

I'm beginning to follow Konrad's thoughts that there is a specific
sequence of events, that persists in hardware state across soft
reboots, occasionally ending up with functioning PCI cards.

Is the fact that the PCI cards fail in dom0 under Xen a hint? Any
debugging I can do with the tuners from the dom0 rather than the domU
with passthrough?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Wed, Oct 12, 2011 at 10:36:12PM +0100, Andy Burns wrote:
> On 12 October 2011 09:01, Andy Burns <xen.lists@burns.me.uk> wrote:
>
> > Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
> >
> >> So it works now right?
> >
> > No.

<Grumble>
> >
> > I think I remember what was the test I did before the boot that
> > started working again
>
> I didn't get much time to test this today ... reverted to same Xen and
> kernel versions that worked briefly last night.
>
> I discovered that (despite what I answered earlier) the PCI tuners
> don't work in dom0 under Xen, they only work if the dom0 is booted as
> baremetal.
>
> If I reboot from dom0 from baremetal with the PCI cards working into
> Xen without powering off, it doesn't "magically" leave the PCI cards
> in a state that allows them to work in the domU.
>
> The thing which *seemed* to put it into a good mood last night was
> booting dom0 with serial console and the domU with the PIC cards but
> without the PCIe card, but that made no difference today.
>
> I'm beginning to follow Konrad's thoughts that there is a specific
> sequence of events, that persists in hardware state across soft
> reboots, occasionally ending up with functioning PCI cards.
>
> Is the fact that the PCI cards fail in dom0 under Xen a hint? Any
> debugging I can do with the tuners from the dom0 rather than the domU
> with passthrough?

That is. That would imply it is not the PCI passthrough code (good!).
It is something related to the driver (as I presume your network card
works in that box). Perhaps it is the VM_IO bug that sometimes creeps
up.. Can you give me the lsmod output please? I want to see which
drivers are loaded for this TV card and I can dig a bit in the
driver to see if there is something fishy.

I saw something about I2C, is there a knob in the driver to _not_
use I2C?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:

> Andy Burns wrote:
>
>> Is the fact that the PCI cards fail in dom0 under Xen a hint?
>
> That is. That would imply it is not the PCI passthrough code (good!).
> It is something related to the driver (as I presume your network card
> works in that box).

Yes, two onboard sky2 gigabit ports and a supermicro 64bit PCI-X JBOD
card with 8 SATA disks

> Perhaps it is the VM_IO bug that sometimes creeps
> up.. Can you give me the lsmod output please? I want to see which
> drivers are loaded for this TV card and I can dig a bit in the
> driver to see if there is something fishy.

Here is an lsmod (from within the domU, but the same modules get used
if it's in dom0 and same kernel 3.1.0-rc9 everywhere now.

Module Size Used by
ds3000 12827 2
dvb_usb_dw2102 41753 27
dvb_usb 14988 1 dvb_usb_dw2102
lockd 70080 0
ip6t_REJECT 3992 2
nf_conntrack_ipv6 7730 2
nf_defrag_ipv6 9083 1 nf_conntrack_ipv6
xt_state 1306 2
nf_conntrack 67597 2 nf_conntrack_ipv6,xt_state
ip6table_filter 1655 1
ip6_tables 16792 1 ip6table_filter
tda1004x 14722 2
saa7134_dvb 27032 12
videobuf_dvb 5146 1 saa7134_dvb
dvb_core 87211 2 dvb_usb,videobuf_dvb
firewire_ohci 26101 0
ir_lirc_codec 4214 0
lirc_dev 12904 1 ir_lirc_codec
firewire_core 49303 1 firewire_ohci
ir_mce_kbd_decoder 4208 0
ir_sony_decoder 2109 0
ir_jvc_decoder 2218 0
ir_rc6_decoder 2682 0
ir_rc5_decoder 2138 0
rc_videomate_tv_pvr 1289 0
ir_nec_decoder 2570 0
saa7134 159679 1 saa7134_dvb
crc_itu_t 1547 1 firewire_core
rc_core 17136 12
dvb_usb,ir_lirc_codec,ir_mce_kbd_decoder,ir_sony_decoder,ir_jvc_decoder,ir_rc6_decoder,ir_rc5_decoder,rc_videomate_tv_pvr,ir_nec_decoder,saa7134
videobuf_dma_sg 8462 2 saa7134_dvb,saa7134
videobuf_core 15780 3 videobuf_dvb,saa7134,videobuf_dma_sg
v4l2_common 6905 1 saa7134
videodev 78689 2 saa7134,v4l2_common
media 11511 1 videodev
v4l2_compat_ioctl32 7665 1 videodev
tveeprom 13045 1 saa7134
i2c_core 25728 9
ds3000,dvb_usb_dw2102,dvb_usb,tda1004x,saa7134_dvb,saa7134,v4l2_common,videodev,tveeprom
sunrpc 200831 2 lockd
xen_pcifront 12182 0
xen_netfront 16358 0
xen_blkfront 12741 4

Specifically the tuner drivers are

ds3000 and dvb_usb_dw2102 for the DVB-S2 PCIe that works

tveeprom, saa7134, saa7134_dvb and tda1004x for the troublesome
DVB-T PCI cards

> I saw something about I2C, is there a knob in the driver to _not_
> use I2C?

I can certainly live without all the LIRC stuff by blacklisting all
the ir_xxx_decoder stuff, I'll check modinfo of the drivers to see
what options exist.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:

> would imply it is not the PCI passthrough code (good!).
> It is something related to the driver
> Perhaps it is the VM_IO bug that sometimes creeps
> up..

I can try *NOT* using pci=resource_realignment=(blah), but I always
had to do that (or rather the old reassigndev equivalent) under
xenified 2.6.18 kernels as the BARs of the PCI cards were smaller than
the page size and I think the drivers for the two instances of the
DVB-T card used to trample all over each other without it.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, Oct 13, 2011 at 09:28:32PM +0100, Andy Burns wrote:
> Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
>
> > would imply it is not the PCI passthrough code (good!).
> > It is something related to the driver
> > Perhaps it is the VM_IO bug that sometimes creeps
> > up..
>
> I can try *NOT* using pci=resource_realignment=(blah), but I always
> had to do that (or rather the old reassigndev equivalent) under
> xenified 2.6.18 kernels as the BARs of the PCI cards were smaller than
> the page size and I think the drivers for the two instances of the
> DVB-T card used to trample all over each other without it.

Is there a good application you use to test this? xawtv?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 14 October 2011 16:55, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:

> Is there a good application you use to test this? xawtv?

Yeah, mythtv *is* a bit too heavy for testing :-P

FOr quick tests I just use scandvb (sometimes known as "dvbscan" or
"scan" depending on distro) but it needs some "seed" information
that's geographically sensitive to start searching for stations e.g.
for me ...

scandvb -a0 /usr/share/dvb-apps/dvb-t/uk-Waltham

I've no idea where you're located and whether you have
DVB-T/DVB-S/ATSC tuners ...

if your distro provides it then "w_scan" is easier as it will go off
and do a brute force scan by itself, you just need to tell it if
you're using terrestrial or satellite, and prefereably a hint to which
country and off it goes ...

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 13 October 2011 19:15, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:

> On Wed, Oct 12, 2011 at 10:36:12PM +0100, Andy Burns wrote:
>
>> I discovered that the PCI tuners don't work in dom0 under Xen,
>> they only work if the dom0 is booted  as baremetal.
>
> That would imply it is not the PCI passthrough code (good!).
> It is something related to the driver

I've been having a go at getting the card running in dom0 rather than
domU, no more success yet.

I've stopped using the PCI resource realignment for now, and enabled
debugging knobs in the tuner modules, seems the I2C transfers used to
program the tuner and receive status bits back from it are working OK,
I can see the driver queueing up DMA using a succession of transfer
buffers, and apparently interrupts signalling their completion.

Presume it's correct that within the dom0, the buffers should *NOT* be
within the SWIOTLB range?

# dmesg | grep ffff8800

[ 0.000000] found SMP MP-table at [ffff8800000ff780] ff780
[ 0.000000] Base memory trampoline at [ffff880000097000] 97000 size 20480
[ 0.000000] PERCPU: Embedded 27 pages/cpu @ffff88001fe43000 s81024
r8192 d21376 u110592
[ 0.000000] Placing 64MB software IO TLB between ffff880015c00000 -
ffff880019c00000

... snip ...

[ 1539.146723] saa7130[0]/ts: buffer_activate [ffff88000c25ca00]
[ 1539.146726] saa7130[0]/ts: - [top] buf=ffff88000c25ca00
next=ffff88000959c800
[ 1539.146737] saa7130[0]/core: buffer_next #2
prev=ffff8800041f8440/next=ffff88000959c840
[ 1539.146790] saa7130[0]/core: buffer_queue ffff88000c25c000
[ 1539.154693] saa7130[0]/core: buffer_finish ffff88000c25ca00
[ 1539.154698] saa7130[0]/core: buffer_next ffff88000959c800
[prev=ffff88000c25c040/next=ffff88000959c840]
[ 1539.154702] saa7130[0]/ts: buffer_activate [ffff88000959c800]
[ 1539.154705] saa7130[0]/ts: - [bottom] buf=ffff88000959c800
next=ffff88000959c600
[ 1539.154716] saa7130[0]/core: buffer_next #2
prev=ffff88000c25c040/next=ffff88000959c640
[ 1540.156114] saa7130[0]/core: timeout on ffff88000959c800
[ 1540.156118] saa7130[0]/core: buffer_finish ffff88000959c800

I assume no meaningful transport stream contents are contained in the
buffers following the transfers, as no stations are found.

Hypervisor, dom0 (and domU when I was using it) are all 64bit, the
tuner device is 32bit, is this likely to be an issue with DMA
transfers? Any extra logging for Xen?

09:00.0 Multimedia controller: Philips Semiconductors SAA7130 Video
Broadcast Decoder (rev 01)
Subsystem: Compro Technology, Inc. Videomate DVB-T200
Flags: bus master, medium devsel, latency 64, IRQ 16
Memory at febffc00 (32-bit, non-prefetchable) [size=1K]
Capabilities: [40] Power Management version 1
Kernel driver in use: saa7134
Kernel modules: saa7134

09:01.0 Multimedia controller: Philips Semiconductors SAA7130 Video
Broadcast Decoder (rev 01)
Subsystem: Compro Technology, Inc. Videomate DVB-T200
Flags: bus master, medium devsel, latency 64, IRQ 17
Memory at febff800 (32-bit, non-prefetchable) [size=1K]
Capabilities: [40] Power Management version 1
Kernel driver in use: saa7134
Kernel modules: saa7134

> Perhaps it is the VM_IO bug that sometimes creeps up..

Any pointers to previous cases of that bug, or the methods to discover it?

> I saw something about I2C, is there a knob in the driver to _not_ use I2C?

Not for my tuner, I've added modprobe options to blacklist devices
that I don't need and turn on debugging for the tuner modules

# cat /etc/modprobe.d/dvb.conf

blacklist ds3000
blacklist dvb_usb_dw2102
blacklist firewire_ehci
blacklist firewire_ohci
options saa7134 disable_ir=1 video_debug=1 ts_debug=1 i2c_debug=1
i2c_scan=1 irq_debug=1 core_debug=1 gpio_tracking=1
options saa7134-dvb debug=1
options tda1004x debug=1

# lsmod
Module Size Used by
lockd 70080 0
bridge 72368 0
stp 1927 1 bridge
llc 4738 2 bridge,stp
ip6t_REJECT 3992 2
nf_conntrack_ipv6 7730 2
nf_defrag_ipv6 9083 1 nf_conntrack_ipv6
xt_state 1306 2
nf_conntrack 67597 2 nf_conntrack_ipv6,xt_state
ip6table_filter 1655 1
ip6_tables 16792 1 ip6table_filter
snd_hda_codec_realtek 312621 1
raid456 54497 1
async_raid6_recov 5358 1 raid456
async_pq 4339 2 raid456,async_raid6_recov
raid6_pq 78299 2 async_raid6_recov,async_pq
async_xor 3255 3 raid456,async_raid6_recov,async_pq
xor 4793 1 async_xor
async_memcpy 1845 2 raid456,async_raid6_recov
async_tx 2702 5
raid456,async_raid6_recov,async_pq,async_xor,async_memcpy
tda1004x 14722 2
snd_hda_intel 24072 0
snd_hda_codec 85181 2 snd_hda_codec_realtek,snd_hda_intel
snd_hwdep 6264 1 snd_hda_codec
saa7134_dvb 27032 0
videobuf_dvb 5146 1 saa7134_dvb
dvb_core 87211 1 videobuf_dvb
snd_seq 52186 0
snd_seq_device 5941 1 snd_seq
iTCO_wdt 12024 0
snd_pcm 78514 2 snd_hda_intel,snd_hda_codec
saa7134 159679 1 saa7134_dvb
rc_core 17136 1 saa7134
videobuf_dma_sg 8462 2 saa7134_dvb,saa7134
videobuf_core 15780 3 videobuf_dvb,saa7134,videobuf_dma_sg
v4l2_common 6905 1 saa7134
videodev 78689 2 saa7134,v4l2_common
media 11511 1 videodev
v4l2_compat_ioctl32 7665 1 videodev
shpchp 24554 0
i2c_i801 9237 0
serio_raw 4298 0
iTCO_vendor_support 2578 1 iTCO_wdt
sky2 42923 0
snd_timer 19372 2 snd_seq,snd_pcm
snd 63124 8
snd_hda_codec_realtek,snd_hda_intel,snd_hda_codec,snd_hwdep,snd_seq,snd_seq_device,snd_pcm,snd_timer
soundcore 6267 1 snd
snd_page_alloc 7311 2 snd_hda_intel,snd_pcm
tveeprom 13045 1 saa7134
asus_atk0110 12395 0
x38_edac 3159 0
edac_core 40154 2 x38_edac
xen_netback 23987 0 [permanent]
xen_blkback 17924 0 [permanent]
xen_gntdev 9019 0
xen_evtchn 5032 1
sunrpc 200831 2 lockd
xenfs 9621 1
raid1 22676 2
ata_generic 3587 0
uas 7775 0
pata_acpi 3419 0
usb_storage 46027 0
sata_mv 24941 8
pata_marvell 3240 0
radeon 690803 1
ttm 54997 1 radeon
drm_kms_helper 26490 1 radeon
drm 194532 3 radeon,ttm,drm_kms_helper
i2c_algo_bit 4958 1 radeon
i2c_core 25728 11
tda1004x,saa7134_dvb,saa7134,v4l2_common,videodev,i2c_i801,tveeprom,radeon,drm_kms_helper,drm,i2c_algo_bit
[root@xen ~]# lsmod|more
Module Size Used by
lockd 70080 0
bridge 72368 0
stp 1927 1 bridge
llc 4738 2 bridge,stp
ip6t_REJECT 3992 2
nf_conntrack_ipv6 7730 2
nf_defrag_ipv6 9083 1 nf_conntrack_ipv6
xt_state 1306 2
nf_conntrack 67597 2 nf_conntrack_ipv6,xt_state
ip6table_filter 1655 1
ip6_tables 16792 1 ip6table_filter
snd_hda_codec_realtek 312621 1
raid456 54497 1
async_raid6_recov 5358 1 raid456
async_pq 4339 2 raid456,async_raid6_recov
raid6_pq 78299 2 async_raid6_recov,async_pq
async_xor 3255 3 raid456,async_raid6_recov,async_pq
xor 4793 1 async_xor
async_memcpy 1845 2 raid456,async_raid6_recov
async_tx 2702 5
raid456,async_raid6_recov,async_pq,async_xor,async_memcpy
tda1004x 14722 2
snd_hda_intel 24072 0
snd_hda_codec 85181 2 snd_hda_codec_realtek,snd_hda_intel
snd_hwdep 6264 1 snd_hda_codec
saa7134_dvb 27032 0
videobuf_dvb 5146 1 saa7134_dvb
dvb_core 87211 1 videobuf_dvb
snd_seq 52186 0
snd_seq_device 5941 1 snd_seq
iTCO_wdt 12024 0
snd_pcm 78514 2 snd_hda_intel,snd_hda_codec
saa7134 159679 1 saa7134_dvb
rc_core 17136 1 saa7134
videobuf_dma_sg 8462 2 saa7134_dvb,saa7134
videobuf_core 15780 3 videobuf_dvb,saa7134,videobuf_dma_sg
v4l2_common 6905 1 saa7134
videodev 78689 2 saa7134,v4l2_common
media 11511 1 videodev
v4l2_compat_ioctl32 7665 1 videodev
shpchp 24554 0
i2c_i801 9237 0
serio_raw 4298 0
iTCO_vendor_support 2578 1 iTCO_wdt
sky2 42923 0
snd_timer 19372 2 snd_seq,snd_pcm
snd 63124 8
snd_hda_codec_realtek,snd_hda_intel,snd_hda_codec,snd_hwdep,snd_seq,snd_seq_device,snd_pcm,snd_timer
soundcore 6267 1 snd
snd_page_alloc 7311 2 snd_hda_intel,snd_pcm
tveeprom 13045 1 saa7134
asus_atk0110 12395 0
x38_edac 3159 0
edac_core 40154 2 x38_edac
xen_netback 23987 0 [permanent]
xen_blkback 17924 0 [permanent]
xen_gntdev 9019 0
xen_evtchn 5032 1
sunrpc 200831 2 lockd
xenfs 9621 1
raid1 22676 2
ata_generic 3587 0
uas 7775 0
pata_acpi 3419 0
usb_storage 46027 0
sata_mv 24941 8
pata_marvell 3240 0
radeon 690803 1
ttm 54997 1 radeon
drm_kms_helper 26490 1 radeon
drm 194532 3 radeon,ttm,drm_kms_helper
i2c_algo_bit 4958 1 radeon
i2c_core 25728 11
tda1004x,saa7134_dvb,saa7134,v4l2_common,videodev,i2c_i801,tveeprom,radeon,drm_kms_helper,drm,i2c_algo_bit

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Sat, 2011-10-15 at 11:07 +0100, Andy Burns wrote:
> On 13 October 2011 19:15, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
>
> > On Wed, Oct 12, 2011 at 10:36:12PM +0100, Andy Burns wrote:
> >
> >> I discovered that the PCI tuners don't work in dom0 under Xen,
> >> they only work if the dom0 is booted as baremetal.
> >
> > That would imply it is not the PCI passthrough code (good!).
> > It is something related to the driver
>
> I've been having a go at getting the card running in dom0 rather than
> domU, no more success yet.
>
> I've stopped using the PCI resource realignment for now, and enabled
> debugging knobs in the tuner modules, seems the I2C transfers used to
> program the tuner and receive status bits back from it are working OK,
> I can see the driver queueing up DMA using a succession of transfer
> buffers, and apparently interrupts signalling their completion.
>
> Presume it's correct that within the dom0, the buffers should *NOT* be
> within the SWIOTLB range?

In general if a driver is correctly using the DMA API things should
never be within the swiotlb.

[...]
> Hypervisor, dom0 (and domU when I was using it) are all 64bit, the
> tuner device is 32bit, is this likely to be an issue with DMA
> transfers? Any extra logging for Xen?

I think you've got 8G of RAM so one thing which might be worth trying is
to give "mem=2G" (or perhaps 3G) on the hypervisor command line. That
ought to rule out addresses which are too high. (just a datapoint, not a
solution)

I wonder if the hypervisor's "dma_bits" option has any relevance here?
Might be worth a go?

For a domU (but not dom0, AFAICT) you can limit the machine addresses
allocated to a guest using "machine_address_size" in the cfg file. Only
seems to be supported by xend and not xl at the moment. That might be
another worthwhile experiment.

> > I saw something about I2C, is there a knob in the driver to _not_ use I2C?
>
> Not for my tuner

In general I think these sorts cards use I2C extensively, i.e. the tuner
etc is on an i2c bus, so I wouldn't expect anything to work at all
without it.

Ian.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 15 October 2011 11:36, Ian Campbell <Ian.Campbell@citrix.com> wrote:

> In general if a driver is correctly using the DMA API things should
> never be within the swiotlb.

OK, they just looked coincidentally similar.

> I think you've got 8G of RAM so one thing which might be worth trying is
> to give "mem=2G" (or perhaps 3G) on the hypervisor command line.

Yes I do have 8GB, will give it a try.

> I wonder if the hypervisor's "dma_bits" option has any relevance here?
> Might be worth a go?

I noticed that parameter, but couldn't see much indication of sensible
values, something a few bits less than 32 perhaps?

> In general I think these sorts cards use I2C extensively, i.e. the tuner
> etc is on an i2c bus, so I wouldn't expect anything to work at all
> without it.

Yes, that part of the card is working, w_scan can control the tuner
and detect when it latches onto valid muxes, receiving the bulk
datastream from the demodulator is the problem.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 15 October 2011 11:36, Ian Campbell <Ian.Campbell@citrix.com> wrote:

> I think you've got 8G of RAM so one thing which might be worth trying is
> to give "mem=2G" (or perhaps 3G) on the hypervisor command line. That
> ought to rule out addresses which are too high. (just a datapoint, not a
> solution)

A coconut for the gentleman!

Working in dom0 and (with page-alignment of the PCI BARs) in the domU,
I did check a couple of reboots and a cold start too, just in case!

So what's to look at for the real cause?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Sat, 2011-10-15 at 11:54 +0100, Andy Burns wrote:
> On 15 October 2011 11:36, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>
> > In general if a driver is correctly using the DMA API things should
> > never be within the swiotlb.
>
> OK, they just looked coincidentally similar.
>
> > I think you've got 8G of RAM so one thing which might be worth trying is
> > to give "mem=2G" (or perhaps 3G) on the hypervisor command line.
>
> Yes I do have 8GB, will give it a try.
>
> > I wonder if the hypervisor's "dma_bits" option has any relevance here?
> > Might be worth a go?
>
> I noticed that parameter, but couldn't see much indication of sensible
> values, something a few bits less than 32 perhaps?

Anything <32 would be a good guess, maybe try 30? Even 32 might be
worthwhile to try (not sure what you are getting by default).

Ian.

> > In general I think these sorts cards use I2C extensively, i.e. the tuner
> > etc is on an i2c bus, so I wouldn't expect anything to work at all
> > without it.
>
> Yes, that part of the card is working, w_scan can control the tuner
> and detect when it latches onto valid muxes, receiving the bulk
> datastream from the demodulator is the problem.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Sat, 2011-10-15 at 12:27 +0100, Andy Burns wrote:
> On 15 October 2011 11:36, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>
> > I think you've got 8G of RAM so one thing which might be worth trying is
> > to give "mem=2G" (or perhaps 3G) on the hypervisor command line. That
> > ought to rule out addresses which are too high. (just a datapoint, not a
> > solution)
>
> A coconut for the gentleman!
>
> Working in dom0 and (with page-alignment of the PCI BARs) in the domU,
> I did check a couple of reboots and a cold start too, just in case!

Excellent. I should read the whole thread before replying.

> So what's to look at for the real cause?

Will have to have a think on Monday.

The VM_IO stuff which Konrad mentioned and perhaps the DMA mask of the
device spring to mind.

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Interesting. Konrad, do you remember that one of my DVB cards did
consume double CPU time
as compared to a) old Xenified 2.6.18 b) Xenified 2.6.34 and c) mem=2G
Dom0? Maybe these
observations are somehow connected.

It did work, though... Unfortunately, I sold these cards to have only
one, which is

Carsten.

-----Ursprüngliche Nachricht-----
Von: Andy Burns [mailto:xen.lists@burns.me.uk]
Gesendet: Samstag, 15. Oktober 2011 13:28
An: Ian Campbell; Konrad Rzeszutek Wilk; xen-devel
Betreff: Re: [Xen-devel] Re: PCI passthrough stopped working, brainache!

On 15 October 2011 11:36, Ian Campbell <Ian.Campbell@citrix.com> wrote:

> I think you've got 8G of RAM so one thing which might be worth trying
is
> to give "mem=2G" (or perhaps 3G) on the hypervisor command line. That
> ought to rule out addresses which are too high. (just a datapoint, not
a
> solution)

A coconut for the gentleman!

Working in dom0 and (with page-alignment of the PCI BARs) in the domU,
I did check a couple of reboots and a cold start too, just in case!

So what's to look at for the real cause?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Sat, Oct 15, 2011 at 05:37:20PM +0200, Carsten Schiers wrote:
> Interesting. Konrad, do you remember that one of my DVB cards did
> consume double CPU time
> as compared to a) old Xenified 2.6.18 b) Xenified 2.6.34 and c) mem=2G

Well, your DVB cards started consuming when you added more than 4GB to your
box. I don't remember the 2.6.18 figuring in this picture.

> Dom0? Maybe these
> observations are somehow connected.
>
> It did work, though... Unfortunately, I sold these cards to have only
> one, which is

.. which is?

.. snip..
> A coconut for the gentleman!

Beer!
>
> Working in dom0 and (with page-alignment of the PCI BARs) in the domU,
> I did check a couple of reboots and a cold start too, just in case!
>
> So what's to look at for the real cause?

There can be one more test to conclude this and that is to raise right about 4GB
(ought to work) or perhaps to 5GB (it should fail there) under Xen .
Either way it sounds like a DMA issue - either we are:

1). Not having the right dma_mask set. Check the /sys/bus/pci/devices/<BDF>/dma_mask
and coherent_dma_mask. Both values should be the same and it ought to be 32

2). The driver is not doing pci_sync_range .. which you should be able
to easily find out. Try booting the Linux kernel in baremetal, with
8GB and with "iommu=soft swiotlb=force". That will bounce buffer _everything_
- which might show the same problem. It also might show problems with some
other drivers.

3). I can send you some patches to instrument Xen SWIOTLB to see what DMA pages
(and where in the driver code is doing it) are using bounce buffer and not
properly doing 2).

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 15 October 2011 12:27, Andy Burns <xen.lists@burns.me.uk> wrote:

> On 15 October 2011 11:36, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>
>> I think you've got 8G of RAM so one thing which might be worth trying is
>> to give "mem=2G"
>
> A coconut for the gentleman!

I can see that the devs have been busy getting their ducks in a row
for when the 3.2 merge window opens (3.1 is released now) so not
wanted to pester about this as it's OK for now with the workarounds.

Initially I tried mem=2G, with dom0_mem=512M and two domUs of 512M and
1G respectively, I presume this caused a bit of ballooning in dom0 as
I caught it with 400M'ish at one point and I had a little instability.

I pushed it up to mem=3G and at the same time added irqpoll for dom0
and the pci domU (because I was seeing a few "nobody cared" messages
in the domU at bootup and in the dom0 after shutting down the domU)
those changes so far have resulted in a stable setup.

Not wanting to reboot the dom0 just yet (would like to see how stable
it really is) but want to think about things to try when I *do* reboot
it,
I'm assuming the DMA problem will rear its head again if I try mem >=
4G, I haven't tried anything with dma_bits yet either, anything other
suggestions or logging that might help?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Mon, 2011-10-24 at 12:30 +0100, Andy Burns wrote:
> On 15 October 2011 12:27, Andy Burns <xen.lists@burns.me.uk> wrote:
>
> > On 15 October 2011 11:36, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> >
> >> I think you've got 8G of RAM so one thing which might be worth trying is
> >> to give "mem=2G"
> >
> > A coconut for the gentleman!
>
> I can see that the devs have been busy getting their ducks in a row
> for when the 3.2 merge window opens (3.1 is released now) so not
> wanted to pester about this as it's OK for now with the workarounds.
>
> Initially I tried mem=2G, with dom0_mem=512M and two domUs of 512M and
> 1G respectively, I presume this caused a bit of ballooning in dom0 as
> I caught it with 400M'ish at one point and I had a little instability.
>
> I pushed it up to mem=3G and at the same time added irqpoll for dom0
> and the pci domU (because I was seeing a few "nobody cared" messages
> in the domU at bootup and in the dom0 after shutting down the domU)
> those changes so far have resulted in a stable setup.
>
> Not wanting to reboot the dom0 just yet (would like to see how stable
> it really is) but want to think about things to try when I *do* reboot
> it,
> I'm assuming the DMA problem will rear its head again if I try mem >=
> 4G, I haven't tried anything with dma_bits yet either, anything other
> suggestions or logging that might help?

Konrad had some suggestions in
<20111020034000.GA2401@phenom.dumpdata.com> which would be worth trying.
(that mail was sent on Thursday but appears to have sat in a queue
somewhere until after you sent this mail so just checking you've seen
it).

Ian.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Andy,

Did I mess the solution for this or are you still working with 'mem=3.99999G'?

Neal Taylor
CA Technologies

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Ian Campbell
Sent: Wednesday, October 26, 2011 1:05 AM
To: Andy Burns
Cc: xen-devel
Subject: Re: [Xen-devel] Re: PCI passthrough stopped working, brainache!

On Mon, 2011-10-24 at 12:30 +0100, Andy Burns wrote:
> On 15 October 2011 12:27, Andy Burns <xen.lists@burns.me.uk> wrote:
>
> > On 15 October 2011 11:36, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> >
> >> I think you've got 8G of RAM so one thing which might be worth trying is
> >> to give "mem=2G"
> >
> > A coconut for the gentleman!
>
> I can see that the devs have been busy getting their ducks in a row
> for when the 3.2 merge window opens (3.1 is released now) so not
> wanted to pester about this as it's OK for now with the workarounds.
>
> Initially I tried mem=2G, with dom0_mem=512M and two domUs of 512M and
> 1G respectively, I presume this caused a bit of ballooning in dom0 as
> I caught it with 400M'ish at one point and I had a little instability.
>
> I pushed it up to mem=3G and at the same time added irqpoll for dom0
> and the pci domU (because I was seeing a few "nobody cared" messages
> in the domU at bootup and in the dom0 after shutting down the domU)
> those changes so far have resulted in a stable setup.
>
> Not wanting to reboot the dom0 just yet (would like to see how stable
> it really is) but want to think about things to try when I *do* reboot
> it,
> I'm assuming the DMA problem will rear its head again if I try mem >=
> 4G, I haven't tried anything with dma_bits yet either, anything other
> suggestions or logging that might help?

Konrad had some suggestions in
<20111020034000.GA2401@phenom.dumpdata.com> which would be worth trying.
(that mail was sent on Thursday but appears to have sat in a queue
somewhere until after you sent this mail so just checking you've seen
it).

Ian.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel