Mailing List Archive: [Question] Do we need to support devices that do not strictly follow the PCI-e specification?

[Question] Do we need to support devices that do not strictly follow the PCI-e specification?

Oct 9, 2008, 1:28 AM

Post #1 of 5 (76 views)

Hi, Keir,

When debugging recently, I found some devices that did not follow the PCI-e specification strictly. Below is an example.

From PCI-e spec, PCI-e capability occupies 0x3c bytes of configuration space. Unimplemented registers are reserved and hardwired to zero. But for device listed below, PCI-e capability should begin at 0x4C and end at 0x88. But this device implements MSI, VPD, MSI-X capabilities in the reserved spaces. Current code can not handle this.

My question is do we need to add hacks to handle such kinds of devices?

Here is the output of lspci:

03:00.0 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 03)
Subsystem: QLogic Corp. Unknown device 0138
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR-
Interrupt: pin A routed to IRQ 18
Region 0: I/O ports at 3000 [disabled] [size=256]
Region 1: Memory at d8500000 (64-bit, non-prefetchable) [disabled] [size=16K]
[virtual] Expansion ROM at d8800000 [disabled] [size=256K]
Capabilities: [44] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [4c] Express Endpoint IRQ 0
Device: Supported: MaxPayload 1024 bytes, PhantFunc 0, ExtTag-
Device: Latency L0s <4us, L1 <1us
Device: AtnBtn+ AtnInd+ PwrInd+
Device: Errors: Correctable- Non-Fatal- Fatal- Unsupported-
Device: RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
Device: MaxPayload 128 bytes, MaxReadReq 512 bytes
Link: Supported Speed 2.5Gb/s, Width x4, ASPM L0s, Port 0
Link: Latency L0s <4us, L1 unlimited
Link: ASPM Disabled RCB 64 bytes CommClk- ExtSynch-
Link: Speed 2.5Gb/s, Width x4
Capabilities: [64] Message Signalled Interrupts: 64bit+ Queue=0/4 Enable-
Address: 00000000fee00000 Data: 40b9
Capabilities: [74] Vital Product Data
Capabilities: [7c] MSI-X: Enable- Mask- TabSize=16
Vector table: BAR=1 offset=00002000
PBA: BAR=1 offset=00003000
00: 77 10 32 24 00 00 10 00 03 00 04 0c 08 00 80 00
10: 01 30 00 00 04 00 50 d8 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 77 10 38 01
30: 00 00 00 00 44 00 00 00 00 00 00 00 0a 01 00 00
40: 00 00 00 00 01 4c 02 00 00 00 00 00 10 64 01 00
50: 83 71 00 00 10 28 00 00 41 e4 03 00 00 00 41 10
60: 00 00 00 00 05 74 88 00 00 00 e0 fe 00 00 00 00
70: b9 40 00 00 03 7c 00 80 82 2e 00 50 11 00 0f 00
80: 01 20 00 00 01 30 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Best Regards
Shan Haitao

Re: [Question] Do we need to support devices that do not strictly follow the PCI-e specification? [ In reply to ]

keir.fraser at eu

Oct 9, 2008, 2:02 AM

Post #2 of 5 (74 views)

Permalink

On 9/10/08 09:28, "Shan, Haitao" <haitao.shan@intel.com> wrote:

> Hi, Keir,
>
> When debugging recently, I found some devices that did not follow the PCI-e
> specification strictly. Below is an example.
>
> From PCI-e spec, PCI-e capability occupies 0x3c bytes of configuration space.
> Unimplemented registers are reserved and hardwired to zero. But for device
> listed below, PCI-e capability should begin at 0x4C and end at 0x88. But this
> device implements MSI, VPD, MSI-X capabilities in the reserved spaces. Current
> code can not handle this.
>
> My question is do we need to add hacks to handle such kinds of devices?
>
I thought PCI capability offsets could be dynamically determined? If so, or
there are other means to easily determine actual capability offsets without
requiring explicit device-quirk lists, we should employ those means.

-- Keir

RE: [Question] Do we need to support devices that do not strictly follow the PCI-e specification? [ In reply to ]

haitao.shan at intel

Oct 9, 2008, 2:08 AM

Post #3 of 5 (74 views)

Permalink

Oh, I made some mistakes. I have just checked PCI-e 1.1 spec. It seems the definition of this capability structure is different with PCI-e 2.0. I will check more and find a solution.

________________________________
From: Keir Fraser [mailto:keir.fraser@eu.citrix.com]
Sent: 2008$BG/(B10$B7n(B9$BF|(B 17:03
To: Shan, Haitao; xen-devel@lists.xensource.com
Cc: Jiang, Yunhong; Li, Xin B; Tian, Kevin; Huang, Zhiteng
Subject: Re: [Question] Do we need to support devices that do not strictly follow the PCI-e specification?

On 9/10/08 09:28, "Shan, Haitao" <haitao.shan@intel.com> wrote:
Hi, Keir,

When debugging recently, I found some devices that did not follow the PCI-e specification strictly. Below is an example.

From PCI-e spec, PCI-e capability occupies 0x3c bytes of configuration space. Unimplemented registers are reserved and hardwired to zero. But for device listed below, PCI-e capability should begin at 0x4C and end at 0x88. But this device implements MSI, VPD, MSI-X capabilities in the reserved spaces. Current code can not handle this.

My question is do we need to add hacks to handle such kinds of devices?
I thought PCI capability offsets could be dynamically determined? If so, or there are other means to easily determine actual capability offsets without requiring explicit device-quirk lists, we should employ those means.

-- Keir

RE: RE: [Question] Do we need to support devices that do not strictly follow the PCI-e specification? [ In reply to ]

haitao.shan at intel

Oct 9, 2008, 2:31 AM

Post #4 of 5 (74 views)

Permalink

Hi, Keir,

There is already a fix for calculating the size of PCI-e capability properly. The changeset is 77fba269. But this changeset is not in xen-3.3-testing. Can you help to check in that changeset to xen-3.3-testing tree?
Thanks!

Shan Haitao

________________________________
From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Shan, Haitao
Sent: 2008$BG/(B10$B7n(B9$BF|(B 17:08
To: Keir Fraser; xen-devel@lists.xensource.com
Cc: Huang, Zhiteng; Tian, Kevin; Jiang, Yunhong; Li, Xin B
Subject: [Xen-devel] RE: [Question] Do we need to support devices that do not strictly follow the PCI-e specification?

Oh, I made some mistakes. I have just checked PCI-e 1.1 spec. It seems the definition of this capability structure is different with PCI-e 2.0. I will check more and find a solution.

________________________________
From: Keir Fraser [mailto:keir.fraser@eu.citrix.com]
Sent: 2008$BG/(B10$B7n(B9$BF|(B 17:03
To: Shan, Haitao; xen-devel@lists.xensource.com
Cc: Jiang, Yunhong; Li, Xin B; Tian, Kevin; Huang, Zhiteng
Subject: Re: [Question] Do we need to support devices that do not strictly follow the PCI-e specification?

On 9/10/08 09:28, "Shan, Haitao" <haitao.shan@intel.com> wrote:
Hi, Keir,

When debugging recently, I found some devices that did not follow the PCI-e specification strictly. Below is an example.

From PCI-e spec, PCI-e capability occupies 0x3c bytes of configuration space. Unimplemented registers are reserved and hardwired to zero. But for device listed below, PCI-e capability should begin at 0x4C and end at 0x88. But this device implements MSI, VPD, MSI-X capabilities in the reserved spaces. Current code can not handle this.

My question is do we need to add hacks to handle such kinds of devices?
I thought PCI capability offsets could be dynamically determined? If so, or there are other means to easily determine actual capability offsets without requiring explicit device-quirk lists, we should employ those means.

-- Keir

Re: RE: [Question] Do we need to support devices that do not strictly follow the PCI-e specification? [ In reply to ]

keir.fraser at eu

Oct 9, 2008, 2:49 AM

Post #5 of 5 (74 views)

Permalink

Will do.

K.

On 9/10/08 10:31, "Shan, Haitao" <haitao.shan@intel.com> wrote:

> Hi, Keir,
>
> There is already a fix for calculating the size of PCI-e capability properly.
> The changeset is 77fba269. But this changeset is not in xen-3.3-testing. Can
> you help to check in that changeset to xen-3.3-testing tree?
> Thanks!
>
> Shan Haitao
>
>
>
> From: xen-devel-bounces@lists.xensource.com
> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Shan, Haitao
> Sent: 2008$BG/(B10$B7n(B9$BF|(B 17:08
> To: Keir Fraser; xen-devel@lists.xensource.com
> Cc: Huang, Zhiteng; Tian, Kevin; Jiang, Yunhong; Li, Xin B
> Subject: [Xen-devel] RE: [Question] Do we need to support devices that do not
> strictly follow the PCI-e specification?
>
> Oh, I made some mistakes. I have just checked PCI-e 1.1 spec. It seems the
> definition of this capability structure is different with PCI-e 2.0. I will
> check more and find a solution.
>
>
>
> From: Keir Fraser [mailto:keir.fraser@eu.citrix.com]
> Sent: 2008$BG/(B10$B7n(B9$BF|(B 17:03
> To: Shan, Haitao; xen-devel@lists.xensource.com
> Cc: Jiang, Yunhong; Li, Xin B; Tian, Kevin; Huang, Zhiteng
> Subject: Re: [Question] Do we need to support devices that do not strictly
> follow the PCI-e specification?
>
>
>
>
> On 9/10/08 09:28, "Shan, Haitao" <haitao.shan@intel.com> wrote:
> Hi, Keir,
>
> When debugging recently, I found some devices that did not follow the PCI-e
> specification strictly. Below is an example.
>
> From PCI-e spec, PCI-e capability occupies 0x3c bytes of configuration space.
> Unimplemented registers are reserved and hardwired to zero. But for device
> listed below, PCI-e capability should begin at 0x4C and end at 0x88. But this
> device implements MSI, VPD, MSI-X capabilities in the reserved spaces. Current
> code can not handle this.
>
> My question is do we need to add hacks to handle such kinds of devices?
> I thought PCI capability offsets could be dynamically determined? If so, or
> there are other means to easily determine actual capability offsets without
> requiring explicit device-quirk lists, we should employ those means.
>
> -- Keir
>