Mailing List Archive

dom0 crash on 4.16.1 involving amd iommu and iov

Full disclosure:  I'm running the 4.16.1-1 package from Arch Linux's
AUR.   It includes a patch from OpenSUSE to allow it to compile on GCC
12.1, which is mostly suppressing errors.

The hardware is a Ryzen 7 5700G in an Asus Prime B450-A II
motherboard.   It has the on board ethernet interface, as well as a quad
ethernet card:

03:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network
Connection (rev 01)
03:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network
Connection (rev 01)
05:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network
Connection (rev 01)
05:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network
Connection (rev 01)
0d:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)

It runs well under 4.16.0, so I've backrev'ed to that in the meantime. 
The same package also works on another, somewhat older AMD machine.  It
is entirely possible that moving to newer gcc has something to do with
it, but the fact that the dom0 dmesg suggests iov might have something
to do with it suggests that I should at least ask about it.   I am
normally using the SR-IOV functions of the card, but that's set up by
the dom0 and that hasn't happened yet (obviously) by the time it
crashes.   Also, I have tried it without dom0=pvh and the failure is the

dmesg follows.   Thanks!


Xen 4.16.1-1-arch
(XEN) Xen version 4.16.1-1-arch (sam@) (gcc (GCC) 12.1.0) debug=n Mon
May 16 10:33:25 PDT 2022
(XEN) Latest ChangeSet:
(XEN) build-id: e9d23853be3606212052f0171a34f616
(XEN) Bootloader: EFI
(XEN) Command line: iommu=verbose dom0_mem=2G,max:2G loglvl=all smt=1
dom0=pvh iommu=1 console=vga com1=38400;8n1 console=com1,vga noreboot
(XEN) Xen image load base address: 0x9a000000
(XEN) Video information:
(XEN)  VGA is graphics mode 1024x768, 32 bpp
(XEN) Disc information:
(XEN)  Found 0 MBR signatures
(XEN)  Found 4 EDD information structures
(XEN) CPU Vendor: AMD, Family 25 (0x19), Model 80 (0x50), Stepping 0
(raw 00a50f00)
(XEN) Enabling Supervisor Shadow Stacks
(XEN) EFI RAM map:
(XEN)  [0000000000000000, 000000000009ffff] (usable)
(XEN)  [00000000000a0000, 00000000000fffff] (reserved)
(XEN)  [0000000000100000, 0000000009bfefff] (usable)
(XEN)  [0000000009bff000, 0000000009ffffff] (reserved)
(XEN)  [000000000a000000, 000000000a1fffff] (usable)
(XEN)  [000000000a200000, 000000000a20efff] (ACPI NVS)
(XEN)  [000000000a20f000, 000000000affffff] (usable)
(XEN)  [000000000b000000, 000000000b01ffff] (reserved)
(XEN)  [000000000b020000, 00000000a3bcdfff] (usable)
(XEN)  [00000000a3bce000, 00000000a3f30fff] (reserved)
(XEN)  [00000000a3f31000, 00000000a4194fff] (ACPI data)
(XEN)  [00000000a4195000, 00000000a5c8afff] (ACPI NVS)
(XEN)  [00000000a5c8b000, 00000000a6bfefff] (reserved)
(XEN)  [00000000a6bff000, 00000000a7ffffff] (usable)
(XEN)  [00000000a8000000, 00000000aaffffff] (reserved)
(XEN)  [00000000af000000, 00000000cfffffff] (reserved)
(XEN)  [00000000f8000000, 00000000fbffffff] (reserved)
(XEN)  [00000000fd200000, 00000000fd2fffff] (reserved)
(XEN)  [00000000fd600000, 00000000fd6fffff] (reserved)
(XEN)  [00000000fea00000, 00000000fea0ffff] (reserved)
(XEN)  [00000000feb80000, 00000000fec01fff] (reserved)
(XEN)  [00000000fec10000, 00000000fec10fff] (reserved)
(XEN)  [00000000fec30000, 00000000fec30fff] (reserved)
(XEN)  [00000000fed00000, 00000000fed00fff] (reserved)
(XEN)  [00000000fed40000, 00000000fed44fff] (reserved)
(XEN)  [00000000fed80000, 00000000fed8ffff] (reserved)
(XEN)  [00000000fedc2000, 00000000fedcffff] (reserved)
(XEN)  [00000000fedd4000, 00000000fedd5fff] (reserved)
(XEN)  [00000000ff000000, 00000000ffffffff] (reserved)
(XEN)  [0000000100000000, 000000082e2fffff] (usable)
(XEN)  [000000082e300000, 000000082fffffff] (reserved)
(XEN) ACPI: RSDP A4194014, 0024 (r2 ALASKA)
(XEN) ACPI: XSDT A4193728, 00BC (r1 ALASKA   A M I   1072009 AMI 1000013)
(XEN) ACPI: FACP A4184000, 0114 (r6 ALASKA   A M I   1072009 AMI     10013)
(XEN) ACPI: DSDT A4178000, B600 (r2 ALASKA   A M I   1072009 INTL 20120913)
(XEN) ACPI: FACS A4C70000, 0040
(XEN) ACPI: IVRS A4192000, 00D0 (r2  AMD   AmdTable        1 AMD         1)
(XEN) ACPI: SSDT A418A000, 723C (r2    AMD   MYRTLE        2 MSFT 4000000)
(XEN) ACPI: SSDT A4186000, 3C43 (r1    AMD  AMD AOD        1 INTL 20120913)
(XEN) ACPI: SSDT A4185000, 00FC (r2 ALASKA  CPUSSDT  1072009 AMI 1072009)
(XEN) ACPI: FIDT A4177000, 009C (r1 ALASKA    A M I  1072009 AMI     10013)
(XEN) ACPI: FPDT A415F000, 0044 (r1 ALASKA   A M I   1072009 AMI 1000013)
(XEN) ACPI: MCFG A4175000, 003C (r1 ALASKA    A M I  1072009 MSFT    10013)
(XEN) ACPI: HPET A4174000, 0038 (r1 ALASKA    A M I  1072009 AMI         5)
(XEN) ACPI: BGRT A4172000, 0038 (r1 ALASKA   A M I   1072009 AMI     10013)
(XEN) ACPI: TPM2 A4171000, 004C (r3 ALASKA   A M I         1 AMI         0)
(XEN) ACPI: SSDT A416B000, 5304 (r2    AMD AmdTable        1 AMD         1)
(XEN) ACPI: CRAT A416A000, 0EE8 (r1    AMD AmdTable        1 AMD         1)
(XEN) ACPI: CDIT A4169000, 0029 (r1    AMD AmdTable        1 AMD         1)
(XEN) ACPI: SSDT A4168000, 0D53 (r1    AMD MYRTLEG2        1 INTL 20120913)
(XEN) ACPI: SSDT A4166000, 10AC (r1    AMD MYRTLEPX        1 INTL 20120913)
(XEN) ACPI: SSDT A4162000, 381A (r1    AMD   MYRTLE        1 INTL 20120913)
(XEN) ACPI: SSDT A4161000, 00BF (r1    AMD AmdTable     1000 INTL 20120913)
(XEN) ACPI: APIC A4160000, 015E (r3 ALASKA   A M I   1072009 AMI     10013)
(XEN) System RAM: 32046MB (32815356kB)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-000000082e300000
(XEN) Domain heap initialised
(XEN) vesafb: framebuffer at 0x00000000e0000000, mapped to
0xffff82c000201000, using 3072k, total 3072k
(XEN) vesafb: mode is 1024x768x32, linelength=4096, font 8x14
(XEN) vesafb: Truecolor: size=8:8:8:8, shift=24:16:8:0
(XEN) SMBIOS 3.3 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x808 (32 bits)
(XEN) ACPI: v5 SLEEP INFO: control[0:0], status[0:0]
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:804,1:0], pm1x_evt[1:800,1:0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT -
a4c70000/0000000000000000, using 32
(XEN) ACPI:             wakeup_vec[a4c7000c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) Overriding APIC driver with bigsmp
(XEN) ACPI: IOAPIC (id[0x11] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 17, version 33, address 0xfec00000, GSI 0-23
(XEN) ACPI: IOAPIC (id[0x12] address[0xfec01000] gsi_base[24])
(XEN) IOAPIC[1]: apic_id 18, version 33, address 0xfec01000, GSI 24-55
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Phys.  Using 2 I/O APICs
(XEN) ACPI: HPET id: 0x10228201 base: 0xfed00000
(XEN) PCI: MCFG configuration 0: base f8000000 segment 0000 buses 00 - 3f
(XEN) PCI: MCFG area at f8000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-3f
(XEN) ACPI: BGRT: invalidating v1 image at 0x9ea53018
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 32 CPUs (16 hotplug CPUs)
(XEN) IRQ limits: 56 GSI, 3272 MSI/MSI-X
(XEN) CPU0: 1400 ... 3800 MHz
(XEN) xstate: size: 0x988 and states: 0x207
(XEN) CPU0: AMD Fam19h machine check reporting enabled
(XEN) Speculative mitigation facilities:
(XEN)   Hardware features: IBPB IBRS STIBP SSBD PSFD
(XEN)   Xen settings: BTI-Thunk JMP, SPEC_CTRL: IBRS+ STIBP+ SSBD-,
(XEN)   Support for HVM VMs: MSR_SPEC_CTRL RSB
(XEN)   Support for PV VMs: None
(XEN)   XPTI (64-bit PV only): Dom0 disabled, DomU disabled (without PCID)
(XEN)   PV L1TF shadowing: Dom0 disabled, DomU disabled
(XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
(XEN) Initializing Credit2 scheduler
(XEN)  load_precision_shift: 18
(XEN)  load_window_shift: 30
(XEN)  underload_balance_tolerance: 0
(XEN)  overload_balance_tolerance: -3
(XEN)  runqueues arrangement: socket
(XEN)  cap enforcement granularity: 10ms
(XEN) load tracking window length 1073741824 ns
(XEN) Platform timer is 14.318MHz HPET
(XEN) Detected 3792.872 MHz processor.
(XEN) EFI memory map:
(XEN)  0000000000000-0000000007fff type=3 attr=000000000000000f
(XEN)  0000000008000-000000003bfff type=7 attr=000000000000000f
(XEN)  000000003c000-000000003dfff type=2 attr=000000000000000f
(XEN)  000000003e000-000000003efff type=4 attr=000000000000000f
(XEN)  000000003f000-000000009efff type=3 attr=000000000000000f
(XEN)  000000009f000-000000009ffff type=4 attr=000000000000000f
(XEN)  0000000100000-0000000ffffff type=7 attr=000000000000000f
(XEN)  0000001000000-000000107ffff type=4 attr=000000000000000f
(XEN)  0000001080000-0000009bfefff type=7 attr=000000000000000f
(XEN)  0000009bff000-0000009ffffff type=0 attr=000000000000000f
(XEN)  000000a000000-000000a1fffff type=7 attr=000000000000000f
(XEN)  000000a200000-000000a20efff type=10 attr=000000000000000f
(XEN)  000000a20f000-000000affffff type=7 attr=000000000000000f
(XEN)  000000b000000-000000b01ffff type=0 attr=000000000000000f
(XEN)  000000b020000-0000098278fff type=7 attr=000000000000000f
(XEN)  0000098279000-0000099ee8fff type=2 attr=000000000000000f
(XEN)  0000099ee9000-0000099ee9fff type=7 attr=000000000000000f
(XEN)  0000099eea000-000009b0e9fff type=1 attr=000000000000000f
(XEN)  000009b0ea000-000009b224fff type=3 attr=000000000000000f
(XEN)  000009b225000-000009b225fff type=2 attr=000000000000000f
(XEN)  000009b226000-000009bbcdfff type=3 attr=000000000000000f
(XEN)  000009bbce000-000009e958fff type=7 attr=000000000000000f
(XEN)  000009e959000-000009e989fff type=4 attr=000000000000000f
(XEN)  000009e98a000-000009e9d9fff type=7 attr=000000000000000f
(XEN)  000009e9da000-000009ea16fff type=4 attr=000000000000000f
(XEN)  000009ea17000-000009ea1bfff type=7 attr=000000000000000f
(XEN)  000009ea1c000-000009ee53fff type=4 attr=000000000000000f
(XEN)  000009ee54000-000009ee5cfff type=7 attr=000000000000000f
(XEN)  000009ee5d000-000009ee99fff type=4 attr=000000000000000f
(XEN)  000009ee9a000-000009ee9cfff type=7 attr=000000000000000f
(XEN)  000009ee9d000-000009eecafff type=4 attr=000000000000000f
(XEN)  000009eecb000-000009eeccfff type=7 attr=000000000000000f
(XEN)  000009eecd000-000009eedafff type=4 attr=000000000000000f
(XEN)  000009eedb000-000009eedcfff type=7 attr=000000000000000f
(XEN)  000009eedd000-000009ef9ffff type=4 attr=000000000000000f
(XEN)  000009efa0000-000009efa0fff type=7 attr=000000000000000f
(XEN)  000009efa1000-000009efb3fff type=4 attr=000000000000000f
(XEN)  000009efb4000-000009efb4fff type=7 attr=000000000000000f
(XEN)  000009efb5000-00000a03d6fff type=4 attr=000000000000000f
(XEN)  00000a03d7000-00000a03d9fff type=7 attr=000000000000000f
(XEN)  00000a03da000-00000a1b6dfff type=4 attr=000000000000000f
(XEN)  00000a1b6e000-00000a1b6efff type=7 attr=000000000000000f
(XEN)  00000a1b6f000-00000a1b71fff type=4 attr=000000000000000f
(XEN)  00000a1b72000-00000a1b72fff type=7 attr=000000000000000f
(XEN)  00000a1b73000-00000a3bcdfff type=4 attr=000000000000000f
(XEN)  00000a3bce000-00000a3f30fff type=0 attr=000000000000000f
(XEN)  00000a3f31000-00000a4194fff type=9 attr=000000000000000f
(XEN)  00000a4195000-00000a5c8afff type=10 attr=000000000000000f
(XEN)  00000a5c8b000-00000a6b6cfff type=6 attr=800000000000000f
(XEN)  00000a6b6d000-00000a6bfefff type=5 attr=800000000000000f
(XEN)  00000a6bff000-00000a6ffffff type=4 attr=000000000000000f
(XEN)  00000a7000000-00000a7087fff type=7 attr=000000000000000f
(XEN)  00000a7088000-00000a7187fff type=4 attr=000000000000000f
(XEN)  00000a7188000-00000a71c0fff type=3 attr=000000000000000f
(XEN)  00000a71c1000-00000a7209fff type=4 attr=000000000000000f
(XEN)  00000a720a000-00000a722bfff type=3 attr=000000000000000f
(XEN)  00000a722c000-00000a7245fff type=4 attr=000000000000000f
(XEN)  00000a7246000-00000a725afff type=3 attr=000000000000000f
(XEN)  00000a725b000-00000a7f67fff type=4 attr=000000000000000f
(XEN)  00000a7f68000-00000a7f6afff type=3 attr=000000000000000f
(XEN)  00000a7f6b000-00000a7f7dfff type=4 attr=000000000000000f
(XEN)  00000a7f7e000-00000a7f7ffff type=3 attr=000000000000000f
(XEN)  00000a7f80000-00000a7f90fff type=4 attr=000000000000000f
(XEN)  00000a7f91000-00000a7f93fff type=3 attr=000000000000000f
(XEN)  00000a7f94000-00000a7fa6fff type=4 attr=000000000000000f
(XEN)  00000a7fa7000-00000a7faafff type=3 attr=000000000000000f
(XEN)  00000a7fab000-00000a7fcefff type=4 attr=000000000000000f
(XEN)  00000a7fcf000-00000a7fd2fff type=3 attr=000000000000000f
(XEN)  00000a7fd3000-00000a7fe6fff type=4 attr=000000000000000f
(XEN)  00000a7fe7000-00000a7ff1fff type=3 attr=000000000000000f
(XEN)  00000a7ff2000-00000a7ff9fff type=4 attr=000000000000000f
(XEN)  00000a7ffa000-00000a7ffffff type=3 attr=000000000000000f
(XEN)  0000100000000-000082e2fffff type=7 attr=000000000000000f
(XEN)  00000000a0000-00000000fffff type=0 attr=000000000000000f
(XEN)  00000a8000000-00000aaffffff type=0 attr=000000000000000f
(XEN)  00000af000000-00000cfffffff type=0 attr=000000000000000f
(XEN)  00000f8000000-00000fbffffff type=11 attr=800000000000100d
(XEN)  00000fd200000-00000fd2fffff type=11 attr=8000000000000001
(XEN)  00000fd600000-00000fd6fffff type=11 attr=8000000000000001
(XEN)  00000fea00000-00000fea0ffff type=11 attr=8000000000000001
(XEN)  00000feb80000-00000fec01fff type=11 attr=8000000000000001
(XEN)  00000fec10000-00000fec10fff type=11 attr=8000000000000001
(XEN)  00000fec30000-00000fec30fff type=11 attr=8000000000000001
(XEN)  00000fed00000-00000fed00fff type=11 attr=8000000000000001
(XEN)  00000fed40000-00000fed44fff type=11 attr=8000000000000001
(XEN)  00000fed80000-00000fed8ffff type=11 attr=8000000000000001
(XEN)  00000fedc2000-00000fedcffff type=11 attr=8000000000000001
(XEN)  00000fedd4000-00000fedd5fff type=11 attr=8000000000000001
(XEN)  00000ff000000-00000ffffffff type=11 attr=8000000000000001
(XEN)  000082e300000-000082fffffff type=0 attr=000000000000000f
(XEN) alt table ffff82d0408925b0 -> ffff82d04089d9b8
(XEN) AMD-Vi: IOMMU Extended Features:
(XEN) - Peripheral Page Service Request
(XEN) - x2APIC
(XEN) - NX bit
(XEN) - Invalidate All Command
(XEN) - Guest APIC
(XEN) - Performance Counters
(XEN) - Host Address Translation Size: 0x2
(XEN) - Guest Address Translation Size: 0
(XEN) - Guest CR3 Root Table Level: 0x1
(XEN) - Maximum PASID: 0xf
(XEN) - SMI Filter Register: 0x1
(XEN) - SMI Filter Register Count: 0x1
(XEN) - Guest Virtual APIC Modes: 0x1
(XEN) - Dual PPR Log: 0x2
(XEN) - Dual Event Log: 0x2
(XEN) - User / Supervisor Page Protection
(XEN) - Device Table Segmentation: 0x3
(XEN) - PPR Log Overflow Early Warning
(XEN) - PPR Automatic Response
(XEN) - Memory Access Routing and Control: 0
(XEN) - Block StopMark Message
(XEN) - Performance Optimization
(XEN) - MSI Capability MMIO Access
(XEN) - Guest I/O Protection
(XEN) - Enhanced PPR Handling
(XEN) - Attribute Forward
(XEN) - Invalidate IOTLB Type
(XEN) - VM Table Size: 0
(XEN) - Guest Access Bit Update Disable
(XEN) AMD-Vi: Disabled HAP memory map sharing with IOMMU
(XEN) ----[ Xen-4.16.1-1-arch  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    e008:[<ffff82d040827fec>] amd_iommu_init+0x38c/0x3d0
(XEN) RFLAGS: 0000000000010246   CONTEXT: hypervisor
(XEN) rax: 00000fff00000000   rbx: 0000000000000001   rcx: 3333333333333333
(XEN) rdx: 0000000000000000   rsi: 0000000000000010   rdi: ffff82d040a0b420
(XEN) rbp: ffff82d040b31d20   rsp: ffff82d040857d28   r8: 0000000000000000
(XEN) r9:  ffff82d040a0b420   r10: ffff82d040857bd8   r11: 0000000000000004
(XEN) r12: 0000000000000000   r13: 00000fff00000000   r14: 0000000000000000
(XEN) r15: ffff830818dbabf0   cr0: 000000008005003b   cr4: 0000000000f506e0
(XEN) cr3: 000000009aa16000   cr2: 0000000000000000
(XEN) fsb: 0000000000000000   gsb: 0000000000000000   gss: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
(XEN) Xen code around <ffff82d040827fec> (amd_iommu_init+0x38c/0x3d0):
(XEN)  00 00 00 00 ff 0f 00 00 <0f> 6e c2 0f 62 c0 49 89 87 c0 00 00 00
41 0f 7f
(XEN) Xen stack trace from rsp=ffff82d040857d28:
(XEN)    0000000018da5fc0 ffff830814800000 0000000000000000 ffff830818da5fc0
(XEN)    ffff82d040a0b8a0 ffff82d040857e50 ffff82d040449ae8 0000000000000020
(XEN)    ffff82d0408282c6 ffff82d04082b41c 00000000ffffffed 0000000000000000
(XEN)    ffff82d04082bd78 ffff82d040857fff 0000000000000000 ffff82d040845195
(XEN)    ffff83009a86b560 0000000000000000 ffff83009a85b3e6 ffff830818dc0000
(XEN)    000000000000000f 0000000003cf9000 ffff83009a86b500 000000000082e300
(XEN)    ffff83009a86b500 0000000000000000 ffff82d040857ef8 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 000000009a86b500
(XEN)    0000000003cf9fff 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000002 ffff82d040b1a000 ffff82d040b1a000
(XEN)    0000000800000000 000000010000006e 0000000000000003 00000000000002f8
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    ffffffffffffffff 0000000099ee9180 0000000000000000 000000009a85b3f5
(XEN)    000000009f4d69a0 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) Xen call trace:
(XEN)    [<ffff82d040827fec>] R amd_iommu_init+0x38c/0x3d0
(XEN)    [<ffff82d0408282c6>] S
(XEN)    [<ffff82d04082b41c>] S iommu_hardware_setup+0x9c/0x100
(XEN)    [<ffff82d04082bd78>] S iommu_setup+0x48/0x1a0
(XEN)    [<ffff82d040845195>] S __start_xen+0x15c5/0x29b5
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) FATAL TRAP: vec 7, #NM[0000]
(XEN) ****************************************
(XEN) Manual reset required ('noreboot' specified)