Mailing List Archive

Hi Jeremy
please open an issue on https://github.com/ntop/PF_RING/issues <https://github.com/ntop/PF_RING/issues> to keep track on this,
please also post a full dmesg output if possible.

Thank you
Alfredo

> On 30 Oct 2017, at 16:55, Jeremy Erb <jeremy.erb@netsweeper.com> wrote:
>
> Upgrading to PF_RING 7.0.0 with the new i40e 2.2.4 causes a kernel panic for us on CentOS 6 kernel. The vanilla i40e 2.2.4 driver does not cause this issue. Here are two kernel dumps. The i40e 1.5.18 driver works in libpfring 6.7.0.
>
>
>
> Pid: 10278, comm: modprobe Not tainted 2.6.32-696.13.2.el6.x86_64 #1 Dell Inc. PowerEdge R910/0JRJM9
> RIP: 0010:[<ffffffffa08c74a3>] [<ffffffffa08c74a3>] i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
> RSP: 0018:ffff880861a53908 EFLAGS: 00010202
> RAX: 0000000000000000 RBX: ffff88106dedcc00 RCX: ffff88106dedcc00
> RDX: ffff88106de0ed40 RSI: 0000000000000010 RDI: ffff88104e0b2000
> RBP: ffff880861a53968 R08: 000000000003ae77 R09: 0000000000000000
> R10: 0000000000000080 R11: 0000000000000001 R12: ffff88104e0b2000
> R13: ffff88106e665f00 R14: 00010102464c457f R15: 0000000000000010
> FS: 00007fd3655b4700(0000) GS:ffff88089c440000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: 00007fee9e121000 CR3: 00000010425ed000 CR4: 00000000000007e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Process modprobe (pid: 10278, threadinfo ffff880861a50000, task ffff8805f8c8f520)
> Stack:
> ffff88104e0b2000 ffff88104e0b2000 ffff881063754000 00000020202ca020
> <d> 0000001000000010 0000000100000010 ffff880861a53968 ffff88104e0b2000
> <d> ffff881063754000 ffff881063754000 ffff88104e0b2028 ffff880861a53998
> Call Trace:
> [<ffffffffa08cd273>] i40e_vsi_setup+0x543/0x880 [i40e_zc]
> [<ffffffffa08e3c9d>] ? i40e_aq_set_switch_config+0x9d/0xd0 [i40e_zc]
> [<ffffffffa08d2d3f>] i40e_setup_pf_switch+0x47f/0x5d0 [i40e_zc]
> [<ffffffffa08fc5cf>] i40e_probe+0xd8a/0x17e8 [i40e_zc]
> [<ffffffff8154a92e>] ? schedule+0x3ee/0xb70
> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
> [<ffffffff812a3ece>] ? number+0x2ee/0x320
> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
> [<ffffffff811b762e>] ? find_inode+0x4e/0x90
> [<ffffffff81217fc0>] ? sysfs_ilookup_test+0x0/0x20
> [<ffffffff811b71d0>] ? iput+0x30/0x70
> [<ffffffff81218bbe>] ? sysfs_addrm_finish+0x4e/0x270
> [<ffffffff8121947e>] ? __sysfs_add_one+0x7e/0xc0
> [<ffffffff812194ec>] ? sysfs_add_one+0x2c/0xd0
> [<ffffffff812c0aa7>] local_pci_probe+0x17/0x20
> [<ffffffff812c1c91>] pci_device_probe+0x101/0x120
> [<ffffffff81383bf2>] ? driver_sysfs_add+0x62/0x90
> [<ffffffff81383e9a>] driver_probe_device+0xaa/0x3a0
> [<ffffffff8138423b>] __driver_attach+0xab/0xb0
> [<ffffffff81384190>] ? __driver_attach+0x0/0xb0
> [<ffffffff81383084>] bus_for_each_dev+0x64/0x90
> [<ffffffff81383b2e>] driver_attach+0x1e/0x20
> [<ffffffff81382898>] bus_add_driver+0x1e8/0x2b0
> [<ffffffff8138444f>] driver_register+0x5f/0xe0
> [<ffffffffa0504000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
> [<ffffffff812c1ef6>] __pci_register_driver+0x56/0xd0
> [<ffffffff8122400b>] ? debugfs_create_dir+0x1b/0x20
> udev: renamed network interface eth10 to eth16
> [<ffffffffa0504000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
> [<ffffffffa05040a1>] i40e_init_module+0xa1/0xa3 [i40e_zc]
> [<ffffffff810020d0>] do_one_initcall+0xc0/0x280
> [<ffffffff810c84f1>] sys_init_module+0xe1/0x250
> [<ffffffff8100b0d2>] system_call_fastpath+0x16/0x1b
> Code: 00 49 89 9d a8 00 00 00 48 8b 83 18 01 00 00 49 89 45 00 66 83 83 30 01 00 00 01 4d 85 f6 4c 89 ab 18 01 00 00 0f 84 05 01 00 00 <49> 89 9e a8 00 00 00 48 8b 83 f0 00 00 00 41 83 c7 01 49 89 06
> RIP [<ffffffffa08c74a3>] i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
> RSP <ffff880861a53908>
> ---[ end trace c4414a8eb6ab10b9 ]---
> Kernel panic - not syncing: Fatal exception
> Pid: 10278, comm: modprobe Tainted: G D -- ------------ 2.6.32-696.13.2.el6.x86_64 #1
> Call Trace:
> [<ffffffff8154a20e>] ? panic+0xa7/0x179
> [<ffffffff8154f0a4>] ? oops_end+0xe4/0x100
> [<ffffffff8101101b>] ? die+0x5b/0x90
> [<ffffffff8154eb62>] ? do_general_protection+0x152/0x160
> [<ffffffff8154e2d5>] ? general_protection+0x25/0x30
> [<ffffffffa08c74a3>] ? i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
> [<ffffffffa08cd273>] ? i40e_vsi_setup+0x543/0x880 [i40e_zc]
> [<ffffffffa08e3c9d>] ? i40e_aq_set_switch_config+0x9d/0xd0 [i40e_zc]
> [<ffffffffa08d2d3f>] ? i40e_setup_pf_switch+0x47f/0x5d0 [i40e_zc]
> [<ffffffffa08fc5cf>] ? i40e_probe+0xd8a/0x17e8 [i40e_zc]
> [<ffffffff8154a92e>] ? schedule+0x3ee/0xb70
> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
> [<ffffffff812a3ece>] ? number+0x2ee/0x320
> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
> [<ffffffff811b762e>] ? find_inode+0x4e/0x90
> [<ffffffff81217fc0>] ? sysfs_ilookup_test+0x0/0x20
> [<ffffffff811b71d0>] ? iput+0x30/0x70
> [<ffffffff81218bbe>] ? sysfs_addrm_finish+0x4e/0x270
> [<ffffffff8121947e>] ? __sysfs_add_one+0x7e/0xc0
> [<ffffffff812194ec>] ? sysfs_add_one+0x2c/0xd0
> [<ffffffff812c0aa7>] ? local_pci_probe+0x17/0x20
> [<ffffffff812c1c91>] ? pci_device_probe+0x101/0x120
> [<ffffffff81383bf2>] ? driver_sysfs_add+0x62/0x90
> [<ffffffff81383e9a>] ? driver_probe_device+0xaa/0x3a0
> [<ffffffff8138423b>] ? __driver_attach+0xab/0xb0
> [<ffffffff81384190>] ? __driver_attach+0x0/0xb0
> [<ffffffff81383084>] ? bus_for_each_dev+0x64/0x90
> [<ffffffff81383b2e>] ? driver_attach+0x1e/0x20
> [<ffffffff81382898>] ? bus_add_driver+0x1e8/0x2b0
> [<ffffffff8138444f>] ? driver_register+0x5f/0xe0
> [<ffffffffa0504000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
> [<ffffffff812c1ef6>] ? __pci_register_driver+0x56/0xd0
> [<ffffffff8122400b>] ? debugfs_create_dir+0x1b/0x20
> [<ffffffffa0504000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
> [<ffffffffa05040a1>] ? i40e_init_module+0xa1/0xa3 [i40e_zc]
> [<ffffffff810020d0>] ? do_one_initcall+0xc0/0x280
> [<ffffffff810c84f1>] ? sys_init_module+0xe1/0x250
> [<ffffffff8100b0d2>] ? system_call_fastpath+0x16/0x1b
>
>
>
>
>
>
>
>
>
>
> And the second dump.
>
>
>
>
>
>
> BUG: unable to handle kernel NULL pointer dereference at 00000000000000a8
> IP: [<ffffffffa0bac4a3>] i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
> PGD 10716fc067 PUD 1063b61067 PMD 0
> Oops: 0002 [#1] SMP
> last sysfs file: /sys/devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth10/type
> CPU 21
> Modules linked in: i40e_zc(+)(U) ixgbe_zc(U) pf_ring(U) vxlan netconsole configfs autofs4 bnx2fc fcoe libfcoe libfc 8021q scsi_transport_fc garp stp scsi_tg
> t llc ipmi_devintf power_meter acpi_ipmi ipmi_si ipmi_msghandler iTCO_wdt iTCO_vendor_support dcdbas serio_raw lpc_ich mfd_core sg i7core_edac edac_core ext
> 4 jbd2 mbcache sr_mod cdrom sd_mod crc_t10dif pata_acpi ata_generic ata_piix igb dca i2c_algo_bit i2c_core bnx2x ptp pps_core libcrc32c crc32c_intel be2iscs
> i bnx2i cnic uio cxgb4i iw_cxgb4 iw_cm cxgb4 cxgb3i libcxgbi iw_cxgb3 ib_core ib_addr ipv6 cxgb3 mdio libiscsi_tcp mpt2sas qla4xxx scsi_transport_sas raid_c
> lass iscsi_boot_sysfs libiscsi bnx2 scsi_transport_iscsi wmi dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ixgbe]
>
> Pid: 6259, comm: modprobe Not tainted 2.6.32-696.13.2.el6.x86_64 #1 Dell Inc. PowerEdge R910/0JRJM9
> RIP: 0010:[<ffffffffa0bac4a3>] [<ffffffffa0bac4a3>] i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
> RSP: 0018:ffff88085ad9f908 EFLAGS: 00010292
> RAX: 0000000000000034 RBX: ffff88106e591000 RCX: 0000000000004df1
> RDX: 0000000000000000 RSI: 0000000000000046 RDI: 0000000000000246
> RBP: ffff88085ad9f968 R08: 000000000002a73e R09: 0000000000000000
> R10: 0000000000000080 R11: 0000000000000001 R12: ffff88106e499000
> R13: ffff8810708e7b00 R14: 0000000000000000 R15: 0000000000000010
> udev: renamed network interface eth10 to eth16
> FS: 00007f8f3620d700(0000) GS:ffff88089c540000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: 00000000000000a8 CR3: 0000001063cbe000 CR4: 00000000000007e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Process modprobe (pid: 6259, threadinfo ffff88085ad9c000, task ffff88085cd16040)
> Stack:
> ffff88106e499000 0000000000000080 ffff881000000000 000000205728d020
> <d> 0000001000000010 0000000100000010 ffff88085ad9f968 ffff88106e499000
> <d> ffff88106e632000 ffff88106e632000 ffff88106e499028 ffff88085ad9f998
> Call Trace:
> [<ffffffffa0bb2273>] i40e_vsi_setup+0x543/0x880 [i40e_zc]
> [<ffffffffa0bc8c9d>] ? i40e_aq_set_switch_config+0x9d/0xd0 [i40e_zc]
> [<ffffffffa0bb7d3f>] i40e_setup_pf_switch+0x47f/0x5d0 [i40e_zc]
> [<ffffffffa0be15cf>] i40e_probe+0xd8a/0x17e8 [i40e_zc]
> [<ffffffff8113e0b9>] ? __alloc_pages_nodemask+0x129/0x950
> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
> [<ffffffff812a3ece>] ? number+0x2ee/0x320
> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
> [<ffffffff811b762e>] ? find_inode+0x4e/0x90
> [<ffffffff81217fc0>] ? sysfs_ilookup_test+0x0/0x20
> [<ffffffff811b71d0>] ? iput+0x30/0x70
> [<ffffffff81218bbe>] ? sysfs_addrm_finish+0x4e/0x270
> [<ffffffff8121947e>] ? __sysfs_add_one+0x7e/0xc0
> [<ffffffff812194ec>] ? sysfs_add_one+0x2c/0xd0
> [<ffffffff812c0aa7>] local_pci_probe+0x17/0x20
> [<ffffffff812c1c91>] pci_device_probe+0x101/0x120
> [<ffffffff81383bf2>] ? driver_sysfs_add+0x62/0x90
> [<ffffffff81383e9a>] driver_probe_device+0xaa/0x3a0
> [<ffffffff8138423b>] __driver_attach+0xab/0xb0
> [<ffffffff81384190>] ? __driver_attach+0x0/0xb0
> [<ffffffff81383084>] bus_for_each_dev+0x64/0x90
> [<ffffffff81383b2e>] driver_attach+0x1e/0x20
> [<ffffffff81382898>] bus_add_driver+0x1e8/0x2b0
> [<ffffffff8138444f>] driver_register+0x5f/0xe0
> [<ffffffffa0c01000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
> [<ffffffff812c1ef6>] __pci_register_driver+0x56/0xd0
> [<ffffffff8122400b>] ? debugfs_create_dir+0x1b/0x20
> [<ffffffffa0c01000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
> [<ffffffffa0c010a1>] i40e_init_module+0xa1/0xa3 [i40e_zc]
> [<ffffffff810020d0>] do_one_initcall+0xc0/0x280
> [<ffffffff810c84f1>] sys_init_module+0xe1/0x250
> [<ffffffff8100b0d2>] system_call_fastpath+0x16/0x1b
> Code: 00 49 89 9d a8 00 00 00 48 8b 83 18 01 00 00 49 89 45 00 66 83 83 30 01 00 00 01 4d 85 f6 4c 89 ab 18 01 00 00 0f 84 05 01 00 00 <49> 89 9e a8 00 00 00 48 8b 83 f0 00 00 00 41 83 c7 01 49 89 06
> RIP [<ffffffffa0bac4a3>] i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
> RSP <ffff88085ad9f908>
> CR2: 00000000000000a8
> ---[ end trace 70024f26b6d6bbcb ]---
> Kernel panic - not syncing: Fatal exception
> Pid: 6259, comm: modprobe Tainted: G D -- ------------ 2.6.32-696.13.2.el6.x86_64 #1
> Call Trace:
> [<ffffffff8154a20e>] ? panic+0xa7/0x179
> [<ffffffff8154f0a4>] ? oops_end+0xe4/0x100
> [<ffffffff8105186b>] ? no_context+0xfb/0x260
> [<ffffffff8105fa0c>] ? check_preempt_curr+0x7c/0x90
> [<ffffffff81051af5>] ? __bad_area_nosemaphore+0x125/0x1e0
> [<ffffffff81051c1e>] ? bad_area+0x4e/0x60
> [<ffffffff81052443>] ? __do_page_fault+0x473/0x500
> [<ffffffff8155100e>] ? do_page_fault+0x3e/0xa0
> [<ffffffff8154e305>] ? page_fault+0x25/0x30
> [<ffffffffa0bac4a3>] ? i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
> [<ffffffffa0bac5e5>] ? i40e_vsi_map_rings_to_vectors+0x235/0x280 [i40e_zc]
> [<ffffffffa0bb2273>] ? i40e_vsi_setup+0x543/0x880 [i40e_zc]
> [<ffffffffa0bc8c9d>] ? i40e_aq_set_switch_config+0x9d/0xd0 [i40e_zc]
> [<ffffffffa0bb7d3f>] ? i40e_setup_pf_switch+0x47f/0x5d0 [i40e_zc]
> [<ffffffffa0be15cf>] ? i40e_probe+0xd8a/0x17e8 [i40e_zc]
> [<ffffffff8113e0b9>] ? __alloc_pages_nodemask+0x129/0x950
> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
> [<ffffffff812a3ece>] ? number+0x2ee/0x320
> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
> [<ffffffff811b762e>] ? find_inode+0x4e/0x90
> [<ffffffff81217fc0>] ? sysfs_ilookup_test+0x0/0x20
> [<ffffffff811b71d0>] ? iput+0x30/0x70
> [<ffffffff81218bbe>] ? sysfs_addrm_finish+0x4e/0x270
> [<ffffffff8121947e>] ? __sysfs_add_one+0x7e/0xc0
> [<ffffffff812194ec>] ? sysfs_add_one+0x2c/0xd0
> [<ffffffff812c0aa7>] ? local_pci_probe+0x17/0x20
> [<ffffffff812c1c91>] ? pci_device_probe+0x101/0x120
> [<ffffffff81383bf2>] ? driver_sysfs_add+0x62/0x90
> [<ffffffff81383e9a>] ? driver_probe_device+0xaa/0x3a0
> [<ffffffff8138423b>] ? __driver_attach+0xab/0xb0
> [<ffffffff81384190>] ? __driver_attach+0x0/0xb0
> [<ffffffff81383084>] ? bus_for_each_dev+0x64/0x90
> [<ffffffff81383b2e>] ? driver_attach+0x1e/0x20
> [<ffffffff81382898>] ? bus_add_driver+0x1e8/0x2b0
> [<ffffffff8138444f>] ? driver_register+0x5f/0xe0
> [<ffffffffa0c01000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
> [<ffffffff812c1ef6>] ? __pci_register_driver+0x56/0xd0
> [<ffffffff8122400b>] ? debugfs_create_dir+0x1b/0x20
> [<ffffffffa0c01000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
> [<ffffffffa0c010a1>] ? i40e_init_module+0xa1/0xa3 [i40e_zc]
> [<ffffffff810020d0>] ? do_one_initcall+0xc0/0x280
> [<ffffffff810c84f1>] ? sys_init_module+0xe1/0x250
> [<ffffffff8100b0d2>] ? system_call_fastpath+0x16/0x1b
>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it <mailto:Ntop-misc@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc <http://listgateway.unipi.it/mailman/listinfo/ntop-misc>

Hi Jeremy
I added more verbosity, please use https://github.com/ntop/PF_RING/tree/dev/drivers/intel/i40e/i40e-2.2.4-zc/src <https://github.com/ntop/PF_RING/tree/dev/drivers/intel/i40e/i40e-2.2.4-zc/src>
loading it with load_driver.sh (this to avoid reinstalling new packages on every test).
Please provide the full dmesg output after a crash.

Thank you
Alfredo

> On 30 Oct 2017, at 17:00, Alfredo Cardigliano <cardigliano@ntop.org> wrote:
>
> Hi Jeremy
> please open an issue on https://github.com/ntop/PF_RING/issues <https://github.com/ntop/PF_RING/issues> to keep track on this,
> please also post a full dmesg output if possible.
>
> Thank you
> Alfredo
>
>> On 30 Oct 2017, at 16:55, Jeremy Erb <jeremy.erb@netsweeper.com <mailto:jeremy.erb@netsweeper.com>> wrote:
>>
>> Upgrading to PF_RING 7.0.0 with the new i40e 2.2.4 causes a kernel panic for us on CentOS 6 kernel. The vanilla i40e 2.2.4 driver does not cause this issue. Here are two kernel dumps. The i40e 1.5.18 driver works in libpfring 6.7.0.
>>
>>
>>
>> Pid: 10278, comm: modprobe Not tainted 2.6.32-696.13.2.el6.x86_64 #1 Dell Inc. PowerEdge R910/0JRJM9
>> RIP: 0010:[<ffffffffa08c74a3>] [<ffffffffa08c74a3>] i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
>> RSP: 0018:ffff880861a53908 EFLAGS: 00010202
>> RAX: 0000000000000000 RBX: ffff88106dedcc00 RCX: ffff88106dedcc00
>> RDX: ffff88106de0ed40 RSI: 0000000000000010 RDI: ffff88104e0b2000
>> RBP: ffff880861a53968 R08: 000000000003ae77 R09: 0000000000000000
>> R10: 0000000000000080 R11: 0000000000000001 R12: ffff88104e0b2000
>> R13: ffff88106e665f00 R14: 00010102464c457f R15: 0000000000000010
>> FS: 00007fd3655b4700(0000) GS:ffff88089c440000(0000) knlGS:0000000000000000
>> CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
>> CR2: 00007fee9e121000 CR3: 00000010425ed000 CR4: 00000000000007e0
>> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>> Process modprobe (pid: 10278, threadinfo ffff880861a50000, task ffff8805f8c8f520)
>> Stack:
>> ffff88104e0b2000 ffff88104e0b2000 ffff881063754000 00000020202ca020
>> <d> 0000001000000010 0000000100000010 ffff880861a53968 ffff88104e0b2000
>> <d> ffff881063754000 ffff881063754000 ffff88104e0b2028 ffff880861a53998
>> Call Trace:
>> [<ffffffffa08cd273>] i40e_vsi_setup+0x543/0x880 [i40e_zc]
>> [<ffffffffa08e3c9d>] ? i40e_aq_set_switch_config+0x9d/0xd0 [i40e_zc]
>> [<ffffffffa08d2d3f>] i40e_setup_pf_switch+0x47f/0x5d0 [i40e_zc]
>> [<ffffffffa08fc5cf>] i40e_probe+0xd8a/0x17e8 [i40e_zc]
>> [<ffffffff8154a92e>] ? schedule+0x3ee/0xb70
>> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
>> [<ffffffff812a3ece>] ? number+0x2ee/0x320
>> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
>> [<ffffffff811b762e>] ? find_inode+0x4e/0x90
>> [<ffffffff81217fc0>] ? sysfs_ilookup_test+0x0/0x20
>> [<ffffffff811b71d0>] ? iput+0x30/0x70
>> [<ffffffff81218bbe>] ? sysfs_addrm_finish+0x4e/0x270
>> [<ffffffff8121947e>] ? __sysfs_add_one+0x7e/0xc0
>> [<ffffffff812194ec>] ? sysfs_add_one+0x2c/0xd0
>> [<ffffffff812c0aa7>] local_pci_probe+0x17/0x20
>> [<ffffffff812c1c91>] pci_device_probe+0x101/0x120
>> [<ffffffff81383bf2>] ? driver_sysfs_add+0x62/0x90
>> [<ffffffff81383e9a>] driver_probe_device+0xaa/0x3a0
>> [<ffffffff8138423b>] __driver_attach+0xab/0xb0
>> [<ffffffff81384190>] ? __driver_attach+0x0/0xb0
>> [<ffffffff81383084>] bus_for_each_dev+0x64/0x90
>> [<ffffffff81383b2e>] driver_attach+0x1e/0x20
>> [<ffffffff81382898>] bus_add_driver+0x1e8/0x2b0
>> [<ffffffff8138444f>] driver_register+0x5f/0xe0
>> [<ffffffffa0504000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
>> [<ffffffff812c1ef6>] __pci_register_driver+0x56/0xd0
>> [<ffffffff8122400b>] ? debugfs_create_dir+0x1b/0x20
>> udev: renamed network interface eth10 to eth16
>> [<ffffffffa0504000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
>> [<ffffffffa05040a1>] i40e_init_module+0xa1/0xa3 [i40e_zc]
>> [<ffffffff810020d0>] do_one_initcall+0xc0/0x280
>> [<ffffffff810c84f1>] sys_init_module+0xe1/0x250
>> [<ffffffff8100b0d2>] system_call_fastpath+0x16/0x1b
>> Code: 00 49 89 9d a8 00 00 00 48 8b 83 18 01 00 00 49 89 45 00 66 83 83 30 01 00 00 01 4d 85 f6 4c 89 ab 18 01 00 00 0f 84 05 01 00 00 <49> 89 9e a8 00 00 00 48 8b 83 f0 00 00 00 41 83 c7 01 49 89 06
>> RIP [<ffffffffa08c74a3>] i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
>> RSP <ffff880861a53908>
>> ---[ end trace c4414a8eb6ab10b9 ]---
>> Kernel panic - not syncing: Fatal exception
>> Pid: 10278, comm: modprobe Tainted: G D -- ------------ 2.6.32-696.13.2.el6.x86_64 #1
>> Call Trace:
>> [<ffffffff8154a20e>] ? panic+0xa7/0x179
>> [<ffffffff8154f0a4>] ? oops_end+0xe4/0x100
>> [<ffffffff8101101b>] ? die+0x5b/0x90
>> [<ffffffff8154eb62>] ? do_general_protection+0x152/0x160
>> [<ffffffff8154e2d5>] ? general_protection+0x25/0x30
>> [<ffffffffa08c74a3>] ? i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
>> [<ffffffffa08cd273>] ? i40e_vsi_setup+0x543/0x880 [i40e_zc]
>> [<ffffffffa08e3c9d>] ? i40e_aq_set_switch_config+0x9d/0xd0 [i40e_zc]
>> [<ffffffffa08d2d3f>] ? i40e_setup_pf_switch+0x47f/0x5d0 [i40e_zc]
>> [<ffffffffa08fc5cf>] ? i40e_probe+0xd8a/0x17e8 [i40e_zc]
>> [<ffffffff8154a92e>] ? schedule+0x3ee/0xb70
>> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
>> [<ffffffff812a3ece>] ? number+0x2ee/0x320
>> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
>> [<ffffffff811b762e>] ? find_inode+0x4e/0x90
>> [<ffffffff81217fc0>] ? sysfs_ilookup_test+0x0/0x20
>> [<ffffffff811b71d0>] ? iput+0x30/0x70
>> [<ffffffff81218bbe>] ? sysfs_addrm_finish+0x4e/0x270
>> [<ffffffff8121947e>] ? __sysfs_add_one+0x7e/0xc0
>> [<ffffffff812194ec>] ? sysfs_add_one+0x2c/0xd0
>> [<ffffffff812c0aa7>] ? local_pci_probe+0x17/0x20
>> [<ffffffff812c1c91>] ? pci_device_probe+0x101/0x120
>> [<ffffffff81383bf2>] ? driver_sysfs_add+0x62/0x90
>> [<ffffffff81383e9a>] ? driver_probe_device+0xaa/0x3a0
>> [<ffffffff8138423b>] ? __driver_attach+0xab/0xb0
>> [<ffffffff81384190>] ? __driver_attach+0x0/0xb0
>> [<ffffffff81383084>] ? bus_for_each_dev+0x64/0x90
>> [<ffffffff81383b2e>] ? driver_attach+0x1e/0x20
>> [<ffffffff81382898>] ? bus_add_driver+0x1e8/0x2b0
>> [<ffffffff8138444f>] ? driver_register+0x5f/0xe0
>> [<ffffffffa0504000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
>> [<ffffffff812c1ef6>] ? __pci_register_driver+0x56/0xd0
>> [<ffffffff8122400b>] ? debugfs_create_dir+0x1b/0x20
>> [<ffffffffa0504000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
>> [<ffffffffa05040a1>] ? i40e_init_module+0xa1/0xa3 [i40e_zc]
>> [<ffffffff810020d0>] ? do_one_initcall+0xc0/0x280
>> [<ffffffff810c84f1>] ? sys_init_module+0xe1/0x250
>> [<ffffffff8100b0d2>] ? system_call_fastpath+0x16/0x1b
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> And the second dump.
>>
>>
>>
>>
>>
>>
>> BUG: unable to handle kernel NULL pointer dereference at 00000000000000a8
>> IP: [<ffffffffa0bac4a3>] i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
>> PGD 10716fc067 PUD 1063b61067 PMD 0
>> Oops: 0002 [#1] SMP
>> last sysfs file: /sys/devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth10/type
>> CPU 21
>> Modules linked in: i40e_zc(+)(U) ixgbe_zc(U) pf_ring(U) vxlan netconsole configfs autofs4 bnx2fc fcoe libfcoe libfc 8021q scsi_transport_fc garp stp scsi_tg
>> t llc ipmi_devintf power_meter acpi_ipmi ipmi_si ipmi_msghandler iTCO_wdt iTCO_vendor_support dcdbas serio_raw lpc_ich mfd_core sg i7core_edac edac_core ext
>> 4 jbd2 mbcache sr_mod cdrom sd_mod crc_t10dif pata_acpi ata_generic ata_piix igb dca i2c_algo_bit i2c_core bnx2x ptp pps_core libcrc32c crc32c_intel be2iscs
>> i bnx2i cnic uio cxgb4i iw_cxgb4 iw_cm cxgb4 cxgb3i libcxgbi iw_cxgb3 ib_core ib_addr ipv6 cxgb3 mdio libiscsi_tcp mpt2sas qla4xxx scsi_transport_sas raid_c
>> lass iscsi_boot_sysfs libiscsi bnx2 scsi_transport_iscsi wmi dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ixgbe]
>>
>> Pid: 6259, comm: modprobe Not tainted 2.6.32-696.13.2.el6.x86_64 #1 Dell Inc. PowerEdge R910/0JRJM9
>> RIP: 0010:[<ffffffffa0bac4a3>] [<ffffffffa0bac4a3>] i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
>> RSP: 0018:ffff88085ad9f908 EFLAGS: 00010292
>> RAX: 0000000000000034 RBX: ffff88106e591000 RCX: 0000000000004df1
>> RDX: 0000000000000000 RSI: 0000000000000046 RDI: 0000000000000246
>> RBP: ffff88085ad9f968 R08: 000000000002a73e R09: 0000000000000000
>> R10: 0000000000000080 R11: 0000000000000001 R12: ffff88106e499000
>> R13: ffff8810708e7b00 R14: 0000000000000000 R15: 0000000000000010
>> udev: renamed network interface eth10 to eth16
>> FS: 00007f8f3620d700(0000) GS:ffff88089c540000(0000) knlGS:0000000000000000
>> CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
>> CR2: 00000000000000a8 CR3: 0000001063cbe000 CR4: 00000000000007e0
>> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>> Process modprobe (pid: 6259, threadinfo ffff88085ad9c000, task ffff88085cd16040)
>> Stack:
>> ffff88106e499000 0000000000000080 ffff881000000000 000000205728d020
>> <d> 0000001000000010 0000000100000010 ffff88085ad9f968 ffff88106e499000
>> <d> ffff88106e632000 ffff88106e632000 ffff88106e499028 ffff88085ad9f998
>> Call Trace:
>> [<ffffffffa0bb2273>] i40e_vsi_setup+0x543/0x880 [i40e_zc]
>> [<ffffffffa0bc8c9d>] ? i40e_aq_set_switch_config+0x9d/0xd0 [i40e_zc]
>> [<ffffffffa0bb7d3f>] i40e_setup_pf_switch+0x47f/0x5d0 [i40e_zc]
>> [<ffffffffa0be15cf>] i40e_probe+0xd8a/0x17e8 [i40e_zc]
>> [<ffffffff8113e0b9>] ? __alloc_pages_nodemask+0x129/0x950
>> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
>> [<ffffffff812a3ece>] ? number+0x2ee/0x320
>> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
>> [<ffffffff811b762e>] ? find_inode+0x4e/0x90
>> [<ffffffff81217fc0>] ? sysfs_ilookup_test+0x0/0x20
>> [<ffffffff811b71d0>] ? iput+0x30/0x70
>> [<ffffffff81218bbe>] ? sysfs_addrm_finish+0x4e/0x270
>> [<ffffffff8121947e>] ? __sysfs_add_one+0x7e/0xc0
>> [<ffffffff812194ec>] ? sysfs_add_one+0x2c/0xd0
>> [<ffffffff812c0aa7>] local_pci_probe+0x17/0x20
>> [<ffffffff812c1c91>] pci_device_probe+0x101/0x120
>> [<ffffffff81383bf2>] ? driver_sysfs_add+0x62/0x90
>> [<ffffffff81383e9a>] driver_probe_device+0xaa/0x3a0
>> [<ffffffff8138423b>] __driver_attach+0xab/0xb0
>> [<ffffffff81384190>] ? __driver_attach+0x0/0xb0
>> [<ffffffff81383084>] bus_for_each_dev+0x64/0x90
>> [<ffffffff81383b2e>] driver_attach+0x1e/0x20
>> [<ffffffff81382898>] bus_add_driver+0x1e8/0x2b0
>> [<ffffffff8138444f>] driver_register+0x5f/0xe0
>> [<ffffffffa0c01000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
>> [<ffffffff812c1ef6>] __pci_register_driver+0x56/0xd0
>> [<ffffffff8122400b>] ? debugfs_create_dir+0x1b/0x20
>> [<ffffffffa0c01000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
>> [<ffffffffa0c010a1>] i40e_init_module+0xa1/0xa3 [i40e_zc]
>> [<ffffffff810020d0>] do_one_initcall+0xc0/0x280
>> [<ffffffff810c84f1>] sys_init_module+0xe1/0x250
>> [<ffffffff8100b0d2>] system_call_fastpath+0x16/0x1b
>> Code: 00 49 89 9d a8 00 00 00 48 8b 83 18 01 00 00 49 89 45 00 66 83 83 30 01 00 00 01 4d 85 f6 4c 89 ab 18 01 00 00 0f 84 05 01 00 00 <49> 89 9e a8 00 00 00 48 8b 83 f0 00 00 00 41 83 c7 01 49 89 06
>> RIP [<ffffffffa0bac4a3>] i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
>> RSP <ffff88085ad9f908>
>> CR2: 00000000000000a8
>> ---[ end trace 70024f26b6d6bbcb ]---
>> Kernel panic - not syncing: Fatal exception
>> Pid: 6259, comm: modprobe Tainted: G D -- ------------ 2.6.32-696.13.2.el6.x86_64 #1
>> Call Trace:
>> [<ffffffff8154a20e>] ? panic+0xa7/0x179
>> [<ffffffff8154f0a4>] ? oops_end+0xe4/0x100
>> [<ffffffff8105186b>] ? no_context+0xfb/0x260
>> [<ffffffff8105fa0c>] ? check_preempt_curr+0x7c/0x90
>> [<ffffffff81051af5>] ? __bad_area_nosemaphore+0x125/0x1e0
>> [<ffffffff81051c1e>] ? bad_area+0x4e/0x60
>> [<ffffffff81052443>] ? __do_page_fault+0x473/0x500
>> [<ffffffff8155100e>] ? do_page_fault+0x3e/0xa0
>> [<ffffffff8154e305>] ? page_fault+0x25/0x30
>> [<ffffffffa0bac4a3>] ? i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
>> [<ffffffffa0bac5e5>] ? i40e_vsi_map_rings_to_vectors+0x235/0x280 [i40e_zc]
>> [<ffffffffa0bb2273>] ? i40e_vsi_setup+0x543/0x880 [i40e_zc]
>> [<ffffffffa0bc8c9d>] ? i40e_aq_set_switch_config+0x9d/0xd0 [i40e_zc]
>> [<ffffffffa0bb7d3f>] ? i40e_setup_pf_switch+0x47f/0x5d0 [i40e_zc]
>> [<ffffffffa0be15cf>] ? i40e_probe+0xd8a/0x17e8 [i40e_zc]
>> [<ffffffff8113e0b9>] ? __alloc_pages_nodemask+0x129/0x950
>> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
>> [<ffffffff812a3ece>] ? number+0x2ee/0x320
>> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
>> [<ffffffff811b762e>] ? find_inode+0x4e/0x90
>> [<ffffffff81217fc0>] ? sysfs_ilookup_test+0x0/0x20
>> [<ffffffff811b71d0>] ? iput+0x30/0x70
>> [<ffffffff81218bbe>] ? sysfs_addrm_finish+0x4e/0x270
>> [<ffffffff8121947e>] ? __sysfs_add_one+0x7e/0xc0
>> [<ffffffff812194ec>] ? sysfs_add_one+0x2c/0xd0
>> [<ffffffff812c0aa7>] ? local_pci_probe+0x17/0x20
>> [<ffffffff812c1c91>] ? pci_device_probe+0x101/0x120
>> [<ffffffff81383bf2>] ? driver_sysfs_add+0x62/0x90
>> [<ffffffff81383e9a>] ? driver_probe_device+0xaa/0x3a0
>> [<ffffffff8138423b>] ? __driver_attach+0xab/0xb0
>> [<ffffffff81384190>] ? __driver_attach+0x0/0xb0
>> [<ffffffff81383084>] ? bus_for_each_dev+0x64/0x90
>> [<ffffffff81383b2e>] ? driver_attach+0x1e/0x20
>> [<ffffffff81382898>] ? bus_add_driver+0x1e8/0x2b0
>> [<ffffffff8138444f>] ? driver_register+0x5f/0xe0
>> [<ffffffffa0c01000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
>> [<ffffffff812c1ef6>] ? __pci_register_driver+0x56/0xd0
>> [<ffffffff8122400b>] ? debugfs_create_dir+0x1b/0x20
>> [<ffffffffa0c01000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
>> [<ffffffffa0c010a1>] ? i40e_init_module+0xa1/0xa3 [i40e_zc]
>> [<ffffffff810020d0>] ? do_one_initcall+0xc0/0x280
>> [<ffffffff810c84f1>] ? sys_init_module+0xe1/0x250
>> [<ffffffff8100b0d2>] ? system_call_fastpath+0x16/0x1b
>>
>> _______________________________________________
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it <mailto:Ntop-misc@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc <http://listgateway.unipi.it/mailman/listinfo/ntop-misc>

Issue 227 opened, will post the output to the ticket.

Regards,

Jeremy




________________________________
From: ntop-misc-bounces@listgateway.unipi.it <ntop-misc-bounces@listgateway.unipi.it> on behalf of Alfredo Cardigliano <cardigliano@ntop.org>
Sent: Monday, October 30, 2017 12:23:06 PM
To: ntop-misc@listgateway.unipi.it
Subject: Re: [Ntop-misc] i40e 2.2.4 pfring 7.0.0

Hi Jeremy
I added more verbosity, please use https://github.com/ntop/PF_RING/tree/dev/drivers/intel/i40e/i40e-2.2.4-zc/src
loading it with load_driver.sh (this to avoid reinstalling new packages on every test).
Please provide the full dmesg output after a crash.

Thank you
Alfredo

On 30 Oct 2017, at 17:00, Alfredo Cardigliano <cardigliano@ntop.org<mailto:cardigliano@ntop.org>> wrote:

Hi Jeremy
please open an issue on https://github.com/ntop/PF_RING/issues to keep track on this,
please also post a full dmesg output if possible.

Thank you
Alfredo

On 30 Oct 2017, at 16:55, Jeremy Erb <jeremy.erb@netsweeper.com<mailto:jeremy.erb@netsweeper.com>> wrote:


Upgrading to PF_RING 7.0.0 with the new i40e 2.2.4 causes a kernel panic for us on CentOS 6 kernel. The vanilla i40e 2.2.4 driver does not cause this issue. Here are two kernel dumps. The i40e 1.5.18 driver works in libpfring 6.7.0.



Pid: 10278, comm: modprobe Not tainted 2.6.32-696.13.2.el6.x86_64 #1 Dell Inc. PowerEdge R910/0JRJM9
RIP: 0010:[<ffffffffa08c74a3>] [<ffffffffa08c74a3>] i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
RSP: 0018:ffff880861a53908 EFLAGS: 00010202
RAX: 0000000000000000 RBX: ffff88106dedcc00 RCX: ffff88106dedcc00
RDX: ffff88106de0ed40 RSI: 0000000000000010 RDI: ffff88104e0b2000
RBP: ffff880861a53968 R08: 000000000003ae77 R09: 0000000000000000
R10: 0000000000000080 R11: 0000000000000001 R12: ffff88104e0b2000
R13: ffff88106e665f00 R14: 00010102464c457f R15: 0000000000000010
FS: 00007fd3655b4700(0000) GS:ffff88089c440000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007fee9e121000 CR3: 00000010425ed000 CR4: 00000000000007e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process modprobe (pid: 10278, threadinfo ffff880861a50000, task ffff8805f8c8f520)
Stack:
ffff88104e0b2000 ffff88104e0b2000 ffff881063754000 00000020202ca020
<d> 0000001000000010 0000000100000010 ffff880861a53968 ffff88104e0b2000
<d> ffff881063754000 ffff881063754000 ffff88104e0b2028 ffff880861a53998
Call Trace:
[<ffffffffa08cd273>] i40e_vsi_setup+0x543/0x880 [i40e_zc]
[<ffffffffa08e3c9d>] ? i40e_aq_set_switch_config+0x9d/0xd0 [i40e_zc]
[<ffffffffa08d2d3f>] i40e_setup_pf_switch+0x47f/0x5d0 [i40e_zc]
[<ffffffffa08fc5cf>] i40e_probe+0xd8a/0x17e8 [i40e_zc]
[<ffffffff8154a92e>] ? schedule+0x3ee/0xb70
[<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
[<ffffffff812a3ece>] ? number+0x2ee/0x320
[<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
[<ffffffff811b762e>] ? find_inode+0x4e/0x90
[<ffffffff81217fc0>] ? sysfs_ilookup_test+0x0/0x20
[<ffffffff811b71d0>] ? iput+0x30/0x70
[<ffffffff81218bbe>] ? sysfs_addrm_finish+0x4e/0x270
[<ffffffff8121947e>] ? __sysfs_add_one+0x7e/0xc0
[<ffffffff812194ec>] ? sysfs_add_one+0x2c/0xd0
[<ffffffff812c0aa7>] local_pci_probe+0x17/0x20
[<ffffffff812c1c91>] pci_device_probe+0x101/0x120
[<ffffffff81383bf2>] ? driver_sysfs_add+0x62/0x90
[<ffffffff81383e9a>] driver_probe_device+0xaa/0x3a0
[<ffffffff8138423b>] __driver_attach+0xab/0xb0
[<ffffffff81384190>] ? __driver_attach+0x0/0xb0
[<ffffffff81383084>] bus_for_each_dev+0x64/0x90
[<ffffffff81383b2e>] driver_attach+0x1e/0x20
[<ffffffff81382898>] bus_add_driver+0x1e8/0x2b0
[<ffffffff8138444f>] driver_register+0x5f/0xe0
[<ffffffffa0504000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
[<ffffffff812c1ef6>] __pci_register_driver+0x56/0xd0
[<ffffffff8122400b>] ? debugfs_create_dir+0x1b/0x20
udev: renamed network interface eth10 to eth16
[<ffffffffa0504000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
[<ffffffffa05040a1>] i40e_init_module+0xa1/0xa3 [i40e_zc]
[<ffffffff810020d0>] do_one_initcall+0xc0/0x280
[<ffffffff810c84f1>] sys_init_module+0xe1/0x250
[<ffffffff8100b0d2>] system_call_fastpath+0x16/0x1b
Code: 00 49 89 9d a8 00 00 00 48 8b 83 18 01 00 00 49 89 45 00 66 83 83 30 01 00 00 01 4d 85 f6 4c 89 ab 18 01 00 00 0f 84 05 01 00 00 <49> 89 9e a8 00 00 00 48 8b 83 f0 00 00 00 41 83 c7 01 49 89 06
RIP [<ffffffffa08c74a3>] i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
RSP <ffff880861a53908>
---[ end trace c4414a8eb6ab10b9 ]---
Kernel panic - not syncing: Fatal exception
Pid: 10278, comm: modprobe Tainted: G D -- ------------ 2.6.32-696.13.2.el6.x86_64 #1
Call Trace:
[<ffffffff8154a20e>] ? panic+0xa7/0x179
[<ffffffff8154f0a4>] ? oops_end+0xe4/0x100
[<ffffffff8101101b>] ? die+0x5b/0x90
[<ffffffff8154eb62>] ? do_general_protection+0x152/0x160
[<ffffffff8154e2d5>] ? general_protection+0x25/0x30
[<ffffffffa08c74a3>] ? i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
[<ffffffffa08cd273>] ? i40e_vsi_setup+0x543/0x880 [i40e_zc]
[<ffffffffa08e3c9d>] ? i40e_aq_set_switch_config+0x9d/0xd0 [i40e_zc]
[<ffffffffa08d2d3f>] ? i40e_setup_pf_switch+0x47f/0x5d0 [i40e_zc]
[<ffffffffa08fc5cf>] ? i40e_probe+0xd8a/0x17e8 [i40e_zc]
[<ffffffff8154a92e>] ? schedule+0x3ee/0xb70
[<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
[<ffffffff812a3ece>] ? number+0x2ee/0x320
[<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
[<ffffffff811b762e>] ? find_inode+0x4e/0x90
[<ffffffff81217fc0>] ? sysfs_ilookup_test+0x0/0x20
[<ffffffff811b71d0>] ? iput+0x30/0x70
[<ffffffff81218bbe>] ? sysfs_addrm_finish+0x4e/0x270
[<ffffffff8121947e>] ? __sysfs_add_one+0x7e/0xc0
[<ffffffff812194ec>] ? sysfs_add_one+0x2c/0xd0
[<ffffffff812c0aa7>] ? local_pci_probe+0x17/0x20
[<ffffffff812c1c91>] ? pci_device_probe+0x101/0x120
[<ffffffff81383bf2>] ? driver_sysfs_add+0x62/0x90
[<ffffffff81383e9a>] ? driver_probe_device+0xaa/0x3a0
[<ffffffff8138423b>] ? __driver_attach+0xab/0xb0
[<ffffffff81384190>] ? __driver_attach+0x0/0xb0
[<ffffffff81383084>] ? bus_for_each_dev+0x64/0x90
[<ffffffff81383b2e>] ? driver_attach+0x1e/0x20
[<ffffffff81382898>] ? bus_add_driver+0x1e8/0x2b0
[<ffffffff8138444f>] ? driver_register+0x5f/0xe0
[<ffffffffa0504000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
[<ffffffff812c1ef6>] ? __pci_register_driver+0x56/0xd0
[<ffffffff8122400b>] ? debugfs_create_dir+0x1b/0x20
[<ffffffffa0504000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
[<ffffffffa05040a1>] ? i40e_init_module+0xa1/0xa3 [i40e_zc]
[<ffffffff810020d0>] ? do_one_initcall+0xc0/0x280
[<ffffffff810c84f1>] ? sys_init_module+0xe1/0x250
[<ffffffff8100b0d2>] ? system_call_fastpath+0x16/0x1b










And the second dump.






BUG: unable to handle kernel NULL pointer dereference at 00000000000000a8
IP: [<ffffffffa0bac4a3>] i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
PGD 10716fc067 PUD 1063b61067 PMD 0
Oops: 0002 [#1] SMP
last sysfs file: /sys/devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth10/type
CPU 21
Modules linked in: i40e_zc(+)(U) ixgbe_zc(U) pf_ring(U) vxlan netconsole configfs autofs4 bnx2fc fcoe libfcoe libfc 8021q scsi_transport_fc garp stp scsi_tg
t llc ipmi_devintf power_meter acpi_ipmi ipmi_si ipmi_msghandler iTCO_wdt iTCO_vendor_support dcdbas serio_raw lpc_ich mfd_core sg i7core_edac edac_core ext
4 jbd2 mbcache sr_mod cdrom sd_mod crc_t10dif pata_acpi ata_generic ata_piix igb dca i2c_algo_bit i2c_core bnx2x ptp pps_core libcrc32c crc32c_intel be2iscs
i bnx2i cnic uio cxgb4i iw_cxgb4 iw_cm cxgb4 cxgb3i libcxgbi iw_cxgb3 ib_core ib_addr ipv6 cxgb3 mdio libiscsi_tcp mpt2sas qla4xxx scsi_transport_sas raid_c
lass iscsi_boot_sysfs libiscsi bnx2 scsi_transport_iscsi wmi dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ixgbe]

Pid: 6259, comm: modprobe Not tainted 2.6.32-696.13.2.el6.x86_64 #1 Dell Inc. PowerEdge R910/0JRJM9
RIP: 0010:[<ffffffffa0bac4a3>] [<ffffffffa0bac4a3>] i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
RSP: 0018:ffff88085ad9f908 EFLAGS: 00010292
RAX: 0000000000000034 RBX: ffff88106e591000 RCX: 0000000000004df1
RDX: 0000000000000000 RSI: 0000000000000046 RDI: 0000000000000246
RBP: ffff88085ad9f968 R08: 000000000002a73e R09: 0000000000000000
R10: 0000000000000080 R11: 0000000000000001 R12: ffff88106e499000
R13: ffff8810708e7b00 R14: 0000000000000000 R15: 0000000000000010
udev: renamed network interface eth10 to eth16
FS: 00007f8f3620d700(0000) GS:ffff88089c540000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00000000000000a8 CR3: 0000001063cbe000 CR4: 00000000000007e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process modprobe (pid: 6259, threadinfo ffff88085ad9c000, task ffff88085cd16040)
Stack:
ffff88106e499000 0000000000000080 ffff881000000000 000000205728d020
<d> 0000001000000010 0000000100000010 ffff88085ad9f968 ffff88106e499000
<d> ffff88106e632000 ffff88106e632000 ffff88106e499028 ffff88085ad9f998
Call Trace:
[<ffffffffa0bb2273>] i40e_vsi_setup+0x543/0x880 [i40e_zc]
[<ffffffffa0bc8c9d>] ? i40e_aq_set_switch_config+0x9d/0xd0 [i40e_zc]
[<ffffffffa0bb7d3f>] i40e_setup_pf_switch+0x47f/0x5d0 [i40e_zc]
[<ffffffffa0be15cf>] i40e_probe+0xd8a/0x17e8 [i40e_zc]
[<ffffffff8113e0b9>] ? __alloc_pages_nodemask+0x129/0x950
[<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
[<ffffffff812a3ece>] ? number+0x2ee/0x320
[<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
[<ffffffff811b762e>] ? find_inode+0x4e/0x90
[<ffffffff81217fc0>] ? sysfs_ilookup_test+0x0/0x20
[<ffffffff811b71d0>] ? iput+0x30/0x70
[<ffffffff81218bbe>] ? sysfs_addrm_finish+0x4e/0x270
[<ffffffff8121947e>] ? __sysfs_add_one+0x7e/0xc0
[<ffffffff812194ec>] ? sysfs_add_one+0x2c/0xd0
[<ffffffff812c0aa7>] local_pci_probe+0x17/0x20
[<ffffffff812c1c91>] pci_device_probe+0x101/0x120
[<ffffffff81383bf2>] ? driver_sysfs_add+0x62/0x90
[<ffffffff81383e9a>] driver_probe_device+0xaa/0x3a0
[<ffffffff8138423b>] __driver_attach+0xab/0xb0
[<ffffffff81384190>] ? __driver_attach+0x0/0xb0
[<ffffffff81383084>] bus_for_each_dev+0x64/0x90
[<ffffffff81383b2e>] driver_attach+0x1e/0x20
[<ffffffff81382898>] bus_add_driver+0x1e8/0x2b0
[<ffffffff8138444f>] driver_register+0x5f/0xe0
[<ffffffffa0c01000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
[<ffffffff812c1ef6>] __pci_register_driver+0x56/0xd0
[<ffffffff8122400b>] ? debugfs_create_dir+0x1b/0x20
[<ffffffffa0c01000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
[<ffffffffa0c010a1>] i40e_init_module+0xa1/0xa3 [i40e_zc]
[<ffffffff810020d0>] do_one_initcall+0xc0/0x280
[<ffffffff810c84f1>] sys_init_module+0xe1/0x250
[<ffffffff8100b0d2>] system_call_fastpath+0x16/0x1b
Code: 00 49 89 9d a8 00 00 00 48 8b 83 18 01 00 00 49 89 45 00 66 83 83 30 01 00 00 01 4d 85 f6 4c 89 ab 18 01 00 00 0f 84 05 01 00 00 <49> 89 9e a8 00 00 00 48 8b 83 f0 00 00 00 41 83 c7 01 49 89 06
RIP [<ffffffffa0bac4a3>] i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
RSP <ffff88085ad9f908>
CR2: 00000000000000a8
---[ end trace 70024f26b6d6bbcb ]---
Kernel panic - not syncing: Fatal exception
Pid: 6259, comm: modprobe Tainted: G D -- ------------ 2.6.32-696.13.2.el6.x86_64 #1
Call Trace:
[<ffffffff8154a20e>] ? panic+0xa7/0x179
[<ffffffff8154f0a4>] ? oops_end+0xe4/0x100
[<ffffffff8105186b>] ? no_context+0xfb/0x260
[<ffffffff8105fa0c>] ? check_preempt_curr+0x7c/0x90
[<ffffffff81051af5>] ? __bad_area_nosemaphore+0x125/0x1e0
[<ffffffff81051c1e>] ? bad_area+0x4e/0x60
[<ffffffff81052443>] ? __do_page_fault+0x473/0x500
[<ffffffff8155100e>] ? do_page_fault+0x3e/0xa0
[<ffffffff8154e305>] ? page_fault+0x25/0x30
[<ffffffffa0bac4a3>] ? i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
[<ffffffffa0bac5e5>] ? i40e_vsi_map_rings_to_vectors+0x235/0x280 [i40e_zc]
[<ffffffffa0bb2273>] ? i40e_vsi_setup+0x543/0x880 [i40e_zc]
[<ffffffffa0bc8c9d>] ? i40e_aq_set_switch_config+0x9d/0xd0 [i40e_zc]
[<ffffffffa0bb7d3f>] ? i40e_setup_pf_switch+0x47f/0x5d0 [i40e_zc]
[<ffffffffa0be15cf>] ? i40e_probe+0xd8a/0x17e8 [i40e_zc]
[<ffffffff8113e0b9>] ? __alloc_pages_nodemask+0x129/0x950
[<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
[<ffffffff812a3ece>] ? number+0x2ee/0x320
[<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
[<ffffffff811b762e>] ? find_inode+0x4e/0x90
[<ffffffff81217fc0>] ? sysfs_ilookup_test+0x0/0x20
[<ffffffff811b71d0>] ? iput+0x30/0x70
[<ffffffff81218bbe>] ? sysfs_addrm_finish+0x4e/0x270
[<ffffffff8121947e>] ? __sysfs_add_one+0x7e/0xc0
[<ffffffff812194ec>] ? sysfs_add_one+0x2c/0xd0
[<ffffffff812c0aa7>] ? local_pci_probe+0x17/0x20
[<ffffffff812c1c91>] ? pci_device_probe+0x101/0x120
[<ffffffff81383bf2>] ? driver_sysfs_add+0x62/0x90
[<ffffffff81383e9a>] ? driver_probe_device+0xaa/0x3a0
[<ffffffff8138423b>] ? __driver_attach+0xab/0xb0
[<ffffffff81384190>] ? __driver_attach+0x0/0xb0
[<ffffffff81383084>] ? bus_for_each_dev+0x64/0x90
[<ffffffff81383b2e>] ? driver_attach+0x1e/0x20
[<ffffffff81382898>] ? bus_add_driver+0x1e8/0x2b0
[<ffffffff8138444f>] ? driver_register+0x5f/0xe0
[<ffffffffa0c01000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
[<ffffffff812c1ef6>] ? __pci_register_driver+0x56/0xd0
[<ffffffff8122400b>] ? debugfs_create_dir+0x1b/0x20
[<ffffffffa0c01000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
[<ffffffffa0c010a1>] ? i40e_init_module+0xa1/0xa3 [i40e_zc]
[<ffffffff810020d0>] ? do_one_initcall+0xc0/0x280
[<ffffffff810c84f1>] ? sys_init_module+0xe1/0x250
[<ffffffff8100b0d2>] ? system_call_fastpath+0x16/0x1b


_______________________________________________
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it<mailto:Ntop-misc@listgateway.unipi.it>
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

We are seeing the same issue on Ubuntu 14.04 with a 4.4.0-98-generic
kernel. We were getting kernel panics using i40e-zc-dkms 2.2.4.1507
from the development repo, so we tried moving to the stable repo and
purging / reinstalling the related packages (cento, pfring,
i40e-zc-dkms, pfring-dkms) but the issue persists. That brought us to
2.2.4.1515 for i40e-zc-dkms. We tried several previous kernel versions
to try to factor that out as an issue, and have had similar
problems with all that we've tried -- we've gone back as far as the
4.4.0-93 kernel.

This brings to mind a couple of questions:

Is there a place that we can find previous versions of these packages
so that we can attempt to go back to what was working for us (we were
on i40e-zc-dkms 1.5.18.1397 before)? Unfortunately, we don't have them
in our local package cache.

Also, it would be great to know what versions of Ubuntu and what
kernels (GA vs HWE) you're working against so that we can try to match
what you're doing as closely as possible, and hopefully that will save
us from running into upgrade issues in the future. We've been running
the HWE kernel because the default i40e driver in the GA kernel didn't
appear to work.

Thanks,
Peter

Here is a snippet of our dmesg output:

[ 636.363874] [PF_RING] Welcome to PF_RING 7.0.0 ($Revision:
7.0.0-stable:96e916d6f2e3017cff1c8f0d44657b946030d6ce$)
[ 636.363874] (C) 2004-17 ntop.org
[ 636.363877] [PF_RING] Min # ring slots 4096
[ 636.363878] [PF_RING] Slot version 16
[ 636.363879] [PF_RING] Capture TX Yes [RX+TX]
[ 636.363880] [PF_RING] IP Defragment No
[ 636.363890] [PF_RING] registered /proc/net/pf_ring/
[ 636.363891] NET: Registered protocol family 27
[ 636.363909] [PF_RING] Initialized correctly
[ 636.387834] i40e 0000:05:00.3: i40e_ptp_stop: removed PHC on p4p4
[ 636.431809] i40e 0000:05:00.3 p4p4: removing xx:xx:xx:xx:xx:xx vid=0
[ 636.472963] i40e 0000:05:00.2: i40e_ptp_stop: removed PHC on p4p3
[ 636.520282] i40e 0000:05:00.2 p4p3: removing xx:xx:xx:xx:xx:xx vid=0
[ 636.552952] i40e 0000:05:00.1: i40e_ptp_stop: removed PHC on p4p2
[ 636.601840] i40e 0000:05:00.1 p4p2: removing xx:xx:xx:xx:xx:xx vid=0
[ 636.652951] i40e 0000:05:00.0: i40e_ptp_stop: removed PHC on p4p1
[ 636.702126] i40e 0000:05:00.0 p4p1: removing xx:xx:xx:xx:xx:xx vid=0
[ 636.855316] i40e: Intel(R) 40-10 Gigabit Ethernet Connection
Network Driver - version 2.2.4
[ 636.855319] i40e: Copyright(c) 2013 - 2017 Intel Corporation.
[ 636.856318] i40e 05:00.0 Core and Global modules ready 0
[ 636.866341] i40e 05:00.0 i40e_set_mac_type found mac: 1, returns: 0
[ 636.867774] i40e 0000:05:00.0: fw 5.0.40043 api 1.5 nvm 5.05
0x80002919 1.1313.0
[ 637.097150] i40e 0000:05:00.0: MAC address: xx:xx:xx:xx:xx:xx
[ 637.097398] i40e 0000:05:00.0: qs_avail=384 FD SB=1 lan_qs=1
lan_tc0=1 vf=32*4 vmdq=8*1, remaining=246
[ 637.099085] i40e 0000:05:00.0 eth0: 1 RX/TX rings requested
[ 637.099088] i40e 0000:05:00.0 eth0: 1 RX/TX rings allocated successfully
[ 637.099098] BUG: unable to handle kernel NULL pointer dereference
at 0000000000000967
[ 637.099118] IP: [<ffffffffc0728aec>]
i40e_vsi_map_rings_to_vectors+0xbc/0x230 [i40e_zc]
[ 637.099144] PGD 0
[ 637.099155] Oops: 0002 [#1] SMP
[ 637.099167] Modules linked in: i40e_zc(OE+) vxlan ip6_udp_tunnel
udp_tunnel pf_ring(OE) configfs uio ipmi_devintf deflate ctr
twofish_generic twofish_avx_x86_64 twofish_x86_64_3way twofish_x86_64
twofish_common camellia_generic camellia_aesni_avx_x86_64
camellia_x86_64 serpent_avx_x86_64 serpent_sse2_x86_64 xts
serpent_generic blowfish_generic blowfish_x86_64 blowfish_common
cast5_avx_x86_64 cast5_generic cast_common des_generic cmac xcbc
rmd160 af_key xfrm_algo nfsd ipt_REJECT nf_reject_ipv4 auth_rpcgss
nf_conntrack_ipv4 nfs_acl nf_defrag_ipv4 nf_log_ipv4 nfs lockd
iptable_filter grace ip_tables sunrpc ip6t_REJECT nf_reject_ipv6
fscache nf_conntrack_ipv6 nf_defrag_ipv6 xt_tcpudp nf_log_ipv6
nf_log_common xt_LOG xt_limit ip6t_rt xt_conntrack nf_conntrack
ip6table_filter ip6_tables x_tables ipmi_ssif 8021q garp mrp stp llc
bonding x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm
irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel
aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd
serio_raw sb_edac joydev input_leds edac_core lpc_ich mei_me mei
ipmi_si 8250_fintek ipmi_msghandler ioatdma shpchp wmi dca mac_hid lp
parport hid_generic usbhid hid psmouse e1000e isci ahci libahci libsas
ptp scsi_transport_sas pps_core fjes [last unloaded: ip6_udp_tunnel]
[ 637.099614] CPU: 0 PID: 272 Comm: kworker/0:2 Tainted: G
OE 4.4.0-97-generic #120~14.04.1-Ubuntu
[ 637.099662] Hardware name: Supermicro
X9DBL-3F/X9DBL-iF/X9DBL-3F/X9DBL-iF, BIOS 3.00 12/06/2013
[ 637.099711] Workqueue: events work_for_cpu_fn
[ 637.099737] task: ffff880c4833f000 ti: ffff880c45600000 task.ti:
ffff880c45600000
[ 637.099780] RIP: 0010:[<ffffffffc0728aec>] [<ffffffffc0728aec>]
i40e_vsi_map_rings_to_vectors+0xbc/0x230 [i40e_zc]
[ 637.099833] RSP: 0018:ffff880c456039f0 EFLAGS: 00010202
[ 637.099859] RAX: 0000000000000000 RBX: ffff880c43f3e400 RCX: ffff880b1f7b1000
[ 637.099888] RDX: ffff880c47560538 RSI: 0000000000000000 RDI: 0000000000000000
[ 637.099916] RBP: ffff880c45603a50 R08: 0000000000000002 R09: 0000000000000010
[ 637.099945] R10: 0000000000000001 R11: 0000000000000002 R12: 00000000000008bf
[ 637.099974] R13: ffff880c47560550 R14: 0000000000000000 R15: 0000000000000010
[ 637.100003] FS: 0000000000000000(0000) GS:ffff880c4fa00000(0000)
knlGS:0000000000000000
[ 637.100048] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 637.100074] CR2: 0000000000000967 CR3: 0000000001e0c000 CR4: 00000000001406f0
[ 637.100103] Stack:
[ 637.100123] ffff880c456039f8 0000000000000002 0000000100000002
0000001600000000
[ 637.100173] ffff880c47560538 0000000000000000 000000184f4032c0
ffff880c4713e000
[ 637.100223] ffff880c4713e000 ffff880b1f7b1000 ffff880c4713e000
ffff880b1f7b1220
[ 637.100273] Call Trace:
[ 637.100302] [<ffffffffc0735a68>] i40e_vsi_setup+0x4f8/0x890 [i40e_zc]
[ 637.100337] [<ffffffffc0744d04>] ?
i40e_aq_set_switch_config+0x84/0xb0 [i40e_zc]
[ 637.100386] [<ffffffffc07365e1>] i40e_setup_pf_switch+0x481/0x580 [i40e_zc]
[ 637.100420] [<ffffffffc0739b9d>] i40e_probe.part.86+0xd8d/0x1930 [i40e_zc]
[ 637.100450] [<ffffffff810d59a7>] ? irq_to_desc+0x17/0x20
[ 637.100479] [<ffffffff813e500d>] ? radix_tree_lookup+0xd/0x10
[ 637.100506] [<ffffffff810d59a7>] ? irq_to_desc+0x17/0x20
[ 637.100533] [<ffffffff810d95fe>] ? irq_get_irq_data+0xe/0x20
[ 637.100562] [<ffffffff81055710>] ? mp_map_pin_to_irq+0xb0/0x310
[ 637.100590] [<ffffffff81055f71>] ? mp_map_gsi_to_irq+0xa1/0xc0
[ 637.100618] [<ffffffff8104d490>] ? acpi_register_gsi_ioapic+0xb0/0x150
[ 637.100648] [<ffffffff816e435a>] ? pci_conf1_read+0xba/0x100
[ 637.100676] [<ffffffff816e7a33>] ? raw_pci_read+0x23/0x40
[ 637.100704] [<ffffffff81424f9e>] ? pci_bus_read_config_word+0x7e/0x90
[ 637.100734] [<ffffffff8142d112>] ? do_pci_enable_device+0xc2/0xf0
[ 637.100765] [<ffffffffc073a75e>] i40e_probe+0x1e/0x30 [i40e_zc]
[ 637.100793] [<ffffffff8142fc35>] local_pci_probe+0x45/0xa0
[ 637.100820] [<ffffffff81093b14>] work_for_cpu_fn+0x14/0x20
[ 637.100848] [<ffffffff81096e10>] process_one_work+0x150/0x3f0
[ 637.100876] [<ffffffff810976ba>] worker_thread+0x24a/0x470
[ 637.100903] [<ffffffff81097470>] ? rescuer_thread+0x310/0x310
[ 637.100931] [<ffffffff8109ce16>] kthread+0xd6/0xf0
[ 637.100957] [<ffffffff8109cd40>] ? kthread_park+0x60/0x60
[ 637.100984] [<ffffffff8180f88f>] ret_from_fork+0x3f/0x70
[ 637.101010] [<ffffffff8109cd40>] ? kthread_park+0x60/0x60
[ 637.101036] Code: ff 85 c0 89 45 b4 88 81 28 01 00 00 89 7d b8 0f
84 ef 00 00 00 41 89 fa 4c 89 f9 45 31 f6 49 83 c2 01 4c 89 5d a8 4d
89 cf eb 2f <49> 89 9c 24 a8 00 00 00 48 8b 83 d8 00 00 00 49 83 c6 01
49 89
[ 637.101237] RIP [<ffffffffc0728aec>]
i40e_vsi_map_rings_to_vectors+0xbc/0x230 [i40e_zc]
[ 637.101286] RSP <ffff880c456039f0>
[ 637.101309] CR2: 0000000000000967
[ 637.101585] ---[ end trace 4c352fd6f332323c ]---


On Mon, Oct 30, 2017 at 9:34 AM,
ntop-misc-request@listgateway.unipi.it
<ntop-misc-request@listgateway.unipi.it> wrote:
> Send Ntop-misc mailing list submissions to
> ntop-misc@listgateway.unipi.it
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
> or, via email, send a message with subject or body 'help' to
> ntop-misc-request@listgateway.unipi.it
>
> You can reach the person managing the list at
> ntop-misc-owner@listgateway.unipi.it
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Ntop-misc digest..."
>
>
> Today's Topics:
>
> 1. Re: i40e 2.2.4 pfring 7.0.0 (Jeremy Erb)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 30 Oct 2017 16:34:07 +0000
> From: Jeremy Erb <jeremy.erb@netsweeper.com>
> To: "ntop-misc@listgateway.unipi.it" <ntop-misc@listgateway.unipi.it>
> Subject: Re: [Ntop-misc] i40e 2.2.4 pfring 7.0.0
> Message-ID:
> <YTOPR0101MB1322677557BB46844FA09D83FE590@YTOPR0101MB1322.CANPRD01.PROD.OUTLOOK.COM>
>
> Content-Type: text/plain; charset="us-ascii"
>
> Issue 227 opened, will post the output to the ticket.
>
> Regards,
>
> Jeremy
>
>
>
>
> ________________________________
> From: ntop-misc-bounces@listgateway.unipi.it <ntop-misc-bounces@listgateway.unipi.it> on behalf of Alfredo Cardigliano <cardigliano@ntop.org>
> Sent: Monday, October 30, 2017 12:23:06 PM
> To: ntop-misc@listgateway.unipi.it
> Subject: Re: [Ntop-misc] i40e 2.2.4 pfring 7.0.0
>
> Hi Jeremy
> I added more verbosity, please use https://github.com/ntop/PF_RING/tree/dev/drivers/intel/i40e/i40e-2.2.4-zc/src
> loading it with load_driver.sh (this to avoid reinstalling new packages on every test).
> Please provide the full dmesg output after a crash.
>
> Thank you
> Alfredo
>
> On 30 Oct 2017, at 17:00, Alfredo Cardigliano <cardigliano@ntop.org<mailto:cardigliano@ntop.org>> wrote:
>
> Hi Jeremy
> please open an issue on https://github.com/ntop/PF_RING/issues to keep track on this,
> please also post a full dmesg output if possible.
>
> Thank you
> Alfredo
>
> On 30 Oct 2017, at 16:55, Jeremy Erb <jeremy.erb@netsweeper.com<mailto:jeremy.erb@netsweeper.com>> wrote:
>
>
> Upgrading to PF_RING 7.0.0 with the new i40e 2.2.4 causes a kernel panic for us on CentOS 6 kernel. The vanilla i40e 2.2.4 driver does not cause this issue. Here are two kernel dumps. The i40e 1.5.18 driver works in libpfring 6.7.0.
>
>
>
> Pid: 10278, comm: modprobe Not tainted 2.6.32-696.13.2.el6.x86_64 #1 Dell Inc. PowerEdge R910/0JRJM9
> RIP: 0010:[<ffffffffa08c74a3>] [<ffffffffa08c74a3>] i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
> RSP: 0018:ffff880861a53908 EFLAGS: 00010202
> RAX: 0000000000000000 RBX: ffff88106dedcc00 RCX: ffff88106dedcc00
> RDX: ffff88106de0ed40 RSI: 0000000000000010 RDI: ffff88104e0b2000
> RBP: ffff880861a53968 R08: 000000000003ae77 R09: 0000000000000000
> R10: 0000000000000080 R11: 0000000000000001 R12: ffff88104e0b2000
> R13: ffff88106e665f00 R14: 00010102464c457f R15: 0000000000000010
> FS: 00007fd3655b4700(0000) GS:ffff88089c440000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: 00007fee9e121000 CR3: 00000010425ed000 CR4: 00000000000007e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Process modprobe (pid: 10278, threadinfo ffff880861a50000, task ffff8805f8c8f520)
> Stack:
> ffff88104e0b2000 ffff88104e0b2000 ffff881063754000 00000020202ca020
> <d> 0000001000000010 0000000100000010 ffff880861a53968 ffff88104e0b2000
> <d> ffff881063754000 ffff881063754000 ffff88104e0b2028 ffff880861a53998
> Call Trace:
> [<ffffffffa08cd273>] i40e_vsi_setup+0x543/0x880 [i40e_zc]
> [<ffffffffa08e3c9d>] ? i40e_aq_set_switch_config+0x9d/0xd0 [i40e_zc]
> [<ffffffffa08d2d3f>] i40e_setup_pf_switch+0x47f/0x5d0 [i40e_zc]
> [<ffffffffa08fc5cf>] i40e_probe+0xd8a/0x17e8 [i40e_zc]
> [<ffffffff8154a92e>] ? schedule+0x3ee/0xb70
> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
> [<ffffffff812a3ece>] ? number+0x2ee/0x320
> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
> [<ffffffff811b762e>] ? find_inode+0x4e/0x90
> [<ffffffff81217fc0>] ? sysfs_ilookup_test+0x0/0x20
> [<ffffffff811b71d0>] ? iput+0x30/0x70
> [<ffffffff81218bbe>] ? sysfs_addrm_finish+0x4e/0x270
> [<ffffffff8121947e>] ? __sysfs_add_one+0x7e/0xc0
> [<ffffffff812194ec>] ? sysfs_add_one+0x2c/0xd0
> [<ffffffff812c0aa7>] local_pci_probe+0x17/0x20
> [<ffffffff812c1c91>] pci_device_probe+0x101/0x120
> [<ffffffff81383bf2>] ? driver_sysfs_add+0x62/0x90
> [<ffffffff81383e9a>] driver_probe_device+0xaa/0x3a0
> [<ffffffff8138423b>] __driver_attach+0xab/0xb0
> [<ffffffff81384190>] ? __driver_attach+0x0/0xb0
> [<ffffffff81383084>] bus_for_each_dev+0x64/0x90
> [<ffffffff81383b2e>] driver_attach+0x1e/0x20
> [<ffffffff81382898>] bus_add_driver+0x1e8/0x2b0
> [<ffffffff8138444f>] driver_register+0x5f/0xe0
> [<ffffffffa0504000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
> [<ffffffff812c1ef6>] __pci_register_driver+0x56/0xd0
> [<ffffffff8122400b>] ? debugfs_create_dir+0x1b/0x20
> udev: renamed network interface eth10 to eth16
> [<ffffffffa0504000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
> [<ffffffffa05040a1>] i40e_init_module+0xa1/0xa3 [i40e_zc]
> [<ffffffff810020d0>] do_one_initcall+0xc0/0x280
> [<ffffffff810c84f1>] sys_init_module+0xe1/0x250
> [<ffffffff8100b0d2>] system_call_fastpath+0x16/0x1b
> Code: 00 49 89 9d a8 00 00 00 48 8b 83 18 01 00 00 49 89 45 00 66 83 83 30 01 00 00 01 4d 85 f6 4c 89 ab 18 01 00 00 0f 84 05 01 00 00 <49> 89 9e a8 00 00 00 48 8b 83 f0 00 00 00 41 83 c7 01 49 89 06
> RIP [<ffffffffa08c74a3>] i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
> RSP <ffff880861a53908>
> ---[ end trace c4414a8eb6ab10b9 ]---
> Kernel panic - not syncing: Fatal exception
> Pid: 10278, comm: modprobe Tainted: G D -- ------------ 2.6.32-696.13.2.el6.x86_64 #1
> Call Trace:
> [<ffffffff8154a20e>] ? panic+0xa7/0x179
> [<ffffffff8154f0a4>] ? oops_end+0xe4/0x100
> [<ffffffff8101101b>] ? die+0x5b/0x90
> [<ffffffff8154eb62>] ? do_general_protection+0x152/0x160
> [<ffffffff8154e2d5>] ? general_protection+0x25/0x30
> [<ffffffffa08c74a3>] ? i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
> [<ffffffffa08cd273>] ? i40e_vsi_setup+0x543/0x880 [i40e_zc]
> [<ffffffffa08e3c9d>] ? i40e_aq_set_switch_config+0x9d/0xd0 [i40e_zc]
> [<ffffffffa08d2d3f>] ? i40e_setup_pf_switch+0x47f/0x5d0 [i40e_zc]
> [<ffffffffa08fc5cf>] ? i40e_probe+0xd8a/0x17e8 [i40e_zc]
> [<ffffffff8154a92e>] ? schedule+0x3ee/0xb70
> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
> [<ffffffff812a3ece>] ? number+0x2ee/0x320
> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
> [<ffffffff811b762e>] ? find_inode+0x4e/0x90
> [<ffffffff81217fc0>] ? sysfs_ilookup_test+0x0/0x20
> [<ffffffff811b71d0>] ? iput+0x30/0x70
> [<ffffffff81218bbe>] ? sysfs_addrm_finish+0x4e/0x270
> [<ffffffff8121947e>] ? __sysfs_add_one+0x7e/0xc0
> [<ffffffff812194ec>] ? sysfs_add_one+0x2c/0xd0
> [<ffffffff812c0aa7>] ? local_pci_probe+0x17/0x20
> [<ffffffff812c1c91>] ? pci_device_probe+0x101/0x120
> [<ffffffff81383bf2>] ? driver_sysfs_add+0x62/0x90
> [<ffffffff81383e9a>] ? driver_probe_device+0xaa/0x3a0
> [<ffffffff8138423b>] ? __driver_attach+0xab/0xb0
> [<ffffffff81384190>] ? __driver_attach+0x0/0xb0
> [<ffffffff81383084>] ? bus_for_each_dev+0x64/0x90
> [<ffffffff81383b2e>] ? driver_attach+0x1e/0x20
> [<ffffffff81382898>] ? bus_add_driver+0x1e8/0x2b0
> [<ffffffff8138444f>] ? driver_register+0x5f/0xe0
> [<ffffffffa0504000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
> [<ffffffff812c1ef6>] ? __pci_register_driver+0x56/0xd0
> [<ffffffff8122400b>] ? debugfs_create_dir+0x1b/0x20
> [<ffffffffa0504000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
> [<ffffffffa05040a1>] ? i40e_init_module+0xa1/0xa3 [i40e_zc]
> [<ffffffff810020d0>] ? do_one_initcall+0xc0/0x280
> [<ffffffff810c84f1>] ? sys_init_module+0xe1/0x250
> [<ffffffff8100b0d2>] ? system_call_fastpath+0x16/0x1b
>
>
>
>
>
>
>
>
>
>
> And the second dump.
>
>
>
>
>
>
> BUG: unable to handle kernel NULL pointer dereference at 00000000000000a8
> IP: [<ffffffffa0bac4a3>] i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
> PGD 10716fc067 PUD 1063b61067 PMD 0
> Oops: 0002 [#1] SMP
> last sysfs file: /sys/devices/pci0000:00/0000:00:07.0/0000:04:00.0/net/eth10/type
> CPU 21
> Modules linked in: i40e_zc(+)(U) ixgbe_zc(U) pf_ring(U) vxlan netconsole configfs autofs4 bnx2fc fcoe libfcoe libfc 8021q scsi_transport_fc garp stp scsi_tg
> t llc ipmi_devintf power_meter acpi_ipmi ipmi_si ipmi_msghandler iTCO_wdt iTCO_vendor_support dcdbas serio_raw lpc_ich mfd_core sg i7core_edac edac_core ext
> 4 jbd2 mbcache sr_mod cdrom sd_mod crc_t10dif pata_acpi ata_generic ata_piix igb dca i2c_algo_bit i2c_core bnx2x ptp pps_core libcrc32c crc32c_intel be2iscs
> i bnx2i cnic uio cxgb4i iw_cxgb4 iw_cm cxgb4 cxgb3i libcxgbi iw_cxgb3 ib_core ib_addr ipv6 cxgb3 mdio libiscsi_tcp mpt2sas qla4xxx scsi_transport_sas raid_c
> lass iscsi_boot_sysfs libiscsi bnx2 scsi_transport_iscsi wmi dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ixgbe]
>
> Pid: 6259, comm: modprobe Not tainted 2.6.32-696.13.2.el6.x86_64 #1 Dell Inc. PowerEdge R910/0JRJM9
> RIP: 0010:[<ffffffffa0bac4a3>] [<ffffffffa0bac4a3>] i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
> RSP: 0018:ffff88085ad9f908 EFLAGS: 00010292
> RAX: 0000000000000034 RBX: ffff88106e591000 RCX: 0000000000004df1
> RDX: 0000000000000000 RSI: 0000000000000046 RDI: 0000000000000246
> RBP: ffff88085ad9f968 R08: 000000000002a73e R09: 0000000000000000
> R10: 0000000000000080 R11: 0000000000000001 R12: ffff88106e499000
> R13: ffff8810708e7b00 R14: 0000000000000000 R15: 0000000000000010
> udev: renamed network interface eth10 to eth16
> FS: 00007f8f3620d700(0000) GS:ffff88089c540000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: 00000000000000a8 CR3: 0000001063cbe000 CR4: 00000000000007e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Process modprobe (pid: 6259, threadinfo ffff88085ad9c000, task ffff88085cd16040)
> Stack:
> ffff88106e499000 0000000000000080 ffff881000000000 000000205728d020
> <d> 0000001000000010 0000000100000010 ffff88085ad9f968 ffff88106e499000
> <d> ffff88106e632000 ffff88106e632000 ffff88106e499028 ffff88085ad9f998
> Call Trace:
> [<ffffffffa0bb2273>] i40e_vsi_setup+0x543/0x880 [i40e_zc]
> [<ffffffffa0bc8c9d>] ? i40e_aq_set_switch_config+0x9d/0xd0 [i40e_zc]
> [<ffffffffa0bb7d3f>] i40e_setup_pf_switch+0x47f/0x5d0 [i40e_zc]
> [<ffffffffa0be15cf>] i40e_probe+0xd8a/0x17e8 [i40e_zc]
> [<ffffffff8113e0b9>] ? __alloc_pages_nodemask+0x129/0x950
> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
> [<ffffffff812a3ece>] ? number+0x2ee/0x320
> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
> [<ffffffff811b762e>] ? find_inode+0x4e/0x90
> [<ffffffff81217fc0>] ? sysfs_ilookup_test+0x0/0x20
> [<ffffffff811b71d0>] ? iput+0x30/0x70
> [<ffffffff81218bbe>] ? sysfs_addrm_finish+0x4e/0x270
> [<ffffffff8121947e>] ? __sysfs_add_one+0x7e/0xc0
> [<ffffffff812194ec>] ? sysfs_add_one+0x2c/0xd0
> [<ffffffff812c0aa7>] local_pci_probe+0x17/0x20
> [<ffffffff812c1c91>] pci_device_probe+0x101/0x120
> [<ffffffff81383bf2>] ? driver_sysfs_add+0x62/0x90
> [<ffffffff81383e9a>] driver_probe_device+0xaa/0x3a0
> [<ffffffff8138423b>] __driver_attach+0xab/0xb0
> [<ffffffff81384190>] ? __driver_attach+0x0/0xb0
> [<ffffffff81383084>] bus_for_each_dev+0x64/0x90
> [<ffffffff81383b2e>] driver_attach+0x1e/0x20
> [<ffffffff81382898>] bus_add_driver+0x1e8/0x2b0
> [<ffffffff8138444f>] driver_register+0x5f/0xe0
> [<ffffffffa0c01000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
> [<ffffffff812c1ef6>] __pci_register_driver+0x56/0xd0
> [<ffffffff8122400b>] ? debugfs_create_dir+0x1b/0x20
> [<ffffffffa0c01000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
> [<ffffffffa0c010a1>] i40e_init_module+0xa1/0xa3 [i40e_zc]
> [<ffffffff810020d0>] do_one_initcall+0xc0/0x280
> [<ffffffff810c84f1>] sys_init_module+0xe1/0x250
> [<ffffffff8100b0d2>] system_call_fastpath+0x16/0x1b
> Code: 00 49 89 9d a8 00 00 00 48 8b 83 18 01 00 00 49 89 45 00 66 83 83 30 01 00 00 01 4d 85 f6 4c 89 ab 18 01 00 00 0f 84 05 01 00 00 <49> 89 9e a8 00 00 00 48 8b 83 f0 00 00 00 41 83 c7 01 49 89 06
> RIP [<ffffffffa0bac4a3>] i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
> RSP <ffff88085ad9f908>
> CR2: 00000000000000a8
> ---[ end trace 70024f26b6d6bbcb ]---
> Kernel panic - not syncing: Fatal exception
> Pid: 6259, comm: modprobe Tainted: G D -- ------------ 2.6.32-696.13.2.el6.x86_64 #1
> Call Trace:
> [<ffffffff8154a20e>] ? panic+0xa7/0x179
> [<ffffffff8154f0a4>] ? oops_end+0xe4/0x100
> [<ffffffff8105186b>] ? no_context+0xfb/0x260
> [<ffffffff8105fa0c>] ? check_preempt_curr+0x7c/0x90
> [<ffffffff81051af5>] ? __bad_area_nosemaphore+0x125/0x1e0
> [<ffffffff81051c1e>] ? bad_area+0x4e/0x60
> [<ffffffff81052443>] ? __do_page_fault+0x473/0x500
> [<ffffffff8155100e>] ? do_page_fault+0x3e/0xa0
> [<ffffffff8154e305>] ? page_fault+0x25/0x30
> [<ffffffffa0bac4a3>] ? i40e_vsi_map_rings_to_vectors+0xf3/0x280 [i40e_zc]
> [<ffffffffa0bac5e5>] ? i40e_vsi_map_rings_to_vectors+0x235/0x280 [i40e_zc]
> [<ffffffffa0bb2273>] ? i40e_vsi_setup+0x543/0x880 [i40e_zc]
> [<ffffffffa0bc8c9d>] ? i40e_aq_set_switch_config+0x9d/0xd0 [i40e_zc]
> [<ffffffffa0bb7d3f>] ? i40e_setup_pf_switch+0x47f/0x5d0 [i40e_zc]
> [<ffffffffa0be15cf>] ? i40e_probe+0xd8a/0x17e8 [i40e_zc]
> [<ffffffff8113e0b9>] ? __alloc_pages_nodemask+0x129/0x950
> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
> [<ffffffff812a3ece>] ? number+0x2ee/0x320
> [<ffffffff8129a800>] ? idr_get_empty_slot+0x110/0x2c0
> [<ffffffff811b762e>] ? find_inode+0x4e/0x90
> [<ffffffff81217fc0>] ? sysfs_ilookup_test+0x0/0x20
> [<ffffffff811b71d0>] ? iput+0x30/0x70
> [<ffffffff81218bbe>] ? sysfs_addrm_finish+0x4e/0x270
> [<ffffffff8121947e>] ? __sysfs_add_one+0x7e/0xc0
> [<ffffffff812194ec>] ? sysfs_add_one+0x2c/0xd0
> [<ffffffff812c0aa7>] ? local_pci_probe+0x17/0x20
> [<ffffffff812c1c91>] ? pci_device_probe+0x101/0x120
> [<ffffffff81383bf2>] ? driver_sysfs_add+0x62/0x90
> [<ffffffff81383e9a>] ? driver_probe_device+0xaa/0x3a0
> [<ffffffff8138423b>] ? __driver_attach+0xab/0xb0
> [<ffffffff81384190>] ? __driver_attach+0x0/0xb0
> [<ffffffff81383084>] ? bus_for_each_dev+0x64/0x90
> [<ffffffff81383b2e>] ? driver_attach+0x1e/0x20
> [<ffffffff81382898>] ? bus_add_driver+0x1e8/0x2b0
> [<ffffffff8138444f>] ? driver_register+0x5f/0xe0
> [<ffffffffa0c01000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
> [<ffffffff812c1ef6>] ? __pci_register_driver+0x56/0xd0
> [<ffffffff8122400b>] ? debugfs_create_dir+0x1b/0x20
> [<ffffffffa0c01000>] ? i40e_init_module+0x0/0xa3 [i40e_zc]
> [<ffffffffa0c010a1>] ? i40e_init_module+0xa1/0xa3 [i40e_zc]
> [<ffffffff810020d0>] ? do_one_initcall+0xc0/0x280
> [<ffffffff810c84f1>] ? sys_init_module+0xe1/0x250
> [<ffffffff8100b0d2>] ? system_call_fastpath+0x16/0x1b
>
>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it<mailto:Ntop-misc@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://listgateway.unipi.it/mailman/private/ntop-misc/attachments/20171030/1f5eaf6d/attachment.htm>
>
> ------------------------------
>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
> End of Ntop-misc Digest, Vol 160, Issue 15
> ******************************************



--
Peter Giles | Senior Developer & Data Analyst | Office of the CISO
| University of Washington
_______________________________________________
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc