Hi all,
Recently, I met a problem of pfring. When the server is connected to
certain traffic, there was something wrong in kernel module and the server
crashed immediately. The traffic is 10Gb/s, which consists of many layers
of VLAN, even more than 10 layers; what’s more, some packet size are more
than 1500 bytes.
Here is the core dump message in /var/crash.
[ 129.671018] Kernel panic - not syncing: stack-protector: Kernel stack is
corrupted in:
ffffffffa077d802
[ 129.671066] CPU: 28 PID: 0 Comm: swapper/28 Tainted: G OE
------------ 3.
10.0-327.13.1.el7.x86_64 #1
[ 129.671099] Hardware name: Supermicro X9DRL-3F/iF/X9DRL-3F/iF, BIOS 3.2
09/22/2015
[ 129.671124] ffffffff81861040 7f670ac0a92dff05 ffff88103fc83868
ffffffff8163571c
[ 129.671157] ffff88103fc838e8 ffffffff8162ef97 0000000000000010
ffff88103fc838f8
[ 129.671189] ffff88103fc83898 7f670ac0a92dff05 0000000000000000
ffffffffa077d802
[ 129.671221] Call Trace:
[ 129.671232] <IRQ> [<ffffffff8163571c>] dump_stack+0x19/0x1b
[ 129.671261] [<ffffffff8162ef97>] panic+0xd8/0x1e7
[ 129.671287] [<ffffffffa077d802>] ? parse_pkt.isra.27+0x3c2/0x3d0
[pf_ring]
[ 129.671313] [<ffffffff8107b00b>] __stack_chk_fail+0x1b/0x20
[ 129.671334] [<ffffffffa077d802>] parse_pkt.isra.27+0x3c2/0x3d0 [pf_ring]
[ 129.671368] [<ffffffffa0781d33>] skb_ring_handler+0x7a3/0x1d30 [pf_ring]
[ 129.671393] [<ffffffff8130c59d>] ? list_del+0xd/0x30
[ 129.671412] [<ffffffff8117036a>] ? __rmqueue+0x8a/0x460
[ 129.671431] [<ffffffff81189849>] ? zone_statistics+0x89/0xa0
[ 129.671452] [<ffffffff81632005>] ? __slab_free+0x10e/0x277
[ 129.671473] [<ffffffff8156632f>] ? ip_rcv+0xcf/0x410
[ 129.671492] [<ffffffffa0783355>] packet_rcv+0x95/0xb0 [pf_ring]
[ 129.671514] [<ffffffff8152a64b>] __netif_receive_skb_core+0x11b/0x7d0
[ 129.671538] [<ffffffff8101c829>] ? read_tsc+0x9/0x10
[ 129.671556] [<ffffffff8152ad18>] __netif_receive_skb+0x18/0x60
[ 129.671577] [<ffffffff8152ada0>] netif_receive_skb+0x40/0xc0
[ 129.671606] [<ffffffff8152b910>] napi_gro_receive+0x80/0xb0
[ 129.671647] [<ffffffffa00dddff>] e1000_receive_skb+0x7f/0xe0 [e1000e]
[ 129.671682] [<ffffffffa00dfc4f>] e1000_clean_rx_irq+0x24f/0x410 [e1000e]
[ 129.671709] [<ffffffffa00e726c>] e1000e_poll+0xbc/0x2e0 [e1000e]
[ 129.671739] [<ffffffff8152b1d2>] net_rx_action+0x152/0x240
[ 129.671759] [<ffffffff81084b0f>] __do_softirq+0xef/0x280
[ 129.671781] [<ffffffff816477dc>] call_softirq+0x1c/0x30
[ 129.671802] [<ffffffff81016fc5>] do_softirq+0x65/0xa0
[ 129.671819] [<ffffffff81084ea5>] irq_exit+0x115/0x120
[ 129.671838] [<ffffffff81648378>] do_IRQ+0x58/0xf0
[ 129.671866] [<ffffffff8163d62d>] common_interrupt+0x6d/0x6d
[ 129.671884] <EOI> [<ffffffff814d46f2>] ? cpuidle_enter_state+0x52/0xc0
[ 129.671913] [<ffffffff814d4839>] cpuidle_idle_call+0xd9/0x210
[ 129.671934] [<ffffffff8101e4be>] arch_cpu_idle+0xe/0x30
[ 129.671954] [<ffffffff810d6325>] cpu_startup_entry+0x245/0x290
[ 129.671976] [<ffffffff810475fa>] start_secondary+0x1ba/0x230
OS information:
CentOS Linux release 7.2.1511 (Core)
3.10.0-327.13.1.el7.x86_64
PF_RING install rpm package:
dkms-2.3-5.20170523git8c3065c.el7.noarch.rpm
pfring-dkms-7.0.0-1541.noarch.rpm
pfring-7.0.0-1541.x86_64.rpm
I want to know whether it is caused by the large volume of traffic or
anything else?
Any insight would be helpful.
Recently, I met a problem of pfring. When the server is connected to
certain traffic, there was something wrong in kernel module and the server
crashed immediately. The traffic is 10Gb/s, which consists of many layers
of VLAN, even more than 10 layers; what’s more, some packet size are more
than 1500 bytes.
Here is the core dump message in /var/crash.
[ 129.671018] Kernel panic - not syncing: stack-protector: Kernel stack is
corrupted in:
ffffffffa077d802
[ 129.671066] CPU: 28 PID: 0 Comm: swapper/28 Tainted: G OE
------------ 3.
10.0-327.13.1.el7.x86_64 #1
[ 129.671099] Hardware name: Supermicro X9DRL-3F/iF/X9DRL-3F/iF, BIOS 3.2
09/22/2015
[ 129.671124] ffffffff81861040 7f670ac0a92dff05 ffff88103fc83868
ffffffff8163571c
[ 129.671157] ffff88103fc838e8 ffffffff8162ef97 0000000000000010
ffff88103fc838f8
[ 129.671189] ffff88103fc83898 7f670ac0a92dff05 0000000000000000
ffffffffa077d802
[ 129.671221] Call Trace:
[ 129.671232] <IRQ> [<ffffffff8163571c>] dump_stack+0x19/0x1b
[ 129.671261] [<ffffffff8162ef97>] panic+0xd8/0x1e7
[ 129.671287] [<ffffffffa077d802>] ? parse_pkt.isra.27+0x3c2/0x3d0
[pf_ring]
[ 129.671313] [<ffffffff8107b00b>] __stack_chk_fail+0x1b/0x20
[ 129.671334] [<ffffffffa077d802>] parse_pkt.isra.27+0x3c2/0x3d0 [pf_ring]
[ 129.671368] [<ffffffffa0781d33>] skb_ring_handler+0x7a3/0x1d30 [pf_ring]
[ 129.671393] [<ffffffff8130c59d>] ? list_del+0xd/0x30
[ 129.671412] [<ffffffff8117036a>] ? __rmqueue+0x8a/0x460
[ 129.671431] [<ffffffff81189849>] ? zone_statistics+0x89/0xa0
[ 129.671452] [<ffffffff81632005>] ? __slab_free+0x10e/0x277
[ 129.671473] [<ffffffff8156632f>] ? ip_rcv+0xcf/0x410
[ 129.671492] [<ffffffffa0783355>] packet_rcv+0x95/0xb0 [pf_ring]
[ 129.671514] [<ffffffff8152a64b>] __netif_receive_skb_core+0x11b/0x7d0
[ 129.671538] [<ffffffff8101c829>] ? read_tsc+0x9/0x10
[ 129.671556] [<ffffffff8152ad18>] __netif_receive_skb+0x18/0x60
[ 129.671577] [<ffffffff8152ada0>] netif_receive_skb+0x40/0xc0
[ 129.671606] [<ffffffff8152b910>] napi_gro_receive+0x80/0xb0
[ 129.671647] [<ffffffffa00dddff>] e1000_receive_skb+0x7f/0xe0 [e1000e]
[ 129.671682] [<ffffffffa00dfc4f>] e1000_clean_rx_irq+0x24f/0x410 [e1000e]
[ 129.671709] [<ffffffffa00e726c>] e1000e_poll+0xbc/0x2e0 [e1000e]
[ 129.671739] [<ffffffff8152b1d2>] net_rx_action+0x152/0x240
[ 129.671759] [<ffffffff81084b0f>] __do_softirq+0xef/0x280
[ 129.671781] [<ffffffff816477dc>] call_softirq+0x1c/0x30
[ 129.671802] [<ffffffff81016fc5>] do_softirq+0x65/0xa0
[ 129.671819] [<ffffffff81084ea5>] irq_exit+0x115/0x120
[ 129.671838] [<ffffffff81648378>] do_IRQ+0x58/0xf0
[ 129.671866] [<ffffffff8163d62d>] common_interrupt+0x6d/0x6d
[ 129.671884] <EOI> [<ffffffff814d46f2>] ? cpuidle_enter_state+0x52/0xc0
[ 129.671913] [<ffffffff814d4839>] cpuidle_idle_call+0xd9/0x210
[ 129.671934] [<ffffffff8101e4be>] arch_cpu_idle+0xe/0x30
[ 129.671954] [<ffffffff810d6325>] cpu_startup_entry+0x245/0x290
[ 129.671976] [<ffffffff810475fa>] start_secondary+0x1ba/0x230
OS information:
CentOS Linux release 7.2.1511 (Core)
3.10.0-327.13.1.el7.x86_64
PF_RING install rpm package:
dkms-2.3-5.20170523git8c3065c.el7.noarch.rpm
pfring-dkms-7.0.0-1541.noarch.rpm
pfring-7.0.0-1541.x86_64.rpm
I want to know whether it is caused by the large volume of traffic or
anything else?
Any insight would be helpful.