Mailing List Archive

Good morning.

Additional info:

Licensed products: n2disk10g, disk2n
Operating System: CentOS Linux release 7.2.1511 (Core)

We installed all here listed RPMs using sudo yum install :

cento-1.3.170308-182.x86_64.rpm
e1000e-zc-3.2.7.1.1152-dkms.noarch.rpm
fm10k-zc-0.20.1.1152-dkms.noarch.rpm
i40e-zc-1.5.18.1152-1dkms.noarch.rpm
igb-zc-5.3.3.5.1152-dkms.noarch.rpm
ixgbe-zc-4.1.5.1152-dkms.noarch.rpm
n2disk-2.7.170308-4774.x86_64.rpm
nbox-2.5.0-553.noarch.rpm
nprobe-7.5.170309-5677.x86_64.rpm
nprobe-7.5.170309-5678.x86_64.rpm
nscrub-0.99.170308-551.x86_64.rpm
ntopng-2.5.170309-2379.x86_64.rpm
ntopng-data-2.5.170308-2377.noarch.rpm
pfring-6.5.0-1152.x86_64.rpm
pfring-dkms-6.5.0-1152.noarch.rpm
pfring-drivers-zc-dkms-1.2-0.noarch.rpm

Hope that helps...

Best regards,
Marcel



Von: ntop-misc-bounces@listgateway.unipi.it [mailto:ntop-misc-bounces@listgateway.unipi.it]
Gesendet: Mittwoch, 17. Mai 2017 16:18
An: ntop-misc@listgateway.unipi.it
Betreff: [Ntop-misc] n2disk/disk2n: Problem with PF_RING ZC

Hello.

I want to maximize the performance for n2disk/disk2n and use PF_RING ZC.
PF_RING only is working as expected, but this doesn't:

[root@myhost]# pfcount -i zc:eno1
pfring_open error [No such device] (pf_ring not loaded or interface zc:eno1 is down ?)

[root@myhost]# pfsend -i zc:eno1
Sending packets on zc:eno1
pfring_open error [No such device] (pf_ring not loaded or interface zc:eno1 is down ?)

[root@myhost]# /usr/local/bin/n2disk10g -i zc:eno1 -o /tmp
17/May/2017 15:51:30 [n2disk.c:5013] Welcome to n2disk10g v.2.7.170308 (r4774) [Haswell]
17/May/2017 15:51:30 [n2disk.c:5040] Running on 2 node(s) system with 32 core(s). NUMA affinity set to node -1.
17/May/2017 15:51:30 [n2disk.c:5081] Using PF_RING for packet capture
17/May/2017 15:51:30 [n2disk.c:5110] Multithread support enabled
17/May/2017 15:51:30 [n2disk.c:5228] Dump files max size is set to 100 MB
17/May/2017 15:51:30 [n2disk.c:5245] Buffer memory is set to 200 MB (x 2 pcap files)
17/May/2017 15:51:30 [n2disk.c:5280] Using directory /tmp for dump files
17/May/2017 15:51:30 [n2disk.c:5285] No sub-directories will be created
17/May/2017 15:51:30 [n2disk.c:5290] Up to 1000 files will be written before overwriting
17/May/2017 15:51:30 [n2disk.c:5342] Dumping data in 0.1 MB chunks
17/May/2017 15:51:30 [n2disk.c:5582] Memory allocated successfully
17/May/2017 15:51:30 [n2disk.c:3525] Using packet timestamps from pf_ring
17/May/2017 15:51:30 [n2disk.c:3537] ERROR: Unable to open PF_RING device zc:eno1 [globals.ro.snaplen=1536]
17/May/2017 15:51:30 [n2disk.c:918] Caught termination signal -1...
17/May/2017 15:51:30 [n2disk.c:5589] ERROR: Unable to initialize packet reader
17/May/2017 15:51:30 [n2disk.c:5590] ERROR: Has the pf_ring kernel module been loaded ?
[root@myhost]#


The kernel modules seem to be loaded:

[root@myhost]# sudo lsmod | grep 'pf_ring\|ixgbe'
pf_ring 1229913 16
ixgbe 263239 0
mdio 13807 1 ixgbe
ptp 19231 2 igb,ixgbe
dca 15130 3 igb,ixgbe,ioatdma


The interface looks OK:

[root@myhost]# ethtool eno1
Settings for eno1:
Supported ports: [ FIBRE ]
Supported link modes: 10000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: No
Advertised link modes: 10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: No
Speed: 10000Mb/s
Duplex: Full
Port: Other
PHYAD: 0
Transceiver: external
Auto-negotiation: off
Supports Wake-on: umbg
Wake-on: g
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes

[root@myhost]# ethtool zc:eno1
Settings for zc:eno1:
Cannot get device settings: No such device
Cannot get wake-on-lan settings: No such device
Cannot get message level: No such device
Cannot get link status: No such device
No data available


Environment:
Server: HPE ProLiant DL380
RAM: 64 GB
Network:
[root@myhost]# lspci | grep -i 'ethernet'
03:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
03:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
0a:00.0 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)
0a:00.1 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)
0a:00.2 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)
0a:00.3 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)


What am I doing wrong?
Thanks for any help!

Regards,
Marcel

Hi Marcel
please provide “cat /proc/net/pf_ring/dev/eno1/info”

Alfredo

> On 17 May 2017, at 16:17, Lüthi Marcel FUB <Marcel.Luethi@vtg.admin.ch> wrote:
>
> Hello.
>
> I want to maximize the performance for n2disk/disk2n and use PF_RING ZC.
> PF_RING only is working as expected, but this doesn't:
>
> [root@myhost]# pfcount -i zc:eno1
> pfring_open error [No such device] (pf_ring not loaded or interface zc:eno1 is down ?)
>
> [root@myhost]# pfsend -i zc:eno1
> Sending packets on zc:eno1
> pfring_open error [No such device] (pf_ring not loaded or interface zc:eno1 is down ?)
>
> [root@myhost]# /usr/local/bin/n2disk10g -i zc:eno1 -o /tmp
> 17/May/2017 15:51:30 [n2disk.c:5013] Welcome to n2disk10g v.2.7.170308 (r4774) [Haswell]
> 17/May/2017 15:51:30 [n2disk.c:5040] Running on 2 node(s) system with 32 core(s). NUMA affinity set to node -1.
> 17/May/2017 15:51:30 [n2disk.c:5081] Using PF_RING for packet capture
> 17/May/2017 15:51:30 [n2disk.c:5110] Multithread support enabled
> 17/May/2017 15:51:30 [n2disk.c:5228] Dump files max size is set to 100 MB
> 17/May/2017 15:51:30 [n2disk.c:5245] Buffer memory is set to 200 MB (x 2 pcap files)
> 17/May/2017 15:51:30 [n2disk.c:5280] Using directory /tmp for dump files
> 17/May/2017 15:51:30 [n2disk.c:5285] No sub-directories will be created
> 17/May/2017 15:51:30 [n2disk.c:5290] Up to 1000 files will be written before overwriting
> 17/May/2017 15:51:30 [n2disk.c:5342] Dumping data in 0.1 MB chunks
> 17/May/2017 15:51:30 [n2disk.c:5582] Memory allocated successfully
> 17/May/2017 15:51:30 [n2disk.c:3525] Using packet timestamps from pf_ring
> 17/May/2017 15:51:30 [n2disk.c:3537] ERROR: Unable to open PF_RING device zc:eno1 [globals.ro.snaplen=1536]
> 17/May/2017 15:51:30 [n2disk.c:918] Caught termination signal -1...
> 17/May/2017 15:51:30 [n2disk.c:5589] ERROR: Unable to initialize packet reader
> 17/May/2017 15:51:30 [n2disk.c:5590] ERROR: Has the pf_ring kernel module been loaded ?
> [root@myhost]#
>
>
> The kernel modules seem to be loaded:
>
> [root@myhost]# sudo lsmod | grep 'pf_ring\|ixgbe'
> pf_ring 1229913 16
> ixgbe 263239 0
> mdio 13807 1 ixgbe
> ptp 19231 2 igb,ixgbe
> dca 15130 3 igb,ixgbe,ioatdma
>
>
> The interface looks OK:
>
> [root@myhost]# ethtool eno1
> Settings for eno1:
> Supported ports: [ FIBRE ]
> Supported link modes: 10000baseT/Full
> Supported pause frame use: No
> Supports auto-negotiation: No
> Advertised link modes: 10000baseT/Full
> Advertised pause frame use: No
> Advertised auto-negotiation: No
> Speed: 10000Mb/s
> Duplex: Full
> Port: Other
> PHYAD: 0
> Transceiver: external
> Auto-negotiation: off
> Supports Wake-on: umbg
> Wake-on: g
> Current message level: 0x00000007 (7)
> drv probe link
> Link detected: yes
>
> [root@myhost]# ethtool zc:eno1
> Settings for zc:eno1:
> Cannot get device settings: No such device
> Cannot get wake-on-lan settings: No such device
> Cannot get message level: No such device
> Cannot get link status: No such device
> No data available
>
>
> Environment:
> Server: HPE ProLiant DL380
> RAM: 64 GB
> Network:
> [root@myhost]# lspci | grep -i 'ethernet'
> 03:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
> 03:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
> 0a:00.0 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)
> 0a:00.1 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)
> 0a:00.2 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)
> 0a:00.3 Ethernet controller: Intel Corporation 82580 Gigabit Network Connection (rev 01)
>
>
> What am I doing wrong?
> Thanks for any help!
>
> Regards,
> Marcel
>
>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it <mailto:Ntop-misc@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc <http://listgateway.unipi.it/mailman/listinfo/ntop-misc>

Hi Alfredo

Here we go:

[root@myhost]# cat /proc/net/pf_ring/dev/eno1/info
Name: eno1
Index: 6
Address: 38:EA:A7:10:9F:F4
Polling Mode: NAPI
Type: Ethernet
Family: Standard NIC
# Bound Sockets: 1
TX Queues: 32
RX Queues: 32

[root@myhost]# cat /proc/net/pf_ring/dev/eno2/info
Name: eno2
Index: 7
Address: 38:EA:A7:10:9F:F5
Polling Mode: NAPI
Type: Ethernet
Family: Standard NIC
# Bound Sockets: 1
TX Queues: 32
RX Queues: 32


Regards,
Marcel



Von: ntop-misc-bounces@listgateway.unipi.it [mailto:ntop-misc-bounces@listgateway.unipi.it] Im Auftrag von Alfredo Cardigliano
Gesendet: Donnerstag, 18. Mai 2017 10:01
An: ntop-misc@listgateway.unipi.it
Betreff: Re: [Ntop-misc] n2disk/disk2n: Problem with PF_RING ZC

Hi Marcel
please provide “cat /proc/net/pf_ring/dev/eno1/info”

Alfredo

Hi Marcel
it seems you did not load the ZC driver (you should see "NAPI/ZC" in “Polling Mode”),
did you follow this guide https://github.com/ntop/PF_RING/blob/dev/doc/README.apt_rpm_packages.md <https://github.com/ntop/PF_RING/blob/dev/doc/README.apt_rpm_packages.md> ?

Alfredo

> On 18 May 2017, at 10:35, Lüthi Marcel FUB <Marcel.Luethi@vtg.admin.ch> wrote:
>
> Hi Alfredo
>
> Here we go:
>
> [root@myhost]# cat /proc/net/pf_ring/dev/eno1/info
> Name: eno1
> Index: 6
> Address: 38:EA:A7:10:9F:F4
> Polling Mode: NAPI
> Type: Ethernet
> Family: Standard NIC
> # Bound Sockets: 1
> TX Queues: 32
> RX Queues: 32
>
> [root@myhost]# cat /proc/net/pf_ring/dev/eno2/info
> Name: eno2
> Index: 7
> Address: 38:EA:A7:10:9F:F5
> Polling Mode: NAPI
> Type: Ethernet
> Family: Standard NIC
> # Bound Sockets: 1
> TX Queues: 32
> RX Queues: 32
>
>
> Regards,
> Marcel
>
>
>
> Von: ntop-misc-bounces@listgateway.unipi.it [mailto:ntop-misc-bounces@listgateway.unipi.it] Im Auftrag von Alfredo Cardigliano
> Gesendet: Donnerstag, 18. Mai 2017 10:01
> An: ntop-misc@listgateway.unipi.it
> Betreff: Re: [Ntop-misc] n2disk/disk2n: Problem with PF_RING ZC
>
> Hi Marcel
> please provide “cat /proc/net/pf_ring/dev/eno1/info”
>
> Alfredo
>
>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Hi Alfredo

We didn’t realize that we have to follow the mentioned guide. ;-)
I had to do a forcestart, but now it works! Great!

Also we didn’t realize we need a “PF_RING ZC license”.
Of course we will buy one…

Thanks for your support!
Have a good day.

Regards,
Marcel


[root@myhost]# ethtool -i eno1 | grep driver
driver: ixgbe

[root@myhost]# echo "RSS=1,1" > /etc/pf_ring/zc/ixgbe/ixgbe.conf
[root@myhost]# touch /etc/pf_ring/zc/ixgbe/ixgbe.start
[root@myhost]# touch /etc/pf_ring/pf_ring.conf
[root@myhost]# touch /etc/pf_ring/pf_ring.start
[root@myhost]# echo "node=0 hugepagenumber=1024" > /etc/pf_ring/hugepages.conf

[root@myhost]# /etc/init.d/pf_ring start
Starting pf_ring (via systemctl): [ OK ]

[root@myhost]# cat /proc/net/pf_ring/dev/eno1/info | grep ZC
[root@myhost]#

[root@myhost]# /etc/init.d/pf_ring status
The following drivers has not been loaded: ixgbe_zc
pf_ring Loaded

[root@myhost]# touch /etc/pf_ring/forcestart
[root@myhost]# /etc/init.d/pf_ring restart
Restarting pf_ring (via systemctl): [ OK ]

[root@myhost]# /etc/init.d/pf_ring status
Drivers Loaded
pf_ring Loaded

[root@myhost]# cat /proc/net/pf_ring/dev/eno1/info | grep ZC
Polling Mode: NAPI/ZC
[root@myhost]# cat /proc/net/pf_ring/dev/eno2/info | grep ZC
Polling Mode: NAPI/ZC

[root@myhost]# pfsend -i zc:eno1 -n 0
Sending packets on zc:eno1
#########################################################################
# ERROR: You do not seem to have a valid PF_RING ZC license 6.5.0.170303 for eno1 [Intel 10 Gbit ixgbe 82599-based]
# ERROR: Please get one at http://shop.ntop.org/.
#########################################################################
# We're now working in demo mode with packet capture and
# transmission limited to 5 minutes
#########################################################################
Using PF_RING v.6.5.0
Dumping statistics on /proc/net/pf_ring/stats/65481-eno1.2
TX rate: [current 12'801'078.38 pps/8.60 Gbps][average 12'801'078.38 pps/8.60 Gbps][total 12'801'232.00 pkts]
TX rate: [current 14'250'492.72 pps/9.58 Gbps][average 13'525'800.41 pps/9.09 Gbps][total 27'052'480.00 pkts]
TX rate: [current 14'037'856.90 pps/9.43 Gbps][average 13'696'485.51 pps/9.20 Gbps][total 41'090'744.00 pkts]
TX rate: [current 14'180'984.39 pps/9.53 Gbps][average 13'817'611.20 pps/9.29 Gbps][total 55'272'324.00 pkts]
TX rate: [current 14'184'381.39 pps/9.53 Gbps][average 13'890'964.71 pps/9.33 Gbps][total 69'457'060.00 pkts]
TX rate: [current 14'186'540.77 pps/9.53 Gbps][average 13'940'227.21 pps/9.37 Gbps][total 83'643'998.00 pkts]
TX rate: [current 14'194'301.33 pps/9.54 Gbps][average 13'976'523.28 pps/9.39 Gbps][total 97'838'640.00 pkts]
TX rate: [current 14'195'033.90 pps/9.54 Gbps][average 14'003'836.88 pps/9.41 Gbps][total 112'033'972.00 pkts]
TX rate: [current 14'182'567.94 pps/9.53 Gbps][average 14'017'274.59 pps/9.42 Gbps][total 121'258'144.00 pkts]
Sent 121258144 packets
[root@myhost]#



Von: ntop-misc-bounces@listgateway.unipi.it [mailto:ntop-misc-bounces@listgateway.unipi.it] Im Auftrag von Alfredo Cardigliano
Gesendet: Donnerstag, 18. Mai 2017 10:40
An: ntop-misc@listgateway.unipi.it
Betreff: Re: [Ntop-misc] n2disk/disk2n: Problem with PF_RING ZC

Hi Marcel
it seems you did not load the ZC driver (you should see "NAPI/ZC" in “Polling Mode”),
did you follow this guide https://github.com/ntop/PF_RING/blob/dev/doc/README.apt_rpm_packages.md ?

Alfredo