Mailing List Archive: starting ntop errors and warnings

starting ntop errors and warnings

Aug 31, 2017, 9:53 AM

Post #1 of 13 (1264 views)

Hi all,

I have an updated ntopng/pfring-zc installation and when I start ntopng
the following messages are produced:

Aug 30 18:05:06 server logger: ntopng start
Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal
Error: NULL interface
[view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal
Error: NULL interface
[view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal
Error: NULL interface
[view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal
Error: NULL interface
[view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
Aug 30 18:05:08 server kernel: device ens160 entered promiscuous mode
Aug 30 18:05:09 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable
to set packet capture direction
Aug 30 18:05:11 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable
to set packet capture direction
Aug 30 18:05:11 server ntopng: Starting ntopng: Unable to start
ntopng[FAILED]
Aug 30 18:05:11 server systemd: Started Start/stop ntopng program.
Aug 30 18:05:12 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable
to set packet capture direction
Aug 30 18:05:14 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable
to set packet capture direction
Aug 30 18:05:14 server kernel: traps: ntopng[2575] general protection
ip:42df01 sp:7ff418f69b60 error:0 in ntopng[400000+264000]
Aug 30 18:05:15 server kernel: device ens160 left promiscuous mode
Aug 30 18:05:15 server systemd: ntopng.service: main process exited,
code=killed, status=11/SEGV

Why the view of all queues does not work? How should i write this line?

When I delete from ntopng.conf, the
"view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3" line, the I get
another bunch of warnings:

Aug 30 18:06:50 server systemd: Starting Start/stop ntopng program...
Aug 30 18:06:50 server logger: ntopng start
Aug 30 18:06:51 server kernel: device ens160 entered promiscuous mode
Aug 30 18:06:52 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable
to set packet capture direction
Aug 30 18:06:53 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable
to set packet capture direction
Aug 30 18:06:55 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable
to set packet capture direction
Aug 30 18:06:55 server ntopng: Starting ntopng: Unable to start
ntopng[FAILED]
Aug 30 18:06:55 server systemd: Started Start/stop ntopng program.
Aug 30 18:06:56 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable
to set packet capture direction
Aug 30 18:06:57 server ZC[2647]: error opening
/dev/hugepages/ens160@1-txrx: Permission denied
Aug 30 18:06:57 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
Aug 30 18:06:58 server ZC[2647]: error opening
/dev/hugepages/ens160@0-txrx: Permission denied
Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
Aug 30 18:06:58 server ZC[2647]: error opening
/dev/hugepages/ens160@2-txrx: Permission denied
Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
Aug 30 18:06:58 server ZC[2647]: error opening
/dev/hugepages/ens160@3-txrx: Permission denied
Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
Aug 30 18:07:01 server systemd: Started Session 6 of user root.
Aug 30 18:07:01 server systemd: Starting Session 6 of user root.

The ntopng then, seems to be working but I would like to get rid of the
errors. What should I do to avoid the hugepages errors? I'm running ntop
as root (though I can see it drops privs).

My conf:

-G=/var/run/ntopng.pid
--data-dir=/mnt/data/ntopng
#--interface=zc:ens160
--interface=zc:ens160@0
--interface=zc:ens160@1
--interface=zc:ens160@2
--interface=zc:ens160@3
#--interface=view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3
--local-networks="xx.xx.0.0/16,xx.xx.cc.0/24,xx.xx.ff.0/22,xx.xx.vv.0/24"
--dns-mode=3
--disable-autologout
--max-num-flows=512000
--max-num-hosts=256000
--sticky-hosts=none
-g -1

# cat /etc/ntopng/ntopng.start
#

# cat /etc/pf_ring/hugepages.conf
node=0 hugepages=1024
# cat /etc/pf_ring/pf_ring.conf
transparent_mode=2

# cat /etc/pf_ring/pf_ring.start
# cat /etc/pf_ring/zc/ixgbe/ixgbe.conf
RSS=0,0,0,0
# cat /etc/pf_ring/zc/ixgbe/ixgbe.start
#

Regards,

Spiros

_______________________________________________
Ntop mailing list
Ntop@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop

Re: starting ntop errors and warnings [ In reply to ]

cardigliano at ntop

Aug 31, 2017, 10:06 AM

Post #2 of 13 (1264 views)

Permalink

Hi Spiros
please provide the output of:

cat /proc/mounts / grep huge
cat /sys/devices/system/node/node*/meminfo | grep Huge

Alfredo

> On 31 Aug 2017, at 18:53, Spiros Papageorgiou <papage@noc.ntua.gr> wrote:
>
> Hi all,
>
>
> I have an updated ntopng/pfring-zc installation and when I start ntopng the following messages are produced:
>
> Aug 30 18:05:06 server logger: ntopng start
> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
> Aug 30 18:05:08 server kernel: device ens160 entered promiscuous mode
> Aug 30 18:05:09 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
> Aug 30 18:05:11 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
> Aug 30 18:05:11 server ntopng: Starting ntopng: Unable to start ntopng[FAILED]
> Aug 30 18:05:11 server systemd: Started Start/stop ntopng program.
> Aug 30 18:05:12 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
> Aug 30 18:05:14 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
> Aug 30 18:05:14 server kernel: traps: ntopng[2575] general protection ip:42df01 sp:7ff418f69b60 error:0 in ntopng[400000+264000]
> Aug 30 18:05:15 server kernel: device ens160 left promiscuous mode
> Aug 30 18:05:15 server systemd: ntopng.service: main process exited, code=killed, status=11/SEGV
>
> Why the view of all queues does not work? How should i write this line?
>
> When I delete from ntopng.conf, the "view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3" line, the I get another bunch of warnings:
>
> Aug 30 18:06:50 server systemd: Starting Start/stop ntopng program...
> Aug 30 18:06:50 server logger: ntopng start
> Aug 30 18:06:51 server kernel: device ens160 entered promiscuous mode
> Aug 30 18:06:52 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
> Aug 30 18:06:53 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
> Aug 30 18:06:55 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
> Aug 30 18:06:55 server ntopng: Starting ntopng: Unable to start ntopng[FAILED]
> Aug 30 18:06:55 server systemd: Started Start/stop ntopng program.
> Aug 30 18:06:56 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
> Aug 30 18:06:57 server ZC[2647]: error opening /dev/hugepages/ens160@1-txrx: Permission denied
> Aug 30 18:06:57 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@0-txrx: Permission denied
> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@2-txrx: Permission denied
> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@3-txrx: Permission denied
> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
> Aug 30 18:07:01 server systemd: Started Session 6 of user root.
> Aug 30 18:07:01 server systemd: Starting Session 6 of user root.
>
> The ntopng then, seems to be working but I would like to get rid of the errors. What should I do to avoid the hugepages errors? I'm running ntop as root (though I can see it drops privs).
>
> My conf:
>
> -G=/var/run/ntopng.pid
> --data-dir=/mnt/data/ntopng
> #--interface=zc:ens160
> --interface=zc:ens160@0
> --interface=zc:ens160@1
> --interface=zc:ens160@2
> --interface=zc:ens160@3
> #--interface=view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3
> --local-networks="xx.xx.0.0/16,xx.xx.cc.0/24,xx.xx.ff.0/22,xx.xx.vv.0/24"
> --dns-mode=3
> --disable-autologout
> --max-num-flows=512000
> --max-num-hosts=256000
> --sticky-hosts=none
> -g -1
>
> # cat /etc/ntopng/ntopng.start
> #
>
> # cat /etc/pf_ring/hugepages.conf
> node=0 hugepages=1024
> # cat /etc/pf_ring/pf_ring.conf
> transparent_mode=2
>
> # cat /etc/pf_ring/pf_ring.start
> # cat /etc/pf_ring/zc/ixgbe/ixgbe.conf
> RSS=0,0,0,0
> # cat /etc/pf_ring/zc/ixgbe/ixgbe.start
> #
>
>
>
> Regards,
>
> Spiros
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop

Re: starting ntop errors and warnings [ In reply to ]

cardigliano at ntop

Aug 31, 2017, 10:06 AM

Post #3 of 13 (1264 views)

Permalink

Re: starting ntop errors and warnings [ In reply to ]

papage at noc

Aug 31, 2017, 11:10 AM

Post #4 of 13 (1261 views)

Permalink

Hi Alfredo,

Here is the output:
[root@server ~]# cat /proc/mounts | grep huge
cgroup /sys/fs/cgroup/hugetlb cgroup
rw,nosuid,nodev,noexec,relatime,hugetlb 0 0
hugetlbfs /dev/hugepages hugetlbfs rw,relatime 0 0

[root@server ~]# cat /sys/devices/system/node/node*/meminfo | grep Huge
Node 0 AnonHugePages:   2488320 kB
Node 0 HugePages_Total: 2048
Node 0 HugePages_Free:   2048
Node 0 HugePages_Surp:      0
[root@server ~]#

Regards,
Sp

On 31/8/2017 8:06 ??, Alfredo Cardigliano wrote:
> Hi Spiros
> please provide the output of:
>
> cat /proc/mounts / grep huge
> cat /sys/devices/system/node/node*/meminfo | grep Huge
>
> Alfredo
>
>> On 31 Aug 2017, at 18:53, Spiros Papageorgiou <papage@noc.ntua.gr> wrote:
>>
>> Hi all,
>>
>>
>> I have an updated ntopng/pfring-zc installation and when I start ntopng the following messages are produced:
>>
>> Aug 30 18:05:06 server logger: ntopng start
>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>> Aug 30 18:05:08 server kernel: device ens160 entered promiscuous mode
>> Aug 30 18:05:09 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>> Aug 30 18:05:11 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>> Aug 30 18:05:11 server ntopng: Starting ntopng: Unable to start ntopng[FAILED]
>> Aug 30 18:05:11 server systemd: Started Start/stop ntopng program.
>> Aug 30 18:05:12 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>> Aug 30 18:05:14 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>> Aug 30 18:05:14 server kernel: traps: ntopng[2575] general protection ip:42df01 sp:7ff418f69b60 error:0 in ntopng[400000+264000]
>> Aug 30 18:05:15 server kernel: device ens160 left promiscuous mode
>> Aug 30 18:05:15 server systemd: ntopng.service: main process exited, code=killed, status=11/SEGV
>>
>> Why the view of all queues does not work? How should i write this line?
>>
>> When I delete from ntopng.conf, the "view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3" line, the I get another bunch of warnings:
>>
>> Aug 30 18:06:50 server systemd: Starting Start/stop ntopng program...
>> Aug 30 18:06:50 server logger: ntopng start
>> Aug 30 18:06:51 server kernel: device ens160 entered promiscuous mode
>> Aug 30 18:06:52 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>> Aug 30 18:06:53 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>> Aug 30 18:06:55 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>> Aug 30 18:06:55 server ntopng: Starting ntopng: Unable to start ntopng[FAILED]
>> Aug 30 18:06:55 server systemd: Started Start/stop ntopng program.
>> Aug 30 18:06:56 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>> Aug 30 18:06:57 server ZC[2647]: error opening /dev/hugepages/ens160@1-txrx: Permission denied
>> Aug 30 18:06:57 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@0-txrx: Permission denied
>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@2-txrx: Permission denied
>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@3-txrx: Permission denied
>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>> Aug 30 18:07:01 server systemd: Started Session 6 of user root.
>> Aug 30 18:07:01 server systemd: Starting Session 6 of user root.
>>
>> The ntopng then, seems to be working but I would like to get rid of the errors. What should I do to avoid the hugepages errors? I'm running ntop as root (though I can see it drops privs).
>>
>> My conf:
>>
>> -G=/var/run/ntopng.pid
>> --data-dir=/mnt/data/ntopng
>> #--interface=zc:ens160
>> --interface=zc:ens160@0
>> --interface=zc:ens160@1
>> --interface=zc:ens160@2
>> --interface=zc:ens160@3
>> #--interface=view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3
>> --local-networks="xx.xx.0.0/16,xx.xx.cc.0/24,xx.xx.ff.0/22,xx.xx.vv.0/24"
>> --dns-mode=3
>> --disable-autologout
>> --max-num-flows=512000
>> --max-num-hosts=256000
>> --sticky-hosts=none
>> -g -1
>>
>> # cat /etc/ntopng/ntopng.start
>> #
>>
>> # cat /etc/pf_ring/hugepages.conf
>> node=0 hugepages=1024
>> # cat /etc/pf_ring/pf_ring.conf
>> transparent_mode=2
>>
>> # cat /etc/pf_ring/pf_ring.start
>> # cat /etc/pf_ring/zc/ixgbe/ixgbe.conf
>> RSS=0,0,0,0
>> # cat /etc/pf_ring/zc/ixgbe/ixgbe.start
>> #
>>
>>
>>
>> Regards,
>>
>> Spiros
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop

Re: starting ntop errors and warnings [ In reply to ]

deri at ntop

Aug 31, 2017, 11:26 PM

Post #5 of 13 (1260 views)

Permalink

Spiros
can you please update ntopng and report?

Thanks Luca

> On 31 Aug 2017, at 20:10, Spiros Papageorgiou <papage@noc.ntua.gr> wrote:
>
> Hi Alfredo,
>
> Here is the output:
> [root@server ~]# cat /proc/mounts | grep huge
> cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0
> hugetlbfs /dev/hugepages hugetlbfs rw,relatime 0 0
>
> [root@server ~]# cat /sys/devices/system/node/node*/meminfo | grep Huge
> Node 0 AnonHugePages: 2488320 kB
> Node 0 HugePages_Total: 2048
> Node 0 HugePages_Free: 2048
> Node 0 HugePages_Surp: 0
> [root@server ~]#
>
> Regards,
> Sp
>
> On 31/8/2017 8:06 ??, Alfredo Cardigliano wrote:
>> Hi Spiros
>> please provide the output of:
>>
>> cat /proc/mounts / grep huge
>> cat /sys/devices/system/node/node*/meminfo | grep Huge
>>
>> Alfredo
>>
>>> On 31 Aug 2017, at 18:53, Spiros Papageorgiou <papage@noc.ntua.gr> <mailto:papage@noc.ntua.gr> wrote:
>>>
>>> Hi all,
>>>
>>>
>>> I have an updated ntopng/pfring-zc installation and when I start ntopng the following messages are produced:
>>>
>>> Aug 30 18:05:06 server logger: ntopng start
>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>> Aug 30 18:05:08 server kernel: device ens160 entered promiscuous mode
>>> Aug 30 18:05:09 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>> Aug 30 18:05:11 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>> Aug 30 18:05:11 server ntopng: Starting ntopng: Unable to start ntopng[FAILED]
>>> Aug 30 18:05:11 server systemd: Started Start/stop ntopng program.
>>> Aug 30 18:05:12 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>> Aug 30 18:05:14 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>> Aug 30 18:05:14 server kernel: traps: ntopng[2575] general protection ip:42df01 sp:7ff418f69b60 error:0 in ntopng[400000+264000]
>>> Aug 30 18:05:15 server kernel: device ens160 left promiscuous mode
>>> Aug 30 18:05:15 server systemd: ntopng.service: main process exited, code=killed, status=11/SEGV
>>>
>>> Why the view of all queues does not work? How should i write this line?
>>>
>>> When I delete from ntopng.conf, the "view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3" line, the I get another bunch of warnings:
>>>
>>> Aug 30 18:06:50 server systemd: Starting Start/stop ntopng program...
>>> Aug 30 18:06:50 server logger: ntopng start
>>> Aug 30 18:06:51 server kernel: device ens160 entered promiscuous mode
>>> Aug 30 18:06:52 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>> Aug 30 18:06:53 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>> Aug 30 18:06:55 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>> Aug 30 18:06:55 server ntopng: Starting ntopng: Unable to start ntopng[FAILED]
>>> Aug 30 18:06:55 server systemd: Started Start/stop ntopng program.
>>> Aug 30 18:06:56 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>> Aug 30 18:06:57 server ZC[2647]: error opening /dev/hugepages/ens160@1-txrx: Permission denied
>>> Aug 30 18:06:57 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@0-txrx: Permission denied
>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@2-txrx: Permission denied
>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@3-txrx: Permission denied
>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>> Aug 30 18:07:01 server systemd: Started Session 6 of user root.
>>> Aug 30 18:07:01 server systemd: Starting Session 6 of user root.
>>>
>>> The ntopng then, seems to be working but I would like to get rid of the errors. What should I do to avoid the hugepages errors? I'm running ntop as root (though I can see it drops privs).
>>>
>>> My conf:
>>>
>>> -G=/var/run/ntopng.pid
>>> --data-dir=/mnt/data/ntopng
>>> #--interface=zc:ens160
>>> --interface=zc:ens160@0
>>> --interface=zc:ens160@1
>>> --interface=zc:ens160@2
>>> --interface=zc:ens160@3
>>> #--interface=view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3
>>> --local-networks="xx.xx.0.0/16,xx.xx.cc.0/24,xx.xx.ff.0/22,xx.xx.vv.0/24"
>>> --dns-mode=3
>>> --disable-autologout
>>> --max-num-flows=512000
>>> --max-num-hosts=256000
>>> --sticky-hosts=none
>>> -g -1
>>>
>>> # cat /etc/ntopng/ntopng.start
>>> #
>>>
>>> # cat /etc/pf_ring/hugepages.conf
>>> node=0 hugepages=1024
>>> # cat /etc/pf_ring/pf_ring.conf
>>> transparent_mode=2
>>>
>>> # cat /etc/pf_ring/pf_ring.start
>>> # cat /etc/pf_ring/zc/ixgbe/ixgbe.conf
>>> RSS=0,0,0,0
>>> # cat /etc/pf_ring/zc/ixgbe/ixgbe.start
>>> #
>>>
>>>
>>>
>>> Regards,
>>>
>>> Spiros
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop

Re: starting ntop errors and warnings [ In reply to ]

papage at noc

Sep 1, 2017, 9:43 PM

Post #6 of 13 (1256 views)

Permalink

Hi Luca,

I updated everything and the error messages seem to be gone. I can see
that huge pages have been allocated:

# cat /sys/devices/system/node/node*/meminfo | grep Huge F10Quit
Node 0 AnonHugePages:    100352 kB
Node 0 HugePages_Total: 2048
Node 0 HugePages_Free:   1852
Node 0 HugePages_Surp:      0

Thanx!

Two more problems:
- What about the "view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3"
which is not working? Is the syntax wrong or something?
- I get "02/Sep/2017 07:34:50 [Lua.cpp:5629]
[@/usr/share/ntopng/scripts/callbacks/5min.lua]:[161] ERROR: Cannot
complete local hosts RRD dump in 5 minutes. Please check your RRD
configuration.". Do I need a stronger server? I have a 4core VM and the
disk (SAN) seems not to be stressed that much. The CPU though is at
about 70%.
- The same goes when I export to ES. The ES is at about 30% CPU
utilization but ntopng reports massive flow drops.

Regards,
Spiros

On 1/9/2017 9:26 ??, Luca Deri wrote:
> Spiros
> can you please update ntopng and report?
>
> Thanks Luca
>
>> On 31 Aug 2017, at 20:10, Spiros Papageorgiou <papage@noc.ntua.gr
>> <mailto:papage@noc.ntua.gr>> wrote:
>>
>> Hi Alfredo,
>>
>> Here is the output:
>> [root@server ~]# cat /proc/mounts | grep huge
>> cgroup /sys/fs/cgroup/hugetlb cgroup
>> rw,nosuid,nodev,noexec,relatime,hugetlb 0 0
>> hugetlbfs /dev/hugepages hugetlbfs rw,relatime 0 0
>>
>> [root@server ~]# cat /sys/devices/system/node/node*/meminfo | grep Huge
>> Node 0 AnonHugePages:   2488320 kB
>> Node 0 HugePages_Total: 2048
>> Node 0 HugePages_Free:   2048
>> Node 0 HugePages_Surp:      0
>> [root@server ~]#
>>
>> Regards,
>> Sp
>>
>> On 31/8/2017 8:06 ??, Alfredo Cardigliano wrote:
>>> Hi Spiros
>>> please provide the output of:
>>>
>>> cat /proc/mounts / grep huge
>>> cat /sys/devices/system/node/node*/meminfo | grep Huge
>>>
>>> Alfredo
>>>
>>>> On 31 Aug 2017, at 18:53, Spiros Papageorgiou<papage@noc.ntua.gr> wrote:
>>>>
>>>> Hi all,
>>>>
>>>>
>>>> I have an updated ntopng/pfring-zc installation and when I start ntopng the following messages are produced:
>>>>
>>>> Aug 30 18:05:06 server logger: ntopng start
>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>> Aug 30 18:05:08 server kernel: device ens160 entered promiscuous mode
>>>> Aug 30 18:05:09 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>> Aug 30 18:05:11 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>> Aug 30 18:05:11 server ntopng: Starting ntopng: Unable to start ntopng[FAILED]
>>>> Aug 30 18:05:11 server systemd: Started Start/stop ntopng program.
>>>> Aug 30 18:05:12 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>> Aug 30 18:05:14 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>> Aug 30 18:05:14 server kernel: traps: ntopng[2575] general protection ip:42df01 sp:7ff418f69b60 error:0 in ntopng[400000+264000]
>>>> Aug 30 18:05:15 server kernel: device ens160 left promiscuous mode
>>>> Aug 30 18:05:15 server systemd: ntopng.service: main process exited, code=killed, status=11/SEGV
>>>>
>>>> Why the view of all queues does not work? How should i write this line?
>>>>
>>>> When I delete from ntopng.conf, the "view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3" line, the I get another bunch of warnings:
>>>>
>>>> Aug 30 18:06:50 server systemd: Starting Start/stop ntopng program...
>>>> Aug 30 18:06:50 server logger: ntopng start
>>>> Aug 30 18:06:51 server kernel: device ens160 entered promiscuous mode
>>>> Aug 30 18:06:52 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>> Aug 30 18:06:53 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>> Aug 30 18:06:55 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>> Aug 30 18:06:55 server ntopng: Starting ntopng: Unable to start ntopng[FAILED]
>>>> Aug 30 18:06:55 server systemd: Started Start/stop ntopng program.
>>>> Aug 30 18:06:56 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>> Aug 30 18:06:57 server ZC[2647]: error opening /dev/hugepages/ens160@1-txrx: Permission denied
>>>> Aug 30 18:06:57 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@0-txrx: Permission denied
>>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@2-txrx: Permission denied
>>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@3-txrx: Permission denied
>>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>> Aug 30 18:07:01 server systemd: Started Session 6 of user root.
>>>> Aug 30 18:07:01 server systemd: Starting Session 6 of user root.
>>>>
>>>> The ntopng then, seems to be working but I would like to get rid of the errors. What should I do to avoid the hugepages errors? I'm running ntop as root (though I can see it drops privs).
>>>>
>>>> My conf:
>>>>
>>>> -G=/var/run/ntopng.pid
>>>> --data-dir=/mnt/data/ntopng
>>>> #--interface=zc:ens160
>>>> --interface=zc:ens160@0
>>>> --interface=zc:ens160@1
>>>> --interface=zc:ens160@2
>>>> --interface=zc:ens160@3
>>>> #--interface=view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3
>>>> --local-networks="xx.xx.0.0/16,xx.xx.cc <http://xx.xx.cc>.0/24,xx.xx.ff.0/22,xx.xx.vv.0/24"
>>>> --dns-mode=3
>>>> --disable-autologout
>>>> --max-num-flows=512000
>>>> --max-num-hosts=256000
>>>> --sticky-hosts=none
>>>> -g -1
>>>>
>>>> # cat /etc/ntopng/ntopng.start
>>>> #
>>>>
>>>> # cat /etc/pf_ring/hugepages.conf
>>>> node=0 hugepages=1024
>>>> # cat /etc/pf_ring/pf_ring.conf
>>>> transparent_mode=2
>>>>
>>>> # cat /etc/pf_ring/pf_ring.start
>>>> # cat /etc/pf_ring/zc/ixgbe/ixgbe.conf
>>>> RSS=0,0,0,0
>>>> # cat /etc/pf_ring/zc/ixgbe/ixgbe.start
>>>> #
>>>>
>>>>
>>>>
>>>> Regards,
>>>>
>>>> Spiros
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop

Re: starting ntop errors and warnings [ In reply to ]

deri at ntop

Sep 3, 2017, 1:46 AM

Post #7 of 13 (1255 views)

Permalink

Spiros,
can you please file an issue on github s we can track this problem? Please open one ticket per issue

thanks Luca

> On 2 Sep 2017, at 06:43, Spiros Papageorgiou <papage@noc.ntua.gr> wrote:
>
> Hi Luca,
>
> I updated everything and the error messages seem to be gone. I can see that huge pages have been allocated:
>
> # cat /sys/devices/system/node/node*/meminfo | grep Huge F10Quit
> Node 0 AnonHugePages: 100352 kB
> Node 0 HugePages_Total: 2048
> Node 0 HugePages_Free: 1852
> Node 0 HugePages_Surp: 0
>
> Thanx!
>
> Two more problems:
> - What about the "view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3" which is not working? Is the syntax wrong or something?
> - I get "02/Sep/2017 07:34:50 [Lua.cpp:5629] [@/usr/share/ntopng/scripts/callbacks/5min.lua]:[161] ERROR: Cannot complete local hosts RRD dump in 5 minutes. Please check your RRD configuration.". Do I need a stronger server? I have a 4core VM and the disk (SAN) seems not to be stressed that much. The CPU though is at about 70%.
> - The same goes when I export to ES. The ES is at about 30% CPU utilization but ntopng reports massive flow drops.
>
> Regards,
> Spiros
>
>
> On 1/9/2017 9:26 ??, Luca Deri wrote:
>> Spiros
>> can you please update ntopng and report?
>>
>> Thanks Luca
>>
>>> On 31 Aug 2017, at 20:10, Spiros Papageorgiou <papage@noc.ntua.gr <mailto:papage@noc.ntua.gr>> wrote:
>>>
>>> Hi Alfredo,
>>>
>>> Here is the output:
>>> [root@server ~]# cat /proc/mounts | grep huge
>>> cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0
>>> hugetlbfs /dev/hugepages hugetlbfs rw,relatime 0 0
>>>
>>> [root@server ~]# cat /sys/devices/system/node/node*/meminfo | grep Huge
>>> Node 0 AnonHugePages: 2488320 kB
>>> Node 0 HugePages_Total: 2048
>>> Node 0 HugePages_Free: 2048
>>> Node 0 HugePages_Surp: 0
>>> [root@server ~]#
>>>
>>> Regards,
>>> Sp
>>>
>>> On 31/8/2017 8:06 ??, Alfredo Cardigliano wrote:
>>>> Hi Spiros
>>>> please provide the output of:
>>>>
>>>> cat /proc/mounts / grep huge
>>>> cat /sys/devices/system/node/node*/meminfo | grep Huge
>>>>
>>>> Alfredo
>>>>
>>>>> On 31 Aug 2017, at 18:53, Spiros Papageorgiou <papage@noc.ntua.gr> <mailto:papage@noc.ntua.gr> wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>>
>>>>> I have an updated ntopng/pfring-zc installation and when I start ntopng the following messages are produced:
>>>>>
>>>>> Aug 30 18:05:06 server logger: ntopng start
>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>>> Aug 30 18:05:08 server kernel: device ens160 entered promiscuous mode
>>>>> Aug 30 18:05:09 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>> Aug 30 18:05:11 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>> Aug 30 18:05:11 server ntopng: Starting ntopng: Unable to start ntopng[FAILED]
>>>>> Aug 30 18:05:11 server systemd: Started Start/stop ntopng program.
>>>>> Aug 30 18:05:12 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>> Aug 30 18:05:14 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>> Aug 30 18:05:14 server kernel: traps: ntopng[2575] general protection ip:42df01 sp:7ff418f69b60 error:0 in ntopng[400000+264000]
>>>>> Aug 30 18:05:15 server kernel: device ens160 left promiscuous mode
>>>>> Aug 30 18:05:15 server systemd: ntopng.service: main process exited, code=killed, status=11/SEGV
>>>>>
>>>>> Why the view of all queues does not work? How should i write this line?
>>>>>
>>>>> When I delete from ntopng.conf, the "view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3" line, the I get another bunch of warnings:
>>>>>
>>>>> Aug 30 18:06:50 server systemd: Starting Start/stop ntopng program...
>>>>> Aug 30 18:06:50 server logger: ntopng start
>>>>> Aug 30 18:06:51 server kernel: device ens160 entered promiscuous mode
>>>>> Aug 30 18:06:52 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>> Aug 30 18:06:53 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>> Aug 30 18:06:55 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>> Aug 30 18:06:55 server ntopng: Starting ntopng: Unable to start ntopng[FAILED]
>>>>> Aug 30 18:06:55 server systemd: Started Start/stop ntopng program.
>>>>> Aug 30 18:06:56 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>> Aug 30 18:06:57 server ZC[2647]: error opening /dev/hugepages/ens160@1-txrx: Permission denied
>>>>> Aug 30 18:06:57 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@0-txrx: Permission denied
>>>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@2-txrx: Permission denied
>>>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@3-txrx: Permission denied
>>>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>>> Aug 30 18:07:01 server systemd: Started Session 6 of user root.
>>>>> Aug 30 18:07:01 server systemd: Starting Session 6 of user root.
>>>>>
>>>>> The ntopng then, seems to be working but I would like to get rid of the errors. What should I do to avoid the hugepages errors? I'm running ntop as root (though I can see it drops privs).
>>>>>
>>>>> My conf:
>>>>>
>>>>> -G=/var/run/ntopng.pid
>>>>> --data-dir=/mnt/data/ntopng
>>>>> #--interface=zc:ens160
>>>>> --interface=zc:ens160@0
>>>>> --interface=zc:ens160@1
>>>>> --interface=zc:ens160@2
>>>>> --interface=zc:ens160@3
>>>>> #--interface=view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3
>>>>> --local-networks="xx.xx.0.0/16,xx.xx.cc <http://xx.xx.cc/>.0/24,xx.xx.ff.0/22,xx.xx.vv.0/24"
>>>>> --dns-mode=3
>>>>> --disable-autologout
>>>>> --max-num-flows=512000
>>>>> --max-num-hosts=256000
>>>>> --sticky-hosts=none
>>>>> -g -1
>>>>>
>>>>> # cat /etc/ntopng/ntopng.start
>>>>> #
>>>>>
>>>>> # cat /etc/pf_ring/hugepages.conf
>>>>> node=0 hugepages=1024
>>>>> # cat /etc/pf_ring/pf_ring.conf
>>>>> transparent_mode=2
>>>>>
>>>>> # cat /etc/pf_ring/pf_ring.start
>>>>> # cat /etc/pf_ring/zc/ixgbe/ixgbe.conf
>>>>> RSS=0,0,0,0
>>>>> # cat /etc/pf_ring/zc/ixgbe/ixgbe.start
>>>>> #
>>>>>
>>>>>
>>>>>
>>>>> Regards,
>>>>>
>>>>> Spiros
>>>>>
>>>>> _______________________________________________
>>>>> Ntop mailing list
>>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop

Re: starting ntop errors and warnings [ In reply to ]

deri at ntop

Sep 3, 2017, 1:46 AM

Post #8 of 13 (1255 views)

Permalink

Re: starting ntop errors and warnings [ In reply to ]

mainardi at ntop

Sep 4, 2017, 12:47 AM

Post #9 of 13 (1251 views)

Permalink

Spiros,

> On 2 Sep 2017, at 06:43, Spiros Papageorgiou <papage@noc.ntua.gr> wrote:
>
> Hi Luca,
>
> I updated everything and the error messages seem to be gone. I can see that huge pages have been allocated:
>
> # cat /sys/devices/system/node/node*/meminfo | grep Huge F10Quit
> Node 0 AnonHugePages: 100352 kB
> Node 0 HugePages_Total: 2048
> Node 0 HugePages_Free: 1852
> Node 0 HugePages_Surp: 0
>
> Thanx!
>
> Two more problems:
> - What about the "view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3" which is not working? Is the syntax wrong or something?

What do you mean with 'is not working'? So you get empty pages? Please, report the issue using our GitHub issue tracker.

> - I get "02/Sep/2017 07:34:50 [Lua.cpp:5629] [@/usr/share/ntopng/scripts/callbacks/5min.lua]:[161] ERROR: Cannot complete local hosts RRD dump in 5 minutes. Please check your RRD configuration.". Do I need a stronger server? I have a 4core VM and the disk (SAN) seems not to be stressed that much. The CPU though is at about 70%.

This means that you have too many local hosts that there's not enough time to update their statistics in 5 minutes. How many local hosts do you have? Disabling data retention of L7 timeseries may alleviate the issue.
The bottleneck here are the several multiple RRD updates made for every local host.

> - The same goes when I export to ES. The ES is at about 30% CPU utilization but ntopng reports massive flow drops.

ElasticSearch is not fast enough to inject flows at the required speed. Hence ntopng flows export queue grows and at some point it overflows resulting in flows drops. Other people experienced this in the past. You must configure your ES deployment to make sure it has enough intake.

>
> Regards,
> Spiros
>
>
> On 1/9/2017 9:26 ??, Luca Deri wrote:
>> Spiros
>> can you please update ntopng and report?
>>
>> Thanks Luca
>>
>>> On 31 Aug 2017, at 20:10, Spiros Papageorgiou <papage@noc.ntua.gr <mailto:papage@noc.ntua.gr>> wrote:
>>>
>>> Hi Alfredo,
>>>
>>> Here is the output:
>>> [root@server ~]# cat /proc/mounts | grep huge
>>> cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0
>>> hugetlbfs /dev/hugepages hugetlbfs rw,relatime 0 0
>>>
>>> [root@server ~]# cat /sys/devices/system/node/node*/meminfo | grep Huge
>>> Node 0 AnonHugePages: 2488320 kB
>>> Node 0 HugePages_Total: 2048
>>> Node 0 HugePages_Free: 2048
>>> Node 0 HugePages_Surp: 0
>>> [root@server ~]#
>>>
>>> Regards,
>>> Sp
>>>
>>> On 31/8/2017 8:06 ??, Alfredo Cardigliano wrote:
>>>> Hi Spiros
>>>> please provide the output of:
>>>>
>>>> cat /proc/mounts / grep huge
>>>> cat /sys/devices/system/node/node*/meminfo | grep Huge
>>>>
>>>> Alfredo
>>>>
>>>>> On 31 Aug 2017, at 18:53, Spiros Papageorgiou <papage@noc.ntua.gr> <mailto:papage@noc.ntua.gr> wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>>
>>>>> I have an updated ntopng/pfring-zc installation and when I start ntopng the following messages are produced:
>>>>>
>>>>> Aug 30 18:05:06 server logger: ntopng start
>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>>> Aug 30 18:05:08 server kernel: device ens160 entered promiscuous mode
>>>>> Aug 30 18:05:09 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>> Aug 30 18:05:11 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>> Aug 30 18:05:11 server ntopng: Starting ntopng: Unable to start ntopng[FAILED]
>>>>> Aug 30 18:05:11 server systemd: Started Start/stop ntopng program.
>>>>> Aug 30 18:05:12 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>> Aug 30 18:05:14 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>> Aug 30 18:05:14 server kernel: traps: ntopng[2575] general protection ip:42df01 sp:7ff418f69b60 error:0 in ntopng[400000+264000]
>>>>> Aug 30 18:05:15 server kernel: device ens160 left promiscuous mode
>>>>> Aug 30 18:05:15 server systemd: ntopng.service: main process exited, code=killed, status=11/SEGV
>>>>>
>>>>> Why the view of all queues does not work? How should i write this line?
>>>>>
>>>>> When I delete from ntopng.conf, the "view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3" line, the I get another bunch of warnings:
>>>>>
>>>>> Aug 30 18:06:50 server systemd: Starting Start/stop ntopng program...
>>>>> Aug 30 18:06:50 server logger: ntopng start
>>>>> Aug 30 18:06:51 server kernel: device ens160 entered promiscuous mode
>>>>> Aug 30 18:06:52 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>> Aug 30 18:06:53 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>> Aug 30 18:06:55 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>> Aug 30 18:06:55 server ntopng: Starting ntopng: Unable to start ntopng[FAILED]
>>>>> Aug 30 18:06:55 server systemd: Started Start/stop ntopng program.
>>>>> Aug 30 18:06:56 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>> Aug 30 18:06:57 server ZC[2647]: error opening /dev/hugepages/ens160@1-txrx: Permission denied
>>>>> Aug 30 18:06:57 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@0-txrx: Permission denied
>>>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@2-txrx: Permission denied
>>>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@3-txrx: Permission denied
>>>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>>> Aug 30 18:07:01 server systemd: Started Session 6 of user root.
>>>>> Aug 30 18:07:01 server systemd: Starting Session 6 of user root.
>>>>>
>>>>> The ntopng then, seems to be working but I would like to get rid of the errors. What should I do to avoid the hugepages errors? I'm running ntop as root (though I can see it drops privs).
>>>>>
>>>>> My conf:
>>>>>
>>>>> -G=/var/run/ntopng.pid
>>>>> --data-dir=/mnt/data/ntopng
>>>>> #--interface=zc:ens160
>>>>> --interface=zc:ens160@0
>>>>> --interface=zc:ens160@1
>>>>> --interface=zc:ens160@2
>>>>> --interface=zc:ens160@3
>>>>> #--interface=view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3
>>>>> --local-networks="xx.xx.0.0/16,xx.xx.cc <http://xx.xx.cc/>.0/24,xx.xx.ff.0/22,xx.xx.vv.0/24"
>>>>> --dns-mode=3
>>>>> --disable-autologout
>>>>> --max-num-flows=512000
>>>>> --max-num-hosts=256000
>>>>> --sticky-hosts=none
>>>>> -g -1
>>>>>
>>>>> # cat /etc/ntopng/ntopng.start
>>>>> #
>>>>>
>>>>> # cat /etc/pf_ring/hugepages.conf
>>>>> node=0 hugepages=1024
>>>>> # cat /etc/pf_ring/pf_ring.conf
>>>>> transparent_mode=2
>>>>>
>>>>> # cat /etc/pf_ring/pf_ring.start
>>>>> # cat /etc/pf_ring/zc/ixgbe/ixgbe.conf
>>>>> RSS=0,0,0,0
>>>>> # cat /etc/pf_ring/zc/ixgbe/ixgbe.start
>>>>> #
>>>>>
>>>>>
>>>>>
>>>>> Regards,
>>>>>
>>>>> Spiros
>>>>>
>>>>> _______________________________________________
>>>>> Ntop mailing list
>>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop

Re: starting ntop errors and warnings [ In reply to ]

mainardi at ntop

Sep 4, 2017, 12:47 AM

Post #10 of 13 (1251 views)

Permalink

Re: starting ntop errors and warnings [ In reply to ]

papage at noc

Sep 4, 2017, 9:39 AM

Post #11 of 13 (1248 views)

Permalink

yeap, ok.

Thanx!
Sp
On 3/9/2017 11:46 ??, Luca Deri wrote:
> Spiros,
> can you please file an issue on github s we can track this problem?
> Please open one ticket per issue
>
> thanks Luca
>
>> On 2 Sep 2017, at 06:43, Spiros Papageorgiou <papage@noc.ntua.gr
>> <mailto:papage@noc.ntua.gr>> wrote:
>>
>> Hi Luca,
>>
>> I updated everything and the error messages seem to be gone. I can
>> see that huge pages have been allocated:
>>
>> # cat /sys/devices/system/node/node*/meminfo | grep Huge F10Quit
>> Node 0 AnonHugePages:    100352 kB
>> Node 0 HugePages_Total: 2048
>> Node 0 HugePages_Free:   1852
>> Node 0 HugePages_Surp:      0
>>
>> Thanx!
>>
>> Two more problems:
>> - What about the
>> "view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3" which is not
>> working? Is the syntax wrong or something?
>> - I get "02/Sep/2017 07:34:50 [Lua.cpp:5629]
>> [@/usr/share/ntopng/scripts/callbacks/5min.lua]:[161] ERROR: Cannot
>> complete local hosts RRD dump in 5 minutes. Please check your RRD
>> configuration.". Do I need a stronger server? I have a 4core VM and
>> the disk (SAN) seems not to be stressed that much. The CPU though is
>> at about 70%.
>> - The same goes when I export to ES. The ES is at about 30% CPU
>> utilization but ntopng reports massive flow drops.
>>
>> Regards,
>> Spiros
>>
>>
>> On 1/9/2017 9:26 ??, Luca Deri wrote:
>>> Spiros
>>> can you please update ntopng and report?
>>>
>>> Thanks Luca
>>>
>>>> On 31 Aug 2017, at 20:10, Spiros Papageorgiou <papage@noc.ntua.gr
>>>> <mailto:papage@noc.ntua.gr>> wrote:
>>>>
>>>> Hi Alfredo,
>>>>
>>>> Here is the output:
>>>> [root@server ~]# cat /proc/mounts | grep huge
>>>> cgroup /sys/fs/cgroup/hugetlb cgroup
>>>> rw,nosuid,nodev,noexec,relatime,hugetlb 0 0
>>>> hugetlbfs /dev/hugepages hugetlbfs rw,relatime 0 0
>>>>
>>>> [root@server ~]# cat /sys/devices/system/node/node*/meminfo | grep Huge
>>>> Node 0 AnonHugePages: 2488320 kB
>>>> Node 0 HugePages_Total: 2048
>>>> Node 0 HugePages_Free: 2048
>>>> Node 0 HugePages_Surp:      0
>>>> [root@server ~]#
>>>>
>>>> Regards,
>>>> Sp
>>>>
>>>> On 31/8/2017 8:06 ??, Alfredo Cardigliano wrote:
>>>>> Hi Spiros
>>>>> please provide the output of:
>>>>>
>>>>> cat /proc/mounts / grep huge
>>>>> cat /sys/devices/system/node/node*/meminfo | grep Huge
>>>>>
>>>>> Alfredo
>>>>>
>>>>>> On 31 Aug 2017, at 18:53, Spiros Papageorgiou<papage@noc.ntua.gr> wrote:
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>>
>>>>>> I have an updated ntopng/pfring-zc installation and when I start ntopng the following messages are produced:
>>>>>>
>>>>>> Aug 30 18:05:06 server logger: ntopng start
>>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>>>> Aug 30 18:05:08 server kernel: device ens160 entered promiscuous mode
>>>>>> Aug 30 18:05:09 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>> Aug 30 18:05:11 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>> Aug 30 18:05:11 server ntopng: Starting ntopng: Unable to start ntopng[FAILED]
>>>>>> Aug 30 18:05:11 server systemd: Started Start/stop ntopng program.
>>>>>> Aug 30 18:05:12 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>> Aug 30 18:05:14 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>> Aug 30 18:05:14 server kernel: traps: ntopng[2575] general protection ip:42df01 sp:7ff418f69b60 error:0 in ntopng[400000+264000]
>>>>>> Aug 30 18:05:15 server kernel: device ens160 left promiscuous mode
>>>>>> Aug 30 18:05:15 server systemd: ntopng.service: main process exited, code=killed, status=11/SEGV
>>>>>>
>>>>>> Why the view of all queues does not work? How should i write this line?
>>>>>>
>>>>>> When I delete from ntopng.conf, the "view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3" line, the I get another bunch of warnings:
>>>>>>
>>>>>> Aug 30 18:06:50 server systemd: Starting Start/stop ntopng program...
>>>>>> Aug 30 18:06:50 server logger: ntopng start
>>>>>> Aug 30 18:06:51 server kernel: device ens160 entered promiscuous mode
>>>>>> Aug 30 18:06:52 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>> Aug 30 18:06:53 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>> Aug 30 18:06:55 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>> Aug 30 18:06:55 server ntopng: Starting ntopng: Unable to start ntopng[FAILED]
>>>>>> Aug 30 18:06:55 server systemd: Started Start/stop ntopng program.
>>>>>> Aug 30 18:06:56 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>> Aug 30 18:06:57 server ZC[2647]: error opening /dev/hugepages/ens160@1-txrx: Permission denied
>>>>>> Aug 30 18:06:57 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@0-txrx: Permission denied
>>>>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@2-txrx: Permission denied
>>>>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@3-txrx: Permission denied
>>>>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>>>> Aug 30 18:07:01 server systemd: Started Session 6 of user root.
>>>>>> Aug 30 18:07:01 server systemd: Starting Session 6 of user root.
>>>>>>
>>>>>> The ntopng then, seems to be working but I would like to get rid of the errors. What should I do to avoid the hugepages errors? I'm running ntop as root (though I can see it drops privs).
>>>>>>
>>>>>> My conf:
>>>>>>
>>>>>> -G=/var/run/ntopng.pid
>>>>>> --data-dir=/mnt/data/ntopng
>>>>>> #--interface=zc:ens160
>>>>>> --interface=zc:ens160@0
>>>>>> --interface=zc:ens160@1
>>>>>> --interface=zc:ens160@2
>>>>>> --interface=zc:ens160@3
>>>>>> #--interface=view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3
>>>>>> --local-networks="xx.xx.0.0/16,xx.xx.cc <http://xx.xx.cc/>.0/24,xx.xx.ff.0/22,xx.xx.vv.0/24"
>>>>>> --dns-mode=3
>>>>>> --disable-autologout
>>>>>> --max-num-flows=512000
>>>>>> --max-num-hosts=256000
>>>>>> --sticky-hosts=none
>>>>>> -g -1
>>>>>>
>>>>>> # cat /etc/ntopng/ntopng.start
>>>>>> #
>>>>>>
>>>>>> # cat /etc/pf_ring/hugepages.conf
>>>>>> node=0 hugepages=1024
>>>>>> # cat /etc/pf_ring/pf_ring.conf
>>>>>> transparent_mode=2
>>>>>>
>>>>>> # cat /etc/pf_ring/pf_ring.start
>>>>>> # cat /etc/pf_ring/zc/ixgbe/ixgbe.conf
>>>>>> RSS=0,0,0,0
>>>>>> # cat /etc/pf_ring/zc/ixgbe/ixgbe.start
>>>>>> #
>>>>>>
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Spiros
>>>>>>
>>>>>> _______________________________________________
>>>>>> Ntop mailing list
>>>>>> Ntop@listgateway.unipi.it
>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Ntop mailing list
>>>>> Ntop@listgateway.unipi.it
>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop

Re: starting ntop errors and warnings [ In reply to ]

papage at noc

Sep 4, 2017, 12:48 PM

Post #12 of 13 (1246 views)

Permalink

On 4/9/2017 10:47 ??, Simone Mainardi wrote:
> Spiros,
>
>> On 2 Sep 2017, at 06:43, Spiros Papageorgiou <papage@noc.ntua.gr
>> <mailto:papage@noc.ntua.gr>> wrote:
>>
>> Hi Luca,
>>
>> I updated everything and the error messages seem to be gone. I can
>> see that huge pages have been allocated:
>>
>> # cat /sys/devices/system/node/node*/meminfo | grep Huge F10Quit
>> Node 0 AnonHugePages:    100352 kB
>> Node 0 HugePages_Total: 2048
>> Node 0 HugePages_Free:   1852
>> Node 0 HugePages_Surp:      0
>>
>> Thanx!
>>
>> Two more problems:
>> - What about the
>> "view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3" which is not
>> working? Is the syntax wrong or something?
>
> What do you mean with 'is not working'? So you get empty pages?
> Please, report the issue using our GitHub issue tracker.
>
>> - I get "02/Sep/2017 07:34:50 [Lua.cpp:5629]
>> [@/usr/share/ntopng/scripts/callbacks/5min.lua]:[161] ERROR: Cannot
>> complete local hosts RRD dump in 5 minutes. Please check your RRD
>> configuration.". Do I need a stronger server? I have a 4core VM and
>> the disk (SAN) seems not to be stressed that much. The CPU though is
>> at about 70%.
>
> This means that you have too many local hosts that there's not enough
> time to update their statistics in 5 minutes. How many local hosts do
> you have? Disabling data retention of L7 timeseries may alleviate the
> issue.
> The bottleneck here are the several multiple RRD updates made for
> every local host.
Ntopng reports about 10K local hosts and 50K remote.
How can i disable L7 timeseries data retention? Can I keep stats for
only local hosts? What kind of control do i have on that aspect?
Is it possible to use somekind of caching, like rrdcache?

>
>> - The same goes when I export to ES. The ES is at about 30% CPU
>> utilization but ntopng reports massive flow drops.
>
> ElasticSearch is not fast enough to inject flows at the required
> speed. Hence ntopng flows export queue grows and at some point it
> overflows resulting in flows drops. Other people experienced this in
> the past. You must configure your ES deployment to make sure it has
> enough intake.
>
>>
>> Regards,
>> Spiros
>>
>>
>> On 1/9/2017 9:26 ??, Luca Deri wrote:
>>> Spiros
>>> can you please update ntopng and report?
>>>
>>> Thanks Luca
>>>
>>>> On 31 Aug 2017, at 20:10, Spiros Papageorgiou <papage@noc.ntua.gr
>>>> <mailto:papage@noc.ntua.gr>> wrote:
>>>>
>>>> Hi Alfredo,
>>>>
>>>> Here is the output:
>>>> [root@server ~]# cat /proc/mounts | grep huge
>>>> cgroup /sys/fs/cgroup/hugetlb cgroup
>>>> rw,nosuid,nodev,noexec,relatime,hugetlb 0 0
>>>> hugetlbfs /dev/hugepages hugetlbfs rw,relatime 0 0
>>>>
>>>> [root@server ~]# cat /sys/devices/system/node/node*/meminfo | grep Huge
>>>> Node 0 AnonHugePages: 2488320 kB
>>>> Node 0 HugePages_Total: 2048
>>>> Node 0 HugePages_Free: 2048
>>>> Node 0 HugePages_Surp:      0
>>>> [root@server ~]#
>>>>
>>>> Regards,
>>>> Sp
>>>>
>>>> On 31/8/2017 8:06 ??, Alfredo Cardigliano wrote:
>>>>> Hi Spiros
>>>>> please provide the output of:
>>>>>
>>>>> cat /proc/mounts / grep huge
>>>>> cat /sys/devices/system/node/node*/meminfo | grep Huge
>>>>>
>>>>> Alfredo
>>>>>
>>>>>> On 31 Aug 2017, at 18:53, Spiros Papageorgiou<papage@noc.ntua.gr> wrote:
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>>
>>>>>> I have an updated ntopng/pfring-zc installation and when I start ntopng the following messages are produced:
>>>>>>
>>>>>> Aug 30 18:05:06 server logger: ntopng start
>>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>>>> Aug 30 18:05:08 server kernel: device ens160 entered promiscuous mode
>>>>>> Aug 30 18:05:09 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>> Aug 30 18:05:11 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>> Aug 30 18:05:11 server ntopng: Starting ntopng: Unable to start ntopng[FAILED]
>>>>>> Aug 30 18:05:11 server systemd: Started Start/stop ntopng program.
>>>>>> Aug 30 18:05:12 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>> Aug 30 18:05:14 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>> Aug 30 18:05:14 server kernel: traps: ntopng[2575] general protection ip:42df01 sp:7ff418f69b60 error:0 in ntopng[400000+264000]
>>>>>> Aug 30 18:05:15 server kernel: device ens160 left promiscuous mode
>>>>>> Aug 30 18:05:15 server systemd: ntopng.service: main process exited, code=killed, status=11/SEGV
>>>>>>
>>>>>> Why the view of all queues does not work? How should i write this line?
>>>>>>
>>>>>> When I delete from ntopng.conf, the "view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3" line, the I get another bunch of warnings:
>>>>>>
>>>>>> Aug 30 18:06:50 server systemd: Starting Start/stop ntopng program...
>>>>>> Aug 30 18:06:50 server logger: ntopng start
>>>>>> Aug 30 18:06:51 server kernel: device ens160 entered promiscuous mode
>>>>>> Aug 30 18:06:52 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>> Aug 30 18:06:53 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>> Aug 30 18:06:55 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>> Aug 30 18:06:55 server ntopng: Starting ntopng: Unable to start ntopng[FAILED]
>>>>>> Aug 30 18:06:55 server systemd: Started Start/stop ntopng program.
>>>>>> Aug 30 18:06:56 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>> Aug 30 18:06:57 server ZC[2647]: error opening /dev/hugepages/ens160@1-txrx: Permission denied
>>>>>> Aug 30 18:06:57 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@0-txrx: Permission denied
>>>>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@2-txrx: Permission denied
>>>>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@3-txrx: Permission denied
>>>>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>>>> Aug 30 18:07:01 server systemd: Started Session 6 of user root.
>>>>>> Aug 30 18:07:01 server systemd: Starting Session 6 of user root.
>>>>>>
>>>>>> The ntopng then, seems to be working but I would like to get rid of the errors. What should I do to avoid the hugepages errors? I'm running ntop as root (though I can see it drops privs).
>>>>>>
>>>>>> My conf:
>>>>>>
>>>>>> -G=/var/run/ntopng.pid
>>>>>> --data-dir=/mnt/data/ntopng
>>>>>> #--interface=zc:ens160
>>>>>> --interface=zc:ens160@0
>>>>>> --interface=zc:ens160@1
>>>>>> --interface=zc:ens160@2
>>>>>> --interface=zc:ens160@3
>>>>>> #--interface=view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3
>>>>>> --local-networks="xx.xx.0.0/16,xx.xx.cc <http://xx.xx.cc/>.0/24,xx.xx.ff.0/22,xx.xx.vv.0/24"
>>>>>> --dns-mode=3
>>>>>> --disable-autologout
>>>>>> --max-num-flows=512000
>>>>>> --max-num-hosts=256000
>>>>>> --sticky-hosts=none
>>>>>> -g -1
>>>>>>
>>>>>> # cat /etc/ntopng/ntopng.start
>>>>>> #
>>>>>>
>>>>>> # cat /etc/pf_ring/hugepages.conf
>>>>>> node=0 hugepages=1024
>>>>>> # cat /etc/pf_ring/pf_ring.conf
>>>>>> transparent_mode=2
>>>>>>
>>>>>> # cat /etc/pf_ring/pf_ring.start
>>>>>> # cat /etc/pf_ring/zc/ixgbe/ixgbe.conf
>>>>>> RSS=0,0,0,0
>>>>>> # cat /etc/pf_ring/zc/ixgbe/ixgbe.start
>>>>>> #
>>>>>>
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Spiros
>>>>>>
>>>>>> _______________________________________________
>>>>>> Ntop mailing list
>>>>>> Ntop@listgateway.unipi.it
>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Ntop mailing list
>>>>> Ntop@listgateway.unipi.it
>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop

Re: starting ntop errors and warnings [ In reply to ]

mainardi at ntop

Sep 5, 2017, 12:29 AM

Post #13 of 13 (1237 views)

Permalink

Spiros,

> On 4 Sep 2017, at 21:48, Spiros Papageorgiou <papage@noc.ntua.gr> wrote:
>
> On 4/9/2017 10:47 ??, Simone Mainardi wrote:
>> Spiros,
>>
>>> On 2 Sep 2017, at 06:43, Spiros Papageorgiou <papage@noc.ntua.gr <mailto:papage@noc.ntua.gr>> wrote:
>>>
>>> Hi Luca,
>>>
>>> I updated everything and the error messages seem to be gone. I can see that huge pages have been allocated:
>>>
>>> # cat /sys/devices/system/node/node*/meminfo | grep Huge F10Quit
>>> Node 0 AnonHugePages: 100352 kB
>>> Node 0 HugePages_Total: 2048
>>> Node 0 HugePages_Free: 1852
>>> Node 0 HugePages_Surp: 0
>>>
>>> Thanx!
>>>
>>> Two more problems:
>>> - What about the "view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3" which is not working? Is the syntax wrong or something?
>>
>> What do you mean with 'is not working'? So you get empty pages? Please, report the issue using our GitHub issue tracker.
>>
>>> - I get "02/Sep/2017 07:34:50 [Lua.cpp:5629] [@/usr/share/ntopng/scripts/callbacks/5min.lua]:[161] ERROR: Cannot complete local hosts RRD dump in 5 minutes. Please check your RRD configuration.". Do I need a stronger server? I have a 4core VM and the disk (SAN) seems not to be stressed that much. The CPU though is at about 70%.
>>
>> This means that you have too many local hosts that there's not enough time to update their statistics in 5 minutes. How many local hosts do you have? Disabling data retention of L7 timeseries may alleviate the issue.
>> The bottleneck here are the several multiple RRD updates made for every local host.
> Ntopng reports about 10K local hosts and 50K remote.
> How can i disable L7 timeseries data retention? Can I keep stats for only local hosts? What kind of control do i have on that aspect?

See the ntopng preferences page, tab "Data Retention".

> Is it possible to use somekind of caching, like rrdcache?

no

>
>>
>>> - The same goes when I export to ES. The ES is at about 30% CPU utilization but ntopng reports massive flow drops.
>>
>> ElasticSearch is not fast enough to inject flows at the required speed. Hence ntopng flows export queue grows and at some point it overflows resulting in flows drops. Other people experienced this in the past. You must configure your ES deployment to make sure it has enough intake.
>>
>>>
>>> Regards,
>>> Spiros
>>>
>>>
>>> On 1/9/2017 9:26 ??, Luca Deri wrote:
>>>> Spiros
>>>> can you please update ntopng and report?
>>>>
>>>> Thanks Luca
>>>>
>>>>> On 31 Aug 2017, at 20:10, Spiros Papageorgiou <papage@noc.ntua.gr <mailto:papage@noc.ntua.gr>> wrote:
>>>>>
>>>>> Hi Alfredo,
>>>>>
>>>>> Here is the output:
>>>>> [root@server ~]# cat /proc/mounts | grep huge
>>>>> cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0
>>>>> hugetlbfs /dev/hugepages hugetlbfs rw,relatime 0 0
>>>>>
>>>>> [root@server ~]# cat /sys/devices/system/node/node*/meminfo | grep Huge
>>>>> Node 0 AnonHugePages: 2488320 kB
>>>>> Node 0 HugePages_Total: 2048
>>>>> Node 0 HugePages_Free: 2048
>>>>> Node 0 HugePages_Surp: 0
>>>>> [root@server ~]#
>>>>>
>>>>> Regards,
>>>>> Sp
>>>>>
>>>>> On 31/8/2017 8:06 ??, Alfredo Cardigliano wrote:
>>>>>> Hi Spiros
>>>>>> please provide the output of:
>>>>>>
>>>>>> cat /proc/mounts / grep huge
>>>>>> cat /sys/devices/system/node/node*/meminfo | grep Huge
>>>>>>
>>>>>> Alfredo
>>>>>>
>>>>>>> On 31 Aug 2017, at 18:53, Spiros Papageorgiou <papage@noc.ntua.gr> <mailto:papage@noc.ntua.gr> wrote:
>>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>>
>>>>>>> I have an updated ntopng/pfring-zc installation and when I start ntopng the following messages are produced:
>>>>>>>
>>>>>>> Aug 30 18:05:06 server logger: ntopng start
>>>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2]
>>>>>>> Aug 30 18:05:08 server kernel: device ens160 entered promiscuous mode
>>>>>>> Aug 30 18:05:09 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>>> Aug 30 18:05:11 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>>> Aug 30 18:05:11 server ntopng: Starting ntopng: Unable to start ntopng[FAILED]
>>>>>>> Aug 30 18:05:11 server systemd: Started Start/stop ntopng program.
>>>>>>> Aug 30 18:05:12 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>>> Aug 30 18:05:14 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>>> Aug 30 18:05:14 server kernel: traps: ntopng[2575] general protection ip:42df01 sp:7ff418f69b60 error:0 in ntopng[400000+264000]
>>>>>>> Aug 30 18:05:15 server kernel: device ens160 left promiscuous mode
>>>>>>> Aug 30 18:05:15 server systemd: ntopng.service: main process exited, code=killed, status=11/SEGV
>>>>>>>
>>>>>>> Why the view of all queues does not work? How should i write this line?
>>>>>>>
>>>>>>> When I delete from ntopng.conf, the "view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3" line, the I get another bunch of warnings:
>>>>>>>
>>>>>>> Aug 30 18:06:50 server systemd: Starting Start/stop ntopng program...
>>>>>>> Aug 30 18:06:50 server logger: ntopng start
>>>>>>> Aug 30 18:06:51 server kernel: device ens160 entered promiscuous mode
>>>>>>> Aug 30 18:06:52 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>>> Aug 30 18:06:53 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>>> Aug 30 18:06:55 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>>> Aug 30 18:06:55 server ntopng: Starting ntopng: Unable to start ntopng[FAILED]
>>>>>>> Aug 30 18:06:55 server systemd: Started Start/stop ntopng program.
>>>>>>> Aug 30 18:06:56 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction
>>>>>>> Aug 30 18:06:57 server ZC[2647]: error opening /dev/hugepages/ens160@1-txrx: Permission denied
>>>>>>> Aug 30 18:06:57 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>>>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@0-txrx: Permission denied
>>>>>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>>>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@2-txrx: Permission denied
>>>>>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>>>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@3-txrx: Permission denied
>>>>>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB
>>>>>>> Aug 30 18:07:01 server systemd: Started Session 6 of user root.
>>>>>>> Aug 30 18:07:01 server systemd: Starting Session 6 of user root.
>>>>>>>
>>>>>>> The ntopng then, seems to be working but I would like to get rid of the errors. What should I do to avoid the hugepages errors? I'm running ntop as root (though I can see it drops privs).
>>>>>>>
>>>>>>> My conf:
>>>>>>>
>>>>>>> -G=/var/run/ntopng.pid
>>>>>>> --data-dir=/mnt/data/ntopng
>>>>>>> #--interface=zc:ens160
>>>>>>> --interface=zc:ens160@0
>>>>>>> --interface=zc:ens160@1
>>>>>>> --interface=zc:ens160@2
>>>>>>> --interface=zc:ens160@3
>>>>>>> #--interface=view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3
>>>>>>> --local-networks="xx.xx.0.0/16,xx.xx.cc <http://xx.xx.cc/>.0/24,xx.xx.ff.0/22,xx.xx.vv.0/24"
>>>>>>> --dns-mode=3
>>>>>>> --disable-autologout
>>>>>>> --max-num-flows=512000
>>>>>>> --max-num-hosts=256000
>>>>>>> --sticky-hosts=none
>>>>>>> -g -1
>>>>>>>
>>>>>>> # cat /etc/ntopng/ntopng.start
>>>>>>> #
>>>>>>>
>>>>>>> # cat /etc/pf_ring/hugepages.conf
>>>>>>> node=0 hugepages=1024
>>>>>>> # cat /etc/pf_ring/pf_ring.conf
>>>>>>> transparent_mode=2
>>>>>>>
>>>>>>> # cat /etc/pf_ring/pf_ring.start
>>>>>>> # cat /etc/pf_ring/zc/ixgbe/ixgbe.conf
>>>>>>> RSS=0,0,0,0
>>>>>>> # cat /etc/pf_ring/zc/ixgbe/ixgbe.start
>>>>>>> #
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Spiros
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Ntop mailing list
>>>>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Ntop mailing list
>>>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>>> _______________________________________________
>>>>> Ntop mailing list
>>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop

Mailing List Archive

Attached Files:

Attached Files: