Mailing List Archive

Some questions
Hello,

I would like to set up Ntopng as a NetFlow collector to monitor a network of virtual machines. I used VirtualBox to create the network (see enclosed image "Network.png"). My goal is to display NetFlow data relative to the flows passing through a Cisco CSR1000v virtual router. I have installed Ntopng in a CentOS 7 VM in Community mode. The load generators are used to simulate traffic through the CSR with Iperf.

The Flexible NetFlow configuration in the CSR is as follows:
flow record netflow-record
collect application name
collect connection initiator
collect connection client counter bytes network long
collect connection client counter packets long
collect connection client ipv4 address
collect connection client transport port
collect connection server counter bytes network long
collect connection server counter packets long
collect connection server ipv4 address
collect connection server transport port
collect counter bytes long
collect counter bytes layer2 long
collect counter packets long
collect datalink source-vlan-id
collect datalink destination-vlan-id
collect datalink mac source address input
collect datalink mac source address output
collect datalink mac destination address input
collect datalink mac destination address output
match flow direction
match interface input
match interface output
match ipv4 source address
collect ipv4 source mask
collect ipv4 source prefix
match ipv4 destination address
collect ipv4 destination mask
collect ipv4 destination prefix
match ipv4 protocol
collect timestamp absolute first
collect timestamp absolute last
match transport source-port
match transport destination-port
collect transport tcp source-port
collect transport tcp destination-port
collect transport udp source-port
collect transport udp destination-port

flow exporter ntopng
destination 10.0.0.9
source GigabitEthernet 1
transport udp 2055

flow monitor netflow-monitor
exporter ntopng
record netflow-record
cache timeout active 30
cache timeout inactive 10

interface GigabitEthernet 1
ip flow monitor netflow-monitor input

interface GigabitEthernet 2
ip flow monitor netflow-monitor input
To collect NetFlow data, the configuration of nProbe in /etc/nprobe/nprobe.conf is the following:
-i none
-n none
-3 2055
--zmq "tcp://*:5556"
-T %IN_BYTES %OUT_BYTES %IN_PKTS %OUT_PKTS %L4_SRC_PORT %L4_DST_PORT %L4_SRC_PORT_MAP %L4_DST_PORT_MAP %L4_SRV_PORT %L4_SRV_PORT_MAP %IPV4_SRC_ADDR %IPV4_DST_ADDR %IPV4_SRC_MASK %IPV4_DST_MASK %TOTAL_BYTES_EXP %TOTAL_PKTS_EXP %TOTAL_FLOWS_EXP %IN_SRC_MAC %OUT_DST_MAC %SRC_VLAN %DST_VLAN %INPUT_SNMP %OUTPUT_SNMP %IP_PROTOCOL_VERSION %PROTOCOL %PROTOCOL_MAP %DIRECTION %APPLICATION_ID %APPLICATION_NAME %EXPORTER_IPV4_ADDRESS %FLOW_START_SEC %FLOW_END_SEC %PACKETS_OBSERVED

That of Ntopng in /etc/ntopng/ntopng.conf is:
--interface="tcp://127.0.0.1:5556"
-m "10.0.0.0/30,10.0.0.4/30"

With these configurations, I get some NetFlow data in Ntopng. The Timeseries graphs work fine and I can see the number of bits/s correctly for the Hosts, Networks and the tcp://127.0.0.1:5556 interface. But I would rather plot data every minute than with a 5 minute interval. Is it possible?

My main problem is that I don't manage to see data relative to packets, ports and protocols for the hosts:

- I can sometimes see the client ports of the Iperf traffic (see enclosed image "Client ports.png") for about 2s. But most of the time, information disappears for minutes and only "100% Other" is displayed for Client and Server ports (see enclosed image "Ports.png"), as if NetFlow collected data is rarely understood by Ntopng.

- The data relative to protocols is never processed by Ntopng, the legends "Other", "Unknown", "Unrated" or "Unspecified" are displayed (see enclosed image "Protocol overviews.png"). The same problem appears for the section Protocols of the tcp://127.0.0.1:5556 interface.

- I cannot see data in the Packets section at all, the Sent and Received distributions are always empty (see enclosed image "Packets.png")

Could you explain to me what I should do to solve these issues please?

Moreover, I would like to have the NetFlow data in Grafana by means of its Ntopng plugin.
Referring to Grafana plugin presentation, I would be able to plot:
Interface Metrics:

- Traffic rates, in bits and packets per second

- Traffic totals, both in Bytes and packets

- Application protocol rates, in bits per second
Host Metrics:

- Traffic rate in bits per second

- Traffic total in Bytes

- Application protocol rates in bits per second.

However, I currently manage to visualize only Traffic rate in bits per second for the hosts (host_10.0.0.1_interface_tcp://127.0.0.1:5556_traffic_bps and host_10.0.0.5_interface_tcp://127.0.0.5:5556_traffic_bps).
For host_10.0.0.X_interface_tcp://127.0.0.1:5556_traffic_total_bytes, the graph displays "Data points outside time range".
For host_10.0.0.X_interface_tcp://127.0.0.1:5556_allprotocols_bps and host_10.0.0.X_interface_tcp://127.0.0.1:5556_allcategories_bps, the graphs display "No data points".

Besides, no interface metric is available.
Do these issues come from Ntopng or Grafana? And how can they be solved please?

Thank you very much in advance for answering my questions.

Best regards,

Arnaud POLOSSAT
Airbus Defence and Space
1, Bvd Jean Moulin, CS 40001
78 996 Elancourt Cedex, France
E-mail: arnaud.polossat@fr.airbus.com<mailto:arnaud.polossat@fr.airbus.com>


***************************************************************
Ce courriel (incluant ses eventuelles pieces jointes) peut contenir des informations confidentielles et/ou protegees ou dont la diffusion est restreinte. Si vous avez recu ce courriel par erreur, vous ne devez ni le copier, ni l'utiliser, ni en divulguer le contenu a quiconque. Merci d'en avertir immediatement l'expediteur et d'effacer ce courriel de votre systeme. Airbus Defence and Space et les sociétés Airbus Group declinent toute responsabilite en cas de corruption par virus, d'alteration ou de falsification de ce courriel lors de sa transmission par voie electronique.
This email (including any attachments) may contain confidential and/or privileged information or information otherwise protected from disclosure. If you are not the intended recipient, please notify the sender immediately, do not copy this message or any attachments and do not use it for any purpose or disclose its content to any person, but delete this message and any attachments from your system. Airbus Defence and Space and Airbus Group companies disclaim any and all liability if this email transmission was virus corrupted, altered or falsified.
---------------------------------------------------------------------
Airbus Defence and Space SAS (393 341 516 RCS Toulouse) - Capital: 29.821.072 EUR - Siege social: 31 rue des Cosmonautes, ZI du Palays, 31402 Toulouse cedex 4, France