Mailing List Archive

Hi Frederic,

The latest ntopng dev build allows you to create virtual interfaces on the
basis of the ingress flow interface. You just have to visit the preferences
page (Expert View), tab "Network Interfaces", and enable "Ingress Flow
Interfaces". This should do the trick.

Regards,
Simone

On Thu, Jan 12, 2017 at 2:06 PM, Frederic Hermann <fhe-ntop@neptune.fr>
wrote:

> Hi guys,
>
> I have a running nprobe+ntopng server, collecting netflow from several
> sources.
>
> My collectors are routers connected to different Peers / Transit providers
> on different interfaces.
>
> Currently, each source send a global flow to my nprobe server, and each
> nprobe instance expose data to ntopng using different ZMQ endpoints.
>
> So I have traffic statistics by router.
>
> Now, I would like to get statistics by peer/provider, and not just by
> router.
> So I would like to have have one zmq interface by peer/provider in ntopng.
>
> To achieve this, I need to be able to filter the collected flow using the
> interface on the source router, but I cannot find the proper way to do
> this, as it seems I cannot use BPF filters in collector mode.
>
>
> Here is the way I try to make it :
>
>
> Peer1;if1 -- zmq
> localhost:5556
> \ /
> \
> Router1 --- (netflow v5 udp:1234) ---> nprobe
> ntopng
> / \
> /
> Peer2:if2 -- zmq
> localhost;5557
>
>
> Do you know how I could achieve this ?
>
> Thaks & regards
>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>

> Hi Frederic,
> The latest ntopng dev build allows you to create virtual interfaces on the basis
> of the ingress flow interface. You just have to visit the preferences page
> (Expert View), tab "Network Interfaces", and enable "Ingress Flow Interfaces".
> This should do the trick.

Thanks Simone,

It seems almost to work - but I have the following error message in ntopng.log :

Ntop.cpp:1276] ERROR: Too many interfaces defined

My netflow routers have many interfaces (physical, vlan and tunnel one) : How many can ntopng actually manage ?

Also, it seems I cannot provide a Custom name for those virtual interfaces, is that correct?

Finally I can see in ntopng interface list ifIdx that are not existing on my router. Maybe it is related to the same error message ?

eg :
on ntopng I have the following interfaces (and many others) :
tcp://127.0.0.1:5558 [ifIdx: 0]
tcp://127.0.0.1:5558 [ifIdx: 903]

Those 2 interface numbers are not related to ifIndex on my router, which is:


IF-MIB::ifIndex.1 = INTEGER: 1
IF-MIB::ifIndex.2 = INTEGER: 2
IF-MIB::ifIndex.3 = INTEGER: 3
IF-MIB::ifIndex.4 = INTEGER: 4
IF-MIB::ifIndex.5 = INTEGER: 5
IF-MIB::ifIndex.6 = INTEGER: 6
IF-MIB::ifIndex.7 = INTEGER: 7
IF-MIB::ifIndex.8 = INTEGER: 8
IF-MIB::ifIndex.9 = INTEGER: 9
IF-MIB::ifIndex.10 = INTEGER: 10
IF-MIB::ifIndex.11 = INTEGER: 11
IF-MIB::ifIndex.12 = INTEGER: 12
IF-MIB::ifIndex.13 = INTEGER: 13
IF-MIB::ifIndex.14 = INTEGER: 14
IF-MIB::ifIndex.15 = INTEGER: 15
IF-MIB::ifIndex.16 = INTEGER: 16
IF-MIB::ifIndex.21 = INTEGER: 21
IF-MIB::ifIndex.23 = INTEGER: 23
IF-MIB::ifIndex.30 = INTEGER: 30
IF-MIB::ifIndex.31 = INTEGER: 31
IF-MIB::ifIndex.32 = INTEGER: 32
IF-MIB::ifIndex.33 = INTEGER: 33
IF-MIB::ifIndex.40 = INTEGER: 40
IF-MIB::ifIndex.41 = INTEGER: 41
IF-MIB::ifIndex.42 = INTEGER: 42
IF-MIB::ifIndex.43 = INTEGER: 43
IF-MIB::ifIndex.44 = INTEGER: 44
IF-MIB::ifIndex.51 = INTEGER: 51
IF-MIB::ifIndex.52 = INTEGER: 52
IF-MIB::ifIndex.53 = INTEGER: 53
IF-MIB::ifIndex.54 = INTEGER: 54
IF-MIB::ifIndex.55 = INTEGER: 55
IF-MIB::ifIndex.56 = INTEGER: 56
IF-MIB::ifIndex.57 = INTEGER: 57
IF-MIB::ifIndex.58 = INTEGER: 58
IF-MIB::ifIndex.59 = INTEGER: 59
IF-MIB::ifIndex.60 = INTEGER: 60
IF-MIB::ifIndex.61 = INTEGER: 61
IF-MIB::ifIndex.62 = INTEGER: 62
IF-MIB::ifIndex.63 = INTEGER: 63
IF-MIB::ifIndex.64 = INTEGER: 64
IF-MIB::ifIndex.65 = INTEGER: 65
IF-MIB::ifIndex.66 = INTEGER: 66
IF-MIB::ifIndex.67 = INTEGER: 67
IF-MIB::ifIndex.68 = INTEGER: 68
IF-MIB::ifIndex.69 = INTEGER: 69
IF-MIB::ifIndex.70 = INTEGER: 70
IF-MIB::ifIndex.71 = INTEGER: 71
IF-MIB::ifIndex.72 = INTEGER: 72
IF-MIB::ifIndex.73 = INTEGER: 73
IF-MIB::ifIndex.74 = INTEGER: 74
IF-MIB::ifIndex.75 = INTEGER: 75
IF-MIB::ifIndex.76 = INTEGER: 76
IF-MIB::ifIndex.77 = INTEGER: 77
IF-MIB::ifIndex.78 = INTEGER: 78
IF-MIB::ifIndex.79 = INTEGER: 79
IF-MIB::ifIndex.80 = INTEGER: 80
IF-MIB::ifIndex.82 = INTEGER: 82
IF-MIB::ifIndex.86 = INTEGER: 86
IF-MIB::ifIndex.87 = INTEGER: 87
IF-MIB::ifIndex.90 = INTEGER: 90
IF-MIB::ifIndex.98 = INTEGER: 98
IF-MIB::ifIndex.99 = INTEGER: 99
IF-MIB::ifIndex.101 = INTEGER: 101
IF-MIB::ifIndex.102 = INTEGER: 102
IF-MIB::ifIndex.15729529 = INTEGER: 15729529
IF-MIB::ifIndex.15729538 = INTEGER: 15729538
IF-MIB::ifIndex.15729541 = INTEGER: 15729541
IF-MIB::ifIndex.15729543 = INTEGER: 15729543



_______________________________________________
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

> My netflow routers have many interfaces (physical, vlan and tunnel one) : How
> many can ntopng actually manage ?

Seems that the current max is 32 interfaces for ntopng.

The problem is : even if I try to limit the number of interfaces to send flows about on my netflow routers to 6 each, I still have the following in the ntopng logs:

18/Jan/2017 15:00:05 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 23] [id: 1]
18/Jan/2017 15:00:05 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 11] [id: 2]
18/Jan/2017 15:00:05 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 24] [id: 3]
18/Jan/2017 15:00:06 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 79] [id: 4]
18/Jan/2017 15:00:06 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 74] [id: 5]
18/Jan/2017 15:00:06 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 53] [id: 6]
18/Jan/2017 15:00:06 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 52] [id: 7]
18/Jan/2017 15:00:06 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 62] [id: 8]
18/Jan/2017 15:00:07 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 60] [id: 9]
18/Jan/2017 15:00:07 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 87] [id: 10]
18/Jan/2017 15:00:07 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 21] [id: 11]
18/Jan/2017 15:00:07 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 43] [id: 12]
18/Jan/2017 15:00:08 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 77] [id: 13]
18/Jan/2017 15:00:08 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 56] [id: 14]
18/Jan/2017 15:00:08 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 76] [id: 15]
18/Jan/2017 15:00:08 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 72] [id: 16]
18/Jan/2017 15:00:09 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 80] [id: 17]
18/Jan/2017 15:00:09 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 40] [id: 18]
18/Jan/2017 15:00:09 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 51] [id: 19]
18/Jan/2017 15:00:09 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 99] [id: 20]
18/Jan/2017 15:00:09 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 82] [id: 21]
18/Jan/2017 15:00:10 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 73] [id: 22]
18/Jan/2017 15:00:10 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 86] [id: 23]
18/Jan/2017 15:00:10 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 61] [id: 24]
18/Jan/2017 15:00:10 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 66] [id: 25]
18/Jan/2017 15:00:11 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 0] [id: 26]
18/Jan/2017 15:00:11 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 90] [id: 27]
18/Jan/2017 15:00:11 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 64] [id: 28]
18/Jan/2017 15:00:11 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 15] [id: 29]
18/Jan/2017 15:00:12 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 44] [id: 30]
18/Jan/2017 15:00:12 [Ntop.cpp:1271] Registered interface tcp://127.0.0.1:5556 [ifIdx: 78] [id: 31]
18/Jan/2017 15:00:12 [Ntop.cpp:1276] ERROR: Too many interfaces defined

I'm still wondering what the ifIdx 0 is for ?
_______________________________________________
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

> Hi Frederic,
> The latest ntopng dev build allows you to create virtual interfaces on the basis
> of the ingress flow interface. You just have to visit the preferences page
> (Expert View), tab "Network Interfaces", and enable "Ingress Flow Interfaces".
> This should do the trick.

Hi Simone,

I opened the following issue on Github, as apparently this feature doesn't work correctly (on my installation ).
Basically, when I enable this, I cannot see any active flow in ntopng, on any interface (real or virtual)

cf https://github.com/ntop/ntopng/issues/936 for full report.

Regards,

FH
_______________________________________________
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Hi, I tried to reproduce but everything works as expected. Full reply is on
the issue page. I encourage you to update both ntopng and nprobe to their
latest versions

On Thu, Jan 19, 2017 at 2:30 PM, Frederic Hermann <fhe-ntop@neptune.fr>
wrote:

>
>
>
> > Hi Frederic,
> > The latest ntopng dev build allows you to create virtual interfaces on
> the basis
> > of the ingress flow interface. You just have to visit the preferences
> page
> > (Expert View), tab "Network Interfaces", and enable "Ingress Flow
> Interfaces".
> > This should do the trick.
>
> Hi Simone,
>
> I opened the following issue on Github, as apparently this feature doesn't
> work correctly (on my installation ).
> Basically, when I enable this, I cannot see any active flow in ntopng, on
> any interface (real or virtual)
>
> cf https://github.com/ntop/ntopng/issues/936 for full report.
>
> Regards,
>
> FH
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>

Hi Simone,

After your message, I did try again, from scratch : I completely removed ntopng install to start again, wiped out redis completely by stopping redis-server and remove /var/lib/redis/dump.rdb,
and reinstalled everything.
Now is seems to work ok when ntopng is configured in collector mode.

I suppose that my first attemps failed because I first configured ntopng and nprobe in standard mode (no --zmq-probe-mode in nprobe, no 'c' in ntopn interface name)

For the record, There is my working configuration :

* My router (mikrotik) send flow to nprobe/ntop server on port 1236

* nprobe configuration:
-n=none -G= --online-license-check -g=/var/run/nprobe.pid --collector-port=1236 --zmq=tcp://127.0.0.1:5560 --zmq-probe-mode

* ntopng configuration:
-G=/var/run/ntopng.pid -n=3 -i=tcp://127.0.0.1:5560c -d=/home/ntopng --daemon= -w=3000 --user=ntopng --group=ntopng --local-networks=100.64.0.0/10,10.0.0.0/8,192.168.0.0/16,172.16.0.0/12

I juqr closed the issue.

Thanks.


----- Mail original -----
> De: "Simone Mainardi" <mainardi@ntop.org>
> À: ntop-misc@listgateway.unipi.it, "Frederic Hermann" <fhe-ntop@neptune.fr>
> Envoyé: Mardi 24 Janvier 2017 11:41:25
> Objet: Re: [Ntop-misc] How to get separate traffic statistic by collector interface

> Hi, I tried to reproduce but everything works as expected. Full reply is on the
> issue page. I encourage you to update both ntopng and nprobe to their latest
> versions
_______________________________________________
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Hello Simone,

When we use this mode, the Traffic Report is done by interface (which is expected, and great).

However, is there any way to get a global report, in addition to the report by interface ? That would make sense for the Top Remote Hosts and Top ASNs, for exemple, to have both informations (global and by interface only).


Regards


> De: "Simone Mainardi" <mainardi@ntop.org>
> À: ntop-misc@listgateway.unipi.it, "Frederic Hermann" <fhe-ntop@neptune.fr>
> Envoyé: Vendredi 13 Janvier 2017 21:09:22
> Objet: Re: [Ntop-misc] How to get separate traffic statistic by collector interface

> Hi Frederic,
> The latest ntopng dev build allows you to create virtual interfaces on the basis
> of the ingress flow interface. You just have to visit the preferences page
> (Expert View), tab "Network Interfaces", and enable "Ingress Flow Interfaces".
> This should do the trick.

_______________________________________________
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Dear Friedrich,


On Wed, Jan 25, 2017 at 11:03 AM, Frederic Hermann <fhe-ntop@neptune.fr>
wrote:

> Hello Simone,
>
> When we use this mode, the Traffic Report is done by interface (which is
> expected, and great).
>
> However, is there any way to get a global report, in addition to the
> report by interface ? That would make sense for the Top Remote Hosts and
> Top ASNs, for exemple, to have both informations (global and by interface
> only).
>

If you use normal interfaces, you can combine them using a view (e.g., -i
eth0 -i eth1 -i view:eth0,eth1) . However, views aren't available for
dynamically created interfaces.

Regards,
Simone


>
>
> Regards
>
>
> > De: "Simone Mainardi" <mainardi@ntop.org>
> > À: ntop-misc@listgateway.unipi.it, "Frederic Hermann" <
> fhe-ntop@neptune.fr>
> > Envoyé: Vendredi 13 Janvier 2017 21:09:22
> > Objet: Re: [Ntop-misc] How to get separate traffic statistic by
> collector interface
>
> > Hi Frederic,
> > The latest ntopng dev build allows you to create virtual interfaces on
> the basis
> > of the ingress flow interface. You just have to visit the preferences
> page
> > (Expert View), tab "Network Interfaces", and enable "Ingress Flow
> Interfaces".
> > This should do the trick.
>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>

Hi Simone,


>> However, is there any way to get a global report, in addition to the report by
>> interface ? That would make sense for the Top Remote Hosts and Top ASNs, for
>> exemple, to have both informations (global and by interface only).

> If you use normal interfaces, you can combine them using a view (e.g., -i eth0
> -i eth1 -i view:eth0,eth1) . However, views aren't available for dynamically
> created interfaces.

Is this working with zmq interfaces ?
ie would : -i view:tcp://127.0.0.1:5559c,tcp://127.0.0.1:5560c
working ?

Regards

_______________________________________________
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Hi,

On Thu, Jan 26, 2017 at 4:15 PM, Frederic Hermann <fhe-ntop@neptune.fr>
wrote:

> Hi Simone,
>
>
> >> However, is there any way to get a global report, in addition to the
> report by
> >> interface ? That would make sense for the Top Remote Hosts and Top
> ASNs, for
> >> exemple, to have both informations (global and by interface only).
>
> > If you use normal interfaces, you can combine them using a view (e.g.,
> -i eth0
> > -i eth1 -i view:eth0,eth1) . However, views aren't available for
> dynamically
> > created interfaces.
>
> Is this working with zmq interfaces ?
> ie would : -i view:tcp://127.0.0.1:5559c,tcp://127.0.0.1:5560c
>

Yes, but not if you create dynamic interfaces (e.g., with the remote probe
address or the remote interface index) out of the zmq interfaces.


> working ?
>
> Regards
>
>