Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. NTop>
  3. Users
Re: nprobe to ntop missing flows
davidlarson at livingstonesa
Sep 4, 2018, 8:31 AM
Post #1 of 4 (729 views)
Permalink
Hello!
I am trying to collect netflow from a cisco 3850 and view it in ntopng. I
am able to see some data, but it appears intermittent. When I view nprobe
with -b 2 option it seems like the flows pause every few seconds then
resume. The real time traffic in ntopng shows traffic, then nothing,
traffic, then nothing. Am I missing a collection rate config or something?

ntopng enterprise and nprobe pro on same machine

ntopng conf
-w=3000
-W=0
-g=-1
-m="<mylocalnetwork>"
-d=/storage/ntopng
-G=/var/run/ntopng.pid
-i=tcp://127.0.0.1:5666
--online-license-check


nprobe -n none --zmq "tcp://*:5666" --collector-port 2166 -T @NTOPNG@
--disable-cache



04/Sep/2018 11:15:39 [nprobe.c:3297] L7 Proto Diff
Total
04/Sep/2018 11:15:39 [nprobe.c:3311] Unknown/0 303.55 MB
741.98 MB
04/Sep/2018 11:15:39 [nprobe.c:3319] Current flow export rate: [131.8
flows/sec]
04/Sep/2018 11:15:39 [nprobe.c:3322] Flow drops: [export queue too
long=0][too many flows=0][ELK queue flow drops=0]
04/Sep/2018 11:15:39 [nprobe.c:3327] Export Queue: 0/512000 [0.0 %]
04/Sep/2018 11:15:39 [nprobe.c:3332] Flow Buckets:
[active=0][allocated=0][toBeExported=0]
04/Sep/2018 11:15:39 [nprobe.c:3369] Collector Threads: [871 pkts@0]
04/Sep/2018 11:15:39 [nprobe.c:3108] Processed packets: 0 (max bucket
search: 5)
04/Sep/2018 11:15:39 [nprobe.c:3091] Fragment queue length: 0
04/Sep/2018 11:15:39 [nprobe.c:3118] Flow collection stats: [collected
pkts: 871][processed flows: 19898]
04/Sep/2018 11:15:39 [nprobe.c:3121] Flow export stats: [0 bytes/0
pkts][0 flows/0 pkts sent]
04/Sep/2018 11:15:39 [nprobe.c:3127] Flow export drop stats: [0 bytes/0
pkts][0 flows]
04/Sep/2018 11:15:39 [nprobe.c:3132] Total flow stats: [0 bytes/0
pkts][0 flows/0 pkts sent]
Re: nprobe to ntop missing flows [ In reply to ]
faranda at ntop
Sep 7, 2018, 1:07 AM
Post #2 of 4 (727 views)
Permalink
Hi David,

Please try to add also --zmq-disable-buffering to the nprobe options.

Emanuele

On 9/4/18 5:31 PM, David Larson wrote:
> Hello!
> I am trying to collect netflow from a cisco 3850 and view it in
> ntopng. I am able to see some data, but it appears intermittent. When
> I view nprobe with -b 2 option it seems like the flows pause every few
> seconds then resume. The real time traffic in ntopng shows traffic,
> then nothing, traffic, then nothing. Am I missing a collection rate
> config or something?
>
> ntopng enterprise and nprobe pro on same machine
>
> ntopng conf
> -w=3000
> -W=0
> -g=-1
> -m="<mylocalnetwork>"
> -d=/storage/ntopng
> -G=/var/run/ntopng.pid
> -i=tcp://127.0.0.1:5666 <http://127.0.0.1:5666>
> --online-license-check
>
>
> nprobe -n none --zmq "tcp://*:5666" --collector-port 2166 -T @NTOPNG@
> --disable-cache
>
>
>
> 04/Sep/2018 11:15:39 [nprobe.c:3297] L7 Proto            Diff      Total
> 04/Sep/2018 11:15:39 [nprobe.c:3311]    Unknown/0          303.55 MB 
> 741.98 MB
> 04/Sep/2018 11:15:39 [nprobe.c:3319] Current flow export rate: [131.8
> flows/sec]
> 04/Sep/2018 11:15:39 [nprobe.c:3322] Flow drops: [export queue too
> long=0][too many flows=0][ELK queue flow drops=0]
> 04/Sep/2018 11:15:39 [nprobe.c:3327] Export Queue: 0/512000 [0.0 %]
> 04/Sep/2018 11:15:39 [nprobe.c:3332] Flow Buckets:
> [active=0][allocated=0][toBeExported=0]
> 04/Sep/2018 11:15:39 [nprobe.c:3369] Collector Threads: [871 pkts@0]
> 04/Sep/2018 11:15:39 [nprobe.c:3108] Processed packets: 0 (max bucket
> search: 5)
> 04/Sep/2018 11:15:39 [nprobe.c:3091] Fragment queue length: 0
> 04/Sep/2018 11:15:39 [nprobe.c:3118] Flow collection stats: 
> [collected pkts: 871][processed flows: 19898]
> 04/Sep/2018 11:15:39 [nprobe.c:3121] Flow export stats:      [0
> bytes/0 pkts][0 flows/0 pkts sent]
> 04/Sep/2018 11:15:39 [nprobe.c:3127] Flow export drop stats: [0
> bytes/0 pkts][0 flows]
> 04/Sep/2018 11:15:39 [nprobe.c:3132] Total flow stats:       [0
> bytes/0 pkts][0 flows/0 pkts sent]
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
Re: nprobe to ntop missing flows [ In reply to ]
mainardi at ntop
Sep 7, 2018, 1:14 AM
Post #3 of 4 (727 views)
Permalink
Also make sure your Cisco is exporting a steady rate.


You can check this by monitoring the output of nprobe with -b 2. In particular, look at

04/Sep/2018 11:15:39 [nprobe.c:3118] Flow collection stats: [collected pkts: 871]

You should see it increasing continuously.


Simone

> On 7 Sep 2018, at 10:07, Emanuele Faranda <faranda@ntop.org> wrote:
>
> Hi David,
>
> Please try to add also --zmq-disable-buffering to the nprobe options.
>
> Emanuele
> On 9/4/18 5:31 PM, David Larson wrote:
>> Hello!
>> I am trying to collect netflow from a cisco 3850 and view it in ntopng. I am able to see some data, but it appears intermittent. When I view nprobe with -b 2 option it seems like the flows pause every few seconds then resume. The real time traffic in ntopng shows traffic, then nothing, traffic, then nothing. Am I missing a collection rate config or something?
>>
>> ntopng enterprise and nprobe pro on same machine
>>
>> ntopng conf
>> -w=3000
>> -W=0
>> -g=-1
>> -m="<mylocalnetwork>"
>> -d=/storage/ntopng
>> -G=/var/run/ntopng.pid
>> -i=tcp://127.0.0.1:5666 <http://127.0.0.1:5666/>
>> --online-license-check
>>
>>
>> nprobe -n none --zmq "tcp://*:5666" --collector-port 2166 -T @NTOPNG@ --disable-cache
>>
>>
>>
>> 04/Sep/2018 11:15:39 [nprobe.c:3297] L7 Proto Diff Total
>> 04/Sep/2018 11:15:39 [nprobe.c:3311] Unknown/0 303.55 MB 741.98 MB
>> 04/Sep/2018 11:15:39 [nprobe.c:3319] Current flow export rate: [131.8 flows/sec]
>> 04/Sep/2018 11:15:39 [nprobe.c:3322] Flow drops: [export queue too long=0][too many flows=0][ELK queue flow drops=0]
>> 04/Sep/2018 11:15:39 [nprobe.c:3327] Export Queue: 0/512000 [0.0 %]
>> 04/Sep/2018 11:15:39 [nprobe.c:3332] Flow Buckets: [active=0][allocated=0][toBeExported=0]
>> 04/Sep/2018 11:15:39 [nprobe.c:3369] Collector Threads: [871 pkts@0]
>> 04/Sep/2018 11:15:39 [nprobe.c:3108] Processed packets: 0 (max bucket search: 5)
>> 04/Sep/2018 11:15:39 [nprobe.c:3091] Fragment queue length: 0
>> 04/Sep/2018 11:15:39 [nprobe.c:3118] Flow collection stats: [collected pkts: 871][processed flows: 19898]
>> 04/Sep/2018 11:15:39 [nprobe.c:3121] Flow export stats: [0 bytes/0 pkts][0 flows/0 pkts sent]
>> 04/Sep/2018 11:15:39 [nprobe.c:3127] Flow export drop stats: [0 bytes/0 pkts][0 flows]
>> 04/Sep/2018 11:15:39 [nprobe.c:3132] Total flow stats: [0 bytes/0 pkts][0 flows/0 pkts sent]
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>_______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
Re: nprobe to ntop missing flows [ In reply to ]
mainardi at ntop
Sep 7, 2018, 1:14 AM
Post #4 of 4 (727 views)
Permalink
Also make sure your Cisco is exporting a steady rate.


You can check this by monitoring the output of nprobe with -b 2. In particular, look at

04/Sep/2018 11:15:39 [nprobe.c:3118] Flow collection stats: [collected pkts: 871]

You should see it increasing continuously.


Simone

> On 7 Sep 2018, at 10:07, Emanuele Faranda <faranda@ntop.org> wrote:
>
> Hi David,
>
> Please try to add also --zmq-disable-buffering to the nprobe options.
>
> Emanuele
> On 9/4/18 5:31 PM, David Larson wrote:
>> Hello!
>> I am trying to collect netflow from a cisco 3850 and view it in ntopng. I am able to see some data, but it appears intermittent. When I view nprobe with -b 2 option it seems like the flows pause every few seconds then resume. The real time traffic in ntopng shows traffic, then nothing, traffic, then nothing. Am I missing a collection rate config or something?
>>
>> ntopng enterprise and nprobe pro on same machine
>>
>> ntopng conf
>> -w=3000
>> -W=0
>> -g=-1
>> -m="<mylocalnetwork>"
>> -d=/storage/ntopng
>> -G=/var/run/ntopng.pid
>> -i=tcp://127.0.0.1:5666 <http://127.0.0.1:5666/>
>> --online-license-check
>>
>>
>> nprobe -n none --zmq "tcp://*:5666" --collector-port 2166 -T @NTOPNG@ --disable-cache
>>
>>
>>
>> 04/Sep/2018 11:15:39 [nprobe.c:3297] L7 Proto Diff Total
>> 04/Sep/2018 11:15:39 [nprobe.c:3311] Unknown/0 303.55 MB 741.98 MB
>> 04/Sep/2018 11:15:39 [nprobe.c:3319] Current flow export rate: [131.8 flows/sec]
>> 04/Sep/2018 11:15:39 [nprobe.c:3322] Flow drops: [export queue too long=0][too many flows=0][ELK queue flow drops=0]
>> 04/Sep/2018 11:15:39 [nprobe.c:3327] Export Queue: 0/512000 [0.0 %]
>> 04/Sep/2018 11:15:39 [nprobe.c:3332] Flow Buckets: [active=0][allocated=0][toBeExported=0]
>> 04/Sep/2018 11:15:39 [nprobe.c:3369] Collector Threads: [871 pkts@0]
>> 04/Sep/2018 11:15:39 [nprobe.c:3108] Processed packets: 0 (max bucket search: 5)
>> 04/Sep/2018 11:15:39 [nprobe.c:3091] Fragment queue length: 0
>> 04/Sep/2018 11:15:39 [nprobe.c:3118] Flow collection stats: [collected pkts: 871][processed flows: 19898]
>> 04/Sep/2018 11:15:39 [nprobe.c:3121] Flow export stats: [0 bytes/0 pkts][0 flows/0 pkts sent]
>> 04/Sep/2018 11:15:39 [nprobe.c:3127] Flow export drop stats: [0 bytes/0 pkts][0 flows]
>> 04/Sep/2018 11:15:39 [nprobe.c:3132] Total flow stats: [0 bytes/0 pkts][0 flows/0 pkts sent]
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>_______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal