Mailing List Archive: TX/RX display always identical.

TX/RX display always identical.

Feb 8, 2017, 7:29 AM

Post #1 of 9 (594 views)

Hello

I have ntopng with nProbe setup on a Ubuntu16 box, the full nBox setup from
the packages.ntop.org repo. I am exporting sflow data from a Fortigate 60D
(OS 5.4.3) to nProbe.

The problem I am having is the Local / Remote traffic is being reported as
the same amount / flow speed. Infact the Ingress and Egress is always
displayed as exactly half of the total throughput at that time. This is
true for the little widget at the bottom next to the rev counter for
ingress and egress and on the home page of a host, when clicked the "Sent
vs Received Traffic Breakdown" is always a perfect 50/50 ratio.

If I export flow data from a Mikrotik on a different network every thing
reports correctly.

What is a little unique on this network is that there are a few /26 subnets
of public IP Addresses behind this firewall. There is no natting. I have
set these subnets as local subnets in ntopng as you can tell from my config
below. The firewall on the WAN side has a public address and a few public
subnets on the LAN side. Would this cause issues with remote/local traffic
differentiation?

I have tried setting V5/V9 etc flow types same issue. I have enabled just
RX or TX from the Fortigate and these when individually enabled display
correctly.

Any help or pointers would be appreciated.

My configs:

root@ntopng:~# cat /etc/ntopng/ntopng.conf
-n=3
-w=3000
-W=0
-g=-1
-m="41.xx.xx.0/26,196.x.x.x/26"
-F=mysql;localhost;ntopng;flows;ntopuser;secretxxx
-d=/storage/ntopng
-G=/var/run/ntopng.pid
-i=tcp://127.0.0.1:5556

root@ntopng:~# cat /etc/nprobe/nprobe-ens18.conf
-n=none
-i=none
-3=2055
-s=128
-t=60
-d=60
-a=0
-e=1
-B=10
-w=128000
-z=0
-S=1:1
-E=0:0
-g=/var/run/nprobe-ens18.pid
--zmq=tcp://127.0.0.1:5556
-V=5
--dump-stats=/var/log/nprobe/ens18-0_flows_stats.txt

The fortigate was configured with the instructions here:
http://kb.fortinet.com/kb/documentLink.do?externalID=FD36460

Thanks and Regards
Jean-Pierre Human

Re: TX/RX display always identical. [ In reply to ]

mainardi at ntop

Feb 8, 2017, 7:49 AM

Post #2 of 9 (594 views)

Permalink

Hi,

If all the hosts seen fall in local networks, then you will see identical
counters local2remote and remote2local. Indeed, the same amount of traffic
is counted as egress (i.e., from a local network) and as ingress (i.e., to
a local network).

Can you please verify if flow src and dst are always in local networks.

Thanks

On Wed, Feb 8, 2017 at 4:29 PM, Jean-Pierre Human <jphuman@gmail.com> wrote:

> Hello
>
> I have ntopng with nProbe setup on a Ubuntu16 box, the full nBox setup
> from the packages.ntop.org repo. I am exporting sflow data from a
> Fortigate 60D (OS 5.4.3) to nProbe.
>
> The problem I am having is the Local / Remote traffic is being reported as
> the same amount / flow speed. Infact the Ingress and Egress is always
> displayed as exactly half of the total throughput at that time. This is
> true for the little widget at the bottom next to the rev counter for
> ingress and egress and on the home page of a host, when clicked the "Sent
> vs Received Traffic Breakdown" is always a perfect 50/50 ratio.
>
> If I export flow data from a Mikrotik on a different network every thing
> reports correctly.
>
> What is a little unique on this network is that there are a few /26
> subnets of public IP Addresses behind this firewall. There is no natting. I
> have set these subnets as local subnets in ntopng as you can tell from my
> config below. The firewall on the WAN side has a public address and a few
> public subnets on the LAN side. Would this cause issues with remote/local
> traffic differentiation?
>
> I have tried setting V5/V9 etc flow types same issue. I have enabled just
> RX or TX from the Fortigate and these when individually enabled display
> correctly.
>
> Any help or pointers would be appreciated.
>
> My configs:
>
> root@ntopng:~# cat /etc/ntopng/ntopng.conf
> -n=3
> -w=3000
> -W=0
> -g=-1
> -m="41.xx.xx.0/26,196.x.x.x/26"
> -F=mysql;localhost;ntopng;flows;ntopuser;secretxxx
> -d=/storage/ntopng
> -G=/var/run/ntopng.pid
> -i=tcp://127.0.0.1:5556
>
>
> root@ntopng:~# cat /etc/nprobe/nprobe-ens18.conf
> -n=none
> -i=none
> -3=2055
> -s=128
> -t=60
> -d=60
> -a=0
> -e=1
> -B=10
> -w=128000
> -z=0
> -S=1:1
> -E=0:0
> -g=/var/run/nprobe-ens18.pid
> --zmq=tcp://127.0.0.1:5556
> -V=5
> --dump-stats=/var/log/nprobe/ens18-0_flows_stats.txt
>
> The fortigate was configured with the instructions here:
> http://kb.fortinet.com/kb/documentLink.do?externalID=FD36460
>
> Thanks and Regards
> Jean-Pierre Human
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>

Re: TX/RX display always identical. [ In reply to ]

mainardi at ntop

Feb 8, 2017, 7:49 AM

Post #3 of 9 (594 views)

Permalink

Re: TX/RX display always identical. [ In reply to ]

deri at ntop

Feb 8, 2017, 11:55 AM

Post #4 of 9 (594 views)

Permalink

Jean-Pierre,
as you using the latest development packages of ntopng and nprobe?

Luca

> On 8 Feb 2017, at 16:29, Jean-Pierre Human <jphuman@gmail.com> wrote:
>
> Hello
>
> I have ntopng with nProbe setup on a Ubuntu16 box, the full nBox setup from the packages.ntop.org <http://packages.ntop.org/> repo. I am exporting sflow data from a Fortigate 60D (OS 5.4.3) to nProbe.
>
> The problem I am having is the Local / Remote traffic is being reported as the same amount / flow speed. Infact the Ingress and Egress is always displayed as exactly half of the total throughput at that time. This is true for the little widget at the bottom next to the rev counter for ingress and egress and on the home page of a host, when clicked the "Sent vs Received Traffic Breakdown" is always a perfect 50/50 ratio.
>
> If I export flow data from a Mikrotik on a different network every thing reports correctly.
>
> What is a little unique on this network is that there are a few /26 subnets of public IP Addresses behind this firewall. There is no natting. I have set these subnets as local subnets in ntopng as you can tell from my config below. The firewall on the WAN side has a public address and a few public subnets on the LAN side. Would this cause issues with remote/local traffic differentiation?
>
> I have tried setting V5/V9 etc flow types same issue. I have enabled just RX or TX from the Fortigate and these when individually enabled display correctly.
>
> Any help or pointers would be appreciated.
>
> My configs:
>
> root@ntopng:~# cat /etc/ntopng/ntopng.conf
> -n=3
> -w=3000
> -W=0
> -g=-1
> -m="41.xx.xx.0/26,196.x.x.x/26"
> -F=mysql;localhost;ntopng;flows;ntopuser;secretxxx
> -d=/storage/ntopng
> -G=/var/run/ntopng.pid
> -i=tcp://127.0.0.1:5556 <http://127.0.0.1:5556/>
>
>
> root@ntopng:~# cat /etc/nprobe/nprobe-ens18.conf
> -n=none
> -i=none
> -3=2055
> -s=128
> -t=60
> -d=60
> -a=0
> -e=1
> -B=10
> -w=128000
> -z=0
> -S=1:1
> -E=0:0
> -g=/var/run/nprobe-ens18.pid
> --zmq=tcp://127.0.0.1:5556 <http://127.0.0.1:5556/>
> -V=5
> --dump-stats=/var/log/nprobe/ens18-0_flows_stats.txt
>
> The fortigate was configured with the instructions here:
> http://kb.fortinet.com/kb/documentLink.do?externalID=FD36460 <http://kb.fortinet.com/kb/documentLink.do?externalID=FD36460>
>
> Thanks and Regards
> Jean-Pierre Human
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop

Re: TX/RX display always identical. [ In reply to ]

deri at ntop

Feb 8, 2017, 11:55 AM

Post #5 of 9 (594 views)

Permalink

Re: TX/RX display always identical. [ In reply to ]

jphuman at gmail

Feb 9, 2017, 2:37 AM

Post #6 of 9 (588 views)

Permalink

Hi,

Sorry about creating a new thread I did not realise digest was on.

Firstly Luca's Question,

"as you using the latest development packages of ntopng and nprobe?
Luca"

To Answer Luca's question I am running the Stable repos, should I be
using Nightly?
ntopng x86_64 v.2.4.170202
nProbe v.7.4.170202 ($Revision: 5334 $)

Then

Simone Mainardi's response

"If all the hosts seen fall in local networks, then you will see identical
counters local2remote and remote2local. Indeed, the same amount of traffic
is counted as egress (i.e., from a local network) and as ingress (i.e., to
a local network).

Can you please verify if flow src and dst are always in local networks."

Hi Simone,

All our subnets which we define as local display as local under "Top Local
Talkers" on the Dashboard. All the remote connections to our locally
defined subnets are shown under "Top Remote Destinations" which seems
correct. To give you more info on the setup the servers on our locally
defined public subnets are public facing web, mailservers etc. With regards
to the flows it does correctly display the client / server fields as remote
or local depending on the direction of the connection.

Thanks
Jean-Pierre Human

Re: TX/RX display always identical. [ In reply to ]

deri at ntop

Feb 9, 2017, 3:08 AM

Post #7 of 9 (588 views)

Permalink

Hi,
please move to nightly. We have made some changes that will likely solve
your issues and that will be included in the next stable

Regards Luca

On 02/09/2017 11:37 AM, Jean-Pierre Human wrote:
> Hi,
>
> Sorry about creating a new thread I did not realise digest was on.
>
> Firstly Luca's Question,
>
> "as you using the latest development packages of ntopng and nprobe?
> Luca"
>
> To Answer Luca's question I am running the Stable repos, should I be
> using Nightly?
> ntopng x86_64 v.2.4.170202
> nProbe v.7.4.170202 ($Revision: 5334 $)
>
> Then
>
> Simone Mainardi's response
>
> "If all the hosts seen fall in local networks, then you will see
> identical
> counters local2remote and remote2local. Indeed, the same amount of
> traffic
> is counted as egress (i.e., from a local network) and as ingress
> (i.e., to
> a local network).
>
> Can you please verify if flow src and dst are always in local networks."
>
> Hi Simone,
>
> All our subnets which we define as local display as local under "Top
> Local Talkers" on the Dashboard. All the remote connections to our
> locally defined subnets are shown under "Top Remote Destinations"
> which seems correct. To give you more info on the setup the servers on
> our locally defined public subnets are public facing web, mailservers
> etc. With regards to the flows it does correctly display the client /
> server fields as remote or local depending on the direction of the
> connection.
>
> Thanks
> Jean-Pierre Human
>
>
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop

Re: TX/RX display always identical. [ In reply to ]

jphuman at gmail

Feb 9, 2017, 4:44 AM

Post #8 of 9 (588 views)

Permalink

Hi Luca

Upgraded to nightly however no changes, see the attached image.

https://imagebin.ca/v/3BnZuBaPazqu

Thanks

Jean-Pierre Human

On Thu, Feb 9, 2017 at 1:08 PM, Luca Deri <deri@ntop.org> wrote:

> Hi,
> please move to nightly. We have made some changes that will likely solve
> your issues and that will be included in the next stable
>
> Regards Luca
>
>
> On 02/09/2017 11:37 AM, Jean-Pierre Human wrote:
>
> Hi,
>
> Sorry about creating a new thread I did not realise digest was on.
>
> Firstly Luca's Question,
>
> "as you using the latest development packages of ntopng and nprobe?
> Luca"
>
> To Answer Luca's question I am running the Stable repos, should I be
> using Nightly?
> ntopng x86_64 v.2.4.170202
> nProbe v.7.4.170202 ($Revision: 5334 $)
>
> Then
>
> Simone Mainardi's response
>
> "If all the hosts seen fall in local networks, then you will see identical
> counters local2remote and remote2local. Indeed, the same amount of traffic
> is counted as egress (i.e., from a local network) and as ingress (i.e., to
> a local network).
>
> Can you please verify if flow src and dst are always in local networks."
>
> Hi Simone,
>
> All our subnets which we define as local display as local under "Top Local
> Talkers" on the Dashboard. All the remote connections to our locally
> defined subnets are shown under "Top Remote Destinations" which seems
> correct. To give you more info on the setup the servers on our locally
> defined public subnets are public facing web, mailservers etc. With regards
> to the flows it does correctly display the client / server fields as remote
> or local depending on the direction of the connection.
>
> Thanks
> Jean-Pierre Human
>
>
>
>
> _______________________________________________
> Ntop mailing listNtop@listgateway.unipi.ithttp://listgateway.unipi.it/mailman/listinfo/ntop
>
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>

Re: TX/RX display always identical. [ In reply to ]

deri at ntop

Feb 10, 2017, 1:11 AM

Post #9 of 9 (583 views)

Permalink

please file a bug on github including all info for tracking the bug

Thanks Luca

On 02/09/2017 01:44 PM, Jean-Pierre Human wrote:
> Hi Luca
>
> Upgraded to nightly however no changes, see the attached image.
>
> https://imagebin.ca/v/3BnZuBaPazqu <https://imagebin.ca/v/3BnZuBaPazqu>
>
> Thanks
>
> Jean-Pierre Human
>
> On Thu, Feb 9, 2017 at 1:08 PM, Luca Deri <deri@ntop.org
> <mailto:deri@ntop.org>> wrote:
>
> Hi,
> please move to nightly. We have made some changes that will likely
> solve your issues and that will be included in the next stable
>
> Regards Luca
>
>
> On 02/09/2017 11:37 AM, Jean-Pierre Human wrote:
>> Hi,
>>
>> Sorry about creating a new thread I did not realise digest was on.
>>
>> Firstly Luca's Question,
>>
>> "as you using the latest development packages of ntopng and nprobe?
>> Luca"
>>
>> To Answer Luca's question I am running the Stable repos, should
>> I be using Nightly?
>> ntopng x86_64 v.2.4.170202
>> nProbe v.7.4.170202 ($Revision: 5334 $)
>>
>> Then
>>
>> Simone Mainardi's response
>>
>> "If all the hosts seen fall in local networks, then you will see
>> identical
>> counters local2remote and remote2local. Indeed, the same amount
>> of traffic
>> is counted as egress (i.e., from a local network) and as ingress
>> (i.e., to
>> a local network).
>>
>> Can you please verify if flow src and dst are always in local
>> networks."
>>
>> Hi Simone,
>>
>> All our subnets which we define as local display as local under
>> "Top Local Talkers" on the Dashboard. All the remote connections
>> to our locally defined subnets are shown under "Top Remote
>> Destinations" which seems correct. To give you more info on the
>> setup the servers on our locally defined public subnets are
>> public facing web, mailservers etc. With regards to the flows it
>> does correctly display the client / server fields as remote or
>> local depending on the direction of the connection.
>>
>> Thanks
>> Jean-Pierre Human
>>
>>
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>> <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
> _______________________________________________ Ntop mailing list
> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop
> <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop