Mailing List Archive: Netflow (NSEL) updates from Cisco ASA

Netflow (NSEL) updates from Cisco ASA

Jun 27, 2017, 5:38 PM

Post #1 of 11 (2733 views)

Hello,

I have a Cisco ASA configured to send Netflow to an instance of nprobe, and then on to ntopng.
The configuration is working, however I have noticed that nprobe is only emitting flows when it receives a flow-teardown event from the ASA. This is causing inaccurate bandwidth reporting for long-lived flows as the total byte count is being recorded as a single spike once the flow is torn down.

My understanding is that Cisco ASA netflow is very non-standard and that this behaviour used to be expected on older version of ASA. However, newer versions of ASA are capable of sending flow-update events using a refresh-interval for active flows. When I run tcpdump on my nprobe server I can see the flow-create and flow-update events being sent from the ASA, however nprobe does not seem to use these events, or act on them in any way. I have enabled verbose logging, but can only see logs being generated for flow-teardown events, not flow-create or flow-update.

My question is, should I expect nprobe to use the flow-updates from Cisco ASA for long-lived active flows, or is it normal for it to only process flow-teardown events?

Nprobe (dev build v.8.1.170626) is running in collector mode with the following settings:

--zmq="tcp://*:5559"
--collector-port=2055
-i=none
-n=none

Regards,
Pelham

This electronic mail is solely for the use of the addressee and may contain information which is confidential or privileged.
If you receive this electronic mail in error, please delete it from your system immediately and notify the sender by electronic mail.
Any opinion expressed in this email is not represented as the opinion of Australian Communication Exchange Limited unless that is stated or
apparent from its terms.

Re: Netflow (NSEL) updates from Cisco ASA [ In reply to ]

mainardi at ntop

Jul 4, 2017, 5:25 AM

Post #2 of 11 (2727 views)

Permalink

Pelham,

ASA flow-updates are received and processed by nprobe. However, I am not
sure they contain all the necessary information required to properly update
flow statistics. Can you please generate and send a .pcap capture file of
your ASA netflow (make sure it contains both templates and data records for
flow-updates and flow-teardown) for our inspection?

By the way, nprobe gives you a couple of configurable timeout that you can
use to periodically export long-lived flows:

[--lifetime-timeout|-t] <timeout> | It specifies the maximum (seconds)
flow
| lifetime [default=120]
[--idle-timeout|-d] <timeout> | It specifies the maximum (seconds)
flow
| idle lifetime [default=30]

Regards,
Simone

On Wed, Jun 28, 2017 at 2:38 AM, Pelham Whitmore <
Pelham.Whitmore@aceinfo.net.au> wrote:

> Hello,
>
>
>
> I have a Cisco ASA configured to send Netflow to an instance of nprobe,
> and then on to ntopng.
>
> The configuration is working, however I have noticed that nprobe is only
> emitting flows when it receives a flow-teardown event from the ASA. This is
> causing inaccurate bandwidth reporting for long-lived flows as the total
> byte count is being recorded as a single spike once the flow is torn down.
>
>
>
> My understanding is that Cisco ASA netflow is very non-standard and that
> this behaviour used to be expected on older version of ASA. However, newer
> versions of ASA are capable of sending flow-update events using a
> refresh-interval for active flows. When I run tcpdump on my nprobe server I
> can see the flow-create and flow-update events being sent from the ASA,
> however nprobe does not seem to use these events, or act on them in any
> way. I have enabled verbose logging, but can only see logs being generated
> for flow-teardown events, not flow-create or flow-update.
>
>
>
> My question is, should I expect nprobe to use the flow-updates from Cisco
> ASA for long-lived active flows, or is it normal for it to only process
> flow-teardown events?
>
>
>
> Nprobe (dev build v.8.1.170626) is running in collector mode with the
> following settings:
>
>
>
> --zmq="tcp://*:5559"
>
> --collector-port=2055
>
> -i=none
>
> -n=none
>
>
>
>
>
>
>
> Regards,
>
> Pelham
>
>
>
>
>
> ------------------------------
>
> This electronic mail is solely for the use of the addressee and may
> contain information which is confidential or privileged. If you receive
> this electronic mail in error, please delete it from your system
> immediately and notify the sender by electronic mail. Any opinion expressed
> in this email is not represented as the opinion of Australian Communication
> Limited unless that is stated or apparent from its terms.
> ------------------------------
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>

Re: Netflow (NSEL) updates from Cisco ASA [ In reply to ]

mainardi at ntop

Jul 4, 2017, 5:25 AM

Post #3 of 11 (2727 views)

Permalink

Re: Netflow (NSEL) updates from Cisco ASA [ In reply to ]

Pelham.Whitmore at aceinfo

Jul 5, 2017, 6:55 PM

Post #4 of 11 (2721 views)

Permalink

Hey Simone,

Thanks for the reply. I have configured the timeout values you mentioned in nprobe however it seemed to have no effect.
I have generated a .pcap file that includes templates, flow create, update, and teardown events.

.pcap file can be downloaded from here:
https://files.aceinfo.net.au/index.php/s/bj0aFU3lpyUjqM4/download

One thing I did notcie from the packet capture is that flow update events are being listed as "Firewall Event: Unknown (5)" .
I'm not sure if that is to be expected.

Regards,
Pelham Whitmore

-----Original Message-----
From: ntop-bounces@listgateway.unipi.it [mailto:ntop-bounces@listgateway.unipi.it] On Behalf Of ntop-request@listgateway.unipi.it
Sent: 05 July 2017 8:00 PM
To: ntop@listgateway.unipi.it
Subject: Ntop Digest, Vol 158, Issue 1

Send Ntop mailing list submissions to
ntop@listgateway.unipi.it

To subscribe or unsubscribe via the World Wide Web, visit
http://listgateway.unipi.it/mailman/listinfo/ntop
or, via email, send a message with subject or body 'help' to
ntop-request@listgateway.unipi.it

You can reach the person managing the list at
ntop-owner@listgateway.unipi.it

When replying, please edit your Subject line so it is more specific than "Re: Contents of Ntop digest..."

Today's Topics:

1. Re: Netflow (NSEL) updates from Cisco ASA (Simone Mainardi)
2. Re: Netflow (NSEL) updates from Cisco ASA (Simone Mainardi)

----------------------------------------------------------------------

Message: 1
Date: Tue, 4 Jul 2017 14:25:19 +0200
From: Simone Mainardi <mainardi@ntop.org>
To: ntop@unipi.it
Cc: "ntop@listgateway.unipi.it" <ntop@listgateway.unipi.it>
Subject: Re: [Ntop] Netflow (NSEL) updates from Cisco ASA
Message-ID:
<CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Pelham,

ASA flow-updates are received and processed by nprobe. However, I am not sure they contain all the necessary information required to properly update flow statistics. Can you please generate and send a .pcap capture file of your ASA netflow (make sure it contains both templates and data records for flow-updates and flow-teardown) for our inspection?

By the way, nprobe gives you a couple of configurable timeout that you can use to periodically export long-lived flows:

[--lifetime-timeout|-t] <timeout> | It specifies the maximum (seconds)
flow
| lifetime [default=120]
[--idle-timeout|-d] <timeout> | It specifies the maximum (seconds)
flow
| idle lifetime [default=30]

Regards,
Simone

On Wed, Jun 28, 2017 at 2:38 AM, Pelham Whitmore < Pelham.Whitmore@aceinfo.net.au> wrote:

> Hello,
>
>
>
> I have a Cisco ASA configured to send Netflow to an instance of
> nprobe, and then on to ntopng.
>
> The configuration is working, however I have noticed that nprobe is
> only emitting flows when it receives a flow-teardown event from the
> ASA. This is causing inaccurate bandwidth reporting for long-lived
> flows as the total byte count is being recorded as a single spike once the flow is torn down.
>
>
>
> My understanding is that Cisco ASA netflow is very non-standard and
> that this behaviour used to be expected on older version of ASA.
> However, newer versions of ASA are capable of sending flow-update
> events using a refresh-interval for active flows. When I run tcpdump
> on my nprobe server I can see the flow-create and flow-update events
> being sent from the ASA, however nprobe does not seem to use these
> events, or act on them in any way. I have enabled verbose logging, but
> can only see logs being generated for flow-teardown events, not flow-create or flow-update.
>
>
>
> My question is, should I expect nprobe to use the flow-updates from
> Cisco ASA for long-lived active flows, or is it normal for it to only
> process flow-teardown events?
>
>
>
> Nprobe (dev build v.8.1.170626) is running in collector mode with the
> following settings:
>
>
>
> --zmq="tcp://*:5559"
>
> --collector-port=2055
>
> -i=none
>
> -n=none
>
>
>
>
>
>
>
> Regards,
>
> Pelham
>
>
>
>
>
> ------------------------------
>
> This electronic mail is solely for the use of the addressee and may
> contain information which is confidential or privileged. If you
> receive this electronic mail in error, please delete it from your
> system immediately and notify the sender by electronic mail. Any
> opinion expressed in this email is not represented as the opinion of
> Australian Communication Limited unless that is stated or apparent from its terms.
> ------------------------------
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0002.htm>

------------------------------

Message: 2
Date: Tue, 4 Jul 2017 14:25:19 +0200
From: Simone Mainardi <mainardi@ntop.org>
To: ntop@unipi.it
Cc: "ntop@listgateway.unipi.it" <ntop@listgateway.unipi.it>
Subject: Re: [Ntop] Netflow (NSEL) updates from Cisco ASA
Message-ID:
<CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Pelham,

ASA flow-updates are received and processed by nprobe. However, I am not sure they contain all the necessary information required to properly update flow statistics. Can you please generate and send a .pcap capture file of your ASA netflow (make sure it contains both templates and data records for flow-updates and flow-teardown) for our inspection?

By the way, nprobe gives you a couple of configurable timeout that you can use to periodically export long-lived flows:

[--lifetime-timeout|-t] <timeout> | It specifies the maximum (seconds)
flow
| lifetime [default=120]
[--idle-timeout|-d] <timeout> | It specifies the maximum (seconds)
flow
| idle lifetime [default=30]

Regards,
Simone

On Wed, Jun 28, 2017 at 2:38 AM, Pelham Whitmore < Pelham.Whitmore@aceinfo.net.au> wrote:

> Hello,
>
>
>
> I have a Cisco ASA configured to send Netflow to an instance of
> nprobe, and then on to ntopng.
>
> The configuration is working, however I have noticed that nprobe is
> only emitting flows when it receives a flow-teardown event from the
> ASA. This is causing inaccurate bandwidth reporting for long-lived
> flows as the total byte count is being recorded as a single spike once the flow is torn down.
>
>
>
> My understanding is that Cisco ASA netflow is very non-standard and
> that this behaviour used to be expected on older version of ASA.
> However, newer versions of ASA are capable of sending flow-update
> events using a refresh-interval for active flows. When I run tcpdump
> on my nprobe server I can see the flow-create and flow-update events
> being sent from the ASA, however nprobe does not seem to use these
> events, or act on them in any way. I have enabled verbose logging, but
> can only see logs being generated for flow-teardown events, not flow-create or flow-update.
>
>
>
> My question is, should I expect nprobe to use the flow-updates from
> Cisco ASA for long-lived active flows, or is it normal for it to only
> process flow-teardown events?
>
>
>
> Nprobe (dev build v.8.1.170626) is running in collector mode with the
> following settings:
>
>
>
> --zmq="tcp://*:5559"
>
> --collector-port=2055
>
> -i=none
>
> -n=none
>
>
>
>
>
>
>
> Regards,
>
> Pelham
>
>
>
>
>
> ------------------------------
>
> This electronic mail is solely for the use of the addressee and may
> contain information which is confidential or privileged. If you
> receive this electronic mail in error, please delete it from your
> system immediately and notify the sender by electronic mail. Any
> opinion expressed in this email is not represented as the opinion of
> Australian Communication Limited unless that is stated or apparent from its terms.
> ------------------------------
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0003.htm>

------------------------------

_______________________________________________
Ntop mailing list
Ntop@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop

End of Ntop Digest, Vol 158, Issue 1
************************************
This electronic mail is solely for the use of the addressee and may contain information which is confidential or privileged.
If you receive this electronic mail in error, please delete it from your system immediately and notify the sender by electronic mail.
Any opinion expressed in this email is not represented as the opinion of Australian Communication Exchange Limited unless that is stated or
apparent from its terms.
_______________________________________________
Ntop mailing list
Ntop@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop

Re: Netflow (NSEL) updates from Cisco ASA [ In reply to ]

mainardi at ntop

Jul 6, 2017, 4:54 AM

Post #5 of 11 (2718 views)

Permalink

Dear Pelham,

Thanks for sharing the pcap. Currently, in nProbe we deliberately ignore
ASA firewall events different from flow-delete. We have made this choice as
we have seen that events other than flow-delete often contain to few
attributes to properly update the flow. This is the reason why other
templates are silently ignored by nprobe.

If you want your nProbe to proxy all the templates received as-is, you may
want to have a look at:
http://www.ntop.org/nprobe/collecting-proprietary-flows-with-nprobe/

Simone

Regards,
Simone

On Thu, Jul 6, 2017 at 3:55 AM, Pelham Whitmore <
Pelham.Whitmore@aceinfo.net.au> wrote:

> Hey Simone,
>
> Thanks for the reply. I have configured the timeout values you mentioned
> in nprobe however it seemed to have no effect.
> I have generated a .pcap file that includes templates, flow create,
> update, and teardown events.
>
> .pcap file can be downloaded from here:
> https://files.aceinfo.net.au/index.php/s/bj0aFU3lpyUjqM4/download
>
> One thing I did notcie from the packet capture is that flow update events
> are being listed as "Firewall Event: Unknown (5)" .
> I'm not sure if that is to be expected.
>
>
> Regards,
> Pelham Whitmore
>
>
> -----Original Message-----
> From: ntop-bounces@listgateway.unipi.it [mailto:ntop-bounces@
> listgateway.unipi.it] On Behalf Of ntop-request@listgateway.unipi.it
> Sent: 05 July 2017 8:00 PM
> To: ntop@listgateway.unipi.it
> Subject: Ntop Digest, Vol 158, Issue 1
>
> Send Ntop mailing list submissions to
> ntop@listgateway.unipi.it
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://listgateway.unipi.it/mailman/listinfo/ntop
> or, via email, send a message with subject or body 'help' to
> ntop-request@listgateway.unipi.it
>
> You can reach the person managing the list at
> ntop-owner@listgateway.unipi.it
>
> When replying, please edit your Subject line so it is more specific than
> "Re: Contents of Ntop digest..."
>
>
> Today's Topics:
>
> 1. Re: Netflow (NSEL) updates from Cisco ASA (Simone Mainardi)
> 2. Re: Netflow (NSEL) updates from Cisco ASA (Simone Mainardi)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 4 Jul 2017 14:25:19 +0200
> From: Simone Mainardi <mainardi@ntop.org>
> To: ntop@unipi.it
> Cc: "ntop@listgateway.unipi.it" <ntop@listgateway.unipi.it>
> Subject: Re: [Ntop] Netflow (NSEL) updates from Cisco ASA
> Message-ID:
> <CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.
> gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Pelham,
>
> ASA flow-updates are received and processed by nprobe. However, I am not
> sure they contain all the necessary information required to properly update
> flow statistics. Can you please generate and send a .pcap capture file of
> your ASA netflow (make sure it contains both templates and data records for
> flow-updates and flow-teardown) for our inspection?
>
> By the way, nprobe gives you a couple of configurable timeout that you can
> use to periodically export long-lived flows:
>
> [--lifetime-timeout|-t] <timeout> | It specifies the maximum (seconds)
> flow
> | lifetime [default=120]
> [--idle-timeout|-d] <timeout> | It specifies the maximum (seconds)
> flow
> | idle lifetime [default=30]
>
> Regards,
> Simone
>
> On Wed, Jun 28, 2017 at 2:38 AM, Pelham Whitmore <
> Pelham.Whitmore@aceinfo.net.au> wrote:
>
> > Hello,
> >
> >
> >
> > I have a Cisco ASA configured to send Netflow to an instance of
> > nprobe, and then on to ntopng.
> >
> > The configuration is working, however I have noticed that nprobe is
> > only emitting flows when it receives a flow-teardown event from the
> > ASA. This is causing inaccurate bandwidth reporting for long-lived
> > flows as the total byte count is being recorded as a single spike once
> the flow is torn down.
> >
> >
> >
> > My understanding is that Cisco ASA netflow is very non-standard and
> > that this behaviour used to be expected on older version of ASA.
> > However, newer versions of ASA are capable of sending flow-update
> > events using a refresh-interval for active flows. When I run tcpdump
> > on my nprobe server I can see the flow-create and flow-update events
> > being sent from the ASA, however nprobe does not seem to use these
> > events, or act on them in any way. I have enabled verbose logging, but
> > can only see logs being generated for flow-teardown events, not
> flow-create or flow-update.
> >
> >
> >
> > My question is, should I expect nprobe to use the flow-updates from
> > Cisco ASA for long-lived active flows, or is it normal for it to only
> > process flow-teardown events?
> >
> >
> >
> > Nprobe (dev build v.8.1.170626) is running in collector mode with the
> > following settings:
> >
> >
> >
> > --zmq="tcp://*:5559"
> >
> > --collector-port=2055
> >
> > -i=none
> >
> > -n=none
> >
> >
> >
> >
> >
> >
> >
> > Regards,
> >
> > Pelham
> >
> >
> >
> >
> >
> > ------------------------------
> >
> > This electronic mail is solely for the use of the addressee and may
> > contain information which is confidential or privileged. If you
> > receive this electronic mail in error, please delete it from your
> > system immediately and notify the sender by electronic mail. Any
> > opinion expressed in this email is not represented as the opinion of
> > Australian Communication Limited unless that is stated or apparent from
> its terms.
> > ------------------------------
> >
> >
> > _______________________________________________
> > Ntop mailing list
> > Ntop@listgateway.unipi.it
> > http://listgateway.unipi.it/mailman/listinfo/ntop
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://listgateway.unipi.it/pipermail/ntop/attachments/
> 20170704/76008a4b/attachment-0002.htm>
>
> ------------------------------
>
> Message: 2
> Date: Tue, 4 Jul 2017 14:25:19 +0200
> From: Simone Mainardi <mainardi@ntop.org>
> To: ntop@unipi.it
> Cc: "ntop@listgateway.unipi.it" <ntop@listgateway.unipi.it>
> Subject: Re: [Ntop] Netflow (NSEL) updates from Cisco ASA
> Message-ID:
> <CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.
> gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Pelham,
>
> ASA flow-updates are received and processed by nprobe. However, I am not
> sure they contain all the necessary information required to properly update
> flow statistics. Can you please generate and send a .pcap capture file of
> your ASA netflow (make sure it contains both templates and data records for
> flow-updates and flow-teardown) for our inspection?
>
> By the way, nprobe gives you a couple of configurable timeout that you can
> use to periodically export long-lived flows:
>
> [--lifetime-timeout|-t] <timeout> | It specifies the maximum (seconds)
> flow
> | lifetime [default=120]
> [--idle-timeout|-d] <timeout> | It specifies the maximum (seconds)
> flow
> | idle lifetime [default=30]
>
> Regards,
> Simone
>
> On Wed, Jun 28, 2017 at 2:38 AM, Pelham Whitmore <
> Pelham.Whitmore@aceinfo.net.au> wrote:
>
> > Hello,
> >
> >
> >
> > I have a Cisco ASA configured to send Netflow to an instance of
> > nprobe, and then on to ntopng.
> >
> > The configuration is working, however I have noticed that nprobe is
> > only emitting flows when it receives a flow-teardown event from the
> > ASA. This is causing inaccurate bandwidth reporting for long-lived
> > flows as the total byte count is being recorded as a single spike once
> the flow is torn down.
> >
> >
> >
> > My understanding is that Cisco ASA netflow is very non-standard and
> > that this behaviour used to be expected on older version of ASA.
> > However, newer versions of ASA are capable of sending flow-update
> > events using a refresh-interval for active flows. When I run tcpdump
> > on my nprobe server I can see the flow-create and flow-update events
> > being sent from the ASA, however nprobe does not seem to use these
> > events, or act on them in any way. I have enabled verbose logging, but
> > can only see logs being generated for flow-teardown events, not
> flow-create or flow-update.
> >
> >
> >
> > My question is, should I expect nprobe to use the flow-updates from
> > Cisco ASA for long-lived active flows, or is it normal for it to only
> > process flow-teardown events?
> >
> >
> >
> > Nprobe (dev build v.8.1.170626) is running in collector mode with the
> > following settings:
> >
> >
> >
> > --zmq="tcp://*:5559"
> >
> > --collector-port=2055
> >
> > -i=none
> >
> > -n=none
> >
> >
> >
> >
> >
> >
> >
> > Regards,
> >
> > Pelham
> >
> >
> >
> >
> >
> > ------------------------------
> >
> > This electronic mail is solely for the use of the addressee and may
> > contain information which is confidential or privileged. If you
> > receive this electronic mail in error, please delete it from your
> > system immediately and notify the sender by electronic mail. Any
> > opinion expressed in this email is not represented as the opinion of
> > Australian Communication Limited unless that is stated or apparent from
> its terms.
> > ------------------------------
> >
> >
> > _______________________________________________
> > Ntop mailing list
> > Ntop@listgateway.unipi.it
> > http://listgateway.unipi.it/mailman/listinfo/ntop
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://listgateway.unipi.it/pipermail/ntop/attachments/
> 20170704/76008a4b/attachment-0003.htm>
>
> ------------------------------
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
> End of Ntop Digest, Vol 158, Issue 1
> ************************************
> This electronic mail is solely for the use of the addressee and may
> contain information which is confidential or privileged.
> If you receive this electronic mail in error, please delete it from your
> system immediately and notify the sender by electronic mail.
> Any opinion expressed in this email is not represented as the opinion of
> Australian Communication Exchange Limited unless that is stated or
> apparent from its terms.
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>

Re: Netflow (NSEL) updates from Cisco ASA [ In reply to ]

mainardi at ntop

Jul 6, 2017, 4:54 AM

Post #6 of 11 (2718 views)

Permalink

Re: Netflow (NSEL) updates from Cisco ASA [ In reply to ]

mkh at rqc

Jul 6, 2017, 5:13 AM

Post #7 of 11 (2718 views)

Permalink

Hello Simone,

I'm also very interested in this issue, observing same behaviour as
Pelham. Do you mean we need a custom fields configuration file to make
this work? How will it be different from your attempts where "events
other than flow-delete often contain to few attributes to properly
update the flow"?

--

With Best Regards,
Marat Khalili

On 06/07/17 14:54, Simone Mainardi wrote:
> Dear Pelham,
>
> Thanks for sharing the pcap. Currently, in nProbe we deliberately
> ignore ASA firewall events different from flow-delete. We have made
> this choice as we have seen that events other than flow-delete often
> contain to few attributes to properly update the flow. This is the
> reason why other templates are silently ignored by nprobe.
>
> If you want your nProbe to proxy all the templates received as-is, you
> may want to have a look at:
> http://www.ntop.org/nprobe/collecting-proprietary-flows-with-nprobe/
>
>
> Simone
>
> Regards,
> Simone
>
>
>
> On Thu, Jul 6, 2017 at 3:55 AM, Pelham Whitmore
> <Pelham.Whitmore@aceinfo.net.au
> <mailto:Pelham.Whitmore@aceinfo.net.au>> wrote:
>
> Hey Simone,
>
> Thanks for the reply. I have configured the timeout values you
> mentioned in nprobe however it seemed to have no effect.
> I have generated a .pcap file that includes templates, flow
> create, update, and teardown events.
>
> .pcap file can be downloaded from here:
> https://files.aceinfo.net.au/index.php/s/bj0aFU3lpyUjqM4/download
> <https://files.aceinfo.net.au/index.php/s/bj0aFU3lpyUjqM4/download>
>
> One thing I did notcie from the packet capture is that flow update
> events are being listed as "Firewall Event: Unknown (5)" .
> I'm not sure if that is to be expected.
>
>
> Regards,
> Pelham Whitmore
>
>
> -----Original Message-----
> From: ntop-bounces@listgateway.unipi.it
> <mailto:ntop-bounces@listgateway.unipi.it>
> [mailto:ntop-bounces@listgateway.unipi.it
> <mailto:ntop-bounces@listgateway.unipi.it>] On Behalf Of
> ntop-request@listgateway.unipi.it
> <mailto:ntop-request@listgateway.unipi.it>
> Sent: 05 July 2017 8:00 PM
> To: ntop@listgateway.unipi.it <mailto:ntop@listgateway.unipi.it>
> Subject: Ntop Digest, Vol 158, Issue 1
>
> Send Ntop mailing list submissions to
> ntop@listgateway.unipi.it <mailto:ntop@listgateway.unipi.it>
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://listgateway.unipi.it/mailman/listinfo/ntop
> <http://listgateway.unipi.it/mailman/listinfo/ntop>
> or, via email, send a message with subject or body 'help' to
> ntop-request@listgateway.unipi.it
> <mailto:ntop-request@listgateway.unipi.it>
>
> You can reach the person managing the list at
> ntop-owner@listgateway.unipi.it
> <mailto:ntop-owner@listgateway.unipi.it>
>
> When replying, please edit your Subject line so it is more
> specific than "Re: Contents of Ntop digest..."
>
>
> Today's Topics:
>
> 1. Re: Netflow (NSEL) updates from Cisco ASA (Simone Mainardi)
> 2. Re: Netflow (NSEL) updates from Cisco ASA (Simone Mainardi)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 4 Jul 2017 14:25:19 +0200
> From: Simone Mainardi <mainardi@ntop.org <mailto:mainardi@ntop.org>>
> To: ntop@unipi.it <mailto:ntop@unipi.it>
> Cc: "ntop@listgateway.unipi.it <mailto:ntop@listgateway.unipi.it>"
> <ntop@listgateway.unipi.it <mailto:ntop@listgateway.unipi.it>>
> Subject: Re: [Ntop] Netflow (NSEL) updates from Cisco ASA
> Message-ID:
>
> <CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com
> <mailto:CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com>>
> Content-Type: text/plain; charset="utf-8"
>
> Pelham,
>
> ASA flow-updates are received and processed by nprobe. However, I
> am not sure they contain all the necessary information required to
> properly update flow statistics. Can you please generate and send
> a .pcap capture file of your ASA netflow (make sure it contains
> both templates and data records for flow-updates and
> flow-teardown) for our inspection?
>
> By the way, nprobe gives you a couple of configurable timeout that
> you can use to periodically export long-lived flows:
>
> [--lifetime-timeout|-t] <timeout> | It specifies the maximum
> (seconds)
> flow
> | lifetime [default=120]
> [--idle-timeout|-d] <timeout> | It specifies the maximum
> (seconds)
> flow
> | idle lifetime [default=30]
>
> Regards,
> Simone
>
> On Wed, Jun 28, 2017 at 2:38 AM, Pelham Whitmore <
> Pelham.Whitmore@aceinfo.net.au
> <mailto:Pelham.Whitmore@aceinfo.net.au>> wrote:
>
> > Hello,
> >
> >
> >
> > I have a Cisco ASA configured to send Netflow to an instance of
> > nprobe, and then on to ntopng.
> >
> > The configuration is working, however I have noticed that nprobe is
> > only emitting flows when it receives a flow-teardown event from the
> > ASA. This is causing inaccurate bandwidth reporting for long-lived
> > flows as the total byte count is being recorded as a single
> spike once the flow is torn down.
> >
> >
> >
> > My understanding is that Cisco ASA netflow is very non-standard and
> > that this behaviour used to be expected on older version of ASA.
> > However, newer versions of ASA are capable of sending flow-update
> > events using a refresh-interval for active flows. When I run tcpdump
> > on my nprobe server I can see the flow-create and flow-update events
> > being sent from the ASA, however nprobe does not seem to use these
> > events, or act on them in any way. I have enabled verbose
> logging, but
> > can only see logs being generated for flow-teardown events, not
> flow-create or flow-update.
> >
> >
> >
> > My question is, should I expect nprobe to use the flow-updates from
> > Cisco ASA for long-lived active flows, or is it normal for it to
> only
> > process flow-teardown events?
> >
> >
> >
> > Nprobe (dev build v.8.1.170626) is running in collector mode
> with the
> > following settings:
> >
> >
> >
> > --zmq="tcp://*:5559"
> >
> > --collector-port=2055
> >
> > -i=none
> >
> > -n=none
> >
> >
> >
> >
> >
> >
> >
> > Regards,
> >
> > Pelham
> >
> >
> >
> >
> >
> > ------------------------------
> >
> > This electronic mail is solely for the use of the addressee and may
> > contain information which is confidential or privileged. If you
> > receive this electronic mail in error, please delete it from your
> > system immediately and notify the sender by electronic mail. Any
> > opinion expressed in this email is not represented as the opinion of
> > Australian Communication Limited unless that is stated or
> apparent from its terms.
> > ------------------------------
> >
> >
> > _______________________________________________
> > Ntop mailing list
> > Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
> > http://listgateway.unipi.it/mailman/listinfo/ntop
> <http://listgateway.unipi.it/mailman/listinfo/ntop>
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0002.htm
> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0002.htm>>
>
> ------------------------------
>
> Message: 2
> Date: Tue, 4 Jul 2017 14:25:19 +0200
> From: Simone Mainardi <mainardi@ntop.org <mailto:mainardi@ntop.org>>
> To: ntop@unipi.it <mailto:ntop@unipi.it>
> Cc: "ntop@listgateway.unipi.it <mailto:ntop@listgateway.unipi.it>"
> <ntop@listgateway.unipi.it <mailto:ntop@listgateway.unipi.it>>
> Subject: Re: [Ntop] Netflow (NSEL) updates from Cisco ASA
> Message-ID:
>
> <CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com
> <mailto:CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com>>
> Content-Type: text/plain; charset="utf-8"
>
> Pelham,
>
> ASA flow-updates are received and processed by nprobe. However, I
> am not sure they contain all the necessary information required to
> properly update flow statistics. Can you please generate and send
> a .pcap capture file of your ASA netflow (make sure it contains
> both templates and data records for flow-updates and
> flow-teardown) for our inspection?
>
> By the way, nprobe gives you a couple of configurable timeout that
> you can use to periodically export long-lived flows:
>
> [--lifetime-timeout|-t] <timeout> | It specifies the maximum
> (seconds)
> flow
> | lifetime [default=120]
> [--idle-timeout|-d] <timeout> | It specifies the maximum
> (seconds)
> flow
> | idle lifetime [default=30]
>
> Regards,
> Simone
>
> On Wed, Jun 28, 2017 at 2:38 AM, Pelham Whitmore <
> Pelham.Whitmore@aceinfo.net.au
> <mailto:Pelham.Whitmore@aceinfo.net.au>> wrote:
>
> > Hello,
> >
> >
> >
> > I have a Cisco ASA configured to send Netflow to an instance of
> > nprobe, and then on to ntopng.
> >
> > The configuration is working, however I have noticed that nprobe is
> > only emitting flows when it receives a flow-teardown event from the
> > ASA. This is causing inaccurate bandwidth reporting for long-lived
> > flows as the total byte count is being recorded as a single
> spike once the flow is torn down.
> >
> >
> >
> > My understanding is that Cisco ASA netflow is very non-standard and
> > that this behaviour used to be expected on older version of ASA.
> > However, newer versions of ASA are capable of sending flow-update
> > events using a refresh-interval for active flows. When I run tcpdump
> > on my nprobe server I can see the flow-create and flow-update events
> > being sent from the ASA, however nprobe does not seem to use these
> > events, or act on them in any way. I have enabled verbose
> logging, but
> > can only see logs being generated for flow-teardown events, not
> flow-create or flow-update.
> >
> >
> >
> > My question is, should I expect nprobe to use the flow-updates from
> > Cisco ASA for long-lived active flows, or is it normal for it to
> only
> > process flow-teardown events?
> >
> >
> >
> > Nprobe (dev build v.8.1.170626) is running in collector mode
> with the
> > following settings:
> >
> >
> >
> > --zmq="tcp://*:5559"
> >
> > --collector-port=2055
> >
> > -i=none
> >
> > -n=none
> >
> >
> >
> >
> >
> >
> >
> > Regards,
> >
> > Pelham
> >
> >
> >
> >
> >
> > ------------------------------
> >
> > This electronic mail is solely for the use of the addressee and may
> > contain information which is confidential or privileged. If you
> > receive this electronic mail in error, please delete it from your
> > system immediately and notify the sender by electronic mail. Any
> > opinion expressed in this email is not represented as the opinion of
> > Australian Communication Limited unless that is stated or
> apparent from its terms.
> > ------------------------------
> >
> >
> > _______________________________________________
> > Ntop mailing list
> > Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
> > http://listgateway.unipi.it/mailman/listinfo/ntop
> <http://listgateway.unipi.it/mailman/listinfo/ntop>
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0003.htm
> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0003.htm>>
>
> ------------------------------
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop
> <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
> End of Ntop Digest, Vol 158, Issue 1
> ************************************
> This electronic mail is solely for the use of the addressee and
> may contain information which is confidential or privileged.
> If you receive this electronic mail in error, please delete it
> from your system immediately and notify the sender by electronic mail.
> Any opinion expressed in this email is not represented as the
> opinion of Australian Communication Exchange Limited unless that
> is stated or
> apparent from its terms.
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop
> <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
>
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop

Re: Netflow (NSEL) updates from Cisco ASA [ In reply to ]

deri at ntop

Jul 12, 2017, 3:26 PM

Post #8 of 11 (2711 views)

Permalink

Hi all,
ASA (and other devices like PaloAlto) generate flows based on firewall events, like flow create, deletion etc. and do not send interim updates as flow devices do. ASA for instance sends only total octets.

Due to this if we take into account flow updates we need to make sure that the frequency of these updates is greater than the flow inactivity timeout in ntopng as otherwise we would generate incorrect values (i.e. we will account more traffic than the one that has really passed on the network). The flow delete instead is not affected by this problem as it is the very last message emitted for a given flow.

This said, reading this thread it seems that you all have different option and this I have modified nProbe to implement the behaviour you expect. Shall you have further comments on this matter, please file ticket on github so we can keep a log of the discussion

Regards Luca

> On 6 Jul 2017, at 14:13, Marat Khalili <mkh@rqc.ru> wrote:
>
> Hello Simone,
>
> I'm also very interested in this issue, observing same behaviour as Pelham. Do you mean we need a custom fields configuration file to make this work? How will it be different from your attempts where "events other than flow-delete often contain to few attributes to properly update the flow"?
> --
>
> With Best Regards,
> Marat Khalili
>
> On 06/07/17 14:54, Simone Mainardi wrote:
>> Dear Pelham,
>>
>> Thanks for sharing the pcap. Currently, in nProbe we deliberately ignore ASA firewall events different from flow-delete. We have made this choice as we have seen that events other than flow-delete often contain to few attributes to properly update the flow. This is the reason why other templates are silently ignored by nprobe.
>>
>> If you want your nProbe to proxy all the templates received as-is, you may want to have a look at: http://www.ntop.org/nprobe/collecting-proprietary-flows-with-nprobe/ <http://www.ntop.org/nprobe/collecting-proprietary-flows-with-nprobe/>
>>
>>
>> Simone
>>
>> Regards,
>> Simone
>>
>>
>>
>> On Thu, Jul 6, 2017 at 3:55 AM, Pelham Whitmore <Pelham.Whitmore@aceinfo.net.au <mailto:Pelham.Whitmore@aceinfo.net.au>> wrote:
>> Hey Simone,
>>
>> Thanks for the reply. I have configured the timeout values you mentioned in nprobe however it seemed to have no effect.
>> I have generated a .pcap file that includes templates, flow create, update, and teardown events.
>>
>> .pcap file can be downloaded from here:
>> https://files.aceinfo.net.au/index.php/s/bj0aFU3lpyUjqM4/download <https://files.aceinfo.net.au/index.php/s/bj0aFU3lpyUjqM4/download>
>>
>> One thing I did notcie from the packet capture is that flow update events are being listed as "Firewall Event: Unknown (5)" .
>> I'm not sure if that is to be expected.
>>
>>
>> Regards,
>> Pelham Whitmore
>>
>>
>> -----Original Message-----
>> From: ntop-bounces@listgateway.unipi.it <mailto:ntop-bounces@listgateway.unipi.it> [mailto:ntop-bounces@listgateway.unipi.it <mailto:ntop-bounces@listgateway.unipi.it>] On Behalf Of ntop-request@listgateway.unipi.it <mailto:ntop-request@listgateway.unipi.it>
>> Sent: 05 July 2017 8:00 PM
>> To: ntop@listgateway.unipi.it <mailto:ntop@listgateway.unipi.it>
>> Subject: Ntop Digest, Vol 158, Issue 1
>>
>> Send Ntop mailing list submissions to
>> ntop@listgateway.unipi.it <mailto:ntop@listgateway.unipi.it>
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>> or, via email, send a message with subject or body 'help' to
>> ntop-request@listgateway.unipi.it <mailto:ntop-request@listgateway.unipi.it>
>>
>> You can reach the person managing the list at
>> ntop-owner@listgateway.unipi.it <mailto:ntop-owner@listgateway.unipi.it>
>>
>> When replying, please edit your Subject line so it is more specific than "Re: Contents of Ntop digest..."
>>
>>
>> Today's Topics:
>>
>> 1. Re: Netflow (NSEL) updates from Cisco ASA (Simone Mainardi)
>> 2. Re: Netflow (NSEL) updates from Cisco ASA (Simone Mainardi)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Tue, 4 Jul 2017 14:25:19 +0200
>> From: Simone Mainardi <mainardi@ntop.org <mailto:mainardi@ntop.org>>
>> To: ntop@unipi.it <mailto:ntop@unipi.it>
>> Cc: "ntop@listgateway.unipi.it <mailto:ntop@listgateway.unipi.it>" <ntop@listgateway.unipi.it <mailto:ntop@listgateway.unipi.it>>
>> Subject: Re: [Ntop] Netflow (NSEL) updates from Cisco ASA
>> Message-ID:
>> <CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com <mailto:CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com>>
>> Content-Type: text/plain; charset="utf-8"
>>
>> Pelham,
>>
>> ASA flow-updates are received and processed by nprobe. However, I am not sure they contain all the necessary information required to properly update flow statistics. Can you please generate and send a .pcap capture file of your ASA netflow (make sure it contains both templates and data records for flow-updates and flow-teardown) for our inspection?
>>
>> By the way, nprobe gives you a couple of configurable timeout that you can use to periodically export long-lived flows:
>>
>> [--lifetime-timeout|-t] <timeout> | It specifies the maximum (seconds)
>> flow
>> | lifetime [default=120]
>> [--idle-timeout|-d] <timeout> | It specifies the maximum (seconds)
>> flow
>> | idle lifetime [default=30]
>>
>> Regards,
>> Simone
>>
>> On Wed, Jun 28, 2017 at 2:38 AM, Pelham Whitmore < Pelham.Whitmore@aceinfo.net.au <mailto:Pelham.Whitmore@aceinfo.net.au>> wrote:
>>
>> > Hello,
>> >
>> >
>> >
>> > I have a Cisco ASA configured to send Netflow to an instance of
>> > nprobe, and then on to ntopng.
>> >
>> > The configuration is working, however I have noticed that nprobe is
>> > only emitting flows when it receives a flow-teardown event from the
>> > ASA. This is causing inaccurate bandwidth reporting for long-lived
>> > flows as the total byte count is being recorded as a single spike once the flow is torn down.
>> >
>> >
>> >
>> > My understanding is that Cisco ASA netflow is very non-standard and
>> > that this behaviour used to be expected on older version of ASA.
>> > However, newer versions of ASA are capable of sending flow-update
>> > events using a refresh-interval for active flows. When I run tcpdump
>> > on my nprobe server I can see the flow-create and flow-update events
>> > being sent from the ASA, however nprobe does not seem to use these
>> > events, or act on them in any way. I have enabled verbose logging, but
>> > can only see logs being generated for flow-teardown events, not flow-create or flow-update.
>> >
>> >
>> >
>> > My question is, should I expect nprobe to use the flow-updates from
>> > Cisco ASA for long-lived active flows, or is it normal for it to only
>> > process flow-teardown events?
>> >
>> >
>> >
>> > Nprobe (dev build v.8.1.170626) is running in collector mode with the
>> > following settings:
>> >
>> >
>> >
>> > --zmq="tcp://*:5559"
>> >
>> > --collector-port=2055
>> >
>> > -i=none
>> >
>> > -n=none
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > Regards,
>> >
>> > Pelham
>> >
>> >
>> >
>> >
>> >
>> > ------------------------------
>> >
>> > This electronic mail is solely for the use of the addressee and may
>> > contain information which is confidential or privileged. If you
>> > receive this electronic mail in error, please delete it from your
>> > system immediately and notify the sender by electronic mail. Any
>> > opinion expressed in this email is not represented as the opinion of
>> > Australian Communication Limited unless that is stated or apparent from its terms.
>> > ------------------------------
>> >
>> >
>> > _______________________________________________
>> > Ntop mailing list
>> > Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> > http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>> >
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL: <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0002.htm <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0002.htm>>
>>
>> ------------------------------
>>
>> Message: 2
>> Date: Tue, 4 Jul 2017 14:25:19 +0200
>> From: Simone Mainardi <mainardi@ntop.org <mailto:mainardi@ntop.org>>
>> To: ntop@unipi.it <mailto:ntop@unipi.it>
>> Cc: "ntop@listgateway.unipi.it <mailto:ntop@listgateway.unipi.it>" <ntop@listgateway.unipi.it <mailto:ntop@listgateway.unipi.it>>
>> Subject: Re: [Ntop] Netflow (NSEL) updates from Cisco ASA
>> Message-ID:
>> <CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com <mailto:CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com>>
>> Content-Type: text/plain; charset="utf-8"
>>
>> Pelham,
>>
>> ASA flow-updates are received and processed by nprobe. However, I am not sure they contain all the necessary information required to properly update flow statistics. Can you please generate and send a .pcap capture file of your ASA netflow (make sure it contains both templates and data records for flow-updates and flow-teardown) for our inspection?
>>
>> By the way, nprobe gives you a couple of configurable timeout that you can use to periodically export long-lived flows:
>>
>> [--lifetime-timeout|-t] <timeout> | It specifies the maximum (seconds)
>> flow
>> | lifetime [default=120]
>> [--idle-timeout|-d] <timeout> | It specifies the maximum (seconds)
>> flow
>> | idle lifetime [default=30]
>>
>> Regards,
>> Simone
>>
>> On Wed, Jun 28, 2017 at 2:38 AM, Pelham Whitmore < Pelham.Whitmore@aceinfo.net.au <mailto:Pelham.Whitmore@aceinfo.net.au>> wrote:
>>
>> > Hello,
>> >
>> >
>> >
>> > I have a Cisco ASA configured to send Netflow to an instance of
>> > nprobe, and then on to ntopng.
>> >
>> > The configuration is working, however I have noticed that nprobe is
>> > only emitting flows when it receives a flow-teardown event from the
>> > ASA. This is causing inaccurate bandwidth reporting for long-lived
>> > flows as the total byte count is being recorded as a single spike once the flow is torn down.
>> >
>> >
>> >
>> > My understanding is that Cisco ASA netflow is very non-standard and
>> > that this behaviour used to be expected on older version of ASA.
>> > However, newer versions of ASA are capable of sending flow-update
>> > events using a refresh-interval for active flows. When I run tcpdump
>> > on my nprobe server I can see the flow-create and flow-update events
>> > being sent from the ASA, however nprobe does not seem to use these
>> > events, or act on them in any way. I have enabled verbose logging, but
>> > can only see logs being generated for flow-teardown events, not flow-create or flow-update.
>> >
>> >
>> >
>> > My question is, should I expect nprobe to use the flow-updates from
>> > Cisco ASA for long-lived active flows, or is it normal for it to only
>> > process flow-teardown events?
>> >
>> >
>> >
>> > Nprobe (dev build v.8.1.170626) is running in collector mode with the
>> > following settings:
>> >
>> >
>> >
>> > --zmq="tcp://*:5559"
>> >
>> > --collector-port=2055
>> >
>> > -i=none
>> >
>> > -n=none
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > Regards,
>> >
>> > Pelham
>> >
>> >
>> >
>> >
>> >
>> > ------------------------------
>> >
>> > This electronic mail is solely for the use of the addressee and may
>> > contain information which is confidential or privileged. If you
>> > receive this electronic mail in error, please delete it from your
>> > system immediately and notify the sender by electronic mail. Any
>> > opinion expressed in this email is not represented as the opinion of
>> > Australian Communication Limited unless that is stated or apparent from its terms.
>> > ------------------------------
>> >
>> >
>> > _______________________________________________
>> > Ntop mailing list
>> > Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> > http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>> >
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL: <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0003.htm <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0003.htm>>
>>
>> ------------------------------
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>
>> End of Ntop Digest, Vol 158, Issue 1
>> ************************************
>> This electronic mail is solely for the use of the addressee and may contain information which is confidential or privileged.
>> If you receive this electronic mail in error, please delete it from your system immediately and notify the sender by electronic mail.
>> Any opinion expressed in this email is not represented as the opinion of Australian Communication Exchange Limited unless that is stated or
>> apparent from its terms.
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop

Re: Netflow (NSEL) updates from Cisco ASA [ In reply to ]

deri at ntop

Jul 12, 2017, 3:26 PM

Post #9 of 11 (2711 views)

Permalink

Re: Netflow (NSEL) updates from Cisco ASA [ In reply to ]

mkh at rqc

Jul 13, 2017, 12:27 AM

Post #10 of 11 (2706 views)

Permalink

> ASA (and other devices like PaloAlto) generate flows based on firewall
> events, like flow create, deletion etc. and do not send interim
> updates as flow devices do. ASA for instance sends only total octets.
According to documentation
<https://www.cisco.com/c/en/us/td/docs/security/asa/special/netflow/guide/asa_netflow.html#pgfId-1324322>
there are Flow-Update events sent every minute by default.
> Shall you have further comments on this matter, please file ticket on
> github so we can keep a log of the discussion
Ok, I'll re-test it my environment and open the ticket with what I'm seeing.

--

With Best Regards,
Marat Khalili

On 13/07/17 01:26, Luca Deri wrote:
> Hi all,
> ASA (and other devices like PaloAlto) generate flows based on firewall
> events, like flow create, deletion etc. and do not send interim
> updates as flow devices do. ASA for instance sends only total octets.
>
> Due to this if we take into account flow updates we need to make sure
> that the frequency of these updates is greater than the flow
> inactivity timeout in ntopng as otherwise we would generate incorrect
> values (i.e. we will account more traffic than the one that has really
> passed on the network). The flow delete instead is not affected by
> this problem as it is the very last message emitted for a given flow.
>
> This said, reading this thread it seems that you all have different
> option and this I have modified nProbe to implement the behaviour you
> expect. Shall you have further comments on this matter, please file
> ticket on github so we can keep a log of the discussion
>
> Regards Luca
>
>> On 6 Jul 2017, at 14:13, Marat Khalili <mkh@rqc.ru
>> <mailto:mkh@rqc.ru>> wrote:
>>
>> Hello Simone,
>>
>> I'm also very interested in this issue, observing same behaviour as
>> Pelham. Do you mean we need a custom fields configuration file to
>> make this work? How will it be different from your attempts where
>> "events other than flow-delete often contain to few attributes to
>> properly update the flow"?
>>
>> --
>>
>> With Best Regards,
>> Marat Khalili
>>
>> On 06/07/17 14:54, Simone Mainardi wrote:
>>> Dear Pelham,
>>>
>>> Thanks for sharing the pcap. Currently, in nProbe we deliberately
>>> ignore ASA firewall events different from flow-delete. We have made
>>> this choice as we have seen that events other than flow-delete often
>>> contain to few attributes to properly update the flow. This is the
>>> reason why other templates are silently ignored by nprobe.
>>>
>>> If you want your nProbe to proxy all the templates received as-is,
>>> you may want to have a look at:
>>> http://www.ntop.org/nprobe/collecting-proprietary-flows-with-nprobe/
>>>
>>>
>>> Simone
>>>
>>> Regards,
>>> Simone
>>>
>>>
>>>
>>> On Thu, Jul 6, 2017 at 3:55 AM, Pelham Whitmore
>>> <Pelham.Whitmore@aceinfo.net.au
>>> <mailto:Pelham.Whitmore@aceinfo.net.au>> wrote:
>>>
>>> Hey Simone,
>>>
>>> Thanks for the reply. I have configured the timeout values you
>>> mentioned in nprobe however it seemed to have no effect.
>>> I have generated a .pcap file that includes templates, flow
>>> create, update, and teardown events.
>>>
>>> .pcap file can be downloaded from here:
>>> https://files.aceinfo.net.au/index.php/s/bj0aFU3lpyUjqM4/download
>>> <https://files.aceinfo.net.au/index.php/s/bj0aFU3lpyUjqM4/download>
>>>
>>> One thing I did notcie from the packet capture is that flow
>>> update events are being listed as "Firewall Event: Unknown (5)" .
>>> I'm not sure if that is to be expected.
>>>
>>>
>>> Regards,
>>> Pelham Whitmore
>>>
>>>
>>> -----Original Message-----
>>> From: ntop-bounces@listgateway.unipi.it
>>> <mailto:ntop-bounces@listgateway.unipi.it>
>>> [mailto:ntop-bounces@listgateway.unipi.it
>>> <mailto:ntop-bounces@listgateway.unipi.it>] On Behalf Of
>>> ntop-request@listgateway.unipi.it
>>> <mailto:ntop-request@listgateway.unipi.it>
>>> Sent: 05 July 2017 8:00 PM
>>> To: ntop@listgateway.unipi.it <mailto:ntop@listgateway.unipi.it>
>>> Subject: Ntop Digest, Vol 158, Issue 1
>>>
>>> Send Ntop mailing list submissions to
>>> ntop@listgateway.unipi.it <mailto:ntop@listgateway.unipi.it>
>>>
>>> To subscribe or unsubscribe via the World Wide Web, visit
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>> <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>> or, via email, send a message with subject or body 'help' to
>>> ntop-request@listgateway.unipi.it
>>> <mailto:ntop-request@listgateway.unipi.it>
>>>
>>> You can reach the person managing the list at
>>> ntop-owner@listgateway.unipi.it
>>> <mailto:ntop-owner@listgateway.unipi.it>
>>>
>>> When replying, please edit your Subject line so it is more
>>> specific than "Re: Contents of Ntop digest..."
>>>
>>>
>>> Today's Topics:
>>>
>>> 1. Re: Netflow (NSEL) updates from Cisco ASA (Simone Mainardi)
>>> 2. Re: Netflow (NSEL) updates from Cisco ASA (Simone Mainardi)
>>>
>>>
>>> ----------------------------------------------------------------------
>>>
>>> Message: 1
>>> Date: Tue, 4 Jul 2017 14:25:19 +0200
>>> From: Simone Mainardi <mainardi@ntop.org <mailto:mainardi@ntop.org>>
>>> To: ntop@unipi.it <mailto:ntop@unipi.it>
>>> Cc: "ntop@listgateway.unipi.it
>>> <mailto:ntop@listgateway.unipi.it>" <ntop@listgateway.unipi.it
>>> <mailto:ntop@listgateway.unipi.it>>
>>> Subject: Re: [Ntop] Netflow (NSEL) updates from Cisco ASA
>>> Message-ID:
>>>
>>> <CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com
>>> <mailto:CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com>>
>>> Content-Type: text/plain; charset="utf-8"
>>>
>>> Pelham,
>>>
>>> ASA flow-updates are received and processed by nprobe. However,
>>> I am not sure they contain all the necessary information
>>> required to properly update flow statistics. Can you please
>>> generate and send a .pcap capture file of your ASA netflow (make
>>> sure it contains both templates and data records for
>>> flow-updates and flow-teardown) for our inspection?
>>>
>>> By the way, nprobe gives you a couple of configurable timeout
>>> that you can use to periodically export long-lived flows:
>>>
>>> [--lifetime-timeout|-t] <timeout> | It specifies the maximum
>>> (seconds)
>>> flow
>>> | lifetime [default=120]
>>> [--idle-timeout|-d] <timeout> | It specifies the maximum
>>> (seconds)
>>> flow
>>> | idle lifetime [default=30]
>>>
>>> Regards,
>>> Simone
>>>
>>> On Wed, Jun 28, 2017 at 2:38 AM, Pelham Whitmore <
>>> Pelham.Whitmore@aceinfo.net.au
>>> <mailto:Pelham.Whitmore@aceinfo.net.au>> wrote:
>>>
>>> > Hello,
>>> >
>>> >
>>> >
>>> > I have a Cisco ASA configured to send Netflow to an instance of
>>> > nprobe, and then on to ntopng.
>>> >
>>> > The configuration is working, however I have noticed that
>>> nprobe is
>>> > only emitting flows when it receives a flow-teardown event
>>> from the
>>> > ASA. This is causing inaccurate bandwidth reporting for long-lived
>>> > flows as the total byte count is being recorded as a single
>>> spike once the flow is torn down.
>>> >
>>> >
>>> >
>>> > My understanding is that Cisco ASA netflow is very
>>> non-standard and
>>> > that this behaviour used to be expected on older version of ASA.
>>> > However, newer versions of ASA are capable of sending flow-update
>>> > events using a refresh-interval for active flows. When I run
>>> tcpdump
>>> > on my nprobe server I can see the flow-create and flow-update
>>> events
>>> > being sent from the ASA, however nprobe does not seem to use these
>>> > events, or act on them in any way. I have enabled verbose
>>> logging, but
>>> > can only see logs being generated for flow-teardown events,
>>> not flow-create or flow-update.
>>> >
>>> >
>>> >
>>> > My question is, should I expect nprobe to use the flow-updates
>>> from
>>> > Cisco ASA for long-lived active flows, or is it normal for it
>>> to only
>>> > process flow-teardown events?
>>> >
>>> >
>>> >
>>> > Nprobe (dev build v.8.1.170626) is running in collector mode
>>> with the
>>> > following settings:
>>> >
>>> >
>>> >
>>> > --zmq="tcp://*:5559"
>>> >
>>> > --collector-port=2055
>>> >
>>> > -i=none
>>> >
>>> > -n=none
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > Regards,
>>> >
>>> > Pelham
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > ------------------------------
>>> >
>>> > This electronic mail is solely for the use of the addressee
>>> and may
>>> > contain information which is confidential or privileged. If you
>>> > receive this electronic mail in error, please delete it from your
>>> > system immediately and notify the sender by electronic mail. Any
>>> > opinion expressed in this email is not represented as the
>>> opinion of
>>> > Australian Communication Limited unless that is stated or
>>> apparent from its terms.
>>> > ------------------------------
>>> >
>>> >
>>> > _______________________________________________
>>> > Ntop mailing list
>>> > Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>> > http://listgateway.unipi.it/mailman/listinfo/ntop
>>> <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>> >
>>> -------------- next part --------------
>>> An HTML attachment was scrubbed...
>>> URL:
>>> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0002.htm
>>> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0002.htm>>
>>>
>>> ------------------------------
>>>
>>> Message: 2
>>> Date: Tue, 4 Jul 2017 14:25:19 +0200
>>> From: Simone Mainardi <mainardi@ntop.org <mailto:mainardi@ntop.org>>
>>> To: ntop@unipi.it <mailto:ntop@unipi.it>
>>> Cc: "ntop@listgateway.unipi.it
>>> <mailto:ntop@listgateway.unipi.it>" <ntop@listgateway.unipi.it
>>> <mailto:ntop@listgateway.unipi.it>>
>>> Subject: Re: [Ntop] Netflow (NSEL) updates from Cisco ASA
>>> Message-ID:
>>>
>>> <CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com
>>> <mailto:CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com>>
>>> Content-Type: text/plain; charset="utf-8"
>>>
>>> Pelham,
>>>
>>> ASA flow-updates are received and processed by nprobe. However,
>>> I am not sure they contain all the necessary information
>>> required to properly update flow statistics. Can you please
>>> generate and send a .pcap capture file of your ASA netflow (make
>>> sure it contains both templates and data records for
>>> flow-updates and flow-teardown) for our inspection?
>>>
>>> By the way, nprobe gives you a couple of configurable timeout
>>> that you can use to periodically export long-lived flows:
>>>
>>> [--lifetime-timeout|-t] <timeout> | It specifies the maximum
>>> (seconds)
>>> flow
>>> | lifetime [default=120]
>>> [--idle-timeout|-d] <timeout> | It specifies the maximum
>>> (seconds)
>>> flow
>>> | idle lifetime [default=30]
>>>
>>> Regards,
>>> Simone
>>>
>>> On Wed, Jun 28, 2017 at 2:38 AM, Pelham Whitmore <
>>> Pelham.Whitmore@aceinfo.net.au
>>> <mailto:Pelham.Whitmore@aceinfo.net.au>> wrote:
>>>
>>> > Hello,
>>> >
>>> >
>>> >
>>> > I have a Cisco ASA configured to send Netflow to an instance of
>>> > nprobe, and then on to ntopng.
>>> >
>>> > The configuration is working, however I have noticed that
>>> nprobe is
>>> > only emitting flows when it receives a flow-teardown event
>>> from the
>>> > ASA. This is causing inaccurate bandwidth reporting for long-lived
>>> > flows as the total byte count is being recorded as a single
>>> spike once the flow is torn down.
>>> >
>>> >
>>> >
>>> > My understanding is that Cisco ASA netflow is very
>>> non-standard and
>>> > that this behaviour used to be expected on older version of ASA.
>>> > However, newer versions of ASA are capable of sending flow-update
>>> > events using a refresh-interval for active flows. When I run
>>> tcpdump
>>> > on my nprobe server I can see the flow-create and flow-update
>>> events
>>> > being sent from the ASA, however nprobe does not seem to use these
>>> > events, or act on them in any way. I have enabled verbose
>>> logging, but
>>> > can only see logs being generated for flow-teardown events,
>>> not flow-create or flow-update.
>>> >
>>> >
>>> >
>>> > My question is, should I expect nprobe to use the flow-updates
>>> from
>>> > Cisco ASA for long-lived active flows, or is it normal for it
>>> to only
>>> > process flow-teardown events?
>>> >
>>> >
>>> >
>>> > Nprobe (dev build v.8.1.170626) is running in collector mode
>>> with the
>>> > following settings:
>>> >
>>> >
>>> >
>>> > --zmq="tcp://*:5559"
>>> >
>>> > --collector-port=2055
>>> >
>>> > -i=none
>>> >
>>> > -n=none
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > Regards,
>>> >
>>> > Pelham
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > ------------------------------
>>> >
>>> > This electronic mail is solely for the use of the addressee
>>> and may
>>> > contain information which is confidential or privileged. If you
>>> > receive this electronic mail in error, please delete it from your
>>> > system immediately and notify the sender by electronic mail. Any
>>> > opinion expressed in this email is not represented as the
>>> opinion of
>>> > Australian Communication Limited unless that is stated or
>>> apparent from its terms.
>>> > ------------------------------
>>> >
>>> >
>>> > _______________________________________________
>>> > Ntop mailing list
>>> > Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>> > http://listgateway.unipi.it/mailman/listinfo/ntop
>>> <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>> >
>>> -------------- next part --------------
>>> An HTML attachment was scrubbed...
>>> URL:
>>> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0003.htm
>>> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0003.htm>>
>>>
>>> ------------------------------
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>> <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>
>>> End of Ntop Digest, Vol 158, Issue 1
>>> ************************************
>>> This electronic mail is solely for the use of the addressee and
>>> may contain information which is confidential or privileged.
>>> If you receive this electronic mail in error, please delete it
>>> from your system immediately and notify the sender by electronic
>>> mail.
>>> Any opinion expressed in this email is not represented as the
>>> opinion of Australian Communication Exchange Limited unless that
>>> is stated or
>>> apparent from its terms.
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>> <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop

Re: Netflow (NSEL) updates from Cisco ASA [ In reply to ]

mkh at rqc

Jul 15, 2017, 10:13 AM

Post #11 of 11 (2705 views)

Permalink

Hello Luca,

I've opened an issue on github as you requested:
https://github.com/ntop/ntopng/issues/1359 . Please contact me off-list
if you need full .pcap or other data.

--

With Best Regards,
Marat Khalili

On 13/07/17 10:27, Marat Khalili wrote:
>
>> ASA (and other devices like PaloAlto) generate flows based on
>> firewall events, like flow create, deletion etc. and do not send
>> interim updates as flow devices do. ASA for instance sends only total
>> octets.
> According to documentation
> <https://www.cisco.com/c/en/us/td/docs/security/asa/special/netflow/guide/asa_netflow.html#pgfId-1324322>
> there are Flow-Update events sent every minute by default.
>> Shall you have further comments on this matter, please file ticket on
>> github so we can keep a log of the discussion
> Ok, I'll re-test it my environment and open the ticket with what I'm
> seeing.
> --
>
> With Best Regards,
> Marat Khalili
>
> On 13/07/17 01:26, Luca Deri wrote:
>> Hi all,
>> ASA (and other devices like PaloAlto) generate flows based on
>> firewall events, like flow create, deletion etc. and do not send
>> interim updates as flow devices do. ASA for instance sends only total
>> octets.
>>
>> Due to this if we take into account flow updates we need to make sure
>> that the frequency of these updates is greater than the flow
>> inactivity timeout in ntopng as otherwise we would generate incorrect
>> values (i.e. we will account more traffic than the one that has
>> really passed on the network). The flow delete instead is not
>> affected by this problem as it is the very last message emitted for a
>> given flow.
>>
>> This said, reading this thread it seems that you all have different
>> option and this I have modified nProbe to implement the behaviour you
>> expect. Shall you have further comments on this matter, please file
>> ticket on github so we can keep a log of the discussion
>>
>> Regards Luca
>>
>>> On 6 Jul 2017, at 14:13, Marat Khalili <mkh@rqc.ru
>>> <mailto:mkh@rqc.ru>> wrote:
>>>
>>> Hello Simone,
>>>
>>> I'm also very interested in this issue, observing same behaviour as
>>> Pelham. Do you mean we need a custom fields configuration file to
>>> make this work? How will it be different from your attempts where
>>> "events other than flow-delete often contain to few attributes to
>>> properly update the flow"?
>>>
>>> --
>>>
>>> With Best Regards,
>>> Marat Khalili
>>>
>>> On 06/07/17 14:54, Simone Mainardi wrote:
>>>> Dear Pelham,
>>>>
>>>> Thanks for sharing the pcap. Currently, in nProbe we deliberately
>>>> ignore ASA firewall events different from flow-delete. We have made
>>>> this choice as we have seen that events other than flow-delete
>>>> often contain to few attributes to properly update the flow. This
>>>> is the reason why other templates are silently ignored by nprobe.
>>>>
>>>> If you want your nProbe to proxy all the templates received as-is,
>>>> you may want to have a look at:
>>>> http://www.ntop.org/nprobe/collecting-proprietary-flows-with-nprobe/
>>>>
>>>>
>>>> Simone
>>>>
>>>> Regards,
>>>> Simone
>>>>
>>>>
>>>>
>>>> On Thu, Jul 6, 2017 at 3:55 AM, Pelham Whitmore
>>>> <Pelham.Whitmore@aceinfo.net.au
>>>> <mailto:Pelham.Whitmore@aceinfo.net.au>> wrote:
>>>>
>>>> Hey Simone,
>>>>
>>>> Thanks for the reply. I have configured the timeout values you
>>>> mentioned in nprobe however it seemed to have no effect.
>>>> I have generated a .pcap file that includes templates, flow
>>>> create, update, and teardown events.
>>>>
>>>> .pcap file can be downloaded from here:
>>>> https://files.aceinfo.net.au/index.php/s/bj0aFU3lpyUjqM4/download
>>>> <https://files.aceinfo.net.au/index.php/s/bj0aFU3lpyUjqM4/download>
>>>>
>>>> One thing I did notcie from the packet capture is that flow
>>>> update events are being listed as "Firewall Event: Unknown (5)" .
>>>> I'm not sure if that is to be expected.
>>>>
>>>>
>>>> Regards,
>>>> Pelham Whitmore
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: ntop-bounces@listgateway.unipi.it
>>>> <mailto:ntop-bounces@listgateway.unipi.it>
>>>> [mailto:ntop-bounces@listgateway.unipi.it
>>>> <mailto:ntop-bounces@listgateway.unipi.it>] On Behalf Of
>>>> ntop-request@listgateway.unipi.it
>>>> <mailto:ntop-request@listgateway.unipi.it>
>>>> Sent: 05 July 2017 8:00 PM
>>>> To: ntop@listgateway.unipi.it <mailto:ntop@listgateway.unipi.it>
>>>> Subject: Ntop Digest, Vol 158, Issue 1
>>>>
>>>> Send Ntop mailing list submissions to
>>>> ntop@listgateway.unipi.it <mailto:ntop@listgateway.unipi.it>
>>>>
>>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>> <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>> or, via email, send a message with subject or body 'help' to
>>>> ntop-request@listgateway.unipi.it
>>>> <mailto:ntop-request@listgateway.unipi.it>
>>>>
>>>> You can reach the person managing the list at
>>>> ntop-owner@listgateway.unipi.it
>>>> <mailto:ntop-owner@listgateway.unipi.it>
>>>>
>>>> When replying, please edit your Subject line so it is more
>>>> specific than "Re: Contents of Ntop digest..."
>>>>
>>>>
>>>> Today's Topics:
>>>>
>>>> 1. Re: Netflow (NSEL) updates from Cisco ASA (Simone Mainardi)
>>>> 2. Re: Netflow (NSEL) updates from Cisco ASA (Simone Mainardi)
>>>>
>>>>
>>>> ----------------------------------------------------------------------
>>>>
>>>> Message: 1
>>>> Date: Tue, 4 Jul 2017 14:25:19 +0200
>>>> From: Simone Mainardi <mainardi@ntop.org
>>>> <mailto:mainardi@ntop.org>>
>>>> To: ntop@unipi.it <mailto:ntop@unipi.it>
>>>> Cc: "ntop@listgateway.unipi.it
>>>> <mailto:ntop@listgateway.unipi.it>" <ntop@listgateway.unipi.it
>>>> <mailto:ntop@listgateway.unipi.it>>
>>>> Subject: Re: [Ntop] Netflow (NSEL) updates from Cisco ASA
>>>> Message-ID:
>>>>
>>>> <CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com
>>>> <mailto:CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com>>
>>>> Content-Type: text/plain; charset="utf-8"
>>>>
>>>> Pelham,
>>>>
>>>> ASA flow-updates are received and processed by nprobe. However,
>>>> I am not sure they contain all the necessary information
>>>> required to properly update flow statistics. Can you please
>>>> generate and send a .pcap capture file of your ASA netflow
>>>> (make sure it contains both templates and data records for
>>>> flow-updates and flow-teardown) for our inspection?
>>>>
>>>> By the way, nprobe gives you a couple of configurable timeout
>>>> that you can use to periodically export long-lived flows:
>>>>
>>>> [--lifetime-timeout|-t] <timeout> | It specifies the maximum
>>>> (seconds)
>>>> flow
>>>> | lifetime [default=120]
>>>> [--idle-timeout|-d] <timeout> | It specifies the maximum (seconds)
>>>> flow
>>>> | idle lifetime [default=30]
>>>>
>>>> Regards,
>>>> Simone
>>>>
>>>> On Wed, Jun 28, 2017 at 2:38 AM, Pelham Whitmore <
>>>> Pelham.Whitmore@aceinfo.net.au
>>>> <mailto:Pelham.Whitmore@aceinfo.net.au>> wrote:
>>>>
>>>> > Hello,
>>>> >
>>>> >
>>>> >
>>>> > I have a Cisco ASA configured to send Netflow to an instance of
>>>> > nprobe, and then on to ntopng.
>>>> >
>>>> > The configuration is working, however I have noticed that
>>>> nprobe is
>>>> > only emitting flows when it receives a flow-teardown event
>>>> from the
>>>> > ASA. This is causing inaccurate bandwidth reporting for
>>>> long-lived
>>>> > flows as the total byte count is being recorded as a single
>>>> spike once the flow is torn down.
>>>> >
>>>> >
>>>> >
>>>> > My understanding is that Cisco ASA netflow is very
>>>> non-standard and
>>>> > that this behaviour used to be expected on older version of ASA.
>>>> > However, newer versions of ASA are capable of sending flow-update
>>>> > events using a refresh-interval for active flows. When I run
>>>> tcpdump
>>>> > on my nprobe server I can see the flow-create and flow-update
>>>> events
>>>> > being sent from the ASA, however nprobe does not seem to use
>>>> these
>>>> > events, or act on them in any way. I have enabled verbose
>>>> logging, but
>>>> > can only see logs being generated for flow-teardown events,
>>>> not flow-create or flow-update.
>>>> >
>>>> >
>>>> >
>>>> > My question is, should I expect nprobe to use the
>>>> flow-updates from
>>>> > Cisco ASA for long-lived active flows, or is it normal for it
>>>> to only
>>>> > process flow-teardown events?
>>>> >
>>>> >
>>>> >
>>>> > Nprobe (dev build v.8.1.170626) is running in collector mode
>>>> with the
>>>> > following settings:
>>>> >
>>>> >
>>>> >
>>>> > --zmq="tcp://*:5559"
>>>> >
>>>> > --collector-port=2055
>>>> >
>>>> > -i=none
>>>> >
>>>> > -n=none
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > Regards,
>>>> >
>>>> > Pelham
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > ------------------------------
>>>> >
>>>> > This electronic mail is solely for the use of the addressee
>>>> and may
>>>> > contain information which is confidential or privileged. If you
>>>> > receive this electronic mail in error, please delete it from your
>>>> > system immediately and notify the sender by electronic mail. Any
>>>> > opinion expressed in this email is not represented as the
>>>> opinion of
>>>> > Australian Communication Limited unless that is stated or
>>>> apparent from its terms.
>>>> > ------------------------------
>>>> >
>>>> >
>>>> > _______________________________________________
>>>> > Ntop mailing list
>>>> > Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>> > http://listgateway.unipi.it/mailman/listinfo/ntop
>>>> <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>> >
>>>> -------------- next part --------------
>>>> An HTML attachment was scrubbed...
>>>> URL:
>>>> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0002.htm
>>>> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0002.htm>>
>>>>
>>>> ------------------------------
>>>>
>>>> Message: 2
>>>> Date: Tue, 4 Jul 2017 14:25:19 +0200
>>>> From: Simone Mainardi <mainardi@ntop.org
>>>> <mailto:mainardi@ntop.org>>
>>>> To: ntop@unipi.it <mailto:ntop@unipi.it>
>>>> Cc: "ntop@listgateway.unipi.it
>>>> <mailto:ntop@listgateway.unipi.it>" <ntop@listgateway.unipi.it
>>>> <mailto:ntop@listgateway.unipi.it>>
>>>> Subject: Re: [Ntop] Netflow (NSEL) updates from Cisco ASA
>>>> Message-ID:
>>>>
>>>> <CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com
>>>> <mailto:CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com>>
>>>> Content-Type: text/plain; charset="utf-8"
>>>>
>>>> Pelham,
>>>>
>>>> ASA flow-updates are received and processed by nprobe. However,
>>>> I am not sure they contain all the necessary information
>>>> required to properly update flow statistics. Can you please
>>>> generate and send a .pcap capture file of your ASA netflow
>>>> (make sure it contains both templates and data records for
>>>> flow-updates and flow-teardown) for our inspection?
>>>>
>>>> By the way, nprobe gives you a couple of configurable timeout
>>>> that you can use to periodically export long-lived flows:
>>>>
>>>> [--lifetime-timeout|-t] <timeout> | It specifies the maximum
>>>> (seconds)
>>>> flow
>>>> | lifetime [default=120]
>>>> [--idle-timeout|-d] <timeout> | It specifies the maximum (seconds)
>>>> flow
>>>> | idle lifetime [default=30]
>>>>
>>>> Regards,
>>>> Simone
>>>>
>>>> On Wed, Jun 28, 2017 at 2:38 AM, Pelham Whitmore <
>>>> Pelham.Whitmore@aceinfo.net.au
>>>> <mailto:Pelham.Whitmore@aceinfo.net.au>> wrote:
>>>>
>>>> > Hello,
>>>> >
>>>> >
>>>> >
>>>> > I have a Cisco ASA configured to send Netflow to an instance of
>>>> > nprobe, and then on to ntopng.
>>>> >
>>>> > The configuration is working, however I have noticed that
>>>> nprobe is
>>>> > only emitting flows when it receives a flow-teardown event
>>>> from the
>>>> > ASA. This is causing inaccurate bandwidth reporting for
>>>> long-lived
>>>> > flows as the total byte count is being recorded as a single
>>>> spike once the flow is torn down.
>>>> >
>>>> >
>>>> >
>>>> > My understanding is that Cisco ASA netflow is very
>>>> non-standard and
>>>> > that this behaviour used to be expected on older version of ASA.
>>>> > However, newer versions of ASA are capable of sending flow-update
>>>> > events using a refresh-interval for active flows. When I run
>>>> tcpdump
>>>> > on my nprobe server I can see the flow-create and flow-update
>>>> events
>>>> > being sent from the ASA, however nprobe does not seem to use
>>>> these
>>>> > events, or act on them in any way. I have enabled verbose
>>>> logging, but
>>>> > can only see logs being generated for flow-teardown events,
>>>> not flow-create or flow-update.
>>>> >
>>>> >
>>>> >
>>>> > My question is, should I expect nprobe to use the
>>>> flow-updates from
>>>> > Cisco ASA for long-lived active flows, or is it normal for it
>>>> to only
>>>> > process flow-teardown events?
>>>> >
>>>> >
>>>> >
>>>> > Nprobe (dev build v.8.1.170626) is running in collector mode
>>>> with the
>>>> > following settings:
>>>> >
>>>> >
>>>> >
>>>> > --zmq="tcp://*:5559"
>>>> >
>>>> > --collector-port=2055
>>>> >
>>>> > -i=none
>>>> >
>>>> > -n=none
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > Regards,
>>>> >
>>>> > Pelham
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > ------------------------------
>>>> >
>>>> > This electronic mail is solely for the use of the addressee
>>>> and may
>>>> > contain information which is confidential or privileged. If you
>>>> > receive this electronic mail in error, please delete it from your
>>>> > system immediately and notify the sender by electronic mail. Any
>>>> > opinion expressed in this email is not represented as the
>>>> opinion of
>>>> > Australian Communication Limited unless that is stated or
>>>> apparent from its terms.
>>>> > ------------------------------
>>>> >
>>>> >
>>>> > _______________________________________________
>>>> > Ntop mailing list
>>>> > Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>> > http://listgateway.unipi.it/mailman/listinfo/ntop
>>>> <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>> >
>>>> -------------- next part --------------
>>>> An HTML attachment was scrubbed...
>>>> URL:
>>>> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0003.htm
>>>> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0003.htm>>
>>>>
>>>> ------------------------------
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>> <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>>
>>>> End of Ntop Digest, Vol 158, Issue 1
>>>> ************************************
>>>> This electronic mail is solely for the use of the addressee and
>>>> may contain information which is confidential or privileged.
>>>> If you receive this electronic mail in error, please delete it
>>>> from your system immediately and notify the sender by
>>>> electronic mail.
>>>> Any opinion expressed in this email is not represented as the
>>>> opinion of Australian Communication Exchange Limited unless
>>>> that is stated or
>>>> apparent from its terms.
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>> <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop