Mailing List Archive

Incorrect timestamp on emitted flows
Hi there,

we use nProbe Pro to provide customers with flow data filtered to only their ASN flows.
Customer uses Andrisoft Wansight for visualization and Wansight complains about flows coming from the future.

After capturing flows via nfcapd from before and after nProbe processing and dumping them with nfdump I noticed the following:

- Before Flows contain timestamp.microseconds

- After Flows contain timestamp.000

- nProbe seems to be rounding up to the next full second

- nProbe is adding 60 seconds to the timestamp as well

I filtered out one IP and used Excel to sort the output by DstPort to make it easier to compare. It was totally consistent with always 1 minute added + rounded to next full second.
Which correlates with our customer reporting flows are between 1 and 55 seconds from the future.

These are our nProbe parameters:
nprobe --sender-address <ip>:2055 --collector-port 2056 --collector <ip>:10000 --flow-version 9 --sample-rate @5000:1:1 --interface none --verbose 1 --in-iface-idx 910 --out-iface-idx 917 -min-num-flows 1 --flows-intra-templ=1

Default ?timestamp-format seems to be 1. When changing it to 0, nfdump only gets 1st Jan 1970 as timestamp.

I tested this on v.8.5.180523 but this seems also to be with v.8.3.180327

I guess this is a bug or are there any options I am missing that would be causing this?


Best regards,

Benjamin Weik
Re: Incorrect timestamp on emitted flows [ In reply to ]
Benjamin,

Thanks for reporting. We've done some changes and fixes that should have addressed the behavior you've reported. Please, hold on until tomorrow for the new build to be available, and then update to the latest 8.5 version.

Simone


> On 24 May 2018, at 12:14, Benjamin Weik <Benjamin.Weik@core-backbone.com> wrote:
>
> Hi there,
>
> we use nProbe Pro to provide customers with flow data filtered to only their ASN flows.
> Customer uses Andrisoft Wansight for visualization and Wansight complains about flows coming from the future.
>
> After capturing flows via nfcapd from before and after nProbe processing and dumping them with nfdump I noticed the following:
> - Before Flows contain timestamp.microseconds
> - After Flows contain timestamp.000
> - nProbe seems to be rounding up to the next full second
> - nProbe is adding 60 seconds to the timestamp as well
>
> I filtered out one IP and used Excel to sort the output by DstPort to make it easier to compare. It was totally consistent with always 1 minute added + rounded to next full second.
> Which correlates with our customer reporting flows are between 1 and 55 seconds from the future.
>
> These are our nProbe parameters:
> nprobe --sender-address <ip>:2055 --collector-port 2056 --collector <ip>:10000 --flow-version 9 --sample-rate @5000:1:1 --interface none --verbose 1 --in-iface-idx 910 --out-iface-idx 917 -min-num-flows 1 --flows-intra-templ=1
>
> Default –timestamp-format seems to be 1. When changing it to 0, nfdump only gets 1st Jan 1970 as timestamp.
>
> I tested this on v.8.5.180523 but this seems also to be with v.8.3.180327
>
> I guess this is a bug or are there any options I am missing that would be causing this?
>
>
> Best regards,
>
> Benjamin Weik
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it <mailto:Ntop-misc@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc <http://listgateway.unipi.it/mailman/listinfo/ntop-misc>
Re: Incorrect timestamp on emitted flows [ In reply to ]
Benjamin, I was wondering if you had the chance to try the latest nprobe. Please, let me know.

> On 25 May 2018, at 17:53, Simone Mainardi <mainardi@ntop.org> wrote:
>
> Benjamin,
>
> Thanks for reporting. We've done some changes and fixes that should have addressed the behavior you've reported. Please, hold on until tomorrow for the new build to be available, and then update to the latest 8.5 version.
>
> Simone
>
>
>> On 24 May 2018, at 12:14, Benjamin Weik <Benjamin.Weik@core-backbone.com <mailto:Benjamin.Weik@core-backbone.com>> wrote:
>>
>> Hi there,
>>
>> we use nProbe Pro to provide customers with flow data filtered to only their ASN flows.
>> Customer uses Andrisoft Wansight for visualization and Wansight complains about flows coming from the future.
>>
>> After capturing flows via nfcapd from before and after nProbe processing and dumping them with nfdump I noticed the following:
>> - Before Flows contain timestamp.microseconds
>> - After Flows contain timestamp.000
>> - nProbe seems to be rounding up to the next full second
>> - nProbe is adding 60 seconds to the timestamp as well
>>
>> I filtered out one IP and used Excel to sort the output by DstPort to make it easier to compare. It was totally consistent with always 1 minute added + rounded to next full second.
>> Which correlates with our customer reporting flows are between 1 and 55 seconds from the future.
>>
>> These are our nProbe parameters:
>> nprobe --sender-address <ip>:2055 --collector-port 2056 --collector <ip>:10000 --flow-version 9 --sample-rate @5000:1:1 --interface none --verbose 1 --in-iface-idx 910 --out-iface-idx 917 -min-num-flows 1 --flows-intra-templ=1
>>
>> Default –timestamp-format seems to be 1. When changing it to 0, nfdump only gets 1st Jan 1970 as timestamp.
>>
>> I tested this on v.8.5.180523 but this seems also to be with v.8.3.180327
>>
>> I guess this is a bug or are there any options I am missing that would be causing this?
>>
>>
>> Best regards,
>>
>> Benjamin Weik
>> _______________________________________________
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it <mailto:Ntop-misc@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc <http://listgateway.unipi.it/mailman/listinfo/ntop-misc>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: Incorrect timestamp on emitted flows [ In reply to ]
Hi Simone,

I installed the new version today and so far it’s looking good.

Best regards,

Benjamin

Von: ntop-misc-bounces@listgateway.unipi.it <ntop-misc-bounces@listgateway.unipi.it> Im Auftrag von Simone Mainardi
Gesendet: Dienstag, 29. Mai 2018 12:32
An: ntop-misc@listgateway.unipi.it
Betreff: Re: [Ntop-misc] Incorrect timestamp on emitted flows

Benjamin, I was wondering if you had the chance to try the latest nprobe. Please, let me know.


On 25 May 2018, at 17:53, Simone Mainardi <mainardi@ntop.org<mailto:mainardi@ntop.org>> wrote:

Benjamin,

Thanks for reporting. We've done some changes and fixes that should have addressed the behavior you've reported. Please, hold on until tomorrow for the new build to be available, and then update to the latest 8.5 version.

Simone



On 24 May 2018, at 12:14, Benjamin Weik <Benjamin.Weik@core-backbone.com<mailto:Benjamin.Weik@core-backbone.com>> wrote:

Hi there,

we use nProbe Pro to provide customers with flow data filtered to only their ASN flows.
Customer uses Andrisoft Wansight for visualization and Wansight complains about flows coming from the future.

After capturing flows via nfcapd from before and after nProbe processing and dumping them with nfdump I noticed the following:
- Before Flows contain timestamp.microseconds
- After Flows contain timestamp.000
- nProbe seems to be rounding up to the next full second
- nProbe is adding 60 seconds to the timestamp as well

I filtered out one IP and used Excel to sort the output by DstPort to make it easier to compare. It was totally consistent with always 1 minute added + rounded to next full second.
Which correlates with our customer reporting flows are between 1 and 55 seconds from the future.

These are our nProbe parameters:
nprobe --sender-address <ip>:2055 --collector-port 2056 --collector <ip>:10000 --flow-version 9 --sample-rate @5000:1:1 --interface none --verbose 1 --in-iface-idx 910 --out-iface-idx 917 -min-num-flows 1 --flows-intra-templ=1

Default –timestamp-format seems to be 1. When changing it to 0, nfdump only gets 1st Jan 1970 as timestamp.

I tested this on v.8.5.180523 but this seems also to be with v.8.3.180327

I guess this is a bug or are there any options I am missing that would be causing this?


Best regards,

Benjamin Weik
_______________________________________________
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it<mailto:Ntop-misc@listgateway.unipi.it>
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

_______________________________________________
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it<mailto:Ntop-misc@listgateway.unipi.it>
http://listgateway.unipi.it/mailman/listinfo/ntop-misc