Mailing List Archive: uburst log headed with 0s

uburst log headed with 0s

May 31, 2018, 8:32 AM

Post #1 of 9 (1943 views)

we have uburst monitoring enabled, but our logs always seem to get headed with a pile of NULs. It seems to be being written as a sparse file so the file doesn't take up much disk, but it's awfully hard to parse. this is with 3.1.180226.

--interface fbcard:0:a:00
-P=/var/run/n2disk-fbcard:0:a:00.pid
-s 9000
--chunk-len 128
--buffer-len 4096
--dump-directory /srv/pcaps/7ticks-A/dumps
--max-file-len 512
--max-file-duration 900
--file-prefix fbcard:0:a:0-
--max-nested-dirs 10
--max-num-files 1000
--hugepages
--timeline-dir /srv/pcaps/7ticks-A/timeline
--reader-cpu-affinity 10
--writer-cpu-affinity 12
--compressor-cpu-affinity 32,34,36,38
--uburst-detection
--uburst-log /var/log/n2disk/uburst-fbcard:0:a:0.log
--uburst-win-size 100
--uburst-link-speed 10000
--uburst-threshold 80
--pcap-compression
--cluster-id 1
--index
--index-on-compressor-threads
--nanoseconds

any thoughts?
_______________________________________________
Ntop mailing list
Ntop@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop

Re: uburst log headed with 0s [ In reply to ]

cardigliano at ntop

May 31, 2018, 8:36 AM

Post #2 of 9 (1943 views)

Permalink

Hi Jeff
could you send me a log file to check what’s happening? I am not sure I understood.

Thank you
Alfredo

> On 31 May 2018, at 17:32, Jeff Bacon <bacon@walleyesoftware.com> wrote:
>
> we have uburst monitoring enabled, but our logs always seem to get headed with a pile of NULs. It seems to be being written as a sparse file so the file doesn't take up much disk, but it's awfully hard to parse. this is with 3.1.180226.
>
> --interface fbcard:0:a:00
> -P=/var/run/n2disk-fbcard:0:a:00.pid
> -s 9000
> --chunk-len 128
> --buffer-len 4096
> --dump-directory /srv/pcaps/7ticks-A/dumps
> --max-file-len 512
> --max-file-duration 900
> --file-prefix fbcard:0:a:0-
> --max-nested-dirs 10
> --max-num-files 1000
> --hugepages
> --timeline-dir /srv/pcaps/7ticks-A/timeline
> --reader-cpu-affinity 10
> --writer-cpu-affinity 12
> --compressor-cpu-affinity 32,34,36,38
> --uburst-detection
> --uburst-log /var/log/n2disk/uburst-fbcard:0:a:0.log
> --uburst-win-size 100
> --uburst-link-speed 10000
> --uburst-threshold 80
> --pcap-compression
> --cluster-id 1
> --index
> --index-on-compressor-threads
> --nanoseconds
>
> any thoughts?
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop

Re: uburst log headed with 0s [ In reply to ]

cardigliano at ntop

May 31, 2018, 8:36 AM

Post #3 of 9 (1943 views)

Permalink

Re: uburst log headed with 0s [ In reply to ]

cardigliano at ntop

Jun 1, 2018, 9:11 AM

Post #4 of 9 (1942 views)

Permalink

Hi Jeff
I tried to reproduce this with no success, that means it happens under some condition,
anyway I made some change to the code to make sure it flushes the buffer correctly,
new packages will be available soon (30 min from now), please check if you still see
this issue and let me know.

Thank you
Alfredo

> On 31 May 2018, at 17:36, Alfredo Cardigliano <cardigliano@ntop.org> wrote:
>
> Hi Jeff
> could you send me a log file to check what’s happening? I am not sure I understood.
>
> Thank you
> Alfredo
>
>> On 31 May 2018, at 17:32, Jeff Bacon <bacon@walleyesoftware.com> wrote:
>>
>> we have uburst monitoring enabled, but our logs always seem to get headed with a pile of NULs. It seems to be being written as a sparse file so the file doesn't take up much disk, but it's awfully hard to parse. this is with 3.1.180226.
>>
>> --interface fbcard:0:a:00
>> -P=/var/run/n2disk-fbcard:0:a:00.pid
>> -s 9000
>> --chunk-len 128
>> --buffer-len 4096
>> --dump-directory /srv/pcaps/7ticks-A/dumps
>> --max-file-len 512
>> --max-file-duration 900
>> --file-prefix fbcard:0:a:0-
>> --max-nested-dirs 10
>> --max-num-files 1000
>> --hugepages
>> --timeline-dir /srv/pcaps/7ticks-A/timeline
>> --reader-cpu-affinity 10
>> --writer-cpu-affinity 12
>> --compressor-cpu-affinity 32,34,36,38
>> --uburst-detection
>> --uburst-log /var/log/n2disk/uburst-fbcard:0:a:0.log
>> --uburst-win-size 100
>> --uburst-link-speed 10000
>> --uburst-threshold 80
>> --pcap-compression
>> --cluster-id 1
>> --index
>> --index-on-compressor-threads
>> --nanoseconds
>>
>> any thoughts?
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop

Re: uburst log headed with 0s [ In reply to ]

cardigliano at ntop

Jun 1, 2018, 9:11 AM

Post #5 of 9 (1942 views)

Permalink

Re: uburst log headed with 0s [ In reply to ]

bacon at walleyesoftware

Jun 4, 2018, 9:40 AM

Post #6 of 9 (1935 views)

Permalink

will this work with the fiberblaze 3.1.0.1 release still?

> -----Original Message-----
> From: ntop-bounces@listgateway.unipi.it [mailto:ntop-
> bounces@listgateway.unipi.it] On Behalf Of Alfredo Cardigliano
> Sent: Friday, June 1, 2018 12:12 PM
> To: ntop@unipi.it
> Cc: ntop@listgateway.unipi.it
> Subject: Re: [Ntop] uburst log headed with 0s
>
> Hi Jeff
> I tried to reproduce this with no success, that means it happens under
> some condition,
> anyway I made some change to the code to make sure it flushes the
> buffer correctly,
> new packages will be available soon (30 min from now), please check if
> you still see
> this issue and let me know.
>
> Thank you
> Alfredo
>
> > On 31 May 2018, at 17:36, Alfredo Cardigliano <cardigliano@ntop.org>
> wrote:
> >
> > Hi Jeff
> > could you send me a log file to check what’s happening? I am not sure
> I understood.
> >
> > Thank you
> > Alfredo
> >
> >> On 31 May 2018, at 17:32, Jeff Bacon <bacon@walleyesoftware.com>
> wrote:
> >>
> >> we have uburst monitoring enabled, but our logs always seem to get
> headed with a pile of NULs. It seems to be being written as a sparse
> file so the file doesn't take up much disk, but it's awfully hard to
> parse. this is with 3.1.180226.
> >>
> >> --interface fbcard:0:a:00
> >> -P=/var/run/n2disk-fbcard:0:a:00.pid
> >> -s 9000
> >> --chunk-len 128
> >> --buffer-len 4096
> >> --dump-directory /srv/pcaps/7ticks-A/dumps
> >> --max-file-len 512
> >> --max-file-duration 900
> >> --file-prefix fbcard:0:a:0-
> >> --max-nested-dirs 10
> >> --max-num-files 1000
> >> --hugepages
> >> --timeline-dir /srv/pcaps/7ticks-A/timeline
> >> --reader-cpu-affinity 10
> >> --writer-cpu-affinity 12
> >> --compressor-cpu-affinity 32,34,36,38
> >> --uburst-detection
> >> --uburst-log /var/log/n2disk/uburst-fbcard:0:a:0.log
> >> --uburst-win-size 100
> >> --uburst-link-speed 10000
> >> --uburst-threshold 80
> >> --pcap-compression
> >> --cluster-id 1
> >> --index
> >> --index-on-compressor-threads
> >> --nanoseconds
> >>
> >> any thoughts?
> >> _______________________________________________
> >> Ntop mailing list
> >> Ntop@listgateway.unipi.it
> >> http://listgateway.unipi.it/mailman/listinfo/ntop
> >
> > _______________________________________________
> > Ntop mailing list
> > Ntop@listgateway.unipi.it
> > http://listgateway.unipi.it/mailman/listinfo/ntop

_______________________________________________
Ntop mailing list
Ntop@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop

Re: uburst log headed with 0s [ In reply to ]

cardigliano at ntop

Jun 4, 2018, 9:44 AM

Post #7 of 9 (1935 views)

Permalink

The binary is compiled with 3.1.1.1, I guess the API is the same, it should work.

Alfredo

> On 4 Jun 2018, at 18:40, Jeff Bacon <bacon@walleyesoftware.com> wrote:
>
> will this work with the fiberblaze 3.1.0.1 release still?
>
>> -----Original Message-----
>> From: ntop-bounces@listgateway.unipi.it [mailto:ntop-
>> bounces@listgateway.unipi.it] On Behalf Of Alfredo Cardigliano
>> Sent: Friday, June 1, 2018 12:12 PM
>> To: ntop@unipi.it
>> Cc: ntop@listgateway.unipi.it
>> Subject: Re: [Ntop] uburst log headed with 0s
>>
>> Hi Jeff
>> I tried to reproduce this with no success, that means it happens under
>> some condition,
>> anyway I made some change to the code to make sure it flushes the
>> buffer correctly,
>> new packages will be available soon (30 min from now), please check if
>> you still see
>> this issue and let me know.
>>
>> Thank you
>> Alfredo
>>
>>> On 31 May 2018, at 17:36, Alfredo Cardigliano <cardigliano@ntop.org>
>> wrote:
>>>
>>> Hi Jeff
>>> could you send me a log file to check what’s happening? I am not sure
>> I understood.
>>>
>>> Thank you
>>> Alfredo
>>>
>>>> On 31 May 2018, at 17:32, Jeff Bacon <bacon@walleyesoftware.com>
>> wrote:
>>>>
>>>> we have uburst monitoring enabled, but our logs always seem to get
>> headed with a pile of NULs. It seems to be being written as a sparse
>> file so the file doesn't take up much disk, but it's awfully hard to
>> parse. this is with 3.1.180226.
>>>>
>>>> --interface fbcard:0:a:00
>>>> -P=/var/run/n2disk-fbcard:0:a:00.pid
>>>> -s 9000
>>>> --chunk-len 128
>>>> --buffer-len 4096
>>>> --dump-directory /srv/pcaps/7ticks-A/dumps
>>>> --max-file-len 512
>>>> --max-file-duration 900
>>>> --file-prefix fbcard:0:a:0-
>>>> --max-nested-dirs 10
>>>> --max-num-files 1000
>>>> --hugepages
>>>> --timeline-dir /srv/pcaps/7ticks-A/timeline
>>>> --reader-cpu-affinity 10
>>>> --writer-cpu-affinity 12
>>>> --compressor-cpu-affinity 32,34,36,38
>>>> --uburst-detection
>>>> --uburst-log /var/log/n2disk/uburst-fbcard:0:a:0.log
>>>> --uburst-win-size 100
>>>> --uburst-link-speed 10000
>>>> --uburst-threshold 80
>>>> --pcap-compression
>>>> --cluster-id 1
>>>> --index
>>>> --index-on-compressor-threads
>>>> --nanoseconds
>>>>
>>>> any thoughts?
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop

Re: uburst log headed with 0s [ In reply to ]

bacon at walleyesoftware

Jul 16, 2018, 8:37 AM

Post #8 of 9 (1895 views)

Permalink

updated to v.3.1.180714, still happening, though not as egregiously -

> -----Original Message-----
> From: ntop-bounces@listgateway.unipi.it [mailto:ntop-
> bounces@listgateway.unipi.it] On Behalf Of Alfredo Cardigliano
> Sent: Monday, June 04, 2018 12:45 PM
> To: ntop@unipi.it
> Subject: Re: [Ntop] uburst log headed with 0s
>
> The binary is compiled with 3.1.1.1, I guess the API is the same, it should
> work.
>
> Alfredo
>
> > On 4 Jun 2018, at 18:40, Jeff Bacon <bacon@walleyesoftware.com> wrote:
> >
> > will this work with the fiberblaze 3.1.0.1 release still?
> >
> >> -----Original Message-----
> >> From: ntop-bounces@listgateway.unipi.it [mailto:ntop-
> >> bounces@listgateway.unipi.it] On Behalf Of Alfredo Cardigliano
> >> Sent: Friday, June 1, 2018 12:12 PM
> >> To: ntop@unipi.it
> >> Cc: ntop@listgateway.unipi.it
> >> Subject: Re: [Ntop] uburst log headed with 0s
> >>
> >> Hi Jeff
> >> I tried to reproduce this with no success, that means it happens under
> >> some condition,
> >> anyway I made some change to the code to make sure it flushes the
> >> buffer correctly,
> >> new packages will be available soon (30 min from now), please check if
> >> you still see
> >> this issue and let me know.
> >>
> >> Thank you
> >> Alfredo
> >>
> >>> On 31 May 2018, at 17:36, Alfredo Cardigliano <cardigliano@ntop.org>
> >> wrote:
> >>>
> >>> Hi Jeff
> >>> could you send me a log file to check what’s happening? I am not sure
> >> I understood.
> >>>
> >>> Thank you
> >>> Alfredo
> >>>
> >>>> On 31 May 2018, at 17:32, Jeff Bacon <bacon@walleyesoftware.com>
> >> wrote:
> >>>>
> >>>> we have uburst monitoring enabled, but our logs always seem to get
> >> headed with a pile of NULs. It seems to be being written as a sparse
> >> file so the file doesn't take up much disk, but it's awfully hard to
> >> parse. this is with 3.1.180226.
> >>>>
> >>>> --interface fbcard:0:a:00
> >>>> -P=/var/run/n2disk-fbcard:0:a:00.pid
> >>>> -s 9000
> >>>> --chunk-len 128
> >>>> --buffer-len 4096
> >>>> --dump-directory /srv/pcaps/7ticks-A/dumps
> >>>> --max-file-len 512
> >>>> --max-file-duration 900
> >>>> --file-prefix fbcard:0:a:0-
> >>>> --max-nested-dirs 10
> >>>> --max-num-files 1000
> >>>> --hugepages
> >>>> --timeline-dir /srv/pcaps/7ticks-A/timeline
> >>>> --reader-cpu-affinity 10
> >>>> --writer-cpu-affinity 12
> >>>> --compressor-cpu-affinity 32,34,36,38
> >>>> --uburst-detection
> >>>> --uburst-log /var/log/n2disk/uburst-fbcard:0:a:0.log
> >>>> --uburst-win-size 100
> >>>> --uburst-link-speed 10000
> >>>> --uburst-threshold 80
> >>>> --pcap-compression
> >>>> --cluster-id 1
> >>>> --index
> >>>> --index-on-compressor-threads
> >>>> --nanoseconds
> >>>>
> >>>> any thoughts?
> >>>> _______________________________________________
> >>>> Ntop mailing list
> >>>> Ntop@listgateway.unipi.it
> >>>> http://listgateway.unipi.it/mailman/listinfo/ntop
> >>>
> >>> _______________________________________________
> >>> Ntop mailing list
> >>> Ntop@listgateway.unipi.it
> >>> http://listgateway.unipi.it/mailman/listinfo/ntop
> >
> > _______________________________________________
> > Ntop mailing list
> > Ntop@listgateway.unipi.it
> > http://listgateway.unipi.it/mailman/listinfo/ntop

Re: uburst log headed with 0s [ In reply to ]

cardigliano at ntop

Jul 17, 2018, 7:21 AM

Post #9 of 9 (1893 views)

Permalink

Hi Jeff
it seems that on your machine the uburst logger is printing null characters in
place of the file header, I made some change, please update and let me know
if you see any difference.

Regards
Alfredo

> On 16 Jul 2018, at 17:37, Jeff Bacon <bacon@walleyesoftware.com> wrote:
>
> updated to v.3.1.180714, still happening, though not as egregiously -
>
>
>
>> -----Original Message-----
>> From: ntop-bounces@listgateway.unipi.it <mailto:ntop-bounces@listgateway.unipi.it> [mailto:ntop-
>> bounces@listgateway.unipi.it <mailto:bounces@listgateway.unipi.it>] On Behalf Of Alfredo Cardigliano
>> Sent: Monday, June 04, 2018 12:45 PM
>> To: ntop@unipi.it <mailto:ntop@unipi.it>
>> Subject: Re: [Ntop] uburst log headed with 0s
>>
>> The binary is compiled with 3.1.1.1, I guess the API is the same, it should
>> work.
>>
>> Alfredo
>>
>>> On 4 Jun 2018, at 18:40, Jeff Bacon <bacon@walleyesoftware.com> wrote:
>>>
>>> will this work with the fiberblaze 3.1.0.1 release still?
>>>
>>>> -----Original Message-----
>>>> From: ntop-bounces@listgateway.unipi.it [mailto:ntop-
>>>> bounces@listgateway.unipi.it] On Behalf Of Alfredo Cardigliano
>>>> Sent: Friday, June 1, 2018 12:12 PM
>>>> To: ntop@unipi.it
>>>> Cc: ntop@listgateway.unipi.it
>>>> Subject: Re: [Ntop] uburst log headed with 0s
>>>>
>>>> Hi Jeff
>>>> I tried to reproduce this with no success, that means it happens under
>>>> some condition,
>>>> anyway I made some change to the code to make sure it flushes the
>>>> buffer correctly,
>>>> new packages will be available soon (30 min from now), please check if
>>>> you still see
>>>> this issue and let me know.
>>>>
>>>> Thank you
>>>> Alfredo
>>>>
>>>>> On 31 May 2018, at 17:36, Alfredo Cardigliano <cardigliano@ntop.org>
>>>> wrote:
>>>>>
>>>>> Hi Jeff
>>>>> could you send me a log file to check what’s happening? I am not sure
>>>> I understood.
>>>>>
>>>>> Thank you
>>>>> Alfredo
>>>>>
>>>>>> On 31 May 2018, at 17:32, Jeff Bacon <bacon@walleyesoftware.com>
>>>> wrote:
>>>>>>
>>>>>> we have uburst monitoring enabled, but our logs always seem to get
>>>> headed with a pile of NULs. It seems to be being written as a sparse
>>>> file so the file doesn't take up much disk, but it's awfully hard to
>>>> parse. this is with 3.1.180226.
>>>>>>
>>>>>> --interface fbcard:0:a:00
>>>>>> -P=/var/run/n2disk-fbcard:0:a:00.pid
>>>>>> -s 9000
>>>>>> --chunk-len 128
>>>>>> --buffer-len 4096
>>>>>> --dump-directory /srv/pcaps/7ticks-A/dumps
>>>>>> --max-file-len 512
>>>>>> --max-file-duration 900
>>>>>> --file-prefix fbcard:0:a:0-
>>>>>> --max-nested-dirs 10
>>>>>> --max-num-files 1000
>>>>>> --hugepages
>>>>>> --timeline-dir /srv/pcaps/7ticks-A/timeline
>>>>>> --reader-cpu-affinity 10
>>>>>> --writer-cpu-affinity 12
>>>>>> --compressor-cpu-affinity 32,34,36,38
>>>>>> --uburst-detection
>>>>>> --uburst-log /var/log/n2disk/uburst-fbcard:0:a:0.log
>>>>>> --uburst-win-size 100
>>>>>> --uburst-link-speed 10000
>>>>>> --uburst-threshold 80
>>>>>> --pcap-compression
>>>>>> --cluster-id 1
>>>>>> --index
>>>>>> --index-on-compressor-threads
>>>>>> --nanoseconds
>>>>>>
>>>>>> any thoughts?
>>>>>> _______________________________________________
>>>>>> Ntop mailing list
>>>>>> Ntop@listgateway.unipi.it
>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>>
>>>>> _______________________________________________
>>>>> Ntop mailing list
>>>>> Ntop@listgateway.unipi.it
>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>
> <uburst.log.tar.gz>_______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>

Mailing List Archive

Attached Files:

Attached Files:

Attached Files:

Attached Files:

Attached Files:

Attached Files:

Attached Files: