Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. NTop>
  3. Misc
n2disk pcap output structure
jeremy.ashton at shopify
Apr 5, 2016, 11:04 AM
Post #1 of 8 (2317 views)
Permalink
I am trying to create a directory structure like the following:

/pcap/0/
/pcap/1/
…
/pcap/n/

Where each sub-directory contains 1,000 files. I would like to do this
without having to restart the file numbering. That way I can have a
<isp>-<file_number>.pcap which will always increment.

Any thoughts as to how I could approach this?

Thanks.
Re: n2disk pcap output structure [ In reply to ]
cardigliano at ntop
Apr 6, 2016, 6:37 AM
Post #2 of 8 (2302 views)
Permalink
Hi Jeremy
if I understand correctly you do not want to restart file numaration, this means that you have to move old files using an external script, otherwise the storage will be full at some point and n2disk will not be able to write new files anymore.
Correct?

Alfredo

> On 05 Apr 2016, at 20:04, Jeremy Ashton <jeremy.ashton@shopify.com> wrote:
>
> I am trying to create a directory structure like the following:
>
> /pcap/0/
> /pcap/1/
> …
> /pcap/n/
>
> Where each sub-directory contains 1,000 files. I would like to do this without having to restart the file numbering. That way I can have a <isp>-<file_number>.pcap which will always increment.
>
> Any thoughts as to how I could approach this?
>
> Thanks.
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Attached Files:

Re: n2disk pcap output structure [ In reply to ]
jeremy.ashton at shopify
Apr 6, 2016, 6:55 AM
Post #3 of 8 (2309 views)
Permalink
Yes, I am currently dealing with the long term retention issues by cleaning
up the disk space.

On Wed, Apr 6, 2016 at 9:37 AM, Alfredo Cardigliano <cardigliano@ntop.org>
wrote:

> Hi Jeremy
> if I understand correctly you do not want to restart file numaration, this
> means that you have to move old files using an external script, otherwise
> the storage will be full at some point and n2disk will not be able to write
> new files anymore.
> Correct?
>
> Alfredo
>
> > On 05 Apr 2016, at 20:04, Jeremy Ashton <jeremy.ashton@shopify.com>
> wrote:
> >
> > I am trying to create a directory structure like the following:
> >
> > /pcap/0/
> > /pcap/1/
> > …
> > /pcap/n/
> >
> > Where each sub-directory contains 1,000 files. I would like to do this
> without having to restart the file numbering. That way I can have a
> <isp>-<file_number>.pcap which will always increment.
> >
> > Any thoughts as to how I could approach this?
> >
> > Thanks.
> > _______________________________________________
> > Ntop-misc mailing list
> > Ntop-misc@listgateway.unipi.it
> > http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
Re: n2disk pcap output structure [ In reply to ]
cardigliano at ntop
Apr 7, 2016, 12:53 AM
Post #4 of 8 (2303 views)
Permalink
Hi Jeremy
please update to latest version from the dev repo, you can specify -n -1 to create unlimited directories, please test it and let as know if it behaves as expected.

Alfredo

> On 06 Apr 2016, at 15:55, Jeremy Ashton <jeremy.ashton@shopify.com> wrote:
>
> Yes, I am currently dealing with the long term retention issues by cleaning up the disk space.
>
> On Wed, Apr 6, 2016 at 9:37 AM, Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>> wrote:
> Hi Jeremy
> if I understand correctly you do not want to restart file numaration, this means that you have to move old files using an external script, otherwise the storage will be full at some point and n2disk will not be able to write new files anymore.
> Correct?
>
> Alfredo
>
> > On 05 Apr 2016, at 20:04, Jeremy Ashton <jeremy.ashton@shopify.com <mailto:jeremy.ashton@shopify.com>> wrote:
> >
> > I am trying to create a directory structure like the following:
> >
> > /pcap/0/
> > /pcap/1/
> > …
> > /pcap/n/
> >
> > Where each sub-directory contains 1,000 files. I would like to do this without having to restart the file numbering. That way I can have a <isp>-<file_number>.pcap which will always increment.
> >
> > Any thoughts as to how I could approach this?
> >
> > Thanks.
> > _______________________________________________
> > Ntop-misc mailing list
> > Ntop-misc@listgateway.unipi.it <mailto:Ntop-misc@listgateway.unipi.it>
> > http://listgateway.unipi.it/mailman/listinfo/ntop-misc <http://listgateway.unipi.it/mailman/listinfo/ntop-misc>
>
>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it <mailto:Ntop-misc@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc <http://listgateway.unipi.it/mailman/listinfo/ntop-misc>
>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Attached Files:

Re: n2disk pcap output structure [ In reply to ]
jeremy.ashton at shopify
Apr 7, 2016, 12:10 PM
Post #5 of 8 (2296 views)
Permalink
This does not appear to work. I now have an unlimited number of
directories with 1000 files in each. But, the file number is incremented
after each 1000. aka same problem.

On Thu, Apr 7, 2016 at 3:53 AM, Alfredo Cardigliano <cardigliano@ntop.org>
wrote:

> Hi Jeremy
> please update to latest version from the dev repo, you can specify -n -1
> to create unlimited directories, please test it and let as know if it
> behaves as expected.
>
> Alfredo
>
> On 06 Apr 2016, at 15:55, Jeremy Ashton <jeremy.ashton@shopify.com> wrote:
>
> Yes, I am currently dealing with the long term retention issues by
> cleaning up the disk space.
>
> On Wed, Apr 6, 2016 at 9:37 AM, Alfredo Cardigliano <cardigliano@ntop.org>
> wrote:
>
>> Hi Jeremy
>> if I understand correctly you do not want to restart file numaration,
>> this means that you have to move old files using an external script,
>> otherwise the storage will be full at some point and n2disk will not be
>> able to write new files anymore.
>> Correct?
>>
>> Alfredo
>>
>> > On 05 Apr 2016, at 20:04, Jeremy Ashton <jeremy.ashton@shopify.com>
>> wrote:
>> >
>> > I am trying to create a directory structure like the following:
>> >
>> > /pcap/0/
>> > /pcap/1/
>> > …
>> > /pcap/n/
>> >
>> > Where each sub-directory contains 1,000 files. I would like to do this
>> without having to restart the file numbering. That way I can have a
>> <isp>-<file_number>.pcap which will always increment.
>> >
>> > Any thoughts as to how I could approach this?
>> >
>> > Thanks.
>> > _______________________________________________
>> > Ntop-misc mailing list
>> > Ntop-misc@listgateway.unipi.it
>> > http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>
>>
>> _______________________________________________
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>
>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
Re: n2disk pcap output structure [ In reply to ]
jeremy.ashton at shopify
Apr 7, 2016, 12:13 PM
Post #6 of 8 (2295 views)
Permalink
To clarify, I am looking for:

/pcaps/1/file-1.pcap
/pcaps/1/file-2.pcap
…
/pcaps/1/file-1000.pcap
/pcaps/2/file-1001.pcap
/pcaps/2/file-1002.pcap
…
/pcaps/2/file-2000.pcap
…

Sorry for the double message.

On Thu, Apr 7, 2016 at 3:10 PM, Jeremy Ashton <jeremy.ashton@shopify.com>
wrote:

> This does not appear to work. I now have an unlimited number of
> directories with 1000 files in each. But, the file number is incremented
> after each 1000. aka same problem.
>
> On Thu, Apr 7, 2016 at 3:53 AM, Alfredo Cardigliano <cardigliano@ntop.org>
> wrote:
>
>> Hi Jeremy
>> please update to latest version from the dev repo, you can specify -n -1
>> to create unlimited directories, please test it and let as know if it
>> behaves as expected.
>>
>> Alfredo
>>
>> On 06 Apr 2016, at 15:55, Jeremy Ashton <jeremy.ashton@shopify.com>
>> wrote:
>>
>> Yes, I am currently dealing with the long term retention issues by
>> cleaning up the disk space.
>>
>> On Wed, Apr 6, 2016 at 9:37 AM, Alfredo Cardigliano <cardigliano@ntop.org
>> > wrote:
>>
>>> Hi Jeremy
>>> if I understand correctly you do not want to restart file numaration,
>>> this means that you have to move old files using an external script,
>>> otherwise the storage will be full at some point and n2disk will not be
>>> able to write new files anymore.
>>> Correct?
>>>
>>> Alfredo
>>>
>>> > On 05 Apr 2016, at 20:04, Jeremy Ashton <jeremy.ashton@shopify.com>
>>> wrote:
>>> >
>>> > I am trying to create a directory structure like the following:
>>> >
>>> > /pcap/0/
>>> > /pcap/1/
>>> > …
>>> > /pcap/n/
>>> >
>>> > Where each sub-directory contains 1,000 files. I would like to do
>>> this without having to restart the file numbering. That way I can have a
>>> <isp>-<file_number>.pcap which will always increment.
>>> >
>>> > Any thoughts as to how I could approach this?
>>> >
>>> > Thanks.
>>> > _______________________________________________
>>> > Ntop-misc mailing list
>>> > Ntop-misc@listgateway.unipi.it
>>> > http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>
>>>
>>> _______________________________________________
>>> Ntop-misc mailing list
>>> Ntop-misc@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>
>>
>> _______________________________________________
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>
>>
>>
>> _______________________________________________
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>
>
>
Re: n2disk pcap output structure [ In reply to ]
cardigliano at ntop
Apr 8, 2016, 3:31 AM
Post #7 of 8 (2297 views)
Permalink
Hi Jeremy
in essence at the moment it creates the structure below, but you want the file numeration to continue on the next dir instead of restarting from 1, did I understand correctly?

/pcaps/1/1.pcap
/pcaps/1/2.pcap
…
/pcaps/1/1000.pcap
/pcaps/2/1.pcap
/pcaps/2/2.pcap
…
/pcaps/2/1000.pcap
…

Alfredo

> On 07 Apr 2016, at 21:13, Jeremy Ashton <jeremy.ashton@shopify.com> wrote:
>
> To clarify, I am looking for:
>
> /pcaps/1/file-1.pcap
> /pcaps/1/file-2.pcap
> …
> /pcaps/1/file-1000.pcap
> /pcaps/2/file-1001.pcap
> /pcaps/2/file-1002.pcap
> …
> /pcaps/2/file-2000.pcap
> …
>
> Sorry for the double message.
>
> On Thu, Apr 7, 2016 at 3:10 PM, Jeremy Ashton <jeremy.ashton@shopify.com <mailto:jeremy.ashton@shopify.com>> wrote:
> This does not appear to work. I now have an unlimited number of directories with 1000 files in each. But, the file number is incremented after each 1000. aka same problem.
>
> On Thu, Apr 7, 2016 at 3:53 AM, Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>> wrote:
> Hi Jeremy
> please update to latest version from the dev repo, you can specify -n -1 to create unlimited directories, please test it and let as know if it behaves as expected.
>
> Alfredo
>
>> On 06 Apr 2016, at 15:55, Jeremy Ashton <jeremy.ashton@shopify.com <mailto:jeremy.ashton@shopify.com>> wrote:
>>
>> Yes, I am currently dealing with the long term retention issues by cleaning up the disk space.
>>
>> On Wed, Apr 6, 2016 at 9:37 AM, Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>> wrote:
>> Hi Jeremy
>> if I understand correctly you do not want to restart file numaration, this means that you have to move old files using an external script, otherwise the storage will be full at some point and n2disk will not be able to write new files anymore.
>> Correct?
>>
>> Alfredo
>>
>> > On 05 Apr 2016, at 20:04, Jeremy Ashton <jeremy.ashton@shopify.com <mailto:jeremy.ashton@shopify.com>> wrote:
>> >
>> > I am trying to create a directory structure like the following:
>> >
>> > /pcap/0/
>> > /pcap/1/
>> > …
>> > /pcap/n/
>> >
>> > Where each sub-directory contains 1,000 files. I would like to do this without having to restart the file numbering. That way I can have a <isp>-<file_number>.pcap which will always increment.
>> >
>> > Any thoughts as to how I could approach this?
>> >
>> > Thanks.
>> > _______________________________________________
>> > Ntop-misc mailing list
>> > Ntop-misc@listgateway.unipi.it <mailto:Ntop-misc@listgateway.unipi.it>
>> > http://listgateway.unipi.it/mailman/listinfo/ntop-misc <http://listgateway.unipi.it/mailman/listinfo/ntop-misc>
>>
>>
>> _______________________________________________
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it <mailto:Ntop-misc@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc <http://listgateway.unipi.it/mailman/listinfo/ntop-misc>
>>
>> _______________________________________________
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it <mailto:Ntop-misc@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc <http://listgateway.unipi.it/mailman/listinfo/ntop-misc>
>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it <mailto:Ntop-misc@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc <http://listgateway.unipi.it/mailman/listinfo/ntop-misc>
>
>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Attached Files:

Re: n2disk pcap output structure [ In reply to ]
jeremy.ashton at shopify
Apr 8, 2016, 4:43 AM
Post #8 of 8 (2298 views)
Permalink
Yes, that is correct.

On Fri, Apr 8, 2016 at 6:31 AM, Alfredo Cardigliano <cardigliano@ntop.org>
wrote:

> Hi Jeremy
> in essence at the moment it creates the structure below, but you want the
> file numeration to continue on the next dir instead of restarting from 1,
> did I understand correctly?
>
> /pcaps/1/1.pcap
> /pcaps/1/2.pcap
> …
> /pcaps/1/1000.pcap
> /pcaps/2/1.pcap
> /pcaps/2/2.pcap
> …
> /pcaps/2/1000.pcap
> …
>
> Alfredo
>
> On 07 Apr 2016, at 21:13, Jeremy Ashton <jeremy.ashton@shopify.com> wrote:
>
> To clarify, I am looking for:
>
> /pcaps/1/file-1.pcap
> /pcaps/1/file-2.pcap
> …
> /pcaps/1/file-1000.pcap
> /pcaps/2/file-1001.pcap
> /pcaps/2/file-1002.pcap
> …
> /pcaps/2/file-2000.pcap
> …
>
> Sorry for the double message.
>
> On Thu, Apr 7, 2016 at 3:10 PM, Jeremy Ashton <jeremy.ashton@shopify.com>
> wrote:
>
>> This does not appear to work. I now have an unlimited number of
>> directories with 1000 files in each. But, the file number is incremented
>> after each 1000. aka same problem.
>>
>> On Thu, Apr 7, 2016 at 3:53 AM, Alfredo Cardigliano <cardigliano@ntop.org
>> > wrote:
>>
>>> Hi Jeremy
>>> please update to latest version from the dev repo, you can specify -n -1
>>> to create unlimited directories, please test it and let as know if it
>>> behaves as expected.
>>>
>>> Alfredo
>>>
>>> On 06 Apr 2016, at 15:55, Jeremy Ashton <jeremy.ashton@shopify.com>
>>> wrote:
>>>
>>> Yes, I am currently dealing with the long term retention issues by
>>> cleaning up the disk space.
>>>
>>> On Wed, Apr 6, 2016 at 9:37 AM, Alfredo Cardigliano <
>>> cardigliano@ntop.org> wrote:
>>>
>>>> Hi Jeremy
>>>> if I understand correctly you do not want to restart file numaration,
>>>> this means that you have to move old files using an external script,
>>>> otherwise the storage will be full at some point and n2disk will not be
>>>> able to write new files anymore.
>>>> Correct?
>>>>
>>>> Alfredo
>>>>
>>>> > On 05 Apr 2016, at 20:04, Jeremy Ashton <jeremy.ashton@shopify.com>
>>>> wrote:
>>>> >
>>>> > I am trying to create a directory structure like the following:
>>>> >
>>>> > /pcap/0/
>>>> > /pcap/1/
>>>> > …
>>>> > /pcap/n/
>>>> >
>>>> > Where each sub-directory contains 1,000 files. I would like to do
>>>> this without having to restart the file numbering. That way I can have a
>>>> <isp>-<file_number>.pcap which will always increment.
>>>> >
>>>> > Any thoughts as to how I could approach this?
>>>> >
>>>> > Thanks.
>>>> > _______________________________________________
>>>> > Ntop-misc mailing list
>>>> > Ntop-misc@listgateway.unipi.it
>>>> > http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>>
>>>>
>>>> _______________________________________________
>>>> Ntop-misc mailing list
>>>> Ntop-misc@listgateway.unipi.it
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>>
>>>
>>> _______________________________________________
>>> Ntop-misc mailing list
>>> Ntop-misc@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>
>>>
>>>
>>> _______________________________________________
>>> Ntop-misc mailing list
>>> Ntop-misc@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>
>>
>>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal