Mailing List Archive: n2disk to network drive?

n2disk to network drive?

Apr 10, 2018, 1:45 PM

Post #1 of 3 (570 views)

Hi,

Is it viable / recommended to write n2disk cache to (low latency) network
storage?

I have multiple servers I would like to capture certain traffic profile on
but then some time later be able to extract pcaps realated to certain
application events. My thinking is I'd rather not have to do the
npacpextract "on box" and then have to work out how to copy them somewhere
else on the network when I could just do the npacpextract directly off
network storage.

Are there any limitations / gotchas I should be aware of if using
npacpextract from a network attached cache while it is being written?

Also, this comes back to a question I asked the other day. but assuming I
wanted to npacpextract a certain pcap that happened between timestamps X
and Y on port Z but I'm not sure yet if that data has been yet flushed to
disk. are my only options either to:

* use the [--max-file-duration|-t] $secs option and wait $secs after the
associated application event until attempting to npacpextract

* force an explicit flush by sending a USR1 signal to the n2disk process.
(this approach would be complicated if the cache is on network storage and
the consumer wishing to run npacpextract is on a different machine than the
n2disk process)

It seems like the first option is the least worst approach. Can you
suggest any others?

Thanks,
RD

Re: n2disk to network drive? [ In reply to ]

cardigliano at ntop

Apr 11, 2018, 1:49 AM

Post #2 of 3 (570 views)

Permalink

Hi Raoul
you should be able to run npcapextract on your network storage
as long as npcapextract is able to access both the timeline and the pcap files
and the timeline links have correct paths (the timeline is a tree of links to the
pcap and index files).
BTW, are you aware of the new ‘nxn' interface? This is a new GUI to run distributed
extractions, included in the nBox package. Please take a look at https://www.ntop.org/guides/nbox/usage/nxn.html
As of flushing data to disk, you have those 2 options you listed below, the first
one requires less efforts on your side as there is no need to broadcast a signal
to all the appliances, with the drawback that you can run queries on 10-min old
data (with the default configuration) or N-secs old data is you configure a max
duration (this depends on your use case).

Alfredo

> On 10 Apr 2018, at 22:45, Raoul Duke <rduke496@gmail.com> wrote:
>
> Hi,
>
> Is it viable / recommended to write n2disk cache to (low latency) network storage?
>
> I have multiple servers I would like to capture certain traffic profile on but then some time later be able to extract pcaps realated to certain application events. My thinking is I'd rather not have to do the npacpextract "on box" and then have to work out how to copy them somewhere else on the network when I could just do the npacpextract directly off network storage.
>
> Are there any limitations / gotchas I should be aware of if using npacpextract from a network attached cache while it is being written?
>
> Also, this comes back to a question I asked the other day. but assuming I wanted to npacpextract a certain pcap that happened between timestamps X and Y on port Z but I'm not sure yet if that data has been yet flushed to disk. are my only options either to:
>
> * use the [--max-file-duration|-t] $secs option and wait $secs after the associated application event until attempting to npacpextract
>
> * force an explicit flush by sending a USR1 signal to the n2disk process. (this approach would be complicated if the cache is on network storage and the consumer wishing to run npacpextract is on a different machine than the n2disk process)
>
> It seems like the first option is the least worst approach. Can you suggest any others?
>
> Thanks,
> RD
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop

Mailing List Archive

Attached Files:

Attached Files: