Mailing List Archive: nprobe scaling...

We are evaluating nprobe, and the results so far look very good. We are looking for advise on the best method to scale nprobe. We have 3 reasonably large linux systems - 32 cpu's/68gb of memory each. We get about 150k flows per second peak, with an avg of about 60k flows per second. So far, we have been running many nprobe instances (over 100) on the same server to scale. Nprobe is using very little cpu or memory, which makes me wonder if there is a better way to scale this, so that one instance can take better advantage of the resources on the server.

Any advice is appreciated.

We have also trying to export the data to Elastic, but it appears that the nprobe can't keep up with the data, as it's exporting in very small batches, in very small sizes. Is there a way to fine-tune how the data is exported?

Thanks.

Scott Bossi
Cyber Threat Operations
Cyber Operations Engineering
Raytheon Company

+1.978.436.3750 business
scott.v.bossi@raytheon.com<mailto:scott.v.bossi@raytheon.com>

880 Technology Park Drive
Billerica, MA 01821-4164 USA
www.raytheon.com<http://www.raytheon.com>

Hi Scott
thanks for using nProbe. A single instance should be able to collect 10-20k+ flows/core, this if you’re able to distribute flows across instances. Export to ElasticSearch has been improved (and extended to support the latest version) recently. What nProbe version are you using?

In order to assist you I would like you to send
- the exact command line you are using to start nprobe
- how do you balance traffic across the probes running on your system

Thanks Luca

> On 26 Jun 2018, at 07:51, Scott Bossi <scott.v.bossi@raytheon.com> wrote:
>
> We are evaluating nprobe, and the results so far look very good. We are looking for advise on the best method to scale nprobe. We have 3 reasonably large linux systems - 32 cpu's/68gb of memory each. We get about 150k flows per second peak, with an avg of about 60k flows per second. So far, we have been running many nprobe instances (over 100) on the same server to scale. Nprobe is using very little cpu or memory, which makes me wonder if there is a better way to scale this, so that one instance can take better advantage of the resources on the server.
> Any advice is appreciated.
>
>
> We have also trying to export the data to Elastic, but it appears that the nprobe can’t keep up with the data, as it’s exporting in very small batches, in very small sizes. Is there a way to fine-tune how the data is exported?
>
> Thanks.
>
> Scott Bossi
> Cyber Threat Operations
> Cyber Operations Engineering
> Raytheon Company
>
> +1.978.436.3750 business
> scott.v.bossi@raytheon.com <mailto:scott.v.bossi@raytheon.com>
>
>
> 880 Technology Park Drive
> Billerica, MA 01821-4164 USA
> www.raytheon.com <http://www.raytheon.com/>
>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it <mailto:Ntop-misc@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc <http://listgateway.unipi.it/mailman/listinfo/ntop-misc>