Hey Alfredo,
Source/destination IPs/networks, protocols/ports, and VLAN #s are usually the only ones used. Occasionally some other oddball queries are used (e.g., looking for fragmented packets), but it's fairly rare.
-Terry
On Sunday, October 29, 2017, 2:58:53 PM EDT, Alfredo Cardigliano <cardigliano@ntop.org> wrote:
Hi TerryI confirm this is not the best use case for zbalance_ipc: in addition to the “flexibility” issue (you need to know in advance the number of tcpdump instances for allocating queues), what happens in this configuration is thatzbalace is distributing 10g on every queue, where a bpf is evaluating the whole traffic. I am not sure this wouldperform much faster then standard kernel filtering.Question: do you have common filters you usually use (e.g. src/dst IP match is enough), or do you use the full power the BPF syntax provides?
RegardsAlfredo
On 27 Oct 2017, at 01:44, Terry <t0psecret@yahoo.com> wrote:
Hey Alfredo,
Thanks. Just Tcpdump -- that's all these servers are used for.
The traffic can be expected to hit line-rate 10G. We distribute the traffic (VLAN-based port mirroring) to eight 10G ports per server. Some of them are fully utilized during peak hours.
When Tcpdump is run, it's not matching all traffic on a port. Captures are run for specific things based on what a user is troubleshooting; e.g., traffic between a specific source/destination IP address.
-Terry
On Thursday, October 26, 2017, 1:22:07 PM EDT, Alfredo Cardigliano <cardigliano@ntop.org> wrote:
Hi Terryyour assumptions are correct, this is not the best use case for zbalance_ipc,in order to help you finding the best configuration, I have a few questions:1. what application do you run on top of zbalance_ipc in addition to tcpdump?2. what is the peak traffic rate?
Alfredo
On 26 Oct 2017, at 18:13, Tom J. <t0psecret@yahoo.com> wrote:
Hey Alfredo,
We dug into this a bit and are trying to figure out how to best use zbalance in our scenario, where lots of people have shell access to a Linux-based sniffer, each running TCPDUMP on various interfaces to troubleshoot network issues.
With zbalance running in the background duplicating traffic to some number of queues, is it correct that at any given time, only one TCPDUMP instance can be running on a given queue? If so I'm having trouble imagining how this can work logistically, as it would seem that users would have to try multiple queues each time until they find one that is free.
Thanks as always.
-Terry
On Tuesday, October 3, 2017, 6:30:17 AM EDT, Alfredo Cardigliano <cardigliano@ntop.org> wrote:
Hi Terryplease find zbalance_ipc+tcpdump examples here:
https://github.com/ntop/PF_RING/blob/dev/userland/examples_zc/README.examples Alfredo
On 2 Oct 2017, at 22:58, Terry <t0psecret@yahoo.com> wrote:
Hey Alfredo,
Thank you. Is there documentation on zbalance beyond what I'm finding via Google? I'm not seeing how to use it to create the queues for Tcpdump to attach to.
-Terry
On Friday, September 29, 2017 1:05 PM, Alfredo Cardigliano <cardigliano@ntop.org> wrote:
Hi Terrydummy interfaces are usually used with Bro because this consumer is well known to be unstable (or at least it crashes from time to time for some reason)leaving ZC queues in an inconsistent state, preventing it from reattaching to the queue again (in order to reattach a zbalance_ipc restart is required).As long as tcpdump is closed correctly, there should be no problem attaching to the queues directly. Please note dummy interfaces are slow as traffic goes through the kernel, and you loose most of the boost provided by ZC.
Alfredo
On 29 Sep 2017, at 18:53, Terry <t0psecret@yahoo.com> wrote:
Hey Alfredo,
Thanks, the zbalance stuff looks encouraging. How would this look in the context of users constantly running/terminating their own instances of tcpdump? I see the "Best practices for using Bro IDS with PF_RING ZC" article, where ZC outputs to dummy interfaces which are then used by the application. Is this how we would do it -- set up one-to-one mappings of ZC Interface -> Dummy Interface, and then have users use the dummy interfaces with tcpdump rather than the ZC interfaces directly?
-Terry
On Friday, September 29, 2017 5:12 AM, Alfredo Cardigliano <cardigliano@ntop.org> wrote:
Hi TerryZC is a kernel-bypass technology, this means that the application takes full control over the NIC in order to access the card memory in zero-copy and maximise the performance. This implies thatone process at a time can open an interface, thus what you are seeing with tcpdump is expected.This said, there is a way to overcome this: you can use zbalance_ipc to open the zc interface, andlet it distribute the traffic (fanout) to multiple applications by means of zc queues. Please note this adds some overhead with respect to opening the zc interface directly from the application, however you should not notice the difference as tcpdump itself is a bottleneck.
Alfredo
On 29 Sep 2017, at 00:29, Tom J. <t0psecret@yahoo.com> wrote:
Hello,
We're exploring using PF-RING ZC for our packet sniffers, but are looking to get clarity on an issue before purchasing licenses.
The sniffers are standard servers running Linux, each with (16) 10G NIC ports connected to SPAN ports on switches. Users log into the system and run TCPDUMP to troubleshoot day-to-day connectivity issues in the environment.
As traffic levels have increased we're seeing more and more drops on the NICs, so the thought was to implement ZC to make things better. But it looks like ZC may limit us to one capture per NIC at any given time. Is this correct? I see text on the product page about not being able to do standard networking activities on a given NIC when ZC is actively running, but how about multiple ZC-enabled TCPDUMPs at once? It doesn't seem to work for us (getting a "No such device" error), but maybe it's something we're doing wrong.
Would appreciate any guidance.
-Terry
_______________________________________________
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
