Mailing List Archive

10/40Gb simple Tap
Hi,
Is it possible to build an inexpensive 10/40Gb simple Tap using PF_RING + ZC? The base requirements is as follows:
1. Monitor a mix of 10Gb and 1Gb interfaces
2. Reduplicate traffic to N x 10/40Gb output interfaces
3. Each output interface may have a network filter
4. Each output interface receives all input packets that match the network filter
5. Simple CLI/GUI

Is this a feasible/documented solution using PF_RING?
What are the limitations? E.g., packet loss if input > output
What are the HW specs for such box? E.g., 8 core CPU, mem, disk, ... for input < 2x 10Gb+2x 1G input ; Enough PCI slots
Are there additional HW/SW costs beyond the PF_RING ZC NICs?

Thanks,
Oren
Re: 10/40Gb simple Tap [ In reply to ]
Hi Oren
traffic aggregation from multiple ingress ports and duplication/distribution
to multiple egress interfaces is available with PF_RING ZC, we have a
tool (zbalance_ipc) which is based on the ZC API and able to do that
A small subset of the functionalities provided by this tool are described at
https://www.ntop.org/guides/pf_ring/rss.html#zc-load-balancing-zbalance-ipc <https://www.ntop.org/guides/pf_ring/rss.html#zc-load-balancing-zbalance-ipc>
please note that it is able to distribute traffic to processes on the same host,
as well as egress interfaces. What is missing is traffic filterins on the egress
interfaces, however that is already supported by the ZC API, and the tool
can support it with small changes.
The main concern here is about performance, it really depends on a few
factors:
1. number of ingress/egress links
2. traffic rate
3. filters
Do you have some number?

Alfredo

> On 28 Aug 2019, at 14:33, Oren N <theoren28@hotmail.com> wrote:
>
> Hi,
> Is it possible to build an inexpensive 10/40Gb simple Tap using PF_RING + ZC? The base requirements is as follows:
> 1. Monitor a mix of 10Gb and 1Gb interfaces
> 2. Reduplicate traffic to N x 10/40Gb output interfaces
> 3. Each output interface may have a network filter
> 4. Each output interface receives all input packets that match the network filter
> 5. Simple CLI/GUI
>
> Is this a feasible/documented solution using PF_RING?
> What are the limitations? E.g., packet loss if input > output
> What are the HW specs for such box? E.g., 8 core CPU, mem, disk, ... for input < 2x 10Gb+2x 1G input ; Enough PCI slots
> Are there additional HW/SW costs beyond the PF_RING ZC NICs?
>
> Thanks,
> Oren
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it <mailto:Ntop-misc@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc <http://listgateway.unipi.it/mailman/listinfo/ntop-misc>
Re: 10/40Gb simple Tap [ In reply to ]
Hi Alfredo,
Thanks for the quick resp.
To answer your questions:

1. Ingress: 1x 10Gb / Egress: 3x 10Gb + 1x 1Gb
2. <5Gbps
3. The 1Gb filter is set by selecting 802.1q VLANs (alternatively, may be replaced by Segment/IP range filter); 10Gb are unfiltered - namely copy everything

Thanks,
Oren

________________________________
From: ntop-misc-bounces@listgateway.unipi.it <ntop-misc-bounces@listgateway.unipi.it> on behalf of Alfredo Cardigliano <cardigliano@ntop.org>
Sent: Wednesday, August 28, 2019 12:59 PM
To: ntop-misc@listgateway.unipi.it <ntop-misc@listgateway.unipi.it>
Subject: Re: [Ntop-misc] 10/40Gb simple Tap

Hi Oren
traffic aggregation from multiple ingress ports and duplication/distribution
to multiple egress interfaces is available with PF_RING ZC, we have a
tool (zbalance_ipc) which is based on the ZC API and able to do that
A small subset of the functionalities provided by this tool are described at
https://www.ntop.org/guides/pf_ring/rss.html#zc-load-balancing-zbalance-ipc
please note that it is able to distribute traffic to processes on the same host,
as well as egress interfaces. What is missing is traffic filterins on the egress
interfaces, however that is already supported by the ZC API, and the tool
can support it with small changes.
The main concern here is about performance, it really depends on a few
factors:
1. number of ingress/egress links
2. traffic rate
3. filters
Do you have some number?

Alfredo

On 28 Aug 2019, at 14:33, Oren N <theoren28@hotmail.com<mailto:theoren28@hotmail.com>> wrote:

Hi,
Is it possible to build an inexpensive 10/40Gb simple Tap using PF_RING + ZC? The base requirements is as follows:
1. Monitor a mix of 10Gb and 1Gb interfaces
2. Reduplicate traffic to N x 10/40Gb output interfaces
3. Each output interface may have a network filter
4. Each output interface receives all input packets that match the network filter
5. Simple CLI/GUI

Is this a feasible/documented solution using PF_RING?
What are the limitations? E.g., packet loss if input > output
What are the HW specs for such box? E.g., 8 core CPU, mem, disk, ... for input < 2x 10Gb+2x 1G input ; Enough PCI slots
Are there additional HW/SW costs beyond the PF_RING ZC NICs?

Thanks,
Oren
_______________________________________________
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it<mailto:Ntop-misc@listgateway.unipi.it>
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: 10/40Gb simple Tap [ In reply to ]
Hi Oren
as of the traffic rate, do you also have numbers about the avg/peak packets/sec rate?

Thank you
Alfredo

> On 5 Sep 2019, at 15:23, Oren N <theoren28@hotmail.com> wrote:
>
> Hi Alfredo,
> Thanks for the quick resp.
> To answer your questions:
> Ingress: 1x 10Gb / Egress: 3x 10Gb + 1x 1Gb
> <5Gbps
> The 1Gb filter is set by selecting 802.1q VLANs (alternatively, may be replaced by Segment/IP range filter); 10Gb are unfiltered - namely copy everything
> Thanks,
> Oren
>
> From: ntop-misc-bounces@listgateway.unipi.it <ntop-misc-bounces@listgateway.unipi.it> on behalf of Alfredo Cardigliano <cardigliano@ntop.org>
> Sent: Wednesday, August 28, 2019 12:59 PM
> To: ntop-misc@listgateway.unipi.it <ntop-misc@listgateway.unipi.it>
> Subject: Re: [Ntop-misc] 10/40Gb simple Tap
>
> Hi Oren
> traffic aggregation from multiple ingress ports and duplication/distribution
> to multiple egress interfaces is available with PF_RING ZC, we have a
> tool (zbalance_ipc) which is based on the ZC API and able to do that
> A small subset of the functionalities provided by this tool are described at
> https://www.ntop.org/guides/pf_ring/rss.html#zc-load-balancing-zbalance-ipc <https://www.ntop.org/guides/pf_ring/rss.html#zc-load-balancing-zbalance-ipc>
> please note that it is able to distribute traffic to processes on the same host,
> as well as egress interfaces. What is missing is traffic filterins on the egress
> interfaces, however that is already supported by the ZC API, and the tool
> can support it with small changes.
> The main concern here is about performance, it really depends on a few
> factors:
> 1. number of ingress/egress links
> 2. traffic rate
> 3. filters
> Do you have some number?
>
> Alfredo
>
>> On 28 Aug 2019, at 14:33, Oren N <theoren28@hotmail.com <mailto:theoren28@hotmail.com>> wrote:
>>
>> Hi,
>> Is it possible to build an inexpensive 10/40Gb simple Tap using PF_RING + ZC? The base requirements is as follows:
>> 1. Monitor a mix of 10Gb and 1Gb interfaces
>> 2. Reduplicate traffic to N x 10/40Gb output interfaces
>> 3. Each output interface may have a network filter
>> 4. Each output interface receives all input packets that match the network filter
>> 5. Simple CLI/GUI
>>
>> Is this a feasible/documented solution using PF_RING?
>> What are the limitations? E.g., packet loss if input > output
>> What are the HW specs for such box? E.g., 8 core CPU, mem, disk, ... for input < 2x 10Gb+2x 1G input ; Enough PCI slots
>> Are there additional HW/SW costs beyond the PF_RING ZC NICs?
>>
>> Thanks,
>> Oren
>> _______________________________________________
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it <mailto:Ntop-misc@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc <http://listgateway.unipi.it/mailman/listinfo/ntop-misc>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it <mailto:Ntop-misc@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc <http://listgateway.unipi.it/mailman/listinfo/ntop-misc>
Re: 10/40Gb simple Tap [ In reply to ]
Hi Alfredo,
This is hard to say as this is for future needs
Interpolating from existing stats.
2MPPS on average
4MPPS peaks
Thanks,
Oren

________________________________
From: ntop-misc-bounces@listgateway.unipi.it <ntop-misc-bounces@listgateway.unipi.it> on behalf of Alfredo Cardigliano <cardigliano@ntop.org>
Sent: Thursday, September 5, 2019 4:29 PM
To: ntop-misc@listgateway.unipi.it <ntop-misc@listgateway.unipi.it>
Subject: Re: [Ntop-misc] 10/40Gb simple Tap

Hi Oren
as of the traffic rate, do you also have numbers about the avg/peak packets/sec rate?

Thank you
Alfredo

On 5 Sep 2019, at 15:23, Oren N <theoren28@hotmail.com<mailto:theoren28@hotmail.com>> wrote:

Hi Alfredo,
Thanks for the quick resp.
To answer your questions:

1. Ingress: 1x 10Gb / Egress: 3x 10Gb + 1x 1Gb
2. <5Gbps
3. The 1Gb filter is set by selecting 802.1q VLANs (alternatively, may be replaced by Segment/IP range filter); 10Gb are unfiltered - namely copy everything

Thanks,
Oren

________________________________
From: ntop-misc-bounces@listgateway.unipi.it<mailto:ntop-misc-bounces@listgateway.unipi.it> <ntop-misc-bounces@listgateway.unipi.it<mailto:ntop-misc-bounces@listgateway.unipi.it>> on behalf of Alfredo Cardigliano <cardigliano@ntop.org<mailto:cardigliano@ntop.org>>
Sent: Wednesday, August 28, 2019 12:59 PM
To: ntop-misc@listgateway.unipi.it<mailto:ntop-misc@listgateway.unipi.it> <ntop-misc@listgateway.unipi.it<mailto:ntop-misc@listgateway.unipi.it>>
Subject: Re: [Ntop-misc] 10/40Gb simple Tap

Hi Oren
traffic aggregation from multiple ingress ports and duplication/distribution
to multiple egress interfaces is available with PF_RING ZC, we have a
tool (zbalance_ipc) which is based on the ZC API and able to do that
A small subset of the functionalities provided by this tool are described at
https://www.ntop.org/guides/pf_ring/rss.html#zc-load-balancing-zbalance-ipc
please note that it is able to distribute traffic to processes on the same host,
as well as egress interfaces. What is missing is traffic filterins on the egress
interfaces, however that is already supported by the ZC API, and the tool
can support it with small changes.
The main concern here is about performance, it really depends on a few
factors:
1. number of ingress/egress links
2. traffic rate
3. filters
Do you have some number?

Alfredo

On 28 Aug 2019, at 14:33, Oren N <theoren28@hotmail.com<mailto:theoren28@hotmail.com>> wrote:

Hi,
Is it possible to build an inexpensive 10/40Gb simple Tap using PF_RING + ZC? The base requirements is as follows:
1. Monitor a mix of 10Gb and 1Gb interfaces
2. Reduplicate traffic to N x 10/40Gb output interfaces
3. Each output interface may have a network filter
4. Each output interface receives all input packets that match the network filter
5. Simple CLI/GUI

Is this a feasible/documented solution using PF_RING?
What are the limitations? E.g., packet loss if input > output
What are the HW specs for such box? E.g., 8 core CPU, mem, disk, ... for input < 2x 10Gb+2x 1G input ; Enough PCI slots
Are there additional HW/SW costs beyond the PF_RING ZC NICs?

Thanks,
Oren
_______________________________________________
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it<mailto:Ntop-misc@listgateway.unipi.it>
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

_______________________________________________
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it<mailto:Ntop-misc@listgateway.unipi.it>
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: 10/40Gb simple Tap [ In reply to ]
Hi Oren
this means receiving 4 Mpps and sending 13.5 Mpps total worst case, while applying a filter.
If you replace the BPF filter with a simple VLAN lookup (since it seems this is enough for you)
and you leverage on RSS, this looks definitely feasible. A 4 core 3 Ghz CPU should be enough,
with the minimum amount of ram to use all cpu channels, no disk is required.

Regards
Alfredo

> On 9 Sep 2019, at 09:40, Oren N <theoren28@hotmail.com> wrote:
>
> Hi Alfredo,
> This is hard to say as this is for future needs
> Interpolating from existing stats.
> 2MPPS on average
> 4MPPS peaks
> Thanks,
> Oren
>
> From: ntop-misc-bounces@listgateway.unipi.it <ntop-misc-bounces@listgateway.unipi.it> on behalf of Alfredo Cardigliano <cardigliano@ntop.org>
> Sent: Thursday, September 5, 2019 4:29 PM
> To: ntop-misc@listgateway.unipi.it <ntop-misc@listgateway.unipi.it>
> Subject: Re: [Ntop-misc] 10/40Gb simple Tap
>
> Hi Oren
> as of the traffic rate, do you also have numbers about the avg/peak packets/sec rate?
>
> Thank you
> Alfredo
>
>> On 5 Sep 2019, at 15:23, Oren N <theoren28@hotmail.com <mailto:theoren28@hotmail.com>> wrote:
>>
>> Hi Alfredo,
>> Thanks for the quick resp.
>> To answer your questions:
>> Ingress: 1x 10Gb / Egress: 3x 10Gb + 1x 1Gb
>> <5Gbps
>> The 1Gb filter is set by selecting 802.1q VLANs (alternatively, may be replaced by Segment/IP range filter); 10Gb are unfiltered - namely copy everything
>> Thanks,
>> Oren
>>
>> From: ntop-misc-bounces@listgateway.unipi.it <mailto:ntop-misc-bounces@listgateway.unipi.it> <ntop-misc-bounces@listgateway.unipi.it <mailto:ntop-misc-bounces@listgateway.unipi.it>> on behalf of Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>>
>> Sent: Wednesday, August 28, 2019 12:59 PM
>> To: ntop-misc@listgateway.unipi.it <mailto:ntop-misc@listgateway.unipi.it> <ntop-misc@listgateway.unipi.it <mailto:ntop-misc@listgateway.unipi.it>>
>> Subject: Re: [Ntop-misc] 10/40Gb simple Tap
>>
>> Hi Oren
>> traffic aggregation from multiple ingress ports and duplication/distribution
>> to multiple egress interfaces is available with PF_RING ZC, we have a
>> tool (zbalance_ipc) which is based on the ZC API and able to do that
>> A small subset of the functionalities provided by this tool are described at
>> https://www.ntop.org/guides/pf_ring/rss.html#zc-load-balancing-zbalance-ipc <https://www.ntop.org/guides/pf_ring/rss.html#zc-load-balancing-zbalance-ipc>
>> please note that it is able to distribute traffic to processes on the same host,
>> as well as egress interfaces. What is missing is traffic filterins on the egress
>> interfaces, however that is already supported by the ZC API, and the tool
>> can support it with small changes.
>> The main concern here is about performance, it really depends on a few
>> factors:
>> 1. number of ingress/egress links
>> 2. traffic rate
>> 3. filters
>> Do you have some number?
>>
>> Alfredo
>>
>>> On 28 Aug 2019, at 14:33, Oren N <theoren28@hotmail.com <mailto:theoren28@hotmail.com>> wrote:
>>>
>>> Hi,
>>> Is it possible to build an inexpensive 10/40Gb simple Tap using PF_RING + ZC? The base requirements is as follows:
>>> 1. Monitor a mix of 10Gb and 1Gb interfaces
>>> 2. Reduplicate traffic to N x 10/40Gb output interfaces
>>> 3. Each output interface may have a network filter
>>> 4. Each output interface receives all input packets that match the network filter
>>> 5. Simple CLI/GUI
>>>
>>> Is this a feasible/documented solution using PF_RING?
>>> What are the limitations? E.g., packet loss if input > output
>>> What are the HW specs for such box? E.g., 8 core CPU, mem, disk, ... for input < 2x 10Gb+2x 1G input ; Enough PCI slots
>>> Are there additional HW/SW costs beyond the PF_RING ZC NICs?
>>>
>>> Thanks,
>>> Oren
>>> _______________________________________________
>>> Ntop-misc mailing list
>>> Ntop-misc@listgateway.unipi.it <mailto:Ntop-misc@listgateway.unipi.it>
>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc <http://listgateway.unipi.it/mailman/listinfo/ntop-misc>
>> _______________________________________________
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it <mailto:Ntop-misc@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc <http://listgateway.unipi.it/mailman/listinfo/ntop-misc>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: 10/40Gb simple Tap [ In reply to ]
Thanks!
Oren

________________________________
From: ntop-misc-bounces@listgateway.unipi.it <ntop-misc-bounces@listgateway.unipi.it> on behalf of Alfredo Cardigliano <cardigliano@ntop.org>
Sent: Monday, September 9, 2019 8:12 AM
To: ntop-misc@listgateway.unipi.it <ntop-misc@listgateway.unipi.it>
Subject: Re: [Ntop-misc] 10/40Gb simple Tap

Hi Oren
this means receiving 4 Mpps and sending 13.5 Mpps total worst case, while applying a filter.
If you replace the BPF filter with a simple VLAN lookup (since it seems this is enough for you)
and you leverage on RSS, this looks definitely feasible. A 4 core 3 Ghz CPU should be enough,
with the minimum amount of ram to use all cpu channels, no disk is required.

Regards
Alfredo

On 9 Sep 2019, at 09:40, Oren N <theoren28@hotmail.com<mailto:theoren28@hotmail.com>> wrote:

Hi Alfredo,
This is hard to say as this is for future needs
Interpolating from existing stats.
2MPPS on average
4MPPS peaks
Thanks,
Oren

________________________________
From: ntop-misc-bounces@listgateway.unipi.it<mailto:ntop-misc-bounces@listgateway.unipi.it> <ntop-misc-bounces@listgateway.unipi.it<mailto:ntop-misc-bounces@listgateway.unipi.it>> on behalf of Alfredo Cardigliano <cardigliano@ntop.org<mailto:cardigliano@ntop.org>>
Sent: Thursday, September 5, 2019 4:29 PM
To: ntop-misc@listgateway.unipi.it<mailto:ntop-misc@listgateway.unipi.it> <ntop-misc@listgateway.unipi.it<mailto:ntop-misc@listgateway.unipi.it>>
Subject: Re: [Ntop-misc] 10/40Gb simple Tap

Hi Oren
as of the traffic rate, do you also have numbers about the avg/peak packets/sec rate?

Thank you
Alfredo

On 5 Sep 2019, at 15:23, Oren N <theoren28@hotmail.com<mailto:theoren28@hotmail.com>> wrote:

Hi Alfredo,
Thanks for the quick resp.
To answer your questions:

1. Ingress: 1x 10Gb / Egress: 3x 10Gb + 1x 1Gb
2. <5Gbps
3. The 1Gb filter is set by selecting 802.1q VLANs (alternatively, may be replaced by Segment/IP range filter); 10Gb are unfiltered - namely copy everything

Thanks,
Oren

________________________________
From: ntop-misc-bounces@listgateway.unipi.it<mailto:ntop-misc-bounces@listgateway.unipi.it> <ntop-misc-bounces@listgateway.unipi.it<mailto:ntop-misc-bounces@listgateway.unipi.it>> on behalf of Alfredo Cardigliano <cardigliano@ntop.org<mailto:cardigliano@ntop.org>>
Sent: Wednesday, August 28, 2019 12:59 PM
To: ntop-misc@listgateway.unipi.it<mailto:ntop-misc@listgateway.unipi.it> <ntop-misc@listgateway.unipi.it<mailto:ntop-misc@listgateway.unipi.it>>
Subject: Re: [Ntop-misc] 10/40Gb simple Tap

Hi Oren
traffic aggregation from multiple ingress ports and duplication/distribution
to multiple egress interfaces is available with PF_RING ZC, we have a
tool (zbalance_ipc) which is based on the ZC API and able to do that
A small subset of the functionalities provided by this tool are described at
https://www.ntop.org/guides/pf_ring/rss.html#zc-load-balancing-zbalance-ipc
please note that it is able to distribute traffic to processes on the same host,
as well as egress interfaces. What is missing is traffic filterins on the egress
interfaces, however that is already supported by the ZC API, and the tool
can support it with small changes.
The main concern here is about performance, it really depends on a few
factors:
1. number of ingress/egress links
2. traffic rate
3. filters
Do you have some number?

Alfredo

On 28 Aug 2019, at 14:33, Oren N <theoren28@hotmail.com<mailto:theoren28@hotmail.com>> wrote:

Hi,
Is it possible to build an inexpensive 10/40Gb simple Tap using PF_RING + ZC? The base requirements is as follows:
1. Monitor a mix of 10Gb and 1Gb interfaces
2. Reduplicate traffic to N x 10/40Gb output interfaces
3. Each output interface may have a network filter
4. Each output interface receives all input packets that match the network filter
5. Simple CLI/GUI

Is this a feasible/documented solution using PF_RING?
What are the limitations? E.g., packet loss if input > output
What are the HW specs for such box? E.g., 8 core CPU, mem, disk, ... for input < 2x 10Gb+2x 1G input ; Enough PCI slots
Are there additional HW/SW costs beyond the PF_RING ZC NICs?

Thanks,
Oren
_______________________________________________
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it<mailto:Ntop-misc@listgateway.unipi.it>
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

_______________________________________________
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it<mailto:Ntop-misc@listgateway.unipi.it>
http://listgateway.unipi.it/mailman/listinfo/ntop-misc

_______________________________________________
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it<mailto:Ntop-misc@listgateway.unipi.it>
http://listgateway.unipi.it/mailman/listinfo/ntop-misc