Mailing List Archive

PF_RING FT API implementation
Hi,

I am new to ntop/ndpi.
I'm looking for source code of PF_RING FT such as implementations of:
pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi lib.

I only see the function defines in userland/lib/pfring_ft.h but not the
implementation.
Could you pls provide some pointers?


Thanks,

Harish
Re: PF_RING FT API implementation [ In reply to ]
Hello,

Sorry but PF_RING FT is not open source.

Regards,
Emanuele


On 06/08/2018 04:00 AM, Harish Patil wrote:
> Hi,
>
> I am new to ntop/ndpi.
> I'm looking for source code of PF_RING FT such as implementations of:
>  pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi lib.
>
> I only see the function defines in userland/lib/pfring_ft.h but not
> the implementation.
> Could you pls provide some pointers?
>
>
> Thanks,
>
> Harish
>
>
>
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
Re: PF_RING FT API implementation [ In reply to ]
Hi Emanuele,
Thanks for the response.
I didn't know that it was not open sourced and didn't expect it either
given that other modules are open sourced.
Are there any plans of open sourcing them?

Thanks.


On Mon, Jun 11, 2018 at 1:37 AM, Emanuele Faranda <faranda@ntop.org> wrote:

> Hello,
>
> Sorry but PF_RING FT is not open source.
>
> Regards,
> Emanuele
>
> On 06/08/2018 04:00 AM, Harish Patil wrote:
>
> Hi,
>
> I am new to ntop/ndpi.
> I'm looking for source code of PF_RING FT such as implementations of:
> pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi lib.
>
> I only see the function defines in userland/lib/pfring_ft.h but not the
> implementation.
> Could you pls provide some pointers?
>
>
> Thanks,
>
> Harish
>
>
>
>
>
>
> _______________________________________________
> Ntop mailing listNtop@listgateway.unipi.ithttp://listgateway.unipi.it/mailman/listinfo/ntop
>
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
Re: PF_RING FT API implementation [ In reply to ]
Hi Emanuele,
Thanks for the response.
I didn't know that it was not open sourced and didn't expect it either
given that other modules are open sourced.
Are there any plans of open sourcing them?

Thanks.


On Mon, Jun 11, 2018 at 1:37 AM, Emanuele Faranda <faranda@ntop.org> wrote:

> Hello,
>
> Sorry but PF_RING FT is not open source.
>
> Regards,
> Emanuele
>
> On 06/08/2018 04:00 AM, Harish Patil wrote:
>
> Hi,
>
> I am new to ntop/ndpi.
> I'm looking for source code of PF_RING FT such as implementations of:
> pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi lib.
>
> I only see the function defines in userland/lib/pfring_ft.h but not the
> implementation.
> Could you pls provide some pointers?
>
>
> Thanks,
>
> Harish
>
>
>
>
>
>
> _______________________________________________
> Ntop mailing listNtop@listgateway.unipi.ithttp://listgateway.unipi.it/mailman/listinfo/ntop
>
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
Re: PF_RING FT API implementation [ In reply to ]
Hi Harish
no plan atm, however we tried to create a flexible framework, meaning
that people should be able to extend it using the API we created.
Please let us know if something is missing for your use case.

Thank you
Alfredo

> On 11 Jun 2018, at 18:10, Harish Patil <patharish@gmail.com> wrote:
>
> Hi Emanuele,
> Thanks for the response.
> I didn't know that it was not open sourced and didn't expect it either given that other modules are open sourced.
> Are there any plans of open sourcing them?
>
> Thanks.
>
>
> On Mon, Jun 11, 2018 at 1:37 AM, Emanuele Faranda <faranda@ntop.org <mailto:faranda@ntop.org>> wrote:
> Hello,
>
> Sorry but PF_RING FT is not open source.
>
> Regards,
> Emanuele
>
> On 06/08/2018 04:00 AM, Harish Patil wrote:
>> Hi,
>>
>> I am new to ntop/ndpi.
>> I'm looking for source code of PF_RING FT such as implementations of:
>> pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi lib.
>>
>> I only see the function defines in userland/lib/pfring_ft.h but not the implementation.
>> Could you pls provide some pointers?
>>
>>
>> Thanks,
>>
>> Harish
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
Re: PF_RING FT API implementation [ In reply to ]
Hi Harish
no plan atm, however we tried to create a flexible framework, meaning
that people should be able to extend it using the API we created.
Please let us know if something is missing for your use case.

Thank you
Alfredo

> On 11 Jun 2018, at 18:10, Harish Patil <patharish@gmail.com> wrote:
>
> Hi Emanuele,
> Thanks for the response.
> I didn't know that it was not open sourced and didn't expect it either given that other modules are open sourced.
> Are there any plans of open sourcing them?
>
> Thanks.
>
>
> On Mon, Jun 11, 2018 at 1:37 AM, Emanuele Faranda <faranda@ntop.org <mailto:faranda@ntop.org>> wrote:
> Hello,
>
> Sorry but PF_RING FT is not open source.
>
> Regards,
> Emanuele
>
> On 06/08/2018 04:00 AM, Harish Patil wrote:
>> Hi,
>>
>> I am new to ntop/ndpi.
>> I'm looking for source code of PF_RING FT such as implementations of:
>> pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi lib.
>>
>> I only see the function defines in userland/lib/pfring_ft.h but not the implementation.
>> Could you pls provide some pointers?
>>
>>
>> Thanks,
>>
>> Harish
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
Re: PF_RING FT API implementation [ In reply to ]
Hi Alfredo,

Thanks for the response.
We are planning to use PF_RING FT in our projects.
We only have shared libs. How can we integrate and make sure it works in
all cases (platforms/GCC)?
If we had the sources we could compile along with sources and would be much
better.

Thanks.
Harish



On Mon, Jun 11, 2018 at 9:20 AM, Alfredo Cardigliano <cardigliano@ntop.org>
wrote:

> Hi Harish
> no plan atm, however we tried to create a flexible framework, meaning
> that people should be able to extend it using the API we created.
> Please let us know if something is missing for your use case.
>
> Thank you
> Alfredo
>
> On 11 Jun 2018, at 18:10, Harish Patil <patharish@gmail.com> wrote:
>
> Hi Emanuele,
> Thanks for the response.
> I didn't know that it was not open sourced and didn't expect it either
> given that other modules are open sourced.
> Are there any plans of open sourcing them?
>
> Thanks.
>
>
> On Mon, Jun 11, 2018 at 1:37 AM, Emanuele Faranda <faranda@ntop.org>
> wrote:
>
>> Hello,
>>
>> Sorry but PF_RING FT is not open source.
>>
>> Regards,
>> Emanuele
>>
>> On 06/08/2018 04:00 AM, Harish Patil wrote:
>>
>> Hi,
>>
>> I am new to ntop/ndpi.
>> I'm looking for source code of PF_RING FT such as implementations of:
>> pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi lib.
>>
>> I only see the function defines in userland/lib/pfring_ft.h but not the
>> implementation.
>> Could you pls provide some pointers?
>>
>>
>> Thanks,
>>
>> Harish
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Ntop mailing listNtop@listgateway.unipi.ithttp://listgateway.unipi.it/mailman/listinfo/ntop
>>
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
Re: PF_RING FT API implementation [ In reply to ]
Hi Alfredo,

Thanks for the response.
We are planning to use PF_RING FT in our projects.
We only have shared libs. How can we integrate and make sure it works in
all cases (platforms/GCC)?
If we had the sources we could compile along with sources and would be much
better.

Thanks.
Harish



On Mon, Jun 11, 2018 at 9:20 AM, Alfredo Cardigliano <cardigliano@ntop.org>
wrote:

> Hi Harish
> no plan atm, however we tried to create a flexible framework, meaning
> that people should be able to extend it using the API we created.
> Please let us know if something is missing for your use case.
>
> Thank you
> Alfredo
>
> On 11 Jun 2018, at 18:10, Harish Patil <patharish@gmail.com> wrote:
>
> Hi Emanuele,
> Thanks for the response.
> I didn't know that it was not open sourced and didn't expect it either
> given that other modules are open sourced.
> Are there any plans of open sourcing them?
>
> Thanks.
>
>
> On Mon, Jun 11, 2018 at 1:37 AM, Emanuele Faranda <faranda@ntop.org>
> wrote:
>
>> Hello,
>>
>> Sorry but PF_RING FT is not open source.
>>
>> Regards,
>> Emanuele
>>
>> On 06/08/2018 04:00 AM, Harish Patil wrote:
>>
>> Hi,
>>
>> I am new to ntop/ndpi.
>> I'm looking for source code of PF_RING FT such as implementations of:
>> pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi lib.
>>
>> I only see the function defines in userland/lib/pfring_ft.h but not the
>> implementation.
>> Could you pls provide some pointers?
>>
>>
>> Thanks,
>>
>> Harish
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Ntop mailing listNtop@listgateway.unipi.ithttp://listgateway.unipi.it/mailman/listinfo/ntop
>>
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
Re: PF_RING FT API implementation [ In reply to ]
Hi Harish
we are currently using those libs to compile our applications on many distributions
including Ubuntu 14/16/18, Centos 6/7, Debian 7/8/9, please let us know if you
experience issues with the compilation, and we will find a way to handle that.

Regards
Alfredo

> On 13 Jun 2018, at 01:40, Harish Patil <patharish@gmail.com> wrote:
>
> Hi Alfredo,
>
> Thanks for the response.
> We are planning to use PF_RING FT in our projects.
> We only have shared libs. How can we integrate and make sure it works in all cases (platforms/GCC)?
> If we had the sources we could compile along with sources and would be much better.
>
> Thanks.
> Harish
>
>
>
> On Mon, Jun 11, 2018 at 9:20 AM, Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>> wrote:
> Hi Harish
> no plan atm, however we tried to create a flexible framework, meaning
> that people should be able to extend it using the API we created.
> Please let us know if something is missing for your use case.
>
> Thank you
> Alfredo
>
>> On 11 Jun 2018, at 18:10, Harish Patil <patharish@gmail.com <mailto:patharish@gmail.com>> wrote:
>>
>> Hi Emanuele,
>> Thanks for the response.
>> I didn't know that it was not open sourced and didn't expect it either given that other modules are open sourced.
>> Are there any plans of open sourcing them?
>>
>> Thanks.
>>
>>
>> On Mon, Jun 11, 2018 at 1:37 AM, Emanuele Faranda <faranda@ntop.org <mailto:faranda@ntop.org>> wrote:
>> Hello,
>>
>> Sorry but PF_RING FT is not open source.
>>
>> Regards,
>> Emanuele
>>
>> On 06/08/2018 04:00 AM, Harish Patil wrote:
>>> Hi,
>>>
>>> I am new to ntop/ndpi.
>>> I'm looking for source code of PF_RING FT such as implementations of:
>>> pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi lib.
>>>
>>> I only see the function defines in userland/lib/pfring_ft.h but not the implementation.
>>> Could you pls provide some pointers?
>>>
>>>
>>> Thanks,
>>>
>>> Harish
>>>
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
Re: PF_RING FT API implementation [ In reply to ]
Hi Harish
we are currently using those libs to compile our applications on many distributions
including Ubuntu 14/16/18, Centos 6/7, Debian 7/8/9, please let us know if you
experience issues with the compilation, and we will find a way to handle that.

Regards
Alfredo

> On 13 Jun 2018, at 01:40, Harish Patil <patharish@gmail.com> wrote:
>
> Hi Alfredo,
>
> Thanks for the response.
> We are planning to use PF_RING FT in our projects.
> We only have shared libs. How can we integrate and make sure it works in all cases (platforms/GCC)?
> If we had the sources we could compile along with sources and would be much better.
>
> Thanks.
> Harish
>
>
>
> On Mon, Jun 11, 2018 at 9:20 AM, Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>> wrote:
> Hi Harish
> no plan atm, however we tried to create a flexible framework, meaning
> that people should be able to extend it using the API we created.
> Please let us know if something is missing for your use case.
>
> Thank you
> Alfredo
>
>> On 11 Jun 2018, at 18:10, Harish Patil <patharish@gmail.com <mailto:patharish@gmail.com>> wrote:
>>
>> Hi Emanuele,
>> Thanks for the response.
>> I didn't know that it was not open sourced and didn't expect it either given that other modules are open sourced.
>> Are there any plans of open sourcing them?
>>
>> Thanks.
>>
>>
>> On Mon, Jun 11, 2018 at 1:37 AM, Emanuele Faranda <faranda@ntop.org <mailto:faranda@ntop.org>> wrote:
>> Hello,
>>
>> Sorry but PF_RING FT is not open source.
>>
>> Regards,
>> Emanuele
>>
>> On 06/08/2018 04:00 AM, Harish Patil wrote:
>>> Hi,
>>>
>>> I am new to ntop/ndpi.
>>> I'm looking for source code of PF_RING FT such as implementations of:
>>> pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi lib.
>>>
>>> I only see the function defines in userland/lib/pfring_ft.h but not the implementation.
>>> Could you pls provide some pointers?
>>>
>>>
>>> Thanks,
>>>
>>> Harish
>>>
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
Re: PF_RING FT API implementation [ In reply to ]
Thanks Alfredo/Emanuele.

I am seeing the below message:

#########################################################################
# ERROR: You do not seem to have a valid PF_RING FT 7.3.0.180617 license
[ECFE456A76066A02]
# ERROR: Please get one at http://shop.ntop.org/.
#########################################################################
# We're now working in demo mode with packet capture and
# transmission limited to 5 minutes

Is there a need to buy a licensed version? Is this newly introduced in the
later releases?

Few questions regarding the PF_RING FT API behavior:

1) Does pfring_ft_process() makes a copy of the packet using the pointer
passed to it? If so, does it copy only the header or entire packet
including payload?

2) Is it safe for me to release/free the buffers after calling
pfring_ft_process()?

3) Do you have performance numbers doing inline DPI for 1G or 10G links?

4) Does PF_RING FT/nDPI maintain state information to detect a flow OR does
it just uses the packet passed to parse L7 info?

Thanks

Harish



On Wed, Jun 13, 2018 at 12:14 AM, Alfredo Cardigliano <cardigliano@ntop.org>
wrote:

> Hi Harish
> we are currently using those libs to compile our applications on many
> distributions
> including Ubuntu 14/16/18, Centos 6/7, Debian 7/8/9, please let us know if
> you
> experience issues with the compilation, and we will find a way to handle
> that.
>
> Regards
> Alfredo
>
>
> On 13 Jun 2018, at 01:40, Harish Patil <patharish@gmail.com> wrote:
>
> Hi Alfredo,
>
> Thanks for the response.
> We are planning to use PF_RING FT in our projects.
> We only have shared libs. How can we integrate and make sure it works in
> all cases (platforms/GCC)?
> If we had the sources we could compile along with sources and would be
> much better.
>
> Thanks.
> Harish
>
>
>
> On Mon, Jun 11, 2018 at 9:20 AM, Alfredo Cardigliano <cardigliano@ntop.org
> > wrote:
>
>> Hi Harish
>> no plan atm, however we tried to create a flexible framework, meaning
>> that people should be able to extend it using the API we created.
>> Please let us know if something is missing for your use case.
>>
>> Thank you
>> Alfredo
>>
>> On 11 Jun 2018, at 18:10, Harish Patil <patharish@gmail.com> wrote:
>>
>> Hi Emanuele,
>> Thanks for the response.
>> I didn't know that it was not open sourced and didn't expect it either
>> given that other modules are open sourced.
>> Are there any plans of open sourcing them?
>>
>> Thanks.
>>
>>
>> On Mon, Jun 11, 2018 at 1:37 AM, Emanuele Faranda <faranda@ntop.org>
>> wrote:
>>
>>> Hello,
>>>
>>> Sorry but PF_RING FT is not open source.
>>>
>>> Regards,
>>> Emanuele
>>>
>>> On 06/08/2018 04:00 AM, Harish Patil wrote:
>>>
>>> Hi,
>>>
>>> I am new to ntop/ndpi.
>>> I'm looking for source code of PF_RING FT such as implementations of:
>>> pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi lib.
>>>
>>> I only see the function defines in userland/lib/pfring_ft.h but not the
>>> implementation.
>>> Could you pls provide some pointers?
>>>
>>>
>>> Thanks,
>>>
>>> Harish
>>>
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Ntop mailing listNtop@listgateway.unipi.ithttp://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
Re: PF_RING FT API implementation [ In reply to ]
Thanks Alfredo/Emanuele.

I am seeing the below message:

#########################################################################
# ERROR: You do not seem to have a valid PF_RING FT 7.3.0.180617 license
[ECFE456A76066A02]
# ERROR: Please get one at http://shop.ntop.org/.
#########################################################################
# We're now working in demo mode with packet capture and
# transmission limited to 5 minutes

Is there a need to buy a licensed version? Is this newly introduced in the
later releases?

Few questions regarding the PF_RING FT API behavior:

1) Does pfring_ft_process() makes a copy of the packet using the pointer
passed to it? If so, does it copy only the header or entire packet
including payload?

2) Is it safe for me to release/free the buffers after calling
pfring_ft_process()?

3) Do you have performance numbers doing inline DPI for 1G or 10G links?

4) Does PF_RING FT/nDPI maintain state information to detect a flow OR does
it just uses the packet passed to parse L7 info?

Thanks

Harish



On Wed, Jun 13, 2018 at 12:14 AM, Alfredo Cardigliano <cardigliano@ntop.org>
wrote:

> Hi Harish
> we are currently using those libs to compile our applications on many
> distributions
> including Ubuntu 14/16/18, Centos 6/7, Debian 7/8/9, please let us know if
> you
> experience issues with the compilation, and we will find a way to handle
> that.
>
> Regards
> Alfredo
>
>
> On 13 Jun 2018, at 01:40, Harish Patil <patharish@gmail.com> wrote:
>
> Hi Alfredo,
>
> Thanks for the response.
> We are planning to use PF_RING FT in our projects.
> We only have shared libs. How can we integrate and make sure it works in
> all cases (platforms/GCC)?
> If we had the sources we could compile along with sources and would be
> much better.
>
> Thanks.
> Harish
>
>
>
> On Mon, Jun 11, 2018 at 9:20 AM, Alfredo Cardigliano <cardigliano@ntop.org
> > wrote:
>
>> Hi Harish
>> no plan atm, however we tried to create a flexible framework, meaning
>> that people should be able to extend it using the API we created.
>> Please let us know if something is missing for your use case.
>>
>> Thank you
>> Alfredo
>>
>> On 11 Jun 2018, at 18:10, Harish Patil <patharish@gmail.com> wrote:
>>
>> Hi Emanuele,
>> Thanks for the response.
>> I didn't know that it was not open sourced and didn't expect it either
>> given that other modules are open sourced.
>> Are there any plans of open sourcing them?
>>
>> Thanks.
>>
>>
>> On Mon, Jun 11, 2018 at 1:37 AM, Emanuele Faranda <faranda@ntop.org>
>> wrote:
>>
>>> Hello,
>>>
>>> Sorry but PF_RING FT is not open source.
>>>
>>> Regards,
>>> Emanuele
>>>
>>> On 06/08/2018 04:00 AM, Harish Patil wrote:
>>>
>>> Hi,
>>>
>>> I am new to ntop/ndpi.
>>> I'm looking for source code of PF_RING FT such as implementations of:
>>> pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi lib.
>>>
>>> I only see the function defines in userland/lib/pfring_ft.h but not the
>>> implementation.
>>> Could you pls provide some pointers?
>>>
>>>
>>> Thanks,
>>>
>>> Harish
>>>
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Ntop mailing listNtop@listgateway.unipi.ithttp://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
Re: PF_RING FT API implementation [ In reply to ]
> On 22 Jun 2018, at 04:32, Harish Patil <patharish@gmail.com> wrote:
>
> Thanks Alfredo/Emanuele.
>
> I am seeing the below message:
>
> #########################################################################
> # ERROR: You do not seem to have a valid PF_RING FT 7.3.0.180617 license [ECFE456A76066A02]
> # ERROR: Please get one at http://shop.ntop.org/ <http://shop.ntop.org/>.
> #########################################################################
> # We're now working in demo mode with packet capture and
> # transmission limited to 5 minutes
>
> Is there a need to buy a licensed version? Is this newly introduced in the later releases?

Yes, PF_RING FT, included in latest stable, requires a license.

> Few questions regarding the PF_RING FT API behavior:
>
> 1) Does pfring_ft_process() makes a copy of the packet using the pointer passed to it? If so, does it copy only the header or entire packet including payload?

No packet copy at all.

> 2) Is it safe for me to release/free the buffers after calling pfring_ft_process()?

Yes

> 3) Do you have performance numbers doing inline DPI for 1G or 10G links?

You should be able to do 10Gbit line-rate, however this also depends on your hw. Please note you can use multiple threads (with RSS or ZC software distribution) if 1 core is not enough.

> 4) Does PF_RING FT/nDPI maintain state information to detect a flow OR does it just uses the packet passed to parse L7 info?

It keeps some state, as 1 packet is often not enough.

Regards
Alfredo

>
> Thanks
>
> Harish
>
>
>
> On Wed, Jun 13, 2018 at 12:14 AM, Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>> wrote:
> Hi Harish
> we are currently using those libs to compile our applications on many distributions
> including Ubuntu 14/16/18, Centos 6/7, Debian 7/8/9, please let us know if you
> experience issues with the compilation, and we will find a way to handle that.
>
> Regards
> Alfredo
>
>
>> On 13 Jun 2018, at 01:40, Harish Patil <patharish@gmail.com <mailto:patharish@gmail.com>> wrote:
>>
>> Hi Alfredo,
>>
>> Thanks for the response.
>> We are planning to use PF_RING FT in our projects.
>> We only have shared libs. How can we integrate and make sure it works in all cases (platforms/GCC)?
>> If we had the sources we could compile along with sources and would be much better.
>>
>> Thanks.
>> Harish
>>
>>
>>
>> On Mon, Jun 11, 2018 at 9:20 AM, Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>> wrote:
>> Hi Harish
>> no plan atm, however we tried to create a flexible framework, meaning
>> that people should be able to extend it using the API we created.
>> Please let us know if something is missing for your use case.
>>
>> Thank you
>> Alfredo
>>
>>> On 11 Jun 2018, at 18:10, Harish Patil <patharish@gmail.com <mailto:patharish@gmail.com>> wrote:
>>>
>>> Hi Emanuele,
>>> Thanks for the response.
>>> I didn't know that it was not open sourced and didn't expect it either given that other modules are open sourced.
>>> Are there any plans of open sourcing them?
>>>
>>> Thanks.
>>>
>>>
>>> On Mon, Jun 11, 2018 at 1:37 AM, Emanuele Faranda <faranda@ntop.org <mailto:faranda@ntop.org>> wrote:
>>> Hello,
>>>
>>> Sorry but PF_RING FT is not open source.
>>>
>>> Regards,
>>> Emanuele
>>>
>>> On 06/08/2018 04:00 AM, Harish Patil wrote:
>>>> Hi,
>>>>
>>>> I am new to ntop/ndpi.
>>>> I'm looking for source code of PF_RING FT such as implementations of:
>>>> pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi lib.
>>>>
>>>> I only see the function defines in userland/lib/pfring_ft.h but not the implementation.
>>>> Could you pls provide some pointers?
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Harish
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
Re: PF_RING FT API implementation [ In reply to ]
> On 22 Jun 2018, at 04:32, Harish Patil <patharish@gmail.com> wrote:
>
> Thanks Alfredo/Emanuele.
>
> I am seeing the below message:
>
> #########################################################################
> # ERROR: You do not seem to have a valid PF_RING FT 7.3.0.180617 license [ECFE456A76066A02]
> # ERROR: Please get one at http://shop.ntop.org/ <http://shop.ntop.org/>.
> #########################################################################
> # We're now working in demo mode with packet capture and
> # transmission limited to 5 minutes
>
> Is there a need to buy a licensed version? Is this newly introduced in the later releases?

Yes, PF_RING FT, included in latest stable, requires a license.

> Few questions regarding the PF_RING FT API behavior:
>
> 1) Does pfring_ft_process() makes a copy of the packet using the pointer passed to it? If so, does it copy only the header or entire packet including payload?

No packet copy at all.

> 2) Is it safe for me to release/free the buffers after calling pfring_ft_process()?

Yes

> 3) Do you have performance numbers doing inline DPI for 1G or 10G links?

You should be able to do 10Gbit line-rate, however this also depends on your hw. Please note you can use multiple threads (with RSS or ZC software distribution) if 1 core is not enough.

> 4) Does PF_RING FT/nDPI maintain state information to detect a flow OR does it just uses the packet passed to parse L7 info?

It keeps some state, as 1 packet is often not enough.

Regards
Alfredo

>
> Thanks
>
> Harish
>
>
>
> On Wed, Jun 13, 2018 at 12:14 AM, Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>> wrote:
> Hi Harish
> we are currently using those libs to compile our applications on many distributions
> including Ubuntu 14/16/18, Centos 6/7, Debian 7/8/9, please let us know if you
> experience issues with the compilation, and we will find a way to handle that.
>
> Regards
> Alfredo
>
>
>> On 13 Jun 2018, at 01:40, Harish Patil <patharish@gmail.com <mailto:patharish@gmail.com>> wrote:
>>
>> Hi Alfredo,
>>
>> Thanks for the response.
>> We are planning to use PF_RING FT in our projects.
>> We only have shared libs. How can we integrate and make sure it works in all cases (platforms/GCC)?
>> If we had the sources we could compile along with sources and would be much better.
>>
>> Thanks.
>> Harish
>>
>>
>>
>> On Mon, Jun 11, 2018 at 9:20 AM, Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>> wrote:
>> Hi Harish
>> no plan atm, however we tried to create a flexible framework, meaning
>> that people should be able to extend it using the API we created.
>> Please let us know if something is missing for your use case.
>>
>> Thank you
>> Alfredo
>>
>>> On 11 Jun 2018, at 18:10, Harish Patil <patharish@gmail.com <mailto:patharish@gmail.com>> wrote:
>>>
>>> Hi Emanuele,
>>> Thanks for the response.
>>> I didn't know that it was not open sourced and didn't expect it either given that other modules are open sourced.
>>> Are there any plans of open sourcing them?
>>>
>>> Thanks.
>>>
>>>
>>> On Mon, Jun 11, 2018 at 1:37 AM, Emanuele Faranda <faranda@ntop.org <mailto:faranda@ntop.org>> wrote:
>>> Hello,
>>>
>>> Sorry but PF_RING FT is not open source.
>>>
>>> Regards,
>>> Emanuele
>>>
>>> On 06/08/2018 04:00 AM, Harish Patil wrote:
>>>> Hi,
>>>>
>>>> I am new to ntop/ndpi.
>>>> I'm looking for source code of PF_RING FT such as implementations of:
>>>> pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi lib.
>>>>
>>>> I only see the function defines in userland/lib/pfring_ft.h but not the implementation.
>>>> Could you pls provide some pointers?
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Harish
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
Re: PF_RING FT API implementation [ In reply to ]
Thanks Alfredo.
Replies below.


On Fri, Jun 22, 2018 at 12:29 AM, Alfredo Cardigliano <cardigliano@ntop.org>
wrote:

>
>
> On 22 Jun 2018, at 04:32, Harish Patil <patharish@gmail.com> wrote:
>
> Thanks Alfredo/Emanuele.
>
> I am seeing the below message:
>
> #########################################################################
> # ERROR: You do not seem to have a valid PF_RING FT 7.3.0.180617 license
> [ECFE456A76066A02]
> # ERROR: Please get one at http://shop.ntop.org/.
> #########################################################################
> # We're now working in demo mode with packet capture and
> # transmission limited to 5 minutes
>
> Is there a need to buy a licensed version? Is this newly introduced in the
> later releases?
>
>
> Yes, PF_RING FT, included in latest stable, requires a license.
>

So at least for now if we stick to the older version then we don't need a
license, right?


>
> Few questions regarding the PF_RING FT API behavior:
>
> 1) Does pfring_ft_process() makes a copy of the packet using the pointer
> passed to it? If so, does it copy only the header or entire packet
> including payload?
>
>
> No packet copy at all.
>

In that case, I can only store packet headers in my local DPDK mempools,
right? The idea is to minimize mempool sizes.

>
> 2) Is it safe for me to release/free the buffers after calling
> pfring_ft_process()?
>
>
> Yes.
>

So it means pfring_ft_process() is synchronous, but where as the callbacks
can be asynchronous.
Ex: pfring_ft_set_flow_export_callback(ft, processFlow, NULL);
The processFlow() callback is asynchronous, will it be scheduled on the
same core?

> 3) Do you have performance numbers doing inline DPI for 1G or 10G links?
>
>
> You should be able to do 10Gbit line-rate, however this also depends on
> your hw. Please note you can use multiple threads (with RSS or ZC software
> distribution) if 1 core is not enough.
>
> 4) Does PF_RING FT/nDPI maintain state information to detect a flow OR
> does it just uses the packet passed to parse L7 info?
>
>
> It keeps some state, as 1 packet is often not enough.
>

Can you please provide any pointers on that?
Another question: Do you know if there is any issues using pf_ring ft lib
as a dynamic module in DPDK secondary process?


>
> Regards
> Alfredo
>
>
> Thanks
>
> Harish
>
>
>
> On Wed, Jun 13, 2018 at 12:14 AM, Alfredo Cardigliano <
> cardigliano@ntop.org> wrote:
>
>> Hi Harish
>> we are currently using those libs to compile our applications on many
>> distributions
>> including Ubuntu 14/16/18, Centos 6/7, Debian 7/8/9, please let us know
>> if you
>> experience issues with the compilation, and we will find a way to handle
>> that.
>>
>> Regards
>> Alfredo
>>
>>
>> On 13 Jun 2018, at 01:40, Harish Patil <patharish@gmail.com> wrote:
>>
>> Hi Alfredo,
>>
>> Thanks for the response.
>> We are planning to use PF_RING FT in our projects.
>> We only have shared libs. How can we integrate and make sure it works in
>> all cases (platforms/GCC)?
>> If we had the sources we could compile along with sources and would be
>> much better.
>>
>> Thanks.
>> Harish
>>
>>
>>
>> On Mon, Jun 11, 2018 at 9:20 AM, Alfredo Cardigliano <
>> cardigliano@ntop.org> wrote:
>>
>>> Hi Harish
>>> no plan atm, however we tried to create a flexible framework, meaning
>>> that people should be able to extend it using the API we created.
>>> Please let us know if something is missing for your use case.
>>>
>>> Thank you
>>> Alfredo
>>>
>>> On 11 Jun 2018, at 18:10, Harish Patil <patharish@gmail.com> wrote:
>>>
>>> Hi Emanuele,
>>> Thanks for the response.
>>> I didn't know that it was not open sourced and didn't expect it either
>>> given that other modules are open sourced.
>>> Are there any plans of open sourcing them?
>>>
>>> Thanks.
>>>
>>>
>>> On Mon, Jun 11, 2018 at 1:37 AM, Emanuele Faranda <faranda@ntop.org>
>>> wrote:
>>>
>>>> Hello,
>>>>
>>>> Sorry but PF_RING FT is not open source.
>>>>
>>>> Regards,
>>>> Emanuele
>>>>
>>>> On 06/08/2018 04:00 AM, Harish Patil wrote:
>>>>
>>>> Hi,
>>>>
>>>> I am new to ntop/ndpi.
>>>> I'm looking for source code of PF_RING FT such as implementations of:
>>>> pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi
>>>> lib.
>>>>
>>>> I only see the function defines in userland/lib/pfring_ft.h but not the
>>>> implementation.
>>>> Could you pls provide some pointers?
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Harish
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing listNtop@listgateway.unipi.ithttp://listgateway.unipi.it/mailman/listinfo/ntop
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
Re: PF_RING FT API implementation [ In reply to ]
Thanks Alfredo.
Replies below.


On Fri, Jun 22, 2018 at 12:29 AM, Alfredo Cardigliano <cardigliano@ntop.org>
wrote:

>
>
> On 22 Jun 2018, at 04:32, Harish Patil <patharish@gmail.com> wrote:
>
> Thanks Alfredo/Emanuele.
>
> I am seeing the below message:
>
> #########################################################################
> # ERROR: You do not seem to have a valid PF_RING FT 7.3.0.180617 license
> [ECFE456A76066A02]
> # ERROR: Please get one at http://shop.ntop.org/.
> #########################################################################
> # We're now working in demo mode with packet capture and
> # transmission limited to 5 minutes
>
> Is there a need to buy a licensed version? Is this newly introduced in the
> later releases?
>
>
> Yes, PF_RING FT, included in latest stable, requires a license.
>

So at least for now if we stick to the older version then we don't need a
license, right?


>
> Few questions regarding the PF_RING FT API behavior:
>
> 1) Does pfring_ft_process() makes a copy of the packet using the pointer
> passed to it? If so, does it copy only the header or entire packet
> including payload?
>
>
> No packet copy at all.
>

In that case, I can only store packet headers in my local DPDK mempools,
right? The idea is to minimize mempool sizes.

>
> 2) Is it safe for me to release/free the buffers after calling
> pfring_ft_process()?
>
>
> Yes.
>

So it means pfring_ft_process() is synchronous, but where as the callbacks
can be asynchronous.
Ex: pfring_ft_set_flow_export_callback(ft, processFlow, NULL);
The processFlow() callback is asynchronous, will it be scheduled on the
same core?

> 3) Do you have performance numbers doing inline DPI for 1G or 10G links?
>
>
> You should be able to do 10Gbit line-rate, however this also depends on
> your hw. Please note you can use multiple threads (with RSS or ZC software
> distribution) if 1 core is not enough.
>
> 4) Does PF_RING FT/nDPI maintain state information to detect a flow OR
> does it just uses the packet passed to parse L7 info?
>
>
> It keeps some state, as 1 packet is often not enough.
>

Can you please provide any pointers on that?
Another question: Do you know if there is any issues using pf_ring ft lib
as a dynamic module in DPDK secondary process?


>
> Regards
> Alfredo
>
>
> Thanks
>
> Harish
>
>
>
> On Wed, Jun 13, 2018 at 12:14 AM, Alfredo Cardigliano <
> cardigliano@ntop.org> wrote:
>
>> Hi Harish
>> we are currently using those libs to compile our applications on many
>> distributions
>> including Ubuntu 14/16/18, Centos 6/7, Debian 7/8/9, please let us know
>> if you
>> experience issues with the compilation, and we will find a way to handle
>> that.
>>
>> Regards
>> Alfredo
>>
>>
>> On 13 Jun 2018, at 01:40, Harish Patil <patharish@gmail.com> wrote:
>>
>> Hi Alfredo,
>>
>> Thanks for the response.
>> We are planning to use PF_RING FT in our projects.
>> We only have shared libs. How can we integrate and make sure it works in
>> all cases (platforms/GCC)?
>> If we had the sources we could compile along with sources and would be
>> much better.
>>
>> Thanks.
>> Harish
>>
>>
>>
>> On Mon, Jun 11, 2018 at 9:20 AM, Alfredo Cardigliano <
>> cardigliano@ntop.org> wrote:
>>
>>> Hi Harish
>>> no plan atm, however we tried to create a flexible framework, meaning
>>> that people should be able to extend it using the API we created.
>>> Please let us know if something is missing for your use case.
>>>
>>> Thank you
>>> Alfredo
>>>
>>> On 11 Jun 2018, at 18:10, Harish Patil <patharish@gmail.com> wrote:
>>>
>>> Hi Emanuele,
>>> Thanks for the response.
>>> I didn't know that it was not open sourced and didn't expect it either
>>> given that other modules are open sourced.
>>> Are there any plans of open sourcing them?
>>>
>>> Thanks.
>>>
>>>
>>> On Mon, Jun 11, 2018 at 1:37 AM, Emanuele Faranda <faranda@ntop.org>
>>> wrote:
>>>
>>>> Hello,
>>>>
>>>> Sorry but PF_RING FT is not open source.
>>>>
>>>> Regards,
>>>> Emanuele
>>>>
>>>> On 06/08/2018 04:00 AM, Harish Patil wrote:
>>>>
>>>> Hi,
>>>>
>>>> I am new to ntop/ndpi.
>>>> I'm looking for source code of PF_RING FT such as implementations of:
>>>> pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi
>>>> lib.
>>>>
>>>> I only see the function defines in userland/lib/pfring_ft.h but not the
>>>> implementation.
>>>> Could you pls provide some pointers?
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Harish
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing listNtop@listgateway.unipi.ithttp://listgateway.unipi.it/mailman/listinfo/ntop
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
Re: PF_RING FT API implementation [ In reply to ]
> On 22 Jun 2018, at 16:45, Harish Patil <patharish@gmail.com> wrote:
>
> Thanks Alfredo.
> Replies below.
>
>
> On Fri, Jun 22, 2018 at 12:29 AM, Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>> wrote:
>
>
>> On 22 Jun 2018, at 04:32, Harish Patil <patharish@gmail.com <mailto:patharish@gmail.com>> wrote:
>>
>> Thanks Alfredo/Emanuele.
>>
>> I am seeing the below message:
>>
>> #########################################################################
>> # ERROR: You do not seem to have a valid PF_RING FT 7.3.0.180617 license [ECFE456A76066A02]
>> # ERROR: Please get one at http://shop.ntop.org/ <http://shop.ntop.org/>.
>> #########################################################################
>> # We're now working in demo mode with packet capture and
>> # transmission limited to 5 minutes
>>
>> Is there a need to buy a licensed version? Is this newly introduced in the later releases?
>
> Yes, PF_RING FT, included in latest stable, requires a license.
>
> So at least for now if we stick to the older version then we don't need a license, right?

What do you mean with “older version”? Just to clarify, please note that:
1. PF_RING does not require a license, if you enable PF_RING FT, which is an extension, that requires a license,
2. PF_RING FT was not present in older versions of PF_RING,
3. you can use PF_RING FT without PF_RING for packet capture

>> Few questions regarding the PF_RING FT API behavior:
>>
>> 1) Does pfring_ft_process() makes a copy of the packet using the pointer passed to it? If so, does it copy only the header or entire packet including payload?
>
> No packet copy at all.
>
> In that case, I can only store packet headers in my local DPDK mempools, right? The idea is to minimize mempool sizes.
>
>> 2) Is it safe for me to release/free the buffers after calling pfring_ft_process()?
>
> Yes.
>
> So it means pfring_ft_process() is synchronous, but where as the callbacks can be asynchronous.
> Ex: pfring_ft_set_flow_export_callback(ft, processFlow, NULL);
> The processFlow() callback is asynchronous, will it be scheduled on the same core?

Callbacks are called inline by the main packet processing thread, both while processing packets
(with process_packet as in the example at https://github.com/ntop/PF_RING/blob/dev/userland/examples_ft/ftflow.c#L253 <https://github.com/ntop/PF_RING/blob/dev/userland/examples_ft/ftflow.c#L253>)
or when idle (see pfring_ft_housekeeping in https://github.com/ntop/PF_RING/blob/dev/userland/examples_ft/ftflow.c#L255 <https://github.com/ntop/PF_RING/blob/dev/userland/examples_ft/ftflow.c#L255> ).

>> 3) Do you have performance numbers doing inline DPI for 1G or 10G links?
>
> You should be able to do 10Gbit line-rate, however this also depends on your hw. Please note you can use multiple threads (with RSS or ZC software distribution) if 1 core is not enough.
>
>> 4) Does PF_RING FT/nDPI maintain state information to detect a flow OR does it just uses the packet passed to parse L7 info?
>
> It keeps some state, as 1 packet is often not enough.
>
> Can you please provide any pointers on that?

This is done internally by FT, it is not visible through the API, you just get teh L7 protocol as soon as it is detected.

> Another question: Do you know if there is any issues using pf_ring ft lib as a dynamic module in DPDK secondary process?

I do not see issues, but I am not 100% sure I got what you mean with "as a dynamic module in DPDK secondary process”.

Regards
Alfredo

>
>
> Regards
> Alfredo
>
>>
>> Thanks
>>
>> Harish
>>
>>
>>
>> On Wed, Jun 13, 2018 at 12:14 AM, Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>> wrote:
>> Hi Harish
>> we are currently using those libs to compile our applications on many distributions
>> including Ubuntu 14/16/18, Centos 6/7, Debian 7/8/9, please let us know if you
>> experience issues with the compilation, and we will find a way to handle that.
>>
>> Regards
>> Alfredo
>>
>>
>>> On 13 Jun 2018, at 01:40, Harish Patil <patharish@gmail.com <mailto:patharish@gmail.com>> wrote:
>>>
>>> Hi Alfredo,
>>>
>>> Thanks for the response.
>>> We are planning to use PF_RING FT in our projects.
>>> We only have shared libs. How can we integrate and make sure it works in all cases (platforms/GCC)?
>>> If we had the sources we could compile along with sources and would be much better.
>>>
>>> Thanks.
>>> Harish
>>>
>>>
>>>
>>> On Mon, Jun 11, 2018 at 9:20 AM, Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>> wrote:
>>> Hi Harish
>>> no plan atm, however we tried to create a flexible framework, meaning
>>> that people should be able to extend it using the API we created.
>>> Please let us know if something is missing for your use case.
>>>
>>> Thank you
>>> Alfredo
>>>
>>>> On 11 Jun 2018, at 18:10, Harish Patil <patharish@gmail.com <mailto:patharish@gmail.com>> wrote:
>>>>
>>>> Hi Emanuele,
>>>> Thanks for the response.
>>>> I didn't know that it was not open sourced and didn't expect it either given that other modules are open sourced.
>>>> Are there any plans of open sourcing them?
>>>>
>>>> Thanks.
>>>>
>>>>
>>>> On Mon, Jun 11, 2018 at 1:37 AM, Emanuele Faranda <faranda@ntop.org <mailto:faranda@ntop.org>> wrote:
>>>> Hello,
>>>>
>>>> Sorry but PF_RING FT is not open source.
>>>>
>>>> Regards,
>>>> Emanuele
>>>>
>>>>
>>>> On 06/08/2018 04:00 AM, Harish Patil wrote:
>>>>> Hi,
>>>>>
>>>>> I am new to ntop/ndpi.
>>>>> I'm looking for source code of PF_RING FT such as implementations of:
>>>>> pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi lib.
>>>>>
>>>>> I only see the function defines in userland/lib/pfring_ft.h but not the implementation.
>>>>> Could you pls provide some pointers?
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Harish
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Ntop mailing list
>>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
Re: PF_RING FT API implementation [ In reply to ]
> On 22 Jun 2018, at 16:45, Harish Patil <patharish@gmail.com> wrote:
>
> Thanks Alfredo.
> Replies below.
>
>
> On Fri, Jun 22, 2018 at 12:29 AM, Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>> wrote:
>
>
>> On 22 Jun 2018, at 04:32, Harish Patil <patharish@gmail.com <mailto:patharish@gmail.com>> wrote:
>>
>> Thanks Alfredo/Emanuele.
>>
>> I am seeing the below message:
>>
>> #########################################################################
>> # ERROR: You do not seem to have a valid PF_RING FT 7.3.0.180617 license [ECFE456A76066A02]
>> # ERROR: Please get one at http://shop.ntop.org/ <http://shop.ntop.org/>.
>> #########################################################################
>> # We're now working in demo mode with packet capture and
>> # transmission limited to 5 minutes
>>
>> Is there a need to buy a licensed version? Is this newly introduced in the later releases?
>
> Yes, PF_RING FT, included in latest stable, requires a license.
>
> So at least for now if we stick to the older version then we don't need a license, right?

What do you mean with “older version”? Just to clarify, please note that:
1. PF_RING does not require a license, if you enable PF_RING FT, which is an extension, that requires a license,
2. PF_RING FT was not present in older versions of PF_RING,
3. you can use PF_RING FT without PF_RING for packet capture

>> Few questions regarding the PF_RING FT API behavior:
>>
>> 1) Does pfring_ft_process() makes a copy of the packet using the pointer passed to it? If so, does it copy only the header or entire packet including payload?
>
> No packet copy at all.
>
> In that case, I can only store packet headers in my local DPDK mempools, right? The idea is to minimize mempool sizes.
>
>> 2) Is it safe for me to release/free the buffers after calling pfring_ft_process()?
>
> Yes.
>
> So it means pfring_ft_process() is synchronous, but where as the callbacks can be asynchronous.
> Ex: pfring_ft_set_flow_export_callback(ft, processFlow, NULL);
> The processFlow() callback is asynchronous, will it be scheduled on the same core?

Callbacks are called inline by the main packet processing thread, both while processing packets
(with process_packet as in the example at https://github.com/ntop/PF_RING/blob/dev/userland/examples_ft/ftflow.c#L253 <https://github.com/ntop/PF_RING/blob/dev/userland/examples_ft/ftflow.c#L253>)
or when idle (see pfring_ft_housekeeping in https://github.com/ntop/PF_RING/blob/dev/userland/examples_ft/ftflow.c#L255 <https://github.com/ntop/PF_RING/blob/dev/userland/examples_ft/ftflow.c#L255> ).

>> 3) Do you have performance numbers doing inline DPI for 1G or 10G links?
>
> You should be able to do 10Gbit line-rate, however this also depends on your hw. Please note you can use multiple threads (with RSS or ZC software distribution) if 1 core is not enough.
>
>> 4) Does PF_RING FT/nDPI maintain state information to detect a flow OR does it just uses the packet passed to parse L7 info?
>
> It keeps some state, as 1 packet is often not enough.
>
> Can you please provide any pointers on that?

This is done internally by FT, it is not visible through the API, you just get teh L7 protocol as soon as it is detected.

> Another question: Do you know if there is any issues using pf_ring ft lib as a dynamic module in DPDK secondary process?

I do not see issues, but I am not 100% sure I got what you mean with "as a dynamic module in DPDK secondary process”.

Regards
Alfredo

>
>
> Regards
> Alfredo
>
>>
>> Thanks
>>
>> Harish
>>
>>
>>
>> On Wed, Jun 13, 2018 at 12:14 AM, Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>> wrote:
>> Hi Harish
>> we are currently using those libs to compile our applications on many distributions
>> including Ubuntu 14/16/18, Centos 6/7, Debian 7/8/9, please let us know if you
>> experience issues with the compilation, and we will find a way to handle that.
>>
>> Regards
>> Alfredo
>>
>>
>>> On 13 Jun 2018, at 01:40, Harish Patil <patharish@gmail.com <mailto:patharish@gmail.com>> wrote:
>>>
>>> Hi Alfredo,
>>>
>>> Thanks for the response.
>>> We are planning to use PF_RING FT in our projects.
>>> We only have shared libs. How can we integrate and make sure it works in all cases (platforms/GCC)?
>>> If we had the sources we could compile along with sources and would be much better.
>>>
>>> Thanks.
>>> Harish
>>>
>>>
>>>
>>> On Mon, Jun 11, 2018 at 9:20 AM, Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>> wrote:
>>> Hi Harish
>>> no plan atm, however we tried to create a flexible framework, meaning
>>> that people should be able to extend it using the API we created.
>>> Please let us know if something is missing for your use case.
>>>
>>> Thank you
>>> Alfredo
>>>
>>>> On 11 Jun 2018, at 18:10, Harish Patil <patharish@gmail.com <mailto:patharish@gmail.com>> wrote:
>>>>
>>>> Hi Emanuele,
>>>> Thanks for the response.
>>>> I didn't know that it was not open sourced and didn't expect it either given that other modules are open sourced.
>>>> Are there any plans of open sourcing them?
>>>>
>>>> Thanks.
>>>>
>>>>
>>>> On Mon, Jun 11, 2018 at 1:37 AM, Emanuele Faranda <faranda@ntop.org <mailto:faranda@ntop.org>> wrote:
>>>> Hello,
>>>>
>>>> Sorry but PF_RING FT is not open source.
>>>>
>>>> Regards,
>>>> Emanuele
>>>>
>>>>
>>>> On 06/08/2018 04:00 AM, Harish Patil wrote:
>>>>> Hi,
>>>>>
>>>>> I am new to ntop/ndpi.
>>>>> I'm looking for source code of PF_RING FT such as implementations of:
>>>>> pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi lib.
>>>>>
>>>>> I only see the function defines in userland/lib/pfring_ft.h but not the implementation.
>>>>> Could you pls provide some pointers?
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Harish
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Ntop mailing list
>>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
Re: PF_RING FT API implementation [ In reply to ]
Hi Alfredo,
Thanks for the response.
Regarding performance question #3.
When you say 10G line rate, are you saying 10G line rate with single core
and is it true for all frame sizes?
Assuming standard Intel 10G NIC (like 82599) which can do line rate @64B
over DPDK/PF_RING, do we have test results that shows how many ports/cores
needed for inline DPI?

Another question: I happen to run some HTTP traffic over DPDK (doing just
L2 forwarding) and did not see any flows detected (using PF_RING FT APIs).
Do you know what typically could cause that and any debug info that can be
gathered?

Thanks,
Harish





On Fri, Jun 22, 2018 at 8:09 AM, Alfredo Cardigliano <cardigliano@ntop.org>
wrote:

>
>
> On 22 Jun 2018, at 16:45, Harish Patil <patharish@gmail.com> wrote:
>
> Thanks Alfredo.
> Replies below.
>
>
> On Fri, Jun 22, 2018 at 12:29 AM, Alfredo Cardigliano <cardigliano@ntop.
> org> wrote:
>
>>
>>
>> On 22 Jun 2018, at 04:32, Harish Patil <patharish@gmail.com> wrote:
>>
>> Thanks Alfredo/Emanuele.
>>
>> I am seeing the below message:
>>
>> #########################################################################
>> # ERROR: You do not seem to have a valid PF_RING FT 7.3.0.180617 license
>> [ECFE456A76066A02]
>> # ERROR: Please get one at http://shop.ntop.org/.
>> #########################################################################
>> # We're now working in demo mode with packet capture and
>> # transmission limited to 5 minutes
>>
>> Is there a need to buy a licensed version? Is this newly introduced in
>> the later releases?
>>
>>
>> Yes, PF_RING FT, included in latest stable, requires a license.
>>
>
> So at least for now if we stick to the older version then we don't need a
> license, right?
>
>
> What do you mean with “older version”? Just to clarify, please note that:
> 1. PF_RING does not require a license, if you enable PF_RING FT, which is
> an extension, that requires a license,
> 2. PF_RING FT was not present in older versions of PF_RING,
> 3. you can use PF_RING FT without PF_RING for packet capture
>
> Few questions regarding the PF_RING FT API behavior:
>>
>> 1) Does pfring_ft_process() makes a copy of the packet using the pointer
>> passed to it? If so, does it copy only the header or entire packet
>> including payload?
>>
>>
>> No packet copy at all.
>>
>
> In that case, I can only store packet headers in my local DPDK mempools,
> right? The idea is to minimize mempool sizes.
>
>>
>> 2) Is it safe for me to release/free the buffers after calling
>> pfring_ft_process()?
>>
>>
>> Yes.
>>
>
> So it means pfring_ft_process() is synchronous, but where as the callbacks
> can be asynchronous.
> Ex: pfring_ft_set_flow_export_callback(ft, processFlow, NULL);
> The processFlow() callback is asynchronous, will it be scheduled on the
> same core?
>
>
> Callbacks are called inline by the main packet processing thread, both
> while processing packets
> (with process_packet as in the example at https://github.com/ntop/PF_
> RING/blob/dev/userland/examples_ft/ftflow.c#L253)
> or when idle (see pfring_ft_housekeeping in https://github.com/ntop/PF_
> RING/blob/dev/userland/examples_ft/ftflow.c#L255 ).
>
> 3) Do you have performance numbers doing inline DPI for 1G or 10G links?
>>
>>
>> You should be able to do 10Gbit line-rate, however this also depends on
>> your hw. Please note you can use multiple threads (with RSS or ZC software
>> distribution) if 1 core is not enough.
>>
>> 4) Does PF_RING FT/nDPI maintain state information to detect a flow OR
>> does it just uses the packet passed to parse L7 info?
>>
>>
>> It keeps some state, as 1 packet is often not enough.
>>
>
> Can you please provide any pointers on that?
>
>
> This is done internally by FT, it is not visible through the API, you just
> get teh L7 protocol as soon as it is detected.
>
> Another question: Do you know if there is any issues using pf_ring ft lib
> as a dynamic module in DPDK secondary process?
>
>
> I do not see issues, but I am not 100% sure I got what you mean with "as a
> dynamic module in DPDK secondary process”.
>
> Regards
> Alfredo
>
>
>
>>
>> Regards
>> Alfredo
>>
>>
>> Thanks
>>
>> Harish
>>
>>
>>
>> On Wed, Jun 13, 2018 at 12:14 AM, Alfredo Cardigliano <cardigliano@ntop.
>> org> wrote:
>>
>>> Hi Harish
>>> we are currently using those libs to compile our applications on many
>>> distributions
>>> including Ubuntu 14/16/18, Centos 6/7, Debian 7/8/9, please let us know
>>> if you
>>> experience issues with the compilation, and we will find a way to handle
>>> that.
>>>
>>> Regards
>>> Alfredo
>>>
>>>
>>> On 13 Jun 2018, at 01:40, Harish Patil <patharish@gmail.com> wrote:
>>>
>>> Hi Alfredo,
>>>
>>> Thanks for the response.
>>> We are planning to use PF_RING FT in our projects.
>>> We only have shared libs. How can we integrate and make sure it works in
>>> all cases (platforms/GCC)?
>>> If we had the sources we could compile along with sources and would be
>>> much better.
>>>
>>> Thanks.
>>> Harish
>>>
>>>
>>>
>>> On Mon, Jun 11, 2018 at 9:20 AM, Alfredo Cardigliano <cardigliano@ntop.
>>> org> wrote:
>>>
>>>> Hi Harish
>>>> no plan atm, however we tried to create a flexible framework, meaning
>>>> that people should be able to extend it using the API we created.
>>>> Please let us know if something is missing for your use case.
>>>>
>>>> Thank you
>>>> Alfredo
>>>>
>>>> On 11 Jun 2018, at 18:10, Harish Patil <patharish@gmail.com> wrote:
>>>>
>>>> Hi Emanuele,
>>>> Thanks for the response.
>>>> I didn't know that it was not open sourced and didn't expect it either
>>>> given that other modules are open sourced.
>>>> Are there any plans of open sourcing them?
>>>>
>>>> Thanks.
>>>>
>>>>
>>>> On Mon, Jun 11, 2018 at 1:37 AM, Emanuele Faranda <faranda@ntop.org>
>>>> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> Sorry but PF_RING FT is not open source.
>>>>>
>>>>> Regards,
>>>>> Emanuele
>>>>>
>>>>> On 06/08/2018 04:00 AM, Harish Patil wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I am new to ntop/ndpi.
>>>>> I'm looking for source code of PF_RING FT such as implementations of:
>>>>> pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi
>>>>> lib.
>>>>>
>>>>> I only see the function defines in userland/lib/pfring_ft.h but not
>>>>> the implementation.
>>>>> Could you pls provide some pointers?
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Harish
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Ntop mailing listNtop@listgateway.unipi.ithttp://listgateway.unipi.it/mailman/listinfo/ntop
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Ntop mailing list
>>>>> Ntop@listgateway.unipi.it
>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
Re: PF_RING FT API implementation [ In reply to ]
Hi Alfredo,
Thanks for the response.
Regarding performance question #3.
When you say 10G line rate, are you saying 10G line rate with single core
and is it true for all frame sizes?
Assuming standard Intel 10G NIC (like 82599) which can do line rate @64B
over DPDK/PF_RING, do we have test results that shows how many ports/cores
needed for inline DPI?

Another question: I happen to run some HTTP traffic over DPDK (doing just
L2 forwarding) and did not see any flows detected (using PF_RING FT APIs).
Do you know what typically could cause that and any debug info that can be
gathered?

Thanks,
Harish





On Fri, Jun 22, 2018 at 8:09 AM, Alfredo Cardigliano <cardigliano@ntop.org>
wrote:

>
>
> On 22 Jun 2018, at 16:45, Harish Patil <patharish@gmail.com> wrote:
>
> Thanks Alfredo.
> Replies below.
>
>
> On Fri, Jun 22, 2018 at 12:29 AM, Alfredo Cardigliano <cardigliano@ntop.
> org> wrote:
>
>>
>>
>> On 22 Jun 2018, at 04:32, Harish Patil <patharish@gmail.com> wrote:
>>
>> Thanks Alfredo/Emanuele.
>>
>> I am seeing the below message:
>>
>> #########################################################################
>> # ERROR: You do not seem to have a valid PF_RING FT 7.3.0.180617 license
>> [ECFE456A76066A02]
>> # ERROR: Please get one at http://shop.ntop.org/.
>> #########################################################################
>> # We're now working in demo mode with packet capture and
>> # transmission limited to 5 minutes
>>
>> Is there a need to buy a licensed version? Is this newly introduced in
>> the later releases?
>>
>>
>> Yes, PF_RING FT, included in latest stable, requires a license.
>>
>
> So at least for now if we stick to the older version then we don't need a
> license, right?
>
>
> What do you mean with “older version”? Just to clarify, please note that:
> 1. PF_RING does not require a license, if you enable PF_RING FT, which is
> an extension, that requires a license,
> 2. PF_RING FT was not present in older versions of PF_RING,
> 3. you can use PF_RING FT without PF_RING for packet capture
>
> Few questions regarding the PF_RING FT API behavior:
>>
>> 1) Does pfring_ft_process() makes a copy of the packet using the pointer
>> passed to it? If so, does it copy only the header or entire packet
>> including payload?
>>
>>
>> No packet copy at all.
>>
>
> In that case, I can only store packet headers in my local DPDK mempools,
> right? The idea is to minimize mempool sizes.
>
>>
>> 2) Is it safe for me to release/free the buffers after calling
>> pfring_ft_process()?
>>
>>
>> Yes.
>>
>
> So it means pfring_ft_process() is synchronous, but where as the callbacks
> can be asynchronous.
> Ex: pfring_ft_set_flow_export_callback(ft, processFlow, NULL);
> The processFlow() callback is asynchronous, will it be scheduled on the
> same core?
>
>
> Callbacks are called inline by the main packet processing thread, both
> while processing packets
> (with process_packet as in the example at https://github.com/ntop/PF_
> RING/blob/dev/userland/examples_ft/ftflow.c#L253)
> or when idle (see pfring_ft_housekeeping in https://github.com/ntop/PF_
> RING/blob/dev/userland/examples_ft/ftflow.c#L255 ).
>
> 3) Do you have performance numbers doing inline DPI for 1G or 10G links?
>>
>>
>> You should be able to do 10Gbit line-rate, however this also depends on
>> your hw. Please note you can use multiple threads (with RSS or ZC software
>> distribution) if 1 core is not enough.
>>
>> 4) Does PF_RING FT/nDPI maintain state information to detect a flow OR
>> does it just uses the packet passed to parse L7 info?
>>
>>
>> It keeps some state, as 1 packet is often not enough.
>>
>
> Can you please provide any pointers on that?
>
>
> This is done internally by FT, it is not visible through the API, you just
> get teh L7 protocol as soon as it is detected.
>
> Another question: Do you know if there is any issues using pf_ring ft lib
> as a dynamic module in DPDK secondary process?
>
>
> I do not see issues, but I am not 100% sure I got what you mean with "as a
> dynamic module in DPDK secondary process”.
>
> Regards
> Alfredo
>
>
>
>>
>> Regards
>> Alfredo
>>
>>
>> Thanks
>>
>> Harish
>>
>>
>>
>> On Wed, Jun 13, 2018 at 12:14 AM, Alfredo Cardigliano <cardigliano@ntop.
>> org> wrote:
>>
>>> Hi Harish
>>> we are currently using those libs to compile our applications on many
>>> distributions
>>> including Ubuntu 14/16/18, Centos 6/7, Debian 7/8/9, please let us know
>>> if you
>>> experience issues with the compilation, and we will find a way to handle
>>> that.
>>>
>>> Regards
>>> Alfredo
>>>
>>>
>>> On 13 Jun 2018, at 01:40, Harish Patil <patharish@gmail.com> wrote:
>>>
>>> Hi Alfredo,
>>>
>>> Thanks for the response.
>>> We are planning to use PF_RING FT in our projects.
>>> We only have shared libs. How can we integrate and make sure it works in
>>> all cases (platforms/GCC)?
>>> If we had the sources we could compile along with sources and would be
>>> much better.
>>>
>>> Thanks.
>>> Harish
>>>
>>>
>>>
>>> On Mon, Jun 11, 2018 at 9:20 AM, Alfredo Cardigliano <cardigliano@ntop.
>>> org> wrote:
>>>
>>>> Hi Harish
>>>> no plan atm, however we tried to create a flexible framework, meaning
>>>> that people should be able to extend it using the API we created.
>>>> Please let us know if something is missing for your use case.
>>>>
>>>> Thank you
>>>> Alfredo
>>>>
>>>> On 11 Jun 2018, at 18:10, Harish Patil <patharish@gmail.com> wrote:
>>>>
>>>> Hi Emanuele,
>>>> Thanks for the response.
>>>> I didn't know that it was not open sourced and didn't expect it either
>>>> given that other modules are open sourced.
>>>> Are there any plans of open sourcing them?
>>>>
>>>> Thanks.
>>>>
>>>>
>>>> On Mon, Jun 11, 2018 at 1:37 AM, Emanuele Faranda <faranda@ntop.org>
>>>> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> Sorry but PF_RING FT is not open source.
>>>>>
>>>>> Regards,
>>>>> Emanuele
>>>>>
>>>>> On 06/08/2018 04:00 AM, Harish Patil wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I am new to ntop/ndpi.
>>>>> I'm looking for source code of PF_RING FT such as implementations of:
>>>>> pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi
>>>>> lib.
>>>>>
>>>>> I only see the function defines in userland/lib/pfring_ft.h but not
>>>>> the implementation.
>>>>> Could you pls provide some pointers?
>>>>>
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Harish
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Ntop mailing listNtop@listgateway.unipi.ithttp://listgateway.unipi.it/mailman/listinfo/ntop
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Ntop mailing list
>>>>> Ntop@listgateway.unipi.it
>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
Re: PF_RING FT API implementation [ In reply to ]
Hi Harish
my comments inline

Alfredo

> On 27 Jun 2018, at 19:08, Harish Patil <patharish@gmail.com> wrote:
>
> Hi Alfredo,
> Thanks for the response.
> Regarding performance question #3.
> When you say 10G line rate, are you saying 10G line rate with single core and is it true for all frame sizes?
> Assuming standard Intel 10G NIC (like 82599) which can do line rate @64B over DPDK/PF_RING, do we have test results that shows how many ports/cores needed for inline DPI?

FT on top of PF_RING ZC is able to process 10G line-rate @64b using a single core,
as of DPI, it is hard to say it would be able to do line-rate at any traffic condition as it
depends on several factors, including the number of new flows per second, the l7
protocol type (some dissectors use more cpu than others), etc. What I can say is that
with mixed internet traffic (what we used for testing) it can keep up with 10G full rate.

> Another question: I happen to run some HTTP traffic over DPDK (doing just L2 forwarding) and did not see any flows detected (using PF_RING FT APIs). Do you know what typically could cause that and any debug info that can be gathered?

Uhm, did you try to process the same traffic using PF_RING or libpcap? Or could you
provide a pcap file that we can use to reproduce it?

Regards
Alfredo

> Thanks,
> Harish
>
>
>
>
>
> On Fri, Jun 22, 2018 at 8:09 AM, Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>> wrote:
>
>
>> On 22 Jun 2018, at 16:45, Harish Patil <patharish@gmail.com <mailto:patharish@gmail.com>> wrote:
>>
>> Thanks Alfredo.
>> Replies below.
>>
>>
>> On Fri, Jun 22, 2018 at 12:29 AM, Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>> wrote:
>>
>>
>>> On 22 Jun 2018, at 04:32, Harish Patil <patharish@gmail.com <mailto:patharish@gmail.com>> wrote:
>>>
>>> Thanks Alfredo/Emanuele.
>>>
>>> I am seeing the below message:
>>>
>>> #########################################################################
>>> # ERROR: You do not seem to have a valid PF_RING FT 7.3.0.180617 license [ECFE456A76066A02]
>>> # ERROR: Please get one at http://shop.ntop.org/ <http://shop.ntop.org/>.
>>> #########################################################################
>>> # We're now working in demo mode with packet capture and
>>> # transmission limited to 5 minutes
>>>
>>> Is there a need to buy a licensed version? Is this newly introduced in the later releases?
>>
>> Yes, PF_RING FT, included in latest stable, requires a license.
>>
>> So at least for now if we stick to the older version then we don't need a license, right?
>
> What do you mean with “older version”? Just to clarify, please note that:
> 1. PF_RING does not require a license, if you enable PF_RING FT, which is an extension, that requires a license,
> 2. PF_RING FT was not present in older versions of PF_RING,
> 3. you can use PF_RING FT without PF_RING for packet capture
>
>>> Few questions regarding the PF_RING FT API behavior:
>>>
>>> 1) Does pfring_ft_process() makes a copy of the packet using the pointer passed to it? If so, does it copy only the header or entire packet including payload?
>>
>> No packet copy at all.
>>
>> In that case, I can only store packet headers in my local DPDK mempools, right? The idea is to minimize mempool sizes.
>>
>>> 2) Is it safe for me to release/free the buffers after calling pfring_ft_process()?
>>
>> Yes.
>>
>> So it means pfring_ft_process() is synchronous, but where as the callbacks can be asynchronous.
>> Ex: pfring_ft_set_flow_export_callback(ft, processFlow, NULL);
>> The processFlow() callback is asynchronous, will it be scheduled on the same core?
>
> Callbacks are called inline by the main packet processing thread, both while processing packets
> (with process_packet as in the example at https://github.com/ntop/PF_RING/blob/dev/userland/examples_ft/ftflow.c#L253 <https://github.com/ntop/PF_RING/blob/dev/userland/examples_ft/ftflow.c#L253>)
> or when idle (see pfring_ft_housekeeping in https://github.com/ntop/PF_RING/blob/dev/userland/examples_ft/ftflow.c#L255 <https://github.com/ntop/PF_RING/blob/dev/userland/examples_ft/ftflow.c#L255> ).
>
>>> 3) Do you have performance numbers doing inline DPI for 1G or 10G links?
>>
>> You should be able to do 10Gbit line-rate, however this also depends on your hw. Please note you can use multiple threads (with RSS or ZC software distribution) if 1 core is not enough.
>>
>>> 4) Does PF_RING FT/nDPI maintain state information to detect a flow OR does it just uses the packet passed to parse L7 info?
>>
>> It keeps some state, as 1 packet is often not enough.
>>
>> Can you please provide any pointers on that?
>
> This is done internally by FT, it is not visible through the API, you just get teh L7 protocol as soon as it is detected.
>
>> Another question: Do you know if there is any issues using pf_ring ft lib as a dynamic module in DPDK secondary process?
>
> I do not see issues, but I am not 100% sure I got what you mean with "as a dynamic module in DPDK secondary process”.
>
> Regards
> Alfredo
>
>>
>>
>> Regards
>> Alfredo
>>
>>>
>>> Thanks
>>>
>>> Harish
>>>
>>>
>>>
>>> On Wed, Jun 13, 2018 at 12:14 AM, Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>> wrote:
>>> Hi Harish
>>> we are currently using those libs to compile our applications on many distributions
>>> including Ubuntu 14/16/18, Centos 6/7, Debian 7/8/9, please let us know if you
>>> experience issues with the compilation, and we will find a way to handle that.
>>>
>>> Regards
>>> Alfredo
>>>
>>>
>>>> On 13 Jun 2018, at 01:40, Harish Patil <patharish@gmail.com <mailto:patharish@gmail.com>> wrote:
>>>>
>>>> Hi Alfredo,
>>>>
>>>> Thanks for the response.
>>>> We are planning to use PF_RING FT in our projects.
>>>> We only have shared libs. How can we integrate and make sure it works in all cases (platforms/GCC)?
>>>> If we had the sources we could compile along with sources and would be much better.
>>>>
>>>> Thanks.
>>>> Harish
>>>>
>>>>
>>>>
>>>> On Mon, Jun 11, 2018 at 9:20 AM, Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>> wrote:
>>>> Hi Harish
>>>> no plan atm, however we tried to create a flexible framework, meaning
>>>> that people should be able to extend it using the API we created.
>>>> Please let us know if something is missing for your use case.
>>>>
>>>> Thank you
>>>> Alfredo
>>>>
>>>>> On 11 Jun 2018, at 18:10, Harish Patil <patharish@gmail.com <mailto:patharish@gmail.com>> wrote:
>>>>>
>>>>> Hi Emanuele,
>>>>> Thanks for the response.
>>>>> I didn't know that it was not open sourced and didn't expect it either given that other modules are open sourced.
>>>>> Are there any plans of open sourcing them?
>>>>>
>>>>> Thanks.
>>>>>
>>>>>
>>>>> On Mon, Jun 11, 2018 at 1:37 AM, Emanuele Faranda <faranda@ntop.org <mailto:faranda@ntop.org>> wrote:
>>>>> Hello,
>>>>>
>>>>> Sorry but PF_RING FT is not open source.
>>>>>
>>>>> Regards,
>>>>> Emanuele
>>>>>
>>>>>
>>>>> On 06/08/2018 04:00 AM, Harish Patil wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I am new to ntop/ndpi.
>>>>>> I'm looking for source code of PF_RING FT such as implementations of:
>>>>>> pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi lib.
>>>>>>
>>>>>> I only see the function defines in userland/lib/pfring_ft.h but not the implementation.
>>>>>> Could you pls provide some pointers?
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Harish
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Ntop mailing list
>>>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>>>
>>>>> _______________________________________________
>>>>> Ntop mailing list
>>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>>>
>>>>> _______________________________________________
>>>>> Ntop mailing list
>>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
Re: PF_RING FT API implementation [ In reply to ]
Hi Harish
my comments inline

Alfredo

> On 27 Jun 2018, at 19:08, Harish Patil <patharish@gmail.com> wrote:
>
> Hi Alfredo,
> Thanks for the response.
> Regarding performance question #3.
> When you say 10G line rate, are you saying 10G line rate with single core and is it true for all frame sizes?
> Assuming standard Intel 10G NIC (like 82599) which can do line rate @64B over DPDK/PF_RING, do we have test results that shows how many ports/cores needed for inline DPI?

FT on top of PF_RING ZC is able to process 10G line-rate @64b using a single core,
as of DPI, it is hard to say it would be able to do line-rate at any traffic condition as it
depends on several factors, including the number of new flows per second, the l7
protocol type (some dissectors use more cpu than others), etc. What I can say is that
with mixed internet traffic (what we used for testing) it can keep up with 10G full rate.

> Another question: I happen to run some HTTP traffic over DPDK (doing just L2 forwarding) and did not see any flows detected (using PF_RING FT APIs). Do you know what typically could cause that and any debug info that can be gathered?

Uhm, did you try to process the same traffic using PF_RING or libpcap? Or could you
provide a pcap file that we can use to reproduce it?

Regards
Alfredo

> Thanks,
> Harish
>
>
>
>
>
> On Fri, Jun 22, 2018 at 8:09 AM, Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>> wrote:
>
>
>> On 22 Jun 2018, at 16:45, Harish Patil <patharish@gmail.com <mailto:patharish@gmail.com>> wrote:
>>
>> Thanks Alfredo.
>> Replies below.
>>
>>
>> On Fri, Jun 22, 2018 at 12:29 AM, Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>> wrote:
>>
>>
>>> On 22 Jun 2018, at 04:32, Harish Patil <patharish@gmail.com <mailto:patharish@gmail.com>> wrote:
>>>
>>> Thanks Alfredo/Emanuele.
>>>
>>> I am seeing the below message:
>>>
>>> #########################################################################
>>> # ERROR: You do not seem to have a valid PF_RING FT 7.3.0.180617 license [ECFE456A76066A02]
>>> # ERROR: Please get one at http://shop.ntop.org/ <http://shop.ntop.org/>.
>>> #########################################################################
>>> # We're now working in demo mode with packet capture and
>>> # transmission limited to 5 minutes
>>>
>>> Is there a need to buy a licensed version? Is this newly introduced in the later releases?
>>
>> Yes, PF_RING FT, included in latest stable, requires a license.
>>
>> So at least for now if we stick to the older version then we don't need a license, right?
>
> What do you mean with “older version”? Just to clarify, please note that:
> 1. PF_RING does not require a license, if you enable PF_RING FT, which is an extension, that requires a license,
> 2. PF_RING FT was not present in older versions of PF_RING,
> 3. you can use PF_RING FT without PF_RING for packet capture
>
>>> Few questions regarding the PF_RING FT API behavior:
>>>
>>> 1) Does pfring_ft_process() makes a copy of the packet using the pointer passed to it? If so, does it copy only the header or entire packet including payload?
>>
>> No packet copy at all.
>>
>> In that case, I can only store packet headers in my local DPDK mempools, right? The idea is to minimize mempool sizes.
>>
>>> 2) Is it safe for me to release/free the buffers after calling pfring_ft_process()?
>>
>> Yes.
>>
>> So it means pfring_ft_process() is synchronous, but where as the callbacks can be asynchronous.
>> Ex: pfring_ft_set_flow_export_callback(ft, processFlow, NULL);
>> The processFlow() callback is asynchronous, will it be scheduled on the same core?
>
> Callbacks are called inline by the main packet processing thread, both while processing packets
> (with process_packet as in the example at https://github.com/ntop/PF_RING/blob/dev/userland/examples_ft/ftflow.c#L253 <https://github.com/ntop/PF_RING/blob/dev/userland/examples_ft/ftflow.c#L253>)
> or when idle (see pfring_ft_housekeeping in https://github.com/ntop/PF_RING/blob/dev/userland/examples_ft/ftflow.c#L255 <https://github.com/ntop/PF_RING/blob/dev/userland/examples_ft/ftflow.c#L255> ).
>
>>> 3) Do you have performance numbers doing inline DPI for 1G or 10G links?
>>
>> You should be able to do 10Gbit line-rate, however this also depends on your hw. Please note you can use multiple threads (with RSS or ZC software distribution) if 1 core is not enough.
>>
>>> 4) Does PF_RING FT/nDPI maintain state information to detect a flow OR does it just uses the packet passed to parse L7 info?
>>
>> It keeps some state, as 1 packet is often not enough.
>>
>> Can you please provide any pointers on that?
>
> This is done internally by FT, it is not visible through the API, you just get teh L7 protocol as soon as it is detected.
>
>> Another question: Do you know if there is any issues using pf_ring ft lib as a dynamic module in DPDK secondary process?
>
> I do not see issues, but I am not 100% sure I got what you mean with "as a dynamic module in DPDK secondary process”.
>
> Regards
> Alfredo
>
>>
>>
>> Regards
>> Alfredo
>>
>>>
>>> Thanks
>>>
>>> Harish
>>>
>>>
>>>
>>> On Wed, Jun 13, 2018 at 12:14 AM, Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>> wrote:
>>> Hi Harish
>>> we are currently using those libs to compile our applications on many distributions
>>> including Ubuntu 14/16/18, Centos 6/7, Debian 7/8/9, please let us know if you
>>> experience issues with the compilation, and we will find a way to handle that.
>>>
>>> Regards
>>> Alfredo
>>>
>>>
>>>> On 13 Jun 2018, at 01:40, Harish Patil <patharish@gmail.com <mailto:patharish@gmail.com>> wrote:
>>>>
>>>> Hi Alfredo,
>>>>
>>>> Thanks for the response.
>>>> We are planning to use PF_RING FT in our projects.
>>>> We only have shared libs. How can we integrate and make sure it works in all cases (platforms/GCC)?
>>>> If we had the sources we could compile along with sources and would be much better.
>>>>
>>>> Thanks.
>>>> Harish
>>>>
>>>>
>>>>
>>>> On Mon, Jun 11, 2018 at 9:20 AM, Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>> wrote:
>>>> Hi Harish
>>>> no plan atm, however we tried to create a flexible framework, meaning
>>>> that people should be able to extend it using the API we created.
>>>> Please let us know if something is missing for your use case.
>>>>
>>>> Thank you
>>>> Alfredo
>>>>
>>>>> On 11 Jun 2018, at 18:10, Harish Patil <patharish@gmail.com <mailto:patharish@gmail.com>> wrote:
>>>>>
>>>>> Hi Emanuele,
>>>>> Thanks for the response.
>>>>> I didn't know that it was not open sourced and didn't expect it either given that other modules are open sourced.
>>>>> Are there any plans of open sourcing them?
>>>>>
>>>>> Thanks.
>>>>>
>>>>>
>>>>> On Mon, Jun 11, 2018 at 1:37 AM, Emanuele Faranda <faranda@ntop.org <mailto:faranda@ntop.org>> wrote:
>>>>> Hello,
>>>>>
>>>>> Sorry but PF_RING FT is not open source.
>>>>>
>>>>> Regards,
>>>>> Emanuele
>>>>>
>>>>>
>>>>> On 06/08/2018 04:00 AM, Harish Patil wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I am new to ntop/ndpi.
>>>>>> I'm looking for source code of PF_RING FT such as implementations of:
>>>>>> pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi lib.
>>>>>>
>>>>>> I only see the function defines in userland/lib/pfring_ft.h but not the implementation.
>>>>>> Could you pls provide some pointers?
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Harish
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Ntop mailing list
>>>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>>>
>>>>> _______________________________________________
>>>>> Ntop mailing list
>>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>>>
>>>>> _______________________________________________
>>>>> Ntop mailing list
>>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it <mailto:Ntop@listgateway.unipi.it>
> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
Re: PF_RING FT API implementation [ In reply to ]
Thanks Alfredo,
Regarding last question, please ignore it, it was a mistake on my side.


On Wed, Jun 27, 2018 at 10:18 AM, Alfredo Cardigliano <cardigliano@ntop.org>
wrote:

> Hi Harish
> my comments inline
>
> Alfredo
>
> On 27 Jun 2018, at 19:08, Harish Patil <patharish@gmail.com> wrote:
>
> Hi Alfredo,
> Thanks for the response.
> Regarding performance question #3.
> When you say 10G line rate, are you saying 10G line rate with single core
> and is it true for all frame sizes?
> Assuming standard Intel 10G NIC (like 82599) which can do line rate @64B
> over DPDK/PF_RING, do we have test results that shows how many ports/cores
> needed for inline DPI?
>
>
> FT on top of PF_RING ZC is able to process 10G line-rate @64b using a
> single core,
> as of DPI, it is hard to say it would be able to do line-rate at any
> traffic condition as it
> depends on several factors, including the number of new flows per second,
> the l7
> protocol type (some dissectors use more cpu than others), etc. What I can
> say is that
> with mixed internet traffic (what we used for testing) it can keep up with
> 10G full rate.
>
> Another question: I happen to run some HTTP traffic over DPDK (doing just
> L2 forwarding) and did not see any flows detected (using PF_RING FT APIs).
> Do you know what typically could cause that and any debug info that can be
> gathered?
>
>
> Uhm, did you try to process the same traffic using PF_RING or libpcap? Or
> could you
> provide a pcap file that we can use to reproduce it?
>
> Regards
> Alfredo
>
> Thanks,
> Harish
>
>
>
>
>
> On Fri, Jun 22, 2018 at 8:09 AM, Alfredo Cardigliano <cardigliano@ntop.org
> > wrote:
>
>>
>>
>> On 22 Jun 2018, at 16:45, Harish Patil <patharish@gmail.com> wrote:
>>
>> Thanks Alfredo.
>> Replies below.
>>
>>
>> On Fri, Jun 22, 2018 at 12:29 AM, Alfredo Cardigliano <cardigliano@ntop.
>> org> wrote:
>>
>>>
>>>
>>> On 22 Jun 2018, at 04:32, Harish Patil <patharish@gmail.com> wrote:
>>>
>>> Thanks Alfredo/Emanuele.
>>>
>>> I am seeing the below message:
>>>
>>> ############################################################
>>> #############
>>> # ERROR: You do not seem to have a valid PF_RING FT 7.3.0.180617 license
>>> [ECFE456A76066A02]
>>> # ERROR: Please get one at http://shop.ntop.org/.
>>> ############################################################
>>> #############
>>> # We're now working in demo mode with packet capture and
>>> # transmission limited to 5 minutes
>>>
>>> Is there a need to buy a licensed version? Is this newly introduced in
>>> the later releases?
>>>
>>>
>>> Yes, PF_RING FT, included in latest stable, requires a license.
>>>
>>
>> So at least for now if we stick to the older version then we don't need a
>> license, right?
>>
>>
>> What do you mean with “older version”? Just to clarify, please note that:
>> 1. PF_RING does not require a license, if you enable PF_RING FT, which is
>> an extension, that requires a license,
>> 2. PF_RING FT was not present in older versions of PF_RING,
>> 3. you can use PF_RING FT without PF_RING for packet capture
>>
>> Few questions regarding the PF_RING FT API behavior:
>>>
>>> 1) Does pfring_ft_process() makes a copy of the packet using the pointer
>>> passed to it? If so, does it copy only the header or entire packet
>>> including payload?
>>>
>>>
>>> No packet copy at all.
>>>
>>
>> In that case, I can only store packet headers in my local DPDK mempools,
>> right? The idea is to minimize mempool sizes.
>>
>>>
>>> 2) Is it safe for me to release/free the buffers after calling
>>> pfring_ft_process()?
>>>
>>>
>>> Yes.
>>>
>>
>> So it means pfring_ft_process() is synchronous, but where as the
>> callbacks can be asynchronous.
>> Ex: pfring_ft_set_flow_export_callback(ft, processFlow, NULL);
>> The processFlow() callback is asynchronous, will it be scheduled on the
>> same core?
>>
>>
>> Callbacks are called inline by the main packet processing thread, both
>> while processing packets
>> (with process_packet as in the example at https://github.com/ntop/PF_
>> RING/blob/dev/userland/examples_ft/ftflow.c#L253)
>> or when idle (see pfring_ft_housekeeping in
>> https://github.com/ntop/PF_RING/blob/dev/userland/examples_
>> ft/ftflow.c#L255 ).
>>
>> 3) Do you have performance numbers doing inline DPI for 1G or 10G links?
>>>
>>>
>>> You should be able to do 10Gbit line-rate, however this also depends on
>>> your hw. Please note you can use multiple threads (with RSS or ZC software
>>> distribution) if 1 core is not enough.
>>>
>>> 4) Does PF_RING FT/nDPI maintain state information to detect a flow OR
>>> does it just uses the packet passed to parse L7 info?
>>>
>>>
>>> It keeps some state, as 1 packet is often not enough.
>>>
>>
>> Can you please provide any pointers on that?
>>
>>
>> This is done internally by FT, it is not visible through the API, you
>> just get teh L7 protocol as soon as it is detected.
>>
>> Another question: Do you know if there is any issues using pf_ring ft lib
>> as a dynamic module in DPDK secondary process?
>>
>>
>> I do not see issues, but I am not 100% sure I got what you mean with "as
>> a dynamic module in DPDK secondary process”.
>>
>> Regards
>> Alfredo
>>
>>
>>
>>>
>>> Regards
>>> Alfredo
>>>
>>>
>>> Thanks
>>>
>>> Harish
>>>
>>>
>>>
>>> On Wed, Jun 13, 2018 at 12:14 AM, Alfredo Cardigliano <cardigliano@ntop.
>>> org> wrote:
>>>
>>>> Hi Harish
>>>> we are currently using those libs to compile our applications on many
>>>> distributions
>>>> including Ubuntu 14/16/18, Centos 6/7, Debian 7/8/9, please let us know
>>>> if you
>>>> experience issues with the compilation, and we will find a way to
>>>> handle that.
>>>>
>>>> Regards
>>>> Alfredo
>>>>
>>>>
>>>> On 13 Jun 2018, at 01:40, Harish Patil <patharish@gmail.com> wrote:
>>>>
>>>> Hi Alfredo,
>>>>
>>>> Thanks for the response.
>>>> We are planning to use PF_RING FT in our projects.
>>>> We only have shared libs. How can we integrate and make sure it works
>>>> in all cases (platforms/GCC)?
>>>> If we had the sources we could compile along with sources and would be
>>>> much better.
>>>>
>>>> Thanks.
>>>> Harish
>>>>
>>>>
>>>>
>>>> On Mon, Jun 11, 2018 at 9:20 AM, Alfredo Cardigliano <cardigliano@ntop.
>>>> org> wrote:
>>>>
>>>>> Hi Harish
>>>>> no plan atm, however we tried to create a flexible framework, meaning
>>>>> that people should be able to extend it using the API we created.
>>>>> Please let us know if something is missing for your use case.
>>>>>
>>>>> Thank you
>>>>> Alfredo
>>>>>
>>>>> On 11 Jun 2018, at 18:10, Harish Patil <patharish@gmail.com> wrote:
>>>>>
>>>>> Hi Emanuele,
>>>>> Thanks for the response.
>>>>> I didn't know that it was not open sourced and didn't expect it either
>>>>> given that other modules are open sourced.
>>>>> Are there any plans of open sourcing them?
>>>>>
>>>>> Thanks.
>>>>>
>>>>>
>>>>> On Mon, Jun 11, 2018 at 1:37 AM, Emanuele Faranda <faranda@ntop.org>
>>>>> wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> Sorry but PF_RING FT is not open source.
>>>>>>
>>>>>> Regards,
>>>>>> Emanuele
>>>>>>
>>>>>> On 06/08/2018 04:00 AM, Harish Patil wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I am new to ntop/ndpi.
>>>>>> I'm looking for source code of PF_RING FT such as implementations of:
>>>>>> pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi
>>>>>> lib.
>>>>>>
>>>>>> I only see the function defines in userland/lib/pfring_ft.h but not
>>>>>> the implementation.
>>>>>> Could you pls provide some pointers?
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Harish
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Ntop mailing listNtop@listgateway.unipi.ithttp://listgateway.unipi.it/mailman/listinfo/ntop
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Ntop mailing list
>>>>>> Ntop@listgateway.unipi.it
>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Ntop mailing list
>>>>> Ntop@listgateway.unipi.it
>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Ntop mailing list
>>>>> Ntop@listgateway.unipi.it
>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
Re: PF_RING FT API implementation [ In reply to ]
Thanks Alfredo,
Regarding last question, please ignore it, it was a mistake on my side.


On Wed, Jun 27, 2018 at 10:18 AM, Alfredo Cardigliano <cardigliano@ntop.org>
wrote:

> Hi Harish
> my comments inline
>
> Alfredo
>
> On 27 Jun 2018, at 19:08, Harish Patil <patharish@gmail.com> wrote:
>
> Hi Alfredo,
> Thanks for the response.
> Regarding performance question #3.
> When you say 10G line rate, are you saying 10G line rate with single core
> and is it true for all frame sizes?
> Assuming standard Intel 10G NIC (like 82599) which can do line rate @64B
> over DPDK/PF_RING, do we have test results that shows how many ports/cores
> needed for inline DPI?
>
>
> FT on top of PF_RING ZC is able to process 10G line-rate @64b using a
> single core,
> as of DPI, it is hard to say it would be able to do line-rate at any
> traffic condition as it
> depends on several factors, including the number of new flows per second,
> the l7
> protocol type (some dissectors use more cpu than others), etc. What I can
> say is that
> with mixed internet traffic (what we used for testing) it can keep up with
> 10G full rate.
>
> Another question: I happen to run some HTTP traffic over DPDK (doing just
> L2 forwarding) and did not see any flows detected (using PF_RING FT APIs).
> Do you know what typically could cause that and any debug info that can be
> gathered?
>
>
> Uhm, did you try to process the same traffic using PF_RING or libpcap? Or
> could you
> provide a pcap file that we can use to reproduce it?
>
> Regards
> Alfredo
>
> Thanks,
> Harish
>
>
>
>
>
> On Fri, Jun 22, 2018 at 8:09 AM, Alfredo Cardigliano <cardigliano@ntop.org
> > wrote:
>
>>
>>
>> On 22 Jun 2018, at 16:45, Harish Patil <patharish@gmail.com> wrote:
>>
>> Thanks Alfredo.
>> Replies below.
>>
>>
>> On Fri, Jun 22, 2018 at 12:29 AM, Alfredo Cardigliano <cardigliano@ntop.
>> org> wrote:
>>
>>>
>>>
>>> On 22 Jun 2018, at 04:32, Harish Patil <patharish@gmail.com> wrote:
>>>
>>> Thanks Alfredo/Emanuele.
>>>
>>> I am seeing the below message:
>>>
>>> ############################################################
>>> #############
>>> # ERROR: You do not seem to have a valid PF_RING FT 7.3.0.180617 license
>>> [ECFE456A76066A02]
>>> # ERROR: Please get one at http://shop.ntop.org/.
>>> ############################################################
>>> #############
>>> # We're now working in demo mode with packet capture and
>>> # transmission limited to 5 minutes
>>>
>>> Is there a need to buy a licensed version? Is this newly introduced in
>>> the later releases?
>>>
>>>
>>> Yes, PF_RING FT, included in latest stable, requires a license.
>>>
>>
>> So at least for now if we stick to the older version then we don't need a
>> license, right?
>>
>>
>> What do you mean with “older version”? Just to clarify, please note that:
>> 1. PF_RING does not require a license, if you enable PF_RING FT, which is
>> an extension, that requires a license,
>> 2. PF_RING FT was not present in older versions of PF_RING,
>> 3. you can use PF_RING FT without PF_RING for packet capture
>>
>> Few questions regarding the PF_RING FT API behavior:
>>>
>>> 1) Does pfring_ft_process() makes a copy of the packet using the pointer
>>> passed to it? If so, does it copy only the header or entire packet
>>> including payload?
>>>
>>>
>>> No packet copy at all.
>>>
>>
>> In that case, I can only store packet headers in my local DPDK mempools,
>> right? The idea is to minimize mempool sizes.
>>
>>>
>>> 2) Is it safe for me to release/free the buffers after calling
>>> pfring_ft_process()?
>>>
>>>
>>> Yes.
>>>
>>
>> So it means pfring_ft_process() is synchronous, but where as the
>> callbacks can be asynchronous.
>> Ex: pfring_ft_set_flow_export_callback(ft, processFlow, NULL);
>> The processFlow() callback is asynchronous, will it be scheduled on the
>> same core?
>>
>>
>> Callbacks are called inline by the main packet processing thread, both
>> while processing packets
>> (with process_packet as in the example at https://github.com/ntop/PF_
>> RING/blob/dev/userland/examples_ft/ftflow.c#L253)
>> or when idle (see pfring_ft_housekeeping in
>> https://github.com/ntop/PF_RING/blob/dev/userland/examples_
>> ft/ftflow.c#L255 ).
>>
>> 3) Do you have performance numbers doing inline DPI for 1G or 10G links?
>>>
>>>
>>> You should be able to do 10Gbit line-rate, however this also depends on
>>> your hw. Please note you can use multiple threads (with RSS or ZC software
>>> distribution) if 1 core is not enough.
>>>
>>> 4) Does PF_RING FT/nDPI maintain state information to detect a flow OR
>>> does it just uses the packet passed to parse L7 info?
>>>
>>>
>>> It keeps some state, as 1 packet is often not enough.
>>>
>>
>> Can you please provide any pointers on that?
>>
>>
>> This is done internally by FT, it is not visible through the API, you
>> just get teh L7 protocol as soon as it is detected.
>>
>> Another question: Do you know if there is any issues using pf_ring ft lib
>> as a dynamic module in DPDK secondary process?
>>
>>
>> I do not see issues, but I am not 100% sure I got what you mean with "as
>> a dynamic module in DPDK secondary process”.
>>
>> Regards
>> Alfredo
>>
>>
>>
>>>
>>> Regards
>>> Alfredo
>>>
>>>
>>> Thanks
>>>
>>> Harish
>>>
>>>
>>>
>>> On Wed, Jun 13, 2018 at 12:14 AM, Alfredo Cardigliano <cardigliano@ntop.
>>> org> wrote:
>>>
>>>> Hi Harish
>>>> we are currently using those libs to compile our applications on many
>>>> distributions
>>>> including Ubuntu 14/16/18, Centos 6/7, Debian 7/8/9, please let us know
>>>> if you
>>>> experience issues with the compilation, and we will find a way to
>>>> handle that.
>>>>
>>>> Regards
>>>> Alfredo
>>>>
>>>>
>>>> On 13 Jun 2018, at 01:40, Harish Patil <patharish@gmail.com> wrote:
>>>>
>>>> Hi Alfredo,
>>>>
>>>> Thanks for the response.
>>>> We are planning to use PF_RING FT in our projects.
>>>> We only have shared libs. How can we integrate and make sure it works
>>>> in all cases (platforms/GCC)?
>>>> If we had the sources we could compile along with sources and would be
>>>> much better.
>>>>
>>>> Thanks.
>>>> Harish
>>>>
>>>>
>>>>
>>>> On Mon, Jun 11, 2018 at 9:20 AM, Alfredo Cardigliano <cardigliano@ntop.
>>>> org> wrote:
>>>>
>>>>> Hi Harish
>>>>> no plan atm, however we tried to create a flexible framework, meaning
>>>>> that people should be able to extend it using the API we created.
>>>>> Please let us know if something is missing for your use case.
>>>>>
>>>>> Thank you
>>>>> Alfredo
>>>>>
>>>>> On 11 Jun 2018, at 18:10, Harish Patil <patharish@gmail.com> wrote:
>>>>>
>>>>> Hi Emanuele,
>>>>> Thanks for the response.
>>>>> I didn't know that it was not open sourced and didn't expect it either
>>>>> given that other modules are open sourced.
>>>>> Are there any plans of open sourcing them?
>>>>>
>>>>> Thanks.
>>>>>
>>>>>
>>>>> On Mon, Jun 11, 2018 at 1:37 AM, Emanuele Faranda <faranda@ntop.org>
>>>>> wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> Sorry but PF_RING FT is not open source.
>>>>>>
>>>>>> Regards,
>>>>>> Emanuele
>>>>>>
>>>>>> On 06/08/2018 04:00 AM, Harish Patil wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I am new to ntop/ndpi.
>>>>>> I'm looking for source code of PF_RING FT such as implementations of:
>>>>>> pfring_ft_create_table() or pfring_ft_process() etc which uses ndpi
>>>>>> lib.
>>>>>>
>>>>>> I only see the function defines in userland/lib/pfring_ft.h but not
>>>>>> the implementation.
>>>>>> Could you pls provide some pointers?
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Harish
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Ntop mailing listNtop@listgateway.unipi.ithttp://listgateway.unipi.it/mailman/listinfo/ntop
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Ntop mailing list
>>>>>> Ntop@listgateway.unipi.it
>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Ntop mailing list
>>>>> Ntop@listgateway.unipi.it
>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Ntop mailing list
>>>>> Ntop@listgateway.unipi.it
>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>