Mailing List Archive

Unknown L7 Protocols
Hello,

I am trying to get L7_PROTO_NAME with nprobe. I am using the nprobe as below

nprobe -G -t 60 -d 15 --elastic "flows;nprobe-%Y.%m.%d;
http://10.X.X.X:9200/_bulk" -i eth1 -T "%IN_BYTES %IN_PKTS %PROTOCOL
%L4_SRC_PORT %IPV4_SRC_ADDR %L4_DST_PORT %IPV4_DST_ADDR %SRC_AS %DST_AS
%OUT_BYTES %OUT_PKTS %SRC_VLAN %DST_VLAN %HTTP_URL %HTTP_METHOD %HTTP_HOST
%HTTP_SITE %L7_PROTO %L7_PROTO_NAME %APPL_LATENCY_MS"


The problem here when I am checking the flows via elasticsearch I get two
differen results for exactly the same request,


L7_PROTO_NAME HTTP

L7_PROTO_NAME Unknown.

So what may be the problem here?

Regards,

Lutfi
Re: Unknown L7 Protocols [ In reply to ]
Hi,

Please, explain how to reproduce. Enclose a pcap if you think it will help
as well.


Simone

On Fri, Oct 28, 2016 at 10:46 AM, Lutfi Oduncuoglu <
lutfioduncuoglu@gmail.com> wrote:

> Hello,
>
> I am trying to get L7_PROTO_NAME with nprobe. I am using the nprobe as
> below
>
> nprobe -G -t 60 -d 15 --elastic "flows;nprobe-%Y.%m.%d;http://
> 10.X.X.X:9200/_bulk" -i eth1 -T "%IN_BYTES %IN_PKTS %PROTOCOL
> %L4_SRC_PORT %IPV4_SRC_ADDR %L4_DST_PORT %IPV4_DST_ADDR %SRC_AS %DST_AS
> %OUT_BYTES %OUT_PKTS %SRC_VLAN %DST_VLAN %HTTP_URL %HTTP_METHOD %HTTP_HOST
> %HTTP_SITE %L7_PROTO %L7_PROTO_NAME %APPL_LATENCY_MS"
>
>
> The problem here when I am checking the flows via elasticsearch I get two
> differen results for exactly the same request,
>
>
> L7_PROTO_NAME HTTP
>
> L7_PROTO_NAME Unknown.
>
> So what may be the problem here?
>
> Regards,
>
> Lutfi
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
Re: Unknown L7 Protocols [ In reply to ]
Hello Simone,

Actually it happens in random. I will try to produce a pcap today. Is it
ok, if I I create a pcap with tcpdump while capturing the flows?

Regards,

Lutfi

On Fri, Oct 28, 2016 at 12:27 PM, Simone Mainardi <mainardi@ntop.org> wrote:

> Hi,
>
> Please, explain how to reproduce. Enclose a pcap if you think it will help
> as well.
>
>
> Simone
>
> On Fri, Oct 28, 2016 at 10:46 AM, Lutfi Oduncuoglu <
> lutfioduncuoglu@gmail.com> wrote:
>
>> Hello,
>>
>> I am trying to get L7_PROTO_NAME with nprobe. I am using the nprobe as
>> below
>>
>> nprobe -G -t 60 -d 15 --elastic "flows;nprobe-%Y.%m.%d;http://
>> 10.X.X.X:9200/_bulk" -i eth1 -T "%IN_BYTES %IN_PKTS %PROTOCOL
>> %L4_SRC_PORT %IPV4_SRC_ADDR %L4_DST_PORT %IPV4_DST_ADDR %SRC_AS %DST_AS
>> %OUT_BYTES %OUT_PKTS %SRC_VLAN %DST_VLAN %HTTP_URL %HTTP_METHOD %HTTP_HOST
>> %HTTP_SITE %L7_PROTO %L7_PROTO_NAME %APPL_LATENCY_MS"
>>
>>
>> The problem here when I am checking the flows via elasticsearch I get two
>> differen results for exactly the same request,
>>
>>
>> L7_PROTO_NAME HTTP
>>
>> L7_PROTO_NAME Unknown.
>>
>> So what may be the problem here?
>>
>> Regards,
>>
>> Lutfi
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>
Re: Unknown L7 Protocols [ In reply to ]
Hello,

I tried the reproduce the situation

Below you can see L7_PROTO_NAME=Unknown
{

- "_index": "nprobe-2016.10.27",
- "_type": "flows",
- "_id": "AVgGH5sfdkghXIQ1kFlQ",
- "_version": 1,
- "_score": 1.4142135,
- "_source": {
- "IN_BYTES": 816,
- "IN_PKTS": 6,
- "PROTOCOL": 6,
- "L4_SRC_PORT": 34229,
- "IPV4_SRC_ADDR": "10.119.0.152",
- "L4_DST_PORT": 80,
- "IPV4_DST_ADDR": "212.252.126.9",
- "SRC_AS": 0,
- "DST_AS": 6822,
- "OUT_BYTES": 348,
- "OUT_PKTS": 3,
- "SRC_VLAN": 0,
- "DST_VLAN": 0,
- "HTTP_URL": "
crl.microsoft.com/pki/crl/products/MicCodSigPCA_08-31-2010.crl",
- "HTTP_METHOD": "GET",
- "HTTP_HOST": "crl.microsoft.com",
- "HTTP_SITE": "microsoft.com",
- "L7_PROTO": 0,
- "L7_PROTO_NAME": "Unknown",
- "APPL_LATENCY_MS": 7.568,
- "@version": "1",
- "@timestamp": "2016-10-27T12:31:19Z",
- "EXPORTER_IPV4_ADDRESS": "0.0.0.0"
}

}

and this is another flow from my network with same url etc.

{

- "_index": "nprobe-2016.10.27",
- "_type": "flows",
- "_id": "AVgGHw33dkghXIQ1kFi5",
- "_version": 1,
- "_score": 1.4142135,
- "_source": {
- "IN_BYTES": 738,
- "IN_PKTS": 4,
- "PROTOCOL": 6,
- "L4_SRC_PORT": 34226,
- "IPV4_SRC_ADDR": "10.119.0.152",
- "L4_DST_PORT": 80,
- "IPV4_DST_ADDR": "212.252.126.9",
- "SRC_AS": 0,
- "DST_AS": 6822,
- "OUT_BYTES": 266,
- "OUT_PKTS": 1,
- "SRC_VLAN": 0,
- "DST_VLAN": 0,
- "HTTP_URL": "
crl.microsoft.com/pki/crl/products/MicCodSigPCA_08-31-2010.crl",
- "HTTP_METHOD": "GET",
- "HTTP_HOST": "crl.microsoft.com",
- "HTTP_SITE": "microsoft.com",
- "L7_PROTO": 219,
- "L7_PROTO_NAME": "HTTP.Office365",
- "APPL_LATENCY_MS": 7.212,
- "@version": "1",
- "@timestamp": "2016-10-27T12:30:39Z",
- "EXPORTER_IPV4_ADDRESS": "0.0.0.0"
}

}

here there is no problem with PROTO_NAME

So what may the problem here?

Regards



On Mon, Oct 31, 2016 at 8:23 AM, Lutfi Oduncuoglu <lutfioduncuoglu@gmail.com
> wrote:

> Hello Simone,
>
> Actually it happens in random. I will try to produce a pcap today. Is it
> ok, if I I create a pcap with tcpdump while capturing the flows?
>
> Regards,
>
> Lutfi
>
> On Fri, Oct 28, 2016 at 12:27 PM, Simone Mainardi <mainardi@ntop.org>
> wrote:
>
>> Hi,
>>
>> Please, explain how to reproduce. Enclose a pcap if you think it will
>> help as well.
>>
>>
>> Simone
>>
>> On Fri, Oct 28, 2016 at 10:46 AM, Lutfi Oduncuoglu <
>> lutfioduncuoglu@gmail.com> wrote:
>>
>>> Hello,
>>>
>>> I am trying to get L7_PROTO_NAME with nprobe. I am using the nprobe as
>>> below
>>>
>>> nprobe -G -t 60 -d 15 --elastic "flows;nprobe-%Y.%m.%d;http://
>>> 10.X.X.X:9200/_bulk" -i eth1 -T "%IN_BYTES %IN_PKTS %PROTOCOL
>>> %L4_SRC_PORT %IPV4_SRC_ADDR %L4_DST_PORT %IPV4_DST_ADDR %SRC_AS %DST_AS
>>> %OUT_BYTES %OUT_PKTS %SRC_VLAN %DST_VLAN %HTTP_URL %HTTP_METHOD %HTTP_HOST
>>> %HTTP_SITE %L7_PROTO %L7_PROTO_NAME %APPL_LATENCY_MS"
>>>
>>>
>>> The problem here when I am checking the flows via elasticsearch I get
>>> two differen results for exactly the same request,
>>>
>>>
>>> L7_PROTO_NAME HTTP
>>>
>>> L7_PROTO_NAME Unknown.
>>>
>>> So what may be the problem here?
>>>
>>> Regards,
>>>
>>> Lutfi
>>>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>
>
Re: Unknown L7 Protocols [ In reply to ]
Uhm, this is strange. It looks like the detection is not always successful
on the same request. Can you capture a small pcap with that causes that
particular event so we can reproduce and debug in our lab?

Regards,
Simone

On Mon, Oct 31, 2016 at 11:54 AM, Lutfi Oduncuoglu <
lutfioduncuoglu@gmail.com> wrote:

> Hello,
>
> I tried the reproduce the situation
>
> Below you can see L7_PROTO_NAME=Unknown
> {
>
> - "_index": "nprobe-2016.10.27",
> - "_type": "flows",
> - "_id": "AVgGH5sfdkghXIQ1kFlQ",
> - "_version": 1,
> - "_score": 1.4142135,
> - "_source": {
> - "IN_BYTES": 816,
> - "IN_PKTS": 6,
> - "PROTOCOL": 6,
> - "L4_SRC_PORT": 34229,
> - "IPV4_SRC_ADDR": "10.119.0.152",
> - "L4_DST_PORT": 80,
> - "IPV4_DST_ADDR": "212.252.126.9",
> - "SRC_AS": 0,
> - "DST_AS": 6822,
> - "OUT_BYTES": 348,
> - "OUT_PKTS": 3,
> - "SRC_VLAN": 0,
> - "DST_VLAN": 0,
> - "HTTP_URL": "crl.microsoft.com/pki/crl/
> products/MicCodSigPCA_08-31-2010.crl
> <http://crl.microsoft.com/pki/crl/products/MicCodSigPCA_08-31-2010.crl>
> ",
> - "HTTP_METHOD": "GET",
> - "HTTP_HOST": "crl.microsoft.com",
> - "HTTP_SITE": "microsoft.com",
> - "L7_PROTO": 0,
> - "L7_PROTO_NAME": "Unknown",
> - "APPL_LATENCY_MS": 7.568,
> - "@version": "1",
> - "@timestamp": "2016-10-27T12:31:19Z",
> - "EXPORTER_IPV4_ADDRESS": "0.0.0.0"
> }
>
> }
>
> and this is another flow from my network with same url etc.
>
> {
>
> - "_index": "nprobe-2016.10.27",
> - "_type": "flows",
> - "_id": "AVgGHw33dkghXIQ1kFi5",
> - "_version": 1,
> - "_score": 1.4142135,
> - "_source": {
> - "IN_BYTES": 738,
> - "IN_PKTS": 4,
> - "PROTOCOL": 6,
> - "L4_SRC_PORT": 34226,
> - "IPV4_SRC_ADDR": "10.119.0.152",
> - "L4_DST_PORT": 80,
> - "IPV4_DST_ADDR": "212.252.126.9",
> - "SRC_AS": 0,
> - "DST_AS": 6822,
> - "OUT_BYTES": 266,
> - "OUT_PKTS": 1,
> - "SRC_VLAN": 0,
> - "DST_VLAN": 0,
> - "HTTP_URL": "crl.microsoft.com/pki/crl/
> products/MicCodSigPCA_08-31-2010.crl
> <http://crl.microsoft.com/pki/crl/products/MicCodSigPCA_08-31-2010.crl>
> ",
> - "HTTP_METHOD": "GET",
> - "HTTP_HOST": "crl.microsoft.com",
> - "HTTP_SITE": "microsoft.com",
> - "L7_PROTO": 219,
> - "L7_PROTO_NAME": "HTTP.Office365",
> - "APPL_LATENCY_MS": 7.212,
> - "@version": "1",
> - "@timestamp": "2016-10-27T12:30:39Z",
> - "EXPORTER_IPV4_ADDRESS": "0.0.0.0"
> }
>
> }
>
> here there is no problem with PROTO_NAME
>
> So what may the problem here?
>
> Regards
>
>
>
> On Mon, Oct 31, 2016 at 8:23 AM, Lutfi Oduncuoglu <
> lutfioduncuoglu@gmail.com> wrote:
>
>> Hello Simone,
>>
>> Actually it happens in random. I will try to produce a pcap today. Is it
>> ok, if I I create a pcap with tcpdump while capturing the flows?
>>
>> Regards,
>>
>> Lutfi
>>
>> On Fri, Oct 28, 2016 at 12:27 PM, Simone Mainardi <mainardi@ntop.org>
>> wrote:
>>
>>> Hi,
>>>
>>> Please, explain how to reproduce. Enclose a pcap if you think it will
>>> help as well.
>>>
>>>
>>> Simone
>>>
>>> On Fri, Oct 28, 2016 at 10:46 AM, Lutfi Oduncuoglu <
>>> lutfioduncuoglu@gmail.com> wrote:
>>>
>>>> Hello,
>>>>
>>>> I am trying to get L7_PROTO_NAME with nprobe. I am using the nprobe as
>>>> below
>>>>
>>>> nprobe -G -t 60 -d 15 --elastic "flows;nprobe-%Y.%m.%d;http://
>>>> 10.X.X.X:9200/_bulk" -i eth1 -T "%IN_BYTES %IN_PKTS %PROTOCOL
>>>> %L4_SRC_PORT %IPV4_SRC_ADDR %L4_DST_PORT %IPV4_DST_ADDR %SRC_AS %DST_AS
>>>> %OUT_BYTES %OUT_PKTS %SRC_VLAN %DST_VLAN %HTTP_URL %HTTP_METHOD %HTTP_HOST
>>>> %HTTP_SITE %L7_PROTO %L7_PROTO_NAME %APPL_LATENCY_MS"
>>>>
>>>>
>>>> The problem here when I am checking the flows via elasticsearch I get
>>>> two differen results for exactly the same request,
>>>>
>>>>
>>>> L7_PROTO_NAME HTTP
>>>>
>>>> L7_PROTO_NAME Unknown.
>>>>
>>>> So what may be the problem here?
>>>>
>>>> Regards,
>>>>
>>>> Lutfi
>>>>
>>>>
>>>> _______________________________________________
>>>> Ntop mailing list
>>>> Ntop@listgateway.unipi.it
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>>
>>>
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>
>>
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
>