Mailing List Archive

Hi Marco
it could be tcpdump is using the standard usec pcap file format for dumping the file, however I connot say more as I guess you forgot to enclose the pcap :-)

Alfredo

> On 01 Jun 2016, at 07:15, Marco Kwok <maruko.kwok@gmail.com> wrote:
>
> Hello all,
>
> My objective is to have nanosecond precision timestamp for packets.
> My settings is:
> NIC: intel i350 on eth1
> root@test:~/Desktop/PF_RING/userland/tcpdump-4.6.2# cat /proc/net/pf_ring/dev/eth1/info
> Name: eth1
> Index: 39
> Address: 2C:53:4A:02:30:40
> Polling Mode: NAPI/ZC
> Type: Ethernet
> Family: Intel igb 82580/i350 HW TS
> Max # TX Queues: 1
> # Used RX Queues: 1
> Num RX Slots: 2048
> Num TX Slots: 2048
>
> OS: ubuntu 14.04
> pf_ring: 6.3.0
> root@test:~/Desktop/PF_RING/userland/tcpdump-4.6.2# cat /proc/net/pf_ring/info
> PF_RING Version : 6.3.0 (dev:db41a41185577ba1b7eb5d1fefc2fdb55d12ec04)
> Total rings : 0
>
> Standard (non DNA/ZC) Options
> Ring slots : 4096
> Slot version : 16
> Capture TX : Yes [RX+TX]
> IP Defragment : No
> Socket Mode : Standard
> Total plugins : 0
> Cluster Fragment Queue : 0
> Cluster Fragment Discard : 0
>
>
> If I use tcpdump to capture packet and disply on screen, the timestamp is in nanosecond precision. For example:
> root@test:~/Desktop/PF_RING/userland/tcpdump-4.6.2# ./tcpdump -i eth1 --time-stamp-precision=nano
> Warning: Kernel filter failed: Bad address
> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
> listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
> 07:51:53.228382757 IP 192.168.30.53.53722 > 224.0.0.251.mdns: 0 PTR (QU)? _spotify-connect._tcp.local. (45)
> 07:51:53.228395385 IP 192.168.30.53.53722 > 224.0.0.251.mdns: 0 PTR (QU)? _spotify-connect._tcp.local. (45)
> 07:51:53.228397614 IP 192.168.30.53.53722 > 224.0.0.251.mdns: 0 PTR (QU)? _spotify-connect._tcp.local. (45)
> 07:51:53.228399436 IP 192.168.30.53.53722 > 224.0.0.251.mdns: 0 PTR (QU)? _spotify-connect._tcp.local. (45)
> 07:51:53.228401157 IP 192.168.30.53.53722 > 224.0.0.251.mdns: 0 PTR (QU)? _spotify-connect._tcp.local. (45)
> 07:51:53.228404600 IP 192.168.30.53.53722 > 224.0.0.251.mdns: 0 PTR (QU)? _spotify-connect._tcp.local. (45)
> 07:51:53.228488883 IP 192.168.30.53.50059 > 239.255.255.250.1900: UDP, length 127
> 07:51:53.228500907 IP 192.168.30.53.50059 > 239.255.255.250.1900: UDP, length 127
> 07:51:53.228502558 IP 192.168.30.53.50059 > 239.255.255.250.1900: UDP, length 127
> 07:51:53.228503806 IP 192.168.30.53.50059 > 239.255.255.250.1900: UDP, length 127
> 07:51:53.228505403 IP 192.168.30.53.50059 > 239.255.255.250.1900: UDP, length 127
> 07:51:53.228506555 IP 192.168.30.53.50059 > 239.255.255.250.1900: UDP, length 127
> 07:51:53.327980623 STP 802.1d, Config, Flags [none], bridge-id 8001.00:19:aa:54:19:00.8001, length 43
> 07:51:53.328532139 STP 802.1d, Config, Flags [none], bridge-id 8001.00:19:aa:54:19:00.8003, length 43
> 07:51:53.328812509 STP 802.1d, Config, Flags [none], bridge-id 8001.00:19:aa:54:19:00.8005, length 43
> 07:51:53.328915619 STP 802.1d, Config, Flags [none], bridge-id 8001.00:19:aa:54:19:00.800d, length 43
> 07:51:53.329010268 STP 802.1d, Config, Flags [none], bridge-id 8001.00:19:aa:54:19:00.800f, length 43
> 07:51:53.329116554 STP 802.1d, Config, Flags [none], bridge-id 8001.00:19:aa:54:19:00.8011, length 43
> 07:51:53.513979065 ARP, Request who-has 192.168.30.200 tell 192.168.30.54, length 46
> 07:51:53.513993983 ARP, Request who-has 192.168.30.200 tell 192.168.30.54, length 46
>
> However, if I capture and write the pcap file using the same command, the nanosecond part is fixed:
> root@test:~/Desktop/PF_RING/userland/tcpdump-4.6.2# ./tcpdump -i eth1 --time-stamp-precision=nano -w b.pcap
> Warning: Kernel filter failed: Bad address
> tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
> 42 packets captured
> 42 packets received by filter
> 0 packets dropped by kernel
> root@test:~/Desktop/PF_RING/userland/tcpdump-4.6.2# ./tcpdump --time-stamp-precision=nano -r b.pcap
> reading from file b.pcap, link-type EN10MB (Ethernet)
> 07:52:10.690324301 IP 192.168.30.22.54915 > 192.168.30.255.54915: UDP, length 263
> 07:52:10.690341301 IP 192.168.30.22.54915 > 192.168.30.255.54915: UDP, length 263
> 07:52:10.690343301 IP 192.168.30.22.54915 > 192.168.30.255.54915: UDP, length 263
> 07:52:10.690345301 IP 192.168.30.22.54915 > 192.168.30.255.54915: UDP, length 263
> 07:52:10.690347301 IP 192.168.30.22.54915 > 192.168.30.255.54915: UDP, length 263
> 07:52:10.690348301 IP 192.168.30.22.54915 > 192.168.30.255.54915: UDP, length 263
> 07:52:10.690436301 IP 192.168.30.22.54915 > 192.168.30.255.54915: UDP, length 263
> 07:52:10.690451301 IP 192.168.30.22.54915 > 192.168.30.255.54915: UDP, length 263
> 07:52:10.690453301 IP 192.168.30.22.54915 > 192.168.30.255.54915: UDP, length 263
> 07:52:10.690454301 IP 192.168.30.22.54915 > 192.168.30.255.54915: UDP, length 263
> 07:52:10.690456301 IP 192.168.30.22.54915 > 192.168.30.255.54915: UDP, length 263
> 07:52:10.690457301 IP 192.168.30.22.54915 > 192.168.30.255.54915: UDP, length 263
> 07:52:11.368855301 STP 802.1d, Config, Flags [none], bridge-id 8001.00:19:aa:54:19:00.8001, length 43
> 07:52:11.369131301 STP 802.1d, Config, Flags [none], bridge-id 8001.00:19:aa:54:19:00.8003, length 43
> 07:52:11.369235301 STP 802.1d, Config, Flags [none], bridge-id 8001.00:19:aa:54:19:00.8005, length 43
> 07:52:11.369327301 STP 802.1d, Config, Flags [none], bridge-id 8001.00:19:aa:54:19:00.800d, length 43
> 07:52:11.369431301 STP 802.1d, Config, Flags [none], bridge-id 8001.00:19:aa:54:19:00.800f, length 43
> 07:52:11.369535301 STP 802.1d, Config, Flags [none], bridge-id 8001.00:19:aa:54:19:00.8011, length 43
>
> Does anyone know the trick to have the nanosecond timestamp written into the pcap file? Or am I doing sometime wrong in parsing the pcap file.
> I attached the pcap file for your reference.
>
> Thank you for your time, appreciate any comments on this.
>
> Best,
> Mark
>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Hi Alfredo,

Ah, yes I forgot to attach the file. Here it is.

Best,
Mark

2016-06-01 15:44 GMT+08:00 Alfredo Cardigliano <cardigliano@ntop.org>:

> Hi Marco
> it could be tcpdump is using the standard usec pcap file format for
> dumping the file, however I connot say more as I guess you forgot to
> enclose the pcap :-)
>
> Alfredo
>
> On 01 Jun 2016, at 07:15, Marco Kwok <maruko.kwok@gmail.com> wrote:
>
> Hello all,
>
> My objective is to have nanosecond precision timestamp for packets.
>
>
>

Hi Mark
the pcap is in nsec format, did you open it with wireshark? It seems the nsec part is correct, it could be a visualisation issue in tcpdump.

Alfredo

> On 02 Jun 2016, at 07:56, Marco Kwok <maruko.kwok@gmail.com> wrote:
>
> Hi Alfredo,
>
> Ah, yes I forgot to attach the file. Here it is.
>
> Best,
> Mark
>
> 2016-06-01 15:44 GMT+08:00 Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>>:
> Hi Marco
> it could be tcpdump is using the standard usec pcap file format for dumping the file, however I connot say more as I guess you forgot to enclose the pcap :-)
>
> Alfredo
>
>> On 01 Jun 2016, at 07:15, Marco Kwok <maruko.kwok@gmail.com <mailto:maruko.kwok@gmail.com>> wrote:
>>
>> Hello all,
>>
>> My objective is to have nanosecond precision timestamp for packets.
>
>
> <b.pcap>_______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Hi Alfredo,

Yes, I opened it on wireshark but the time stamp is incorrect. It shift the
whole fraction by 1000 times as follows.

​Best,
Mark

2016-06-02 16:38 GMT+08:00 Alfredo Cardigliano <cardigliano@ntop.org>:

> Hi Mark
> the pcap is in nsec format, did you open it with wireshark? It seems the
> nsec part is correct, it could be a visualisation issue in tcpdump.
>
> Alfredo
>
> On 02 Jun 2016, at 07:56, Marco Kwok <maruko.kwok@gmail.com> wrote:
>
> Hi Alfredo,
>
> Ah, yes I forgot to attach the file. Here it is.
>
> Best,
> Mark
>
> 2016-06-01 15:44 GMT+08:00 Alfredo Cardigliano <cardigliano@ntop.org>:
>
>> Hi Marco
>> it could be tcpdump is using the standard usec pcap file format for
>> dumping the file, however I connot say more as I guess you forgot to
>> enclose the pcap :-)
>>
>> Alfredo
>>
>> On 01 Jun 2016, at 07:15, Marco Kwok <maruko.kwok@gmail.com> wrote:
>>
>> Hello all,
>>
>> My objective is to have nanosecond precision timestamp for packets.
>>
>>
>>
> <b.pcap>_______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
>

Hi Marco
you are right, the pcap header is nsec but the time in the packet header is actually usec, thus wireshark treats it as nsec,
I made some changes to libpcap on github, please update and let me know if it fixes this issue.

Alfredo

> On 02 Jun 2016, at 11:24, Marco Kwok <maruko.kwok@gmail.com> wrote:
>
> Hi Alfredo,
>
> Yes, I opened it on wireshark but the time stamp is incorrect. It shift the whole fraction by 1000 times as follows.
> <bpcap-min.png>
> ​Best,
> Mark
>
> 2016-06-02 16:38 GMT+08:00 Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>>:
> Hi Mark
> the pcap is in nsec format, did you open it with wireshark? It seems the nsec part is correct, it could be a visualisation issue in tcpdump.
>
> Alfredo
>
>> On 02 Jun 2016, at 07:56, Marco Kwok <maruko.kwok@gmail.com <mailto:maruko.kwok@gmail.com>> wrote:
>>
>> Hi Alfredo,
>>
>> Ah, yes I forgot to attach the file. Here it is.
>>
>> Best,
>> Mark
>>
>> 2016-06-01 15:44 GMT+08:00 Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>>:
>> Hi Marco
>> it could be tcpdump is using the standard usec pcap file format for dumping the file, however I connot say more as I guess you forgot to enclose the pcap :-)
>>
>> Alfredo
>>
>>> On 01 Jun 2016, at 07:15, Marco Kwok <maruko.kwok@gmail.com <mailto:maruko.kwok@gmail.com>> wrote:
>>>
>>> Hello all,
>>>
>>> My objective is to have nanosecond precision timestamp for packets.
>>
>>
>> <b.pcap>_______________________________________________
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it <mailto:Ntop-misc@listgateway.unipi.it>
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc <http://listgateway.unipi.it/mailman/listinfo/ntop-misc>
>
> _______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc

Hi Alfredo,

After checkout the newest commit, I observed 1 fix needed and
3 behavioral changes:

1. one more line need to be changed to have proper time stamp:

/userland/libpcap-1.7.4/pcap-linux.c
@@ -1709,7 +1709,7 @@ pcap_read_packet(pcap_t *handle, pcap_handler
callback, u_char *userdata)
caplen = pcap_header.caplen, packet_len =
pcap_header.len;
if (pcap_header.extended_hdr.timestamp_ns
&& handle->opt.tstamp_precision == PCAP_TSTAMP_PRECISION_NANO) {
pcap_header.ts.tv_sec =
pcap_header.extended_hdr.timestamp_ns / 1000000000;
- pcap_header.ts.tv_usec =
pcap_header.extended_hdr.timestamp_ns % 1000;
+ pcap_header.ts.tv_usec =
pcap_header.extended_hdr.timestamp_ns % 1000000000;


1. time stamp read by tcpdump different from wireshark

In wireshark, the time stamp is read correctly with
nanosecond precision. In tcpdump, the nanosecond fraction is a constant and
vary everytime we read the file.

1. incorrect packet content and filter not applicable.

The whole packet frame have wrong length and unable to
decode the correct content from MAC level.
Attached the pcap file captured.
Best,
Mark

2016-06-03 1:35 GMT+08:00 Alfredo Cardigliano <cardigliano@ntop.org>:

> Hi Marco
> you are right, the pcap header is nsec but the time in the packet header
> is actually usec, thus wireshark treats it as nsec,
> I made some changes to libpcap on github, please update and let me know if
> it fixes this issue.
>
> Alfredo
>

> On 06 Jun 2016, at 06:10, Marco Kwok <maruko.kwok@gmail.com> wrote:
>
> Hi Alfredo,
>
> After checkout the newest commit, I observed 1 fix needed and 3 behavioral changes:
> one more line need to be changed to have proper time stamp:
> /userland/libpcap-1.7.4/pcap-linux.c
> @@ -1709,7 +1709,7 @@ pcap_read_packet(pcap_t *handle, pcap_handler callback, u_char *userdata)
> caplen = pcap_header.caplen, packet_len = pcap_header.len;
> if (pcap_header.extended_hdr.timestamp_ns && handle->opt.tstamp_precision == PCAP_TSTAMP_PRECISION_NANO) {
> pcap_header.ts.tv_sec = pcap_header.extended_hdr.timestamp_ns / 1000000000;
> - pcap_header.ts.tv_usec = pcap_header.extended_hdr.timestamp_ns % 1000;
> + pcap_header.ts.tv_usec = pcap_header.extended_hdr.timestamp_ns % 1000000000;
>
Fixed, my mistake, thank you.
> time stamp read by tcpdump different from wireshark
> In wireshark, the time stamp is read correctly with nanosecond precision. In tcpdump, the nanosecond fraction is a constant and vary everytime we read the file.

I made more fixes now, please update, it should work as expected.
> incorrect packet content and filter not applicable.

What drivers are you using? Please update everything now as I made a few fixes in several places.

Alfredo

> The whole packet frame have wrong length and unable to decode the correct content from MAC level.
> Attached the pcap file captured.
> Best,
> Mark
>
> 2016-06-03 1:35 GMT+08:00 Alfredo Cardigliano <cardigliano@ntop.org <mailto:cardigliano@ntop.org>>:
> Hi Marco
> you are right, the pcap header is nsec but the time in the packet header is actually usec, thus wireshark treats it as nsec,
> I made some changes to libpcap on github, please update and let me know if it fixes this issue.
>
> Alfredo
>
> <c.pcap>_______________________________________________
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc