I have met this issue too.
The reason is that the link layer (network interface card driver program)
reassemble several packets into one packet and submit it to the network
layer (TCP/IP protocol stack ).
How to solve it? Modify the network card configuration, disable the
packet-reassemble option. You may use "ethtool -K "to do it.
key words : LRO£¨Large Receive offload£©, TSO(TCP Segmentation Offload)
NJISPLVS001:~ # ethtool -k eth4
Offload parameters for eth4:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: on
udp fragmentation offload: off
generic segmentation offload: on
large receive offload: off
"§¥§Ö§ß§Ú§ã§Ü§Ñ-§â§Ö§Õ§Ú§ã§Ü§Ñ" <slim@inbox.lv>
·¢¼þÈË: lvs-users-bounces@linuxvirtualserver.org
2013-07-11 22:17
Çë´ð¸´ ¸ø
"LinuxVirtualServer.org users mailing list."
<lvs-users@linuxvirtualserver.org>
ÊÕ¼þÈË
lvs-users@linuxvirtualserver.org
³ËÍ
Ö÷Ìâ
[lvs-users] LVS-DR: director cuts packets with payload size > MSS (1460)
hello,
i have strange issue with LVS in direct routing mode - Director
forwards
between interfaces packets with payload < 1460 only
see below dumps from two interfaces on same box where eth1 is incoming
interface where VIP is up, and eth0 is outgoing interface to
real-servers.
A packet with id 11270 (length 2960) is present in incoming traffic but
absent in outgoing. Same with packet id 11279
why this may happens ?
I am running kernel 3.8.13 without conntrack.
look what happens on interfaces as tcpdump shows:
eth1 is incoming interface where VIP 10.1.1.60 is up:
(virtual) foto1 ~ # tcpdump -i eth1 -vv -n port 80 and host 10.1.1.145
tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size
68
bytes
16:38:34.896032 IP (tos 0x0, ttl 128, id 11268, offset 0, flags [DF],
proto
TCP (6), length 52) 10.1.1.145.63094 > 10.1.1.60.80: S
1211165489:1211165489(0) win 8192 <mss 1460,[|tcp]>
16:38:34.896843 IP (tos 0x0, ttl 128, id 11269, offset 0, flags [DF],
proto
TCP (6), length 40) 10.1.1.145.63094 > 10.1.1.60.80: ., cksum 0x6192
(correct), 1211165490:1211165490(0) ack 2881536955 win 64240
16:38:34.896978 IP (tos 0x0, ttl 128, id 11270, offset 0, flags [DF],
proto
TCP (6), length 2960) 10.1.1.145.63094 > 10.1.1.60.80: . 0:2920(2920)
ack 1
win 64240
16:38:35.206653 IP (tos 0x0, ttl 128, id 11273, offset 0, flags [DF],
proto
TCP (6), length 1500) 10.1.1.145.63094 > 10.1.1.60.80: . 0:1460(1460)
ack 1
win 64240
16:38:35.207487 IP (tos 0x0, ttl 128, id 11274, offset 0, flags [DF],
proto
TCP (6), length 1500) 10.1.1.145.63094 > 10.1.1.60.80: .
2920:4380(1460) ack
1 win 64240
16:38:35.207538 IP (tos 0x0, ttl 128, id 11275, offset 0, flags [DF],
proto
TCP (6), length 1500) 10.1.1.145.63094 > 10.1.1.60.80: .
4380:5840(1460) ack
1 win 64240
16:38:35.208593 IP (tos 0x0, ttl 128, id 11276, offset 0, flags [DF],
proto
TCP (6), length 1500) 10.1.1.145.63094 > 10.1.1.60.80: .
1460:2920(1460) ack
1 win 64240
16:38:35.209105 IP (tos 0x0, ttl 128, id 11277, offset 0, flags [DF],
proto
TCP (6), length 2960) 10.1.1.145.63094 > 10.1.1.60.80: .
5840:8760(2920) ack
1 win 64240
16:38:35.209145 IP (tos 0x0, ttl 128, id 11279, offset 0, flags [DF],
proto
TCP (6), length 2960) 10.1.1.145.63094 > 10.1.1.60.80: .
8760:11680(2920)
ack 1 win 64240
eth0 is outgoing to real-servers interface on same server:
(virtual) foto1 ~ # tcpdump -i eth0 -vv -n port 80 and host 10.1.1.145
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size
68
bytes
16:38:34.896070 IP (tos 0x0, ttl 128, id 11268, offset 0, flags [DF],
proto
TCP (6), length 52) 10.1.1.145.63094 > 10.1.1.60.80: S
1211165489:1211165489(0) win 8192 <mss 1460,[|tcp]>
16:38:34.896863 IP (tos 0x0, ttl 128, id 11269, offset 0, flags [DF],
proto
TCP (6), length 40) 10.1.1.145.63094 > 10.1.1.60.80: ., cksum 0x6192
(correct), 1211165490:1211165490(0) ack 2881536955 win 64240
16:38:35.206692 IP (tos 0x0, ttl 128, id 11273, offset 0, flags [DF],
proto
TCP (6), length 1500) 10.1.1.145.63094 > 10.1.1.60.80: . 0:1460(1460)
ack 1
win 64240
16:38:35.207505 IP (tos 0x0, ttl 128, id 11274, offset 0, flags [DF],
proto
TCP (6), length 1500) 10.1.1.145.63094 > 10.1.1.60.80: .
2920:4380(1460) ack
1 win 64240
16:38:35.207542 IP (tos 0x0, ttl 128, id 11275, offset 0, flags [DF],
proto
TCP (6), length 1500) 10.1.1.145.63094 > 10.1.1.60.80: .
4380:5840(1460) ack
1 win 64240
16:38:35.208610 IP (tos 0x0, ttl 128, id 11276, offset 0, flags [DF],
proto
TCP (6), length 1500) 10.1.1.145.63094 > 10.1.1.60.80: .
1460:2920(1460) ack
1 win 64240
16:38:35.520432 IP (tos 0x0, ttl 128, id 11282, offset 0, flags [DF],
proto
TCP (6), length 1500) 10.1.1.145.63094 > 10.1.1.60.80: .
5840:7300(1460) ack
1 win 64240
16:38:35.523322 IP (tos 0x0, ttl 128, id 11283, offset 0, flags [DF],
proto
TCP (6), length 1500) 10.1.1.145.63094 > 10.1.1.60.80: .
11680:13140(1460)
ack 1 win 64240
Thanks in advance,
Deniss
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
--------------------------------------------------------
ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s). If you are not an intended recipient, any disclosure, reproduction, distribution or other dissemination or use of the information contained is strictly prohibited. If you have received this mail in error, please delete it and notify us immediately.
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
The reason is that the link layer (network interface card driver program)
reassemble several packets into one packet and submit it to the network
layer (TCP/IP protocol stack ).
How to solve it? Modify the network card configuration, disable the
packet-reassemble option. You may use "ethtool -K "to do it.
key words : LRO£¨Large Receive offload£©, TSO(TCP Segmentation Offload)
NJISPLVS001:~ # ethtool -k eth4
Offload parameters for eth4:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp segmentation offload: on
udp fragmentation offload: off
generic segmentation offload: on
large receive offload: off
"§¥§Ö§ß§Ú§ã§Ü§Ñ-§â§Ö§Õ§Ú§ã§Ü§Ñ" <slim@inbox.lv>
·¢¼þÈË: lvs-users-bounces@linuxvirtualserver.org
2013-07-11 22:17
Çë´ð¸´ ¸ø
"LinuxVirtualServer.org users mailing list."
<lvs-users@linuxvirtualserver.org>
ÊÕ¼þÈË
lvs-users@linuxvirtualserver.org
³ËÍ
Ö÷Ìâ
[lvs-users] LVS-DR: director cuts packets with payload size > MSS (1460)
hello,
i have strange issue with LVS in direct routing mode - Director
forwards
between interfaces packets with payload < 1460 only
see below dumps from two interfaces on same box where eth1 is incoming
interface where VIP is up, and eth0 is outgoing interface to
real-servers.
A packet with id 11270 (length 2960) is present in incoming traffic but
absent in outgoing. Same with packet id 11279
why this may happens ?
I am running kernel 3.8.13 without conntrack.
look what happens on interfaces as tcpdump shows:
eth1 is incoming interface where VIP 10.1.1.60 is up:
(virtual) foto1 ~ # tcpdump -i eth1 -vv -n port 80 and host 10.1.1.145
tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size
68
bytes
16:38:34.896032 IP (tos 0x0, ttl 128, id 11268, offset 0, flags [DF],
proto
TCP (6), length 52) 10.1.1.145.63094 > 10.1.1.60.80: S
1211165489:1211165489(0) win 8192 <mss 1460,[|tcp]>
16:38:34.896843 IP (tos 0x0, ttl 128, id 11269, offset 0, flags [DF],
proto
TCP (6), length 40) 10.1.1.145.63094 > 10.1.1.60.80: ., cksum 0x6192
(correct), 1211165490:1211165490(0) ack 2881536955 win 64240
16:38:34.896978 IP (tos 0x0, ttl 128, id 11270, offset 0, flags [DF],
proto
TCP (6), length 2960) 10.1.1.145.63094 > 10.1.1.60.80: . 0:2920(2920)
ack 1
win 64240
16:38:35.206653 IP (tos 0x0, ttl 128, id 11273, offset 0, flags [DF],
proto
TCP (6), length 1500) 10.1.1.145.63094 > 10.1.1.60.80: . 0:1460(1460)
ack 1
win 64240
16:38:35.207487 IP (tos 0x0, ttl 128, id 11274, offset 0, flags [DF],
proto
TCP (6), length 1500) 10.1.1.145.63094 > 10.1.1.60.80: .
2920:4380(1460) ack
1 win 64240
16:38:35.207538 IP (tos 0x0, ttl 128, id 11275, offset 0, flags [DF],
proto
TCP (6), length 1500) 10.1.1.145.63094 > 10.1.1.60.80: .
4380:5840(1460) ack
1 win 64240
16:38:35.208593 IP (tos 0x0, ttl 128, id 11276, offset 0, flags [DF],
proto
TCP (6), length 1500) 10.1.1.145.63094 > 10.1.1.60.80: .
1460:2920(1460) ack
1 win 64240
16:38:35.209105 IP (tos 0x0, ttl 128, id 11277, offset 0, flags [DF],
proto
TCP (6), length 2960) 10.1.1.145.63094 > 10.1.1.60.80: .
5840:8760(2920) ack
1 win 64240
16:38:35.209145 IP (tos 0x0, ttl 128, id 11279, offset 0, flags [DF],
proto
TCP (6), length 2960) 10.1.1.145.63094 > 10.1.1.60.80: .
8760:11680(2920)
ack 1 win 64240
eth0 is outgoing to real-servers interface on same server:
(virtual) foto1 ~ # tcpdump -i eth0 -vv -n port 80 and host 10.1.1.145
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size
68
bytes
16:38:34.896070 IP (tos 0x0, ttl 128, id 11268, offset 0, flags [DF],
proto
TCP (6), length 52) 10.1.1.145.63094 > 10.1.1.60.80: S
1211165489:1211165489(0) win 8192 <mss 1460,[|tcp]>
16:38:34.896863 IP (tos 0x0, ttl 128, id 11269, offset 0, flags [DF],
proto
TCP (6), length 40) 10.1.1.145.63094 > 10.1.1.60.80: ., cksum 0x6192
(correct), 1211165490:1211165490(0) ack 2881536955 win 64240
16:38:35.206692 IP (tos 0x0, ttl 128, id 11273, offset 0, flags [DF],
proto
TCP (6), length 1500) 10.1.1.145.63094 > 10.1.1.60.80: . 0:1460(1460)
ack 1
win 64240
16:38:35.207505 IP (tos 0x0, ttl 128, id 11274, offset 0, flags [DF],
proto
TCP (6), length 1500) 10.1.1.145.63094 > 10.1.1.60.80: .
2920:4380(1460) ack
1 win 64240
16:38:35.207542 IP (tos 0x0, ttl 128, id 11275, offset 0, flags [DF],
proto
TCP (6), length 1500) 10.1.1.145.63094 > 10.1.1.60.80: .
4380:5840(1460) ack
1 win 64240
16:38:35.208610 IP (tos 0x0, ttl 128, id 11276, offset 0, flags [DF],
proto
TCP (6), length 1500) 10.1.1.145.63094 > 10.1.1.60.80: .
1460:2920(1460) ack
1 win 64240
16:38:35.520432 IP (tos 0x0, ttl 128, id 11282, offset 0, flags [DF],
proto
TCP (6), length 1500) 10.1.1.145.63094 > 10.1.1.60.80: .
5840:7300(1460) ack
1 win 64240
16:38:35.523322 IP (tos 0x0, ttl 128, id 11283, offset 0, flags [DF],
proto
TCP (6), length 1500) 10.1.1.145.63094 > 10.1.1.60.80: .
11680:13140(1460)
ack 1 win 64240
Thanks in advance,
Deniss
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
--------------------------------------------------------
ZTE Information Security Notice: The information contained in this mail (and any attachment transmitted herewith) is privileged and confidential and is intended for the exclusive use of the addressee(s). If you are not an intended recipient, any disclosure, reproduction, distribution or other dissemination or use of the information contained is strictly prohibited. If you have received this mail in error, please delete it and notify us immediately.
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/
LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users