Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. Linux Virtual Server>
  3. Users
[lvs-users] IPVS Table update failed - maybe arp suffering issue ?
hv at remsnet
May 2, 2013, 4:11 PM
Post #1 of 1 (852 views)
Permalink
Im sems to suffer from an stange issue with LVS-DR


IPVS-DR Setup:

VIP .240
lb1= .230, lb2= .231
real1= .252, real2= .253

Distro : debian 7.0 ARM, RPI
ARCH : arm6 , PREEMPT kernel
Linux lb1 3.8.8-4-raspberrypi #2 PREEMPT Wed Apr 24 19:31:24 CEST 2013
armv6l GNU/Linux

----------------------------------------------------------------------------
#sysctl -p
kernel.printk = 3 4 1 3
net.ipv4.tcp_syncookies = 1
net.ipv4.ip_forward = 0
net.ipv6.conf.all.forwarding = 0
net.ipv4.conf.all.log_martians = 1
vm.swappiness = 1
vm.min_free_kbytes = 8192
kernel.msgmax = 65536
kernel.msgmnb = 65536
kernel.sem = 250 256000 32 1024
net.ipv4.vs.debug_level = 2

lb1/lb2 #lsmod
Module Size Used by
dlm 138857 17
sctp 188067 3 dlm
dm_mod 72808 1
ip_vs_wrr 2260 0
ip_vs_wlc 1433 0
ip_vs_sh 2589 0
ip_vs_sed 1483 0
ip_vs_rr 1634 0
ip_vs_nq 1492 0
ip_vs_lc 1436 0
ip_vs_lblcr 6889 0
ip_vs_lblc 4909 0
ip_vs_dh 2295 0
ip_vs 158484 21
ip_vs_dh,ip_vs_lc,ip_vs_nq,ip_vs_rr,ip_vs_sh,ip_vs_sed,ip_vs_wlc,ip_vs_wrr,i
p_vs_lblcr,ip_vs_lblc
nf_conntrack 83599 1 ip_vs
i2c_dev 5291 0
rtl8150 9106 0
leds_gpio 3199 0
led_class 3590 1 leds_gpio
root@lb2:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state
UP qlen 1000
link/ether b8:27:eb:b6:7c:e3 brd ff:ff:ff:ff:ff:ff
inet 192.168.17.231/24 brd 192.168.43.255 scope global eth0
inet 192.168.17.240/32 brd 192.168.17.240 scope global eth0:0
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state
UNKNOWN qlen 1000
link/ether 00:0b:2b:0d:48:98 brd ff:ff:ff:ff:ff:ff
inet 10.10.10.2/24 brd 10.10.10.255 scope global eth1
inet 10.10.10.1/24 brd 10.10.10.255 scope global eth1
root@lb1:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state
UP qlen 1000
link/ether b8:27:eb:0d:7e:b6 brd ff:ff:ff:ff:ff:ff
inet 192.168.17.230/24 brd 192.168.43.255 scope global eth0
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state
UNKNOWN qlen 1000
link/ether 00:0b:2b:0f:a2:89 brd ff:ff:ff:ff:ff:ff
inet 10.10.10.1/24 brd 10.10.10.255 scope global eth1

real1#ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet 192.168.17.240/32 brd 192.168.17.240 scope global lo:0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state
UP qlen 1000
link/ether b8:27:eb:97:81:07 brd ff:ff:ff:ff:ff:ff
inet 192.168.17.250/24 brd 192.168.43.255 scope global eth0
inet 192.168.17.252/24 brd 192.168.43.255 scope global secondary eth0:1

ip / real2 a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet 192.168.17.240/32 brd 192.168.17.240 scope global lo:0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state
UP qlen 1000
link/ether b8:27:eb:6a:98:da brd ff:ff:ff:ff:ff:ff
inet 192.168.17.251/24 brd 192.168.43.255 scope global eth0
inet 192.168.17.253/24 brd 192.168.43.255 scope global secondary eth0:1


real1/real2 # sysctl -p
kernel.printk = 3 4 1 3
vm.swappiness = 1
vm.min_free_kbytes = 8192
kernel.msgmax = 65536
kernel.msgmnb = 65536
kernel.sem = 250 256000 32 1024
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 0
net.ipv4.conf.all.arp_announce = 0
----------------------------------------------------------------------------
lb2 # ifconfig eth0:0 192.168.17.240 netmask 255.255.255.255 broadcast
192.168.17.240 arp up
lb2 # /sbin/ipvsadm -A -t 192.168.17.240:80 -s wrr
lb2 # /sbin/ipvsadm -a -t 192.168.17.240:80 -r 192.168.17.254:80 -g -w 100
lb2 # /sbin/ipvsadm -a -t 192.168.17.240:80 -r 192.168.17.255:80 -g -w 100
lb2 # ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.17.240:80 wrr
-> 192.168.17.253:80 Route 100 0 0
-------Manual check -------------------
client1 ~ $ curl real1
<html><body><h1> www1 index.html !</h1>
<p>www1</p>
</body></html>
client1 ~ $ curl real2
<html><body><h1> real2 index.html !</h1>
<p>www2</p>
</body></html>
client1 ~ $ curl vip
<html><body><h1> real1 index.html !</h1>
<p>real1</p>
</body></html>

lb2 # ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.17.240:80 wrr
-> 192.168.17.254:80 Route 100 0 1
---- Manual check-----------results
- got answer ( Ok )
- IPVS answers..
- ipvsadm faild to show the second host
-------stress check -------------------
while true; do curl realip; done
...
<html><body><h1> real2 index.html !</h1>
<p>real2</p>
</body></html>
<html><body><h1> real1 index.html !</h1>
<p>real1</p>
</body></html>
<html><body><h1> real2 index.html !</h1>
<p>real2</p>
</body></html>
<html><body><h1> real1 index.html !</h1>
<p>real1</p>
</body></html>
<html><body><h1> real2 index.html !</h1>
<p>real2</p>
</body></html>
<html><body><h1> real1 index.html !</h1>
<p>real1</p>
</body></html>
^C
...

ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.17.240:80 wrr
-> 192.168.17.253:80 Route 100 0 403
-------stress check ----results---------------

- seems goten answer of BOTH real servers via IPVS ...
!!> how can that bee if one is dropt at ipvs´s list to be ok ?!??!

- ipvs(adm) faild to show the second host
- If i stop real1 , the failover works for ~ 10sek secons and failed then.

- IPVS table not updated when shutdown one of the real servers

- ldirectord or keepalived failed due kernel IPVS table failed updated.


Has someone an basic Idea what mabye wrong here ?




--
Mit freundlichen Grüßen / Best Regards
Horst Venzke ; PGP NET : 1024G/082F2E6D ; http://www.remsnet.de
Legal Notice: This transmittal and/or attachments may be privileged or
confidential. It is intended solely for the addressee named above. Any
review, dissemination, or copying is strictly prohibited. If you received
this transmittal in error, please notify us immediately by reply and
immediately delete this message and all its attachments. Thank you.
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal