Mailing List Archive

Hello Darren,

Please check your Figure against the LVS-DR standart .
i.e http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.LVS-DR.html

Please note that the LVS-DR Common setup are 2 ( or more ) real servers and 2 directors .

Please note its Common that the real servers are seperat nodes

Please note that Corosync & pachemaker needs to be have an Group for the VIP + ldirectord , otherwise that not work out.
Corosync setup Virtual IP, Colocation of IP - sample ( see i.e http://clusterlabs.org/wiki/Using_ldirectord )

crm configure primitive WEBIP ocf:heartbeat:IPaddr2 op monitor interval="60" timeout="20" params ip="192.168.43.14" nic="eth0:1" cidr_netmask="32" \
lvs_support="true" is-managed="true" multiple-active="stop_start" target-role="Started"

crm configure primitive LVS ocf:heartbeat:ldirectord params configfile="/etc/ha.d/ldirectord.cf" op monitor interval="60" timeout="120" op start timeout="120" op stop timeout="120"

crm configure colocation LVS_Colo inf: LVS WEBIP:Master
crm configure order LVS_after_WEBIP inf: WEBIP:promote LVS:start
crm configure group LVSGROUP MAILIP SQLIP WEBIP LDAPIP LVS



This are from my LB1(debian 7):
adm@lb1 ~ $ cat /etc/network/interfaces
# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)

auto lo

# The loopback interface
iface lo inet loopback

iface eth0 inet static
address 192.168.3.13
netmask 255.255.255.0
broadcast 192.168.3.255
gateway 192.168.3.1

dns-nameservers 192.168.3.10 192.168.3.11

iface eth1 inet static
address 10.11.10.1
netmask 255.255.255.0
broadcast 10.11.10.255


----------
adm@app1 ~ $ cat /etc/network/interfaces
# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)

# The loopback interface
auto lo
iface lo inet loopback

# IPv4 eth0 Interface
auto eth0
iface eth0 inet static
address 192.168.3.15
netmask 255.255.255.0
broadcast 192.168.3.255
gateway 192.168.3.1

dns-nameservers 192.168.3.10 192.168.43.11

# webip
auto lo:0
iface lo:0 inet static
address 192.168.3.14
netmask 255.255.255.255
broadcast 192.168.3.14
gateway 192.168.3.1

----------------
rnot@app1 ~ $ cat /etc/sysctl.conf
#
# /etc/sysctl.conf - Configuration file for setting system variables
# See /etc/sysctl.d/ for additonal system variables
# See sysctl.conf (5) for information.
#

# Uncomment the following to stop low-level messages on console
kernel.printk = 3 4 1 3


# rpi tweaks
vm.swappiness=1
vm.min_free_kbytes = 8192


# Increase defaults for IPC (bnc#146656)
kernel.msgmax = 65536
kernel.msgmnb = 65536
kernel.sem = 250 256000 32 1024

# https://www.masarlabs.com/noarp/
# ipvs settings for realservers:

net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 0
net.ipv4.conf.all.arp_announce = 0
adm@app1 ~ $




--
Mit freundlichen Grüßen / Best Regards

Horst Venzke ; PGP NET : 1024G/082F2E6D ; http://www.remsnet.de

Legal Notice: This transmittal and/or attachments may be privileged or confidential. It is intended solely for the addressee named above. Any review, dissemination, or copying is strictly prohibited. If you received this transmittal in error, please notify us immediately by reply and immediately delete this message and all its attachments. Thank you.


> Gesendet: Donnerstag, 04. Juli 2013 um 23:55 Uhr
> Von: "Darren Mansell" <darren.mansell@gmail.com>
> An: lvs-users@linuxvirtualserver.org
> Betreff: [lvs-users] Can't Access Other Director/Real Server
>
> Hi all,
>
> I have a 2 node Pacemaker cluster using ldirector to control my IPVS rules.
>
> I'm setting it up in the same way I've done for years, put the VIP on
> loopback as /32, and also as an extra IP on eth0 as /24.
>
> The cluster will add the VIP to eth0 as 192.168.1.10/24 on one of the
> nodes, and the same VIP will be on lo as 192.168.1.10/32 on both nodes.
>
> If I try to access 192.168.1.10 on the node with the VIP active on eth0 and
> with the IPVS rules active, the requests directed to the local node work,
> but fail going to the other node. IPVS is using gate (direct server return).
>
> This has worked fine for years but my latest build using Ubuntu 12.04 isn't
> working. I've tried capturing some traffic but nothing jumps out.
>
> I've tried to make a diagram below. Any help would be appreciated. Thanks.
>
> Client
> |
> |
> Real/director1--------------------------------------Real/director2
> eth0:192.168.1.1 eth0:192.168.1.2
> eth0:192.168.1.10(VIP)
> lo:192.168.1.10(VIP) lo:192.168.1.10(VIP)
>
> Real/director1 only gets responses when the VIP traffic is sent to itself.
> When traffic is directed to the other node it fails (times out)
>
> Traffic originating from the client is balanced and directed fine.
>
> The standard sysctl.conf arp params are on there.
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

Hi Horst,

Many thanks for your reply. Your config is almost exactly the same as mine,
apart from I have the real servers combined with the directors.

As I say it's worked before with this exact config, I just can't figure out
why it's suddenly not.

Thanks again.
Darren


On 5 July 2013 09:45, Horst Venzke-Fa Remsnet Ltd <support@remsnet.de>wrote:

> Hello Darren,
>
> Please check your Figure against the LVS-DR standart .
> i.e http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.LVS-DR.html
>
> Please note that the LVS-DR Common setup are 2 ( or more ) real servers
> and 2 directors .
>
> Please note its Common that the real servers are seperat nodes
>
> Please note that Corosync & pachemaker needs to be have an Group for the
> VIP + ldirectord , otherwise that not work out.
> Corosync setup Virtual IP, Colocation of IP - sample ( see i.e
> http://clusterlabs.org/wiki/Using_ldirectord )
>
> crm configure primitive WEBIP ocf:heartbeat:IPaddr2 op monitor
> interval="60" timeout="20" params ip="192.168.43.14" nic="eth0:1"
> cidr_netmask="32" \
> lvs_support="true" is-managed="true" multiple-active="stop_start"
> target-role="Started"
>
> crm configure primitive LVS ocf:heartbeat:ldirectord params
> configfile="/etc/ha.d/ldirectord.cf" op monitor interval="60"
> timeout="120" op start timeout="120" op stop timeout="120"
>
> crm configure colocation LVS_Colo inf: LVS WEBIP:Master
> crm configure order LVS_after_WEBIP inf: WEBIP:promote LVS:start
> crm configure group LVSGROUP MAILIP SQLIP WEBIP LDAPIP LVS
>
>
>
> This are from my LB1(debian 7):
> adm@lb1 ~ $ cat /etc/network/interfaces
> # /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)
>
> auto lo
>
> # The loopback interface
> iface lo inet loopback
>
> iface eth0 inet static
> address 192.168.3.13
> netmask 255.255.255.0
> broadcast 192.168.3.255
> gateway 192.168.3.1
>
> dns-nameservers 192.168.3.10 192.168.3.11
>
> iface eth1 inet static
> address 10.11.10.1
> netmask 255.255.255.0
> broadcast 10.11.10.255
>
>
> ----------
> adm@app1 ~ $ cat /etc/network/interfaces
> # /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)
>
> # The loopback interface
> auto lo
> iface lo inet loopback
>
> # IPv4 eth0 Interface
> auto eth0
> iface eth0 inet static
> address 192.168.3.15
> netmask 255.255.255.0
> broadcast 192.168.3.255
> gateway 192.168.3.1
>
> dns-nameservers 192.168.3.10 192.168.43.11
>
> # webip
> auto lo:0
> iface lo:0 inet static
> address 192.168.3.14
> netmask 255.255.255.255
> broadcast 192.168.3.14
> gateway 192.168.3.1
>
> ----------------
> rnot@app1 ~ $ cat /etc/sysctl.conf
> #
> # /etc/sysctl.conf - Configuration file for setting system variables
> # See /etc/sysctl.d/ for additonal system variables
> # See sysctl.conf (5) for information.
> #
>
> # Uncomment the following to stop low-level messages on console
> kernel.printk = 3 4 1 3
>
>
> # rpi tweaks
> vm.swappiness=1
> vm.min_free_kbytes = 8192
>
>
> # Increase defaults for IPC (bnc#146656)
> kernel.msgmax = 65536
> kernel.msgmnb = 65536
> kernel.sem = 250 256000 32 1024
>
> # https://www.masarlabs.com/noarp/
> # ipvs settings for realservers:
>
> net.ipv4.conf.lo.arp_ignore = 1
> net.ipv4.conf.lo.arp_announce = 2
> net.ipv4.conf.all.arp_ignore = 0
> net.ipv4.conf.all.arp_announce = 0
> adm@app1 ~ $
>
>
>
>
> --
> Mit freundlichen Grüßen / Best Regards
>
> Horst Venzke ; PGP NET : 1024G/082F2E6D ; http://www.remsnet.de
>
> Legal Notice: This transmittal and/or attachments may be privileged or
> confidential. It is intended solely for the addressee named above. Any
> review, dissemination, or copying is strictly prohibited. If you received
> this transmittal in error, please notify us immediately by reply and
> immediately delete this message and all its attachments. Thank you.
>
>
> > Gesendet: Donnerstag, 04. Juli 2013 um 23:55 Uhr
> > Von: "Darren Mansell" <darren.mansell@gmail.com>
> > An: lvs-users@linuxvirtualserver.org
> > Betreff: [lvs-users] Can't Access Other Director/Real Server
> >
> > Hi all,
> >
> > I have a 2 node Pacemaker cluster using ldirector to control my IPVS
> rules.
> >
> > I'm setting it up in the same way I've done for years, put the VIP on
> > loopback as /32, and also as an extra IP on eth0 as /24.
> >
> > The cluster will add the VIP to eth0 as 192.168.1.10/24 on one of the
> > nodes, and the same VIP will be on lo as 192.168.1.10/32 on both nodes.
> >
> > If I try to access 192.168.1.10 on the node with the VIP active on eth0
> and
> > with the IPVS rules active, the requests directed to the local node work,
> > but fail going to the other node. IPVS is using gate (direct server
> return).
> >
> > This has worked fine for years but my latest build using Ubuntu 12.04
> isn't
> > working. I've tried capturing some traffic but nothing jumps out.
> >
> > I've tried to make a diagram below. Any help would be appreciated.
> Thanks.
> >
> > Client
> > |
> > |
> > Real/director1--------------------------------------Real/director2
> > eth0:192.168.1.1 eth0:192.168.1.2
> > eth0:192.168.1.10(VIP)
> > lo:192.168.1.10(VIP) lo:192.168.1.10(VIP)
> >
> > Real/director1 only gets responses when the VIP traffic is sent to
> itself.
> > When traffic is directed to the other node it fails (times out)
> >
> > Traffic originating from the client is balanced and directed fine.
> >
> > The standard sysctl.conf arp params are on there.
> > _______________________________________________
> > Please read the documentation before posting - it's available at:
> > http://www.linuxvirtualserver.org/
> >
> > LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> > Send requests to lvs-users-request@LinuxVirtualServer.org
> > or go to http://lists.graemef.net/mailman/listinfo/lvs-users
> >
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

On 7/5/13 11:41 AM, Darren Mansell wrote:
> Hi Horst,
>
> Many thanks for your reply. Your config is almost exactly the same as mine,
> apart from I have the real servers combined with the directors.
>
> As I say it's worked before with this exact config, I just can't figure out
> why it's suddenly not.
What source IP is the system using when it is talking to the VIP? Is it
perhaps using the VIP itself as a source IP, then the packets get routed
over to the other box and it sends them to 'itself'? Quick look at
tcpdump on both boxes should tell you.

For what it is worth, my RHEL systems with /32s on loopbacks show the
src address being the the same address:

# ip ro get 172.31.0.5
local 172.31.0.5 dev lo src 172.31.0.5
cache <local> mtu 16436 advmss 16396 hoplimit 64

You could probably just add a SNAT rule to your OUTPUT chain (in NAT
table) and force it to use the eth0 address to talk to the loopback.

David

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users