Mailing List Archive

Re: [lvs-users] DR : real server unable to reach VIP
Am Mittwoch, 20. Mai 2015, 12:00:01 schrieb Florent B:
> Some precisions :
>
> In fact packets are computed by LVS, the problem is that RS receives a
> packets with :
> - its proper eth0 IP in src, but with mac address of LVS server

"its": which IP in the source field? The source IP address should always be the
IP address of the client. The source address should never be the IP adress of
the real server.

MAC address of the director: That is corret.

> - VIP in dest with its eth0 mac address.

VIP in the destination field should be the VIP if you use direct routing.
Destination MAC address should ne of the real server, otherwise the packet
would not reach the real server.

> I think packet is filtered somewhere in kernel, but I can't find which
> setting to change.

If I understand correct, the source address in the packet that the real server
gets is the IP address of the real server. Of course this is an error. If the
kernel recieves a packet with a soure with its own IP address it discards it.

What does change the source IP address of the packet? Do you have additional
NAT configurured on the director?

Mit freundlichen Grüßen,

Michael Schwartzkopff

--
[*] sys4 AG

http://sys4.de, +49 (89) 30 90 46 64, +49 (162) 165 0044
Franziskanerstraße 15, 81669 München

Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
Re: [lvs-users] DR : real server unable to reach VIP [ In reply to ]
Am Mittwoch, 20. Mai 2015, 12:59:33 schrieb Florent B:
> Thank you a lot for your answer.
>
> On 05/20/2015 12:21 PM, Michael Schwartzkopff wrote:
> > Am Mittwoch, 20. Mai 2015, 12:00:01 schrieb Florent B:
> >> Some precisions :
> >>
> >> In fact packets are computed by LVS, the problem is that RS receives a
> >> packets with :
> >> - its proper eth0 IP in src, but with mac address of LVS server
> >
> > "its": which IP in the source field? The source IP address should
>
> always be the
>
> > IP address of the client. The source address should never be the IP
>
> adress of
>
> > the real server.
> >
> > MAC address of the director: That is corret.
>
> The source IP address is the IP address of the client, but the client
> here is my real server. It tries to connect to VIP.

Why? It could use its own IP address for connetions (127.0.0.1, ::1).
Otherwise you are in trouble.

Mit freundlichen Grüßen,

Michael Schwartzkopff

--
[*] sys4 AG

http://sys4.de, +49 (89) 30 90 46 64, +49 (162) 165 0044
Franziskanerstraße 15, 81669 München

Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
Re: [lvs-users] DR : real server unable to reach VIP [ In reply to ]
On 20.05.2015, Florent B wrote:
> On 05/20/2015 12:21 PM, Michael Schwartzkopff wrote:
> > Am Mittwoch, 20. Mai 2015, 12:00:01 schrieb Florent B:
> >> Some precisions :
> >>
> >> In fact packets are computed by LVS, the problem is that RS receives a
> >> packets with :
> >> - its proper eth0 IP in src, but with mac address of LVS server
> >
> > "its": which IP in the source field? The source IP address should
> always be the
> > IP address of the client. The source address should never be the IP
> adress of
> > the real server.
> >
> > MAC address of the director: That is corret.
>
> The source IP address is the IP address of the client, but the client
> here is my real server. It tries to connect to VIP.

Hi Florent,

You've hit a very special corner case.

When you're trying to access your VIP from a real server, your real server does discover the VIP to be a local IP address. As a consequence, the traffic isn't bpassed onto your local network, but routed via loopback and this traffic is not being loadbalanced at all, 100% of this traffic is delivered to the same real server.

If your client application doesn't specify a specific source IP address, the Linux kernel does select an IP address which is (in terms of routing) most close to the destination address. In this special cornercase, it is the very same IP address than the destination address.

You can verify this behaviour by running "ip route get VIP" on your realserver (replace "VIP" by the VIP address). As an example:

$ ip route get 172.17.24.42
local 172.17.24.42 dev lo src 172.17.24.42
cache <local>

This reads as following:
- 172.17.24.42 is locally configured on this host.
- the packet is being routed via the loopback device ("dev lo").
- source IP address will be 172.17.24.42


Best,

Anders
--
1&1 Internet AG Expert Systems Architect (IT Operations)
Brauerstrasse 50 v://49.721.91374.0
D-76135 Karlsruhe http://www.1und1.de/

Amtsgericht Montabaur HRB 6484
Vorstand: Frank Einhellinger, Robert Hoffmann, Markus Huhn,
Hans-Henning Kettler, Uwe Lamnek
Aufsichtsratsvorsitzender: Michael Scheeren

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] DR : real server unable to reach VIP [ In reply to ]
Have a look at xt_ipvs, it was added some time ago.

http://www.gossamer-threads.com/lists/lvs/users/22563

On Wednesday, May 20, 2015, Florent B <florent@coppint.com> wrote:

> I just want to do what is explained here :
>
> http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.lvs_clients_on_realservers.html#realserver_as_client_in_LVS-DR
>
> But it uses old iptables syntax and I can't find how to do it...
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>


--

Jonathan Petersson
Sr. Systems Administrator
Cxense Sweden AB
+46732001678
jonathan.petersson@cxense.com
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] DR : real server unable to reach VIP [ In reply to ]
Dear Florent,

>
> I'm using LVS with keepalived on Debian Wheezy.
>
> I have 3 Real Servers:
> 10.111.17.170
> 10.111.17.171
> 10.111.17.172
>
> And One VIP:
> 10.111.17.202
>
> My service is running on port 3306.
>

OK .. looks like an Mysql-NDB or MariaDB-Galea Cluster...


> VIP is not configured on RS (because I had others problems), so I use
> the iptables strategy on RS : iptables -t nat -A PREROUTING -p tcp -d
> 10.111.17.202 -j REDIRECT

you configured NAT - but expected funtions of DR - and from that mixed up.

if you want LVS-NAT , then the realservers must BEHIND the LVS LoadBalancer at an second network connected to the LB1/LB1 in LVS-NAT Mode.


See LVS docus :
- http://www.austintek.com/LVS/LVS-HOWTO/mini-HOWTO/LVS-mini-HOWTO.html#example_LVS-NAT
- http://www.austintek.com/LVS/LVS-HOWTO/mini-HOWTO/LVS-mini-HOWTO.html#example_lvs_dr

see keepalive docu
- http://www.hbyconsultancy.com/blog/two-nodes-load-balance-and-failover-with-keepalived-and-ubuntu-server-10-04-x64.html


Hope this helps.

--
Mit freundlichen Grüßen / Best Regards

Horst Venzke ; PGP NET : 1024G/082F2E6D ; http://www.remsnet.de - 1995 - 2015 - 20 Jahre Linux/Unix Support.

Legal Notice: This transmittal and/or attachments may be privileged or confidential. It is intended solely for the addressee named above. Any review, dissemination, or copying is strictly prohibited. If you received this transmittal in error, please notify us immediately by reply and immediately delete this message and all


> Gesendet: Mittwoch, 20. Mai 2015 um 11:25 Uhr
> Von: "Florent B" <florent@coppint.com>
> An: lvs-users@linuxvirtualserver.org
> Betreff: [lvs-users] DR : real server unable to reach VIP
>
> Hi everyone,
>
> I'm using LVS with keepalived on Debian Wheezy.
>
> I have 3 Real Servers:
> 10.111.17.170
> 10.111.17.171
> 10.111.17.172
>
> And One VIP:
> 10.111.17.202
>
> My service is running on port 3306.
>
> VIP is not configured on RS (because I had others problems), so I use
> the iptables strategy on RS : iptables -t nat -A PREROUTING -p tcp -d
> 10.111.17.202 -j REDIRECT
>
> LVS is configured to send all connections to VIP in priority on
> 10.111.17.170 if it's alive.
>
> My problem is that I can't connect to VIP:3306 from this real server !
>
> Packets are sent, received by LVS but it seems to ignore them when
> source mac address is the same as (computed) destination mac address.
>
> I really need to go through LVS and not REDIRECT OUTPUT packets on each RS.
>
> Has anyone an idea of what's wrong with my setup and to fix it ?
>
> Thank you a lot.
>
> Florent
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>