Mailing List Archive

[lvs-users] Connections to VIPs on the same machine (in BACKUP state)
Hi list,

I've a doubt about how connections to a VIP initiated on the same machine
works. Let me explain with an example:
I have 2 machines (lvs1 and lvs2) with keepalived (vrrp+LVS-DR). The
cluster has a virtual server (10.0.1.1) with some real servers behind.
lvs1 is the master and lvs2 is the backup.

The strange thing I'm seeing and that I don't understand (at least as a
feature) is that ig on lvs2 I try to connect to 10.0.1.1
it goes directly to the real servers without passing through lvs1. But
10.0.1.1 is not present on any lvs2 interfaces (ifconfig, ip addr) but only
in the keepalived configuration. It's not even present in the ARP cache
table.

I was thinking that maybe, since it's known to LVS, this IP is somewhere in
the ip_vs module and it's in earlier stage of the network stack, so any
connection to it is handled by the LVS stack as if lvs2 were the MASTER. If
I remove the virtual server from lvs2 keepalived, then a connection to
10.0.1.1 from lvs2 goes to the real servers through lvs1 as expected.

Is this normal? Is this the expected behavior? If so, why?

Thank you very much
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] Connections to VIPs on the same machine (in BACKUP state) [ In reply to ]
So I am curious if you have made the following changes to your real servers:
net.ipv4.conf.lo.arp_ignore=1
net.ipv4.conf.lo.arp_announce=2
net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2

What might be happening is lvs2 could be seeing the arp reply from the
servers for the VIP IP. The above settings will prevent internal/inside
traffic from bypassing your LVS cluster etc.





> Hi list,
>
> I've a doubt about how connections to a VIP initiated on the same machine
> works. Let me explain with an example:
> I have 2 machines (lvs1 and lvs2) with keepalived (vrrp+LVS-DR). The
> cluster has a virtual server (10.0.1.1) with some real servers behind.
> lvs1 is the master and lvs2 is the backup.
>
> The strange thing I'm seeing and that I don't understand (at least as a
> feature) is that ig on lvs2 I try to connect to 10.0.1.1
> it goes directly to the real servers without passing through lvs1. But
> 10.0.1.1 is not present on any lvs2 interfaces (ifconfig, ip addr) but
> only
> in the keepalived configuration. It's not even present in the ARP cache
> table.
>
> I was thinking that maybe, since it's known to LVS, this IP is somewhere
> in
> the ip_vs module and it's in earlier stage of the network stack, so any
> connection to it is handled by the LVS stack as if lvs2 were the MASTER.
> If
> I remove the virtual server from lvs2 keepalived, then a connection to
> 10.0.1.1 from lvs2 goes to the real servers through lvs1 as expected.
>
> Is this normal? Is this the expected behavior? If so, why?
>
> Thank you very much
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>


--



_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] Connections to VIPs on the same machine (in BACKUP state) [ In reply to ]
Absolutely, it's the standard real server configuration with sysctl arp
tuning. Aa I said lvs2 doesn't know any mac of the real IP (arp -a empty)
On 10 Jul 2015 12:58 am, "Jamie Dahl" <jamied@meatball.net> wrote:

>
> So I am curious if you have made the following changes to your real
> servers:
> net.ipv4.conf.lo.arp_ignore=1
> net.ipv4.conf.lo.arp_announce=2
> net.ipv4.conf.all.arp_ignore=1
> net.ipv4.conf.all.arp_announce=2
>
> What might be happening is lvs2 could be seeing the arp reply from the
> servers for the VIP IP. The above settings will prevent internal/inside
> traffic from bypassing your LVS cluster etc.
>
>
>
>
>
> > Hi list,
> >
> > I've a doubt about how connections to a VIP initiated on the same machine
> > works. Let me explain with an example:
> > I have 2 machines (lvs1 and lvs2) with keepalived (vrrp+LVS-DR). The
> > cluster has a virtual server (10.0.1.1) with some real servers behind.
> > lvs1 is the master and lvs2 is the backup.
> >
> > The strange thing I'm seeing and that I don't understand (at least as a
> > feature) is that ig on lvs2 I try to connect to 10.0.1.1
> > it goes directly to the real servers without passing through lvs1. But
> > 10.0.1.1 is not present on any lvs2 interfaces (ifconfig, ip addr) but
> > only
> > in the keepalived configuration. It's not even present in the ARP cache
> > table.
> >
> > I was thinking that maybe, since it's known to LVS, this IP is somewhere
> > in
> > the ip_vs module and it's in earlier stage of the network stack, so any
> > connection to it is handled by the LVS stack as if lvs2 were the MASTER.
> > If
> > I remove the virtual server from lvs2 keepalived, then a connection to
> > 10.0.1.1 from lvs2 goes to the real servers through lvs1 as expected.
> >
> > Is this normal? Is this the expected behavior? If so, why?
> >
> > Thank you very much
> > _______________________________________________
> > Please read the documentation before posting - it's available at:
> > http://www.linuxvirtualserver.org/
> >
> > LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> > Send requests to lvs-users-request@LinuxVirtualServer.org
> > or go to http://lists.graemef.net/mailman/listinfo/lvs-users
> >
>
>
> --
>
>
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users
Re: [lvs-users] Connections to VIPs on the same machine (in BACKUP state) [ In reply to ]
Hello,

On Thu, 9 Jul 2015, Davide Ferrari wrote:

> Hi list,
>
> I've a doubt about how connections to a VIP initiated on the same machine
> works. Let me explain with an example:
> I have 2 machines (lvs1 and lvs2) with keepalived (vrrp+LVS-DR). The
> cluster has a virtual server (10.0.1.1) with some real servers behind.
> lvs1 is the master and lvs2 is the backup.
>
> The strange thing I'm seeing and that I don't understand (at least as a
> feature) is that ig on lvs2 I try to connect to 10.0.1.1
> it goes directly to the real servers without passing through lvs1. But
> 10.0.1.1 is not present on any lvs2 interfaces (ifconfig, ip addr) but only
> in the keepalived configuration. It's not even present in the ARP cache
> table.
>
> I was thinking that maybe, since it's known to LVS, this IP is somewhere in
> the ip_vs module and it's in earlier stage of the network stack, so any
> connection to it is handled by the LVS stack as if lvs2 were the MASTER. If
> I remove the virtual server from lvs2 keepalived, then a connection to
> 10.0.1.1 from lvs2 goes to the real servers through lvs1 as expected.
>
> Is this normal? Is this the expected behavior? If so, why?

I think, what happens is that connection is originated
from some unique IP and IPVS at LOCAL_OUT forwards it to the
remote real server. IPVS does not need VIP to be configured on
directors for local clients to be balanced to remote real servers.
But for traffic from remote clients VIP should be configured
as IP because IPVS works at LOCAL_IN.

As for any ARP sysctl settings, they are used only
for remote clients when director and DR real servers are on same
LAN.

If you want to avoid the IPVS configuration to work
in backup mode and to leave packets to master server for balancing,
you can set the backup_only flag. Commit log talks about
remote clients but actually it should work for local clients
too:

>From commit 0c12582fbcdea0cbb0dfd224e1c5f9a8428ffa18:

Even when the backup function is enabled we continue to forward
traffic and schedule new connections when the current master is using
the backup server as real server. While this is not a problem for NAT,
for DR and TUN method the backup server can not determine if a request
comes from client or from director.

To avoid such loops add new sysctl flag backup_only. It can be needed
for DR/TUN setups that do not need backup and director function at the
same time. When the backup function is enabled we stop any forwarding
and pass the traffic to the local stack (real server mode). The flag
disables the director function when the backup function is enabled.

Regards

--
Julian Anastasov <ja@ssi.bg>

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users