Mailing List Archive

Re: LVS only allows 2 connections...?
On 2000-11-20T21:44:41,
iceam@talk21.com said:

> The route output for the real servers is as follows:
> For real 1:
> real1.server.ne * 255.255.255.255 UH 0 0 0 eth0
> 10.1.1.0 * 255.255.255.0 U 0 0 0 eth0
> 127.0.0.0 * 255.0.0.0 U 0 0 0 eth0
> default director.ser 0.0.0.0 UG 0 0 0 eth0

Okay, the director is your default gateway, and you are using masquerading,
and you have everything on the same ethernet segment if I didn't miss
anything.

My guess would be that the load balancer sends ICMP redirects to the clients,
saying "Hey, you can reach that machine directly at this MAC address" - this
shouldn't happen ;), but is a bug in the ipchains masquerading code.

What do your arp -a tables look like?

You can disable this with
echo 0 >/proc/sys/net/ipv4/conf/all/send_redirects
echo 0 >/proc/sys/net/ipv4/conf/eth0/send_redirects

and see whether that helps.

Sincerely,
Lars Marowsky-Brée <lmb@suse.de>
Development HA

--
Perfection is our goal, excellence will be tolerated. -- J. Yahl
Re: LVS only allows 2 connections...? [ In reply to ]
Joe,

The output from the arp -a command is as follows:

? (192.168.1.254 at 00:08:C7:50:96:19 [ether] on eth0
? (10.0.0.1) at 00:60:97:1D:03:12 [ether] on eth0
real2.server.net (10.1.1.3) at 00:08:C7:50:96:19 [ether] on eth0
real1.server.net (10.1.1.2) at 00:08:C7:A8:E0:69 [ether] on eth0


I have changed the default gateway for the director to be the client (I had already tried this, but was unsure if it was right) and run the following two commands also, as suggested by someone else:

echo 0 > /proc/sys/net/ipv4/conf/all/send_redirects
echo 0 > /proc/sys/net/ipv4/conf/eth0/send_redirects

Now I can get multiple reloads for one server but the other one does not seem to want to reply!

I dunno!

Andy
> On Mon, 20 Nov 2000 iceam@talk21.com wrote:
>
> Hi all, thanks for all your help in pointing me in the
> direction in the past few days/week.
>
> I now have a new problem with LVS-NAT. I still have the same
> set-up as before:
>
> 1 client, 1 director and 2 real servers:
> Client IP = 192.168.1.254
> Director OUtside IP = 192.168.1.110
> Director Inside IP = 10.1.1.1
> Real Server 1 IP = 10.1.1.2
> Real Server 2 IP = 10.1.1.3
>
> And I have upgraded the director kernel to 2.2.16, patched
> it and compiled the latest ipvsadm. Running Joe's scripts
> works, o.k. on the director, but as I don't have a default
> gateway for the director, the two real server scripts don't
> run properly.
>
>ah - the script does a check when configuring the realservers
>to check that the realservers can''t connect directly to the
>director's default gw. You don't need the default gw for the VS-NAT
>to work, but since a production VS-NAT LVS will need a default gw,
>the script checks that it is a reasonable one. In your case set the
>default gw for the director to be the client.
>
>I couldn't imagine why you needed a default route for the director
>sorry.
>
> Do the real servers also have to be kernel 2.2.16?
>
>anything, they don't even need to be running Linux.
>
>
> I was under the impression that they don't, but at the
> moment if I try to view pages using my client, the lvs will
> only provide two connections to each real server and then
> stop responding
>
>so active and inactive connections to both machines increase
>and then decrease after you stop making requests (about 2mins for the
>inactive connections to drop)?
>
>Joe
>
> (If I reload the pages I get nothing, the
> browser says the www connection cannot be made, but I can
> still ping the machines from the director, and still ping
> the director from the client.)
>
>
> The output of IPVSADM on the director is as follows:
>
> IP Virtual Server 0.9.15 (size=4096)
>
> TCP extdirector.icepicc.co.uk:www wlc
> -> real2.server.net:www Masq 1 0 0
> -> real1.server.net:www Masq 1 0 0
> TCP extdirector.icepicc.co.uk:telnet rr
> -> rea1.server.net:telnet Masq 1 0 0
>
> The directors 'route' output is as follows:
>
> CORSICA.icepicc * 255.255.255.255 UH 0 0 0 eth0
> extdirector.ice * 255.255.255.255 UH 0 0 0 eth0
> director.server * 255.255.255.255 UH 0 0 0 eth0
> 10.0.0.0 * 255.255.255.0 U 0 0 0 eth0
> 192.168.1.0 * 255.255.255.0 U 0 0 0 eth0
> 10.1.1.0 * 255.255.255.0 U 0 0 0 eth0
> 127.0.0.0 * 255.0.0.0 U 0 0 0 lo
>
> The IP chains output for the director is as follows:
> Chain input (Policy ACCEPT)
> Chain forward (policy ACCEPT)
> MASQ tcp ----- real1.server.net anywhere www -> any
> MASQ tcp ----- real2.server.net anywhere www -> any
> MASQ tcp ----- real1.server.net anywhere telnet -> any
> Chain output (policy ACCEPT)
>
> There is no ipchains output for the real servers.
>
> The route output for the real servers is as follows:
> For real 1:
> real1.server.ne * 255.255.255.255 UH 0 0 0 eth0
> 10.1.1.0 * 255.255.255.0 U 0 0 0 eth0
> 127.0.0.0 * 255.0.0.0 U 0 0 0 eth0
> default director.ser 0.0.0.0 UG 0 0 0 eth0
>
> For real 2:
> real2.server.ne * 255.255.255.255 UH 0 0 0 eth0
> 10.1.1.0 * 255.255.255.0 U 0 0 0 eth0
> 127.0.0.0 * 255.0.0.0 U 0 0 0 eth0
> default director.ser 0.0.0.0 UG 0 0 0 eth0
>
> Have I missed something?
>
>
>
> --------------------
> talk21 your FREE portable and private address on the net at <a Target='_new' Href='http://www.talk21.com/redirect.html?http://www.talk21.com'>http://www.talk21.com</a>
>
>
> ----------------------------------------------------------------------
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> To unsubscribe, e-mail: lvs-users-unsubscribe@LinuxVirtualServer.org
> For additional commands, e-mail: lvs-users-help@LinuxVirtualServer.org
>
>
>
>--
>Joseph Mack mack@ncifcrf.gov



--------------------
talk21 your FREE portable and private address on the net at http://www.talk21.com
Re: LVS only allows 2 connections...? [ In reply to ]
Joe,

The output from the arp -a command is as follows:

? (192.168.1.254 at 00:08:C7:50:96:19 [ether] on eth0
? (10.0.0.1) at 00:60:97:1D:03:12 [ether] on eth0
real2.server.net (10.1.1.3) at 00:08:C7:50:96:19 [ether] on eth0
real1.server.net (10.1.1.2) at 00:08:C7:A8:E0:69 [ether] on eth0


I have changed the default gateway for the director to be the client (I had already tried this, but was unsure if it was right) and run the following two commands also, as suggested by someone else:

echo 0 > /proc/sys/net/ipv4/conf/all/send_redirects
echo 0 > /proc/sys/net/ipv4/conf/eth0/send_redirects

Now I can get multiple reloads for one server but the other one does not seem to want to reply!

I dunno!

Andy
> On Mon, 20 Nov 2000 iceam@talk21.com wrote:
>
> Hi all, thanks for all your help in pointing me in the
> direction in the past few days/week.
>
> I now have a new problem with LVS-NAT. I still have the same
> set-up as before:
>
> 1 client, 1 director and 2 real servers:
> Client IP = 192.168.1.254
> Director OUtside IP = 192.168.1.110
> Director Inside IP = 10.1.1.1
> Real Server 1 IP = 10.1.1.2
> Real Server 2 IP = 10.1.1.3
>
> And I have upgraded the director kernel to 2.2.16, patched
> it and compiled the latest ipvsadm. Running Joe's scripts
> works, o.k. on the director, but as I don't have a default
> gateway for the director, the two real server scripts don't
> run properly.
>
>ah - the script does a check when configuring the realservers
>to check that the realservers can''t connect directly to the
>director's default gw. You don't need the default gw for the VS-NAT
>to work, but since a production VS-NAT LVS will need a default gw,
>the script checks that it is a reasonable one. In your case set the
>default gw for the director to be the client.
>
>I couldn't imagine why you needed a default route for the director
>sorry.
>
> Do the real servers also have to be kernel 2.2.16?
>
>anything, they don't even need to be running Linux.
>
>
> I was under the impression that they don't, but at the
> moment if I try to view pages using my client, the lvs will
> only provide two connections to each real server and then
> stop responding
>
>so active and inactive connections to both machines increase
>and then decrease after you stop making requests (about 2mins for the
>inactive connections to drop)?
>
>Joe
>
> (If I reload the pages I get nothing, the
> browser says the www connection cannot be made, but I can
> still ping the machines from the director, and still ping
> the director from the client.)
>
>
> The output of IPVSADM on the director is as follows:
>
> IP Virtual Server 0.9.15 (size=4096)
>
> TCP extdirector.icepicc.co.uk:www wlc
> -> real2.server.net:www Masq 1 0 0
> -> real1.server.net:www Masq 1 0 0
> TCP extdirector.icepicc.co.uk:telnet rr
> -> rea1.server.net:telnet Masq 1 0 0
>
> The directors 'route' output is as follows:
>
> CORSICA.icepicc * 255.255.255.255 UH 0 0 0 eth0
> extdirector.ice * 255.255.255.255 UH 0 0 0 eth0
> director.server * 255.255.255.255 UH 0 0 0 eth0
> 10.0.0.0 * 255.255.255.0 U 0 0 0 eth0
> 192.168.1.0 * 255.255.255.0 U 0 0 0 eth0
> 10.1.1.0 * 255.255.255.0 U 0 0 0 eth0
> 127.0.0.0 * 255.0.0.0 U 0 0 0 lo
>
> The IP chains output for the director is as follows:
> Chain input (Policy ACCEPT)
> Chain forward (policy ACCEPT)
> MASQ tcp ----- real1.server.net anywhere www -> any
> MASQ tcp ----- real2.server.net anywhere www -> any
> MASQ tcp ----- real1.server.net anywhere telnet -> any
> Chain output (policy ACCEPT)
>
> There is no ipchains output for the real servers.
>
> The route output for the real servers is as follows:
> For real 1:
> real1.server.ne * 255.255.255.255 UH 0 0 0 eth0
> 10.1.1.0 * 255.255.255.0 U 0 0 0 eth0
> 127.0.0.0 * 255.0.0.0 U 0 0 0 eth0
> default director.ser 0.0.0.0 UG 0 0 0 eth0
>
> For real 2:
> real2.server.ne * 255.255.255.255 UH 0 0 0 eth0
> 10.1.1.0 * 255.255.255.0 U 0 0 0 eth0
> 127.0.0.0 * 255.0.0.0 U 0 0 0 eth0
> default director.ser 0.0.0.0 UG 0 0 0 eth0
>
> Have I missed something?
>
>
>
> --------------------
> talk21 your FREE portable and private address on the net at <a Target='_new' Href='http://www.talk21.com/redirect.html?http://www.talk21.com'>http://www.talk21.com</a>
>
>
> ----------------------------------------------------------------------
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> To unsubscribe, e-mail: lvs-users-unsubscribe@LinuxVirtualServer.org
> For additional commands, e-mail: lvs-users-help@LinuxVirtualServer.org
>
>
>
>--
>Joseph Mack mack@ncifcrf.gov



--------------------
talk21 your FREE portable and private address on the net at http://www.talk21.com
Re: LVS only allows 2 connections...? [ In reply to ]
D'oh!

The server is serving up the page so quickly that it has
the same number of connections as the other one (0) faster
than I can reload the page. I have sorted it now. Many
thanks to everyone who has helped me in the past few days,
I know I have been pestering. You've been cool, especially Joe. Ta

Andy
> Joe,
>
>The output from the arp -a command is as follows:
>
>? (192.168.1.254 at 00:08:C7:50:96:19 [ether] on eth0
>? (10.0.0.1) at 00:60:97:1D:03:12 [ether] on eth0
>real2.server.net (10.1.1.3) at 00:08:C7:50:96:19 [ether] on eth0
>real1.server.net (10.1.1.2) at 00:08:C7:A8:E0:69 [ether] on eth0
>
>
>I have changed the default gateway for the director to be the client (I had already tried this, but was unsure if it was right) and run the following two commands also, as suggested by someone else:
>
>echo 0 > /proc/sys/net/ipv4/conf/all/send_redirects
>echo 0 > /proc/sys/net/ipv4/conf/eth0/send_redirects
>
>Now I can get multiple reloads for one server but the other one does not seem to want to reply!
>
>I dunno!
>
>Andy
> On Mon, 20 Nov 2000 iceam@talk21.com wrote:
>
> Hi all, thanks for all your help in pointing me in the
> direction in the past few days/week.
>
> I now have a new problem with LVS-NAT. I still have the same
> set-up as before:
>
> 1 client, 1 director and 2 real servers:
> Client IP = 192.168.1.254
> Director OUtside IP = 192.168.1.110
> Director Inside IP = 10.1.1.1
> Real Server 1 IP = 10.1.1.2
> Real Server 2 IP = 10.1.1.3
>
> And I have upgraded the director kernel to 2.2.16, patched
> it and compiled the latest ipvsadm. Running Joe's scripts
> works, o.k. on the director, but as I don't have a default
> gateway for the director, the two real server scripts don't
> run properly.
>
>ah - the script does a check when configuring the realservers
>to check that the realservers can''t connect directly to the
>director's default gw. You don't need the default gw for the VS-NAT
>to work, but since a production VS-NAT LVS will need a default gw,
>the script checks that it is a reasonable one. In your case set the
>default gw for the director to be the client.
>
>I couldn't imagine why you needed a default route for the director
>sorry.
>
> Do the real servers also have to be kernel 2.2.16?
>
>anything, they don't even need to be running Linux.
>
>
> I was under the impression that they don't, but at the
> moment if I try to view pages using my client, the lvs will
> only provide two connections to each real server and then
> stop responding
>
>so active and inactive connections to both machines increase
>and then decrease after you stop making requests (about 2mins for the
>inactive connections to drop)?
>
>Joe
>
> (If I reload the pages I get nothing, the
> browser says the www connection cannot be made, but I can
> still ping the machines from the director, and still ping
> the director from the client.)
>
>
> The output of IPVSADM on the director is as follows:
>
> IP Virtual Server 0.9.15 (size=4096)
>
> TCP extdirector.icepicc.co.uk:www wlc
> -> real2.server.net:www Masq 1 0 0
> -> real1.server.net:www Masq 1 0 0
> TCP extdirector.icepicc.co.uk:telnet rr
> -> rea1.server.net:telnet Masq 1 0 0
>
> The directors 'route' output is as follows:
>
> CORSICA.icepicc * 255.255.255.255 UH 0 0 0 eth0
> extdirector.ice * 255.255.255.255 UH 0 0 0 eth0
> director.server * 255.255.255.255 UH 0 0 0 eth0
> 10.0.0.0 * 255.255.255.0 U 0 0 0 eth0
> 192.168.1.0 * 255.255.255.0 U 0 0 0 eth0
> 10.1.1.0 * 255.255.255.0 U 0 0 0 eth0
> 127.0.0.0 * 255.0.0.0 U 0 0 0 lo
>
> The IP chains output for the director is as follows:
> Chain input (Policy ACCEPT)
> Chain forward (policy ACCEPT)
> MASQ tcp ----- real1.server.net anywhere www -> any
> MASQ tcp ----- real2.server.net anywhere www -> any
> MASQ tcp ----- real1.server.net anywhere telnet -> any
> Chain output (policy ACCEPT)
>
> There is no ipchains output for the real servers.
>
> The route output for the real servers is as follows:
> For real 1:
> real1.server.ne * 255.255.255.255 UH 0 0 0 eth0
> 10.1.1.0 * 255.255.255.0 U 0 0 0 eth0
> 127.0.0.0 * 255.0.0.0 U 0 0 0 eth0
> default director.ser 0.0.0.0 UG 0 0 0 eth0
>
> For real 2:
> real2.server.ne * 255.255.255.255 UH 0 0 0 eth0
> 10.1.1.0 * 255.255.255.0 U 0 0 0 eth0
> 127.0.0.0 * 255.0.0.0 U 0 0 0 eth0
> default director.ser 0.0.0.0 UG 0 0 0 eth0
>
> Have I missed something?
>
>
>
> --------------------
> talk21 your FREE portable and private address on the net at <a Target='_new' Href='<a Target='_new' Href='http://www.talk21.com/redirect.html?http://www.talk21.com/redirect.html?http://www.talk21.com''>http://www.talk21.com/redirect.html?http://www.talk21.com'</a>><a Target='_new' Href='http://www.talk21.com/redirect.html?http://www.talk21.com'>http://www.talk21.com</a></a>
>
>
> ----------------------------------------------------------------------
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> To unsubscribe, e-mail: lvs-users-unsubscribe@LinuxVirtualServer.org
> For additional commands, e-mail: lvs-users-help@LinuxVirtualServer.org
>
>
>
>--
>Joseph Mack mack@ncifcrf.gov
>
>
>
>--------------------
>talk21 your FREE portable and private address on the net at <a Target='_new' Href='http://www.talk21.com/redirect.html?http://www.talk21.com'>http://www.talk21.com</a>
>
>
>_______________________________________________
>LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
>Send requests to lvs-users-request@LinuxVirtualServer.org
>or go to <a Target='_new' Href='http://www.talk21.com/redirect.html?http://www.in-addr.de/mailman/listinfo/lvs-users'>http://www.in-addr.de/mailman/listinfo/lvs-users</a>



--------------------
talk21 your FREE portable and private address on the net at http://www.talk21.com