Mailing List Archive: [lvs-users] Configuring LVS-Tun on Ubuntu

[lvs-users] Configuring LVS-Tun on Ubuntu

Feb 4, 2013, 6:01 PM

Post #1 of 3 (2407 views)

Hello Everyone,

I have so far been able to configure LVS-DR and LVS-NAT and trying to
complete the trilogy. Here is the setup I am using:
2 realservers, 1 director and 1 client

*Director interfaces and other info*:
$ ifconfig
eth0 Link encap:Ethernet HWaddr 00:0c:29:07:96:cf
inet addr:192.168.25.135 Bcast:192.168.25.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe07:96cf/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3571 errors:0 dropped:0 overruns:0 frame:0
TX packets:2861 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1776401 (1.7 MB) TX bytes:292263 (292.2 KB)
Interrupt:19 Base address:0x2000

eth0:110 Link encap:Ethernet HWaddr 00:0c:29:07:96:cf
inet addr:192.168.25.110 Bcast:192.168.25.110
Mask:255.255.255.255
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:19 Base address:0x2000

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

$ sudo /sbin/ipvsadm -l -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.25.110:8080 rr
-> 192.168.25.131:8080 Tunnel 1 0 1
-> 192.168.25.140:8080 Tunnel 1 0 0

$ route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
0.0.0.0 192.168.25.2 0.0.0.0 UG 0 0 0 eth0
192.168.25.0 0.0.0.0 255.255.255.0 U 1 0 0 eth0
192.168.25.110 0.0.0.0 255.255.255.255 UH 0 0 0 eth0

*One of the servers*

$ ifconfig
eth0 Link encap:Ethernet HWaddr 00:0c:29:ae:54:3c
inet addr:192.168.25.140 Bcast:192.168.25.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:feae:543c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2892 errors:0 dropped:0 overruns:0 frame:0
TX packets:1892 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1628504 (1.6 MB) TX bytes:184065 (184.0 KB)
Interrupt:19 Base address:0x2024

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

tunl0 Link encap:IPIP Tunnel HWaddr
inet addr:192.168.25.110 Mask:255.255.255.255
UP RUNNING NOARP MTU:1480 Metric:1
RX packets:36 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2160 (2.1 KB) TX bytes:0 (0.0 B)

$ route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
0.0.0.0 192.168.25.2 0.0.0.0 UG 0 0 0 eth0
192.168.25.0 0.0.0.0 255.255.255.0 U 1 0 0 eth0
192.168.25.110 0.0.0.0 255.255.255.255 UH 0 0 0
tunl0

I use arptables to solve the ARP problem. I have confirmed using wireshark
traces that only the director sends ARP replies for the VIP. I start
servers on the client using the following command:
$while true ; do nc -l 8080 < index.html ; done

>From the client I execute:
$ lynx -dump http://192.168.25.140:8080/
This is fake WWW server 1

$ lynx -dump http://192.168.25.131:8080/
This is fake WWW server 2

$ lynx -dump http://192.168.25.110:8080/

Looking up 192.168.25.110:8080
Making HTTP connection to 192.168.25.110:8080
Alert!: Unable to connect to remote host.

lynx: Can't access startfile http://192.168.25.110:8080/

So, I ran a wireshark trace on tunl0 on one of the realservers. I see a
bunch of TCP SYNs with scr IP 192.168.25.128 and dst IP 192.168.25.110 but
the server never sends back a SYN ACK. I have been trying to search online
to find a solution for this but no luck so far. Can someone please tell me
what I might be missing?

Thanks,

Dushyant
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

Re: [lvs-users] Configuring LVS-Tun on Ubuntu [ In reply to ]

khosrow.ebrahimpour at ssc-spc

Feb 5, 2013, 6:30 AM

Post #2 of 3 (2318 views)

Permalink

Hi Dushyant,

On February 4, 2013 09:01:32 PM Dushyant Arora wrote:
> Hello Everyone,
>
> I have so far been able to configure LVS-DR and LVS-NAT and trying to
> complete the trilogy. Here is the setup I am using:
> 2 realservers, 1 director and 1 client
>
> *Director interfaces and other info*:
> $ ifconfig
> eth0 Link encap:Ethernet HWaddr 00:0c:29:07:96:cf
> inet addr:192.168.25.135 Bcast:192.168.25.255 Mask:255.255.255.0
> inet6 addr: fe80::20c:29ff:fe07:96cf/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:3571 errors:0 dropped:0 overruns:0 frame:0
> TX packets:2861 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:1776401 (1.7 MB) TX bytes:292263 (292.2 KB)
> Interrupt:19 Base address:0x2000
>
> eth0:110 Link encap:Ethernet HWaddr 00:0c:29:07:96:cf
> inet addr:192.168.25.110 Bcast:192.168.25.110
> Mask:255.255.255.255
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> Interrupt:19 Base address:0x2000
>
> lo Link encap:Local Loopback
> inet addr:127.0.0.1 Mask:255.0.0.0
> inet6 addr: ::1/128 Scope:Host
> UP LOOPBACK RUNNING MTU:16436 Metric:1
> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
>
>
> $ sudo /sbin/ipvsadm -l -n
> IP Virtual Server version 1.2.1 (size=4096)
> Prot LocalAddress:Port Scheduler Flags
> -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> TCP 192.168.25.110:8080 rr
> -> 192.168.25.131:8080 Tunnel 1 0 1
> -> 192.168.25.140:8080 Tunnel 1 0 0
>
> $ route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use
> Iface
> 0.0.0.0 192.168.25.2 0.0.0.0 UG 0 0 0 eth0
> 192.168.25.0 0.0.0.0 255.255.255.0 U 1 0 0
> eth0 192.168.25.110 0.0.0.0 255.255.255.255 UH 0 0
> 0 eth0
>
>
> *One of the servers*
>
> $ ifconfig
> eth0 Link encap:Ethernet HWaddr 00:0c:29:ae:54:3c
> inet addr:192.168.25.140 Bcast:192.168.25.255 Mask:255.255.255.0
> inet6 addr: fe80::20c:29ff:feae:543c/64 Scope:Link
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:2892 errors:0 dropped:0 overruns:0 frame:0
> TX packets:1892 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:1628504 (1.6 MB) TX bytes:184065 (184.0 KB)
> Interrupt:19 Base address:0x2024
>
> lo Link encap:Local Loopback
> inet addr:127.0.0.1 Mask:255.0.0.0
> inet6 addr: ::1/128 Scope:Host
> UP LOOPBACK RUNNING MTU:16436 Metric:1
> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
>
> tunl0 Link encap:IPIP Tunnel HWaddr
> inet addr:192.168.25.110 Mask:255.255.255.255
> UP RUNNING NOARP MTU:1480 Metric:1
> RX packets:36 errors:0 dropped:0 overruns:0 frame:0
> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:2160 (2.1 KB) TX bytes:0 (0.0 B)
>
>
> $ route -n
> Kernel IP routing table
> Destination Gateway Genmask Flags Metric Ref Use
> Iface
> 0.0.0.0 192.168.25.2 0.0.0.0 UG 0 0 0 eth0
> 192.168.25.0 0.0.0.0 255.255.255.0 U 1 0 0
> eth0 192.168.25.110 0.0.0.0 255.255.255.255 UH 0 0
> 0 tunl0
>
> I use arptables to solve the ARP problem. I have confirmed using wireshark
> traces that only the director sends ARP replies for the VIP. I start
> servers on the client using the following command:
> $while true ; do nc -l 8080 < index.html ; done
>
> >From the client I execute:
> $ lynx -dump http://192.168.25.140:8080/
> This is fake WWW server 1
>
> $ lynx -dump http://192.168.25.131:8080/
> This is fake WWW server 2
>
> $ lynx -dump http://192.168.25.110:8080/
>
> Looking up 192.168.25.110:8080
> Making HTTP connection to 192.168.25.110:8080
> Alert!: Unable to connect to remote host.
>
> lynx: Can't access startfile http://192.168.25.110:8080/
>
> So, I ran a wireshark trace on tunl0 on one of the realservers. I see a
> bunch of TCP SYNs with scr IP 192.168.25.128 and dst IP 192.168.25.110 but
> the server never sends back a SYN ACK. I have been trying to search online
> to find a solution for this but no luck so far. Can someone please tell me
> what I might be missing?
>
> Thanks,
>
> Dushyant

Looks like you've got everything setup correctly.

By default Ubuntu enables rp_filter. In my own setup on Ubuntu I've had to
disable the rp_filter kernel parameter on the real servers, and that's fixed the
issue.

_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users

Re: [lvs-users] Configuring LVS-Tun on Ubuntu [ In reply to ]

dushyantarora13 at gmail

Feb 5, 2013, 9:46 AM

Post #3 of 3 (2312 views)

Permalink

That was it. Thanks Khosrow!

On Tue, Feb 5, 2013 at 9:30 AM, Khosrow Ebrahimpour <
khosrow.ebrahimpour@ssc-spc.gc.ca> wrote:

> Hi Dushyant,
>
> On February 4, 2013 09:01:32 PM Dushyant Arora wrote:
> > Hello Everyone,
> >
> > I have so far been able to configure LVS-DR and LVS-NAT and trying to
> > complete the trilogy. Here is the setup I am using:
> > 2 realservers, 1 director and 1 client
> >
> > *Director interfaces and other info*:
> > $ ifconfig
> > eth0 Link encap:Ethernet HWaddr 00:0c:29:07:96:cf
> > inet addr:192.168.25.135 Bcast:192.168.25.255
> Mask:255.255.255.0
> > inet6 addr: fe80::20c:29ff:fe07:96cf/64 Scope:Link
> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> > RX packets:3571 errors:0 dropped:0 overruns:0 frame:0
> > TX packets:2861 errors:0 dropped:0 overruns:0 carrier:0
> > collisions:0 txqueuelen:1000
> > RX bytes:1776401 (1.7 MB) TX bytes:292263 (292.2 KB)
> > Interrupt:19 Base address:0x2000
> >
> > eth0:110 Link encap:Ethernet HWaddr 00:0c:29:07:96:cf
> > inet addr:192.168.25.110 Bcast:192.168.25.110
> > Mask:255.255.255.255
> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> > Interrupt:19 Base address:0x2000
> >
> > lo Link encap:Local Loopback
> > inet addr:127.0.0.1 Mask:255.0.0.0
> > inet6 addr: ::1/128 Scope:Host
> > UP LOOPBACK RUNNING MTU:16436 Metric:1
> > RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> > collisions:0 txqueuelen:0
> > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
> >
> >
> > $ sudo /sbin/ipvsadm -l -n
> > IP Virtual Server version 1.2.1 (size=4096)
> > Prot LocalAddress:Port Scheduler Flags
> > -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> > TCP 192.168.25.110:8080 rr
> > -> 192.168.25.131:8080 Tunnel 1 0 1
> > -> 192.168.25.140:8080 Tunnel 1 0 0
> >
> > $ route -n
> > Kernel IP routing table
> > Destination Gateway Genmask Flags Metric Ref Use
> > Iface
> > 0.0.0.0 192.168.25.2 0.0.0.0 UG 0 0 0
> eth0
> > 192.168.25.0 0.0.0.0 255.255.255.0 U 1 0 0
> > eth0 192.168.25.110 0.0.0.0 255.255.255.255 UH 0 0
> > 0 eth0
> >
> >
> > *One of the servers*
> >
> > $ ifconfig
> > eth0 Link encap:Ethernet HWaddr 00:0c:29:ae:54:3c
> > inet addr:192.168.25.140 Bcast:192.168.25.255
> Mask:255.255.255.0
> > inet6 addr: fe80::20c:29ff:feae:543c/64 Scope:Link
> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> > RX packets:2892 errors:0 dropped:0 overruns:0 frame:0
> > TX packets:1892 errors:0 dropped:0 overruns:0 carrier:0
> > collisions:0 txqueuelen:1000
> > RX bytes:1628504 (1.6 MB) TX bytes:184065 (184.0 KB)
> > Interrupt:19 Base address:0x2024
> >
> > lo Link encap:Local Loopback
> > inet addr:127.0.0.1 Mask:255.0.0.0
> > inet6 addr: ::1/128 Scope:Host
> > UP LOOPBACK RUNNING MTU:16436 Metric:1
> > RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> > collisions:0 txqueuelen:0
> > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
> >
> > tunl0 Link encap:IPIP Tunnel HWaddr
> > inet addr:192.168.25.110 Mask:255.255.255.255
> > UP RUNNING NOARP MTU:1480 Metric:1
> > RX packets:36 errors:0 dropped:0 overruns:0 frame:0
> > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> > collisions:0 txqueuelen:0
> > RX bytes:2160 (2.1 KB) TX bytes:0 (0.0 B)
> >
> >
> > $ route -n
> > Kernel IP routing table
> > Destination Gateway Genmask Flags Metric Ref Use
> > Iface
> > 0.0.0.0 192.168.25.2 0.0.0.0 UG 0 0 0
> eth0
> > 192.168.25.0 0.0.0.0 255.255.255.0 U 1 0 0
> > eth0 192.168.25.110 0.0.0.0 255.255.255.255 UH 0 0
> > 0 tunl0
> >
> > I use arptables to solve the ARP problem. I have confirmed using
> wireshark
> > traces that only the director sends ARP replies for the VIP. I start
> > servers on the client using the following command:
> > $while true ; do nc -l 8080 < index.html ; done
> >
> > >From the client I execute:
> > $ lynx -dump http://192.168.25.140:8080/
> > This is fake WWW server 1
> >
> > $ lynx -dump http://192.168.25.131:8080/
> > This is fake WWW server 2
> >
> > $ lynx -dump http://192.168.25.110:8080/
> >
> > Looking up 192.168.25.110:8080
> > Making HTTP connection to 192.168.25.110:8080
> > Alert!: Unable to connect to remote host.
> >
> > lynx: Can't access startfile http://192.168.25.110:8080/
> >
> > So, I ran a wireshark trace on tunl0 on one of the realservers. I see a
> > bunch of TCP SYNs with scr IP 192.168.25.128 and dst IP 192.168.25.110
> but
> > the server never sends back a SYN ACK. I have been trying to search
> online
> > to find a solution for this but no luck so far. Can someone please tell
> me
> > what I might be missing?
> >
> > Thanks,
> >
> > Dushyant
>
> Looks like you've got everything setup correctly.
>
> By default Ubuntu enables rp_filter. In my own setup on Ubuntu I've had to
> disable the rp_filter kernel parameter on the real servers, and that's
> fixed the
> issue.
>
>
>
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users
>
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users