Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. Linux Virtual Server>
  3. Users
1 2  View All
Setting up a one network VS-NAT LVS
mack at ncifcrf
Jan 18, 2001, 1:41 PM
Post #1 of 37 (4591 views)
Permalink
From the next version of the HOWTO. Comments gladly accepted.

Joe

7.1.2 One network VS-NAT

The disadvantage of the 2 network VS-NAT is that the real-servers
are not able to connect to machines in the network of the VIP.
You couldn't make a VS-NAT setup out of machines already on
your LAN, which were also required for other purposes to stay
on the LAN network.

Here's a 1 network VS-NAT LVS.
________
| |
| client |
|________|
CIP=192.168.1.254
|
(router)
|
__________ |
| | | VIP=192.168.1.110 (eth0:110)
| director |---| DIP=192.168.1.1 (eth0)
|__________| | DIIP=192.168.1.9 (eth0:9)
|
|
------------------------------------
| | |
| | |
RIP1=192.168.1.2 RIP2=192.168.1.3 RIP3=10.1.1.4 (all eth0)
_____________ _____________ _____________
| | | | | |
| real-server | | real-server | | real-server |
|_____________| |_____________| |_____________|

The problem:

A return packet from the real-server (with address RIP->CIP)
will be sent to the real-server's default gw (the director).
ICMP redirects will be sent from the director telling the
real-server of the better route directly to the client.
The real-server will then send the packet directly to the
client and it will not be demasqueraded by the director.
The client will get a reply from the RIP rather than the VIP
and the connection will hang.

The cure:

In the previous HOWTO I said that initial attempts to handle this
by turning off redirects had not worked. The problem appears now
to be solved.

Thanks to michael_e_brown@dell.com and Julian ja@ssi.bg
for sorting this out.

To get a VS-NAT LVS to work on one network -

1. On the director, turn off icmp redirects

director:/etc/lvs# echo 0 > /proc/sys/net/ipv4/conf/all/send_redirects
director:/etc/lvs# echo 0 > /proc/sys/net/ipv4/conf/default/send_redirects
director:/etc/lvs# echo 0 > /proc/sys/net/ipv4/conf/eth0/send_redirects

(Note: eth0 may be eth1 etc, on your machine).

2. Make the director the default and only route for outgoing packets.

You will probably have set the routing on the real-server up like this

realserver:/etc/lvs# netstat -r
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
0.0.0.0 director 0.0.0.0 UG 0 0 0 eth0

Note the route to 192.168.1.0/24. This allows the real-server to send packets
to the client by just putting them out on eth0, where the client will
pick them up directly (without being demasqueraded) and the LVS will
not work.

Remove the route to 192.168.1.0/24.

realserver:/etc/lvs#route del -net 192.168.1.0 netmask 255.255.255.0 dev eth0

This will leave you with

realserver:/etc/lvs# netstat -r
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
0.0.0.0 director 0.0.0.0 UG 0 0 0 eth0

The VS-NAT LVS now works. If LVS is forwarding telnet, you can
telnet from the client to the VIP and connect to the real-server.

You can ping from the client to the real-server.

You can also connect _directly_ to services on the real-server not
being forwarded by LVS (in this case e.g. ftp).

You can no longer connect directly to the real-server for services
being forwarded by the LVS. Here telnet ports are not being rewritten
by the LVS (ie telnet->telnet).

client:~# telnet realserver
Trying 192.168.1.11...
^C

(i.e. connection hangs)

Here's tcpdump on the director (which can see all packets).
The client initiates telnet. `netstat -a` on the client
shows a SYN_SENT from port 4121.

director:/etc/lvs# tcpdump
tcpdump: listening on eth0
16:37:04.655036 realserver.telnet > client.4121: S 354934654:354934654(0) ack 1183118745 win 32120 <mss 1460,sackOK,timestamp 111425176[|tcp]> (DF)
16:37:04.655284 director > realserver: icmp: client tcp port 4 121 unreachable [tos 0xc0]

(repeats every second until I kill telnet on client)

I don't know why I don't see the connect request from client->real-server.
The first packet I see is the ack from the real-server,
which will be forwarded via the director.
The director will rewrite the ack to be from the director.
The client will not accept an ack to port 4121 from director:telnet.

Various debugging techniques (in discussion with Julian) in
case you want to know more:

The routes added with the route command go into the kernel FIB
(Forwarding information base) route table. The contents are
displayed with the route (or netstat -a) command.

Following an icmp redirect, the route updates go into the kernel's
route cache (route -C).

Here's the route cache on the real-server before any packets are sent.
(You can flush the route cache with

echo 1 > /proc/sys/net/ipv4/route/flush
or
ip route flush cache
)

realserver:/etc/rc.d# route -C
Kernel IP routing cache
Source Destination Gateway Flags Metric Ref Use Iface
realserver director director 0 1 0 eth0
director realserver realserver il 0 0 9 lo

With icmp redirects enabled on the director, repeatedly running
traceroute to the client shows the routes changing from 2 hops to 1 hop.

realserver:/etc/rc.d# traceroute client
traceroute to client (192.168.1.254), 30 hops max, 40 byte packets
1 director (192.168.1.9) 0.932 ms 0.562 ms 0.503 ms
2 client (192.168.1.254) 1.174 ms 0.597 ms 0.571 ms
realserver:/etc/rc.d# traceroute client
traceroute to client (192.168.1.254), 30 hops max, 40 byte packets
1 director (192.168.1.9) 0.72 ms 0.581 ms 0.532 ms
2 client (192.168.1.254) 0.845 ms 0.559 ms 0.5 ms
realserver:/etc/rc.d# traceroute client
traceroute to client (192.168.1.254), 30 hops max, 40 byte packets
1 client (192.168.1.254) 0.69 ms * 0.579 ms

Although the route command shows no change in the FIB, the route cache
has changed. (The new route of interest is bracketted by >< signs.)

realserver:/etc/rc.d# route -C
Kernel IP routing cache
Source Destination Gateway Flags Metric Ref Use Iface
client realserver realserver l 0 0 8 lo
realserver realserver realserver l 0 0 1038 lo
realserver director director 0 1 138 eth0
>realserver client client 0 0 6 eth0<
director realserver realserver l 0 0 9 lo
director realserver realserver l 0 0 168 lo

Packets to the client now go directly to the client instead of via
the director.

It takes about 10mins for the client's route cache to expire
(experimental result). The timeouts may be in /proc/sys/net/ipv4/route/gc_*,
but their location and values are well encrypted in the sources :).
Here's the route cache after 10mins.

realserver:/etc/rc.d# route -C
Kernel IP routing cache
Source Destination Gateway Flags Metric Ref Use Iface
realserver realserver realserver l 0 0 1049 lo
realserver director director 0 1 139 eth0
director realserver realserver l 0 0 0 lo
director realserver realserver l 0 0 236 lo

There are no routes to the client anymore. Checking with traceroute,
shows that 2 hops are initially required to get to the client (i.e.
the routing cache has reverted to using the director as the route
to the client). After 2 iterations, icmp redirects route the packets
directly to the client again.

realserver:/etc/rc.d# traceroute client
traceroute to client (192.168.1.254), 30 hops max, 40 byte packets
1 director (192.168.1.9) 0.908 ms 0.572 ms 0.537 ms
2 client (192.168.1.254) 1.179 ms 0.6 ms 0.577 ms
realserver:/etc/rc.d# traceroute client
traceroute to client (192.168.1.254), 30 hops max, 40 byte packets
1 director (192.168.1.9) 0.695 ms 0.552 ms 0.492 ms
2 client (192.168.1.254) 0.804 ms 0.55 ms 0.502 ms
realserver:/etc/rc.d# traceroute client
traceroute to client (192.168.1.254), 30 hops max, 40 byte packets
1 client (192.168.1.254) 0.686 ms 0.533 ms *

If you now turn off icmp redirects on the director.

director:/etc/lvs# echo 0 > /proc/sys/net/ipv4/conf/all/send_redirects
director:/etc/lvs# echo 0 > /proc/sys/net/ipv4/conf/default/send_redirects
director:/etc/lvs# echo 0 > /proc/sys/net/ipv4/conf/eth0/send_redirects

Checking routes on the real-server -

realserver:/etc/lvs# netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
0.0.0.0 director 0.0.0.0 UG 0 0 0 eth0

nothing changed here.

Flush the kernel routing table and show the kernel routing table -

realserver:/etc/lvs# ip route flush cache
realserver:/etc/lvs# route -C
Kernel IP routing cache
Source Destination Gateway Flags Metric Ref Use Iface
realserver director director 0 1 0 eth0
director realserver realserver l 0 0 1 lo

There are no routes to the client.

Now when you send packet to the client, the route stays via the director
needing 2 hops to get to the client. There are no one hop packets
to the client.

realserver:/etc/rc.d# traceroute client
traceroute to client (192.168.1.254), 30 hops max, 40 byte packets
1 director (192.168.1.9) 0.951 ms 0.56 ms 0.491 ms
2 client (192.168.1.254) 0.76 ms 0.599 ms 0.574 ms
realserver:/etc/rc.d# traceroute client
traceroute to client (192.168.1.254), 30 hops max, 40 byte packets
1 director (192.168.1.9) 0.696 ms 0.562 ms 0.583 ms
2 client (192.168.1.254) 0.62 ms 0.603 ms 0.576 ms
realserver:/etc/rc.d# traceroute client
traceroute to client (192.168.1.254), 30 hops max, 40 byte packets
1 director (192.168.1.9) 0.692 ms * 0.599 ms
2 client (192.168.1.254) 0.667 ms 0.603 ms 0.579 ms
realserver:/etc/rc.d# traceroute client
traceroute to client (192.168.1.254), 30 hops max, 40 byte packets
1 director (192.168.1.9) 0.689 ms 0.558 ms 0.487 ms
2 client (192.168.1.254) 0.61 ms 0.63 ms 0.567 ms
realserver:/etc/rc.d# traceroute client
traceroute to client (192.168.1.254), 30 hops max, 40 byte packets
1 director (192.168.1.9) 0.705 ms 0.563 ms 0.526 ms
2 client (192.168.1.254) 0.611 ms 0.595 ms *
realserver:/etc/rc.d# traceroute client
traceroute to client (192.168.1.254), 30 hops max, 40 byte packets
1 director (192.168.1.9) 0.706 ms 0.558 ms 0.535 ms
2 client (192.168.1.254) 0.614 ms 0.593 ms 0.573 ms

The kernel route cache

realserver:/etc/rc.d# route -C
Kernel IP routing cache
Source Destination Gateway Flags Metric Ref Use Iface
client realserver realserver l 0 0 17 lo
realserver realserver realserver l 0 0 2 lo
realserver director director 0 1 0 eth0
>realserver client director 0 0 35 eth0<
director realserver realserver l 0 0 16 lo
director realserver realserver l 0 0 63 lo

shows the the only route to the client (labelled with >< ) is via the director.

#end

--
Joseph Mack mack@ncifcrf.gov
Re: Setting up a one network VS-NAT LVS [ In reply to ]
idf at weewannabe
Jan 18, 2001, 3:02 PM
Post #2 of 37 (4574 views)
Permalink
Well, now I am kinda confused, becasue I did what Joe said (thx Joe!!) and
now the thing is working!!??

So, just to clarify, what exaclty is the "masquerading?" (I will RTFM in the
meantime.)

My setup is:

client - 192.168.168.5
director - 192.168.168.2
real - 192.168.168.3

I did as described by joe's e-mail and my previous postings, and this works,
i.e., when I browse to 192.168.168.2, I get the 192.168.168.3 website. If I
try to browse 192.168.168.3 directly, it hangs.

Perhaps I misunderstood - Please clarify.

Ivan


----- Original Message -----
From: "Julian Anastasov" <ja@ssi.bg>
To: "Joseph Mack" <mack@ncifcrf.gov>
Cc: <lvs-users@LinuxVirtualServer.org>
Sent: Thursday, January 18, 2001 5:46 PM
Subject: Re: Setting up a one network VS-NAT LVS


>
> Hello,
>
> On Thu, 18 Jan 2001, Joseph Mack wrote:
>
> > >From the next version of the HOWTO. Comments gladly accepted.
> >
> > Joe
> >
> > 7.1.2 One network VS-NAT
> >
> > The disadvantage of the 2 network VS-NAT is that the real-servers
> > are not able to connect to machines in the network of the VIP.
> > You couldn't make a VS-NAT setup out of machines already on
> > your LAN, which were also required for other purposes to stay
> > on the LAN network.
> >
> > Here's a 1 network VS-NAT LVS.
> > ________
> > | |
> > | client |
> > |________|
> > CIP=192.168.1.254
> > |
> > (router)
> > |
> > __________ |
> > | | | VIP=192.168.1.110 (eth0:110)
> > | director |---| DIP=192.168.1.1 (eth0)
> > |__________| | DIIP=192.168.1.9 (eth0:9)
> > |
> > |
> > ------------------------------------
> > | | |
> > | | |
> > RIP1=192.168.1.2 RIP2=192.168.1.3 RIP3=10.1.1.4 (all eth0)
> > _____________ _____________ _____________
> > | | | | | |
> > | real-server | | real-server | | real-server |
> > |_____________| |_____________| |_____________|
> >
>
> Joe, the setup with two different logical networks can work
> but when the client and the real server share same logical network
> the masquerading can't work.
>
> Here is a setup that I assume will work:
>
> 192.168.1.1 (DIP)
> 192.168.1.110 (VIP)
> 10.0.0.1 (DIP)
> Client 192.168.1.254 Director 10.0.0.2
> | | Real server
> One physical ---+-----------------------+-------------------+--- ...
> network, aka LAN
> Can be provided
> from switched
> or shared hub
>
>
> So, we have two logical networks: 192.168.1.0/24 and 10.0.0.0/24
> but over one physical connection (shared media).
>
> All NAT rules apply:
>
> - 10.0.0.1 is def gw in real server 10.0.0.2
> - both ends connect through the director
> - send_redirects are stopped in the director, so the client and the
> real server don't receive ICMP redirects and they both don't detect
> that they share same physical network.
>
> > realserver:/etc/lvs# netstat -r
> > Kernel IP routing table
> > Destination Gateway Genmask Flags MSS Window irtt
Iface
> > 192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0
eth0
> > 127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0
lo
> > 0.0.0.0 director 0.0.0.0 UG 0 0 0
eth0
> >
> > Note the route to 192.168.1.0/24. This allows the real-server to send
packets
> > to the client by just putting them out on eth0, where the client will
> > pick them up directly (without being demasqueraded) and the LVS will
> > not work.
>
> Yes, this is a direct connection. One of the both ends (the
> real server) connects directly to the client and the packets reach
> the client unmasqueraded avoiding the translation in the director.
>
> > Remove the route to 192.168.1.0/24.
> >
> > realserver:/etc/lvs#route del -net 192.168.1.0 netmask 255.255.255.0 dev
eth0
> >
> > This will leave you with
> >
> > realserver:/etc/lvs# netstat -r
> > Kernel IP routing table
> > Destination Gateway Genmask Flags MSS Window irtt
Iface
> > 127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0
lo
> > 0.0.0.0 director 0.0.0.0 UG 0 0 0
eth0
>
> Wow, if you don't have RIP how the LVS traffic is working :)
>
> >
> > The VS-NAT LVS now works. If LVS is forwarding telnet, you can
> > telnet from the client to the VIP and connect to the real-server.
>
> May be to another real server with RIP configured ?
>
> > You can ping from the client to the real-server.
> >
> > You can also connect _directly_ to services on the real-server not
> > being forwarded by LVS (in this case e.g. ftp).
>
> To which IP ? Hm, are the packets really accepted from the
> real server?
>
> >
> > You can no longer connect directly to the real-server for services
> > being forwarded by the LVS. Here telnet ports are not being rewritten
> > by the LVS (ie telnet->telnet).
> >
> > client:~# telnet realserver
> > Trying 192.168.1.11...
> > ^C
> >
> > (i.e. connection hangs)
>
> The theory is true here. What can happen is that when a packet
> reaches the real server and there is no RIP in the real server, the
> packet will be forwarded to the next hop: the director because it is
> a default gateway. May be the packet will loop until the IP TTL is
> expired or may be it will be dropped. But soon after the ARP entry in
> the director's cache is invalidated the traffic from LVS to the
> director will stop.
>
> >
> > Here's tcpdump on the director (which can see all packets).
> > The client initiates telnet. `netstat -a` on the client
> > shows a SYN_SENT from port 4121.
> >
> > director:/etc/lvs# tcpdump
> > tcpdump: listening on eth0
> > 16:37:04.655036 realserver.telnet > client.4121: S
354934654:354934654(0) ack 1183118745 win 32120 <mss 1460,sackOK,timestamp
111425176[|tcp]> (DF)
> > 16:37:04.655284 director > realserver: icmp: client tcp port 4 121
unreachable [tos 0xc0]
> >
> > (repeats every second until I kill telnet on client)
> >
> > I don't know why I don't see the connect request from
client->real-server.
>
> Because the director redirects the client via ICMP redirect
> directly to the real server. The real servers sends SYN+ACK through the
> director (showed in the tcpdump output) to the client. But the director
> does not have such connection entry (the initail SYN skipped the director
> after the ICMP redirect) and replies with PORT_UNREACH to the real
> server. This terminates the handshake in the real server.
>
> I assume the tcpdump will show ICMP redirects from the director
> to the client: "go to the real server directly". Are there such redirects?
>
> > The first packet I see is the ack from the real-server,
> > which will be forwarded via the director.
> > The director will rewrite the ack to be from the director.
>
> But there is no such LVS entry, PORT_UNREACH is replied.
>
> Joe, can you perform tests with two different Class C networks: one
> for the client, the external DIP and the VIP and one "internal" network
> for the NAT-ed real servers and the internal director DIP used as def
> gw in the real servers. Something like the above picture, with
> send_redirects=0. I assume this should work without any problems.
>
>
> Regards
>
> --
> Julian Anastasov <ja@ssi.bg>
>
>
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
>
Re: Setting up a one network VS-NAT LVS [ In reply to ]
ja at ssi
Jan 18, 2001, 4:46 PM
Post #3 of 37 (4575 views)
Permalink
Hello,

On Thu, 18 Jan 2001, Joseph Mack wrote:

> >From the next version of the HOWTO. Comments gladly accepted.
>
> Joe
>
> 7.1.2 One network VS-NAT
>
> The disadvantage of the 2 network VS-NAT is that the real-servers
> are not able to connect to machines in the network of the VIP.
> You couldn't make a VS-NAT setup out of machines already on
> your LAN, which were also required for other purposes to stay
> on the LAN network.
>
> Here's a 1 network VS-NAT LVS.
> ________
> | |
> | client |
> |________|
> CIP=192.168.1.254
> |
> (router)
> |
> __________ |
> | | | VIP=192.168.1.110 (eth0:110)
> | director |---| DIP=192.168.1.1 (eth0)
> |__________| | DIIP=192.168.1.9 (eth0:9)
> |
> |
> ------------------------------------
> | | |
> | | |
> RIP1=192.168.1.2 RIP2=192.168.1.3 RIP3=10.1.1.4 (all eth0)
> _____________ _____________ _____________
> | | | | | |
> | real-server | | real-server | | real-server |
> |_____________| |_____________| |_____________|
>

Joe, the setup with two different logical networks can work
but when the client and the real server share same logical network
the masquerading can't work.

Here is a setup that I assume will work:

192.168.1.1 (DIP)
192.168.1.110 (VIP)
10.0.0.1 (DIP)
Client 192.168.1.254 Director 10.0.0.2
| | Real server
One physical ---+-----------------------+-------------------+--- ...
network, aka LAN
Can be provided
from switched
or shared hub


So, we have two logical networks: 192.168.1.0/24 and 10.0.0.0/24
but over one physical connection (shared media).

All NAT rules apply:

- 10.0.0.1 is def gw in real server 10.0.0.2
- both ends connect through the director
- send_redirects are stopped in the director, so the client and the
real server don't receive ICMP redirects and they both don't detect
that they share same physical network.

> realserver:/etc/lvs# netstat -r
> Kernel IP routing table
> Destination Gateway Genmask Flags MSS Window irtt Iface
> 192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
> 127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
> 0.0.0.0 director 0.0.0.0 UG 0 0 0 eth0
>
> Note the route to 192.168.1.0/24. This allows the real-server to send packets
> to the client by just putting them out on eth0, where the client will
> pick them up directly (without being demasqueraded) and the LVS will
> not work.

Yes, this is a direct connection. One of the both ends (the
real server) connects directly to the client and the packets reach
the client unmasqueraded avoiding the translation in the director.

> Remove the route to 192.168.1.0/24.
>
> realserver:/etc/lvs#route del -net 192.168.1.0 netmask 255.255.255.0 dev eth0
>
> This will leave you with
>
> realserver:/etc/lvs# netstat -r
> Kernel IP routing table
> Destination Gateway Genmask Flags MSS Window irtt Iface
> 127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
> 0.0.0.0 director 0.0.0.0 UG 0 0 0 eth0

Wow, if you don't have RIP how the LVS traffic is working :)

>
> The VS-NAT LVS now works. If LVS is forwarding telnet, you can
> telnet from the client to the VIP and connect to the real-server.

May be to another real server with RIP configured ?

> You can ping from the client to the real-server.
>
> You can also connect _directly_ to services on the real-server not
> being forwarded by LVS (in this case e.g. ftp).

To which IP ? Hm, are the packets really accepted from the
real server?

>
> You can no longer connect directly to the real-server for services
> being forwarded by the LVS. Here telnet ports are not being rewritten
> by the LVS (ie telnet->telnet).
>
> client:~# telnet realserver
> Trying 192.168.1.11...
> ^C
>
> (i.e. connection hangs)

The theory is true here. What can happen is that when a packet
reaches the real server and there is no RIP in the real server, the
packet will be forwarded to the next hop: the director because it is
a default gateway. May be the packet will loop until the IP TTL is
expired or may be it will be dropped. But soon after the ARP entry in
the director's cache is invalidated the traffic from LVS to the
director will stop.

>
> Here's tcpdump on the director (which can see all packets).
> The client initiates telnet. `netstat -a` on the client
> shows a SYN_SENT from port 4121.
>
> director:/etc/lvs# tcpdump
> tcpdump: listening on eth0
> 16:37:04.655036 realserver.telnet > client.4121: S 354934654:354934654(0) ack 1183118745 win 32120 <mss 1460,sackOK,timestamp 111425176[|tcp]> (DF)
> 16:37:04.655284 director > realserver: icmp: client tcp port 4 121 unreachable [tos 0xc0]
>
> (repeats every second until I kill telnet on client)
>
> I don't know why I don't see the connect request from client->real-server.

Because the director redirects the client via ICMP redirect
directly to the real server. The real servers sends SYN+ACK through the
director (showed in the tcpdump output) to the client. But the director
does not have such connection entry (the initail SYN skipped the director
after the ICMP redirect) and replies with PORT_UNREACH to the real
server. This terminates the handshake in the real server.

I assume the tcpdump will show ICMP redirects from the director
to the client: "go to the real server directly". Are there such redirects?

> The first packet I see is the ack from the real-server,
> which will be forwarded via the director.
> The director will rewrite the ack to be from the director.

But there is no such LVS entry, PORT_UNREACH is replied.

Joe, can you perform tests with two different Class C networks: one
for the client, the external DIP and the VIP and one "internal" network
for the NAT-ed real servers and the internal director DIP used as def
gw in the real servers. Something like the above picture, with
send_redirects=0. I assume this should work without any problems.


Regards

--
Julian Anastasov <ja@ssi.bg>
Re: Setting up a one network VS-NAT LVS [ In reply to ]
mack at ncifcrf
Jan 18, 2001, 5:28 PM
Post #4 of 37 (4574 views)
Permalink
On Thu, 18 Jan 2001, Julian Anastasov wrote:

> > |
> > ------------------------------------
> > | | |
> > | | |
> > RIP1=192.168.1.2 RIP2=192.168.1.3 RIP3=10.1.1.4 (all eth0)
^^^^^^^^
hmm this should be 192.168.1.4
> > _____________ _____________ _____________
> > | | | | | |
> > | real-server | | real-server | | real-server |
> > |_____________| |_____________| |_____________|
> >
>
> Joe, the setup with two different logical networks can work
> but when the client and the real server share same logical network
> the masquerading can't work.

hmm I'm not clear on "logical" network here (you use this term a bit
so I'd better find out what you mean).

In the original HOWTO I had a 1 NIC director with client,VIP on
192.168.1.0/24 and the real-servers,director_insideip on 10.1.1.1/24.
This works fine. (I assume this is 1 physical network, 2 logical
networks.)

A 2 NIC director with 2 networks is easy.

Here I've got a VS-NAT to work with 1 physical network, and all machines
on the same 192.168.1.0/24 network (1 logical network).

> > Remove the route to 192.168.1.0/24.
> >
> > realserver:/etc/lvs#route del -net 192.168.1.0 netmask 255.255.255.0 dev eth0
> >
> > This will leave you with
> >
> > realserver:/etc/lvs# netstat -r
> > Kernel IP routing table
> > Destination Gateway Genmask Flags MSS Window irtt Iface
> > 127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
> > 0.0.0.0 director 0.0.0.0 UG 0 0 0 eth0
>
> Wow, if you don't have RIP how the LVS traffic is working :)

this is the output of netstat -r, not ifconfig -a. The RIP is still there.

> > The VS-NAT LVS now works. If LVS is forwarding telnet, you can
> > telnet from the client to the VIP and connect to the real-server.
>
> May be to another real server with RIP configured ?

no I watched the hostname at the login prompt when I telnet'ed in.

> > You can ping from the client to the real-server.
> >
> > You can also connect _directly_ to services on the real-server NOT
> > being forwarded by LVS (in this case e.g. ftp).
>
> To which IP ? Hm, are the packets really accepted from the
> real server?

yes, when you ftp to the RIP, you get the correct hostname at the prompt.
(ftp is NOT being LVS'ed)

> The theory is true here. What can happen is that when a packet
> reaches the real server and there is no RIP in the real server, the
> packet will be forwarded to the next hop: the director because it is
> a default gateway. May be the packet will loop until the IP TTL is
> expired or may be it will be dropped. But soon after the ARP entry in
> the director's cache is invalidated the traffic from LVS to the
> director will stop.

do you have an explanation if there is a RIP on the real-server?

> > I don't know why I don't see the connect request from client->real-server.
>
> Because the director redirects the client via ICMP redirect
> directly to the real server. The real servers sends SYN+ACK through the
> director (showed in the tcpdump output) to the client. But the director
> does not have such connection entry (the initail SYN skipped the director
> after the ICMP redirect) and replies with PORT_UNREACH to the real
> server. This terminates the handshake in the real server.
>
> I assume the tcpdump will show ICMP redirects from the director
> to the client: "go to the real server directly". Are there such redirects?

redirects are off on the director :-)

however tcpdump is not showing PORT_UNREACH packets etc.

> > The first packet I see is the ack from the real-server,
> > which will be forwarded via the director.
> > The director will rewrite the ack to be from the director.
>
> But there is no such LVS entry, PORT_UNREACH is replied.

yes

> Joe, can you perform tests with two different Class C networks: one
> for the client, the external DIP and the VIP and one "internal" network
> for the NAT-ed real servers and the internal director DIP used as def
> gw in the real servers. Something like the above picture, with
> send_redirects=0. I assume this should work without any problems.

yes it does. It's in the HOWTO :-)

Joe

--
Joseph Mack mack@ncifcrf.gov
Re: Setting up a one network VS-NAT LVS [ In reply to ]
ja at ssi
Jan 19, 2001, 7:01 AM
Post #5 of 37 (4573 views)
Permalink
Hello,

On Thu, 18 Jan 2001, Joseph Mack wrote:

> On Thu, 18 Jan 2001, Julian Anastasov wrote:
>
> > > |
> > > ------------------------------------
> > > | | |
> > > | | |
> > > RIP1=192.168.1.2 RIP2=192.168.1.3 RIP3=10.1.1.4 (all eth0)
> ^^^^^^^^
> hmm this should be 192.168.1.4

This is a different network address, internal.

> hmm I'm not clear on "logical" network here (you use this term a bit
> so I'd better find out what you mean).

Different subnets. Many logical networks in one physical.
"Subnets" is not a correct term. 10.0.0/24 is not a subnet in 192.168/16.
May be both are subnets in 0.0.0.0/0 :)

>
> In the original HOWTO I had a 1 NIC director with client,VIP on
> 192.168.1.0/24 and the real-servers,director_insideip on 10.1.1.1/24.
> This works fine. (I assume this is 1 physical network, 2 logical

Wonderful

> networks.)
>
> A 2 NIC director with 2 networks is easy.
>
> Here I've got a VS-NAT to work with 1 physical network, and all machines
> on the same 192.168.1.0/24 network (1 logical network).

Hm, I now understand. So, is it really working or there are
problems?

>
> > > Remove the route to 192.168.1.0/24.
> > >
> > > realserver:/etc/lvs#route del -net 192.168.1.0 netmask 255.255.255.0 dev eth0
> > >
> > > This will leave you with
> > >
> > > realserver:/etc/lvs# netstat -r
> > > Kernel IP routing table
> > > Destination Gateway Genmask Flags MSS Window irtt Iface
> > > 127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
> > > 0.0.0.0 director 0.0.0.0 UG 0 0 0 eth0
> >
> > Wow, if you don't have RIP how the LVS traffic is working :)
>
> this is the output of netstat -r, not ifconfig -a. The RIP is still there.

Oh, well.

> > The theory is true here. What can happen is that when a packet
> > reaches the real server and there is no RIP in the real server, the
> > packet will be forwarded to the next hop: the director because it is
> > a default gateway. May be the packet will loop until the IP TTL is
> > expired or may be it will be dropped. But soon after the ARP entry in
> > the director's cache is invalidated the traffic from LVS to the
> > director will stop.
>
> do you have an explanation if there is a RIP on the real-server?

The gateway address is not reachable. It must be onlink, i.e.
on directly connected link. This is true for any network mask for the
RIP address if you delete the link route. If you delete the def gw and
try to add it this operation must fail. Of course, I may be wrong.

> > > I don't know why I don't see the connect request from client->real-server.
> >
> > Because the director redirects the client via ICMP redirect
> > directly to the real server. The real servers sends SYN+ACK through the
> > director (showed in the tcpdump output) to the client. But the director
> > does not have such connection entry (the initail SYN skipped the director
> > after the ICMP redirect) and replies with PORT_UNREACH to the real
> > server. This terminates the handshake in the real server.
> >
> > I assume the tcpdump will show ICMP redirects from the director
> > to the client: "go to the real server directly". Are there such redirects?
>
> redirects are off on the director :-)
>
> however tcpdump is not showing PORT_UNREACH packets etc.

Well, the redirects are disabled. If they are enabled I assume the
client will be redirected using ICMP message to the real server. Can
you repeat this test: make sure send_redirects are 0 and always flush
the routing cache in each host.

> > > The first packet I see is the ack from the real-server,
> > > which will be forwarded via the director.
> > > The director will rewrite the ack to be from the director.
> >
> > But there is no such LVS entry, PORT_UNREACH is replied.
>
> yes
>
> > Joe, can you perform tests with two different Class C networks: one
> > for the client, the external DIP and the VIP and one "internal" network
> > for the NAT-ed real servers and the internal director DIP used as def
> > gw in the real servers. Something like the above picture, with
> > send_redirects=0. I assume this should work without any problems.
>
> yes it does. It's in the HOWTO :-)

Very good!

> Joe
>
> --
> Joseph Mack mack@ncifcrf.gov


Regards

--
Julian Anastasov <ja@ssi.bg>
Re: Setting up a one network VS-NAT LVS [ In reply to ]
ja at ssi
Jan 19, 2001, 7:19 AM
Post #6 of 37 (4580 views)
Permalink
Hello,

On Thu, 18 Jan 2001, Ivan Figueredo wrote:

> Well, now I am kinda confused, becasue I did what Joe said (thx Joe!!) and
> now the thing is working!!??
>
> So, just to clarify, what exaclty is the "masquerading?" (I will RTFM in the
> meantime.)
>
> My setup is:
>
> client - 192.168.168.5
> director - 192.168.168.2
> real - 192.168.168.3
>
> I did as described by joe's e-mail and my previous postings, and this works,
> i.e., when I browse to 192.168.168.2, I get the 192.168.168.3 website. If I
> try to browse 192.168.168.3 directly, it hangs.

This is with deleted link route? I.e. 192.168.168.3/24 configured
on the real server and then the 192.168.168.0/24 route deleted? Can you
tcpdump in the director and in the real server?

> Perhaps I misunderstood - Please clarify.

I'm not sure what you exactly configure. A tcpdump output will
be enough.

> Ivan

Regards

--
Julian Anastasov <ja@ssi.bg>
Re: Setting up a one network VS-NAT LVS [ In reply to ]
mack.joseph at epa
Jan 19, 2001, 7:53 AM
Post #7 of 37 (4574 views)
Permalink
Julian Anastasov wrote:

>
> Hm, I now understand. So, is it really working or there are
> problems?

Both Ivan and I have got it to work. I assume it's working.


> > > Joe, can you perform tests with two different Class C networks: one
> > > for the client, the external DIP and the VIP and one "internal" network
> > > for the NAT-ed real servers and the internal director DIP used as def
> > > gw in the real servers. Something like the above picture, with
> > > send_redirects=0. I assume this should work without any problems.

will let you know what happens.

Joe

--
Joseph Mack PhD, Senior Systems Engineer, Lockheed Martin
contractor to the National Environmental Supercomputer Center,
mailto:mack.joseph@epa.gov ph# 919-541-0007, RTP, NC, USA
Re: Setting up a one network VS-NAT LVS [ In reply to ]
idf at weewannabe
Jan 19, 2001, 9:41 AM
Post #8 of 37 (4573 views)
Permalink
Julian,

If your time allows, please give me *very* specific instructions on what to
type and I will get the output to you right away.

Sorry, I do not have the expertise you require.

Ivan

----- Original Message -----
From: "Julian Anastasov" <ja@ssi.bg>
To: "Ivan Figueredo" <idf@weewannabe.com>
Cc: <lvs-users@LinuxVirtualServer.org>
Sent: Friday, January 19, 2001 8:19 AM
Subject: Re: Setting up a one network VS-NAT LVS


>
> Hello,
>
> On Thu, 18 Jan 2001, Ivan Figueredo wrote:
>
> > Well, now I am kinda confused, becasue I did what Joe said (thx Joe!!)
and
> > now the thing is working!!??
> >
> > So, just to clarify, what exaclty is the "masquerading?" (I will RTFM in
the
> > meantime.)
> >
> > My setup is:
> >
> > client - 192.168.168.5
> > director - 192.168.168.2
> > real - 192.168.168.3
> >
> > I did as described by joe's e-mail and my previous postings, and this
works,
> > i.e., when I browse to 192.168.168.2, I get the 192.168.168.3 website.
If I
> > try to browse 192.168.168.3 directly, it hangs.
>
> This is with deleted link route? I.e. 192.168.168.3/24 configured
> on the real server and then the 192.168.168.0/24 route deleted? Can you
> tcpdump in the director and in the real server?
>
> > Perhaps I misunderstood - Please clarify.
>
> I'm not sure what you exactly configure. A tcpdump output will
> be enough.
>
> > Ivan
>
> Regards
>
> --
> Julian Anastasov <ja@ssi.bg>
>
>
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
>
Re: Setting up a one network VS-NAT LVS [ In reply to ]
idf at weewannabe
Jan 19, 2001, 9:49 AM
Post #9 of 37 (4580 views)
Permalink
Joe, Julian,

I do seem to be having one problem - I believe that this is related to LVS
as per my one "logical and physical" net setup of LVS, as this was not
happening pre-LVS.

I do not seem to be able to telnet to realserver (192.168.168.3) It times
out. As you may recall, as per man page, on the Director, I did:

------
[root@wee2 httpd]# echo 1 > /proc/sys/net/ipv4/ip_forward
[root@wee2 httpd]# echo 1 > /proc/sys/net/ipv4/ip_always_defrag
[root@wee2 httpd]# ipchains -A forward -j MASQ -s 192.168.168.0/24 -d
0.0.0.0/0

Then I did:

[root@wee2 httpd]# ipvsadmin -A -t 192.168.168.2:80 -s rr
[root@wee2 httpd]# ipvsadm -a -t 192.168.168.2:80 -r 192.168.168.3 -m
-----

FWIW, even with this problem, and even if it cannot be resolved, it was
worth it for me to gain the confidence to get something working *very* fast
and *very* straightforward. For the less advanced networking/TCP-IP people,
this would be a win if it were included in the How-To's, as long as the
proper caveats were given.

I now feel like taking more baby steps...

Ivan


----- Original Message -----
From: "Joseph Mack" <mack.joseph@epa.gov>
To: <lvs-users@LinuxVirtualServer.org>
Sent: Friday, January 19, 2001 8:53 AM
Subject: Re: Setting up a one network VS-NAT LVS


> Julian Anastasov wrote:
>
> >
> > Hm, I now understand. So, is it really working or there are
> > problems?
>
> Both Ivan and I have got it to work. I assume it's working.
>
>
> > > > Joe, can you perform tests with two different Class C networks:
one
> > > > for the client, the external DIP and the VIP and one "internal"
network
> > > > for the NAT-ed real servers and the internal director DIP used as
def
> > > > gw in the real servers. Something like the above picture, with
> > > > send_redirects=0. I assume this should work without any problems.
>
> will let you know what happens.
>
> Joe
>
> --
> Joseph Mack PhD, Senior Systems Engineer, Lockheed Martin
> contractor to the National Environmental Supercomputer Center,
> mailto:mack.joseph@epa.gov ph# 919-541-0007, RTP, NC, USA
>
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
>
Re: Setting up a one network VS-NAT LVS [ In reply to ]
idf at weewannabe
Jan 19, 2001, 9:55 AM
Post #10 of 37 (4574 views)
Permalink
I also am not able to ftp to realserver from client. Do all the machines
have to have the director as their gateway?

Real server is a RH Linux 6.x box.
Director is a RH Linux 6.x box.
Client is NT 4.0 SP6.

Ivan


> ----- Original Message -----
> From: "Joseph Mack" <mack.joseph@epa.gov>
> To: <lvs-users@LinuxVirtualServer.org>
> Sent: Friday, January 19, 2001 8:53 AM
> Subject: Re: Setting up a one network VS-NAT LVS
>
>
> > Julian Anastasov wrote:
> >
> > >
> > > Hm, I now understand. So, is it really working or there are
> > > problems?
> >
> > Both Ivan and I have got it to work. I assume it's working.
> >
> >
> > > > > Joe, can you perform tests with two different Class C
networks:
> one
> > > > > for the client, the external DIP and the VIP and one "internal"
> network
> > > > > for the NAT-ed real servers and the internal director DIP used as
> def
> > > > > gw in the real servers. Something like the above picture, with
> > > > > send_redirects=0. I assume this should work without any problems.
> >
> > will let you know what happens.
> >
> > Joe
> >
> > --
> > Joseph Mack PhD, Senior Systems Engineer, Lockheed Martin
> > contractor to the National Environmental Supercomputer Center,
> > mailto:mack.joseph@epa.gov ph# 919-541-0007, RTP, NC, USA
> >
> > _______________________________________________
> > LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> > Send requests to lvs-users-request@LinuxVirtualServer.org
> > or go to http://www.in-addr.de/mailman/listinfo/lvs-users
> >
>
>
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
>
Re: Setting up a one network VS-NAT LVS [ In reply to ]
mack.joseph at epa
Jan 19, 2001, 11:26 AM
Post #11 of 37 (4572 views)
Permalink
Ivan Figueredo wrote:
>
> I also am not able to ftp to realserver from client. Do all the machines
> have to have the director as their gateway?

yes (well the real-servers anyhow).

Am testing all of this 1 net VS-NAT. Will get back when I have
results

Joe

--
Joseph Mack PhD, Senior Systems Engineer, Lockheed Martin
contractor to the National Environmental Supercomputer Center,
mailto:mack.joseph@epa.gov ph# 919-541-0007, RTP, NC, USA
Re: Setting up a one network VS-NAT LVS [ In reply to ]
mack.joseph at epa
Jan 19, 2001, 2:00 PM
Post #12 of 37 (4571 views)
Permalink
Ivan Figueredo wrote:
>
> I also am not able to ftp to realserver from client.

when you set up your masq rules with ipchains on the director,
you probably masqueraded all ports. This means that any packets
coming from the real-servers will be masqueraded, even if they
aren't from services that have been LVS'ed on the way in.

Neat security feature huh? You can't connect to any services
on the real-servers, except those that have been LVS'ed :-(

Instead for each service that is being LVS'ed, on the director you
set up masquerading by running a line like this.

director:# ipchains -A forward -p tcp -j MASQ -s realserver_name service -d
0.0.0.0/0

where service = telnet, http

My configure script (on the web site) does this for you.
It doesn't handle the 1 net VS-NAT LVS (yet). To do this
you'll need to run the send_redirects on the director,
and set up the routes on the real-servers(s) first.

Joe
--
Joseph Mack PhD, Senior Systems Engineer, Lockheed Martin
contractor to the National Environmental Supercomputer Center,
mailto:mack.joseph@epa.gov ph# 919-541-0007, RTP, NC, USA
Re: Setting up a one network VS-NAT LVS [ In reply to ]
ja at ssi
Jan 19, 2001, 2:06 PM
Post #13 of 37 (4574 views)
Permalink
Hello,

On Fri, 19 Jan 2001, Ivan Figueredo wrote:

> Julian,
>
> If your time allows, please give me *very* specific instructions on what to
> type and I will get the output to you right away.

On client, director and real server:

tcpdump -len net 192.168.168.0 mask 255.255.255.0

analyze the MAC addresses from the outputs. Are they the
expected. We want to see the ICMP messages too, not only the TCP
packets. Before each test run the following command on each host:

echo 1 > /proc/sys/net/ipv4/route/flush

We want to see the possible ICMP redirect generated from the director.

> Sorry, I do not have the expertise you require.
>
> Ivan


Regards

--
Julian Anastasov <ja@ssi.bg>
Re: Setting up a one network VS-NAT LVS [ In reply to ]
idf at weewannabe
Jan 19, 2001, 3:13 PM
Post #14 of 37 (4574 views)
Permalink
Joe,

----- Original Message -----
From: "Joseph Mack" <mack.joseph@epa.gov>
To: <lvs-users@LinuxVirtualServer.org>
Sent: Friday, January 19, 2001 3:00 PM
Subject: Re: Setting up a one network VS-NAT LVS


> Ivan Figueredo wrote:
> >
> > I also am not able to ftp to realserver from client.
>
> when you set up your masq rules with ipchains on the director,
> you probably masqueraded all ports. This means that any packets
> coming from the real-servers will be masqueraded, even if they
> aren't from services that have been LVS'ed on the way in.

I am beginning to get he hang of it. I see...

> Neat security feature huh? You can't connect to any services
> on the real-servers, except those that have been LVS'ed :-(

Yes!

> Instead for each service that is being LVS'ed, on the director you
> set up masquerading by running a line like this.
>
> director:# ipchains -A forward -p tcp -j MASQ -s realserver_name
service -d
> 0.0.0.0/0

If I have already entered:
[root@wee2 httpd]# ipchains -A forward -j MASQ -s 192.168.168.0/24 -d
0.0.0.0/0

*and* I now enter your ipchains command above, will the new one override the
old one? Or do they both become in effect, matching the last one, etc?

> where service = telnet, http
>
> My configure script (on the web site) does this for you.
> It doesn't handle the 1 net VS-NAT LVS (yet). To do this
> you'll need to run the send_redirects on the director,
> and set up the routes on the real-servers(s) first.

I think I have already done this, as it was part of your e-mail that stared
this thread, no?

Thx,

Ivan

> Joe
> --
> Joseph Mack PhD, Senior Systems Engineer, Lockheed Martin
> contractor to the National Environmental Supercomputer Center,
> mailto:mack.joseph@epa.gov ph# 919-541-0007, RTP, NC, USA
>
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
>
Re: Setting up a one network VS-NAT LVS [ In reply to ]
mack at ncifcrf
Jan 19, 2001, 6:16 PM
Post #15 of 37 (4575 views)
Permalink
> >
> > director:# ipchains -A forward -p tcp -j MASQ -s realserver_name
> service -d
> > 0.0.0.0/0
>
> If I have already entered:
> [root@wee2 httpd]# ipchains -A forward -j MASQ -s 192.168.168.0/24 -d
> 0.0.0.0/0
>
> *and* I now enter your ipchains command above, will the new one override the
> old one? Or do they both become in effect, matching the last one, etc?
>
clear your ipchains tables (man ipchains) then run the command I gave

> > where service = telnet, http
> >
> > My configure script (on the web site) does this for you.
> > It doesn't handle the 1 net VS-NAT LVS (yet). To do this
> > you'll need to run the send_redirects on the director,
> > and set up the routes on the real-servers(s) first.
>
> I think I have already done this, as it was part of your e-mail that stared
> this thread, no?

yes

Joe

--
Joseph Mack mack@ncifcrf.gov
Re: Setting up a one network VS-NAT LVS [ In reply to ]
idf at weewannabe
Jan 19, 2001, 11:48 PM
Post #16 of 37 (4572 views)
Permalink
That works! Now, I see *no* problems whatsoever. IMHO, this is a big win in
getting LVS up FAST and should be part of the mini-how-to!! I will be
testing for Julian to see what the tcpdump stuff gives (and learn something
too!)

I will be attempting more "complex" settings next, ultimately moving it to a
production enviroment.

Next on my list is to add a secod real server. It would be easiest to use
the current director. I think I saw something about this in the How-To,
something about Julian getting code to work with it. Does this work yet, and
if so, how is it done?

Then, get mon working on this simple 1 Network LVS.

Then, as I already have the director and realserver communicating through
the serial port, and I have to NICs on each UP, I assume that adding a
second director and using "heartbeat" between them to isn't to tough.

Finally, I think for me it is best to use VS-DRin my environment, as I have
a FBSD, Linux and Win2000 AS environment in production. I assume that for
VS-DR one *must* use the two network approach?

Thx Joe,

Regards,

Ivan

----- Original Message -----
From: "Joseph Mack" <mack@ncifcrf.gov>
To: <lvs-users@LinuxVirtualServer.org>
Sent: Friday, January 19, 2001 7:16 PM
Subject: Re: Setting up a one network VS-NAT LVS


> > >
> > > director:# ipchains -A forward -p tcp -j MASQ -s realserver_name
> > service -d
> > > 0.0.0.0/0
> >
> > If I have already entered:
> > [root@wee2 httpd]# ipchains -A forward -j MASQ -s 192.168.168.0/24 -d
> > 0.0.0.0/0
> >
> > *and* I now enter your ipchains command above, will the new one override
the
> > old one? Or do they both become in effect, matching the last one, etc?
> >
> clear your ipchains tables (man ipchains) then run the command I gave
>
> > > where service = telnet, http
> > >
> > > My configure script (on the web site) does this for you.
> > > It doesn't handle the 1 net VS-NAT LVS (yet). To do this
> > > you'll need to run the send_redirects on the director,
> > > and set up the routes on the real-servers(s) first.
> >
> > I think I have already done this, as it was part of your e-mail that
stared
> > this thread, no?
>
> yes
>
> Joe
>
> --
> Joseph Mack mack@ncifcrf.gov
>
>
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
>
Re: Setting up a one network VS-NAT LVS [ In reply to ]
idf at weewannabe
Jan 20, 2001, 12:50 AM
Post #17 of 37 (4573 views)
Permalink
Julian,


----- Original Message -----
From: "Julian Anastasov" <ja@ssi.bg>
To: "Ivan Figueredo" <idf@weewannabe.com>
Cc: <lvs-users@LinuxVirtualServer.org>
Sent: Friday, January 19, 2001 3:06 PM
Subject: Re: Setting up a one network VS-NAT LVS


>
> Hello,
>
> On Fri, 19 Jan 2001, Ivan Figueredo wrote:
>
> > Julian,
> >
> > If your time allows, please give me *very* specific instructions on what
to
> > type and I will get the output to you right away.
>
> On client, director and real server:
>
> tcpdump -len net 192.168.168.0 mask 255.255.255.0

Two of my hosts are linux, but the client is an NT machine. I do not see a
tcpdump anywhere. It is probably called something else. I will see if I can
find it, but if you beat me to it, send it my way.

> analyze the MAC addresses from the outputs. Are they the
> expected. We want to see the ICMP messages too, not only the TCP
> packets. Before each test run the following command on each host:
>
> echo 1 > /proc/sys/net/ipv4/route/flush

Again, I can do this on the director and realserver, but not client. Is this
necessary on NT client?

> We want to see the possible ICMP redirect generated from the director.
>
> > Sorry, I do not have the expertise you require.
> >
> > Ivan
>
>
> Regards


Ivan

> --
> Julian Anastasov <ja@ssi.bg>
>
>
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
>
Re: Setting up a one network VS-NAT LVS [ In reply to ]
ja at ssi
Jan 20, 2001, 4:55 AM
Post #18 of 37 (4575 views)
Permalink
Hello,

On Sat, 20 Jan 2001, Ivan Figueredo wrote:

> Again, I can do this on the director and realserver, but not client. Is this
> necessary on NT client?

No :) Ivan, Don't worry for these output, I see Joe have this setup
working. BTW, the same level of security can be achieved using LVS/DR
where the real servers have private addresses as in the NAT setup. May
be Joe have this info in the HOWTO.

> Ivan


Regards

--
Julian Anastasov <ja@ssi.bg>
Re: Setting up a one network VS-NAT LVS [ In reply to ]
idf at weewannabe
Jan 20, 2001, 9:19 AM
Post #19 of 37 (4578 views)
Permalink
Julian,

----- Original Message -----
From: "Julian Anastasov" <ja@ssi.bg>
To: "Ivan Figueredo" <idf@weewannabe.com>
Cc: <lvs-users@LinuxVirtualServer.org>
Sent: Saturday, January 20, 2001 5:55 AM
Subject: Re: Setting up a one network VS-NAT LVS


>
> Hello,
>
> On Sat, 20 Jan 2001, Ivan Figueredo wrote:
>
> > Again, I can do this on the director and realserver, but not client. Is
this
> > necessary on NT client?
>
> No :) Ivan, Don't worry for these output, I see Joe have this setup
> working.

OK - Is there a web site or book that you can recommend that shows how to
debug/understand TCP/IP packets?

>...BTW, the same level of security can be achieved using LVS/DR
> where the real servers have private addresses as in the NAT setup. May
> be Joe have this info in the HOWTO.

You have anticipated my next question! Thx. this IS the way I will
eventually need to set it up, as REAL IP addresses on the Internet are
scarce.

Regards,

Ivan

> > Ivan
>
>
> Regards
>
> --
> Julian Anastasov <ja@ssi.bg>
>
>
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
>
RE: Setting up a one network VS-NAT LVS [ In reply to ]
Steve.Gonczi at networkengines
Jan 20, 2001, 9:23 AM
Post #20 of 37 (4572 views)
Permalink
>OK - Is there a web site or book that you can recommend that shows how to
>debug/understand TCP/IP packets?

-W. Richard Stevens: TCP-IP Illustrated, Vol 1.
(this is a good intro into packet layouts and protocol basics)
-tcpdump MAN pages
Note, that there is a working tcpdump (a.k.a windump)
for NT4 and W2K as well
RE: Setting up a one network VS-NAT LVS [ In reply to ]
Steve.Gonczi at networkengines
Jan 20, 2001, 9:36 AM
Post #21 of 37 (4571 views)
Permalink
On NT _server_,
there is a uSoft supplied packet capture utility as well.

/sG

-----Original Message-----
From: Ivan Figueredo [mailto:idf@weewannabe.com]
Sent: Saturday, January 20, 2001 11:37 AM
To: lvs-users@LinuxVirtualServer.org
Subject: Re: Setting up a one network VS-NAT LVS


Steve,

Thx.I will look into this book. Also, I found windump at
http://netgroup-serv.polito.it/windump/

Regards,

Ivan

----- Original Message -----
From: "Steve Gonczi" <Steve.Gonczi@networkengines.com>
To: <lvs-users@LinuxVirtualServer.org>
Sent: Saturday, January 20, 2001 10:23 AM
Subject: RE: Setting up a one network VS-NAT LVS


>
> >OK - Is there a web site or book that you can recommend that shows how to
> >debug/understand TCP/IP packets?
>
> -W. Richard Stevens: TCP-IP Illustrated, Vol 1.
> (this is a good intro into packet layouts and protocol basics)
> -tcpdump MAN pages
> Note, that there is a working tcpdump (a.k.a windump)
> for NT4 and W2K as well
>
>
>
>
>
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
>


_______________________________________________
LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://www.in-addr.de/mailman/listinfo/lvs-users
Re: Setting up a one network VS-NAT LVS [ In reply to ]
idf at weewannabe
Jan 20, 2001, 9:37 AM
Post #22 of 37 (4574 views)
Permalink
Steve,

Thx.I will look into this book. Also, I found windump at
http://netgroup-serv.polito.it/windump/

Regards,

Ivan

----- Original Message -----
From: "Steve Gonczi" <Steve.Gonczi@networkengines.com>
To: <lvs-users@LinuxVirtualServer.org>
Sent: Saturday, January 20, 2001 10:23 AM
Subject: RE: Setting up a one network VS-NAT LVS


>
> >OK - Is there a web site or book that you can recommend that shows how to
> >debug/understand TCP/IP packets?
>
> -W. Richard Stevens: TCP-IP Illustrated, Vol 1.
> (this is a good intro into packet layouts and protocol basics)
> -tcpdump MAN pages
> Note, that there is a working tcpdump (a.k.a windump)
> for NT4 and W2K as well
>
>
>
>
>
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
>
Re: Setting up a one network VS-NAT LVS [ In reply to ]
mack.joseph at epa
Jan 20, 2001, 9:38 AM
Post #23 of 37 (4576 views)
Permalink
Steve Gonczi wrote:
>

> -W. Richard Stevens:

In a skit on (I think) "Saturday Night Live" they had a
geeky guy hold one of Steven's books as a prop (to show he
was a real geek)

Joe

--
Joseph Mack PhD, Senior Systems Engineer, Lockheed Martin
contractor to the National Environmental Supercomputer Center,
mailto:mack.joseph@epa.gov ph# 919-541-0007, RTP, NC, USA
Re: Setting up a one network VS-NAT LVS [ In reply to ]
mack.joseph at epa
Jan 20, 2001, 9:41 AM
Post #24 of 37 (4575 views)
Permalink
Ivan Figueredo wrote:


> Next on my list is to add a secod real server. It would be easiest to use
> the current director. I think I saw something about this in the How-To,
> something about Julian getting code to work with it. Does this work yet, and
> if so, how is it done?

> Then, get mon working on this simple 1 Network LVS.

see the HOWTO

> Then, as I already have the director and realserver communicating through
> the serial port, and I have to NICs on each UP, I assume that adding a
> second director and using "heartbeat" between them to isn't to tough.

currently heartbeat etc is setup with ldirectord. It could use mon, but no-one
has done the grunt work.

> Finally, I think for me it is best to use VS-DRin my environment, as I have
> a FBSD, Linux and Win2000 AS environment in production. I assume that for
> VS-DR one *must* use the two network approach?

see the HOWTO.

Joe
--
Joseph Mack PhD, Senior Systems Engineer, Lockheed Martin
contractor to the National Environmental Supercomputer Center,
mailto:mack.joseph@epa.gov ph# 919-541-0007, RTP, NC, USA
Re: Setting up a one network VS-NAT LVS [ In reply to ]
idf at weewannabe
Jan 20, 2001, 10:03 AM
Post #25 of 37 (4573 views)
Permalink
Joe,

> > Then, as I already have the director and realserver communicating
through
> > the serial port, and I have to NICs on each UP, I assume that adding a
> > second director and using "heartbeat" between them to isn't to tough.
>
> currently heartbeat etc is setup with ldirectord. It could use mon, but
no-one
> has done the grunt work.

Hmm, I think I misunderstand the HOW-TO. There, I thought it said that mon
was used to monitor the realserver services, whereas heartbeat is used to
monitor the directors. Perhaps I misunderstand your statement above.

Part of understanding LVS is unravel all its' tools. As I have essentially
set up my LVS by using RH out of the box (I did not rebuild or patch the
kernel,) and have configured LVS "manually" by looking at the man page and
following your lead on this thread (I have not used any of the "automation"
tools provided,) I may be diverging from the "standard" way to set it up as
per your documentation in the HOW-TO's.

However, IMHO, this route of manually setting it up should also be
documented in the HOW-TO's as an alternative, as it may suit some people's
psychology better. It is simple enough to follow, and you get a better feel
as to how the parts fit together.

Ivan

>
> Joe
> --
> Joseph Mack PhD, Senior Systems Engineer, Lockheed Martin
> contractor to the National Environmental Supercomputer Center,
> mailto:mack.joseph@epa.gov ph# 919-541-0007, RTP, NC, USA
>
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
>
1 2  View All

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal