Mailing List Archive

----- Original Message -----
From: "Ivan Figueredo" <idf@weewannabe.com>
To: "Julian Anastasov" <ja@ssi.bg>
Sent: Saturday, January 20, 2001 10:53 AM
Subject: Re: Setting up a one network VS-NAT LVS


Julian,

>
> ----- Original Message -----
> From: "Julian Anastasov" <ja@ssi.bg>
> To: "Ivan Figueredo" <idf@weewannabe.com>
> Cc: <lvs-users@LinuxVirtualServer.org>
> Sent: Saturday, January 20, 2001 12:43 PM
> Subject: Re: Setting up a one network VS-NAT LVS
>
>
> >
> > Hello,
> >
> > On Sat, 20 Jan 2001, Ivan Figueredo wrote:
> >
> > > OK - Is there a web site or book that you can recommend that shows how
> to
> > > debug/understand TCP/IP packets?
> >
> > The RFC documents are your friends:
>

Will do.

>
> > http://www.ietf.cnri.reston.va.us/rfc.html
> >
> > The numbers you need:
> >
> > 793 TRANSMISSION CONTROL PROTOCOL
> > 1122 Requirements for Internet Hosts -- Communication Layers
> > 1812 Requirements for IP Version 4 Routers
> > 826 An Ethernet Address Resolution Protocol
> >
> > man tcpdump can help to understand its outputs. I don't
> > remember for other documents. May be someone else has better
> > information :)
> >
> > > >...BTW, the same level of security can be achieved using LVS/DR
> > > > where the real servers have private addresses as in the NAT setup.
May
> > > > be Joe have this info in the HOWTO.
> > >
> > > You have anticipated my next question! Thx. this IS the way I will
> > > eventually need to set it up, as REAL IP addresses on the Internet are
> > > scarce.
> >
> > Yes, put the same private addresses in the real servers, the
> > same def gw IP from the private network and add the VIPs on the loopback
> > adapter. I don't remember for other requirements. By default, when
> > the devices where the VIPs are defined in the real server are hidden,
> > so this feature does not allow the VIPs to be autoselected from the
> > kernel as source address for outgoing connections. VIP can be used
> > in connections if you bind to VIP and when the director feeds us
> > with packets with daddr=VIP. So, if you don't put other publicly
> > visible IP addresses in the real servers I don't see a reason why the
> > NAT setup will be more secure than this one.
>

Hmm, Interesting...I will trying this when I am ready for showtime...

Ivan

>
> > > Regards,
> > >
> > > Ivan
> >
> >
> > Regards
> >
> > --
> > Julian Anastasov <ja@ssi.bg>
> >
> >
>

Steve,

I did not find this on the Internet, but I will look harder.

Ivan

----- Original Message -----
From: "Steve Gonczi" <Steve.Gonczi@networkengines.com>
To: <lvs-users@LinuxVirtualServer.org>
Sent: Saturday, January 20, 2001 10:36 AM
Subject: RE: Setting up a one network VS-NAT LVS


> On NT _server_,
> there is a uSoft supplied packet capture utility as well.
>
> /sG
>
> -----Original Message-----
> From: Ivan Figueredo [mailto:idf@weewannabe.com]
> Sent: Saturday, January 20, 2001 11:37 AM
> To: lvs-users@LinuxVirtualServer.org
> Subject: Re: Setting up a one network VS-NAT LVS
>
>
> Steve,
>
> Thx.I will look into this book. Also, I found windump at
> http://netgroup-serv.polito.it/windump/
>
> Regards,
>
> Ivan
>
> ----- Original Message -----
> From: "Steve Gonczi" <Steve.Gonczi@networkengines.com>
> To: <lvs-users@LinuxVirtualServer.org>
> Sent: Saturday, January 20, 2001 10:23 AM
> Subject: RE: Setting up a one network VS-NAT LVS
>
>
> >
> > >OK - Is there a web site or book that you can recommend that shows how
to
> > >debug/understand TCP/IP packets?
> >
> > -W. Richard Stevens: TCP-IP Illustrated, Vol 1.
> > (this is a good intro into packet layouts and protocol basics)
> > -tcpdump MAN pages
> > Note, that there is a working tcpdump (a.k.a windump)
> > for NT4 and W2K as well
> >
> >
> >
> >
> >
> > _______________________________________________
> > LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> > Send requests to lvs-users-request@LinuxVirtualServer.org
> > or go to http://www.in-addr.de/mailman/listinfo/lvs-users
> >
>
>
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
>
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
>

On Sat, 20 Jan 2001, Ivan Figueredo wrote:

> Joe,
>

> Hmm, I think I misunderstand the HOW-TO. There, I thought it said that mon
> was used to monitor the realserver services, whereas heartbeat is used to
> monitor the directors. Perhaps I misunderstand your statement above.

No you have it right. However the Ultramonkey version of HA, uses
ldirectord to monitor the services and state of the directors. The
Ultramonkey code was released after the last version of the HOWTO.

> Part of understanding LVS is unravel all its' tools.

yes, it's all complicated, no doubt about that.

> As I have essentially
> set up my LVS by using RH out of the box (I did not rebuild or patch the
> kernel,) and have configured LVS "manually" by looking at the man page and
> following your lead on this thread (I have not used any of the "automation"
> tools provided,) I may be diverging from the "standard" way to set it up as
> per your documentation in the HOW-TO's.
>
> However, IMHO, this route of manually setting it up should also be
> documented in the HOW-TO's as an alternative,

all the info needed is in the docs on the website. The number of ways you
can get it wrong are large and after the first few LVS setups I decided to
do it via scripts.

Joe
--
Joseph Mack mack@ncifcrf.gov

Joe,

----- Original Message -----
From: "Joseph Mack" <mack@ncifcrf.gov>
To: <lvs-users@LinuxVirtualServer.org>
Sent: Saturday, January 20, 2001 11:17 AM
Subject: Re: Setting up a one network VS-NAT LVS


> No you have it right. However the Ultramonkey version of HA, uses
> ldirectord to monitor the services and state of the directors. The
> Ultramonkey code was released after the last version of the HOWTO.

Got it! I will have to look into this Ultramonkey. Any recommendations as to
the route to take?

( i.e., mon+heartbeat set up by hand, or UM I guess using ldirectord, which
probably either incorporates some of the source from mon&heartbeat, or just
calls them in their whole form some script.)


Ivan

> all the info needed is in the docs on the website. The number of ways you
> can get it wrong are large and after the first few LVS setups I decided to
> do it via scripts.

> Joe
> --
> Joseph Mack mack@ncifcrf.gov
>
>
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
>

Ivan Figueredo wrote:
>
> Joe,
>
> ----- Original Message -----
> From: "Joseph Mack" <mack@ncifcrf.gov>
> To: <lvs-users@LinuxVirtualServer.org>
> Sent: Saturday, January 20, 2001 11:17 AM
> Subject: Re: Setting up a one network VS-NAT LVS
>
> > No you have it right. However the Ultramonkey version of HA, uses
> > ldirectord to monitor the services and state of the directors. The
> > Ultramonkey code was released after the last version of the HOWTO.
>
> Got it! I will have to look into this Ultramonkey. Any recommendations as to
> the route to take?
>
> ( i.e., mon+heartbeat set up by hand, or UM I guess using ldirectord, which
> probably either incorporates some of the source from mon&heartbeat, or just
> calls them in their whole form some script.)

You understand what's available now.
UM is ready to go as a package.
The other stuff will need a bit of hand guidance.
I haven't tried ldirectord yet,
so don't have any opinions about it.
Mon works, but the author doesn't reply to e-mail messages,
which means that we either have to take over maintaining it,
or abandon it.

Joe

--
Joseph Mack PhD, Senior Systems Engineer, Lockheed Martin
contractor to the National Environmental Supercomputer Center,
mailto:mack.joseph@epa.gov ph# 919-541-0007, RTP, NC, USA

Hello,

On Sat, 20 Jan 2001, Ivan Figueredo wrote:

> OK - Is there a web site or book that you can recommend that shows how to
> debug/understand TCP/IP packets?

The RFC documents are your friends:

http://www.ietf.cnri.reston.va.us/rfc.html

The numbers you need:

793 TRANSMISSION CONTROL PROTOCOL
1122 Requirements for Internet Hosts -- Communication Layers
1812 Requirements for IP Version 4 Routers
826 An Ethernet Address Resolution Protocol

man tcpdump can help to understand its outputs. I don't
remember for other documents. May be someone else has better
information :)

> >...BTW, the same level of security can be achieved using LVS/DR
> > where the real servers have private addresses as in the NAT setup. May
> > be Joe have this info in the HOWTO.
>
> You have anticipated my next question! Thx. this IS the way I will
> eventually need to set it up, as REAL IP addresses on the Internet are
> scarce.

Yes, put the same private addresses in the real servers, the
same def gw IP from the private network and add the VIPs on the loopback
adapter. I don't remember for other requirements. By default, when
the devices where the VIPs are defined in the real server are hidden,
so this feature does not allow the VIPs to be autoselected from the
kernel as source address for outgoing connections. VIP can be used
in connections if you bind to VIP and when the director feeds us
with packets with daddr=VIP. So, if you don't put other publicly
visible IP addresses in the real servers I don't see a reason why the
NAT setup will be more secure than this one.

> Regards,
>
> Ivan


Regards

--
Julian Anastasov <ja@ssi.bg>

> > Got it! I will have to look into this Ultramonkey. Any recommendations
as to
> > the route to take?
> >
> > ( i.e., mon+heartbeat set up by hand, or UM I guess using ldirectord,
which
> > probably either incorporates some of the source from mon&heartbeat, or
just
> > calls them in their whole form some script.)
>
> You understand what's available now.
> UM is ready to go as a package.
> The other stuff will need a bit of hand guidance.
> I haven't tried ldirectord yet,
> so don't have any opinions about it.
> Mon works, but the author doesn't reply to e-mail messages,
> which means that we either have to take over maintaining it,
> or abandon it.

Hmm, that's a pretty strong argument against mon in my book, unless the LVS
people choose to start maintain it, that is.

Ivan

> Joe
>
> --
> Joseph Mack PhD, Senior Systems Engineer, Lockheed Martin
> contractor to the National Environmental Supercomputer Center,
> mailto:mack.joseph@epa.gov ph# 919-541-0007, RTP, NC, USA
>
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
>

> director:/etc/lvs# echo 0 > /proc/sys/net/ipv4/conf/all/send_redirects
> director:/etc/lvs# echo 0 > /proc/sys/net/ipv4/conf/default/send_redirects
> director:/etc/lvs# echo 0 > /proc/sys/net/ipv4/conf/eth0/send_redirects
>
> (Note: eth0 may be eth1 etc, on your machine).
>
> 2. Make the director the default and only route for outgoing packets.
>
> You will probably have set the routing on the real-server up like this
>
> realserver:/etc/lvs# netstat -r
> Kernel IP routing table
> Destination Gateway Genmask Flags MSS Window irtt
Iface
> 192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0
eth0
> 127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0
lo
> 0.0.0.0 director 0.0.0.0 UG 0 0 0
eth0
>
> Note the route to 192.168.1.0/24. This allows the real-server to send
packets
> to the client by just putting them out on eth0, where the client will
> pick them up directly (without being demasqueraded) and the LVS will
> not work.
>
> Remove the route to 192.168.1.0/24.
>
> realserver:/etc/lvs#route del -net 192.168.1.0 netmask 255.255.255.0 dev
eth0
>
> This will leave you with
>
> realserver:/etc/lvs# netstat -r
> Kernel IP routing table
> Destination Gateway Genmask Flags MSS Window irtt
Iface
> 127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0
lo
> 0.0.0.0 director 0.0.0.0 UG 0 0 0
eth0
>
> The VS-NAT LVS now works. If LVS is forwarding telnet, you can
> telnet from the client to the VIP and connect to the real-server.


Both of these (the echoes an the remove the route) do not seem to survive a
reboot. How can I get it to survive the reboots?

Thx.

Ivan

On Wed, 24 Jan 2001, Ivan Figueredo wrote:

> Both of these (the echoes an the remove the route) do not seem to survive a
> reboot. How can I get it to survive the reboots?

nothing survives a reboot. You have to run it all from a script each time

Joe

--
Joseph Mack mack@ncifcrf.gov

Julian Anastasov wrote:


> Joe, can you perform tests with two different Class C networks: one
> for the client, the external DIP and the VIP and one "internal" network
> for the NAT-ed real servers and the internal director DIP used as def
> gw in the real servers. Something like the above picture, with
> send_redirects=0. I assume this should work without any problems.

(As I said previously) The one network VS-NAT (client, VIP, real-servers
all on 192.168.1.0/24) works with send_redirects=1

The two network, 2 NIC director with VS-NAT (outside addresses -
client, VIP 192.168.2.0/24; inside addresses - DIIP, real-servers
192.168.1.0/24) works with send_redirects=0 (this is trivial)

I just checked the 2 logical network, 1 physical network (outside addresses
192.168.2.0/24, inside addresses 192.168.1.0/24 all on the same wire)
in VS-NAT works with send-redirects=1 (yes that's 1). (This is how I did it in
the
HOWTO - at that time I wasn't paying attention to send_redirects).
The real-servers have the default route to the director and a network
route to the inside network.



Joe

--
Joseph Mack PhD, Senior Systems Engineer, Lockheed Martin
contractor to the National Environmental Supercomputer Center,
mailto:mack.joseph@epa.gov ph# 919-541-0007, RTP, NC, USA

Hello,

On Thu, 25 Jan 2001, Joseph Mack wrote:

> (As I said previously) The one network VS-NAT (client, VIP, real-servers
> all on 192.168.1.0/24) works with send_redirects=1

This is funny :)

> The two network, 2 NIC director with VS-NAT (outside addresses -
> client, VIP 192.168.2.0/24; inside addresses - DIIP, real-servers
> 192.168.1.0/24) works with send_redirects=0 (this is trivial)

Yes, this is the default setup

> I just checked the 2 logical network, 1 physical network (outside addresses
> 192.168.2.0/24, inside addresses 192.168.1.0/24 all on the same wire)
> in VS-NAT works with send-redirects=1 (yes that's 1). (This is how I did it in
> the

Hm, so it looks like redirects are not sent here, I assume this
is Linux 2.2 because in 2.4 I'm sure redirects will not be sent. LVS
in 2.4 lookups the NAT-ed real server using the output route function
while LVS for 2.2 lookups the real server using the input route function
and a redirect is usually returned when the forwarding device matches
the input device, which is our case. But may be I'm missing something in
theory. I'll check it this weekend too.

> HOWTO - at that time I wasn't paying attention to send_redirects).
> The real-servers have the default route to the director and a network
> route to the inside network.
>
>
>
> Joe
>
> --
> Joseph Mack PhD, Senior Systems Engineer, Lockheed Martin
> contractor to the National Environmental Supercomputer Center,
> mailto:mack.joseph@epa.gov ph# 919-541-0007, RTP, NC, USA


Regards

--
Julian Anastasov <ja@ssi.bg>

Julian Anastasov wrote:
>
> Hello,
>
> On Thu, 25 Jan 2001, Joseph Mack wrote:
>
> > (As I said previously) The one network VS-NAT (client, VIP, real-servers
> > all on 192.168.1.0/24) works with send_redirects=1
>
> This is funny :)

Ow! the redirects are off - sorry

(I get mixed up which is on/off)

> > The two network, 2 NIC director with VS-NAT (outside addresses -
> > client, VIP 192.168.2.0/24; inside addresses - DIIP, real-servers
> > 192.168.1.0/24) works with send_redirects=0 (this is trivial)
>
> Yes, this is the default setup

redirects are on.

> > I just checked the 2 logical network, 1 physical network (outside addresses
> > 192.168.2.0/24, inside addresses 192.168.1.0/24 all on the same wire)
> > in VS-NAT works with send-redirects=1 (yes that's 1). (This is how I did it in
> > the

redirects are on.

> Hm, so it looks like redirects are not sent here, I assume this
> is Linux 2.2

yes

> in 2.4 lookups the NAT-ed real server using the output route function
> while LVS for 2.2 lookups the real server using the input route function
> and a redirect is usually returned when the forwarding device matches
> the input device, which is our case. But may be I'm missing something in
> theory. I'll check it this weekend too.

Joe

--
Joseph Mack PhD, Senior Systems Engineer, Lockheed Martin
contractor to the National Environmental Supercomputer Center,
mailto:mack.joseph@epa.gov ph# 919-541-0007, RTP, NC, USA