Mailing List Archive

--- Wensong Zhang <wensong@gnuchina.org> wrote:
>
> Status: we will try to keep the IPVS Netfilter module version 0.1.2 or
> later for kernel 2.4 ready for production. New stuff will go to version
> 0.2.x.

Now, is 0.1.2 REALLY ready for production? Can you bet your life on it?
(because i will bet my career if i'll use it instead of the 2.2 version)

--
Florin


__________________________________________________
Do You Yahoo!?
Yahoo! Photos - Share your holiday photos online!
http://photos.yahoo.com/

At 11:50 PM 1/10/2001 +0000, Julian Anastasov wrote:

> Hello,
>
>On Wed, 10 Jan 2001, Florin Andrei wrote:
>
>> --- Wensong Zhang <wensong@gnuchina.org> wrote:
>> >
>> > Status: we will try to keep the IPVS Netfilter module version 0.1.2 or
>> > later for kernel 2.4 ready for production. New stuff will go to version
>> > 0.2.x.
>>
>> Now, is 0.1.2 REALLY ready for production? Can you bet your life on it?
>> (because i will bet my career if i'll use it instead of the 2.2 version)
>
> Wow, I don't know many people in production with kernel 2.4 :)

Julian is right. So far, no one has officially support
2.4 kernel, not even Red Hat.

>We don't know for bugs in LVS for 2.4 but may be you need an opinion
>from other happy people in production. You can notice one difference
>between LVS for 2.2 and 2.4: the NAT setup is tricky to build if you
>rely on netfilter's new connection tracking using iptables. The LVS/DR
>and LVS/TUN methods and the other functionalities are working in the same
>way. The other difference is that in 2.4 LVS can serve FTP without any
>help from other modules, even for LVS/NAT. In 2.2 you need ip_masq_ftp.
>
> The users with more CPUs and NICs can build now more powerful
>boxes, thanks to the new 2.4 networking.
>
> But don't ignore your tests. It is always risky to jump to
>something new. Someone can be happy but may be the new LVS can't work
>for your setup. It is up to you to make your transition plan safe :)
>For me, the 2.2 kernel is good enough for LVS box and I don't need to
>change it to 2.4. I prefer to burn some real servers than to kill
>my lovely LVS box :)
>
>
>Regards
>
>--
>Julian Anastasov <ja@ssi.bg>
>
>
>_______________________________________________
>LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
>Send requests to lvs-users-request@LinuxVirtualServer.org
>or go to http://www.in-addr.de/mailman/listinfo/lvs-users

--- Julian Anastasov <ja@ssi.bg> wrote:
>
> Wow, I don't know many people in production with kernel 2.4 :)

You're right. Maybe i'm too eager to put 2.4 to work. I think i'll delay it a
little bit.
I just wonder, those nice guys at Sourceforge, Real.com, and so on, what
kernel and what lvs version they are using in production?... ;-)

> between LVS for 2.2 and 2.4: the NAT setup is tricky to build if you
> rely on netfilter's new connection tracking using iptables.

Why? Can you detail a little bit?
I used netfilter since 2.3.*, with plain masquerading (no LVS, just usual masq
stuff), port forwarding and other goodies. It worked without any problem.

> The LVS/DR
> and LVS/TUN methods and the other functionalities are working in the same
> way. The other difference is that in 2.4 LVS can serve FTP without any
> help from other modules, even for LVS/NAT. In 2.2 you need ip_masq_ftp.

But isn't there an ftp module in netfilter?

> The users with more CPUs and NICs can build now more powerful
> boxes, thanks to the new 2.4 networking.

That's the idea. My future LVS will have a dual-CPU motherboard, and at least
3 interfaces. I saw that 2.4 can use a multiCPU machine much better, so this is
why i tried to push 2.4 into production.

> But don't ignore your tests. It is always risky to jump to
> something new. Someone can be happy but may be the new LVS can't work
> for your setup. It is up to you to make your transition plan safe :)
> For me, the 2.2 kernel is good enough for LVS box and I don't need to
> change it to 2.4. I prefer to burn some real servers than to kill
> my lovely LVS box :)

Another thing is that netfilter is soooo powerful. By using 2.4, i'll have
another strong reason to prevent Cisco to take over the LVS functionality, and
keep Linux on the director node. ;-)
(you know, with 2.4 you can do many clever Cisco-like tricks with the
packets...)

--
Florin

__________________________________________________
Do You Yahoo!?
Yahoo! Photos - Share your holiday photos online!
http://photos.yahoo.com/

--- Julian Anastasov <ja@ssi.bg> wrote:
>
> Wow, I don't know many people in production with kernel 2.4 :)

You're right. Maybe i'm too eager to put 2.4 to work. I think i'll delay it a
little bit.
I just wonder, those nice guys at Sourceforge, Real.com, and so on, what
kernel and what lvs version they are using in production?... ;-)

> between LVS for 2.2 and 2.4: the NAT setup is tricky to build if you
> rely on netfilter's new connection tracking using iptables.

Why? Can you detail a little bit?
I used netfilter since 2.3.*, with plain masquerading (no LVS, just usual masq
stuff), port forwarding and other goodies. It worked without any problem.

> The LVS/DR
> and LVS/TUN methods and the other functionalities are working in the same
> way. The other difference is that in 2.4 LVS can serve FTP without any
> help from other modules, even for LVS/NAT. In 2.2 you need ip_masq_ftp.

But isn't there an ftp module in netfilter?

> The users with more CPUs and NICs can build now more powerful
> boxes, thanks to the new 2.4 networking.

That's the idea. My future LVS will have a dual-CPU motherboard, and at least
3 interfaces. I saw that 2.4 can use a multiCPU machine much better, so this is
why i tried to push 2.4 into production.

> But don't ignore your tests. It is always risky to jump to
> something new. Someone can be happy but may be the new LVS can't work
> for your setup. It is up to you to make your transition plan safe :)
> For me, the 2.2 kernel is good enough for LVS box and I don't need to
> change it to 2.4. I prefer to burn some real servers than to kill
> my lovely LVS box :)

Another thing is that netfilter is soooo powerful. By using 2.4, i'll have
another strong reason to prevent Cisco to take over the LVS functionality, and
keep Linux on the director node. ;-)
(you know, with 2.4 you can do many clever Cisco-like tricks with the
packets...)

--
Florin

__________________________________________________
Do You Yahoo!?
Yahoo! Photos - Share your holiday photos online!
http://photos.yahoo.com/

Hi,

Florin Andrei wrote:
>
> --- Julian Anastasov <ja@ssi.bg> wrote:
> >
> > Wow, I don't know many people in production with kernel 2.4 :)
>
> You're right. Maybe i'm too eager to put 2.4 to work. I think i'll delay it a
> little bit.
> I just wonder, those nice guys at Sourceforge, Real.com, and so on, what
> kernel and what lvs version they are using in production?... ;-)
>

our biggest LVS has >70 real server and is running
a 2.2.14 Kernel with a LVS 0.9.7 (+ patch against
wrong counting); the masq table size is 2^18 (a
bit low, but is still working)
all services are running in DR-mode with the wlc
scheduler
the hardware ist single P3 500 with 512MB ram and
a tulip NIC.

the monitoring of the realservers is done with
ldirectord (tcp-connect)

I don´t want to switch to 2.4 in the next time,
becaus everthing is working fine and we had never
a crash of the Server.
we had in the past more problems with broken
hardware and buggy NIC-drivers (only on the real
Servers) than with the LVS-Kernel itself.

cu,

Chris

Hello,

On Wed, 10 Jan 2001, Florin Andrei wrote:

> --- Wensong Zhang <wensong@gnuchina.org> wrote:
> >
> > Status: we will try to keep the IPVS Netfilter module version 0.1.2 or
> > later for kernel 2.4 ready for production. New stuff will go to version
> > 0.2.x.
>
> Now, is 0.1.2 REALLY ready for production? Can you bet your life on it?
> (because i will bet my career if i'll use it instead of the 2.2 version)

Wow, I don't know many people in production with kernel 2.4 :)
We don't know for bugs in LVS for 2.4 but may be you need an opinion
from other happy people in production. You can notice one difference
between LVS for 2.2 and 2.4: the NAT setup is tricky to build if you
rely on netfilter's new connection tracking using iptables. The LVS/DR
and LVS/TUN methods and the other functionalities are working in the same
way. The other difference is that in 2.4 LVS can serve FTP without any
help from other modules, even for LVS/NAT. In 2.2 you need ip_masq_ftp.

The users with more CPUs and NICs can build now more powerful
boxes, thanks to the new 2.4 networking.

But don't ignore your tests. It is always risky to jump to
something new. Someone can be happy but may be the new LVS can't work
for your setup. It is up to you to make your transition plan safe :)
For me, the 2.2 kernel is good enough for LVS box and I don't need to
change it to 2.4. I prefer to burn some real servers than to kill
my lovely LVS box :)


Regards

--
Julian Anastasov <ja@ssi.bg>

--- Chris <chris@isg.de> wrote:
>
> our biggest LVS has >70 real server and is running
> a 2.2.14 Kernel with a LVS 0.9.7 (+ patch against
> wrong counting); the masq table size is 2^18 (a
> bit low, but is still working)
> all services are running in DR-mode with the wlc
> scheduler
> the hardware ist single P3 500 with 512MB ram and
> a tulip NIC.

Why so much RAM? Are you running some services on the director?

How big is the system load on the director? (load average, CPU %)
How many requests/second you serve with the director?
How many bytes/second?

--
Florin


__________________________________________________
Do You Yahoo!?
Yahoo! Photos - Share your holiday photos online!
http://photos.yahoo.com/

Hello,

On Wed, 10 Jan 2001, Florin Andrei wrote:

> > between LVS for 2.2 and 2.4: the NAT setup is tricky to build if you
> > rely on netfilter's new connection tracking using iptables.
>
> Why? Can you detail a little bit?
> I used netfilter since 2.3.*, with plain masquerading (no LVS, just usual masq
> stuff), port forwarding and other goodies. It worked without any problem.

Yes, netfilter is working very good ... without LVS :)
If you browse my last postings related to the 2.4 kernel you can find
more info. Just search for "netfilter" in the mailing list:

http://marc.theaimsgroup.com/?l=linux-virtual-server&r=1&w=2

Some references:

http://marc.theaimsgroup.com/?l=linux-virtual-server&m=97415815026910&w=2
http://marc.theaimsgroup.com/?l=linux-virtual-server&m=97245361623266&w=2
http://marc.theaimsgroup.com/?l=linux-virtual-server&m=97463283012922&w=2

In short, there is a requirement in LVS to stay in LOCAL_IN
chain but ip_conntrack is working before LVS in the pre_routing.
Currently, the 2.4 routing is based on the data in the IP header.
LVS requires (for DR method) other kind of control. LVS can't benefit
from "mangling packets in the pre_routing". We build our decision
based on the input route function, the fwmarking, etc. Now netfilter
creates always connection structure, even when LVS will create its
own. There is no way to support multiple connection tracking modules.
And so it is difficult for LVS to stick with the netfilter requirements.
Some of the functionalities simply can't be ported to the current model.
Solutions? We still are trying to find them. But it is difficult in
code-freeze. There are some fundamental concepts that prevent to
intergrate LVS clearly but may be there are some workarounds we can
implement. This is in our TODO list.

> > The LVS/DR
> > and LVS/TUN methods and the other functionalities are working in the same
> > way. The other difference is that in 2.4 LVS can serve FTP without any
> > help from other modules, even for LVS/NAT. In 2.2 you need ip_masq_ftp.
>
> But isn't there an ftp module in netfilter?

Oh, yes. There is. But read above.

>
> > The users with more CPUs and NICs can build now more powerful
> > boxes, thanks to the new 2.4 networking.
>
> That's the idea. My future LVS will have a dual-CPU motherboard, and at least
> 3 interfaces. I saw that 2.4 can use a multiCPU machine much better, so this is
> why i tried to push 2.4 into production.

My recommendation: go with LVS/DR. Linux 2.4 will be ready for
production soon but this is my opinion.

> > But don't ignore your tests. It is always risky to jump to
> > something new. Someone can be happy but may be the new LVS can't work
> > for your setup. It is up to you to make your transition plan safe :)
> > For me, the 2.2 kernel is good enough for LVS box and I don't need to
> > change it to 2.4. I prefer to burn some real servers than to kill
> > my lovely LVS box :)
>
> Another thing is that netfilter is soooo powerful. By using 2.4, i'll have
> another strong reason to prevent Cisco to take over the LVS functionality, and
> keep Linux on the director node. ;-)

Yes, iptables is powerful enough for many different things.
If you talk about the advanced routing you can find most of the code
in 2.2 too.

> (you know, with 2.4 you can do many clever Cisco-like tricks with the
> packets...)


Regards

--
Julian Anastasov <ja@ssi.bg>

Hi,

Florin Andrei wrote:
>
> --- Chris <chris@isg.de> wrote:
> >
> > our biggest LVS has >70 real server and is running
> > a 2.2.14 Kernel with a LVS 0.9.7 (+ patch against
> > wrong counting); the masq table size is 2^18 (a
> > bit low, but is still working)
> > all services are running in DR-mode with the wlc
> > scheduler
> > the hardware ist single P3 500 with 512MB ram and
> > a tulip NIC.
>
> Why so much RAM? Are you running some services on the director?
>
> How big is the system load on the director? (load average, CPU %)
> How many requests/second you serve with the director?
> How many bytes/second?

the RAM was an mistake by me, because of the
masq-table size (I first thought ipvsadm
shows kbyte and not byte :-)
the only service on the director is ldirectord

in peaks we have about 7000 req/s on the director
with an load of 0.07

the only Problem, that we don´t have more Requests
is, that the real servers are serving dynamic
webpages
and they ran out of CPU-time

incomming traffic is about 15 m/bit and outgoing
ca 60 m/bit, but we never verified the numbers.

cu,

Chris


>
> --
> Florin
>
> __________________________________________________
> Do You Yahoo!?
> Yahoo! Photos - Share your holiday photos online!
> http://photos.yahoo.com/
>
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users