Mailing List Archive

Hello,

On Sun, 4 Mar 2001, William Clements wrote:

> Hi,
>
> I'm using testlvs to test throughput on a test system. I've
> noted the following:
>
> >From a client machine, I use:
> testlvs 192.168.1.250:80 -tcp -srcnum 10 -packets 10000

Is there an ipchains -I input -s 10.0.0.0/24 -j DENY rule in
the real servers? I assume you try to test the throughput in the
director and not to flood your web services. The traffic can be
redirected to another place if you don't follow the instructions
to block it in the real servers. You have to describe your setup.

> Using show_traffic.sh I get the following:
>
> director: ~ 1002 packets/sec
> webserver1: ~ 50 packets/sec
> webserver2: ~ 50 packets/sec
>
> (as per the show_traffic.sh default, all measurements are over
> a 10 second period of time)
> So, I'm wondering what happened to the "other" packets.
> Any thoughts?

It seems the packets are not scheduled. What shows telnet from
the client host to 192.168.1.250 ? ipvsadm -Ln ?

> Much Thanks
>
> Will


Regards

--
Julian Anastasov <ja@ssi.bg>

* On 03/05/01 ja@ssi.bg wrote:
>
> On Sun, 4 Mar 2001, William Clements wrote:
>
> > I'm using testlvs to test throughput on a test system. I've
> > noted the following:
> >
> > >From a client machine, I use:
> > testlvs 192.168.1.250:80 -tcp -srcnum 10 -packets 10000
>
> Is there an ipchains -I input -s 10.0.0.0/24 -j DENY rule in
> the real servers? I assume you try to test the throughput in the
> director and not to flood your web services. The traffic can be
> redirected to another place if you don't follow the instructions
> to block it in the real servers. You have to describe your setup.
>

I'm not using ipchains anywhere. I used "route" on the
real webservers to drop the bogus route.

> > Using show_traffic.sh I get the following:
> >
> > director: ~ 1002 packets/sec
> > webserver1: ~ 50 packets/sec
> > webserver2: ~ 50 packets/sec
> >
> > (as per the show_traffic.sh default, all measurements are over
> > a 10 second period of time)
> > So, I'm wondering what happened to the "other" packets.
> > Any thoughts?
>
> It seems the packets are not scheduled. What shows telnet from
> the client host to 192.168.1.250 ? ipvsadm -Ln ?
>

Output from ipvsadm -L -n:

IP Virtual Server version 1.0.5 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.1.250:80 lc
-> 192.168.1.221:80 Route 1 0 0
-> 192.168.1.220:80 Route 1 0 0

Conectivity is not the issue. I've also used "ab" and "ptester"
simulate real traffic. (Unfortunately the 10mb pathway I'm using quicky
gets saturated if the return traffic is to large).

Typical "ab" traffic:

ab -n 10000 -c 100 http://192.168.1.250/test.html

director: ~ 6562 packets/sec
webserver1: ~ 3297 packets/sec
webserver2: ~ 3254 packets/sec

Server Software: Apache/1.3.17
Server Hostname: 192.168.1.250
Server Port: 80

Document Path: /test.html
Document Length: 81145 bytes

Concurrency Level: 100
Time taken for tests: 69.665 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 815721360 bytes
HTML transferred: 813210610 bytes
Requests per second: 143.54
Transfer rate: 11709.20 kb/s received

Connnection Times (ms)
min avg max
Connect: 4 217 9557
Processing: 27 467 3408
Total: 31 684 12965

---------------------------------------

Also of note: I've just switched over to the 1.0.5-2.2.18 lvs patch.


Much Thanks

Will

Hello,

On Mon, 5 Mar 2001, William Clements wrote:

> > Is there an ipchains -I input -s 10.0.0.0/24 -j DENY rule in
> > the real servers? I assume you try to test the throughput in the
> > director and not to flood your web services. The traffic can be
> > redirected to another place if you don't follow the instructions
> > to block it in the real servers. You have to describe your setup.
> >
>
> I'm not using ipchains anywhere. I used "route" on the
> real webservers to drop the bogus route.

OK

> > > Using show_traffic.sh I get the following:
> > >
> > > director: ~ 1002 packets/sec
> > > webserver1: ~ 50 packets/sec
> > > webserver2: ~ 50 packets/sec
> > >
> > > (as per the show_traffic.sh default, all measurements are over
> > > a 10 second period of time)
> > > So, I'm wondering what happened to the "other" packets.
> > > Any thoughts?
> >
> > It seems the packets are not scheduled. What shows telnet from
> > the client host to 192.168.1.250 ? ipvsadm -Ln ?
> >
>
> Output from ipvsadm -L -n:
>
> IP Virtual Server version 1.0.5 (size=4096)
> Prot LocalAddress:Port Scheduler Flags
> -> RemoteAddress:Port Forward Weight ActiveConn InActConn
> TCP 192.168.1.250:80 lc
> -> 192.168.1.221:80 Route 1 0 0
> -> 192.168.1.220:80 Route 1 0 0

Hm, may be the packets are dropped in the director. May be
you have rp_filter=1 and no route to 10/8. What about adding in the
director:

ip route add default via some_gateway dev eth0
or just
ip route add 10/8 via the_same_gateway dev eth0

The gateway can simply be the client host where you run
testlvs, i.e. the device where the requests come from if we don't want
to cause more troubles when using rp_filter.

If you don't have route to the used spoofed source addresses,
which are from 10.0.0.0/8 by default, the rp_filter in the director
just drops them. This is the only difference I can see when using
testlvs and other L7 clients (ab, ptester, etc) which use normal
source addresses.

> Conectivity is not the issue. I've also used "ab" and "ptester"
> simulate real traffic. (Unfortunately the 10mb pathway I'm using quicky
> gets saturated if the return traffic is to large).
>
> Typical "ab" traffic:
>
> ab -n 10000 -c 100 http://192.168.1.250/test.html
>
> director: ~ 6562 packets/sec
> webserver1: ~ 3297 packets/sec
> webserver2: ~ 3254 packets/sec
...

> Much Thanks
>
> Will


Regards

--
Julian Anastasov <ja@ssi.bg>

* On 03/05/01 ja@ssi.bg wrote:

> On Mon, 5 Mar 2001, William Clements wrote:
>
> > > Is there an ipchains -I input -s 10.0.0.0/24 -j DENY rule in
> > > the real servers? I assume you try to test the throughput in the
> > > director and not to flood your web services. The traffic can be
> > > redirected to another place if you don't follow the instructions
> > > to block it in the real servers. You have to describe your setup.
> > >
> >
> > I'm not using ipchains anywhere. I used "route" on the
> > real webservers to drop the bogus route.
>
> OK
>
> > > > Using show_traffic.sh I get the following:
> > > >
> > > > director: ~ 1002 packets/sec
> > > > webserver1: ~ 50 packets/sec
> > > > webserver2: ~ 50 packets/sec
> > > >
> > > > (as per the show_traffic.sh default, all measurements are over
> > > > a 10 second period of time)
> > > > So, I'm wondering what happened to the "other" packets.
> > > > Any thoughts?
>
> Hm, may be the packets are dropped in the director. May be
> you have rp_filter=1 and no route to 10/8. What about adding in the
> director:
>
> ip route add default via some_gateway dev eth0
> or just
> ip route add 10/8 via the_same_gateway dev eth0
>
> The gateway can simply be the client host where you run
> testlvs, i.e. the device where the requests come from if we don't want
> to cause more troubles when using rp_filter.
>
> If you don't have route to the used spoofed source addresses,
> which are from 10.0.0.0/8 by default, the rp_filter in the director
> just drops them. This is the only difference I can see when using
> testlvs and other L7 clients (ab, ptester, etc) which use normal
> source addresses.
>

Humm....

/sbin/sysctl -a | grep rp_filter:

net.ipv4.conf.eth0.arp_filter = 0
net.ipv4.conf.eth0.rp_filter = 0
net.ipv4.conf.lo.arp_filter = 0
net.ipv4.conf.lo.rp_filter = 0
net.ipv4.conf.default.arp_filter = 0
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.all.arp_filter = 0
net.ipv4.conf.all.rp_filter = 0

(I reset net.ipv4.conf.all.rp_filter )

As for the gateway, my default gateway should catch all the 10.0.0.0/8
IP's, and this traffic should show up on my firewall (As is actually the
case for the real webservers before I dropped in a routing rule). I
But I have not seen any traffic from the director. I took your
suggestion and added the additional route for the director anyway, but
still the same result.

I guess I'll try tcpdump next. Much thanks for the help.

Will