Mailing List Archive

Dear 胡西宁,

please share more information about your setup:

- distro & release
- what kernel ( uname -r )
release, parameter ( sysctl ? )

- what kenrel ipvs moduules used

- lvs config, ipvs/lvs release
- ipvsadm -v

server logs ..
iptables setup ?
routing setup ?

Otherwise it may be to hard GUEssing how your setup looks like :-)


--
Mit freundlichen Grüßen / Best Regards

Horst Venzke ; PGP NET : 1024G/082F2E6D ; http://www.remsnet.de

Legal Notice: This transmittal and/or attachments may be privileged or confidential. It is intended solely for the addressee named above. Any review, dissemination, or copying is strictly prohibited. If you received this transmittal in error, please notify us immediately by reply and immediately delete this message and all its attachments. Thank you.


> Gesendet: Mittwoch, 09. Juli 2014 um 05:10 Uhr
> Von: 胡西宁 <huxining@yeah.net>
> An: lvs-users@linuxvirtualserver.org
> Betreff: [lvs-users] help: lvs performance testing problem
>
> hi everyone:
> I do test throughput of lvs server, the mode is dr. Num of lvs server is only one. test tool is jmeter, when throughput more than 20,000， It's not increase, I do not know what is the reason, I optimization params of kernal, include limits of open file, tcp connection. etc, thank you
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
>
> LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
> Send requests to lvs-users-request@LinuxVirtualServer.org
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users

add some information:


my kernal and release of lvs are taobao's,



optimization params:
net.ipv4.ip_forward=1
net.nf_conntrack_max = 25000000
net.netfilter.nf_conntrack_max = 25000000
net.netfilter.nf_conntrack_tcp_timeout_established = 1500
net.ipv4.tcp_max_tw_buckets = 6000
net.ipv4.tcp_sack = 1
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_rmem = 4096 87380 4194304
net.ipv4.tcp_wmem = 4096 16384 4194304
net.core.wmem_default = 8388608
net.core.rmem_default = 8388608
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.core.netdev_max_backlog = 262144
net.core.somaxconn = 262144
net.ipv4.tcp_max_orphans = 3276800
net.ipv4.tcp_max_syn_backlog = 262144
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_synack_retries = 1
net.ipv4.tcp_syn_retries = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_mem = 94500000 915000000 927000000
net.ipv4.tcp_fin_timeout = 100
net.ipv4.tcp_keepalive_time = 30
net.ipv4.ip_local_port_range = 1024 65000
net.ipv4.route.flush = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.tcp_syncookies = 0
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv4.tcp_max_syn_backlog = 819200
net.ipv4.tcp_synack_retries = 1
net.ipv4.tcp_max_tw_buckets = 819200
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.core.netdev_max_backlog = 500000
net.ipv4.ip_forward=1
net.ipv4.tcp_tw_recyle=1
net.ipv4.tcp_tw_reuse=1
net.ipv4.tcp_max_syn_backlog=8192
net.ipv4.tcp_keepalive_time=1800
net.ipv4.tcp_fin_timeout=30
net.core.rmem_max=16777216
net.core.wmem_max=16777216
net.ipv4.tcp_rmem=4096 87380 16777216
net.ipv4.tcp_wmem=4096 65536 16777216
net.core.netdev_max_backlog=3000


ipvs
ipvsadm -ln：
IP Virtual Server version 1.2.1 (size=4194304)


ipvs route:
[root@10.153.72.18 ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4194304)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 20.0.0.1 rr
-> 10.153.72.2:80 Route 100 0 0
-> 10.153.74.84:80 Route 100 0 0
-> 10.153.74.85:80 Route 100 0 0
-> 10.153.74.139:80 Route 100 0 0
-> 10.153.74.140:80 Route 100 0 0
-> 10.153.74.141:80 Route 100 0 0
-> 10.153.74.142:80 Route 100 0 0
-> 10.153.74.248:80 Route 100 0 0
-> 10.153.74.249:80 Route 100 0 0
-> 10.153.74.250:80 Route 100 0 0
-> 10.153.74.251:80 Route 100 0 0


ipvs config:
[root@10.153.72.18 ~]# cat /etc/keepalived/119.188.143.208-80.conf
virtual_server_group http {
20.0.0.1 80
}






virtual_server group http {
delay_loop 7
lb_algo rr
lb_kind DR
protocol TCP
//syn_proxy
laddr_group_name laddr_g1
alpha //open alpha module, automatic bind vip
omega //open omega module,automatci unbind vip
quorum 1
hysteresis 0


quorum_up "ip addr add 20.0.0.1/32 dev eth0;"
quorum_down "ip addr del 20.0.0.1/32 dev eth0;"


include 20.0.0.1-80~10.153.74.85-80.conf
include 20.0.0.1-80~10.153.74.84-80.conf
include 20.0.0.1-80~10.153.74.139-80.conf
include 20.0.0.1-80~10.153.74.140-80.conf
include 20.0.0.1-80~10.153.74.141-80.conf
include 20.0.0.1-80~10.153.74.142-80.conf
include 20.0.0.1-80~10.153.74.248-80.conf
include 20.0.0.1-80~10.153.74.249-80.conf
include 20.0.0.1-80~10.153.74.250-80.conf
include 20.0.0.1-80~10.153.74.251-80.conf
include 20.0.0.1-80~10.153.72.2-80.conf






}


for each realserver file, contents of file is same:
real_server 10.153.74.250 80 {
weight 100
inhibit_on_failure
HTTP_GET {
url {
path /index.html
status_code 200
}
connect_timeout 3
nb_get_retry 2
delay_before_retry 5
}
}







在 2014-07-09 11:10:20，"胡西宁" <huxining@yeah.net> 写道：
>hi everyone:
> I do test throughput of lvs server, the mode is dr. Num of lvs server is only one. test tool is jmeter, when throughput more than 20,000， It's not increase, I do not know what is the reason, I optimization params of kernal, include limits of open file, tcp connection. etc, thank you
>_______________________________________________
>Please read the documentation before posting - it's available at:
>http://www.linuxvirtualserver.org/
>
>LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
>Send requests to lvs-users-request@LinuxVirtualServer.org
>or go to http://lists.graemef.net/mailman/listinfo/lvs-users
_______________________________________________
Please read the documentation before posting - it's available at:
http://www.linuxvirtualserver.org/

LinuxVirtualServer.org mailing list - lvs-users@LinuxVirtualServer.org
Send requests to lvs-users-request@LinuxVirtualServer.org
or go to http://lists.graemef.net/mailman/listinfo/lvs-users