Hi!
As this is my first post to this list I would like to thank to all developers of this excellent software.
We put our images behind varnish-1.0.4 serving ~1.5k req/sec and load is staying low which is great.
Though when reaching 2k req/sec we start to encounter connection closures. Connection gets closed immediately after connection is established.
I have raised thread limits but that did not help. I would look into logs if this happened for another application but I am not really sure where to look for varnish.
my configuration:
05:56:16 janis[pts/2]@host ~$ telnet localhost 81
Trying 127.0.0.1.81...
Connected to localhost.
Escape character is '^]'.
param.show
200 709
default_ttl 120 [seconds]
thread_pools 5 [pools]
thread_pool_max 100000 [threads]
thread_pool_min 100 [threads]
thread_pool_timeout 10 [seconds]
overflow_max 100 [%]
http_workspace 8192 [bytes]
sess_timeout 5 [seconds]
pipe_timeout 60 [seconds]
send_timeout 600 [seconds]
auto_restart on [bool]
fetch_chunksize 128 [kilobytes]
sendfile_threshold unlimited [bytes]
vcl_trace off [bool]
listen_address :80
listen_depth 4096 [connections]
srcaddr_hash 1049 [buckets]
srcaddr_ttl 30 [seconds]
backend_http11 off [bool]
client_http11 off [bool]
ping_interval 3 [seconds]
varnishstat:
00:01:22
Hitrate ratio: 10 19 19
Hitrate avg: 1.0059 1.0096 1.0096
82740 1016.84 1009.02 Client connections accepted
182469 2259.17 2225.23 Client requests received
180928 2256.78 2206.44 Cache hits
0 0.00 0.00 Cache hits for pass
1541 2.40 18.79 Cache misses
1541 2.40 18.79 Backend connections success
2 0.00 0.02 Backend connections failures
1507 2.40 18.38 Backend connections reuses
1511 2.40 18.43 Backend connections recycles
3 0.00 0.04 Backend connections unused
1614 6.30 19.68 N struct srcaddr
311 -3.70 3.79 N active struct srcaddr
906 0.00 11.05 N struct sess_mem
875 -1.00 10.67 N struct sess
1604 2.60 19.56 N struct object
1604 2.60 19.56 N struct objecthead
1523 2.40 18.57 N struct smf
0 0.00 0.00 N small free smf
2 0.00 0.02 N large free smf
6 0.00 0.07 N struct vbe_conn
63 0.20 0.77 N worker threads
63 0.20 0.77 N worker threads created
0 0.00 0.00 N worker threads not created
0 0.00 0.00 N worker threads limited
0 0.00 0.00 N queued work requests
63 0.20 0.77 N overflowed work requests
0 0.00 0.00 N dropped work requests
0 0.00 0.00 N expired objects
0 0.00 0.00 N objects on deathrow
0 0.00 0.00 HTTP header overflows
0 0.00 0.00 Objects sent with sendfile
95445 1159.97 1163.96 Objects sent with write
77391 928.18 943.79 Total Sessions
182469 2259.37 2225.23 Total Requests
0 0.00 0.00 Total pipe
0 0.00 0.00 Total pass
1541 2.40 18.79 Total fetch
38575489 476006.89 470432.79 Total header bytes
7861511491 91999201.08 95872091.35 Total body bytes
61101 712.18 745.13 Session Closed
272 5.80 3.32 Session Pipeline
137 3.00 1.67 Session Read Ahead
121024 1539.50 1475.90 Session herd
5833297 71817.11 71137.77 SHM records
425680 5241.62 5191.22 SHM writes
128 2.70 1.56 SHM MTX contention
netstat summary:
FIN_WAIT1: 93
TIME_WAIT: 6825
CLOSE_WAIT: 5
LAST_ACK: 7
FIN_WAIT2: 462
ESTABLISHED: 751
SYN_RECV: 112
LISTEN: 7
load:
06:05:55 root[pts/1]@host ~# uptime
06:05:58 up 52 days, 20:27, 4 users, load average: 0.55, 0.39, 0.31
process info:
06:19:35 root[pts/1]@host ~/install# ps auxf|egrep [v]arnish
root 25226 0.0 0.0 10020 740 ? Ss 05:33 0:00 varnishd -f /etc/varnishd/varnishd.conf -T 127.0.0.1:81 -P /var/run/varnishd.pid -s file,/var/spool/varnishd/varnishd_storage.bin,1574176153 -p thread_pools 5 -p listen_depth 4096 -w100,100000,10
root 26973 14.0 0.6 2282680 56608 ? Sl 06:03 2:13 \_ varnishd -f /etc/varnishd/varnishd.conf -T 127.0.0.1:81 -P /var/run/varnishd.pid -s file,/var/spool/varnishd/varnishd_storage.bin,1574176153 -p thread_pools 5 -p listen_depth 4096 -w100,100000,10
06:19:42 root[pts/1]@host ~/install# uname -a
Linux host 2.6.16.52-1smp #1 SMP Thu May 31 19:32:54 CEST 2007 i686 Intel(R)_Xeon(R)_CPU____________5150__ at _2.66GHz Linux
06:23:04 root[pts/1]@host ~/install# free
total used free shared buffers cached
Mem: 8114 721 7392 0 306 314
-/+ buffers/cache: 101 8013
Swap: 9765 0 9765
connection closure:
06:05:58 root[pts/1]@host ~# telnet host.mydomain.lv 80
Trying 11.11.11.11.80...
Connected to 11.11.11.11.
Escape character is '^]'.
Connection closed by foreign host.
Also in varnishstat Client counters gets reset from time to time. Don't know if it is problem or not.
Thank you in advance.
--
Janis Putrams
As this is my first post to this list I would like to thank to all developers of this excellent software.
We put our images behind varnish-1.0.4 serving ~1.5k req/sec and load is staying low which is great.
Though when reaching 2k req/sec we start to encounter connection closures. Connection gets closed immediately after connection is established.
I have raised thread limits but that did not help. I would look into logs if this happened for another application but I am not really sure where to look for varnish.
my configuration:
05:56:16 janis[pts/2]@host ~$ telnet localhost 81
Trying 127.0.0.1.81...
Connected to localhost.
Escape character is '^]'.
param.show
200 709
default_ttl 120 [seconds]
thread_pools 5 [pools]
thread_pool_max 100000 [threads]
thread_pool_min 100 [threads]
thread_pool_timeout 10 [seconds]
overflow_max 100 [%]
http_workspace 8192 [bytes]
sess_timeout 5 [seconds]
pipe_timeout 60 [seconds]
send_timeout 600 [seconds]
auto_restart on [bool]
fetch_chunksize 128 [kilobytes]
sendfile_threshold unlimited [bytes]
vcl_trace off [bool]
listen_address :80
listen_depth 4096 [connections]
srcaddr_hash 1049 [buckets]
srcaddr_ttl 30 [seconds]
backend_http11 off [bool]
client_http11 off [bool]
ping_interval 3 [seconds]
varnishstat:
00:01:22
Hitrate ratio: 10 19 19
Hitrate avg: 1.0059 1.0096 1.0096
82740 1016.84 1009.02 Client connections accepted
182469 2259.17 2225.23 Client requests received
180928 2256.78 2206.44 Cache hits
0 0.00 0.00 Cache hits for pass
1541 2.40 18.79 Cache misses
1541 2.40 18.79 Backend connections success
2 0.00 0.02 Backend connections failures
1507 2.40 18.38 Backend connections reuses
1511 2.40 18.43 Backend connections recycles
3 0.00 0.04 Backend connections unused
1614 6.30 19.68 N struct srcaddr
311 -3.70 3.79 N active struct srcaddr
906 0.00 11.05 N struct sess_mem
875 -1.00 10.67 N struct sess
1604 2.60 19.56 N struct object
1604 2.60 19.56 N struct objecthead
1523 2.40 18.57 N struct smf
0 0.00 0.00 N small free smf
2 0.00 0.02 N large free smf
6 0.00 0.07 N struct vbe_conn
63 0.20 0.77 N worker threads
63 0.20 0.77 N worker threads created
0 0.00 0.00 N worker threads not created
0 0.00 0.00 N worker threads limited
0 0.00 0.00 N queued work requests
63 0.20 0.77 N overflowed work requests
0 0.00 0.00 N dropped work requests
0 0.00 0.00 N expired objects
0 0.00 0.00 N objects on deathrow
0 0.00 0.00 HTTP header overflows
0 0.00 0.00 Objects sent with sendfile
95445 1159.97 1163.96 Objects sent with write
77391 928.18 943.79 Total Sessions
182469 2259.37 2225.23 Total Requests
0 0.00 0.00 Total pipe
0 0.00 0.00 Total pass
1541 2.40 18.79 Total fetch
38575489 476006.89 470432.79 Total header bytes
7861511491 91999201.08 95872091.35 Total body bytes
61101 712.18 745.13 Session Closed
272 5.80 3.32 Session Pipeline
137 3.00 1.67 Session Read Ahead
121024 1539.50 1475.90 Session herd
5833297 71817.11 71137.77 SHM records
425680 5241.62 5191.22 SHM writes
128 2.70 1.56 SHM MTX contention
netstat summary:
FIN_WAIT1: 93
TIME_WAIT: 6825
CLOSE_WAIT: 5
LAST_ACK: 7
FIN_WAIT2: 462
ESTABLISHED: 751
SYN_RECV: 112
LISTEN: 7
load:
06:05:55 root[pts/1]@host ~# uptime
06:05:58 up 52 days, 20:27, 4 users, load average: 0.55, 0.39, 0.31
process info:
06:19:35 root[pts/1]@host ~/install# ps auxf|egrep [v]arnish
root 25226 0.0 0.0 10020 740 ? Ss 05:33 0:00 varnishd -f /etc/varnishd/varnishd.conf -T 127.0.0.1:81 -P /var/run/varnishd.pid -s file,/var/spool/varnishd/varnishd_storage.bin,1574176153 -p thread_pools 5 -p listen_depth 4096 -w100,100000,10
root 26973 14.0 0.6 2282680 56608 ? Sl 06:03 2:13 \_ varnishd -f /etc/varnishd/varnishd.conf -T 127.0.0.1:81 -P /var/run/varnishd.pid -s file,/var/spool/varnishd/varnishd_storage.bin,1574176153 -p thread_pools 5 -p listen_depth 4096 -w100,100000,10
06:19:42 root[pts/1]@host ~/install# uname -a
Linux host 2.6.16.52-1smp #1 SMP Thu May 31 19:32:54 CEST 2007 i686 Intel(R)_Xeon(R)_CPU____________5150__ at _2.66GHz Linux
06:23:04 root[pts/1]@host ~/install# free
total used free shared buffers cached
Mem: 8114 721 7392 0 306 314
-/+ buffers/cache: 101 8013
Swap: 9765 0 9765
connection closure:
06:05:58 root[pts/1]@host ~# telnet host.mydomain.lv 80
Trying 11.11.11.11.80...
Connected to 11.11.11.11.
Escape character is '^]'.
Connection closed by foreign host.
Also in varnishstat Client counters gets reset from time to time. Don't know if it is problem or not.
Thank you in advance.
--
Janis Putrams