Hey,
Sorry if this is the wrong place to be posting this, but I didn't see
any other lists that seemed to fit the question.
Basically, we are currently using varnish for some our photo caching,
and it's working perfectly. Here's the relevant portion of varnishstat
from a photo cache:
client_conn 4478824 49.86 Client connections accepted
client_req 10146386 112.96 Client requests received
cache_hit 9101426 101.32 Cache hits
cache_hitpass 27 0.00 Cache hits for pass
cache_miss 1044704 11.63 Cache misses
backend_conn 1044954 11.63 Backend connections success
backend_fail 0 0.00 Backend connections failures
backend_reuse 979195 10.90 Backend connections reuses
backend_recycle 985646 10.97 Backend connections recycles
backend_unused 39 0.00 Backend connections unused
We start varnishd up with this commandline:
/usr/local/sbin/varnishd -a :80 -f /etc/varnish/default.vcl -T
127.0.0.1:6082 -t 120 -w 1,1000,120 -s
file,/c01/varnish/varnish_storage.bin,40% -u varnish -g varnish -n
/var/varnish -P /var/run/varnish.pid
Varnish is working exactly as we expected with photos, and exceptionally
better than squid (which we also use). We have a problem when we use
varnish for serving audio and video files though. Using the same
commandline, varnish eats up all the memory in the system, and then
starts to eat into swap, which in turn makes the system unusable. During
our test, the load rose on the machine to ~950 (!), at which point we
shut it off and went back to using squid, where the load hovers around
~6.
The machine running squid is a dell pe1950, 8gb of ram, 1 xeon 2.33 ghz.
The pe1950 has 4 drives in the system formatted as a raid 10, and is
connected to a md1000 (external storage) consisting of 15 73gb 15k scsi
drives formatted in a raid 10+HS. This is formatted at /c01, which, as
you can see above, is where we keep the backend storage for varnish.
Unfortunately, I don't have the output from varnish on the audio/video
system. I'm just wondering if anyone has any suggestions on anything I
may be doing wrong. Is 40% (~190GB) too much for a backend storage? The
default.vcl file on the audio/video system is about as simple as you
get, with something added for TTL's. I know varnish is being used on
some pretty big sites out there, so I'm hoping people have some
pointers...
Thanks,
--
Andrew Knapp | systems engineer | imeem, inc. | 415.762.0135 o |
415.994.5752 m | 415.836.0088 f
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://projects.linpro.no/pipermail/varnish-misc/attachments/20070803/a94101fc/attachment.htm
Sorry if this is the wrong place to be posting this, but I didn't see
any other lists that seemed to fit the question.
Basically, we are currently using varnish for some our photo caching,
and it's working perfectly. Here's the relevant portion of varnishstat
from a photo cache:
client_conn 4478824 49.86 Client connections accepted
client_req 10146386 112.96 Client requests received
cache_hit 9101426 101.32 Cache hits
cache_hitpass 27 0.00 Cache hits for pass
cache_miss 1044704 11.63 Cache misses
backend_conn 1044954 11.63 Backend connections success
backend_fail 0 0.00 Backend connections failures
backend_reuse 979195 10.90 Backend connections reuses
backend_recycle 985646 10.97 Backend connections recycles
backend_unused 39 0.00 Backend connections unused
We start varnishd up with this commandline:
/usr/local/sbin/varnishd -a :80 -f /etc/varnish/default.vcl -T
127.0.0.1:6082 -t 120 -w 1,1000,120 -s
file,/c01/varnish/varnish_storage.bin,40% -u varnish -g varnish -n
/var/varnish -P /var/run/varnish.pid
Varnish is working exactly as we expected with photos, and exceptionally
better than squid (which we also use). We have a problem when we use
varnish for serving audio and video files though. Using the same
commandline, varnish eats up all the memory in the system, and then
starts to eat into swap, which in turn makes the system unusable. During
our test, the load rose on the machine to ~950 (!), at which point we
shut it off and went back to using squid, where the load hovers around
~6.
The machine running squid is a dell pe1950, 8gb of ram, 1 xeon 2.33 ghz.
The pe1950 has 4 drives in the system formatted as a raid 10, and is
connected to a md1000 (external storage) consisting of 15 73gb 15k scsi
drives formatted in a raid 10+HS. This is formatted at /c01, which, as
you can see above, is where we keep the backend storage for varnish.
Unfortunately, I don't have the output from varnish on the audio/video
system. I'm just wondering if anyone has any suggestions on anything I
may be doing wrong. Is 40% (~190GB) too much for a backend storage? The
default.vcl file on the audio/video system is about as simple as you
get, with something added for TTL's. I know varnish is being used on
some pretty big sites out there, so I'm hoping people have some
pointers...
Thanks,
--
Andrew Knapp | systems engineer | imeem, inc. | 415.762.0135 o |
415.994.5752 m | 415.836.0088 f
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://projects.linpro.no/pipermail/varnish-misc/attachments/20070803/a94101fc/attachment.htm