Mailing List Archive: varnish on flash drives

varnish on flash drives

Jan 19, 2008, 2:14 PM

Post #1 of 4 (200 views)

I'd like to run a reverse proxy on a machine with solid state drives. The
hope is to get more queries served by utilizing high random I/O performance
(the drive I have supports about 5K requests per second of average 4KB
objects).
I'll have 16GB of RAM, 128GB of flash, and tons of CPU on the machine. The
entire 128GB of cache will be active in general.

The question is whether I should run Varnish on this hardware. SSD random
write performance is very poor, and I don't understand how Varnish replaces
objects. LRU won't be friendly to SSD. FIFO would be good enough for my
purpose. Is it possible to configure it that way?

Also, when using the flash drive as a disk, would I be guaranteed to make
exactly one read on the flash per object I fetch from it (for objects that
don't reside in the memory)?

Thanks.
Rob
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://projects.linpro.no/pipermail/varnish-misc/attachments/20080119/b7a58a8c/attachment.htm

varnish on flash drives [ In reply to ]

fragfutter at gmail

Jan 19, 2008, 3:22 PM

Post #2 of 4 (195 views)

Permalink

On Jan 19, 2008 10:14 PM, Rob Tucker <rob.g.tucker at gmail.com> wrote:
> I'd like to run a reverse proxy on a machine with solid state drives. The
> hope is to get more queries served by utilizing high random I/O performance
> (the drive I have supports about 5K requests per second of average 4KB
> objects).
> I'll have 16GB of RAM, 128GB of flash, and tons of CPU on the machine. The
> entire 128GB of cache will be active in general.

Flash Drives are bad for systems with a high io-load (Flash Drives
have poor write Performance and a limited count of write access).
Get more Ram. You should be able to fit 32GB of ram in any
Serversystem for a reasonable Price. To get a good harddisk IO
Performance, get hardware raidcontroller an stripe a lot of small fast
scsi-disks.

Greetings
Christoph

varnish on flash drives [ In reply to ]

rob.g.tucker at gmail

Jan 19, 2008, 8:17 PM

Post #3 of 4 (197 views)

Permalink

The flash drive I have support 1M cycles so if I treat it as circular
buffer and keep writing to it, at my rate, it would last me more than
10 years.
So it is not a concern. So if I go ahead with this config, I wonder if
it would work out ok. i.e.
1) Can I choose a FIFO replacement policy that ensures writes happen
sequentially to the media.
2) Would randomly reading one object that is not in memory result in
more than 1 access on the underlying device. Objects I'm talking about
would rarely exceed 10KB.

Rob

>Flash Drives are bad for systems with a high io-load (Flash Drives
have poor write Performance and a limited count of write access).
> Get more Ram. You should be able to fit 32GB of ram in any Serversystem for a reasonable Price. To get a good harddisk IO Performance, get hardware raidcontroller an stripe a lot of small fast scsi-disks.

>Greetings
> Christoph

On Jan 19, 2008 1:14 PM, Rob Tucker <rob.g.tucker at gmail.com> wrote:
> I'd like to run a reverse proxy on a machine with solid state drives. The hope is to get more queries served by utilizing high random I/O performance (the drive I have supports about 5K requests per second of average 4KB objects).
> I'll have 16GB of RAM, 128GB of flash, and tons of CPU on the machine. The entire 128GB of cache will be active in general.
>
> The question is whether I should run Varnish on this hardware. SSD random write performance is very poor, and I don't understand how Varnish replaces objects. LRU won't be friendly to SSD. FIFO would be good enough for my purpose. Is it possible to configure it that way?
>
> Also, when using the flash drive as a disk, would I be guaranteed to make exactly one read on the flash per object I fetch from it (for objects that don't reside in the memory)?
>
> Thanks.
> Rob
>
>

varnish on flash drives [ In reply to ]

phk at phk

Jan 28, 2008, 5:29 AM

Post #4 of 4 (201 views)

Permalink

In message <f4b65cf70801191314q91a6b47of3a8d95d0b508838 at mail.gmail.com>, "Rob T
ucker" writes:

>I'd like to run a reverse proxy on a machine with solid state drives.

If you run it on the "real" kind of solid state drives, you can just do
it with no concerns. I've been recommended "MTRON" as a really good
and some of the fastest drives for this kind of purpose.

If you run it on "lets stick a SATA converter in front of a camera-flash"
drives, you'll suffer terrible write service time (0.1+ second) and wear
them out in no time.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.