Mailing List Archive: varnish consuming too much memory?

varnish consuming too much memory?

jodok at lovelysystems

Mar 12, 2008, 11:26 AM

Post #1 of 24 (1806 views)

hi,

i'm wondering how varnish determines how much memory it should
consume...
we're running multiple varnishes on one box and now and then one
instance starts to use more and more memory.
right now it's using 65% out of 16GB ram, which is pretty much :)

varnishlog shows that pretty much traffic is going on :)
varnishhist shows that the requests are beeing served really fast
varnishtop shows: the following:

list length
569
hu

4537.38 VCL_retur deliver
2655.35 TxProtoco HTTP/1.1
2397.58 RxProtoco HTTP/1.0
2397.58 RxHeader Host: www.zoomer.de
2397.58 RxHeader Connection: close
2397.58 VCL_call recv
2397.58 VCL_retur lookup
2397.58 VCL_call hash
2397.58 VCL_retur hash
2397.58 VCL_call deliver
2397.58 TxHeader Via: 1.1 varnish
2397.58 TxHeader Connection: close
2397.58 SessionCl Connection: close
2378.81 RxRequest GET
2261.07 TxHeader Server: nginx/0.5.35
2261.07 TxHeader Keep-Alive: timeout=20
2252.21 TxStatus 200
2245.30 TxHeader X-Powered-By: Zope (www.zope.org), Python (www.python.org
)
2236.44 TxRespons Ok
2139.80 VCL_call hit
1450.22 RxHeader Accept: */*
1028.78 TxHeader Content-Type: text/html;charset=utf-8
826.47 RxHeader Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
820.48 RxHeader Accept-Encoding: gzip,deflate
796.95 RxHeader User-Agent: Wget/1.10.2
796.95 RxHeader Authorization: Basic aHVtYm9sZHQ6QWxleDA3
737.62 RxHeader Accept-Encoding: gzip, deflate
720.28 RxHeader Accept-Language: de-de,de;q=0.8,en-
us;q=0.5,en;q=0.3
685.15 RxHeader Cookie: uid=5BbkCkfYHlkLGChJBTgCAg==
614.14 RxHeader User-Agent: Mozilla/5.0 (Windows; U; Windows NT
5.1; de; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.1
585.94 RxHeader Accept: text/xml,application/xml,application/xhtml
+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/
484.43 TxHeader Content-Type: text/xml;charset=utf-8
480.47 RxURL /xml/generic/humboldt/top1.xml
480.47 Hit 350226425
480.47 Length 1555
480.47 TxHeader Last-Modified: Wed, 12 Mar 2008 18:16:06 GMT

any idea?

jodok

--
"Beautiful is better than ugly."
-- The Zen of Python, by Tim Peters

Jodok Batlogg, Lovely Systems GmbH
Schmelzh?tterstra?e 26a, 6850 Dornbirn, Austria
mobile: +43 676 5683591, phone: +43 5572 908060

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2454 bytes
Desc: not available
Url : http://projects.linpro.no/pipermail/varnish-dev/attachments/20080312/d1822930/attachment.bin

varnish consuming too much memory? [ In reply to ]

jodok at lovelysystems

Mar 12, 2008, 11:26 AM

Post #2 of 24 (1788 views)

hi,

i'm wondering how varnish determines how much memory it should
consume...
we're running multiple varnishes on one box and now and then one
instance starts to use more and more memory.
right now it's using 65% out of 16GB ram, which is pretty much :)

varnishlog shows that pretty much traffic is going on :)
varnishhist shows that the requests are beeing served really fast
varnishtop shows: the following:

list length
569
hu

4537.38 VCL_retur deliver
2655.35 TxProtoco HTTP/1.1
2397.58 RxProtoco HTTP/1.0
2397.58 RxHeader Host: www.zoomer.de
2397.58 RxHeader Connection: close
2397.58 VCL_call recv
2397.58 VCL_retur lookup
2397.58 VCL_call hash
2397.58 VCL_retur hash
2397.58 VCL_call deliver
2397.58 TxHeader Via: 1.1 varnish
2397.58 TxHeader Connection: close
2397.58 SessionCl Connection: close
2378.81 RxRequest GET
2261.07 TxHeader Server: nginx/0.5.35
2261.07 TxHeader Keep-Alive: timeout=20
2252.21 TxStatus 200
2245.30 TxHeader X-Powered-By: Zope (www.zope.org), Python (www.python.org
)
2236.44 TxRespons Ok
2139.80 VCL_call hit
1450.22 RxHeader Accept: */*
1028.78 TxHeader Content-Type: text/html;charset=utf-8
826.47 RxHeader Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
820.48 RxHeader Accept-Encoding: gzip,deflate
796.95 RxHeader User-Agent: Wget/1.10.2
796.95 RxHeader Authorization: Basic aHVtYm9sZHQ6QWxleDA3
737.62 RxHeader Accept-Encoding: gzip, deflate
720.28 RxHeader Accept-Language: de-de,de;q=0.8,en-
us;q=0.5,en;q=0.3
685.15 RxHeader Cookie: uid=5BbkCkfYHlkLGChJBTgCAg==
614.14 RxHeader User-Agent: Mozilla/5.0 (Windows; U; Windows NT
5.1; de; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.1
585.94 RxHeader Accept: text/xml,application/xml,application/xhtml
+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/
484.43 TxHeader Content-Type: text/xml;charset=utf-8
480.47 RxURL /xml/generic/humboldt/top1.xml
480.47 Hit 350226425
480.47 Length 1555
480.47 TxHeader Last-Modified: Wed, 12 Mar 2008 18:16:06 GMT

any idea?

jodok

--
"Beautiful is better than ugly."
-- The Zen of Python, by Tim Peters

Jodok Batlogg, Lovely Systems GmbH
Schmelzh?tterstra?e 26a, 6850 Dornbirn, Austria
mobile: +43 676 5683591, phone: +43 5572 908060

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2454 bytes
Desc: not available
Url : http://projects.linpro.no/pipermail/varnish-dev/attachments/20080312/d1822930/attachment-0001.bin

varnish consuming too much memory? [ In reply to ]

Mar 12, 2008, 11:59 AM

Post #3 of 24 (1795 views)

>i'm wondering how varnish determines how much memory it should
>consume...

It uses all memory it can for caching.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

varnish consuming too much memory? [ In reply to ]

Mar 12, 2008, 11:59 AM

Post #4 of 24 (1789 views)

>i'm wondering how varnish determines how much memory it should
>consume...

It uses all memory it can for caching.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

varnish consuming too much memory? [ In reply to ]

jodok at lovelysystems

Mar 12, 2008, 3:03 PM

Post #5 of 24 (1792 views)

On 12.03.2008, at 19:59, Poul-Henning Kamp wrote:

>
>> i'm wondering how varnish determines how much memory it should
>> consume...
>
> It uses all memory it can for caching.

- is there a way to limit it?
i'd like to assign the 4 varnishes running on this box 3 GB each and
leave 4GB for linux disk/nfs cache

- what effect has the size of the VARNISH_STORAGE_SIZE?
is this cache persistent? when are objects moved from memory to disk?

thanks

jodok

>
>
> --
> Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG | TCP/IP since RFC 956
> FreeBSD committer | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by
> incompetence.

--
"Beautiful is better than ugly."
-- The Zen of Python, by Tim Peters

Jodok Batlogg, Lovely Systems GmbH
Schmelzh?tterstra?e 26a, 6850 Dornbirn, Austria
mobile: +43 676 5683591, phone: +43 5572 908060

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2454 bytes
Desc: not available
Url : http://projects.linpro.no/pipermail/varnish-dev/attachments/20080312/2339e655/attachment.bin

varnish consuming too much memory? [ In reply to ]

jodok at lovelysystems

Mar 12, 2008, 3:03 PM

Post #6 of 24 (1794 views)

On 12.03.2008, at 19:59, Poul-Henning Kamp wrote:

>
>> i'm wondering how varnish determines how much memory it should
>> consume...
>
> It uses all memory it can for caching.

- is there a way to limit it?
i'd like to assign the 4 varnishes running on this box 3 GB each and
leave 4GB for linux disk/nfs cache

- what effect has the size of the VARNISH_STORAGE_SIZE?
is this cache persistent? when are objects moved from memory to disk?

thanks

jodok

>
>
> --
> Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG | TCP/IP since RFC 956
> FreeBSD committer | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by
> incompetence.

--
"Beautiful is better than ugly."
-- The Zen of Python, by Tim Peters

Jodok Batlogg, Lovely Systems GmbH
Schmelzh?tterstra?e 26a, 6850 Dornbirn, Austria
mobile: +43 676 5683591, phone: +43 5572 908060

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2454 bytes
Desc: not available
Url : http://projects.linpro.no/pipermail/varnish-dev/attachments/20080312/2339e655/attachment-0001.bin

varnish consuming too much memory? [ In reply to ]

Mar 13, 2008, 2:56 AM

Post #7 of 24 (1790 views)

In message <33CD53F7-74C0-447C-8E0C-8F40365B7D7D at lovelysystems.com>, Jodok Batl
ogg writes:

>>> i'm wondering how varnish determines how much memory it should
>>> consume...
>>
>> It uses all memory it can for caching.
>
>- is there a way to limit it?

no.

>i'd like to assign the 4 varnishes running on this box 3 GB each and
>leave 4GB for linux disk/nfs cache

Why are you running multiple varnishes on the same box ?

>- what effect has the size of the VARNISH_STORAGE_SIZE?

That sets the size of the file varnish maps for storage, and in a sort
of way limits the amount of memory varnish can use, but also severely
limits your hit-rate when it runs full.

>is this cache persistent? when are objects moved from memory to disk?

No.

Whenever the kernel VM subsystem decide it should happen.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

varnish consuming too much memory? [ In reply to ]

Mar 13, 2008, 2:56 AM

Post #8 of 24 (1792 views)

In message <33CD53F7-74C0-447C-8E0C-8F40365B7D7D at lovelysystems.com>, Jodok Batl
ogg writes:

>>> i'm wondering how varnish determines how much memory it should
>>> consume...
>>
>> It uses all memory it can for caching.
>
>- is there a way to limit it?

no.

>i'd like to assign the 4 varnishes running on this box 3 GB each and
>leave 4GB for linux disk/nfs cache

Why are you running multiple varnishes on the same box ?

>- what effect has the size of the VARNISH_STORAGE_SIZE?

That sets the size of the file varnish maps for storage, and in a sort
of way limits the amount of memory varnish can use, but also severely
limits your hit-rate when it runs full.

>is this cache persistent? when are objects moved from memory to disk?

No.

Whenever the kernel VM subsystem decide it should happen.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

varnish consuming too much memory? [ In reply to ]

jodok at lovelysystems

Mar 13, 2008, 3:27 AM

Post #9 of 24 (1786 views)

On 13.03.2008, at 10:56, Poul-Henning Kamp wrote:

> In message <33CD53F7-74C0-447C-8E0C-8F40365B7D7D at lovelysystems.com>,
> Jodok Batl
> ogg writes:
>
>>>> i'm wondering how varnish determines how much memory it should
>>>> consume...
>>>
>>> It uses all memory it can for caching.
>>
>> - is there a way to limit it?
>
> no.
>
>> i'd like to assign the 4 varnishes running on this box 3 GB each and
>> leave 4GB for linux disk/nfs cache
>
> Why are you running multiple varnishes on the same box ?

we're hosting multiple sites on our cluster. the cluster has two load
balanced reverse proxies that handle delivery of static content
(nginx) and caching (varnish).
the different sites are running under different users. i'd like to
share the resources "fair" between them. the sites have different
rules for content expiry, allowed purge hosts,...

some of the static files are beeing delivered from NFS, with netfs
we're using linux cache memory to cache the files as well.
right now (one) varnish eats up all memory, leaves no memory for
caching and the system starts swapping.

>> - what effect has the size of the VARNISH_STORAGE_SIZE?
>
> That sets the size of the file varnish maps for storage, and in a sort
> of way limits the amount of memory varnish can use, but also severely
> limits your hit-rate when it runs full.

can you elaborate or point me to some documentation?

>> is this cache persistent? when are objects moved from memory to disk?
>
> No.
>
> Whenever the kernel VM subsystem decide it should happen.

so it is a kind of home-made swap memory for varnish?

thanks

jodok

>
>
> --
> Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG | TCP/IP since RFC 956
> FreeBSD committer | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by
> incompetence.

--
"Beautiful is better than ugly."
-- The Zen of Python, by Tim Peters

Jodok Batlogg, Lovely Systems GmbH
Schmelzh?tterstra?e 26a, 6850 Dornbirn, Austria
mobile: +43 676 5683591, phone: +43 5572 908060

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2454 bytes
Desc: not available
Url : http://projects.linpro.no/pipermail/varnish-dev/attachments/20080313/f1cfc89c/attachment.bin

varnish consuming too much memory? [ In reply to ]

jodok at lovelysystems

Mar 13, 2008, 3:27 AM

Post #10 of 24 (1790 views)

On 13.03.2008, at 10:56, Poul-Henning Kamp wrote:

> In message <33CD53F7-74C0-447C-8E0C-8F40365B7D7D at lovelysystems.com>,
> Jodok Batl
> ogg writes:
>
>>>> i'm wondering how varnish determines how much memory it should
>>>> consume...
>>>
>>> It uses all memory it can for caching.
>>
>> - is there a way to limit it?
>
> no.
>
>> i'd like to assign the 4 varnishes running on this box 3 GB each and
>> leave 4GB for linux disk/nfs cache
>
> Why are you running multiple varnishes on the same box ?

we're hosting multiple sites on our cluster. the cluster has two load
balanced reverse proxies that handle delivery of static content
(nginx) and caching (varnish).
the different sites are running under different users. i'd like to
share the resources "fair" between them. the sites have different
rules for content expiry, allowed purge hosts,...

some of the static files are beeing delivered from NFS, with netfs
we're using linux cache memory to cache the files as well.
right now (one) varnish eats up all memory, leaves no memory for
caching and the system starts swapping.

>> - what effect has the size of the VARNISH_STORAGE_SIZE?
>
> That sets the size of the file varnish maps for storage, and in a sort
> of way limits the amount of memory varnish can use, but also severely
> limits your hit-rate when it runs full.

can you elaborate or point me to some documentation?

>> is this cache persistent? when are objects moved from memory to disk?
>
> No.
>
> Whenever the kernel VM subsystem decide it should happen.

so it is a kind of home-made swap memory for varnish?

thanks

jodok

>
>
> --
> Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG | TCP/IP since RFC 956
> FreeBSD committer | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by
> incompetence.

--
"Beautiful is better than ugly."
-- The Zen of Python, by Tim Peters

Jodok Batlogg, Lovely Systems GmbH
Schmelzh?tterstra?e 26a, 6850 Dornbirn, Austria
mobile: +43 676 5683591, phone: +43 5572 908060

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2454 bytes
Desc: not available
Url : http://projects.linpro.no/pipermail/varnish-dev/attachments/20080313/f1cfc89c/attachment-0001.bin

varnish consuming too much memory? [ In reply to ]

Mar 13, 2008, 3:47 AM

Post #11 of 24 (1788 views)

Hi Jodok

Please take a look at

http://varnish.projects.linpro.no/wiki/ArchitectNotes

It should give you a pretty clear idea about the design choices that
the varnish team have made.

Cheers

Dave

On 13/03/2008, at 9:27 PM, Jodok Batlogg wrote:

> can you elaborate or point me to some documentation?
>
>>> is this cache persistent? when are objects moved from memory to
>>> disk?
>>
>> No.
>>
>> Whenever the kernel VM subsystem decide it should happen.
>
> so it is a kind of home-made swap memory for varnish?
>
> thanks
>
> jodok

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://projects.linpro.no/pipermail/varnish-dev/attachments/20080313/b3571f8b/attachment.htm

varnish consuming too much memory? [ In reply to ]

Mar 13, 2008, 3:47 AM

Post #12 of 24 (1787 views)

Hi Jodok

Please take a look at

http://varnish.projects.linpro.no/wiki/ArchitectNotes

It should give you a pretty clear idea about the design choices that
the varnish team have made.

Cheers

Dave

On 13/03/2008, at 9:27 PM, Jodok Batlogg wrote:

> can you elaborate or point me to some documentation?
>
>>> is this cache persistent? when are objects moved from memory to
>>> disk?
>>
>> No.
>>
>> Whenever the kernel VM subsystem decide it should happen.
>
> so it is a kind of home-made swap memory for varnish?
>
> thanks
>
> jodok

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://projects.linpro.no/pipermail/varnish-dev/attachments/20080313/b3571f8b/attachment-0001.htm

varnish consuming too much memory? [ In reply to ]

ask at develooper

Mar 13, 2008, 7:43 AM

Post #13 of 24 (1791 views)

On Mar 13, 2008, at 3:27, Jodok Batlogg wrote:

> the different sites are running under different users. i'd like to
> share the resources "fair" between them.

If your platform supports virtualization then you can split up your
system into N virtual boxes to limit how much real memory varnish uses
for each...

- ask

--
http://develooper.com/ - http://askask.com/

varnish consuming too much memory? [ In reply to ]

ask at develooper

Mar 13, 2008, 7:43 AM

Post #14 of 24 (1793 views)

On Mar 13, 2008, at 3:27, Jodok Batlogg wrote:

> the different sites are running under different users. i'd like to
> share the resources "fair" between them.

If your platform supports virtualization then you can split up your
system into N virtual boxes to limit how much real memory varnish uses
for each...

- ask

--
http://develooper.com/ - http://askask.com/

varnish consuming too much memory? [ In reply to ]

jodok at lovelysystems

Mar 13, 2008, 2:35 PM

Post #15 of 24 (1786 views)

On 13.03.2008, at 11:47, Dave Cheney wrote:

> Hi Jodok
>
> Please take a look at
>
> http://varnish.projects.linpro.no/wiki/ArchitectNotes
>
> It should give you a pretty clear idea about the design choices
> that the varnish team have made.

i've read that document beforehand - but i don't get the point why it
should be impossible to limit system resources?

jodok

>
> Cheers
>
> Dave
>
> On 13/03/2008, at 9:27 PM, Jodok Batlogg wrote:
>
>> can you elaborate or point me to some documentation?
>>
>>>> is this cache persistent? when are objects moved from memory to
>>>> disk?
>>>
>>> No.
>>>
>>> Whenever the kernel VM subsystem decide it should happen.
>>
>> so it is a kind of home-made swap memory for varnish?
>>
>> thanks
>>
>> jodok
>

--
"Beautiful is better than ugly."
-- The Zen of Python, by Tim Peters

Jodok Batlogg, Lovely Systems GmbH
Schmelzh?tterstra?e 26a, 6850 Dornbirn, Austria
mobile: +43 676 5683591, phone: +43 5572 908060

varnish consuming too much memory? [ In reply to ]

jodok at lovelysystems

Mar 13, 2008, 2:35 PM

Post #16 of 24 (1788 views)

On 13.03.2008, at 11:47, Dave Cheney wrote:

> Hi Jodok
>
> Please take a look at
>
> http://varnish.projects.linpro.no/wiki/ArchitectNotes
>
> It should give you a pretty clear idea about the design choices
> that the varnish team have made.

i've read that document beforehand - but i don't get the point why it
should be impossible to limit system resources?

jodok

>
> Cheers
>
> Dave
>
> On 13/03/2008, at 9:27 PM, Jodok Batlogg wrote:
>
>> can you elaborate or point me to some documentation?
>>
>>>> is this cache persistent? when are objects moved from memory to
>>>> disk?
>>>
>>> No.
>>>
>>> Whenever the kernel VM subsystem decide it should happen.
>>
>> so it is a kind of home-made swap memory for varnish?
>>
>> thanks
>>
>> jodok
>

--
"Beautiful is better than ugly."
-- The Zen of Python, by Tim Peters

Jodok Batlogg, Lovely Systems GmbH
Schmelzh?tterstra?e 26a, 6850 Dornbirn, Austria
mobile: +43 676 5683591, phone: +43 5572 908060

varnish consuming too much memory? [ In reply to ]

jodok at lovelysystems

Mar 13, 2008, 2:38 PM

Post #17 of 24 (1788 views)

On 13.03.2008, at 15:43, Ask Bj?rn Hansen wrote:

>
> On Mar 13, 2008, at 3:27, Jodok Batlogg wrote:
>
>> the different sites are running under different users. i'd like to
>> share the resources "fair" between them.
>
> If your platform supports virtualization then you can split up your
> system into N virtual boxes to limit how much real memory varnish
> uses for each...

well - if you're serving 500mbit/s you don't want a virtualized
system...
probably i really need to create a more complicated vcl that allows to
run all applications within the same cache space.

but still it's open how it is possible to reserve some GB of RAM for
cache memory...

or is it really true that i need dedicated boxes for caching only?

jodok

>
>
>
>
> - ask
>
> --
> http://develooper.com/ - http://askask.com/
>
>

--
"Beautiful is better than ugly."
-- The Zen of Python, by Tim Peters

Jodok Batlogg, Lovely Systems GmbH
Schmelzh?tterstra?e 26a, 6850 Dornbirn, Austria
mobile: +43 676 5683591, phone: +43 5572 908060

varnish consuming too much memory? [ In reply to ]

jodok at lovelysystems

Mar 13, 2008, 2:38 PM

Post #18 of 24 (1787 views)

On 13.03.2008, at 15:43, Ask Bj?rn Hansen wrote:

>
> On Mar 13, 2008, at 3:27, Jodok Batlogg wrote:
>
>> the different sites are running under different users. i'd like to
>> share the resources "fair" between them.
>
> If your platform supports virtualization then you can split up your
> system into N virtual boxes to limit how much real memory varnish
> uses for each...

well - if you're serving 500mbit/s you don't want a virtualized
system...
probably i really need to create a more complicated vcl that allows to
run all applications within the same cache space.

but still it's open how it is possible to reserve some GB of RAM for
cache memory...

or is it really true that i need dedicated boxes for caching only?

jodok

>
>
>
>
> - ask
>
> --
> http://develooper.com/ - http://askask.com/
>
>

--
"Beautiful is better than ugly."
-- The Zen of Python, by Tim Peters

Jodok Batlogg, Lovely Systems GmbH
Schmelzh?tterstra?e 26a, 6850 Dornbirn, Austria
mobile: +43 676 5683591, phone: +43 5572 908060

varnish consuming too much memory? [ In reply to ]

Mar 13, 2008, 2:45 PM

Post #19 of 24 (1788 views)

In message <7FC400FB-7BB0-4979-81DE-7EE538AED27A at lovelysystems.com>, Jodok Batl
ogg writes:

>or is it really true that i need dedicated boxes for caching only?

If your operating system has a VM system worth its salt, no.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

varnish consuming too much memory? [ In reply to ]

Mar 13, 2008, 2:45 PM

Post #20 of 24 (1785 views)

In message <7FC400FB-7BB0-4979-81DE-7EE538AED27A at lovelysystems.com>, Jodok Batl
ogg writes:

>or is it really true that i need dedicated boxes for caching only?

If your operating system has a VM system worth its salt, no.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

varnish consuming too much memory? [ In reply to ]

jodok at lovelysystems

Mar 13, 2008, 2:50 PM

Post #21 of 24 (1795 views)

On 13.03.2008, at 22:45, Poul-Henning Kamp wrote:

> In message <7FC400FB-7BB0-4979-81DE-7EE538AED27A at lovelysystems.com>,
> Jodok Batl
> ogg writes:
>
>> or is it really true that i need dedicated boxes for caching only?
>
> If your operating system has a VM system worth its salt, no.

so you mean - in case the box is busy accessing objects from cache
memory and busy accessing some items from varnish cache and another
application is busy and accesses some other objects from memory
frequently my 2.6 linux kernel is intelligent enough to decide what to
keep in memory and what to put to swap.
that would be great :)

what is varnish's disk file exactly for? is varnish using "real"
memory only and "swapping" to its own file?

thanks

jodok

>
>
>
> --
> Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG | TCP/IP since RFC 956
> FreeBSD committer | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by
> incompetence.

--
"Beautiful is better than ugly."
-- The Zen of Python, by Tim Peters

Jodok Batlogg, Lovely Systems GmbH
Schmelzh?tterstra?e 26a, 6850 Dornbirn, Austria
mobile: +43 676 5683591, phone: +43 5572 908060

varnish consuming too much memory? [ In reply to ]

jodok at lovelysystems

Mar 13, 2008, 2:50 PM

Post #22 of 24 (1786 views)

On 13.03.2008, at 22:45, Poul-Henning Kamp wrote:

> In message <7FC400FB-7BB0-4979-81DE-7EE538AED27A at lovelysystems.com>,
> Jodok Batl
> ogg writes:
>
>> or is it really true that i need dedicated boxes for caching only?
>
> If your operating system has a VM system worth its salt, no.

so you mean - in case the box is busy accessing objects from cache
memory and busy accessing some items from varnish cache and another
application is busy and accesses some other objects from memory
frequently my 2.6 linux kernel is intelligent enough to decide what to
keep in memory and what to put to swap.
that would be great :)

what is varnish's disk file exactly for? is varnish using "real"
memory only and "swapping" to its own file?

thanks

jodok

>
>
>
> --
> Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG | TCP/IP since RFC 956
> FreeBSD committer | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by
> incompetence.

--
"Beautiful is better than ugly."
-- The Zen of Python, by Tim Peters

Jodok Batlogg, Lovely Systems GmbH
Schmelzh?tterstra?e 26a, 6850 Dornbirn, Austria
mobile: +43 676 5683591, phone: +43 5572 908060

varnish consuming too much memory? [ In reply to ]

Mar 13, 2008, 3:00 PM

Post #23 of 24 (1789 views)

In message <438E9577-CF3C-4328-B108-402128196ED6 at lovelysystems.com>, Jodok Batl
ogg writes:

>what is varnish's disk file exactly for? is varnish using "real" =20
>memory only and "swapping" to its own file?

it is the backingstore to which the kernel can page the virtual memory
which Varnish uses.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

varnish consuming too much memory? [ In reply to ]

Mar 13, 2008, 3:00 PM

Post #24 of 24 (1786 views)

In message <438E9577-CF3C-4328-B108-402128196ED6 at lovelysystems.com>, Jodok Batl
ogg writes:

>what is varnish's disk file exactly for? is varnish using "real" =20
>memory only and "swapping" to its own file?

it is the backingstore to which the kernel can page the virtual memory
which Varnish uses.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.