Hello,
We have been using the open source (free) version of Varnish Cache for my company's website for several months but are repeatedly running into an issue where Varnish's memory usage increases until all of the server's available memory is consumed, rendering the server unresponsive.? This continues to happen despite having Varnish's malloc and transient malloc storage settings dialed way down (currently set at 50% combined of available RAM).? Here's an excerpt from our storage backend configuration showing these settings.? I've attached our full backend storage settings to this message for review.
# Configure Varnish listening port, default "vcl" file location, memory allocation, etc:
ExecStart=/usr/sbin/varnishd -a :80 -f /etc/varnish/default.vcl -s malloc,14G -s Transient=malloc,512M -T 0.0.0.0:2000
ExecReload=/usr/sbin/varnishreload
We're running version 6.0.2 of Varnish Cache on a CentOS 8 virtual server.?? The server has 30 GB of RAM, 29.3 GB of which are available for use.
I've read a number of technical posts from other Varnish users complaining about the same issue, including this one:??https://stackoverflow.com/questions/65450044/varnish-6-lts-w-centos-8-not-respecting-memory-limits"]https://stackoverflow.com/questions/65450044/varnish-6-lts-w-centos-8-not-respecting-memory-limits.? Unfortunately, I've not seen any published resolution for this problem, but several tuning articles I've read talk about adjusting the server's memory allocation manager to more aggressively cleanup fragmentation, expired objects, etc.?? Here's one such post that talks about adjusting the "jemalloc" memory allocation manager settings as a possible fix:? https://info.varnish-software.com/blog/understanding-varnish-cache-memory-usage"]https://info.varnish-software.com/blog/understanding-varnish-cache-memory-usage.? I searched our CentOS 8 server to see what packages are installed but jemalloc is not in the list.
I'm still relatively new to Varnish Cache and am not sure what the next steps should be for troubleshooting & identifying the issue.? FWIW, I reached out to the folks at Varnish Cache asking if they could offer any suggestions, but they said we'd have to upgrade to their Enterprise version, which uses something called a "massive storage engine" that would eliminate this problem.? Not sure what the differences are between the paid / free versions, but I'm hoping to find a solution to this problem here before having to upgrade.
Thanks in advance for any assistance the community can provide.
John Kormanec
We have been using the open source (free) version of Varnish Cache for my company's website for several months but are repeatedly running into an issue where Varnish's memory usage increases until all of the server's available memory is consumed, rendering the server unresponsive.? This continues to happen despite having Varnish's malloc and transient malloc storage settings dialed way down (currently set at 50% combined of available RAM).? Here's an excerpt from our storage backend configuration showing these settings.? I've attached our full backend storage settings to this message for review.
# Configure Varnish listening port, default "vcl" file location, memory allocation, etc:
ExecStart=/usr/sbin/varnishd -a :80 -f /etc/varnish/default.vcl -s malloc,14G -s Transient=malloc,512M -T 0.0.0.0:2000
ExecReload=/usr/sbin/varnishreload
We're running version 6.0.2 of Varnish Cache on a CentOS 8 virtual server.?? The server has 30 GB of RAM, 29.3 GB of which are available for use.
I've read a number of technical posts from other Varnish users complaining about the same issue, including this one:??https://stackoverflow.com/questions/65450044/varnish-6-lts-w-centos-8-not-respecting-memory-limits"]https://stackoverflow.com/questions/65450044/varnish-6-lts-w-centos-8-not-respecting-memory-limits.? Unfortunately, I've not seen any published resolution for this problem, but several tuning articles I've read talk about adjusting the server's memory allocation manager to more aggressively cleanup fragmentation, expired objects, etc.?? Here's one such post that talks about adjusting the "jemalloc" memory allocation manager settings as a possible fix:? https://info.varnish-software.com/blog/understanding-varnish-cache-memory-usage"]https://info.varnish-software.com/blog/understanding-varnish-cache-memory-usage.? I searched our CentOS 8 server to see what packages are installed but jemalloc is not in the list.
I'm still relatively new to Varnish Cache and am not sure what the next steps should be for troubleshooting & identifying the issue.? FWIW, I reached out to the folks at Varnish Cache asking if they could offer any suggestions, but they said we'd have to upgrade to their Enterprise version, which uses something called a "massive storage engine" that would eliminate this problem.? Not sure what the differences are between the paid / free versions, but I'm hoping to find a solution to this problem here before having to upgrade.
Thanks in advance for any assistance the community can provide.
John Kormanec