Mailing List Archive

#1723: High CPU load and exponential peak of objects in a281a10
--------------------------+----------------------------------
Reporter: zaterio@… | Owner:
Type: defect | Status: new
Priority: normal | Milestone: Varnish 4.0 release
Component: varnishd | Version: trunk
Severity: major | Resolution:
Keywords: load objects |
--------------------------+----------------------------------

Old description:

> I am using varnishd (varnish-trunk review a281a10), with 2 storage
> backends:
> ram1: default storage (malloc, 8GB)
> vod1: only for mp4 files (file, 50 GB)
>
> in vcl_backend_response, whe have the following rule:
>

> set beresp.storage_hint = "ram1";
> set beresp.http.x-storage = "ram1";
>
> if (bereq.url ~ "\.mp4$")
> {
> set beresp.storage_hint = "vod1";
> set beresp.http.x-storage =
> "vod1";
> set beresp.do_stream = true;
> set beresp.ttl = 10h;
> set beresp.http.Cache-Control =
> "max-age=604800, public";
> set beresp.do_esi = true;
> return (deliver);
> }
>
> On this machine we have traffic peaks of 1 Gbits, but yesterady we
> accomodate new service and trafic increased to 2 Gbps.
>
> In this situation, every 30 min, the load increases, up to 100 (5
> minutes), and MAIN.n_objectcore and
> MAIN.n_objecthead rises to 16970000000000 (normal is 5.5k), and then
> uptime returns to 0, and load decreases.
>
> We have another server with identical hardware and varnish configuration
> (but more traffic, 5Gbps), This server does not have the above described
> behavior, this server is running varnish-trunk revision 7746e30.
> Whe downgrade the problematic machine from a281a10 to 7746e30 and after
> 10 hours looks normal.
>
> DAEMON_OPTS="-a XXX.XXX.XXX.XXX:80, \
> -T XXX:XXX:XXX:XXX:6082 \
> -f /etc/varnish/default.vcl \
> -h classic,16383 \
> -s ram1=malloc,8G \
> -s vod1=file,/varnishcache/varnish.bin,50G \
> -p thread_pools=2 \
> -p thread_pool_min=500 \
> -p thread_pool_max=3000 \
> -p thread_pool_add_delay=2 \
> -p auto_restart=on \
> -p ping_interval=3 \
> -p send_timeout=5000 \
> -p workspace_session=1M \
> -p cli_timeout=25 \
> -p http_gzip_support=off \
> -p tcp_keepalive_time=600 \
> -p listen_depth=8192 \
> -p cli_buffer=32k \
> -p cli_limit=96k \
> -p ban_dups=on"

New description:

I am using varnishd (varnish-trunk review a281a10), with 2 storage
backends:
ram1: default storage (malloc, 8GB)
vod1: only for mp4 files (file, 50 GB)

in vcl_backend_response, whe have the following rule:

{{{
set beresp.storage_hint = "ram1";
set beresp.http.x-storage = "ram1";

if (bereq.url ~ "\.mp4$")
{
set beresp.storage_hint = "vod1";
set beresp.http.x-storage =
"vod1";
set beresp.do_stream = true;
set beresp.ttl = 10h;
set beresp.http.Cache-Control =
"max-age=604800, public";
set beresp.do_esi = true;
return (deliver);
}
}}}
On this machine we have traffic peaks of 1 Gbits, but yesterady we
accomodate new service and trafic increased to 2 Gbps.

In this situation, every 30 min, the load increases, up to 100 (5
minutes), and
MAIN.n_objectcore and
MAIN.n_objecthead rises to 16970000000000 (normal is 5.5k), and then
uptime returns to 0, and load decreases.

We have another server with identical hardware and varnish configuration
(but more traffic, 5Gbps), This server does not have the above described
behavior, this server is running varnish-trunk revision 7746e30.
Whe downgrade the problematic machine from a281a10 to 7746e30 and after 10
hours looks normal.

{{{
DAEMON_OPTS="-a XXX.XXX.XXX.XXX:80, \
-T XXX:XXX:XXX:XXX:6082 \
-f /etc/varnish/default.vcl \
-h classic,16383 \
-s ram1=malloc,8G \
-s vod1=file,/varnishcache/varnish.bin,50G \
-p thread_pools=2 \
-p thread_pool_min=500 \
-p thread_pool_max=3000 \
-p thread_pool_add_delay=2 \
-p auto_restart=on \
-p ping_interval=3 \
-p send_timeout=5000 \
-p workspace_session=1M \
-p cli_timeout=25 \
-p http_gzip_support=off \
-p tcp_keepalive_time=600 \
-p listen_depth=8192 \
-p cli_buffer=32k \
-p cli_limit=96k \
-p ban_dups=on"
}}}

--

Comment (by phk):

Can you try "varnishadm panic.show" and send us the output ?

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1723#comment:1>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs

#1723: High CPU load and exponential peak of objects in a281a10
--------------------------+----------------------------------
Reporter: zaterio@… | Owner:
Type: defect | Status: new
Priority: normal | Milestone: Varnish 4.0 release
Component: varnishd | Version: trunk
Severity: major | Resolution:
Keywords: load objects |
--------------------------+----------------------------------

Comment (by zaterio@…):

I think this error is related to #1735, after using trunk
12ace86f88c18a1ec85b18578e6f36ae2d95d501, it has not happened again.
Regards

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1723#comment:2>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs

#1723: High CPU load and exponential peak of objects in a281a10
--------------------------+----------------------------------
Reporter: zaterio@… | Owner:
Type: defect | Status: closed
Priority: normal | Milestone: Varnish 4.0 release
Component: varnishd | Version: trunk
Severity: major | Resolution: worksforme
Keywords: load objects |
--------------------------+----------------------------------
Changes (by daghf):

* status: new => closed
* resolution: => worksforme


Comment:

timed out.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1723#comment:3>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs