Mailing List Archive

#1842: Assert error in hsh_rush()
#1842: Assert error in hsh_rush()
---------------------+--------------------
Reporter: Sesse | Type: defect
Status: new | Priority: normal
Milestone: | Component: build
Version: unknown | Severity: normal
Keywords: |
---------------------+--------------------
Hi,

I ran 4.1.1-beta1 on Debian jessie (from your packages). Immediately as
the load increased, it started crashing repeatedly. I picked out the last
panic, and got this:

Last panic at: Sat, 16 Jan 2016 13:39:46 GMT
"Assert error in hsh_rush(), cache/cache_hash.c line 526:
Condition((wl) != NULL) not true.
errno = 11 (Resource temporarily unavailable)
thread = (cache-worker)
version = varnish-4.1.1-beta1 revision 1fd0d39
ident = Linux,4.3.0,x86_64,-junix,-smalloc,-smalloc,-hcritbit,epoll
Backtrace:
0x433985: varnishd() [0x433985]
0x427e5a: varnishd() [0x427e5a]
0x428fc1: varnishd(HSH_DerefObjHead+0xa1) [0x428fc1]
0x4299c5: varnishd(HSH_Lookup+0x825) [0x4299c5]
0x436989: varnishd(CNT_Request+0x5b9) [0x436989]
0x44e713: varnishd(HTTP1_Session+0xe3) [0x44e713]
0x439d4d: varnishd(SES_Proto_Req+0x5d) [0x439d4d]
0x449062: varnishd() [0x449062]
0x4494db: varnishd() [0x4494db]
0x7f1c43b940a4: libpthread.so.0(+0x80a4) [0x7f1c43b940a4]
req = 0x7f1c2f00f020 {
vxid = 229379, step = R_STP_LOOKUP,
req_body = R_BODY_NONE,
restarts = 0, esi_level = 0,
sp = 0x7f1c30c0e420 {
fd = 35, vxid = 229377,
client = 84.211.184.160 49650,
step = S_STP_H1PROC,
},
worker = 0x7f1c434c5c80 {
stack = {0x7f1c434c6000 -> 0x7f1c434ba000},
ws = 0x7f1c434c5e78 {
id = \"wrk\",
{s,f,r,e} = {0x7f1c434c5420,0x7f1c434c5420,(nil),+2040},
},
VCL::method = HASH,
VCL::return = lookup,
VCL::methods = {},
},
ws = 0x7f1c2f00f200 {
id = \"req\",
{s,f,r,e} = {0x7f1c2f011000,+608,+90104,+90104},
},
http_conn = 0x7f1c2f00f128 {
fd = 35,
doclose = NULL,
ws = 0x7f1c2f00f200,
{rxbuf_b, rxbuf_e} = {0x7f1c2f011000, 0x7f1c2f0111a3},
{pipeline_b, pipeline_e} = {(nil), (nil)},
content_length = -1,
body_status = none,
first_byte_timeout = 0.000000,
between_bytes_timeout = 0.000000,
},
http[req] = 0x7f1c2f00f298 {
ws[req] = 0x7f1c2f00f200,
hdrs {
\"GET\",
\"/analysis.pl?ims=1452951576000&unique=0.15300315176136792\",
\"HTTP/1.1\",
\"Host: analysis.sesse.net\",
\"Connection: keep-alive\",
\"Accept: */*\",
\"X-Requested-With: XMLHttpRequest\",
\"User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36\",
\"Referer: http://analysis.sesse.net/\",
\"Accept-Language: en-US,en;q=0.8,nb;q=0.6\",
\"DNT: 1\",
\"X-Forwarded-For: 84.211.184.160, 84.211.184.160\",
\"x-analysis-backend: backend1\",
\"Accept-Encoding: gzip\",
},
},
vcl = {
temp = warm
srcname = {
\"/etc/varnish/default.vcl\",
\"Builtin\",
},
},
flags = {
},
},

"

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1842>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1842: Assert error in hsh_rush() [ In reply to ]
#1842: Assert error in hsh_rush()
----------------------+----------------------
Reporter: Sesse | Owner:
Type: defect | Status: new
Priority: normal | Milestone:
Component: varnishd | Version: unknown
Severity: normal | Resolution:
Keywords: |
----------------------+----------------------
Changes (by lkarsten):

* component: build => varnishd


Old description:

> Hi,
>
> I ran 4.1.1-beta1 on Debian jessie (from your packages). Immediately as
> the load increased, it started crashing repeatedly. I picked out the last
> panic, and got this:
>
> Last panic at: Sat, 16 Jan 2016 13:39:46 GMT
> "Assert error in hsh_rush(), cache/cache_hash.c line 526:
> Condition((wl) != NULL) not true.
> errno = 11 (Resource temporarily unavailable)
> thread = (cache-worker)
> version = varnish-4.1.1-beta1 revision 1fd0d39
> ident = Linux,4.3.0,x86_64,-junix,-smalloc,-smalloc,-hcritbit,epoll
> Backtrace:
> 0x433985: varnishd() [0x433985]
> 0x427e5a: varnishd() [0x427e5a]
> 0x428fc1: varnishd(HSH_DerefObjHead+0xa1) [0x428fc1]
> 0x4299c5: varnishd(HSH_Lookup+0x825) [0x4299c5]
> 0x436989: varnishd(CNT_Request+0x5b9) [0x436989]
> 0x44e713: varnishd(HTTP1_Session+0xe3) [0x44e713]
> 0x439d4d: varnishd(SES_Proto_Req+0x5d) [0x439d4d]
> 0x449062: varnishd() [0x449062]
> 0x4494db: varnishd() [0x4494db]
> 0x7f1c43b940a4: libpthread.so.0(+0x80a4) [0x7f1c43b940a4]
> req = 0x7f1c2f00f020 {
> vxid = 229379, step = R_STP_LOOKUP,
> req_body = R_BODY_NONE,
> restarts = 0, esi_level = 0,
> sp = 0x7f1c30c0e420 {
> fd = 35, vxid = 229377,
> client = 84.211.184.160 49650,
> step = S_STP_H1PROC,
> },
> worker = 0x7f1c434c5c80 {
> stack = {0x7f1c434c6000 -> 0x7f1c434ba000},
> ws = 0x7f1c434c5e78 {
> id = \"wrk\",
> {s,f,r,e} = {0x7f1c434c5420,0x7f1c434c5420,(nil),+2040},
> },
> VCL::method = HASH,
> VCL::return = lookup,
> VCL::methods = {},
> },
> ws = 0x7f1c2f00f200 {
> id = \"req\",
> {s,f,r,e} = {0x7f1c2f011000,+608,+90104,+90104},
> },
> http_conn = 0x7f1c2f00f128 {
> fd = 35,
> doclose = NULL,
> ws = 0x7f1c2f00f200,
> {rxbuf_b, rxbuf_e} = {0x7f1c2f011000, 0x7f1c2f0111a3},
> {pipeline_b, pipeline_e} = {(nil), (nil)},
> content_length = -1,
> body_status = none,
> first_byte_timeout = 0.000000,
> between_bytes_timeout = 0.000000,
> },
> http[req] = 0x7f1c2f00f298 {
> ws[req] = 0x7f1c2f00f200,
> hdrs {
> \"GET\",
> \"/analysis.pl?ims=1452951576000&unique=0.15300315176136792\",
> \"HTTP/1.1\",
> \"Host: analysis.sesse.net\",
> \"Connection: keep-alive\",
> \"Accept: */*\",
> \"X-Requested-With: XMLHttpRequest\",
> \"User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106
> Safari/537.36\",
> \"Referer: http://analysis.sesse.net/\",
> \"Accept-Language: en-US,en;q=0.8,nb;q=0.6\",
> \"DNT: 1\",
> \"X-Forwarded-For: 84.211.184.160, 84.211.184.160\",
> \"x-analysis-backend: backend1\",
> \"Accept-Encoding: gzip\",
> },
> },
> vcl = {
> temp = warm
> srcname = {
> \"/etc/varnish/default.vcl\",
> \"Builtin\",
> },
> },
> flags = {
> },
> },
>
> "

New description:

Hi,

I ran 4.1.1-beta1 on Debian jessie (from your packages). Immediately as
the load increased, it started crashing repeatedly. I picked out the last
panic, and got this:

{{{
Last panic at: Sat, 16 Jan 2016 13:39:46 GMT
"Assert error in hsh_rush(), cache/cache_hash.c line 526:
Condition((wl) != NULL) not true.
errno = 11 (Resource temporarily unavailable)
thread = (cache-worker)
version = varnish-4.1.1-beta1 revision 1fd0d39
ident = Linux,4.3.0,x86_64,-junix,-smalloc,-smalloc,-hcritbit,epoll
Backtrace:
0x433985: varnishd() [0x433985]
0x427e5a: varnishd() [0x427e5a]
0x428fc1: varnishd(HSH_DerefObjHead+0xa1) [0x428fc1]
0x4299c5: varnishd(HSH_Lookup+0x825) [0x4299c5]
0x436989: varnishd(CNT_Request+0x5b9) [0x436989]
0x44e713: varnishd(HTTP1_Session+0xe3) [0x44e713]
0x439d4d: varnishd(SES_Proto_Req+0x5d) [0x439d4d]
0x449062: varnishd() [0x449062]
0x4494db: varnishd() [0x4494db]
0x7f1c43b940a4: libpthread.so.0(+0x80a4) [0x7f1c43b940a4]
req = 0x7f1c2f00f020 {
vxid = 229379, step = R_STP_LOOKUP,
req_body = R_BODY_NONE,
restarts = 0, esi_level = 0,
sp = 0x7f1c30c0e420 {
fd = 35, vxid = 229377,
client = 84.211.184.160 49650,
step = S_STP_H1PROC,
},
worker = 0x7f1c434c5c80 {
stack = {0x7f1c434c6000 -> 0x7f1c434ba000},
ws = 0x7f1c434c5e78 {
id = \"wrk\",
{s,f,r,e} = {0x7f1c434c5420,0x7f1c434c5420,(nil),+2040},
},
VCL::method = HASH,
VCL::return = lookup,
VCL::methods = {},
},
ws = 0x7f1c2f00f200 {
id = \"req\",
{s,f,r,e} = {0x7f1c2f011000,+608,+90104,+90104},
},
http_conn = 0x7f1c2f00f128 {
fd = 35,
doclose = NULL,
ws = 0x7f1c2f00f200,
{rxbuf_b, rxbuf_e} = {0x7f1c2f011000, 0x7f1c2f0111a3},
{pipeline_b, pipeline_e} = {(nil), (nil)},
content_length = -1,
body_status = none,
first_byte_timeout = 0.000000,
between_bytes_timeout = 0.000000,
},
http[req] = 0x7f1c2f00f298 {
ws[req] = 0x7f1c2f00f200,
hdrs {
\"GET\",
\"/analysis.pl?ims=1452951576000&unique=0.15300315176136792\",
\"HTTP/1.1\",
\"Host: analysis.sesse.net\",
\"Connection: keep-alive\",
\"Accept: */*\",
\"X-Requested-With: XMLHttpRequest\",
\"User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36\",
\"Referer: http://analysis.sesse.net/\",
\"Accept-Language: en-US,en;q=0.8,nb;q=0.6\",
\"DNT: 1\",
\"X-Forwarded-For: 84.211.184.160, 84.211.184.160\",
\"x-analysis-backend: backend1\",
\"Accept-Encoding: gzip\",
},
},
vcl = {
temp = warm
srcname = {
\"/etc/varnish/default.vcl\",
\"Builtin\",
},
},
flags = {
},
},
}}}

--

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1842#comment:1>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1842: Assert error in hsh_rush() [ In reply to ]
#1842: Assert error in hsh_rush()
----------------------+--------------------------
Reporter: Sesse | Owner:
Type: defect | Status: new
Priority: normal | Milestone:
Component: varnishd | Version: 4.1.1-beta1
Severity: normal | Resolution:
Keywords: |
----------------------+--------------------------
Changes (by lkarsten):

* version: unknown => 4.1.1-beta1


--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1842#comment:2>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1842: Assert error in hsh_rush() [ In reply to ]
#1842: Assert error in hsh_rush()
----------------------+--------------------------
Reporter: Sesse | Owner:
Type: defect | Status: new
Priority: normal | Milestone:
Component: varnishd | Version: 4.1.1-beta1
Severity: normal | Resolution:
Keywords: |
----------------------+--------------------------

Comment (by lkarsten):

Note for Monday: verify that 32af38d43c9cefa10ef24775ef999fbd61fd49ed does
what it was supposed to do.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1842#comment:3>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1842: Assert error in hsh_rush() [ In reply to ]
#1842: Assert error in hsh_rush()
----------------------+--------------------------
Reporter: Sesse | Owner:
Type: defect | Status: new
Priority: normal | Milestone:
Component: varnishd | Version: 4.1.1-beta1
Severity: normal | Resolution:
Keywords: |
----------------------+--------------------------

Comment (by lkarsten):

https://gist.github.com/lkarsten/18d35c409d2227bd7d74

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1842#comment:4>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1842: Assert error in hsh_rush() [ In reply to ]
#1842: Assert error in hsh_rush()
----------------------+--------------------------------------------
Reporter: Sesse | Owner: Lasse Karstensen <lkarsten@…>
Type: defect | Status: closed
Priority: normal | Milestone:
Component: varnishd | Version: 4.1.1-beta1
Severity: normal | Resolution: fixed
Keywords: |
----------------------+--------------------------------------------
Changes (by Lasse Karstensen <lkarsten@…>):

* status: new => closed
* owner: => Lasse Karstensen <lkarsten@…>
* resolution: => fixed


Comment:

In [d8ac05a3c4b7b5838eee2d94150e6853cfc64d43]:
{{{
#!CommitTicketReference repository=""
revision="d8ac05a3c4b7b5838eee2d94150e6853cfc64d43"
Handle missing waiting list gracefully.

In 5c8268062bf06a2700dd27331c29c48d650c9197 the checks are relaxed,
but this commit (or part of) was not brought into 4.1.

Fixes: #1842
}}}

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1842#comment:5>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs