Mailing List Archive: Query about HSH_EXPBUSY & HSH_EXP in HSH

Query about HSH_EXPBUSY & HSH_EXP in HSH_Lookup

Feb 1, 2018, 12:46 PM

Post #1 of 3 (561 views)

Hi,

I was trying to understand the function

enum lookup_e
HSH_Lookup(struct req *req, struct objcore **ocp, struct objcore **bocp,
int always_insert)

located in bin/varnishd/cache/cache_hash.c

My understanding is, if the lookedup-object in the cache is expired, the
code seems to look if the object is busy or not and accordingly returns
HSH_EXPBUSY or HSH_BUSY.

lines 474 to 501 in cache_hash.c
if (exp_oc != NULL && exp_oc->flags & OC_F_PASS) {
wrk->stats->cache_hitmiss++;
VSLb(req->vsl, SLT_HitMiss, "%u %.6f", ObjGetXID(wrk, exp_oc),
EXP_Dttl(req, exp_oc));
exp_oc = NULL;
busy_found = 0;
}

if (exp_oc != NULL) {
assert(oh->refcnt > 1);
assert(exp_oc->objhead == oh);
exp_oc->refcnt++;

if (!busy_found) {
*bocp = hsh_insert_busyobj(wrk, oh);
retval = HSH_EXPBUSY;
} else {
AZ(req->hash_ignore_busy);
retval = HSH_EXP;
}
if (exp_oc->hits < LONG_MAX)
exp_oc->hits++;
Lck_Unlock(&oh->mtx);
if (retval == HSH_EXP)
assert(HSH_DerefObjHead(wrk, &oh));
*ocp = exp_oc;
return (retval);
}

if (!busy_found) {
/* Insert objcore in objecthead and release mutex */
*bocp = hsh_insert_busyobj(wrk, oh);
/* NB: no deref of objhead, new object inherits reference */
Lck_Unlock(&oh->mtx);
return (HSH_MISS);
}

If busy, is the variable busy_found set to 0 or 1? For expired objects, it
looks like if it is not busy, then you set the return value to HSH_EXPBUSY,
shouldn't it be HSH_EXP? Can anyone help me understand this piece of code?
Because below, the code returns HSH_MISS

?Thanks,
Arvind

Re: Query about HSH_EXPBUSY & HSH_EXP in HSH_Lookup [ In reply to ]

dridi at varni

Feb 1, 2018, 1:36 PM

Post #2 of 3 (561 views)

Permalink

On Thu, Feb 1, 2018 at 9:46 PM, Arvind Narayanan <arvind@cs.umn.edu> wrote:
<snip>
> If busy, is the variable busy_found set to 0 or 1? For expired objects, it
> looks like if it is not busy, then you set the return value to HSH_EXPBUSY,
> shouldn't it be HSH_EXP? Can anyone help me understand this piece of code?
> Because below, the code returns HSH_MISS

It's a bit embarrassing because I've stared hard at this code a couple
months ago, and don't remember the details. I think that HSH_EXPBUSY
means that the object you found during the lookup was expired but
there is an ongoing transaction also matching (hash and variants) your
lookup. So I think in that case your hit is actually a graced object.

But I only tried to remember from the snippet you posted, I didn't
look at the full picture.

Dridi
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Re: Query about HSH_EXPBUSY & HSH_EXP in HSH_Lookup [ In reply to ]

hermunn at varnish-software

Feb 2, 2018, 4:23 AM

Post #3 of 3 (556 views)

Permalink

2018-02-01 22:36 GMT+01:00 Dridi Boukelmoune <dridi@varni.sh>:
> On Thu, Feb 1, 2018 at 9:46 PM, Arvind Narayanan <arvind@cs.umn.edu> wrote:
> <snip>
>> If busy, is the variable busy_found set to 0 or 1? For expired objects, it
>> looks like if it is not busy, then you set the return value to HSH_EXPBUSY,
>> shouldn't it be HSH_EXP? Can anyone help me understand this piece of code?
>> Because below, the code returns HSH_MISS
>
> It's a bit embarrassing because I've stared hard at this code a couple
> months ago, and don't remember the details.

I had to spend a lot of time with this code, and I think I remember
most of it. I did a small essay on an old bug,

https://github.com/varnishcache/varnish-cache/pull/2519

and, as you can see, I propose a change in the code that will make it
even harder to follow. Later I discovered that my PR was not enough
to fix the problems, and I am working on a new PR that will change
how TTLs are calculated.

> I think that HSH_EXPBUSY means that the object you found during the lookup
> was expired but there is an ongoing transaction also matching (hash and
> variants) your lookup.

This is correct.

> So I think in that case your hit is actually a graced object.

It can also be an expired object that is in "keep". However, even if
you have 0s keep and 0s grace, you can get an object here. This
happens when the expiry thread is lagging a bit, or when you are
affected by the very slightly different ways TTLs are calculated in
VCL and the core.

> But I only tried to remember from the snippet you posted, I didn't
> look at the full picture.

If you look at the code that calls HSH_Lookup, you will see that the
return value is not always considered, but the result of the lookup
is deduced from the values that HSH_Lookup did or did not set
(struct objcore **ocp, struct objcore **bocp).

When reading the code, maybe looking at the variables ocp and
bocp is key to understanding what is going on.

Pål
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc