Mailing List Archive

#1788: ObjIter has terrible performance profile when busyobj != NULL
-------------------+--------------------
Reporter: tnt | Type: defect
Status: new | Priority: normal
Milestone: | Component: build
Version: 4.0.3 | Severity: normal
Keywords: |
-------------------+--------------------
The loop of ObjIter when there is a busyobj (i.e. dl from the backend
isn't over), will do a linear scan of all the 'struct storage' for every
chunk served to the client. This mean a N^2 complexity which is _really_
bad.

This causes issues when the file is largish (i.e hundreds of Mbytes), the
download time is several order of magnitude larger through varnish than it
is hitting the backend directly.

Especially when the client connection is about the same speed as the
backend connection (and so busyobj is != NULL for a while)

I tested this on 4.1-rp1 and 4.0.3.

To reproduce, use this .vtc

{{{
vcl 4.0;

backend be {
.host = "127.0.0.1";
.port = "5000";
}

sub vcl_recv {
return (pass);
}

sub vcl_backend_response {
set beresp.do_stream = true;
}
}}}

and download like a 1G file through varnish. In my case it takes 2m30s to
download (from 127.0.0.1) and my cpu is 100% the whole time. While hitting
the backend directly it takes 3s.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1788>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs