Mailing List Archive: [RFC] version 2 state diagram (take 2)

[RFC] version 2 state diagram (take 2)

Feb 12, 2007, 5:38 AM

Post #1 of 3 (662 views)

I would appreciate if you call could take a moment to look at this
diagram and see if it makes sense:

http://phk.freebsd.dk/misc/varnish.png

One way or another, something like this will be the cornerstone in
explaning VCL to Varnish users, so I would appreciate feedback from
you guys that this is actually understandable.

The plot has gotten simpler, partly because I did it by hand instead
of using dot(1), but more so because I found out that I needed to
trust VCL to do more things.

I think the result is an improvement.

There are some new vcl functions as a result, so let me present
them all:

vcl_recv{}
Good place to check ACLs, rewrite URLS and generally fix
up things.

Pipe must be selected from here if necessary.

If you select pass from here it will not result in a cached
"pass this one" entry. This is useful for giving a limited
subset of clients direct access to the backend.

vcl_hash{}
Place holder for the idea that it might be desirable to
hash on more or different fields than (URL+Host:).

Not fully thought through yet, may or may not happen.

vcl_hit{}
Not really sure what you would do here, but you get
the chance if you dream up something.

One possible thing would be to react to high hitrates
and initiate compression.

vcl_miss{}
Fix up headers before requesting from backend

vcl_pass{}
Fix up headers before requesting from backend

vcl_fetch{}
Fix up headers received from backend.
Decide if object gets cached or not.
Determine Time To Live

vcl_pipe{}
Fix up headers before requesting from backend
Hint: Insert "Connection: close" header to terminate
pipe session after one request.

vcl_error{}
Issue error object to client.

Poul-Henning

PS: Computer Science connoisseurs will recognize, the graph as
inspired by figure 3 in the 1975 paper by S?ren Lauesen "A Large
Semaphore Based Operating System": All the arrows point downwards
as intuitive proof that no loops are possible.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

[RFC] version 2 state diagram (take 2) [ In reply to ]

des at linpro

Feb 12, 2007, 5:57 AM

Post #2 of 3 (651 views)

Permalink

Poul-Henning Kamp <phk at phk.freebsd.dk> writes:
> I would appreciate if you call could take a moment to look at this
> diagram and see if it makes sense:
>
> http://phk.freebsd.dk/misc/varnish.png

It certainly is a lot more readable than the complete diagram
(http://phk.freebsd.dk/misc/varnish.gif).

I assume all the red arrows point to vcl_error()?

> The plot has gotten simpler, partly because I did it by hand instead
> of using dot(1), but more so because I found out that I needed to
> trust VCL to do more things.

Absolutely.

> vcl_hash{}
> Place holder for the idea that it might be desirable to
> hash on more or different fields than (URL+Host:).
>
> Not fully thought through yet, may or may not happen.

Strictly speaking, you need to hash on every header mentioned in
Vary:, but you don't even have that information until you've retrieved
at least one version of the document from the backend - and even then,
when a new request comes in, you need to identify the requested
document to know which headers to hash on - so it looks like we need a
two-level hash.

> vcl_hit{}
> Not really sure what you would do here, but you get
> the chance if you dream up something.
>
> One possible thing would be to react to high hitrates
> and initiate compression.

...or check the time remaining before expiry and initiate a prefetch?
You don't mention vcl_timeout, which in any case isn't called until
after the document has expired.

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no

[RFC] version 2 state diagram (take 2) [ In reply to ]

phk at phk

Feb 12, 2007, 7:42 AM

Post #3 of 3 (656 views)

Permalink

In message <ujrodnza5z8.fsf at cat.linpro.no>, Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?=
writes:

>I assume all the red arrows point to vcl_error()?

Yes, I didn't want to clutter the drawing with them.

>> vcl_hash{}
>> Place holder for the idea that it might be desirable to
>> hash on more or different fields than (URL+Host:).
>>
>> Not fully thought through yet, may or may not happen.
>
>Strictly speaking, you need to hash on every header mentioned in
>Vary:, but you don't even have that information until you've retrieved
>at least one version of the document from the backend - and even then,
>when a new request comes in, you need to identify the requested
>document to know which headers to hash on - so it looks like we need a
>two-level hash.

You can't hash on Vary, it's a response side header. Vary must be
handled by traversing the list of possible objects looking for matches.

>> vcl_hit{}
>> Not really sure what you would do here, but you get
>> the chance if you dream up something.
>>
>> One possible thing would be to react to high hitrates
>> and initiate compression.
>
>...or check the time remaining before expiry and initiate a prefetch?
>You don't mention vcl_timeout, which in any case isn't called until
>after the document has expired.

My plans for vcl_timeout{} are not firm yet, but the idea is to
call vcl_timeout{} before expiry to decide on prefetching and
possibly compression.

The comment above therefore hints at the vcl_hit{} being able
to reduce the timeout for an object, so vcl_timeout{} can get
a shot at compressing it.

But lets concentrate on the "traffic" part for now, vcl_timeout{}
needs a bit more time with the thinking cap.

--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.