Author: des
Date: 2006-02-22 17:18:23 +0100 (Wed, 22 Feb 2006)
New Revision: 12
Added:
trunk/varnish-doc/en/
trunk/varnish-doc/en/varnish-specification/
trunk/varnish-doc/en/varnish-specification/article.xml
Log:
Draft specification in DocBook format.
Added: trunk/varnish-doc/en/varnish-specification/article.xml
===================================================================
--- trunk/varnish-doc/en/varnish-specification/article.xml 2006-02-22 16:18:07 UTC (rev 11)
+++ trunk/varnish-doc/en/varnish-specification/article.xml 2006-02-22 16:18:23 UTC (rev 12)
@@ -0,0 +1,636 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!DOCTYPE article
+ PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
+ "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
+<article lang="en">
+ <articleinfo>
+ <releaseinfo role="cvs">$Id$</releaseinfo>
+ <title>Varnish HTTP Accelerator Draft Specification</title>
+ </articleinfo>
+
+ <section>
+ <title>Introduction</title>
+
+ <section>
+ <title>Overview</title>
+
+ <para>Varnish is a high-performance HTTP accelerator.</para>
+
+ <para>XXX</para>
+ </section>
+
+ <section>
+ <title>Terminology</title>
+
+ <para>The key words ?MUST?, ?MUST NOT?, ?REQUIRED?, ?SHALL?,
+ ?SHALL NOT?, ?SHOULD?, ?SHOULD NOT?, ?RECOMMENDED?, ?MAY?, and
+ ?OPTIONAL? in this document are to be interpreted as described
+ in IETF RFC 2119.</para>
+
+ <para>XXX at this time, the above is incorrect because we
+ started out using MoSCoW prioritisation before realising it was
+ inadequate for the task.</para>
+ </section>
+
+ <section>
+ <title>To do</title>
+
+ <para>Use consistent terminology throughout the document (see
+ previous section)</para>
+
+ <para>Clarify the following terms: configuration facility;
+ logging and statistics facility (split in two?); monitoring and
+ tuning facility (split in two? does this overlap with
+ configuration facility?)</para>
+ </section>
+ </section>
+
+ <section>
+ <title>General requirements</title>
+
+ <section>
+ <title>License</title>
+
+ <para>XXX two-clause BSD license</para>
+ </section>
+
+ <section>
+ <title>Version control</title>
+
+ <para>All source code and documentation must be kept under
+ version control in a publicly accessible repository.</para>
+ </section>
+
+ <section>
+ <title>Software platform</title>
+
+ <para>Varnish must be fully functional on the platforms listed
+ in this section.</para>
+
+ <para>Varnish should also be fully functional on other
+ POSIX-derived platforms, insofar as this does not require
+ unreasonable effort.</para>
+
+ <section>
+ <title>FreeBSD</title>
+
+ <para>Varnish must be fully functional on FreeBSD 6.0 or later
+ officially supported releases, including security and errata
+ branches.</para>
+
+ <para>The reference platform for FreeBSD compatibility is
+ FreeBSD 6.1-RELEASE.</para>
+ </section>
+
+ <section>
+ <title>GNU/Linux</title>
+
+ <para>Varnish must be fully functional on GNU/Linux systems
+ with Linux 2.6.12 or later and GNU libc 2.3.2 or later.</para>
+
+ <para>The reference platform for GNU/Linux compatibility is
+ Ubuntu 5.10 ?Breezy Badger?.</para>
+ </section>
+ </section>
+
+ <section>
+ <title>Hardware platform</title>
+
+ <para>Varnish must be fully functional on both 32-bit and 64-bit
+ Intel-compatible hardware, but may place different limits on
+ operating parameters (such as cache size) depending on the
+ platform and the amount of physical and / or virtual memory
+ available. Varnish must support and take advantage of multiple
+ CPUs or CPU cores if present.</para>
+
+ <para>The reference hardware platform is a dual-CPU AMD Opteron
+ system with 4?GB of RAM divided evenly between the CPUs.</para>
+ </section>
+
+ <section>
+ <title>Language</title>
+
+ <para>Varnish must be implemented in C with as few compiler- and
+ platform-specific extensions as possible.</para>
+ </section>
+
+ <section>
+ <title>Compiler and toolchain</title>
+
+ <para>Varnish must be compilable with the GNU C compiler (GCC)
+ 3.3.5 or later and toolchain (binutils) 2.15 or later.</para>
+
+ <para>Alternative compilers and toolchains should be supported
+ insofar as this does not require unreasonable effort.</para>
+ </section>
+
+ <section>
+ <title>Compile-time configuration</title>
+
+ <para>Varnish must use the GNU Autotools for compile-time
+ configuration. The Makefile templates must be written to work
+ with any POSIX-compliant make(1) utility.</para>
+ </section>
+
+ <section>
+ <title>Third-party dependencies</title>
+
+ <para>Varnish must not depend on any third-party packages other
+ than the compiler, toolchain and configuration tools.</para>
+ </section>
+
+ <section>
+ <title>Incidental tools</title>
+
+ <para>Varnish may be accompanied by incidental tools for
+ purposes such as creating or editing configuration files,
+ warming up the cache, manipulating or generating source code,
+ etc. Insofar as these tools are not required at compilation or
+ installation time nor for the daily operation of Varnish, they
+ may be written in Perl 5.8 or later, and depend on third-party
+ Perl modules available from CPAN.</para>
+ </section>
+
+ <section>
+ <title>Coding standards</title>
+
+ <para>All C source code must conform to the FreeBSD style(9)
+ coding standard.</para>
+ </section>
+
+ <section>
+ <title>Documentation</title>
+
+ <para>Varnish must be accompanied by complete internal and
+ external documentation.</para>
+
+ <para>All documentation must be in English.</para>
+
+ <para>All documentation must be made available online in HTML
+ form, and may be made available online in additional formats
+ such as PDF.</para>
+
+ <section>
+ <title>Internal documentation</title>
+
+ <para>The internal documentation consists of:</para>
+ <orderedlist>
+ <listitem>
+ <para>Code comments.</para>
+ </listitem>
+ <listitem>
+ <para>Manual pages describing Varnish internals.</para>
+ </listitem>
+ <listitem>
+ <para>Version control history.</para>
+ </listitem>
+ <listitem>
+ <para>Requirements and specification in DocBook XML
+ format.</para>
+ </listitem>
+ <listitem>
+ <para>System architecture in DocBook XML format.</para>
+ </listitem>
+ <listitem>
+ <para>Developer guidelines and other incidental
+ documentation either in the project Wiki or in DocBook XML
+ format.</para>
+ </listitem>
+ </orderedlist>
+ </section>
+
+ <section>
+ <title>External documentation</title>
+
+ <para>The external documentation consists of:</para>
+ <orderedlist>
+ <listitem>
+ <para>Manual pages for all daemons and command-line
+ tools.</para>
+ </listitem>
+ <listitem>
+ <para>Installation instructions in DocBook XML
+ format.</para>
+ </listitem>
+ <listitem>
+ <para>Administrator's handbook in DocBook XML
+ format.</para>
+ </listitem>
+ <listitem>
+ <para>Sample configuration files.</para>
+ </listitem>
+ </orderedlist>
+ </section>
+ </section>
+ </section>
+
+ <section>
+ <title>Functional requirements</title>
+
+ <section>
+ <title>Functional description</title>
+
+ <para>Varnish accepts HTTP requests from clients and satisfy
+ them with documents retrieved from its cache (disk- and / or
+ memory-based). Documents which are not present in the cache
+ must be retrieved from a set of preconfigured content servers.
+ Requests for documents from other servers than the preconfigured
+ content servers are ignored.</para>
+ </section>
+
+ <section>
+ <title>Protocol support</title>
+
+ <section>
+ <title>HTTP</title>
+
+ <para>Varnish must be able to accept HTTP/1.0 and HTTP/1.1
+ requests from both IPv4 and IPv6 clients.</para>
+
+ <para>Varnish must, in general terms, handle these requests in
+ conformance with IETF RFC 2616.</para>
+
+ <para>Varnish may handle HTTP/0.9 requests in any way it sees
+ fit, including but not limited to returning a 400 Bad Request
+ response or simply closing the connection.</para>
+
+ <para>Varnish must use HTTP/1.1 in its communications with the
+ content servers.</para>
+
+ <para>Varnish may deviate from IETF RFC 2616 when this is
+ necessary for interoperability with non-conforming clients or
+ content servers.</para>
+
+ <para>Varnish may deviate from IETF RFC 2616 in cases where
+ doing so provides a considerable performance advantage without
+ causing significant harm to interoperability. Any such
+ deviation must be documented.</para>
+
+ <para>In its communications with clients, Varnish must
+ interpret IETF RFC 2616 as if it were an origin server. In
+ its communications with content servers, Varnish must
+ interpret IETF RFC 2616 as if it were a cache.</para>
+ </section>
+
+ <section>
+ <title>ICP</title>
+
+ <para>Varnish may support ICP for inter-cache coordination.
+ ICP support may be a compile-time option.</para>
+ </section>
+
+ <section>
+ <title>HTCP</title>
+
+ <para>Varnish may support HTCP for inter-cache coordination.
+ HTCP support may be a compile-time option.</para>
+ </section>
+ </section>
+
+ <section>
+ <title>Content manipulation</title>
+
+ <para>Varnish won't implement content manipulation at this
+ time.</para>
+
+ <para>Varnish should be designed in such a manner as to make it
+ possible to implement various kinds of content manipulation
+ (such as ESI) at a future date.</para>
+
+ <para>XXX ICAP may be worth looking into (but is probably a
+ performance killer)</para>
+ </section>
+
+ <section>
+ <title>Caching</title>
+
+ <para>Varnish must maintain a local cache of the documets
+ present on the content server.</para>
+
+ <section>
+ <title>Cached set</title>
+
+ <para>If the amount of memory and / or disk available to
+ Varnish is not sufficient to cache the entire document set,
+ Varnish must attempt to identify a subset to cache which
+ minimizes load on the content servers.</para>
+
+ <para>Varnish should offer multiple alternative cache control
+ algorithms. At the very least, the LRU (least-recently-used)
+ and WLRU (LRU weighted by document size) algorithms should be
+ implemented.</para>
+ </section>
+
+ <section>
+ <title>Expiry</title>
+
+ <para>If a cached document has an expiry time associated with
+ it, and that time has not yet been reached, Varnish may serve
+ the document from cache without contacting the content
+ server.</para>
+ </section>
+
+ <section>
+ <title>Non-cacheable content</title>
+
+ <para>XXX</para>
+ </section>
+ </section>
+
+ <section>
+ <title>Management</title>
+
+ <section>
+ <title>Configuration</title>
+
+ <para>XXX</para>
+ </section>
+
+ <section>
+ <title>Logging and statistics</title>
+
+ <para>A separate application is responsible for collecting,
+ collating and analyzing log data which Varnish makes available
+ in circular shared memory buffers.</para>
+
+ <para>Varnish must provide the data necessary to compute
+ lifetime totals and sliding averages for the following:</para>
+ <orderedlist>
+ <listitem>
+ <para>Total size of documents served to clients</para>
+ </listitem>
+ <listitem>
+ <para>Total size of data transmitted to clients, including
+ headers, error messages, etc.</para>
+ </listitem>
+ <listitem>
+ <para>Total size of data received from clients, including
+ request headers etc.</para>
+ </listitem>
+ <listitem>
+ <para>Number of client connections received</para>
+ </listitem>
+ <listitem>
+ <para>Number of client requests served</para>
+ </listitem>
+ <listitem>
+ <para>Client requests broken down by result code</para>
+ </listitem>
+ <listitem>
+ <para>Total size of documents retrieved from content
+ servers</para>
+ </listitem>
+ <listitem>
+ <para>Total size of data received from content servers,
+ including headers, error messages, etc.</para>
+ </listitem>
+ <listitem>
+ <para>Total size of data sent to content servers,
+ including request headers etc.</para>
+ </listitem>
+ <listitem>
+ <para>Number of content server connections
+ initiated</para>
+ </listitem>
+ <listitem>
+ <para>Number of content server requests sent</para>
+ </listitem>
+ <listitem>
+ <para>Content server requests broken down by result
+ code</para>
+ </listitem>
+ <listitem>
+ <para>Cache effectiveness as the ratio of bytes served to
+ clients to bytes requested from content servers</para>
+ </listitem>
+ <listitem>
+ <para>Cache effectiveness as the ratio of client requests
+ to content server requests</para>
+ </listitem>
+ <listitem>
+ <para>Number of active server processes / threads, broken
+ down by process / thread type</para>
+ </listitem>
+ <listitem>
+ <para>XXX length of request queues</para>
+ </listitem>
+ </orderedlist>
+
+ <para>In addition, Varnish must provide the data necessary to
+ compute the average, median and distribution for the
+ following:</para>
+ <orderedlist>
+ <listitem>
+ <para>Size of documents served, per unique document</para>
+ </listitem>
+ <listitem>
+ <para>Size of documents served, per request</para>
+ </listitem>
+ <listitem>
+ <para>Client connection duration</para>
+ </listitem>
+ <listitem>
+ <para>Requests per client connection</para>
+ </listitem>
+ <listitem>
+ <para>Client request completion time</para>
+ </listitem>
+ <listitem>
+ <para>Content server connection duration</para>
+ </listitem>
+ <listitem>
+ <para>Requests per content server connection</para>
+ </listitem>
+ <listitem>
+ <para>Content server request completion time</para>
+ </listitem>
+ <listitem>
+ <para>XXX time spent in request queues</para>
+ </listitem>
+ </orderedlist>
+ </section>
+ </section>
+
+ <section>
+ <title>Run-time monitoring and tuning</title>
+
+ <para>Varnish must provide low-level monitoring and tuning
+ facilities. A separate application is responsible for providing
+ a user-friendly interface to these facilities.</para>
+
+ <para>The following monitoring operations must be
+ supported:</para>
+
+ <orderedlist>
+ <listitem>
+ <para>Cache status of individual documents</para>
+ </listitem>
+ <listitem>
+ <para>Cache status of documents matching a glob or regular
+ expression</para>
+ </listitem>
+ <listitem>
+ <para>Access statistics of individual documents</para>
+ </listitem>
+ <listitem>
+ <para>Access statistics of documents matching a glob or
+ regular expression</para>
+ </listitem>
+ <listitem>
+ <para>XXX</para>
+ </listitem>
+ </orderedlist>
+
+ <para>The following tuning operations must be supported:</para>
+
+ <orderedlist>
+ <listitem>
+ <para>Forced invalidation of individual documents</para>
+ </listitem>
+ <listitem>
+ <para>Forced invalidation of documents matching a glob or regular expression</para>
+ </listitem>
+ <listitem>
+ <para>XXX</para>
+ </listitem>
+ </orderedlist>
+ </section>
+ <section>
+ <title>Clustering</title>
+
+ <para>Clustering is defined in this context as a situation where
+ multiple servers are set up to run Varnish with the same
+ configuration, serving data from the same content servers to the
+ same set of clients.</para>
+
+ <section>
+ <title>Configuration</title>
+
+ <para>When multiple Varnish servers act together as a cluster,
+ the configuration facility is responsible for ensuring that
+ all nodes share the same configuration and that configuration
+ changes are applied to all nodes in a timely fashion.</para>
+ </section>
+
+ <section>
+ <title>Logging and statistics</title>
+
+ <para>When multiple Varnish servers act together as a cluster,
+ the logging and statistics facilities must base its reports on
+ aggregated data as if the cluster were a single Varnish
+ server.</para>
+
+ <para>Per-node data may optionally be made available in
+ addition to aggregated data.</para>
+ </section>
+
+ <section>
+ <title>Run-time monitoring and tuning</title>
+
+ <para>When multiple Varnish servers act together as a cluster,
+ the run-time monitoring and tuning facilities must propagate
+ invalidation requests and other administrative commands to all
+ servers in the cluster.</para>
+ </section>
+ </section>
+ </section>
+
+ <section>
+ <title>Application structure</title>
+
+ <section>
+ <title>Components</title>
+
+ <para>This section lists the major components in Varnish.</para>
+
+ <section>
+ <title>Listener</title>
+
+ <para>The Listener monitors the listening socket and accepts
+ incoming client connections. Once the connection is
+ established, it is passed to the Accepter.</para>
+
+ <para>The Listener should take advantage of accept filters or
+ similar technologies on systems where they are
+ available.</para>
+ </section>
+
+ <section>
+ <title>Accepter</title>
+
+ <para>The Accepter reads an HTTP request from a client
+ connection. It parses the request line and header only to the
+ extent necessary to establish well-formedness and determine
+ the requested URL.</para>
+
+ <para>The Accepter then queries the Keeper about the status of
+ the requested document (identified by its full URL). If the
+ document is present and valid in the cache, the request is
+ passed directly to a Sender. Otherwise, it is passed to a
+ Retriever queue.</para>
+ </section>
+
+ <section>
+ <title>Keeper</title>
+
+ <para>The Keeper manages the document cache. XXX</para>
+ </section>
+
+ <section>
+ <title>Sender</title>
+
+ <para>The Sender transfers the contents of the requested
+ document to the client. It examines the HTTP request header
+ to determine the correct way in which to do this ? Range,
+ If-Modified-Since, Content-Encoding and other options may
+ affect the type and amount of data transferred.</para>
+
+ <para>There may be multiple concurrent Sender threads.</para>
+ </section>
+
+ <section>
+ <title>Retriever</title>
+
+ <para>The Retriever is responsible for retrieving documents
+ from the content servers. It is triggered either by an
+ Accepter trying to satisfy a request for a document which is
+ not in the cache, or by the Janitor when a ?hot? document is
+ nearing expiry. Either way, there may be a queue of requests
+ waiting for the document to arrive; when it does, the
+ Retriever passes those requests to a Sender.</para>
+
+ <para>There may be multiple concurrent Retriever
+ threads.</para>
+ </section>
+
+ <section>
+ <title>Janitor</title>
+
+ <para>The Janitor keeps track of the expiry time of cached
+ documents and attempts to retrieve fresh copies of documents
+ which are soon to expire.</para>
+ </section>
+
+ <section>
+ <title>Logger</title>
+
+ <para>The Logger keeps logs of various types of events in
+ circular shared-memory buffers. These buffers must be managed
+ using either POSIX shared memory primitives or file-backed
+ mmap(2).</para>
+
+ <para>It is the responsibility of each module to feed relevant
+ log data to the Logger.</para>
+ </section>
+ </section>
+ </section>
+
+ <bibliography>
+ <title>References</title>
+
+ <biblioentry id="RFC2119"/>
+ <biblioentry id="RFC2186"/>
+ <biblioentry id="RFC2616"/>
+ <biblioentry id="RFC2756"/>
+ </bibliography>
+</article>
Property changes on: trunk/varnish-doc/en/varnish-specification/article.xml
___________________________________________________________________
Name: svn:mime-type
+ text/xml
Name: svn:charset
+ utf-8
Date: 2006-02-22 17:18:23 +0100 (Wed, 22 Feb 2006)
New Revision: 12
Added:
trunk/varnish-doc/en/
trunk/varnish-doc/en/varnish-specification/
trunk/varnish-doc/en/varnish-specification/article.xml
Log:
Draft specification in DocBook format.
Added: trunk/varnish-doc/en/varnish-specification/article.xml
===================================================================
--- trunk/varnish-doc/en/varnish-specification/article.xml 2006-02-22 16:18:07 UTC (rev 11)
+++ trunk/varnish-doc/en/varnish-specification/article.xml 2006-02-22 16:18:23 UTC (rev 12)
@@ -0,0 +1,636 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!DOCTYPE article
+ PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
+ "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
+<article lang="en">
+ <articleinfo>
+ <releaseinfo role="cvs">$Id$</releaseinfo>
+ <title>Varnish HTTP Accelerator Draft Specification</title>
+ </articleinfo>
+
+ <section>
+ <title>Introduction</title>
+
+ <section>
+ <title>Overview</title>
+
+ <para>Varnish is a high-performance HTTP accelerator.</para>
+
+ <para>XXX</para>
+ </section>
+
+ <section>
+ <title>Terminology</title>
+
+ <para>The key words ?MUST?, ?MUST NOT?, ?REQUIRED?, ?SHALL?,
+ ?SHALL NOT?, ?SHOULD?, ?SHOULD NOT?, ?RECOMMENDED?, ?MAY?, and
+ ?OPTIONAL? in this document are to be interpreted as described
+ in IETF RFC 2119.</para>
+
+ <para>XXX at this time, the above is incorrect because we
+ started out using MoSCoW prioritisation before realising it was
+ inadequate for the task.</para>
+ </section>
+
+ <section>
+ <title>To do</title>
+
+ <para>Use consistent terminology throughout the document (see
+ previous section)</para>
+
+ <para>Clarify the following terms: configuration facility;
+ logging and statistics facility (split in two?); monitoring and
+ tuning facility (split in two? does this overlap with
+ configuration facility?)</para>
+ </section>
+ </section>
+
+ <section>
+ <title>General requirements</title>
+
+ <section>
+ <title>License</title>
+
+ <para>XXX two-clause BSD license</para>
+ </section>
+
+ <section>
+ <title>Version control</title>
+
+ <para>All source code and documentation must be kept under
+ version control in a publicly accessible repository.</para>
+ </section>
+
+ <section>
+ <title>Software platform</title>
+
+ <para>Varnish must be fully functional on the platforms listed
+ in this section.</para>
+
+ <para>Varnish should also be fully functional on other
+ POSIX-derived platforms, insofar as this does not require
+ unreasonable effort.</para>
+
+ <section>
+ <title>FreeBSD</title>
+
+ <para>Varnish must be fully functional on FreeBSD 6.0 or later
+ officially supported releases, including security and errata
+ branches.</para>
+
+ <para>The reference platform for FreeBSD compatibility is
+ FreeBSD 6.1-RELEASE.</para>
+ </section>
+
+ <section>
+ <title>GNU/Linux</title>
+
+ <para>Varnish must be fully functional on GNU/Linux systems
+ with Linux 2.6.12 or later and GNU libc 2.3.2 or later.</para>
+
+ <para>The reference platform for GNU/Linux compatibility is
+ Ubuntu 5.10 ?Breezy Badger?.</para>
+ </section>
+ </section>
+
+ <section>
+ <title>Hardware platform</title>
+
+ <para>Varnish must be fully functional on both 32-bit and 64-bit
+ Intel-compatible hardware, but may place different limits on
+ operating parameters (such as cache size) depending on the
+ platform and the amount of physical and / or virtual memory
+ available. Varnish must support and take advantage of multiple
+ CPUs or CPU cores if present.</para>
+
+ <para>The reference hardware platform is a dual-CPU AMD Opteron
+ system with 4?GB of RAM divided evenly between the CPUs.</para>
+ </section>
+
+ <section>
+ <title>Language</title>
+
+ <para>Varnish must be implemented in C with as few compiler- and
+ platform-specific extensions as possible.</para>
+ </section>
+
+ <section>
+ <title>Compiler and toolchain</title>
+
+ <para>Varnish must be compilable with the GNU C compiler (GCC)
+ 3.3.5 or later and toolchain (binutils) 2.15 or later.</para>
+
+ <para>Alternative compilers and toolchains should be supported
+ insofar as this does not require unreasonable effort.</para>
+ </section>
+
+ <section>
+ <title>Compile-time configuration</title>
+
+ <para>Varnish must use the GNU Autotools for compile-time
+ configuration. The Makefile templates must be written to work
+ with any POSIX-compliant make(1) utility.</para>
+ </section>
+
+ <section>
+ <title>Third-party dependencies</title>
+
+ <para>Varnish must not depend on any third-party packages other
+ than the compiler, toolchain and configuration tools.</para>
+ </section>
+
+ <section>
+ <title>Incidental tools</title>
+
+ <para>Varnish may be accompanied by incidental tools for
+ purposes such as creating or editing configuration files,
+ warming up the cache, manipulating or generating source code,
+ etc. Insofar as these tools are not required at compilation or
+ installation time nor for the daily operation of Varnish, they
+ may be written in Perl 5.8 or later, and depend on third-party
+ Perl modules available from CPAN.</para>
+ </section>
+
+ <section>
+ <title>Coding standards</title>
+
+ <para>All C source code must conform to the FreeBSD style(9)
+ coding standard.</para>
+ </section>
+
+ <section>
+ <title>Documentation</title>
+
+ <para>Varnish must be accompanied by complete internal and
+ external documentation.</para>
+
+ <para>All documentation must be in English.</para>
+
+ <para>All documentation must be made available online in HTML
+ form, and may be made available online in additional formats
+ such as PDF.</para>
+
+ <section>
+ <title>Internal documentation</title>
+
+ <para>The internal documentation consists of:</para>
+ <orderedlist>
+ <listitem>
+ <para>Code comments.</para>
+ </listitem>
+ <listitem>
+ <para>Manual pages describing Varnish internals.</para>
+ </listitem>
+ <listitem>
+ <para>Version control history.</para>
+ </listitem>
+ <listitem>
+ <para>Requirements and specification in DocBook XML
+ format.</para>
+ </listitem>
+ <listitem>
+ <para>System architecture in DocBook XML format.</para>
+ </listitem>
+ <listitem>
+ <para>Developer guidelines and other incidental
+ documentation either in the project Wiki or in DocBook XML
+ format.</para>
+ </listitem>
+ </orderedlist>
+ </section>
+
+ <section>
+ <title>External documentation</title>
+
+ <para>The external documentation consists of:</para>
+ <orderedlist>
+ <listitem>
+ <para>Manual pages for all daemons and command-line
+ tools.</para>
+ </listitem>
+ <listitem>
+ <para>Installation instructions in DocBook XML
+ format.</para>
+ </listitem>
+ <listitem>
+ <para>Administrator's handbook in DocBook XML
+ format.</para>
+ </listitem>
+ <listitem>
+ <para>Sample configuration files.</para>
+ </listitem>
+ </orderedlist>
+ </section>
+ </section>
+ </section>
+
+ <section>
+ <title>Functional requirements</title>
+
+ <section>
+ <title>Functional description</title>
+
+ <para>Varnish accepts HTTP requests from clients and satisfy
+ them with documents retrieved from its cache (disk- and / or
+ memory-based). Documents which are not present in the cache
+ must be retrieved from a set of preconfigured content servers.
+ Requests for documents from other servers than the preconfigured
+ content servers are ignored.</para>
+ </section>
+
+ <section>
+ <title>Protocol support</title>
+
+ <section>
+ <title>HTTP</title>
+
+ <para>Varnish must be able to accept HTTP/1.0 and HTTP/1.1
+ requests from both IPv4 and IPv6 clients.</para>
+
+ <para>Varnish must, in general terms, handle these requests in
+ conformance with IETF RFC 2616.</para>
+
+ <para>Varnish may handle HTTP/0.9 requests in any way it sees
+ fit, including but not limited to returning a 400 Bad Request
+ response or simply closing the connection.</para>
+
+ <para>Varnish must use HTTP/1.1 in its communications with the
+ content servers.</para>
+
+ <para>Varnish may deviate from IETF RFC 2616 when this is
+ necessary for interoperability with non-conforming clients or
+ content servers.</para>
+
+ <para>Varnish may deviate from IETF RFC 2616 in cases where
+ doing so provides a considerable performance advantage without
+ causing significant harm to interoperability. Any such
+ deviation must be documented.</para>
+
+ <para>In its communications with clients, Varnish must
+ interpret IETF RFC 2616 as if it were an origin server. In
+ its communications with content servers, Varnish must
+ interpret IETF RFC 2616 as if it were a cache.</para>
+ </section>
+
+ <section>
+ <title>ICP</title>
+
+ <para>Varnish may support ICP for inter-cache coordination.
+ ICP support may be a compile-time option.</para>
+ </section>
+
+ <section>
+ <title>HTCP</title>
+
+ <para>Varnish may support HTCP for inter-cache coordination.
+ HTCP support may be a compile-time option.</para>
+ </section>
+ </section>
+
+ <section>
+ <title>Content manipulation</title>
+
+ <para>Varnish won't implement content manipulation at this
+ time.</para>
+
+ <para>Varnish should be designed in such a manner as to make it
+ possible to implement various kinds of content manipulation
+ (such as ESI) at a future date.</para>
+
+ <para>XXX ICAP may be worth looking into (but is probably a
+ performance killer)</para>
+ </section>
+
+ <section>
+ <title>Caching</title>
+
+ <para>Varnish must maintain a local cache of the documets
+ present on the content server.</para>
+
+ <section>
+ <title>Cached set</title>
+
+ <para>If the amount of memory and / or disk available to
+ Varnish is not sufficient to cache the entire document set,
+ Varnish must attempt to identify a subset to cache which
+ minimizes load on the content servers.</para>
+
+ <para>Varnish should offer multiple alternative cache control
+ algorithms. At the very least, the LRU (least-recently-used)
+ and WLRU (LRU weighted by document size) algorithms should be
+ implemented.</para>
+ </section>
+
+ <section>
+ <title>Expiry</title>
+
+ <para>If a cached document has an expiry time associated with
+ it, and that time has not yet been reached, Varnish may serve
+ the document from cache without contacting the content
+ server.</para>
+ </section>
+
+ <section>
+ <title>Non-cacheable content</title>
+
+ <para>XXX</para>
+ </section>
+ </section>
+
+ <section>
+ <title>Management</title>
+
+ <section>
+ <title>Configuration</title>
+
+ <para>XXX</para>
+ </section>
+
+ <section>
+ <title>Logging and statistics</title>
+
+ <para>A separate application is responsible for collecting,
+ collating and analyzing log data which Varnish makes available
+ in circular shared memory buffers.</para>
+
+ <para>Varnish must provide the data necessary to compute
+ lifetime totals and sliding averages for the following:</para>
+ <orderedlist>
+ <listitem>
+ <para>Total size of documents served to clients</para>
+ </listitem>
+ <listitem>
+ <para>Total size of data transmitted to clients, including
+ headers, error messages, etc.</para>
+ </listitem>
+ <listitem>
+ <para>Total size of data received from clients, including
+ request headers etc.</para>
+ </listitem>
+ <listitem>
+ <para>Number of client connections received</para>
+ </listitem>
+ <listitem>
+ <para>Number of client requests served</para>
+ </listitem>
+ <listitem>
+ <para>Client requests broken down by result code</para>
+ </listitem>
+ <listitem>
+ <para>Total size of documents retrieved from content
+ servers</para>
+ </listitem>
+ <listitem>
+ <para>Total size of data received from content servers,
+ including headers, error messages, etc.</para>
+ </listitem>
+ <listitem>
+ <para>Total size of data sent to content servers,
+ including request headers etc.</para>
+ </listitem>
+ <listitem>
+ <para>Number of content server connections
+ initiated</para>
+ </listitem>
+ <listitem>
+ <para>Number of content server requests sent</para>
+ </listitem>
+ <listitem>
+ <para>Content server requests broken down by result
+ code</para>
+ </listitem>
+ <listitem>
+ <para>Cache effectiveness as the ratio of bytes served to
+ clients to bytes requested from content servers</para>
+ </listitem>
+ <listitem>
+ <para>Cache effectiveness as the ratio of client requests
+ to content server requests</para>
+ </listitem>
+ <listitem>
+ <para>Number of active server processes / threads, broken
+ down by process / thread type</para>
+ </listitem>
+ <listitem>
+ <para>XXX length of request queues</para>
+ </listitem>
+ </orderedlist>
+
+ <para>In addition, Varnish must provide the data necessary to
+ compute the average, median and distribution for the
+ following:</para>
+ <orderedlist>
+ <listitem>
+ <para>Size of documents served, per unique document</para>
+ </listitem>
+ <listitem>
+ <para>Size of documents served, per request</para>
+ </listitem>
+ <listitem>
+ <para>Client connection duration</para>
+ </listitem>
+ <listitem>
+ <para>Requests per client connection</para>
+ </listitem>
+ <listitem>
+ <para>Client request completion time</para>
+ </listitem>
+ <listitem>
+ <para>Content server connection duration</para>
+ </listitem>
+ <listitem>
+ <para>Requests per content server connection</para>
+ </listitem>
+ <listitem>
+ <para>Content server request completion time</para>
+ </listitem>
+ <listitem>
+ <para>XXX time spent in request queues</para>
+ </listitem>
+ </orderedlist>
+ </section>
+ </section>
+
+ <section>
+ <title>Run-time monitoring and tuning</title>
+
+ <para>Varnish must provide low-level monitoring and tuning
+ facilities. A separate application is responsible for providing
+ a user-friendly interface to these facilities.</para>
+
+ <para>The following monitoring operations must be
+ supported:</para>
+
+ <orderedlist>
+ <listitem>
+ <para>Cache status of individual documents</para>
+ </listitem>
+ <listitem>
+ <para>Cache status of documents matching a glob or regular
+ expression</para>
+ </listitem>
+ <listitem>
+ <para>Access statistics of individual documents</para>
+ </listitem>
+ <listitem>
+ <para>Access statistics of documents matching a glob or
+ regular expression</para>
+ </listitem>
+ <listitem>
+ <para>XXX</para>
+ </listitem>
+ </orderedlist>
+
+ <para>The following tuning operations must be supported:</para>
+
+ <orderedlist>
+ <listitem>
+ <para>Forced invalidation of individual documents</para>
+ </listitem>
+ <listitem>
+ <para>Forced invalidation of documents matching a glob or regular expression</para>
+ </listitem>
+ <listitem>
+ <para>XXX</para>
+ </listitem>
+ </orderedlist>
+ </section>
+ <section>
+ <title>Clustering</title>
+
+ <para>Clustering is defined in this context as a situation where
+ multiple servers are set up to run Varnish with the same
+ configuration, serving data from the same content servers to the
+ same set of clients.</para>
+
+ <section>
+ <title>Configuration</title>
+
+ <para>When multiple Varnish servers act together as a cluster,
+ the configuration facility is responsible for ensuring that
+ all nodes share the same configuration and that configuration
+ changes are applied to all nodes in a timely fashion.</para>
+ </section>
+
+ <section>
+ <title>Logging and statistics</title>
+
+ <para>When multiple Varnish servers act together as a cluster,
+ the logging and statistics facilities must base its reports on
+ aggregated data as if the cluster were a single Varnish
+ server.</para>
+
+ <para>Per-node data may optionally be made available in
+ addition to aggregated data.</para>
+ </section>
+
+ <section>
+ <title>Run-time monitoring and tuning</title>
+
+ <para>When multiple Varnish servers act together as a cluster,
+ the run-time monitoring and tuning facilities must propagate
+ invalidation requests and other administrative commands to all
+ servers in the cluster.</para>
+ </section>
+ </section>
+ </section>
+
+ <section>
+ <title>Application structure</title>
+
+ <section>
+ <title>Components</title>
+
+ <para>This section lists the major components in Varnish.</para>
+
+ <section>
+ <title>Listener</title>
+
+ <para>The Listener monitors the listening socket and accepts
+ incoming client connections. Once the connection is
+ established, it is passed to the Accepter.</para>
+
+ <para>The Listener should take advantage of accept filters or
+ similar technologies on systems where they are
+ available.</para>
+ </section>
+
+ <section>
+ <title>Accepter</title>
+
+ <para>The Accepter reads an HTTP request from a client
+ connection. It parses the request line and header only to the
+ extent necessary to establish well-formedness and determine
+ the requested URL.</para>
+
+ <para>The Accepter then queries the Keeper about the status of
+ the requested document (identified by its full URL). If the
+ document is present and valid in the cache, the request is
+ passed directly to a Sender. Otherwise, it is passed to a
+ Retriever queue.</para>
+ </section>
+
+ <section>
+ <title>Keeper</title>
+
+ <para>The Keeper manages the document cache. XXX</para>
+ </section>
+
+ <section>
+ <title>Sender</title>
+
+ <para>The Sender transfers the contents of the requested
+ document to the client. It examines the HTTP request header
+ to determine the correct way in which to do this ? Range,
+ If-Modified-Since, Content-Encoding and other options may
+ affect the type and amount of data transferred.</para>
+
+ <para>There may be multiple concurrent Sender threads.</para>
+ </section>
+
+ <section>
+ <title>Retriever</title>
+
+ <para>The Retriever is responsible for retrieving documents
+ from the content servers. It is triggered either by an
+ Accepter trying to satisfy a request for a document which is
+ not in the cache, or by the Janitor when a ?hot? document is
+ nearing expiry. Either way, there may be a queue of requests
+ waiting for the document to arrive; when it does, the
+ Retriever passes those requests to a Sender.</para>
+
+ <para>There may be multiple concurrent Retriever
+ threads.</para>
+ </section>
+
+ <section>
+ <title>Janitor</title>
+
+ <para>The Janitor keeps track of the expiry time of cached
+ documents and attempts to retrieve fresh copies of documents
+ which are soon to expire.</para>
+ </section>
+
+ <section>
+ <title>Logger</title>
+
+ <para>The Logger keeps logs of various types of events in
+ circular shared-memory buffers. These buffers must be managed
+ using either POSIX shared memory primitives or file-backed
+ mmap(2).</para>
+
+ <para>It is the responsibility of each module to feed relevant
+ log data to the Logger.</para>
+ </section>
+ </section>
+ </section>
+
+ <bibliography>
+ <title>References</title>
+
+ <biblioentry id="RFC2119"/>
+ <biblioentry id="RFC2186"/>
+ <biblioentry id="RFC2616"/>
+ <biblioentry id="RFC2756"/>
+ </bibliography>
+</article>
Property changes on: trunk/varnish-doc/en/varnish-specification/article.xml
___________________________________________________________________
Name: svn:mime-type
+ text/xml
Name: svn:charset
+ utf-8