Mailing List Archive

Author: des
Date: 2007-07-13 16:54:51 +0200 (Fri, 13 Jul 2007)
New Revision: 1693

Added:
branches/1.1/lib/libvcl/vcc_string.c
Modified:
branches/1.1/
branches/1.1/bin/varnishd/Makefile.am
branches/1.1/bin/varnishd/cache.h
branches/1.1/bin/varnishd/cache_acceptor.c
branches/1.1/bin/varnishd/cache_acceptor.h
branches/1.1/bin/varnishd/cache_acceptor_epoll.c
branches/1.1/bin/varnishd/cache_acceptor_kqueue.c
branches/1.1/bin/varnishd/cache_acceptor_poll.c
branches/1.1/bin/varnishd/cache_backend.c
branches/1.1/bin/varnishd/cache_center.c
branches/1.1/bin/varnishd/cache_expire.c
branches/1.1/bin/varnishd/cache_fetch.c
branches/1.1/bin/varnishd/cache_hash.c
branches/1.1/bin/varnishd/cache_http.c
branches/1.1/bin/varnishd/cache_lru.c
branches/1.1/bin/varnishd/cache_main.c
branches/1.1/bin/varnishd/cache_pipe.c
branches/1.1/bin/varnishd/cache_pool.c
branches/1.1/bin/varnishd/cache_response.c
branches/1.1/bin/varnishd/cache_session.c
branches/1.1/bin/varnishd/cache_synthetic.c
branches/1.1/bin/varnishd/cache_vrt.c
branches/1.1/bin/varnishd/cache_vrt_re.c
branches/1.1/bin/varnishd/mgt_child.c
branches/1.1/bin/varnishd/mgt_cli.c
branches/1.1/bin/varnishd/mgt_cli.h
branches/1.1/bin/varnishd/mgt_event.c
branches/1.1/bin/varnishd/rfc2616.c
branches/1.1/bin/varnishd/shmlog.c
branches/1.1/bin/varnishd/varnishd.1
branches/1.1/doc/changes-1.0.4-1.1.xml
branches/1.1/include/cli.h
branches/1.1/include/libvarnish.h
branches/1.1/include/shmlog_tags.h
branches/1.1/include/vrt.h
branches/1.1/include/vrt_obj.h
branches/1.1/lib/libvarnish/Makefile.am
branches/1.1/lib/libvarnish/time.c
branches/1.1/lib/libvcl/Makefile.am
branches/1.1/lib/libvcl/syntax.txt
branches/1.1/lib/libvcl/vcc_acl.c
branches/1.1/lib/libvcl/vcc_action.c
branches/1.1/lib/libvcl/vcc_compile.c
branches/1.1/lib/libvcl/vcc_compile.h
branches/1.1/lib/libvcl/vcc_fixed_token.c
branches/1.1/lib/libvcl/vcc_gen_obj.tcl
branches/1.1/lib/libvcl/vcc_obj.c
branches/1.1/lib/libvcl/vcc_parse.c
branches/1.1/lib/libvcl/vcc_var.c
branches/1.1/lib/libvcl/vcc_xref.c
branches/1.1/man/vcl.7
Log:
Merged revisions 1649-1650,1655-1692 via svnmerge from
svn+ssh://projects.linpro.no/svn/varnish/trunk/varnish-cache

........
r1657 | phk | 2007-07-05 23:08:15 +0200 (Thu, 05 Jul 2007) | 2 lines

Clean up FlexeLint fluff.
........
r1658 | phk | 2007-07-06 12:07:30 +0200 (Fri, 06 Jul 2007) | 3 lines

Don't rewrite pipe'ed requests to "GET".
........
r1659 | phk | 2007-07-09 22:23:41 +0200 (Mon, 09 Jul 2007) | 10 lines

Make all protocol header fields writable, except obj.status and resp.status
(which are numeric, they'll follow shortly)

Unify the shmemlog tag used for failure to rewrite something, the Rx/Tx/Obj
distinction is not helpful enough to warrant the complexity of it.
........
r1660 | phk | 2007-07-09 22:34:59 +0200 (Mon, 09 Jul 2007) | 2 lines

Allow assignment to INT type variables
........
r1661 | phk | 2007-07-09 22:35:20 +0200 (Mon, 09 Jul 2007) | 2 lines

Allow assignment to obj.status and resp.status
........
r1662 | phk | 2007-07-10 21:46:16 +0200 (Tue, 10 Jul 2007) | 6 lines

Move string stuff to vcc_string.c, there's going to be a fair bit of it.

Give vcc_StringVal() a return value to say if it did anything so we can
emit better error messages when confused.
........
r1663 | phk | 2007-07-10 21:59:39 +0200 (Tue, 10 Jul 2007) | 5 lines

Add conversion from IP to string format to allow things like:

set bereq.http.HeyYou = client.ip " asked for " req.url;
........
r1664 | phk | 2007-07-10 22:07:07 +0200 (Tue, 10 Jul 2007) | 3 lines

Properly emit the header name in VRT_SetHdr();
........
r1665 | phk | 2007-07-10 22:08:39 +0200 (Tue, 10 Jul 2007) | 2 lines

Fix VRT_SetHdr() prototype
........
r1666 | phk | 2007-07-10 22:43:24 +0200 (Tue, 10 Jul 2007) | 2 lines

Add compiler side support for regsub() but only a dummy function in VRT.
........
r1667 | phk | 2007-07-10 23:30:47 +0200 (Tue, 10 Jul 2007) | 60 lines

Add "regsub" support for string manipulation.

Notice this facility is subject to change!

"regsub" is short for regular expression substitution and it is probably
easiest to explain with some examples:

sub vcl_recv {
set req.url = regsub(req.url, "#.*", "");
}

This will replace the requests URL with the output of the regsub() function

regsub() takes three arguments: the string to be examined, a regular
expression and a replacement string.

In this case, everything after the first '#' is removed (replaced
with nothing).

The replacement string recognizes the following magic sequences:
& - insert everything matched by the regexp
$0 - ditto.
$1 - replace with the first submatch of the regexp
$2 - replace with the second submatch of the regexp
...
$9 - replace with the ninth submatch of the regexp

(The $0..$9 syntax was chosen over the \0...\9 syntax in order to avoid
a nightmare of escape characters in the VCL source code. Arguments and
suggestions are welcome).

A more advanced example:

set bereq.http.ClientIP = regsub(client.ip, "(.*):(.*)", "$2 $1");

The client.ip variable expands to IP:port number, for instance
127.0.0.1:54662

The regular expression "(.*):(.*)" results in the the following matches:
& + $0 "127.0.0.1:54662"
$1 "127.0.0.1"
$2 "54662"

So the replacement string "$2 $1" results in "54662 127.0.0.1"

And the completed header which is sent to the backend will look like:

"ClientIP: 54662 127.0.0.1"

An even more advanced example would be:

set bereq.http.magic = "Client IP = " regsub(client.ip, ":", " port = ");

Where we also exploint the string concatenation ability of the "set" statement.

The result string is built in the request workspace, so you may need
to increase the workspace size if you do a lot of regsub()'s.

Currently there is no decent error handling for running out of workspace.
........
r1668 | phk | 2007-07-12 11:04:54 +0200 (Thu, 12 Jul 2007) | 6 lines

Add TIM_mono() and TIM_real() which return double representations of
timestamps on a monotonic and the UTC timescales respectively.

Doubles are much more convenient than timespecs for comparisons etc.
........
r1669 | phk | 2007-07-12 11:25:07 +0200 (Thu, 12 Jul 2007) | 2 lines

Replace ev_now() with TIM_mono().
........
r1670 | phk | 2007-07-12 11:25:45 +0200 (Thu, 12 Jul 2007) | 2 lines

Replace Uptime() with TIM_mono()
........
r1671 | phk | 2007-07-12 11:49:26 +0200 (Thu, 12 Jul 2007) | 6 lines

Change all timekeeping to use doubles instead of time_t and struct timespec.

Eliminate all direct calls to time(2) and clockgettime(2) and use TIM_real()
and TIM_mono() exclusively.
........
r1672 | phk | 2007-07-12 12:00:13 +0200 (Thu, 12 Jul 2007) | 2 lines

Document timescale of srcaddr->ttl
........
r1673 | phk | 2007-07-12 12:13:29 +0200 (Thu, 12 Jul 2007) | 2 lines

Convert the last time(2) calls to TIM_real()
........
r1674 | cecilihf | 2007-07-12 12:20:33 +0200 (Thu, 12 Jul 2007) | 2 lines

Added a new cli option, status, for checking the status of the varnish child process. This is for use in the webmin plugin.
........
r1675 | cecilihf | 2007-07-12 12:27:37 +0200 (Thu, 12 Jul 2007) | 2 lines

Fixed typo
........
r1676 | des | 2007-07-12 18:00:04 +0200 (Thu, 12 Jul 2007) | 2 lines

RT_LIBS dependency has moved from varnishd to libvarnish.
........
r1677 | des | 2007-07-12 18:02:47 +0200 (Thu, 12 Jul 2007) | 2 lines

Add missing semicolon.
........
r1678 | des | 2007-07-12 19:37:44 +0200 (Thu, 12 Jul 2007) | 2 lines

sockaddr.sa_len is not portable.
........
r1682 | phk | 2007-07-13 09:11:54 +0200 (Fri, 13 Jul 2007) | 2 lines

Initialize all timestamps in the session to NAN
........
r1683 | phk | 2007-07-13 09:21:46 +0200 (Fri, 13 Jul 2007) | 4 lines

Unify the recycle functionality of the acceptors, all three used the same
method.
........
r1684 | phk | 2007-07-13 09:27:50 +0200 (Fri, 13 Jul 2007) | 2 lines

Clean all but t_open timestamps to NAN at end of transaction.
........
r1685 | phk | 2007-07-13 09:47:45 +0200 (Fri, 13 Jul 2007) | 9 lines

Rename the "idle" field of struct worker to "used", which is more precise.

Don't use the "used" field to signal suicide for worker threads,
use the "wrq" field which is much more natural.

Set the "used" field to NAN before doing anything and assert that
somebody updated during the task.
........
r1686 | phk | 2007-07-13 09:53:08 +0200 (Fri, 13 Jul 2007) | 2 lines

Clarify XXX comment
........
r1687 | phk | 2007-07-13 09:58:11 +0200 (Fri, 13 Jul 2007) | 3 lines

Move setting of t_resp up to before we build the response.
........
r1688 | phk | 2007-07-13 10:05:14 +0200 (Fri, 13 Jul 2007) | 2 lines

Add an XXX comment
........
r1690 | des | 2007-07-13 13:42:02 +0200 (Fri, 13 Jul 2007) | 2 lines

Add an entry for r1531.
........
r1691 | des | 2007-07-13 16:27:55 +0200 (Fri, 13 Jul 2007) | 2 lines

Document regsub().
........
r1692 | des | 2007-07-13 16:53:48 +0200 (Fri, 13 Jul 2007) | 2 lines

Document recent changes.
........



Property changes on: branches/1.1
___________________________________________________________________
Name: svnmerge-integrated
- /trunk/varnish-cache:1-1648,1651-1654
+ /trunk/varnish-cache:1-1692

Modified: branches/1.1/bin/varnishd/Makefile.am
===================================================================
--- branches/1.1/bin/varnishd/Makefile.am 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/Makefile.am 2007-07-13 14:54:51 UTC (rev 1693)
@@ -68,4 +68,4 @@
$(top_builddir)/lib/libvarnish/libvarnish.la \
$(top_builddir)/lib/libcompat/libcompat.a \
$(top_builddir)/lib/libvcl/libvcl.la \
- ${DL_LIBS} ${RT_LIBS} ${PTHREAD_LIBS}
+ ${DL_LIBS} ${PTHREAD_LIBS}

Modified: branches/1.1/bin/varnishd/cache.h
===================================================================
--- branches/1.1/bin/varnishd/cache.h 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/cache.h 2007-07-13 14:54:51 UTC (rev 1693)
@@ -131,7 +131,7 @@
/*--------------------------------------------------------------------*/

struct acct {
- time_t first;
+ double first;
uint64_t sess;
uint64_t req;
uint64_t pipe;
@@ -149,7 +149,7 @@
struct objhead *nobjhead;
struct object *nobj;

- time_t idle;
+ double used;

int pipe[2];

@@ -247,11 +247,11 @@
unsigned busy;
unsigned len;

- time_t age;
- time_t entered;
- time_t ttl;
+ double age;
+ double entered;
+ double ttl;

- time_t last_modified;
+ double last_modified;

struct http http;
TAILQ_ENTRY(object) list;
@@ -262,7 +262,7 @@

TAILQ_HEAD(, sess) waitinglist;

- time_t lru_stamp;
+ double lru_stamp;
TAILQ_ENTRY(object) lru;
};

@@ -300,10 +300,11 @@
const char *doclose;
struct http *http;

- struct timespec t_open;
- struct timespec t_req;
- struct timespec t_resp;
- struct timespec t_end;
+ /* Timestamps, all on TIM_real() timescale */
+ double t_open;
+ double t_req;
+ double t_resp;
+ double t_end;

enum step step;
unsigned handling;
@@ -359,6 +360,7 @@
void vca_close_session(struct sess *sp, const char *why);
void VCA_Prep(struct sess *sp);
void VCA_Init(void);
+extern int vca_pipes[2];

/* cache_backend.c */
void VBE_Init(void);
@@ -412,6 +414,7 @@
void http_PutResponse(struct worker *w, int fd, struct http *to, const char *response);
void http_PrintfHeader(struct worker *w, int fd, struct http *to, const char *fmt, ...);
void http_SetHeader(struct worker *w, int fd, struct http *to, const char *hdr);
+void http_SetH(struct http *to, unsigned n, const char *fm);
void http_Setup(struct http *ht, void *space, unsigned len);
int http_GetHdr(struct http *hp, const char *hdr, char **ptr);
int http_GetHdrField(struct http *hp, const char *hdr, const char *field, char **ptr);
@@ -428,7 +431,6 @@
void http_DoConnection(struct sess *sp);
void http_CopyHome(struct worker *w, int fd, struct http *hp);
void http_Unset(struct http *hp, const char *hdr);
-void http_LogLostHeader(struct worker *w, int fd, struct http *hp, const char *hdr);


#define HTTPH(a, b, c, d, e, f, g) extern char b[];
@@ -492,11 +494,11 @@

/* cache_lru.c */
// void LRU_Init(void);
-void LRU_Enter(struct object *o, time_t stamp);
+void LRU_Enter(struct object *o, double stamp);
void LRU_Remove(struct object *o);
int LRU_DiscardOne(void);
int LRU_DiscardSpace(int64_t quota);
-int LRU_DiscardTime(time_t cutoff);
+int LRU_DiscardTime(double cutoff);

#define VCL_RET_MAC(l,u,b,n)
#define VCL_MET_MAC(l,u,b) void VCL_##l##_method(struct sess *);

Modified: branches/1.1/bin/varnishd/cache_acceptor.c
===================================================================
--- branches/1.1/bin/varnishd/cache_acceptor.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/cache_acceptor.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -44,10 +44,6 @@
#include <sys/types.h>
#include <sys/socket.h>

-#ifndef HAVE_CLOCK_GETTIME
-#include "compat/clock_gettime.h"
-#endif
-
#ifndef HAVE_SRANDOMDEV
#include "compat/srandomdev.h"
#endif
@@ -80,6 +76,8 @@

static unsigned char need_sndtimeo, need_rcvtimeo, need_linger, need_test;

+int vca_pipes[2];
+
static void
sock_test(int fd)
{
@@ -116,7 +114,7 @@
TCP_name(sp->sockaddr, sp->sockaddrlen,
sp->addr, sizeof sp->addr, sp->port, sizeof sp->port);
VSL(SLT_SessionOpen, sp->fd, "%s %s", sp->addr, sp->port);
- sp->acct.first = sp->t_open.tv_sec;
+ sp->acct.first = sp->t_open;
if (need_test)
sock_test(sp->fd);
if (need_linger)
@@ -195,7 +193,7 @@

sp->fd = i;
sp->id = i;
- (void)clock_gettime(CLOCK_REALTIME, &sp->t_open);
+ sp->t_open = TIM_real();

http_RecvPrep(sp->http);
sp->step = STP_FIRST;
@@ -259,7 +257,10 @@
CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
AZ(sp->obj);
AZ(sp->vcl);
- vca_act->recycle(sp);
+ if (sp->fd < 0)
+ SES_Delete(sp);
+ else
+ assert(sizeof sp == write(vca_pipes[1], &sp, sizeof sp));
}


@@ -277,6 +278,7 @@
fprintf(stderr, "No acceptor in program\n");
exit (2);
}
+ AZ(pipe(vca_pipes));
vca_act->init();
AZ(pthread_create(&vca_thread_acct, NULL, vca_acct, NULL));
}

Modified: branches/1.1/bin/varnishd/cache_acceptor.h
===================================================================
--- branches/1.1/bin/varnishd/cache_acceptor.h 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/cache_acceptor.h 2007-07-13 14:54:51 UTC (rev 1693)
@@ -32,12 +32,10 @@
struct sess;

typedef void acceptor_init_f(void);
-typedef void acceptor_recycle_f(struct sess *);

struct acceptor {
const char *name;
acceptor_init_f *init;
- acceptor_recycle_f *recycle;
};

#if defined(HAVE_EPOLL_CTL)

Modified: branches/1.1/bin/varnishd/cache_acceptor_epoll.c
===================================================================
--- branches/1.1/bin/varnishd/cache_acceptor_epoll.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/cache_acceptor_epoll.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -41,10 +41,6 @@

#include <sys/epoll.h>

-#ifndef HAVE_CLOCK_GETTIME
-#include "compat/clock_gettime.h"
-#endif
-
#include "heritage.h"
#include "shmlog.h"
#include "cache.h"
@@ -52,7 +48,6 @@

static pthread_t vca_epoll_thread;
static int epfd = -1;
-static int pipes[2];

static TAILQ_HEAD(,sess) sesshead = TAILQ_HEAD_INITIALIZER(sesshead);

@@ -74,7 +69,7 @@
vca_main(void *arg)
{
struct epoll_event ev;
- struct timespec ts;
+ double deadline;
struct sess *sp, *sp2;
int i;

@@ -83,12 +78,12 @@
epfd = epoll_create(16);
assert(epfd >= 0);

- vca_add(pipes[0], pipes);
+ vca_add(vca_pipes[0], vca_pipes);

while (1) {
if (epoll_wait(epfd, &ev, 1, 100) > 0) {
- if (ev.data.ptr == pipes) {
- i = read(pipes[0], &sp, sizeof sp);
+ if (ev.data.ptr == vca_pipes) {
+ i = read(vca_pipes[0], &sp, sizeof sp);
assert(i == sizeof sp);
CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
TAILQ_INSERT_TAIL(&sesshead, sp, list);
@@ -108,15 +103,11 @@
}
}
/* check for timeouts */
- clock_gettime(CLOCK_REALTIME, &ts);
- ts.tv_sec -= params->sess_timeout;
+ deadline = TIM_real() - params->sess_timeout;
TAILQ_FOREACH_SAFE(sp, &sesshead, list, sp2) {
CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
- if (sp->t_open.tv_sec > ts.tv_sec)
+ if (sp->t_open > deadline)
continue;
- if (sp->t_open.tv_sec == ts.tv_sec &&
- sp->t_open.tv_nsec > ts.tv_nsec)
- continue;
TAILQ_REMOVE(&sesshead, sp, list);
vca_del(sp->fd);
vca_close_session(sp, "timeout");
@@ -128,27 +119,15 @@
/*--------------------------------------------------------------------*/

static void
-vca_epoll_recycle(struct sess *sp)
-{
-
- if (sp->fd < 0)
- SES_Delete(sp);
- else
- assert(sizeof sp == write(pipes[1], &sp, sizeof sp));
-}
-
-static void
vca_epoll_init(void)
{

- AZ(pipe(pipes));
AZ(pthread_create(&vca_epoll_thread, NULL, vca_main, NULL));
}

struct acceptor acceptor_epoll = {
.name = "epoll",
.init = vca_epoll_init,
- .recycle = vca_epoll_recycle,
};

#endif /* defined(HAVE_EPOLL_CTL) */

Modified: branches/1.1/bin/varnishd/cache_acceptor_kqueue.c
===================================================================
--- branches/1.1/bin/varnishd/cache_acceptor_kqueue.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/cache_acceptor_kqueue.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -43,10 +43,6 @@

#include <sys/event.h>

-#ifndef HAVE_CLOCK_GETTIME
-#include "compat/clock_gettime.h"
-#endif
-
#include "heritage.h"
#include "shmlog.h"
#include "cache.h"
@@ -56,7 +52,6 @@
static int kq = -1;

static TAILQ_HEAD(,sess) sesshead = TAILQ_HEAD_INITIALIZER(sesshead);
-static int pipes[2];

#define NKEV 100

@@ -85,9 +80,9 @@
struct sess *ss[NKEV];

AN(kp->udata);
- if (kp->udata == pipes) {
+ if (kp->udata == vca_pipes) {
j = 0;
- i = read(pipes[0], ss, sizeof ss);
+ i = read(vca_pipes[0], ss, sizeof ss);
if (i == -1 && errno == EAGAIN)
return;
while (i >= sizeof ss[0]) {
@@ -129,7 +124,7 @@
{
struct kevent ke[NKEV], *kp;
int j, n, dotimer;
- struct timespec ts;
+ double deadline;
struct sess *sp;

(void)arg;
@@ -139,7 +134,7 @@

j = 0;
EV_SET(&ke[j++], 0, EVFILT_TIMER, EV_ADD, 0, 100, NULL);
- EV_SET(&ke[j++], pipes[0], EVFILT_READ, EV_ADD, 0, 0, pipes);
+ EV_SET(&ke[j++], vca_pipes[0], EVFILT_READ, EV_ADD, 0, 0, vca_pipes);
AZ(kevent(kq, ke, j, NULL, 0, NULL));

nki = 0;
@@ -160,17 +155,13 @@
}
if (!dotimer)
continue;
- clock_gettime(CLOCK_REALTIME, &ts);
- ts.tv_sec -= params->sess_timeout;
+ deadline = TIM_real() - params->sess_timeout;
for (;;) {
sp = TAILQ_FIRST(&sesshead);
if (sp == NULL)
break;
- if (sp->t_open.tv_sec > ts.tv_sec)
+ if (sp->t_open > deadline)
break;
- if (sp->t_open.tv_sec == ts.tv_sec &&
- sp->t_open.tv_nsec > ts.tv_nsec)
- break;
TAILQ_REMOVE(&sesshead, sp, list);
vca_close_session(sp, "timeout");
SES_Delete(sp);
@@ -181,24 +172,13 @@
/*--------------------------------------------------------------------*/

static void
-vca_kqueue_recycle(struct sess *sp)
-{
-
- if (sp->fd < 0)
- SES_Delete(sp);
- else
- assert(write(pipes[1], &sp, sizeof sp) == sizeof sp);
-}
-
-static void
vca_kqueue_init(void)
{
int i;

- AZ(pipe(pipes));
- i = fcntl(pipes[0], F_GETFL);
+ i = fcntl(vca_pipes[0], F_GETFL);
i |= O_NONBLOCK;
- i = fcntl(pipes[0], F_SETFL, i);
+ i = fcntl(vca_pipes[0], F_SETFL, i);

AZ(pthread_create(&vca_kqueue_thread, NULL, vca_kqueue_main, NULL));
}
@@ -206,7 +186,6 @@
struct acceptor acceptor_kqueue = {
.name = "kqueue",
.init = vca_kqueue_init,
- .recycle = vca_kqueue_recycle,
};

#endif /* defined(HAVE_KQUEUE) */

Modified: branches/1.1/bin/varnishd/cache_acceptor_poll.c
===================================================================
--- branches/1.1/bin/varnishd/cache_acceptor_poll.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/cache_acceptor_poll.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -42,10 +42,6 @@
#include <unistd.h>
#include <poll.h>

-#ifndef HAVE_CLOCK_GETTIME
-#include "compat/clock_gettime.h"
-#endif
-
#include "heritage.h"
#include "shmlog.h"
#include "cache.h"
@@ -55,8 +51,6 @@
static struct pollfd *pollfd;
static unsigned npoll;

-static int pipes[2];
-
static TAILQ_HEAD(,sess) sesshead = TAILQ_HEAD_INITIALIZER(sesshead);

/*--------------------------------------------------------------------*/
@@ -108,25 +102,24 @@
{
unsigned v;
struct sess *sp, *sp2;
- struct timespec ts;
+ double deadline;
int i, fd;

(void)arg;

- vca_poll(pipes[0]);
+ vca_poll(vca_pipes[0]);

while (1) {
v = poll(pollfd, npoll, 100);
- if (v && pollfd[pipes[0]].revents) {
+ if (v && pollfd[vca_pipes[0]].revents) {
v--;
- i = read(pipes[0], &sp, sizeof sp);
+ i = read(vca_pipes[0], &sp, sizeof sp);
assert(i == sizeof sp);
CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
TAILQ_INSERT_TAIL(&sesshead, sp, list);
vca_poll(sp->fd);
}
- clock_gettime(CLOCK_REALTIME, &ts);
- ts.tv_sec -= params->sess_timeout;
+ deadline = TIM_real() - params->sess_timeout;
TAILQ_FOREACH_SAFE(sp, &sesshead, list, sp2) {
if (v == 0)
break;
@@ -145,11 +138,8 @@
SES_Delete(sp);
continue;
}
- if (sp->t_open.tv_sec > ts.tv_sec)
+ if (sp->t_open > deadline)
continue;
- if (sp->t_open.tv_sec == ts.tv_sec &&
- sp->t_open.tv_nsec > ts.tv_nsec)
- continue;
TAILQ_REMOVE(&sesshead, sp, list);
vca_unpoll(fd);
vca_close_session(sp, "timeout");
@@ -161,26 +151,15 @@
/*--------------------------------------------------------------------*/

static void
-vca_poll_recycle(struct sess *sp)
-{
-
- if (sp->fd < 0)
- SES_Delete(sp);
- else
- assert(sizeof sp == write(pipes[1], &sp, sizeof sp));
-}
-
-static void
vca_poll_init(void)
{
- AZ(pipe(pipes));
+
AZ(pthread_create(&vca_poll_thread, NULL, vca_main, NULL));
}

struct acceptor acceptor_poll = {
.name = "poll",
.init = vca_poll_init,
- .recycle = vca_poll_recycle,
};

#endif /* defined(HAVE_POLL) */

Modified: branches/1.1/bin/varnishd/cache_backend.c
===================================================================
--- branches/1.1/bin/varnishd/cache_backend.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/cache_backend.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -48,10 +48,6 @@
#include <sys/select.h>
#include <sys/ioctl.h>

-#ifndef HAVE_CLOCK_GETTIME
-#include "compat/clock_gettime.h"
-#endif
-
#include "heritage.h"
#include "shmlog.h"
#include "cache.h"
@@ -64,19 +60,7 @@
static MTX vbemtx;

/*--------------------------------------------------------------------*/
-/* XXX: belongs a more general place */

-static double
-Uptime(void)
-{
- struct timespec ts;
-
- assert(clock_gettime(CLOCK_MONOTONIC, &ts) == 0);
- return (ts.tv_sec + ts.tv_nsec * 1e-9);
-}
-
-/*--------------------------------------------------------------------*/
-
struct bereq *
vbe_new_bereq(void)
{
@@ -153,7 +137,7 @@
error = getaddrinfo(bp->hostname,
bp->portname == NULL ? "http" : bp->portname,
&hint, &res);
- bp->dnstime = Uptime();
+ bp->dnstime = TIM_mono();
if (error) {
if (res != NULL)
freeaddrinfo(res);
@@ -207,7 +191,7 @@
}
}

- if (bp->dnstime + bp->dnsttl >= Uptime())
+ if (bp->dnstime + bp->dnsttl >= TIM_mono())
return (-1);

/* Then do another lookup to catch DNS changes */

Modified: branches/1.1/bin/varnishd/cache_center.c
===================================================================
--- branches/1.1/bin/varnishd/cache_center.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/cache_center.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -58,14 +58,11 @@

#include <stdio.h>
#include <errno.h>
+#include <math.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

-#ifndef HAVE_CLOCK_GETTIME
-#include "compat/clock_gettime.h"
-#endif
-
#ifndef HAVE_SRANDOMDEV
#include "compat/srandomdev.h"
#endif
@@ -149,6 +146,7 @@
cnt_deliver(struct sess *sp)
{

+ sp->t_resp = TIM_real();
RES_BuildHttp(sp);
VCL_deliver_method(sp);
if (sp->handling != VCL_RET_DELIVER)
@@ -172,16 +170,6 @@
DOT ]
*/

-static double
-cnt_dt(struct timespec *t1, struct timespec *t2)
-{
- double dt;
-
- dt = (t2->tv_sec - t1->tv_sec);
- dt += (t2->tv_nsec - t1->tv_nsec) * 1e-9;
- return (dt);
-}
-
static int
cnt_done(struct sess *sp)
{
@@ -197,24 +185,24 @@
sp->vcl = NULL;
}

- clock_gettime(CLOCK_REALTIME, &sp->t_end);
- sp->wrk->idle = sp->t_end.tv_sec;
+ sp->t_end = TIM_real();
+ sp->wrk->used = sp->t_end;
if (sp->xid == 0) {
sp->t_req = sp->t_end;
sp->t_resp = sp->t_end;
}
- dp = cnt_dt(&sp->t_req, &sp->t_resp);
- da = cnt_dt(&sp->t_resp, &sp->t_end);
- dh = cnt_dt(&sp->t_open, &sp->t_req);
- WSL(sp->wrk, SLT_ReqEnd, sp->id, "%u %ld.%09ld %ld.%09ld %.9f %.9f %.9f",
- sp->xid,
- (long)sp->t_req.tv_sec, (long)sp->t_req.tv_nsec,
- (long)sp->t_end.tv_sec, (long)sp->t_end.tv_nsec,
- dh, dp, da);
+ dp = sp->t_resp - sp->t_req;
+ da = sp->t_end - sp->t_resp;
+ dh = sp->t_req - sp->t_open;
+ WSL(sp->wrk, SLT_ReqEnd, sp->id, "%u %.9f %.9f %.9f %.9f %.9f",
+ sp->xid, sp->t_req, sp->t_end, dh, dp, da);

sp->xid = 0;
- sp->t_open = sp->t_end;
SES_Charge(sp);
+ sp->t_open = sp->t_end;
+ sp->t_req = NAN;
+ sp->t_resp = NAN;
+ sp->t_end = NAN;
WSL_Flush(sp->wrk);
if (sp->fd >= 0 && sp->doclose != NULL)
vca_close_session(sp, sp->doclose);
@@ -345,7 +333,7 @@

assert(sp->xid == 0);
VCA_Prep(sp);
- sp->wrk->idle = sp->t_open.tv_sec;
+ sp->wrk->used = sp->t_open;
sp->wrk->acct.sess++;
SES_RefSrcAddr(sp);
do
@@ -672,8 +660,8 @@

/* Update stats of various sorts */
VSL_stats->client_req++; /* XXX not locked */
- clock_gettime(CLOCK_REALTIME, &sp->t_req);
- sp->wrk->idle = sp->t_req.tv_sec;
+ sp->t_req = TIM_real();
+ sp->wrk->used = sp->t_req;
sp->wrk->acct.req++;

/* Assign XID and log */

Modified: branches/1.1/bin/varnishd/cache_expire.c
===================================================================
--- branches/1.1/bin/varnishd/cache_expire.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/cache_expire.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -103,11 +103,11 @@
exp_hangman(void *arg)
{
struct object *o;
- time_t t;
+ double t;

(void)arg;

- t = time(NULL);
+ t = TIM_real();
while (1) {
LOCK(&exp_mtx);
TAILQ_FOREACH(o, &exp_deathrow, deathrow) {
@@ -127,7 +127,7 @@
if (o == NULL) {
UNLOCK(&exp_mtx);
AZ(sleep(1));
- t = time(NULL);
+ t = TIM_real();
continue;
}
TAILQ_REMOVE(&exp_deathrow, o, deathrow);
@@ -153,7 +153,7 @@
{
struct worker ww;
struct object *o;
- time_t t;
+ double t;
struct sess *sp;
struct object *o2;

@@ -168,7 +168,7 @@

sleep(10); /* Takes time for VCL to arrive */
VCL_Get(&sp->vcl);
- t = time(NULL);
+ t = TIM_real();
while (1) {
LOCK(&exp_mtx);
o = binheap_root(exp_heap);
@@ -178,7 +178,7 @@
UNLOCK(&exp_mtx);
AZ(sleep(1));
VCL_Refresh(&sp->vcl);
- t = time(NULL);
+ t = TIM_real();
continue;
}
binheap_delete(exp_heap, o->heap_idx);

Modified: branches/1.1/bin/varnishd/cache_fetch.c
===================================================================
--- branches/1.1/bin/varnishd/cache_fetch.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/cache_fetch.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -303,7 +303,7 @@
CHECK_OBJ_NOTNULL(sp->wrk, WORKER_MAGIC);
CHECK_OBJ_NOTNULL(sp->obj, OBJECT_MAGIC);

- sp->obj->entered = time(NULL);
+ sp->obj->entered = TIM_real();

assert(sp->obj->busy != 0);


Modified: branches/1.1/bin/varnishd/cache_hash.c
===================================================================
--- branches/1.1/bin/varnishd/cache_hash.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/cache_hash.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -152,7 +152,7 @@
/* ignore */
} else if (o->ttl == 0) {
/* Object banned but not reaped yet */
- } else if (o->ttl <= sp->t_req.tv_sec) {
+ } else if (o->ttl <= sp->t_req) {
/* Object expired */
} else if (BAN_CheckObject(o, h->hd[HTTP_HDR_URL].b)) {
o->ttl = 0;
@@ -166,7 +166,7 @@
if (o != NULL) {
UNLOCK(&oh->mtx);
(void)hash->deref(oh);
- LRU_Enter(o, sp->t_req.tv_sec);
+ LRU_Enter(o, sp->t_req);
return (o);
}

@@ -178,7 +178,7 @@
/* NB: do not deref objhead the new object inherits our reference */
UNLOCK(&oh->mtx);
BAN_NewObj(o);
- LRU_Enter(o, sp->t_req.tv_sec);
+ LRU_Enter(o, sp->t_req);
return (o);
}


Modified: branches/1.1/bin/varnishd/cache_http.c
===================================================================
--- branches/1.1/bin/varnishd/cache_http.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/cache_http.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -58,7 +58,6 @@
HTTP_T_URL,
HTTP_T_Protocol,
HTTP_T_Header,
- HTTP_T_LostHeader,
};

#define LOGMTX2(ax, bx) \
@@ -71,7 +70,6 @@
LOGMTX2(ax, URL), \
LOGMTX2(ax, Protocol), \
LOGMTX2(ax, Header), \
- LOGMTX2(ax, LostHeader) \
}

static enum shmlogtag logmtx[3][7] = {
@@ -86,7 +84,7 @@

CHECK_OBJ_NOTNULL(hp, HTTP_MAGIC);
assert(/* hp->logtag >= HTTP_Rx && */hp->logtag <= HTTP_Obj);
- assert(/* t >= HTTP_T_Request && */t <= HTTP_T_LostHeader);
+ assert(/* t >= HTTP_T_Request && */t <= HTTP_T_Header);
return (logmtx[hp->logtag][t]);
}

@@ -97,12 +95,6 @@
WSLR(w, http2shmlog(hp, t), fd, hp->hd[hdr].b, hp->hd[hdr].e);
}

-void
-http_LogLostHeader(struct worker *w, int fd, struct http *hp, const char *hdr)
-{
- WSLR(w, http2shmlog(hp, HTTP_T_LostHeader), fd, hdr + 1, hdr + hdr[0]);
-}
-
/*--------------------------------------------------------------------*/
/* List of canonical HTTP response code names from RFC2616 */

@@ -430,7 +422,7 @@
hp->nhd++;
} else {
VSL_stats->losthdr++;
- WSLR(w, http2shmlog(hp, HTTP_T_LostHeader), fd, p, q);
+ WSLR(w, SLT_LostHeader, fd, p, q);
}
}
return (0);
@@ -686,8 +678,8 @@

/*--------------------------------------------------------------------*/

-static void
-http_seth(struct http *to, unsigned n, const char *fm)
+void
+http_SetH(struct http *to, unsigned n, const char *fm)
{

assert(n < HTTP_HDR_MAX);
@@ -709,14 +701,17 @@
}

static void
-http_getreq(struct http *to, struct http *fm)
+http_copyreq(struct http *to, struct http *fm, int forceget)
{

CHECK_OBJ_NOTNULL(fm, HTTP_MAGIC);
CHECK_OBJ_NOTNULL(to, HTTP_MAGIC);
- http_seth(to, HTTP_HDR_REQ, "GET");
+ if (forceget)
+ http_SetH(to, HTTP_HDR_REQ, "GET");
+ else
+ http_copyh(to, fm, HTTP_HDR_REQ);
http_copyh(to, fm, HTTP_HDR_URL);
- http_seth(to, HTTP_HDR_PROTO, "HTTP/1.1");
+ http_SetH(to, HTTP_HDR_PROTO, "HTTP/1.1");
}

void
@@ -726,7 +721,7 @@
CHECK_OBJ_NOTNULL(fm, HTTP_MAGIC);
CHECK_OBJ_NOTNULL(to, HTTP_MAGIC);
if (params->client_http11)
- http_seth(to, HTTP_HDR_PROTO, "HTTP/1.1");
+ http_SetH(to, HTTP_HDR_PROTO, "HTTP/1.1");
else
http_copyh(to, fm, HTTP_HDR_PROTO);
http_copyh(to, fm, HTTP_HDR_STATUS);
@@ -738,9 +733,9 @@
{

CHECK_OBJ_NOTNULL(to, HTTP_MAGIC);
- http_seth(to, HTTP_HDR_PROTO, proto);
- http_seth(to, HTTP_HDR_STATUS, status);
- http_seth(to, HTTP_HDR_RESPONSE, response);
+ http_SetH(to, HTTP_HDR_PROTO, proto);
+ http_SetH(to, HTTP_HDR_STATUS, status);
+ http_SetH(to, HTTP_HDR_RESPONSE, response);
}

static void
@@ -757,7 +752,7 @@
to->nhd++;
} else {
VSL_stats->losthdr++;
- WSLH(w, HTTP_T_LostHeader, fd, fm, n);
+ WSLH(w, SLT_LostHeader, fd, fm, n);
}
}

@@ -797,7 +792,7 @@
hp = bereq->http;
hp->logtag = HTTP_Tx;

- http_getreq(hp, sp->http);
+ http_copyreq(hp, sp->http, how != HTTPH_R_PIPE);
http_FilterFields(sp->wrk, sp->fd, hp, sp->http, how);
http_PrintfHeader(sp->wrk, sp->fd, hp, "X-Varnish: %u", sp->xid);
http_PrintfHeader(sp->wrk, sp->fd, hp,
@@ -850,7 +845,7 @@
hp->hd[u].b = p;
hp->hd[u].e = p + l;
} else {
- WSLH(w, HTTP_T_LostHeader, fd, hp, u);
+ WSLH(w, SLT_LostHeader, fd, hp, u);
hp->hd[u].b = NULL;
hp->hd[u].e = NULL;
}
@@ -864,7 +859,7 @@
{

CHECK_OBJ_NOTNULL(to, HTTP_MAGIC);
- /* XXX ??? to->f = to->v; Not sure this is valid */
+ /* XXX: don't to->f = to->v; it would kill pipelining */
to->nhd = HTTP_HDR_FIRST;
memset(to->hd, 0, sizeof to->hd);
}
@@ -878,10 +873,10 @@
CHECK_OBJ_NOTNULL(to, HTTP_MAGIC);
if (to->nhd >= HTTP_HDR_MAX) {
VSL_stats->losthdr++;
- WSL(w, http2shmlog(to, HTTP_T_LostHeader), fd, "%s", hdr);
+ WSL(w, SLT_LostHeader, fd, "%s", hdr);
return;
}
- http_seth(to, to->nhd++, hdr);
+ http_SetH(to, to->nhd++, hdr);
}

/*--------------------------------------------------------------------*/
@@ -898,7 +893,7 @@
l = (e - string);
p = WS_Alloc(to->ws, l + 1);
if (p == NULL) {
- WSL(w, http2shmlog(to, HTTP_T_LostHeader), fd, "%s", string);
+ WSL(w, SLT_LostHeader, fd, "%s", string);
to->hd[field].b = NULL;
to->hd[field].e = NULL;
} else {
@@ -945,7 +940,7 @@
va_end(ap);
if (n + 1 >= l || to->nhd >= HTTP_HDR_MAX) {
VSL_stats->losthdr++;
- WSL(w, http2shmlog(to, HTTP_T_LostHeader), fd, "%s", to->ws->f);
+ WSL(w, SLT_LostHeader, fd, "%s", to->ws->f);
WS_Release(to->ws, 0);
} else {
to->hd[to->nhd].b = to->ws->f;

Modified: branches/1.1/bin/varnishd/cache_lru.c
===================================================================
--- branches/1.1/bin/varnishd/cache_lru.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/cache_lru.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -71,7 +71,7 @@
* if it's already in it and hasn't moved in a while.
*/
void
-LRU_Enter(struct object *o, time_t stamp)
+LRU_Enter(struct object *o, double stamp)
{

CHECK_OBJ_NOTNULL(o, OBJECT_MAGIC);
@@ -199,7 +199,7 @@
* number of objects that were discarded.
*/
int
-LRU_DiscardTime(time_t cutoff)
+LRU_DiscardTime(double cutoff)
{
struct object *first = TAILQ_FIRST(&lru_list);
struct object *o;

Modified: branches/1.1/bin/varnishd/cache_main.c
===================================================================
--- branches/1.1/bin/varnishd/cache_main.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/cache_main.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -71,7 +71,7 @@
stevedore->open(stevedore);

printf("Ready\n");
- VSL_stats->start_time = time(NULL);
+ VSL_stats->start_time = TIM_real();

CLI_Init();


Modified: branches/1.1/bin/varnishd/cache_pipe.c
===================================================================
--- branches/1.1/bin/varnishd/cache_pipe.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/cache_pipe.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -38,10 +38,6 @@
#include <stdlib.h>
#include <sys/socket.h>

-#ifndef HAVE_CLOCK_GETTIME
-#include "compat/clock_gettime.h"
-#endif
-
#include "shmlog.h"
#include "heritage.h"
#include "cache.h"
@@ -106,7 +102,7 @@
vbe_free_bereq(bereq);
bereq = NULL;

- clock_gettime(CLOCK_REALTIME, &sp->t_resp);
+ sp->t_resp = TIM_real();

memset(fds, 0, sizeof fds);
fds[0].fd = vc->fd;

Modified: branches/1.1/bin/varnishd/cache_pool.c
===================================================================
--- branches/1.1/bin/varnishd/cache_pool.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/cache_pool.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -46,6 +46,7 @@

#include <errno.h>
#include <stdio.h>
+#include <math.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
@@ -181,6 +182,7 @@
struct workreq *wrq;

AN(w->wrq);
+ w->used = NAN;
wrq = w->wrq;
CHECK_OBJ_NOTNULL(wrq->sess, SESS_MAGIC);
wrq->sess->wrk = w;
@@ -193,6 +195,7 @@
CHECK_OBJ(w->nobj, OBJECT_MAGIC);
if (w->nobjhead != NULL)
CHECK_OBJ(w->nobjhead, OBJHEAD_MAGIC);
+ assert(!isnan(w->used));
w->wrq = NULL;
}

@@ -207,7 +210,7 @@
w = &ww;
memset(w, 0, sizeof *w);
w->magic = WORKER_MAGIC;
- w->idle = time(NULL);
+ w->used = TIM_real();
w->wlp = w->wlog;
w->wle = w->wlog + sizeof w->wlog;
AZ(pipe(w->pipe));
@@ -236,10 +239,10 @@

LOCK(&qp->mtx);
TAILQ_INSERT_HEAD(&qp->idle, w, list);
- assert(w->idle != 0);
+ assert(!isnan(w->used));
UNLOCK(&qp->mtx);
assert(1 == read(w->pipe[0], &c, 1));
- if (w->idle == 0)
+ if (w->wrq == NULL)
break;
wrk_do_one(w);
}
@@ -381,7 +384,7 @@
static void *
wrk_reaperthread(void *priv)
{
- time_t now;
+ double now;
struct worker *w;
struct wq *qp;
unsigned u;
@@ -392,13 +395,13 @@
sleep(1);
if (VSL_stats->n_wrk <= params->wthread_min)
continue;
- now = time(NULL);
+ now = TIM_real();
for (u = 0; u < nwq; u++) {
qp = wq[u];
LOCK(&qp->mtx);
w = TAILQ_LAST(&qp->idle, workerhead);
if (w != NULL &&
- (w->idle + params->wthread_timeout < now ||
+ (w->used + params->wthread_timeout < now ||
VSL_stats->n_wrk > params->wthread_max))
TAILQ_REMOVE(&qp->idle, w, list);
else
@@ -406,7 +409,7 @@
UNLOCK(&qp->mtx);
if (w == NULL)
continue;
- w->idle = 0;
+ AZ(w->wrq);
assert(1 == write(w->pipe[1], w, 1));
}
}

Modified: branches/1.1/bin/varnishd/cache_response.c
===================================================================
--- branches/1.1/bin/varnishd/cache_response.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/cache_response.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -32,10 +32,6 @@
#include <sys/types.h>
#include <sys/time.h>

-#ifndef HAVE_CLOCK_GETTIME
-#include "compat/clock_gettime.h"
-#endif
-
#include "shmlog.h"
#include "heritage.h"
#include "cache.h"
@@ -75,7 +71,7 @@
sp->http->logtag = HTTP_Tx;
http_SetResp(sp->http,
"HTTP/1.1", "304", "Not Modified");
- TIM_format(sp->t_req.tv_sec, lm);
+ TIM_format(sp->t_req, lm);
http_PrintfHeader(sp->wrk, sp->fd, sp->http, "Date: %s", lm);
http_SetHeader(sp->wrk, sp->fd, sp->http, "Via: 1.1 varnish");
http_PrintfHeader(sp->wrk, sp->fd, sp->http, "X-Varnish: %u", sp->xid);
@@ -95,12 +91,12 @@
res_do_conds(struct sess *sp)
{
char *p;
- time_t ims;
+ double ims;

if (sp->obj->last_modified > 0 &&
http_GetHdr(sp->http, H_If_Modified_Since, &p)) {
ims = TIM_parse(p);
- if (ims > sp->t_req.tv_sec) /* [RFC2616 14.25] */
+ if (ims > sp->t_req) /* [RFC2616 14.25] */
return (0);
if (sp->obj->last_modified > ims) {
return (0);
@@ -134,8 +130,8 @@
else
http_PrintfHeader(sp->wrk, sp->fd, sp->http,
"X-Varnish: %u", sp->xid);
- http_PrintfHeader(sp->wrk, sp->fd, sp->http, "Age: %u",
- sp->obj->age + sp->t_resp.tv_sec - sp->obj->entered);
+ http_PrintfHeader(sp->wrk, sp->fd, sp->http, "Age: %.0f",
+ sp->obj->age + sp->t_resp - sp->obj->entered);
http_SetHeader(sp->wrk, sp->fd, sp->http, "Via: 1.1 varnish");
if (sp->doclose != NULL)
http_SetHeader(sp->wrk, sp->fd, sp->http, "Connection: close");
@@ -151,7 +147,6 @@

CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);

- clock_gettime(CLOCK_REALTIME, &sp->t_resp);
WRK_Reset(sp->wrk, &sp->fd);
sp->wrk->acct.hdrbytes += http_Write(sp->wrk, sp->http, 1);
CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);

Modified: branches/1.1/bin/varnishd/cache_session.c
===================================================================
--- branches/1.1/bin/varnishd/cache_session.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/cache_session.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -50,6 +50,7 @@

#include <stdlib.h>
#include <string.h>
+#include <math.h>
#include <sys/uio.h>
#include <sys/socket.h>

@@ -91,7 +92,8 @@
char addr[TCP_ADDRBUFSIZE];
unsigned nref;

- time_t ttl;
+ /* How long to keep entry around. Inherits timescale from t_open */
+ double ttl;

struct acct acct;
};
@@ -120,7 +122,7 @@
unsigned u, v;
struct srcaddr *c, *c2, *c3;
struct srcaddrhead *ch;
- time_t now;
+ double now;

if (params->srcaddr_ttl == 0) {
sp->srcaddr = NULL;
@@ -131,7 +133,7 @@
v = u % nsrchash;
ch = &srchash[v];
CHECK_OBJ(ch, SRCADDRHEAD_MAGIC);
- now = sp->t_open.tv_sec;
+ now = sp->t_open;
if (sp->wrk->srcaddr == NULL) {
sp->wrk->srcaddr = calloc(sizeof *sp->wrk->srcaddr, 1);
XXXAN(sp->wrk->srcaddr);
@@ -227,14 +229,15 @@

ses_sum_acct(&sp->acct, a);
if (sp->srcaddr != NULL) {
+ /* XXX: only report once per second ? */
CHECK_OBJ(sp->srcaddr, SRCADDR_MAGIC);
LOCK(&sp->srcaddr->sah->mtx);
ses_sum_acct(&sp->srcaddr->acct, a);
b = sp->srcaddr->acct;
UNLOCK(&sp->srcaddr->sah->mtx);
WSL(sp->wrk, SLT_StatAddr, 0,
- "%s 0 %d %ju %ju %ju %ju %ju %ju %ju",
- sp->srcaddr->addr, sp->t_end.tv_sec - b.first,
+ "%s 0 %.0f %ju %ju %ju %ju %ju %ju %ju",
+ sp->srcaddr->addr, sp->t_end - b.first,
b.sess, b.req, b.pipe, b.pass,
b.fetch, b.hdrbytes, b.bodybytes);
}
@@ -307,6 +310,10 @@
sp->mysockaddr = (void*)(&sm->sockaddr[1]);
sp->mysockaddrlen = sizeof(sm->sockaddr[1]);
sp->sockaddr->sa_family = sp->mysockaddr->sa_family = PF_UNSPEC;
+ sp->t_open = NAN;
+ sp->t_req = NAN;
+ sp->t_resp = NAN;
+ sp->t_end = NAN;

assert(len <= sp->sockaddrlen);
if (addr != NULL) {
@@ -333,8 +340,8 @@
AZ(sp->vcl);
VSL_stats->n_sess--;
ses_relsrcaddr(sp);
- VSL(SLT_StatSess, sp->id, "%s %s %d %ju %ju %ju %ju %ju %ju %ju",
- sp->addr, sp->port, sp->t_end.tv_sec - b->first,
+ VSL(SLT_StatSess, sp->id, "%s %s %.0f %ju %ju %ju %ju %ju %ju %ju",
+ sp->addr, sp->port, sp->t_end - b->first,
b->sess, b->req, b->pipe, b->pass,
b->fetch, b->hdrbytes, b->bodybytes);
if (sm->workspace != params->mem_workspace) {

Modified: branches/1.1/bin/varnishd/cache_synthetic.c
===================================================================
--- branches/1.1/bin/varnishd/cache_synthetic.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/cache_synthetic.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -33,10 +33,6 @@

#include <stdlib.h>

-#ifndef HAVE_CLOCK_GETTIME
-#include "compat/clock_gettime.h"
-#endif
-
#include "shmlog.h"
#include "heritage.h"
#include "cache.h"
@@ -56,7 +52,7 @@
struct vsb vsb;
const char *msg;
char date[40];
- time_t now;
+ double now;
int fd;

assert(status >= 100 && status <= 999);
@@ -71,7 +67,7 @@
fd = sp->fd;
o = sp->obj;
h = &o->http;
- time(&now);
+ now = TIM_real();

/* look up HTTP response */
msg = http_StatusMessage(status);

Modified: branches/1.1/bin/varnishd/cache_vrt.c
===================================================================
--- branches/1.1/bin/varnishd/cache_vrt.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/cache_vrt.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -32,7 +32,10 @@
*/

#include <sys/types.h>
+#include <sys/socket.h>

+#include <netinet/in.h>
+
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
@@ -112,45 +115,60 @@

/*--------------------------------------------------------------------*/

+static char *
+vrt_assemble_string(struct http *hp, const char *h, const char *p, va_list ap)
+{
+ char *b, *e;
+ unsigned u, x;
+
+ u = WS_Reserve(hp->ws, 0);
+ e = b = hp->ws->f;
+ *e = '\0';
+ if (h != NULL) {
+ x = strlen(h);
+ if (x + 2 < u) {
+ memcpy(e, h, x);
+ e[x] = ' ';
+ e[x + 1] = '\0';
+ }
+ e += x + 1;
+ }
+ while (p != NULL) {
+ x = strlen(p);
+ if (x + 1 < u)
+ memcpy(e, p, x);
+ e += x;
+ p = va_arg(ap, const char *);
+ }
+ *e = '\0';
+ if (e > b + u) {
+ WS_Release(hp->ws, 0);
+ return (NULL);
+ } else {
+ WS_Release(hp->ws, 1 + e - b);
+ return (b);
+ }
+}
+
+/*--------------------------------------------------------------------*/
+
void
-VRT_SetHdr(struct sess *sp , enum gethdr_e where, const char *hdr, ...)
+VRT_SetHdr(struct sess *sp , enum gethdr_e where, const char *hdr, const char *p, ...)
{
struct http *hp;
va_list ap;
- const char *p;
- char *b, *e;
- unsigned u, x;
+ char *b;

CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
hp = vrt_selecthttp(sp, where);
- va_start(ap, hdr);
- p = va_arg(ap, const char *);
+ va_start(ap, p);
if (p == NULL) {
http_Unset(hp, hdr);
} else {
- u = WS_Reserve(hp->ws, 0);
- e = b = hp->ws->f;
- *e = '\0';
- x = strlen(hdr + 1);
- if (x + 1 < u)
- memcpy(e, hdr + 1, x);
- e += x;
- if (1 + 1 < u)
- *e++ = ' ';
- while (p != NULL) {
- x = strlen(p);
- if (x + 1 < u)
- memcpy(e, p, x);
- e += x;
- p = va_arg(ap, const char *);
- }
- *e = '\0';
- if (e > b + u) {
- http_LogLostHeader(sp->wrk, sp->fd, hp, hdr);
- WS_Release(hp->ws, 0);
-
+ b = vrt_assemble_string(hp, hdr + 1, p, ap);
+ if (b == NULL) {
+ VSL(SLT_LostHeader, sp->fd, hdr + 1);
} else {
- WS_Release(hp->ws, 1 + e - b);
http_Unset(hp, hdr);
http_SetHeader(sp->wrk, sp->fd, hp, b);
}
@@ -160,7 +178,77 @@

/*--------------------------------------------------------------------*/

+static void
+vrt_do_string(struct worker *w, int fd, struct http *hp, int fld, const char *err, const char *p, va_list ap)
+{
+ char *b;
+
+ AN(p);
+ AN(hp);
+ b = vrt_assemble_string(hp, NULL, p, ap);
+ if (b == NULL) {
+ WSL(w, SLT_LostHeader, fd, err);
+ } else {
+ http_SetH(hp, fld, b);
+ }
+ va_end(ap);
+}
+
+#define VRT_DO_HDR(obj, hdr, http, fld) \
+void \
+VRT_l_##obj##_##hdr(struct sess *sp, const char *p, ...) \
+{ \
+ va_list ap; \
+ \
+ AN(p); \
+ va_start(ap, p); \
+ vrt_do_string(sp->wrk, sp->fd, \
+ http, fld, #obj "." #hdr, p, ap); \
+ va_end(ap); \
+}
+
+VRT_DO_HDR(req, request, sp->http, HTTP_HDR_REQ)
+VRT_DO_HDR(req, url, sp->http, HTTP_HDR_URL)
+VRT_DO_HDR(req, proto, sp->http, HTTP_HDR_PROTO)
+VRT_DO_HDR(bereq, request, sp->bereq->http, HTTP_HDR_REQ)
+VRT_DO_HDR(bereq, url, sp->bereq->http, HTTP_HDR_URL)
+VRT_DO_HDR(bereq, proto, sp->bereq->http, HTTP_HDR_PROTO)
+VRT_DO_HDR(obj, proto, &sp->obj->http, HTTP_HDR_PROTO)
+VRT_DO_HDR(obj, response, &sp->obj->http, HTTP_HDR_RESPONSE)
+VRT_DO_HDR(resp, proto, sp->http, HTTP_HDR_PROTO)
+VRT_DO_HDR(resp, response, sp->http, HTTP_HDR_RESPONSE)
+
void
+VRT_l_obj_status(struct sess *sp, int num)
+{
+ char *p;
+
+ assert(num >= 100 && num <= 999);
+ p = WS_Alloc(sp->obj->http.ws, 4);
+ if (p == NULL)
+ WSL(sp->wrk, SLT_LostHeader, sp->fd, "obj.status");
+ else
+ sprintf(p, "%d", num);
+ http_SetH(&sp->obj->http, HTTP_HDR_STATUS, p);
+}
+
+void
+VRT_l_resp_status(struct sess *sp, int num)
+{
+ char *p;
+
+ assert(num >= 100 && num <= 999);
+ p = WS_Alloc(sp->http->ws, 4);
+ if (p == NULL)
+ WSL(sp->wrk, SLT_LostHeader, sp->fd, "resp.status");
+ else
+ sprintf(p, "%d", num);
+ http_SetH(sp->http, HTTP_HDR_STATUS, p);
+}
+
+/*--------------------------------------------------------------------*/
+
+void
VRT_handling(struct sess *sp, unsigned hand)
{

@@ -239,11 +327,11 @@

CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
CHECK_OBJ_NOTNULL(sp->obj, OBJECT_MAGIC); /* XXX */
- WSL(sp->wrk, SLT_TTL, sp->fd, "%u VCL %.0f %u",
- sp->obj->xid, a, sp->t_req.tv_sec);
+ WSL(sp->wrk, SLT_TTL, sp->fd, "%u VCL %.0f %.0f",
+ sp->obj->xid, a, sp->t_req);
if (a < 0)
a = 0;
- sp->obj->ttl = sp->t_req.tv_sec + (int)a;
+ sp->obj->ttl = sp->t_req + a;
if (sp->obj->heap_idx != 0)
EXP_TTLchange(sp->obj);
}
@@ -253,7 +341,7 @@
{
CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
CHECK_OBJ_NOTNULL(sp->obj, OBJECT_MAGIC); /* XXX */
- return (sp->obj->ttl - sp->t_req.tv_sec);
+ return (sp->obj->ttl - sp->t_req);
}

/*--------------------------------------------------------------------*/
@@ -375,22 +463,43 @@
double
VRT_r_now(struct sess *sp)
{
- struct timespec now;

(void)sp;
- /* XXX use of clock_gettime() needs review */
- clock_gettime(CLOCK_MONOTONIC, &now);
- return (now.tv_sec);
+ return (TIM_mono());
}

double
VRT_r_obj_lastuse(struct sess *sp)
{
- struct timespec now;

CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
CHECK_OBJ_NOTNULL(sp->obj, OBJECT_MAGIC); /* XXX */
- /* XXX use of clock_gettime() needs review */
- clock_gettime(CLOCK_MONOTONIC, &now);
- return (now.tv_sec - sp->obj->lru_stamp);
+ return (TIM_mono() - sp->obj->lru_stamp);
}
+
+/*--------------------------------------------------------------------*/
+
+char *
+VRT_IP_string(struct sess *sp, struct sockaddr *sa)
+{
+ char h[64], p[8], *q;
+ socklen_t len = 0;
+
+ /* XXX can't rely on sockaddr.sa_len */
+ switch (sa->sa_family) {
+ case AF_INET:
+ len = sizeof(struct sockaddr_in);
+ break;
+ case AF_INET6:
+ len = sizeof(struct sockaddr_in6);
+ break;
+ }
+ XXXAN(len);
+ TCP_name(sa, len, h, sizeof h, p, sizeof p);
+ q = WS_Alloc(sp->http->ws, strlen(h) + strlen(p) + 2);
+ AN(q);
+ strcpy(q, h);
+ strcat(q, ":");
+ strcat(q, p);
+ return (q);
+}

Modified: branches/1.1/bin/varnishd/cache_vrt_re.c
===================================================================
--- branches/1.1/bin/varnishd/cache_vrt_re.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/cache_vrt_re.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -35,6 +35,7 @@

#include <stdio.h>
#include <string.h>
+#include <ctype.h>
#include <stdlib.h>
#include <regex.h>

@@ -45,14 +46,14 @@
#include "cache.h"

void
-VRT_re_init(void **rep, const char *re)
+VRT_re_init(void **rep, const char *re, int sub)
{
regex_t *t;

t = calloc(sizeof *t, 1);
XXXAN(t);
/* This was already check-compiled by the VCL compiler */
- AZ(regcomp(t, re, REG_EXTENDED | REG_NOSUB));
+ AZ(regcomp(t, re, REG_EXTENDED | (sub ? 0 : REG_NOSUB)));
*rep = t;
}

@@ -82,14 +83,14 @@
}

int
-VRT_re_test(struct vsb *sb, const char *re)
+VRT_re_test(struct vsb *sb, const char *re, int sub)
{
int i;
regex_t t;
char buf[BUFSIZ];

memset(&t, 0, sizeof t);
- i = regcomp(&t, re, REG_EXTENDED | REG_NOSUB);
+ i = regcomp(&t, re, REG_EXTENDED | (sub ? 0 : REG_NOSUB));
if (i == 0) {
regfree(&t);
return (0);
@@ -99,3 +100,73 @@
regfree(&t);
return (1);
}
+
+const char *
+VRT_regsub(struct sess *sp, const char *str, void *re, const char *sub)
+{
+ regmatch_t pm[10];
+ regex_t *t;
+ int i, l;
+ char *b, *p, *e;
+ unsigned u, x;
+
+ AN(re);
+ t = re;
+ i = regexec(t, str, 10, pm, 0);
+
+ /* If it didn't match, we can return the original string */
+ if (i == REG_NOMATCH)
+ return(str);
+
+ u = WS_Reserve(sp->http->ws, 0);
+ e = p = b = sp->http->ws->f;
+ e += u;
+
+ /* Copy prefix to match */
+ if (pm[0].rm_so > 0) {
+ if (p + pm[0].rm_so < e)
+ memcpy(p, str, pm[0].rm_so);
+ p += pm[0].rm_so;
+ }
+
+ for ( ; *sub != '\0'; sub++ ) {
+ if (*sub == '&') {
+ l = pm[0].rm_eo - pm[0].rm_so;
+ if (l > 0) {
+ if (p + l < e)
+ memcpy(p, str + pm[0].rm_so, l);
+ p += l;
+ }
+ } else if (*sub == '$' && isdigit(sub[1])) {
+ x = sub[1] - '0';
+ sub++;
+ l = pm[x].rm_eo - pm[x].rm_so;
+ if (l > 0) {
+ if (p + l < e)
+ memcpy(p, str + pm[x].rm_so, l);
+ p += l;
+ }
+ } else {
+ if (p + 1 < e)
+ *p = *sub;
+ p++;
+ }
+ }
+
+ /* Copy suffix to match */
+ l = strlen(str + pm[0].rm_eo);
+ if (l > 0) {
+ if (p + l < e)
+ memcpy(p, str + pm[0].rm_eo, l);
+ p += l;
+ }
+ if (p + 1 < e)
+ *p++ = '\0';
+ xxxassert(p <= e);
+ if (p > e) {
+ WS_Release(sp->http->ws, 0);
+ return (str);
+ }
+ WS_Release(sp->http->ws, p - b);
+ return (b);
+}

Modified: branches/1.1/bin/varnishd/mgt_child.c
===================================================================
--- branches/1.1/bin/varnishd/mgt_child.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/mgt_child.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -445,3 +445,13 @@
cli_out(cli, "Child in state %s", ch_state[child_state]);
}
}
+
+/*--------------------------------------------------------------------*/
+
+void
+mcf_server_status(struct cli *cli, char **av, void *priv)
+{
+ (void)av;
+ (void)priv;
+ cli_out(cli, "Child in state %s", ch_state[child_state]);
+}

Modified: branches/1.1/bin/varnishd/mgt_cli.c
===================================================================
--- branches/1.1/bin/varnishd/mgt_cli.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/mgt_cli.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -140,6 +140,7 @@
/* XXX: what order should this list be in ? */
static struct cli_proto mgt_cli_proto[] = {
{ CLI_PING, cli_func_ping },
+ { CLI_SERVER_STATUS, mcf_server_status, NULL },
{ CLI_SERVER_START, mcf_server_startstop, NULL },
{ CLI_SERVER_STOP, mcf_server_startstop, &cli_proto },
{ CLI_STATS, mcf_stats, NULL },

Modified: branches/1.1/bin/varnishd/mgt_cli.h
===================================================================
--- branches/1.1/bin/varnishd/mgt_cli.h 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/mgt_cli.h 2007-07-13 14:54:51 UTC (rev 1693)
@@ -31,6 +31,7 @@

/* mgt_child.c */
cli_func_t mcf_server_startstop;
+cli_func_t mcf_server_status;

/* mgt_param.c */
cli_func_t mcf_param_show;

Modified: branches/1.1/bin/varnishd/mgt_event.c
===================================================================
--- branches/1.1/bin/varnishd/mgt_event.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/mgt_event.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -37,10 +37,6 @@
#include <string.h>
#include <stdlib.h>

-#ifndef HAVE_CLOCK_GETTIME
-#include "compat/clock_gettime.h"
-#endif
-
#include "mgt.h"
#include "mgt_event.h"
#include "miniobj.h"
@@ -76,19 +72,6 @@

/*--------------------------------------------------------------------*/

-static double
-ev_now(void)
-{
- double t;
- struct timespec ts;
-
- assert(clock_gettime(CLOCK_MONOTONIC, &ts) == 0);
- t = ts.tv_sec + ts.tv_nsec * 1e-9;
- return (t);
-}
-
-/*--------------------------------------------------------------------*/
-
static void
ev_bh_update(void *priv, void *a, unsigned u)
{
@@ -265,7 +248,7 @@
e->magic = EV_MAGIC; /* before binheap_insert() */

if (e->timeout != 0.0) {
- e->__when += ev_now() + e->timeout;
+ e->__when += TIM_mono() + e->timeout;
binheap_insert(evb->binheap, e);
assert(e->__binheap_idx > 0);
} else {
@@ -430,7 +413,7 @@
if (e != NULL) {
CHECK_OBJ_NOTNULL(e, EV_MAGIC);
assert(e->__binheap_idx == 1);
- t = ev_now();
+ t = TIM_mono();
if (e->__when <= t)
return (ev_sched_timeout(evb, e, t));
tmo = (int)((e->__when - t) * 1e3);
@@ -453,7 +436,7 @@
return (ev_sched_signal(evb));
if (i == 0) {
assert(e != NULL);
- t = ev_now();
+ t = TIM_mono();
if (e->__when <= t)
return (ev_sched_timeout(evb, e, t));
}

Modified: branches/1.1/bin/varnishd/rfc2616.c
===================================================================
--- branches/1.1/bin/varnishd/rfc2616.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/rfc2616.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -98,12 +98,12 @@
ttd = min(ttd, our_clock + hard_upper_ttl)
#endif

-static time_t
+static double
RFC2616_Ttl(struct sess *sp, struct http *hp, struct object *obj)
{
int retirement_age;
unsigned u1, u2;
- time_t h_date, h_expires, ttd;
+ double h_date, h_expires, ttd;
char *p;

retirement_age = INT_MAX;
@@ -121,6 +121,14 @@
retirement_age = u1 - u2;
}

+ /*
+ * XXX: if the backends time is too skewed relative to our own
+ * XXX: we should blacklist the backend, to avoid getting totally
+ * XXX: bogus results further down. Exactly what "too skewed" means
+ * XXX: in this context is a good question. It could be determined
+ * XXX: out according to the backends headers, but a simple fixed
+ * XXX: tolerance of a minute either way would be more predictable.
+ */
h_date = 0;
if (http_GetHdr(hp, H_Date, &p))
h_date = TIM_parse(p);

Modified: branches/1.1/bin/varnishd/shmlog.c
===================================================================
--- branches/1.1/bin/varnishd/shmlog.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/shmlog.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -289,7 +289,7 @@
/* XXX more check sanity of loghead ? */
logstart = (unsigned char *)loghead + loghead->start;
MTX_INIT(&vsl_mtx);
- loghead->starttime = time(NULL);
+ loghead->starttime = TIM_real();
memset(VSL_stats, 0, sizeof *VSL_stats);
}


Modified: branches/1.1/bin/varnishd/varnishd.1
===================================================================
--- branches/1.1/bin/varnishd/varnishd.1 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/bin/varnishd/varnishd.1 2007-07-13 14:54:51 UTC (rev 1693)
@@ -316,6 +316,8 @@
better idea of the current situation, use the
.Xr varnishstat 1
utility.
+.It Cm status
+Check the status of the child process.
.It Cm stop
Stop the child process.
.It Cm url.purge Ar regexp

Modified: branches/1.1/doc/changes-1.0.4-1.1.xml
===================================================================
--- branches/1.1/doc/changes-1.0.4-1.1.xml 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/doc/changes-1.0.4-1.1.xml 2007-07-13 14:54:51 UTC (rev 1693)
@@ -155,6 +155,29 @@
corresponding variable, and strip an HTTP header by using the
<command>remove</command> keyword.</para>
</change>
+
+ <change type="enh" ref="1661,1662">
+ <para>VCL scripts can now modify the HTTP status code of cached
+ objects (<varname>obj.status</varname>) and responses
+ (<varname>resp.status</varname>)</para>
+ </change>
+
+ <change type="enh" ref="1663">
+ <para>Numeric and other non-textual variables in VCL can now be
+ assigned to textual variables; they will be converted as
+ needed.</para>
+ </change>
+
+ <change type="enh" ref="1666,1667">
+ <para>VCL scripts can now apply regular expression substitutions
+ to textual variables using the <function>regsub</function>
+ function.</para>
+ </change>
+
+ <change type="enh" ref="1674,1675">
+ <para>A new management command, <command>status</command>,
+ returns the state of the child.</para>
+ </change>
</subsystem>

<subsystem>
@@ -189,6 +212,12 @@
<command>varnishncsa</command> can now also process log data
from backend traffic.</para>
</change>
+
+ <change type="bug" ref="1531">
+ <para>A bug that would cause <command>varnishncsa</command> to
+ segfault when it encountered an empty HTTP header in the log
+ file has been fixed.</para>
+ </change>
</subsystem>

<subsystem>

Modified: branches/1.1/include/cli.h
===================================================================
--- branches/1.1/include/cli.h 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/include/cli.h 2007-07-13 14:54:51 UTC (rev 1693)
@@ -211,6 +211,12 @@
"\tClose connection", \
0, 0

+# define CLI_SERVER_STATUS \
+ "status", \
+ "status", \
+ "\tCheck status of Varnish cache process.", \
+ 0, 0
+
/*
* Status/return codes in the CLI protocol
*/

Modified: branches/1.1/include/libvarnish.h
===================================================================
--- branches/1.1/include/libvarnish.h 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/include/libvarnish.h 2007-07-13 14:54:51 UTC (rev 1693)
@@ -47,6 +47,8 @@
/* from libvarnish/time.c */
void TIM_format(time_t t, char *p);
time_t TIM_parse(const char *p);
+double TIM_mono(void);
+double TIM_real(void);

/* from libvarnish/version.c */
void varnish_version(const char *);

Modified: branches/1.1/include/shmlog_tags.h
===================================================================
--- branches/1.1/include/shmlog_tags.h 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/include/shmlog_tags.h 2007-07-13 14:54:51 UTC (rev 1693)
@@ -62,7 +62,6 @@
SLTM(RxURL)
SLTM(RxProtocol)
SLTM(RxHeader)
-SLTM(RxLostHeader)

SLTM(TxRequest)
SLTM(TxResponse)
@@ -70,7 +69,6 @@
SLTM(TxURL)
SLTM(TxProtocol)
SLTM(TxHeader)
-SLTM(TxLostHeader)

SLTM(ObjRequest)
SLTM(ObjResponse)
@@ -78,8 +76,9 @@
SLTM(ObjURL)
SLTM(ObjProtocol)
SLTM(ObjHeader)
-SLTM(ObjLostHeader)

+SLTM(LostHeader)
+
SLTM(TTL)
SLTM(VCL_acl)
SLTM(VCL_call)

Modified: branches/1.1/include/vrt.h
===================================================================
--- branches/1.1/include/vrt.h 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/include/vrt.h 2007-07-13 14:54:51 UTC (rev 1693)
@@ -64,10 +64,11 @@
void VRT_acl_fini(struct vrt_acl *);

/* Regexp related */
-void VRT_re_init(void **, const char *);
+void VRT_re_init(void **, const char *, int sub);
void VRT_re_fini(void *);
int VRT_re_match(const char *, void *re);
-int VRT_re_test(struct vsb *, const char *);
+int VRT_re_test(struct vsb *, const char *, int sub);
+const char *VRT_regsub(struct sess *sp, const char *, void *, const char *);

void VRT_count(struct sess *, unsigned);
int VRT_rewrite(const char *, const char *);
@@ -76,7 +77,7 @@

enum gethdr_e { HDR_REQ, HDR_RESP, HDR_OBJ, HDR_BEREQ };
char *VRT_GetHdr(struct sess *, enum gethdr_e where, const char *);
-void VRT_SetHdr(struct sess *, enum gethdr_e where, const char *, ...);
+void VRT_SetHdr(struct sess *, enum gethdr_e where, const char *, const char *, ...);
void VRT_handling(struct sess *sp, unsigned hand);

/* Backend related */
@@ -85,6 +86,7 @@
void VRT_free_backends(struct VCL_conf *cp);
void VRT_fini_backend(struct backend *be);

+char *VRT_IP_string(struct sess *sp, struct sockaddr *sa);

#define VRT_done(sp, hand) \
do { \

Modified: branches/1.1/include/vrt_obj.h
===================================================================
--- branches/1.1/include/vrt_obj.h 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/include/vrt_obj.h 2007-07-13 14:54:51 UTC (rev 1693)
@@ -12,23 +12,26 @@
struct sockaddr * VRT_r_client_ip(struct sess *);
struct sockaddr * VRT_r_server_ip(struct sess *);
const char * VRT_r_req_request(struct sess *);
+void VRT_l_req_request(struct sess *, const char *, ...);
const char * VRT_r_req_url(struct sess *);
+void VRT_l_req_url(struct sess *, const char *, ...);
const char * VRT_r_req_proto(struct sess *);
+void VRT_l_req_proto(struct sess *, const char *, ...);
void VRT_l_req_hash(struct sess *, const char *);
struct backend * VRT_r_req_backend(struct sess *);
void VRT_l_req_backend(struct sess *, struct backend *);
const char * VRT_r_bereq_request(struct sess *);
-void VRT_l_bereq_request(struct sess *, const char *);
+void VRT_l_bereq_request(struct sess *, const char *, ...);
const char * VRT_r_bereq_url(struct sess *);
-void VRT_l_bereq_url(struct sess *, const char *);
+void VRT_l_bereq_url(struct sess *, const char *, ...);
const char * VRT_r_bereq_proto(struct sess *);
-void VRT_l_bereq_proto(struct sess *, const char *);
+void VRT_l_bereq_proto(struct sess *, const char *, ...);
const char * VRT_r_obj_proto(struct sess *);
-void VRT_l_obj_proto(struct sess *, const char *);
+void VRT_l_obj_proto(struct sess *, const char *, ...);
int VRT_r_obj_status(struct sess *);
void VRT_l_obj_status(struct sess *, int);
const char * VRT_r_obj_response(struct sess *);
-void VRT_l_obj_response(struct sess *, const char *);
+void VRT_l_obj_response(struct sess *, const char *, ...);
unsigned VRT_r_obj_valid(struct sess *);
void VRT_l_obj_valid(struct sess *, unsigned);
unsigned VRT_r_obj_cacheable(struct sess *);
@@ -37,9 +40,9 @@
void VRT_l_obj_ttl(struct sess *, double);
double VRT_r_obj_lastuse(struct sess *);
const char * VRT_r_resp_proto(struct sess *);
-void VRT_l_resp_proto(struct sess *, const char *);
+void VRT_l_resp_proto(struct sess *, const char *, ...);
int VRT_r_resp_status(struct sess *);
void VRT_l_resp_status(struct sess *, int);
const char * VRT_r_resp_response(struct sess *);
-void VRT_l_resp_response(struct sess *, const char *);
+void VRT_l_resp_response(struct sess *, const char *, ...);
double VRT_r_now(struct sess *);

Modified: branches/1.1/lib/libvarnish/Makefile.am
===================================================================
--- branches/1.1/lib/libvarnish/Makefile.am 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/lib/libvarnish/Makefile.am 2007-07-13 14:54:51 UTC (rev 1693)
@@ -19,3 +19,5 @@
vss.c

libvarnish_la_CFLAGS = -include config.h
+
+libvarnish_la_LIBADD = ${RT_LIBS}

Modified: branches/1.1/lib/libvarnish/time.c
===================================================================
--- branches/1.1/lib/libvarnish/time.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/lib/libvarnish/time.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -50,8 +50,30 @@
#include <string.h>
#include <time.h>

+#ifndef HAVE_CLOCK_GETTIME
+#include "compat/clock_gettime.h"
+#endif
+
#include "libvarnish.h"

+double
+TIM_mono(void)
+{
+ struct timespec ts;
+
+ assert(clock_gettime(CLOCK_MONOTONIC, &ts) == 0);
+ return (ts.tv_sec + 1e-9 * ts.tv_nsec);
+}
+
+double
+TIM_real(void)
+{
+ struct timespec ts;
+
+ assert(clock_gettime(CLOCK_REALTIME, &ts) == 0);
+ return (ts.tv_sec + 1e-9 * ts.tv_nsec);
+}
+
void
TIM_format(time_t t, char *p)
{

Modified: branches/1.1/lib/libvcl/Makefile.am
===================================================================
--- branches/1.1/lib/libvcl/Makefile.am 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/lib/libvcl/Makefile.am 2007-07-13 14:54:51 UTC (rev 1693)
@@ -16,6 +16,7 @@
vcc_parse.c \
vcc_fixed_token.c \
vcc_obj.c \
+ vcc_string.c \
vcc_token.c \
vcc_var.c \
vcc_xref.c

Modified: branches/1.1/lib/libvcl/syntax.txt
===================================================================
--- branches/1.1/lib/libvcl/syntax.txt 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/lib/libvcl/syntax.txt 2007-07-13 14:54:51 UTC (rev 1693)
@@ -128,6 +128,7 @@
'call' ident ';'
'rewrite' cstr cstr ';'
'set' assignment ';'
+ 'remove' variable ';'

# see variable 'returns' in vcc_gen_fixed_token.tcl
return_action:
@@ -162,6 +163,7 @@
'-=' cnum
'=' cnum

+
ratio:
'*=' double
'/=' double

Modified: branches/1.1/lib/libvcl/vcc_acl.c
===================================================================
--- branches/1.1/lib/libvcl/vcc_acl.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/lib/libvcl/vcc_acl.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -30,13 +30,13 @@
*/

#include <stdio.h>
-#include <assert.h>
#include <stdlib.h>

#include "vsb.h"

#include "vcc_priv.h"
#include "vcc_compile.h"
+#include "libvarnish.h"

static void
vcc_acl_top(struct tokenlist *tl, const char *acln)
@@ -108,7 +108,7 @@
}

void
-vcc_Cond_Ip(struct var *vp, struct tokenlist *tl)
+vcc_Cond_Ip(const struct var *vp, struct tokenlist *tl)
{
unsigned tcond;
char *acln;

Modified: branches/1.1/lib/libvcl/vcc_action.c
===================================================================
--- branches/1.1/lib/libvcl/vcc_action.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/lib/libvcl/vcc_action.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -94,7 +94,7 @@
/*--------------------------------------------------------------------*/

static void
-illegal_assignment(struct tokenlist *tl, const char *type)
+illegal_assignment(const struct tokenlist *tl, const char *type)
{

vsb_printf(tl->sb, "Invalid assignment operator ");
@@ -104,7 +104,7 @@
}

static void
-check_writebit(struct tokenlist *tl, struct var *vp)
+check_writebit(struct tokenlist *tl, const struct var *vp)
{

if (vp->access == V_RW || vp->access == V_WO)
@@ -155,6 +155,8 @@
vcc_RateVal(tl);
else if (vp->fmt == FLOAT)
Fb(tl, 0, "%g", vcc_DoubleVal(tl));
+ else if (vp->fmt == INT)
+ Fb(tl, 0, "%u", vcc_UintVal(tl));
else {
vsb_printf(tl->sb, "Cannot assign this variable type.\n");
vcc_ErrWhere(tl, vt);
@@ -195,28 +197,38 @@
vcc_NextToken(tl);
Fb(tl, 0, ");\n");
break;
- return;
case HASH:
ExpectErr(tl, T_INCR);
vcc_NextToken(tl);
- vcc_StringVal(tl);
+ if (!vcc_StringVal(tl)) {
+ ERRCHK(tl);
+ vcc_ExpectedStringval(tl);
+ return;
+ }
Fb(tl, 0, ");\n");
- return;
+ break;
case STRING:
if (tl->t->tok != '=') {
illegal_assignment(tl, "strings");
return;
}
vcc_NextToken(tl);
- vcc_StringVal(tl);
- if (vp->ishdr) {
- while (tl->t->tok != ';') {
- Fb(tl, 0, ", ");
- vcc_StringVal(tl);
- }
- Fb(tl, 0, ", 0");
+ if (!vcc_StringVal(tl)) {
+ ERRCHK(tl);
+ vcc_ExpectedStringval(tl);
+ return;
}
- Fb(tl, 0, ");\n");
+ do
+ Fb(tl, 0, ", ");
+ while (vcc_StringVal(tl));
+ if (tl->t->tok != ';') {
+ ERRCHK(tl);
+ vsb_printf(tl->sb,
+ "Expected variable, string or semicolon\n");
+ vcc_ErrWhere(tl, tl->t);
+ return;
+ }
+ Fb(tl, 0, "0);\n");
break;
default:
vsb_printf(tl->sb,
@@ -232,13 +244,13 @@
parse_remove(struct tokenlist *tl)
{
struct var *vp;
- struct token *vt;

vcc_NextToken(tl);
ExpectErr(tl, VAR);
- vt = tl->t;
vp = vcc_FindVar(tl, tl->t, vcc_vars);
- if (vp->fmt != STRING || !vp->ishdr) {
+ ERRCHK(tl);
+ assert(vp != NULL);
+ if (vp->fmt != STRING || vp->hdr == NULL) {
vsb_printf(tl->sb, "Only http header lines can be removed.\n");
vcc_ErrWhere(tl, tl->t);
return;

Modified: branches/1.1/lib/libvcl/vcc_compile.c
===================================================================
--- branches/1.1/lib/libvcl/vcc_compile.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/lib/libvcl/vcc_compile.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -86,7 +86,7 @@
#include "vcl_returns.h"
#undef VCL_MET_MAC
#undef VCL_RET_MAC
- { NULL, 0U }
+ { NULL, 0U, 0}
};

/*--------------------------------------------------------------------*/

Modified: branches/1.1/lib/libvcl/vcc_compile.h
===================================================================
--- branches/1.1/lib/libvcl/vcc_compile.h 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/lib/libvcl/vcc_compile.h 2007-07-13 14:54:51 UTC (rev 1693)
@@ -121,7 +121,7 @@
const char *rname;
const char *lname;
enum {V_RO, V_RW, V_WO} access;
- char ishdr;
+ const char *hdr;
unsigned methods;
};

@@ -136,7 +136,7 @@
/* vcc_acl.c */

void vcc_Acl(struct tokenlist *tl);
-void vcc_Cond_Ip(struct var *vp, struct tokenlist *tl);
+void vcc_Cond_Ip(const struct var *vp, struct tokenlist *tl);

/* vcc_action.c */
void vcc_ParseAction(struct tokenlist *tl);
@@ -167,6 +167,11 @@
unsigned vcc_UintVal(struct tokenlist *tl);
double vcc_DoubleVal(struct tokenlist *tl);

+/* vcc_string.c */
+char *vcc_regexp(struct tokenlist *tl, int sub);
+int vcc_StringVal(struct tokenlist *tl);
+void vcc_ExpectedStringval(struct tokenlist *tl);
+
/* vcc_token.c */
void vcc_ErrToken(const struct tokenlist *tl, const struct token *t);
void vcc_ErrWhere(struct tokenlist *tl, const struct token *t);
@@ -180,7 +185,6 @@
void vcc_FreeToken(struct token *t);

/* vcc_var.c */
-void vcc_StringVal(struct tokenlist *tl);
struct var *vcc_FindVar(struct tokenlist *tl, const struct token *t, struct var *vl);

/* vcc_xref.c */

Modified: branches/1.1/lib/libvcl/vcc_fixed_token.c
===================================================================
--- branches/1.1/lib/libvcl/vcc_fixed_token.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/lib/libvcl/vcc_fixed_token.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -420,10 +420,11 @@
vsb_cat(sb, "void VRT_acl_fini(struct vrt_acl *);\n");
vsb_cat(sb, "\n");
vsb_cat(sb, "/* Regexp related */\n");
- vsb_cat(sb, "void VRT_re_init(void **, const char *);\n");
+ vsb_cat(sb, "void VRT_re_init(void **, const char *, int sub);\n");
vsb_cat(sb, "void VRT_re_fini(void *);\n");
vsb_cat(sb, "int VRT_re_match(const char *, void *re);\n");
- vsb_cat(sb, "int VRT_re_test(struct vsb *, const char *);\n");
+ vsb_cat(sb, "int VRT_re_test(struct vsb *, const char *, int sub);\n");
+ vsb_cat(sb, "const char *VRT_regsub(struct sess *sp, const char *, void *, const char *);\n");
vsb_cat(sb, "\n");
vsb_cat(sb, "void VRT_count(struct sess *, unsigned);\n");
vsb_cat(sb, "int VRT_rewrite(const char *, const char *);\n");
@@ -432,7 +433,7 @@
vsb_cat(sb, "\n");
vsb_cat(sb, "enum gethdr_e { HDR_REQ, HDR_RESP, HDR_OBJ, HDR_BEREQ };\n");
vsb_cat(sb, "char *VRT_GetHdr(struct sess *, enum gethdr_e where, const char *);\n");
- vsb_cat(sb, "void VRT_SetHdr(struct sess *, enum gethdr_e where, const char *, ...);\n");
+ vsb_cat(sb, "void VRT_SetHdr(struct sess *, enum gethdr_e where, const char *, const char *, ...);\n");
vsb_cat(sb, "void VRT_handling(struct sess *sp, unsigned hand);\n");
vsb_cat(sb, "\n");
vsb_cat(sb, "/* Backend related */\n");
@@ -441,6 +442,7 @@
vsb_cat(sb, "void VRT_free_backends(struct VCL_conf *cp);\n");
vsb_cat(sb, "void VRT_fini_backend(struct backend *be);\n");
vsb_cat(sb, "\n");
+ vsb_cat(sb, "char *VRT_IP_string(struct sess *sp, struct sockaddr *sa);\n");
vsb_cat(sb, "\n");
vsb_cat(sb, "#define VRT_done(sp, hand) \\\n");
vsb_cat(sb, " do { \\\n");
@@ -461,23 +463,26 @@
vsb_cat(sb, "struct sockaddr * VRT_r_client_ip(struct sess *);\n");
vsb_cat(sb, "struct sockaddr * VRT_r_server_ip(struct sess *);\n");
vsb_cat(sb, "const char * VRT_r_req_request(struct sess *);\n");
+ vsb_cat(sb, "void VRT_l_req_request(struct sess *, const char *, ...);\n");
vsb_cat(sb, "const char * VRT_r_req_url(struct sess *);\n");
+ vsb_cat(sb, "void VRT_l_req_url(struct sess *, const char *, ...);\n");
vsb_cat(sb, "const char * VRT_r_req_proto(struct sess *);\n");
+ vsb_cat(sb, "void VRT_l_req_proto(struct sess *, const char *, ...);\n");
vsb_cat(sb, "void VRT_l_req_hash(struct sess *, const char *);\n");
vsb_cat(sb, "struct backend * VRT_r_req_backend(struct sess *);\n");
vsb_cat(sb, "void VRT_l_req_backend(struct sess *, struct backend *);\n");
vsb_cat(sb, "const char * VRT_r_bereq_request(struct sess *);\n");
- vsb_cat(sb, "void VRT_l_bereq_request(struct sess *, const char *);\n");
+ vsb_cat(sb, "void VRT_l_bereq_request(struct sess *, const char *, ...);\n");
vsb_cat(sb, "const char * VRT_r_bereq_url(struct sess *);\n");
- vsb_cat(sb, "void VRT_l_bereq_url(struct sess *, const char *);\n");
+ vsb_cat(sb, "void VRT_l_bereq_url(struct sess *, const char *, ...);\n");
vsb_cat(sb, "const char * VRT_r_bereq_proto(struct sess *);\n");
- vsb_cat(sb, "void VRT_l_bereq_proto(struct sess *, const char *);\n");
+ vsb_cat(sb, "void VRT_l_bereq_proto(struct sess *, const char *, ...);\n");
vsb_cat(sb, "const char * VRT_r_obj_proto(struct sess *);\n");
- vsb_cat(sb, "void VRT_l_obj_proto(struct sess *, const char *);\n");
+ vsb_cat(sb, "void VRT_l_obj_proto(struct sess *, const char *, ...);\n");
vsb_cat(sb, "int VRT_r_obj_status(struct sess *);\n");
vsb_cat(sb, "void VRT_l_obj_status(struct sess *, int);\n");
vsb_cat(sb, "const char * VRT_r_obj_response(struct sess *);\n");
- vsb_cat(sb, "void VRT_l_obj_response(struct sess *, const char *);\n");
+ vsb_cat(sb, "void VRT_l_obj_response(struct sess *, const char *, ...);\n");
vsb_cat(sb, "unsigned VRT_r_obj_valid(struct sess *);\n");
vsb_cat(sb, "void VRT_l_obj_valid(struct sess *, unsigned);\n");
vsb_cat(sb, "unsigned VRT_r_obj_cacheable(struct sess *);\n");
@@ -486,10 +491,10 @@
vsb_cat(sb, "void VRT_l_obj_ttl(struct sess *, double);\n");
vsb_cat(sb, "double VRT_r_obj_lastuse(struct sess *);\n");
vsb_cat(sb, "const char * VRT_r_resp_proto(struct sess *);\n");
- vsb_cat(sb, "void VRT_l_resp_proto(struct sess *, const char *);\n");
+ vsb_cat(sb, "void VRT_l_resp_proto(struct sess *, const char *, ...);\n");
vsb_cat(sb, "int VRT_r_resp_status(struct sess *);\n");
vsb_cat(sb, "void VRT_l_resp_status(struct sess *, int);\n");
vsb_cat(sb, "const char * VRT_r_resp_response(struct sess *);\n");
- vsb_cat(sb, "void VRT_l_resp_response(struct sess *, const char *);\n");
+ vsb_cat(sb, "void VRT_l_resp_response(struct sess *, const char *, ...);\n");
vsb_cat(sb, "double VRT_r_now(struct sess *);\n");
}

Modified: branches/1.1/lib/libvcl/vcc_gen_obj.tcl
===================================================================
--- branches/1.1/lib/libvcl/vcc_gen_obj.tcl 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/lib/libvcl/vcc_gen_obj.tcl 2007-07-13 14:54:51 UTC (rev 1693)
@@ -32,9 +32,9 @@

# Objects available in backends
set beobj {
- { backend.host WO HOSTNAME }
- { backend.port WO PORTNAME }
- { backend.dnsttl WO TIME }
+ { backend.host WO HOSTNAME {} }
+ { backend.port WO PORTNAME {} }
+ { backend.dnsttl WO TIME {} }
}

# Variables available in sessions
@@ -56,20 +56,21 @@

# Request paramters
{ req.request
- RO STRING
+ RW STRING
{recv pipe pass hash miss hit fetch }
}
{ req.url
- RO STRING
+ RW STRING
{recv pipe pass hash miss hit fetch }
}
{ req.proto
- RO STRING
+ RW STRING
{recv pipe pass hash miss hit fetch }
}
{ req.http.
RW HEADER
{recv pipe pass hash miss hit fetch }
+ HDR_REQ
}

# Possibly misnamed, not really part of the request
@@ -98,6 +99,7 @@
{ bereq.http.
RW HEADER
{ pipe pass miss }
+ HDR_BEREQ
}

# The (possibly) cached object
@@ -116,6 +118,7 @@
{ obj.http.
RW HEADER
{ hit fetch }
+ HDR_OBJ
}

{ obj.valid
@@ -151,6 +154,7 @@
{ resp.http.
RW HEADER
{ deliver }
+ HDR_RESP
}

# Miscellaneous
@@ -200,6 +204,9 @@
append l " | "
append l VCL_MET_[string toupper $i]
}
+ if {$l == ""} {
+ return "0"
+ }
return [string range $l 3 end]
}

@@ -224,14 +231,21 @@
}
if {$a == "WO" || $a == "RW"} {
puts $fo "\t \"VRT_l_${m}($pa, \","
- if {$t != "HEADER"} {
+ if {$t == "HEADER"} {
+ } elseif {$t == "STRING"} {
+ puts $fp "void VRT_l_${m}($ty, $tt($t), ...);"
+ } else {
puts $fp "void VRT_l_${m}($ty, $tt($t));"
}
} else {
puts $fo "\t NULL,"
}
puts $fo "\t V_$a,"
- puts $fo "\t 0,"
+ if {$t != "HEADER"} {
+ puts $fo "\t 0,"
+ } else {
+ puts $fo "\t \"[lindex $v 4]\","
+ }
puts $fo "\t [method_map [lindex $v 3]]"
puts $fo "\t\},"


Modified: branches/1.1/lib/libvcl/vcc_obj.c
===================================================================
--- branches/1.1/lib/libvcl/vcc_obj.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/lib/libvcl/vcc_obj.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -15,21 +15,21 @@
"VRT_l_backend_host(backend, ",
V_WO,
0,
-
+ 0
},
{ "backend.port", PORTNAME, 12,
NULL,
"VRT_l_backend_port(backend, ",
V_WO,
0,
-
+ 0
},
{ "backend.dnsttl", TIME, 14,
NULL,
"VRT_l_backend_dnsttl(backend, ",
V_WO,
0,
-
+ 0
},
{ NULL }
};
@@ -51,22 +51,22 @@
},
{ "req.request", STRING, 11,
"VRT_r_req_request(sp)",
- NULL,
- V_RO,
+ "VRT_l_req_request(sp, ",
+ V_RW,
0,
VCL_MET_RECV | VCL_MET_PIPE | VCL_MET_PASS | VCL_MET_HASH | VCL_MET_MISS | VCL_MET_HIT | VCL_MET_FETCH
},
{ "req.url", STRING, 7,
"VRT_r_req_url(sp)",
- NULL,
- V_RO,
+ "VRT_l_req_url(sp, ",
+ V_RW,
0,
VCL_MET_RECV | VCL_MET_PIPE | VCL_MET_PASS | VCL_MET_HASH | VCL_MET_MISS | VCL_MET_HIT | VCL_MET_FETCH
},
{ "req.proto", STRING, 9,
"VRT_r_req_proto(sp)",
- NULL,
- V_RO,
+ "VRT_l_req_proto(sp, ",
+ V_RW,
0,
VCL_MET_RECV | VCL_MET_PIPE | VCL_MET_PASS | VCL_MET_HASH | VCL_MET_MISS | VCL_MET_HIT | VCL_MET_FETCH
},
@@ -74,7 +74,7 @@
"VRT_r_req_http_(sp)",
"VRT_l_req_http_(sp, ",
V_RW,
- 0,
+ "HDR_REQ",
VCL_MET_RECV | VCL_MET_PIPE | VCL_MET_PASS | VCL_MET_HASH | VCL_MET_MISS | VCL_MET_HIT | VCL_MET_FETCH
},
{ "req.hash", HASH, 8,
@@ -116,7 +116,7 @@
"VRT_r_bereq_http_(sp)",
"VRT_l_bereq_http_(sp, ",
V_RW,
- 0,
+ "HDR_BEREQ",
VCL_MET_PIPE | VCL_MET_PASS | VCL_MET_MISS
},
{ "obj.proto", STRING, 9,
@@ -144,7 +144,7 @@
"VRT_r_obj_http_(sp)",
"VRT_l_obj_http_(sp, ",
V_RW,
- 0,
+ "HDR_OBJ",
VCL_MET_HIT | VCL_MET_FETCH
},
{ "obj.valid", BOOL, 9,
@@ -200,7 +200,7 @@
"VRT_r_resp_http_(sp)",
"VRT_l_resp_http_(sp, ",
V_RW,
- 0,
+ "HDR_RESP",
VCL_MET_DELIVER
},
{ "now", TIME, 3,

Modified: branches/1.1/lib/libvcl/vcc_parse.c
===================================================================
--- branches/1.1/lib/libvcl/vcc_parse.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/lib/libvcl/vcc_parse.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -221,38 +221,16 @@
/*--------------------------------------------------------------------*/

static void
-vcc_re(struct tokenlist *tl, const char *str, const struct token *re)
-{
- char buf[32];
-
- assert(re->tok == CSTR);
- if (VRT_re_test(tl->sb, re->dec)) {
- vcc_ErrWhere(tl, re);
- return;
- }
- sprintf(buf, "VGC_re_%u", tl->recnt++);
-
- Fb(tl, 1, "VRT_re_match(%s, %s)\n", str, buf);
- Fh(tl, 0, "void *%s;\n", buf);
- Fi(tl, 0, "\tVRT_re_init(&%s, ",buf);
- EncToken(tl->fi, re);
- Fi(tl, 0, ");\n");
- Ff(tl, 0, "\tVRT_re_fini(%s);\n", buf);
-}
-
-
-/*--------------------------------------------------------------------*/
-
-static void
Cond_String(const struct var *vp, struct tokenlist *tl)
{
+ char *p;

switch (tl->t->tok) {
case '~':
vcc_NextToken(tl);
- ExpectErr(tl, CSTR);
- vcc_re(tl, vp->rname, tl->t);
+ p = vcc_regexp(tl, 0);
vcc_NextToken(tl);
+ Fb(tl, 1, "VRT_re_match(%s, %s)\n", vp->rname, p);
break;
case T_EQ:
case T_NEQ:

Copied: branches/1.1/lib/libvcl/vcc_string.c (from rev 1692, trunk/varnish-cache/lib/libvcl/vcc_string.c)
===================================================================
--- branches/1.1/lib/libvcl/vcc_string.c (rev 0)
+++ branches/1.1/lib/libvcl/vcc_string.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -0,0 +1,164 @@
+/*-
+ * Copyright (c) 2006 Verdens Gang AS
+ * Copyright (c) 2006-2007 Linpro AS
+ * All rights reserved.
+ *
+ * Author: Poul-Henning Kamp <phk at phk.freebsd.dk>
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * $Id$
+ */
+
+#include <stdio.h>
+#include <string.h>
+
+#include "vsb.h"
+
+#include "vcc_priv.h"
+#include "vcc_compile.h"
+#include "libvarnish.h"
+
+#include "vrt.h"
+
+/*--------------------------------------------------------------------*/
+
+char *
+vcc_regexp(struct tokenlist *tl, int sub)
+{
+ char buf[32], *p;
+
+ Expect(tl, CSTR);
+ if (VRT_re_test(tl->sb, tl->t->dec, sub)) {
+ vcc_ErrWhere(tl, tl->t);
+ return (NULL);
+ }
+ sprintf(buf, "VGC_re_%u", tl->recnt++);
+ p = TlAlloc(tl, strlen(buf) + 1);
+ strcpy(p, buf);
+
+ Fh(tl, 0, "void *%s;\n", buf);
+ Fi(tl, 0, "\tVRT_re_init(&%s, ",buf);
+ EncToken(tl->fi, tl->t);
+ Fi(tl, 0, ", %d);\n", sub);
+ Ff(tl, 0, "\tVRT_re_fini(%s);\n", buf);
+ return (p);
+}
+
+/*--------------------------------------------------------------------*/
+
+static int
+vcc_regsub(struct tokenlist *tl)
+{
+ char *p;
+
+ vcc_NextToken(tl);
+
+ Fb(tl, 0, "VRT_regsub(sp, ");
+
+ Expect(tl, '(');
+ vcc_NextToken(tl);
+
+ if (!vcc_StringVal(tl)) {
+ vcc_ExpectedStringval(tl);
+ return (0);
+ }
+
+ Expect(tl, ',');
+ vcc_NextToken(tl);
+
+ Expect(tl, CSTR);
+ p = vcc_regexp(tl, 1);
+ vcc_NextToken(tl);
+ Fb(tl, 0, ", %s, ", p);
+
+ Expect(tl, ',');
+ vcc_NextToken(tl);
+
+ Expect(tl, CSTR);
+ if (!vcc_StringVal(tl)) {
+ vcc_ExpectedStringval(tl);
+ return (0);
+ }
+
+ Expect(tl, ')');
+ vcc_NextToken(tl);
+ Fb(tl, 0, ")");
+
+ return (1);
+}
+
+/*--------------------------------------------------------------------
+ * Parse a string value and emit something that results in a usable
+ * "const char *".
+ * There are three possible outcomes:
+ * tl->err != 0 means something bad happened and a message is emitted.
+ * return (0) means "could not use this token"
+ * return (1) means "done"
+ */
+
+int
+vcc_StringVal(struct tokenlist *tl)
+{
+ struct var *vp;
+
+ if (tl->t->tok == CSTR) {
+ EncToken(tl->fb, tl->t);
+ vcc_NextToken(tl);
+ return (1);
+ }
+ if (tl->t->tok == ID && vcc_IdIs(tl->t, "regsub"))
+ return (vcc_regsub(tl));
+ if (tl->t->tok == VAR) {
+ vp = vcc_FindVar(tl, tl->t, vcc_vars);
+ if (tl->err)
+ return (0);
+ assert(vp != NULL);
+ switch (vp->fmt) {
+ case STRING:
+ Fb(tl, 0, "%s", vp->rname);
+ break;
+ case IP:
+ Fb(tl, 0, "VRT_IP_string(sp, %s)", vp->rname);
+ break;
+ default:
+ vsb_printf(tl->sb,
+ "String representation of '%s' not implemented yet.\n",
+ vp->name);
+ vcc_ErrWhere(tl, tl->t);
+ return (0);
+ }
+ vcc_NextToken(tl);
+ return (1);
+ }
+ return (0);
+}
+
+void
+vcc_ExpectedStringval(struct tokenlist *tl)
+{
+
+ if (!tl->err) {
+ vsb_printf(tl->sb, "Expected string variable or constant\n");
+ vcc_ErrWhere(tl, tl->t);
+ }
+}

Modified: branches/1.1/lib/libvcl/vcc_var.c
===================================================================
--- branches/1.1/lib/libvcl/vcc_var.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/lib/libvcl/vcc_var.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -40,44 +40,10 @@

/*--------------------------------------------------------------------*/

-void
-vcc_StringVal(struct tokenlist *tl)
-{
- struct var *vp;
-
- if (tl->t->tok == CSTR) {
- EncToken(tl->fb, tl->t);
- vcc_NextToken(tl);
- return;
- } else if (tl->t->tok != VAR) {
- vsb_printf(tl->sb, "Expected string variable or constant\n");
- vcc_ErrWhere(tl, tl->t);
- return;
- }
- ERRCHK(tl);
- vp = vcc_FindVar(tl, tl->t, vcc_vars);
- ERRCHK(tl);
- switch (vp->fmt) {
- case STRING:
- Fb(tl, 0, "%s", vp->rname);
- break;
- default:
- vsb_printf(tl->sb,
- "String representation of '%s' not implemented yet.\n",
- vp->name);
- vcc_ErrWhere(tl, tl->t);
- return;
- }
- vcc_NextToken(tl);
-}
-
-/*--------------------------------------------------------------------*/
-
static struct var *
HeaderVar(struct tokenlist *tl, const struct token *t, const struct var *vh)
{
char *p;
- const char *wh;
struct var *v;
int i;

@@ -93,23 +59,13 @@
v->name = p;
v->access = V_RW;
v->fmt = STRING;
- v->ishdr = 1;
+ v->hdr = vh->hdr;
v->methods = vh->methods;
- if (!memcmp(vh->name, "req.", 4))
- wh = "HDR_REQ";
- else if (!memcmp(vh->name, "resp.", 5))
- wh = "HDR_RESP";
- else if (!memcmp(vh->name, "obj.", 4))
- wh = "HDR_OBJ";
- else if (!memcmp(vh->name, "bereq.", 6))
- wh = "HDR_BEREQ";
- else
- assert(0 == 1);
- asprintf(&p, "VRT_GetHdr(sp, %s, \"\\%03o%s:\")", wh,
+ asprintf(&p, "VRT_GetHdr(sp, %s, \"\\%03o%s:\")", v->hdr,
(unsigned)(strlen(v->name + vh->len) + 1), v->name + vh->len);
AN(p);
v->rname = p;
- asprintf(&p, "VRT_SetHdr(sp, %s, \"\\%03o%s:\", ", wh,
+ asprintf(&p, "VRT_SetHdr(sp, %s, \"\\%03o%s:\", ", v->hdr,
(unsigned)(strlen(v->name + vh->len) + 1), v->name + vh->len);
AN(p);
v->lname = p;

Modified: branches/1.1/lib/libvcl/vcc_xref.c
===================================================================
--- branches/1.1/lib/libvcl/vcc_xref.c 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/lib/libvcl/vcc_xref.c 2007-07-13 14:54:51 UTC (rev 1693)
@@ -328,7 +328,7 @@
}

static struct procuse *
-vcc_FindIllegalUse(struct proc *p, struct method *m)
+vcc_FindIllegalUse(const struct proc *p, const struct method *m)
{
struct procuse *pu;

@@ -339,7 +339,7 @@
}

static int
-vcc_CheckUseRecurse(struct tokenlist *tl, struct proc *p, struct method *m)
+vcc_CheckUseRecurse(struct tokenlist *tl, const struct proc *p, struct method *m)
{
struct proccall *pc;
struct procuse *pu;

Modified: branches/1.1/man/vcl.7
===================================================================
--- branches/1.1/man/vcl.7 2007-07-13 14:53:48 UTC (rev 1692)
+++ branches/1.1/man/vcl.7 2007-07-13 14:54:51 UTC (rev 1693)
@@ -28,7 +28,7 @@
.\"
.\" $Id$
.\"
-.Dd July 5, 2007
+.Dd July 13, 2007
.Dt VCL 7
.Os
.Sh NAME
@@ -116,6 +116,27 @@
pipe;
}
.Ed
+.Ss Functions
+The following built-in functions are available:
+.Bl -tag -width indent
+.It Fn regsub "str" "regex" "sub"
+Returns a copy of
+.Fa str
+with all occurrences of the regular expression
+.Fa regex
+replaced with
+.Fa sub .
+Within
+.Fa sub ,
+.Va $0
+(which can also be spelled
+.Va & )
+is replaced with the entire matched string, and
+.Va $n
+is replaced with the contents of subgroup
+.Ar n
+in the matched string.
+.El
.Ss Subroutines
A subroutine is used to group code for legibility or reusability:
.Bd -literal -offset 4n
@@ -145,7 +166,7 @@
request should be handled.
Each subroutine terminates by calling one of a small number of
keywords which indicates the desired outcome.
-.Bl -tag -width "vcl_timeout"
+.Bl -tag -width indent
.\" vcl_recv
.It Cm vcl_recv
Called at the beginning of a request, after the complete request has
@@ -156,7 +177,7 @@
The
.Cm vcl_recv
subroutine may terminate with one of the following keywords:
-.Bl -tag -width "discard"
+.Bl -tag -width indent
.It Cm error Ar code Op Ar reason
Return the specified error code to the client and abandon the
request.
@@ -186,7 +207,7 @@
The
.Cm vcl_pipe
subroutine may terminate with one of the following keywords:
-.Bl -tag -width "discard"
+.Bl -tag -width indent
.It Cm error Ar code Op Ar reason
Return the specified error code to the client and abandon the
request.
@@ -205,7 +226,7 @@
The
.Cm vcl_pass
subroutine may terminate with one of the following keywords:
-.Bl -tag -width "discard"
+.Bl -tag -width indent
.It Cm error Ar code Op Ar reason
Return the specified error code to the client and abandon the
request.
@@ -218,7 +239,7 @@
The
.Cm vcl_hash
subroutine may terminate with one of the following keywords:
-.Bl -tag -width "discard"
+.Bl -tag -width indent
.It Cm hash
Proceed.
.El
@@ -230,7 +251,7 @@
The
.Cm vcl_hit
subroutine may terminate with one of the following keywords:
-.Bl -tag -width "discard"
+.Bl -tag -width indent
.It Cm error Ar code Op Ar reason
Return the specified error code to the client and abandon the
request.
@@ -253,7 +274,7 @@
The
.Cm vcl_miss
subroutine may terminate with one of the following keywords:
-.Bl -tag -width "discard"
+.Bl -tag -width indent
.It Cm error Ar code Op Ar reason
Return the specified error code to the client and abandon the
request.
@@ -274,7 +295,7 @@
The
.Cm vcl_fetch
subroutine may terminate with one of the following keywords:
-.Bl -tag -width "discard"
+.Bl -tag -width indent
.It Cm error Ar code Op Ar reason
Return the specified error code to the client and abandon the
request.
@@ -294,7 +315,7 @@
The
.Cm vcl_deliver
subroutine may terminate with one of the following keywords:
-.Bl -tag -width "discard"
+.Bl -tag -width indent
.It Cm error Ar code Op Ar reason
Return the specified error code to the client and abandon the
request.
@@ -309,7 +330,7 @@
The
.Cm vcl_timeout
subroutine may terminate with one of the following keywords:
-.Bl -tag -width "discard"
+.Bl -tag -width indent
.It Cm fetch
Request a fresh copy of the object from the backend.
.It Cm discard
@@ -324,7 +345,7 @@
The
.Cm vcl_discard
subroutine may terminate with one of the following keywords:
-.Bl -tag -width "discard"
+.Bl -tag -width indent
.It Cm discard
Discard the object.
.It Cm keep