Mailing List Archive

Virtual hosts and logfiles -- some problems
Hi!

I refer to DES' mail to the list 4th April, where he proposed using:

$ varnishlog -w /dev/stdout -c RxHeader '^Host: tinderbox.des.no' |
varnishncsa -r /dev/stdin

We've tried to get NCSA logs out of Varnish based on virtual hosts.
The method described doesn't work -- it does not sort by virtual
hosts as -o
isn't specified. However, -o and -w don't work together, so it's not
a matter of simply adding -o.

So far, what we've done is to output with -o and the RxHeader rule on
the server side and redirected the output to a file. We then want to
do the conversion on a separate server. However, varnishncsa doesn't
seem to work very welll with stored files on any server. :(

varnishncsa -d -r raw.log -- with or without the -d outputs nothing.
Some times it segfaults. varnishlog has no problems reading the log
when passed the same parameters, although the output is clearly not
as intended (see below):

11 RxHeader c Accept-Language: en-gb
11 RxHeader c Accept
25455 (null) ding: gzip, deflate
11 RxHeader c Connection: Keep-Alive
11 VCL_call c recv pass
11 Backend
8291 ObjRequest 13 default
11 RxProtocol c
21584 (null) /1.1
11 RxStatus c 200
11 RxResponse c OK
11 RxHeader c Date: Tue
14112 ObjRequest Apr 2007 11:57:38 GMT
11 RxHeader c Server
28769 ObjRequest che
11 RxHeader c Last-Modified: Wed, 04 Apr 2007 16:20:57
21514 (null) 11 RxHeader c Content-Length: 166
11 RxHeader c Content-Type: i
25903 (null) gif
11 RxHeader c X-Varnish: 1753174786
11 RxHeader c X-Forwarded-for: 194.75.128.200

The version of Varnish used is 1.0.3. This happens both with the
standard Gentoo package and a compiled version of the source tarball.

If anyone knows any way to reliably get log data per virtual host
into NCSA style logfiles, that would be appreciated.

Stephan Nedregaard,
Ymogen Ltd.
Virtual hosts and logfiles -- some problems [ In reply to ]
Hi!

I refer to DES` mail to the list 4th April, where he proposed using:

des at dma <http://projects.linpro.no/mailman/listinfo/varnish-misc> ~% varnishlog -w /dev/stdout -c RxHeader '^Host: tinderbox.des.no' | varnishncsa -r /dev/stdin


We've tried to get NCSA logs out of Varnish based on virtual hosts. The
method described doesn't work -- it does not sort by virtual hosts as -o
isn't specified. However, -o and -w don't work together, so it's not a
matter of simply adding -o.

So far, what we've done is to output with -o and the RxHeader rule on
the server side and redirected the output to a file. We then want to do
the conversion on a separate server. However, varnishncsa doesn't seem
to work very welll with stored files on any server. :(

varnishncsa -d -r raw.log -- with or without the -d outputs nothing.
Some times it segfaults. varnishlog has no problems reading the log when
passed the same parameters, although the output is clearly not as
intended (see below):

11 RxHeader c Accept-Language: en-gb
11 RxHeader c Accept
25455 (null) ding: gzip, deflate
11 RxHeader c Connection: Keep-Alive
11 VCL_call c recv pass
11 Backend
8291 ObjRequest 13 default
11 RxProtocol c
21584 (null) /1.1
11 RxStatus c 200
11 RxResponse c OK
11 RxHeader c Date: Tue
14112 ObjRequest Apr 2007 11:57:38 GMT
11 RxHeader c Server
28769 ObjRequest che
11 RxHeader c Last-Modified: Wed, 04 Apr 2007 16:20:57
21514 (null) 11 RxHeader c Content-Length: 166
11 RxHeader c Content-Type: i
25903 (null) gif
11 RxHeader c X-Varnish: 1753174786
11 RxHeader c X-Forwarded-for: 194.75.128.200

The version of Varnish used is 1.0.3. This happens both with the
standard Gentoo package and a compiled version of the source tarball.

If anyone knows any way to reliably get log data per virtual host, that
would be appreciated.

Stephan Nedregaard
Virtual hosts and logfiles -- some problems [ In reply to ]
Hi!

I refer to DES` mail to the list 4th April, where he proposed using:

des at dma <http://projects.linpro.no/mailman/listinfo/varnish-misc> ~%
varnishlog -w /dev/stdout -c RxHeader '^Host: tinderbox.des.no' |
varnishncsa -r /dev/stdin


We've tried to get NCSA logs out of Varnish based on virtual hosts. The
method described doesn't work -- it does not sort by virtual hosts as -o
isn't specified. However, -o and -w don't work together, so it's not a
matter of simply adding -o.

So far, what we've done is to output with -o and the RxHeader rule on
the server side and redirected the output to a file. We then want to do
the conversion on a separate server. However, varnishncsa doesn't seem
to work very welll with stored files on any server. :(

varnishncsa -d -r raw.log -- with or without the -d outputs nothing.
Some times it segfaults. varnishlog has no problems reading the log when
passed the same parameters, although the output is clearly not as
intended (see below):

11 RxHeader c Accept-Language: en-gb
11 RxHeader c Accept
25455 (null) ding: gzip, deflate
11 RxHeader c Connection: Keep-Alive
11 VCL_call c recv pass
11 Backend
8291 ObjRequest 13 default
11 RxProtocol c
21584 (null) /1.1
11 RxStatus c 200
11 RxResponse c OK
11 RxHeader c Date: Tue
14112 ObjRequest Apr 2007 11:57:38 GMT
11 RxHeader c Server
28769 ObjRequest che
11 RxHeader c Last-Modified: Wed, 04 Apr 2007 16:20:57
21514 (null) 11 RxHeader c Content-Length: 166
11 RxHeader c Content-Type: i
25903 (null) gif
11 RxHeader c X-Varnish: 1753174786
11 RxHeader c X-Forwarded-for: 194.75.128.200

The version of Varnish used is 1.0.3. This happens both with the
standard Gentoo package and a compiled version of the source tarball.

If anyone knows any way to reliably get log data per virtual host, that
would be appreciated.

Stephan Nedregaard
Virtual hosts and logfiles -- some problems [ In reply to ]
Stephan Nedregaard <stephan at ymogen.net> writes:
> So far, what we've done is to output with -o and the RxHeader rule on
> the server side and redirected the output to a file. We then want to
> do the conversion on a separate server. However, varnishncsa doesn't
> seem to work very welll with stored files on any server. :(

Of course - varnishncsa can't read the output from varnishlog. It can
only read raw logs either from memory or from a file written with
varnishlog -w.

Basically, we need to fix varnishncsa so you can apply the filter
there (i.e. 'varnishncsa RxHeader '^Host: host.example.com')

DES
--
Dag-Erling Sm?rgrav
Senior Software Developer
Linpro AS - www.linpro.no