Mailing List Archive

Varnish and awstats
Is there a trick to getting awstats to work with varnish?

I tried doing varnishncsa -a -w filename and then pointed awstats to this file but it gives me an error that the log format must be incorrect

Create/Update database for config "/etc/awstats/awstats.mmvk.hidden.net.conf" by AWStats version 6.7 (build 1.892) From data in log file "hidden/lb.log"... Phase 1 : First bypass old records, searching new record... Direct access after last parsed record (after line 9835) AWStats did not find any valid log lines that match your LogFormat parameter, in the 50th first non commented lines read of your log. Your log file /hidden/lb.log must have a bad format or LogFormat parameter setup does not match this format. Your AWStats LogFormat parameter is: 4 This means each line in your web server log file need to have "common log format" like this
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://projects.linpro.no/pipermail/varnish-misc/attachments/20070822/4e235007/attachment.htm
Varnish and awstats [ In reply to ]
From: jdouglas <jdouglas@bksnetworks.com>
>Is there a trick to getting awstats to work with varnish?
>
>I tried doing varnishncsa -a -w filename and then pointed awstats to this file but it gives me an error that the log format must be incorrect
>
>a bad format or LogFormat parameter setup does not match this format. Your AWStats
>LogFormat parameter is: 4 This means each line in your web server log file need to
>have "common log format" like this

Try setting the "Combined" logformat instead, which is what varnishncsa produces..
From the varnishncsa man page : "Display Varnish logs in Apache / NCSA combined log format"

Regards
--
Denis Braekhus - Teknisk Ansvarlig ABC Startsiden AS
http://www.startsiden.no
Varnish and awstats [ In reply to ]
Weird, your varnishncsa seems to output differently from mine, anything special configured?

A line from my output :
85.19.*.* - - [22/Aug/2007:09:53:56 +0200] "GET /img/frimerkertopbg.gif HTTP/1.0" 304 - "http://www.startsiden.no/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)"

74.93.253.*** - - 22/Aug/2007:03:20:44 -0400 "GET /apps/WebObjects/BKSMMVK.woa/6/wo/bEBLQd7bB7Vl3jOsSMiiZM/11.1.0 HTTP/1.1" 3850 "Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"

compared, it seems your datestring misses the [] brackets, and referer bits are missing too.

Unfortunately I do not know why this is, but a solution would of course be to specify the type of loglines in a custom logdefinition in AWStats. AFAIK that is possible with the configuration. Check up on the AWStats docs.

Regards
--
Denis

----- Original Message -----
From: jdouglas <jdouglas@bksnetworks.com>
To: Denis Br?khus <denis at startsiden.no>
Sent: Wednesday, August 22, 2007 9:53:56 AM GMT+0100 Europe/Berlin
Subject: Re: Varnish and awstats

From the varnish generated log file
74.93.253.*** - - 22/Aug/2007:03:20:44 -0400 "GET /apps/WebObjects/BKSMMVK.woa/6/wo/bEBLQd7bB7Vl3jOsSMiiZM/11.1.0 HTTP/1.1" 3850 "Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"

The awstats error

Create/Update database for config "/etc/awstats/awstats.mmvk.brandkey.net.conf" by AWStats version 6.7 (build 1.892) From data in log file "/home/deploy/lb.log"... Phase 1 : First bypass old records, searching new record... Searching new records from beginning of log file... AWStats did not find any valid log lines that match your LogFormat parameter, in the 50th first non commented lines read of your log. Your log file /hidden/lb.log must have a bad format or LogFormat parameter setup does not match this format. Your AWStats LogFormat parameter is: 1 This means each line in your web server log file need to have "combined log format" like this: 111.22.33.44 - - [10/Jan/2001:02:14:14 +0200] "GET / HTTP/1.1" 200 1234 "http://www.fromserver.com/from.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)" And this is an example of records AWStats found in your log file (the record number 50 in your log): 74.93.253.*** - - 22/Aug/2007:03:20:42 -0400 "GET /bksstdskins/039_0014M.gif HTTP/1.1" 1041 "http://hidden/apps/WebObjects/BKSMMVK.woa/6/wo/bEBLQd7bB7Vl3jOsSMiiZM/2.3.0.5.0" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"



From: Denis Br?khus [mailto:denis@startsiden.no]
To: jdouglas [mailto:jdouglas at bksnetworks.com]
Sent: Wed, 22 Aug 2007 03:43:47 -0400
Subject: Re: Varnish and awstats

Could you paste one line from that log? And paste the relevant part of the AWstats conf file?

----- Original Message -----
From: jdouglas < jdouglas@bksnetworks.com >
To: Denis Br?khus < denis at startsiden.no >
Sent: Wednesday, August 22, 2007 9:24:38 AM GMT+0100 Europe/Berlin
Subject: Re: Varnish and awstats

Changing this makes no difference. I get the same error.





From: Denis Br?khus [mailto: denis@startsiden.no ]
To: varnish-misc [mailto: varnish-misc at projects.linpro.no ]
Cc: jdouglas [mailto: jdouglas at bksnetworks.com ]
Sent: Wed, 22 Aug 2007 02:05:35 -0400
Subject: Re: Varnish and awstats

From: jdouglas < jdouglas@bksnetworks.com >
>Is there a trick to getting awstats to work with varnish?
>
>I tried doing varnishncsa -a -w filename and then pointed awstats to this file but it gives me an error that the log format must be incorrect
>
>a bad format or LogFormat parameter setup does not match this format. Your AWStats
>LogFormat parameter is: 4 This means each line in your web server log file need to
>have "common log format" like this

Try setting the "Combined" logformat instead, which is what varnishncsa produces..
From the varnishncsa man page : "Display Varnish logs in Apache / NCSA combined log format"

Regards
--
Denis Braekhus - Teknisk Ansvarlig ABC Startsiden AS
http://www.startsiden.no



--
Denis Braekhus - Teknisk Ansvarlig ABC Startsiden AS
http://www.startsiden.no


--
Denis Braekhus - Teknisk Ansvarlig ABC Startsiden AS
http://www.startsiden.no