Mailing List Archive

On 7/7/2008 7:19 AM, Arnab Ganguly wrote:
> Hi All,
> I am using Analog for the first time.In my Apache I have tailor made the
> log in the following format
>
> *LogFormat "%h %l %u %t \"%r\" \"Transaction time in Sec= %T\" \"Bytes
> received = %I\" %>s %b" common
>
> *My requirement is Analog should be able to understand the above format.
> I tried out the following in the analog.cfg file,
>
> APACHELOGFORMAT (%h %l %u %t \"%r\" \"Transaction time in Sec= %T\"
> \"Bytes received = %I\" %>s %b)
> LOGFILE /data/servers/testgm1/logs/access_log
> OUTFILE /data/servers/testgm1/pages/output.html
> HOSTNAME "10.146.163.301"
> HOSTNAME "http://10.146.163.301"
>
> but the output.html file generated is not proper.Any help would be very
> much appreciated.

Can you post 2 or 3 lines from your logfile? Analogs debug output
usually makes it possible to figure out any discrepancies in the log format.

> Also one more question, is it possible for Analog to monitor logs other
> than Apache?

Analog can be configured to read log files from most web servers (Analog
predates Apache). It's sometimes possible to use it to parse log files
that aren't web server logs, but that depends on the format of the
logfiles, and the type of information you're actually interested in.

Aengus
+------------------------------------------------------------------------
| TO UNSUBSCRIBE from this list:
| http://lists.meer.net/mailman/listinfo/analog-help
|
| Analog Documentation: http://analog.cx/docs/Readme.html
| List archives: http://www.analog.cx/docs/mailing.html#listarchives
| Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+------------------------------------------------------------------------

I have given below the log file snippets:

10.146.163.80 - - [27/Jun/2008:15:27:49 +0530] "POST
/spiral-bin/Collector.dll HTTP/1.0" "Transaction time in Sec= 1" "Bytes
received = 238" 200 85
10.146.163.80 - - [27/Jun/2008:15:27:49 +0530] "POST
/spiral-bin/Collector.dll HTTP/1.0" "Transaction time in Sec= 1" "Bytes
received = 304" 200 183
10.146.163.80 - - [27/Jun/2008:15:27:48 +0530] "POST
/spiral-bin/Collector.dll HTTP/1.0" "Transaction time in Sec= 4" "Bytes
received = 106142" 200 117
10.146.163.80 - - [27/Jun/2008:15:27:49 +0530] "POST
/spiral-bin/Collector.dll HTTP/1.0" "Transaction time in Sec= 4" "Bytes
received = 304" 200 142
10.146.163.80 - - [27/Jun/2008:15:27:49 +0530] "POST
/spiral-bin/Collector.dll HTTP/1.0" "Transaction time in Sec= 5" "Bytes
received = 306" 200 183
10.146.163.80 - - [27/Jun/2008:15:27:49 +0530] "POST
/spiral-bin/Collector.dll HTTP/1.0" "Transaction time in Sec= 5" "Bytes
received = 100614" 200 174
10.146.163.80 - - [27/Jun/2008:15:27:49 +0530] "POST
/spiral-bin/Collector.dll HTTP/1.0" "Transaction time in Sec= 6" "Bytes
received = 304" 200 142
10.146.163.80 - - [27/Jun/2008:15:27:49 +0530] "POST
/spiral-bin/Collector.dll HTTP/1.0" "Transaction time in Sec= 6" "Bytes
received = 306" 200 142
10.146.163.80 - - [27/Jun/2008:15:27:49 +0530] "POST
/spiral-bin/Collector.dll HTTP/1.0" "Transaction time in Sec= 7" "Bytes
received = 306" 200 183
10.146.163.80 - - [27/Jun/2008:15:27:49 +0530] "POST
/spiral-bin/Collector.dll HTTP/1.0" "Transaction time in Sec= 7" "Bytes
received = 262" 200 124

Thanks
Arnab


On Mon, Jul 7, 2008 at 6:33 PM, Aengus <analog07@eircom.net> wrote:

> On 7/7/2008 7:19 AM, Arnab Ganguly wrote:
>
>> Hi All,
>> I am using Analog for the first time.In my Apache I have tailor made the
>> log in the following format
>>
>> *LogFormat "%h %l %u %t \"%r\" \"Transaction time in Sec= %T\" \"Bytes
>> received = %I\" %>s %b" common
>>
>> *My requirement is Analog should be able to understand the above format.
>> I tried out the following in the analog.cfg file,
>>
>> APACHELOGFORMAT (%h %l %u %t \"%r\" \"Transaction time in Sec= %T\"
>> \"Bytes received = %I\" %>s %b)
>> LOGFILE /data/servers/testgm1/logs/access_log
>> OUTFILE /data/servers/testgm1/pages/output.html
>> HOSTNAME "10.146.163.301"
>> HOSTNAME "http://10.146.163.301"
>>
>> but the output.html file generated is not proper.Any help would be very
>> much appreciated.
>>
>
> Can you post 2 or 3 lines from your logfile? Analogs debug output usually
> makes it possible to figure out any discrepancies in the log format.
>
> Also one more question, is it possible for Analog to monitor logs other
>> than Apache?
>>
>
> Analog can be configured to read log files from most web servers (Analog
> predates Apache). It's sometimes possible to use it to parse log files that
> aren't web server logs, but that depends on the format of the logfiles, and
> the type of information you're actually interested in.
>
> Aengus
> +------------------------------------------------------------------------
> | TO UNSUBSCRIBE from this list:
> | http://lists.meer.net/mailman/listinfo/analog-help
> |
> | Analog Documentation: http://analog.cx/docs/Readme.html
> | List archives: http://www.analog.cx/docs/mailing.html#listarchives
> | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
> +------------------------------------------------------------------------
>

On 7/7/2008 7:19 AM, Arnab Ganguly wrote:
> Hi All,
> I am using Analog for the first time.In my Apache I have tailor made the
> log in the following format
>
> *LogFormat "%h %l %u %t \"%r\" \"Transaction time in Sec= %T\" \"Bytes
> received = %I\" %>s %b" common
>
> *My requirement is Analog should be able to understand the above format.
> I tried out the following in the analog.cfg file,
>
> APACHELOGFORMAT (%h %l %u %t \"%r\" \"Transaction time in Sec= %T\"
> \"Bytes received = %I\" %>s %b)
> LOGFILE /data/servers/testgm1/logs/access_log
> OUTFILE /data/servers/testgm1/pages/output.html
> HOSTNAME "10.146.163.301"
> HOSTNAME "http://10.146.163.301"
>
> but the output.html file generated is not proper.Any help would be very
> much appreciated.

I'm sorry, I jumped to the wrong conclusion when I saw a question that
referenced a Log format. It looks like your APACHELOGFORMAT is fine - it
parses the sample logfile lines that you posted.

What do you mean when you say that "the output.html file generated is
not proper"?

Aengus
+------------------------------------------------------------------------
| TO UNSUBSCRIBE from this list:
| http://lists.meer.net/mailman/listinfo/analog-help
|
| Analog Documentation: http://analog.cx/docs/Readme.html
| List archives: http://www.analog.cx/docs/mailing.html#listarchives
| Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+------------------------------------------------------------------------

I rechecked again.Actually the pie charts are not getting generated?OS
information is also blank.How do I generate the pie charts?
Thanks
Arnab

On Mon, Jul 7, 2008 at 4:49 PM, Arnab Ganguly <aganguly01@gmail.com> wrote:

> Hi All,
> I am using Analog for the first time.In my Apache I have tailor made the
> log in the following format
>
> *LogFormat "%h %l %u %t \"%r\" \"Transaction time in Sec= %T\" \"Bytes
> received = %I\" %>s %b" common
>
> *My requirement is Analog should be able to understand the above format.
> I tried out the following in the analog.cfg file,
>
> APACHELOGFORMAT (%h %l %u %t \"%r\" \"Transaction time in Sec= %T\" \"Bytes
> received = %I\" %>s %b)
> LOGFILE /data/servers/testgm1/logs/access_log
> OUTFILE /data/servers/testgm1/pages/output.html
> HOSTNAME "10.146.163.301"
> HOSTNAME "http://10.146.163.301"
>
> but the output.html file generated is not proper.Any help would be very
> much appreciated.
>
> Also one more question, is it possible for Analog to monitor logs other
> than Apache?
> Thanks in advance.
> Regards
> Arnab*
>
> *

Arnab Ganguly <aganguly01@gmail.com> wrote:
>> I rechecked again.Actually the pie charts are not getting
>> generated?OS information is also blank.

The OS Information comes from User Agent string, which isn't in your logs.

>> How do I generate the pie charts?

I get Pie Charts for File Size and Processing Time from your sample log file - if you don't have those two reports turned on, you won't get any charts (at least from the sample lines you provided - the charts for the Domain, Organisation. Host, Status Code, File Type, Directory and Request Reports aren't generated, because there is only one entry in each of those reports (for the 10 sample lines that you provided).

When I ran Analog with "ALL ON" to turn on all available reports, Analog generated the following warnings:

analog: Warning R: Turning off empty Redirection Report
analog: Warning R: Turning off empty Failure Report
analog: Warning R: Turning off empty Host Redirection Report
analog: Warning R: Turning off empty Host Failure Report
analog: Warning R: Turning off empty Referrer Report
analog: Warning R: Turning off empty Referring Site Report
analog: Warning R: Turning off empty Redirected Referrer Report
analog: Warning R: Turning off empty Failed Referrer Report
analog: Warning R: Turning off empty Browser Report
analog: Warning R: Turning off empty Virtual Host Report
analog: Warning R: Turning off empty Virtual Host Redirection Report
analog: Warning R: Turning off empty Virtual Host Failure Report
analog: Warning R: Turning off empty User Report
analog: Warning R: Turning off empty User Redirection Report
analog: Warning R: Turning off empty User Failure Report
analog: Warning R: Turning off empty Search Query Report
analog: Warning R: Turning off empty Search Word Report
analog: Warning R: Turning off empty Internal Search Query Report
analog: Warning R: Turning off empty Internal Search Word Report
analog: Warning R: Turning off empty Browser Summary
analog: Warning R: Turning off empty Operating System Report
analog: Warning R: In Domain Report, turning off pie chart of only one wedge
analog: Warning R: In Organisation Report, turning off pie chart of only one wedge
analog: Warning R: In Host Report, turning off pie chart of only one wedge
analog: Warning R: In Status Code Report, turning off pie chart of only one wedge
F: Opening Charts/proctime.png as pie chart file
F: Closing Charts/proctime.png
F: Opening Charts/size.png as pie chart file
F: Closing Charts/size.png
analog: Warning R: In File Type Report, turning off pie chart of only one wedge
analog: Warning R: In Directory Report, turning off pie chart of only one wedge
analog: Warning R: In Request Report, turning off pie chart of only one wedge

The Empty Report warnings occur because there isn't enough information in the logfile to generate those reports (no Referrer or no User Agent, in most cases. Some of the other reports may show up in your real logs, such as the Failure Report).

The "turning off pie chart" warnings occur because all of your sample lines come from the same IP address, and request the same file, so there is only one entry, therefore Analog doesn't generate a Pie Chart.

Aengus

+------------------------------------------------------------------------
| TO UNSUBSCRIBE from this list:
| http://lists.meer.net/mailman/listinfo/analog-help
|
| Analog Documentation: http://analog.cx/docs/Readme.html
| List archives: http://www.analog.cx/docs/mailing.html#listarchives
| Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+------------------------------------------------------------------------

2008/7/7 Arnab Ganguly <aganguly01@gmail.com>:
> I rechecked again.Actually the pie charts are not getting generated?OS
> information is also blank.How do I generate the pie charts?
> Thanks
> Arnab
>

There is no OS information because you don't have that information in
your logfiles!

As for the pie charts, check out the CHARTDIR and LOCALCHARTDIR
commands. You probably need to specify both of these.

--
Stephen Turner
+------------------------------------------------------------------------
| TO UNSUBSCRIBE from this list:
| http://lists.meer.net/mailman/listinfo/analog-help
|
| Analog Documentation: http://analog.cx/docs/Readme.html
| List archives: http://www.analog.cx/docs/mailing.html#listarchives
| Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+------------------------------------------------------------------------

Hi All,
Thanks a lot for all your help.Explanation was really comprehensive and
things are clear.Lastly I face one more issue.I am able to generate the
output.html file and related png files.But issue is that when I try to
access those files from the browser the output in the web page doesn't come
properly below I have given the snippet

<?xml version="1.0" encoding="ISO-8859-1"?>
<?xml-stylesheet href="#internalStyle" type="text/css"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Web Server Statistics for http://10.146.163.30</title>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1" />
<meta name="robots" content="noindex,nofollow" />
<meta name="generator" content="analog 6.0/Unix" />
<style type="text/css" id="internalStyle">
h2 {
background-color: #A0C0F0;
width: 98%;
padding: 3px 6px;
}
table {
text-align: right;
margin-left: 30px;
background-color: #D0E0F0;
border-collapse: collapse;
}


But when I copy the files locally and try to access it the pages comes
properly.I guess some more settings are required.Please let me know.
Thanks and regards
Arnab

On Mon, Jul 7, 2008 at 11:36 PM, Aengus <analog07@eircom.net> wrote:

> Arnab Ganguly <aganguly01@gmail.com> wrote:
> >> I rechecked again.Actually the pie charts are not getting
> >> generated?OS information is also blank.
>
> The OS Information comes from User Agent string, which isn't in your logs.
>
> >> How do I generate the pie charts?
>
> I get Pie Charts for File Size and Processing Time from your sample log
> file - if you don't have those two reports turned on, you won't get any
> charts (at least from the sample lines you provided - the charts for the
> Domain, Organisation. Host, Status Code, File Type, Directory and Request
> Reports aren't generated, because there is only one entry in each of those
> reports (for the 10 sample lines that you provided).
>
> When I ran Analog with "ALL ON" to turn on all available reports, Analog
> generated the following warnings:
>
> analog: Warning R: Turning off empty Redirection Report
> analog: Warning R: Turning off empty Failure Report
> analog: Warning R: Turning off empty Host Redirection Report
> analog: Warning R: Turning off empty Host Failure Report
> analog: Warning R: Turning off empty Referrer Report
> analog: Warning R: Turning off empty Referring Site Report
> analog: Warning R: Turning off empty Redirected Referrer Report
> analog: Warning R: Turning off empty Failed Referrer Report
> analog: Warning R: Turning off empty Browser Report
> analog: Warning R: Turning off empty Virtual Host Report
> analog: Warning R: Turning off empty Virtual Host Redirection Report
> analog: Warning R: Turning off empty Virtual Host Failure Report
> analog: Warning R: Turning off empty User Report
> analog: Warning R: Turning off empty User Redirection Report
> analog: Warning R: Turning off empty User Failure Report
> analog: Warning R: Turning off empty Search Query Report
> analog: Warning R: Turning off empty Search Word Report
> analog: Warning R: Turning off empty Internal Search Query Report
> analog: Warning R: Turning off empty Internal Search Word Report
> analog: Warning R: Turning off empty Browser Summary
> analog: Warning R: Turning off empty Operating System Report
> analog: Warning R: In Domain Report, turning off pie chart of only one
> wedge
> analog: Warning R: In Organisation Report, turning off pie chart of only
> one wedge
> analog: Warning R: In Host Report, turning off pie chart of only one wedge
> analog: Warning R: In Status Code Report, turning off pie chart of only one
> wedge
> F: Opening Charts/proctime.png as pie chart file
> F: Closing Charts/proctime.png
> F: Opening Charts/size.png as pie chart file
> F: Closing Charts/size.png
> analog: Warning R: In File Type Report, turning off pie chart of only one
> wedge
> analog: Warning R: In Directory Report, turning off pie chart of only one
> wedge
> analog: Warning R: In Request Report, turning off pie chart of only one
> wedge
>
> The Empty Report warnings occur because there isn't enough information in
> the logfile to generate those reports (no Referrer or no User Agent, in most
> cases. Some of the other reports may show up in your real logs, such as the
> Failure Report).
>
> The "turning off pie chart" warnings occur because all of your sample lines
> come from the same IP address, and request the same file, so there is only
> one entry, therefore Analog doesn't generate a Pie Chart.
>
> Aengus
>
> +------------------------------------------------------------------------
> | TO UNSUBSCRIBE from this list:
> | http://lists.meer.net/mailman/listinfo/analog-help
> |
> | Analog Documentation: http://analog.cx/docs/Readme.html
> | List archives: http://www.analog.cx/docs/mailing.html#listarchives
> | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
> +------------------------------------------------------------------------
>

On 7/8/2008 3:44 AM, Arnab Ganguly wrote:
> Hi All,
> Thanks a lot for all your help.Explanation was really comprehensive and
> things are clear.Lastly I face one more issue.I am able to generate the
> output.html file and related png files.But issue is that when I try to
> access those files from the browser the output in the web page doesn't
> come properly below I have given the snippet

If the problem is that the HTML page itself is not displayed properly,
then it sounds like you have some sort of issue with MIME Types on your
web server - I really have no idea what might be going on there.
(Thought I have a vague recollection that there was a problem with using
anlgform.pl and IE6 at one point).

If the web page is displaying properly on your server, but the graphs
aren't being displayed, then you need to look into the CHARTDIR and
LOCALCHARTDIR commands.

http://analog.cx/docs/othreps.html#CHARTDIR

Aengus
+------------------------------------------------------------------------
| TO UNSUBSCRIBE from this list:
| http://lists.meer.net/mailman/listinfo/analog-help
|
| Analog Documentation: http://analog.cx/docs/Readme.html
| List archives: http://www.analog.cx/docs/mailing.html#listarchives
| Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+------------------------------------------------------------------------