Mailing List Archive

Darren Munt <darrenm@ardex.com.au> wrote:
> I'm trying to suppress internal referrers from the Referrer Report.
> I've put the following lines in the config file:
>
> REFREPEXCLUDE http://mydomain.com.au/
> REFSITEEXCLUDE http://mydomain.com.au/
>
> However this doesn't suppress requests from mydomain.com.au from the
> report. Obviously I'm missing something but I can't see what?

REFREPEXCLUDE http://mydomain.com.au/
will only exclude http://mydomain.com.au/ as a referrer. It won't exclude http://mydomain.com.au/index.html (for example).

Try
REFREPEXCLUDE http://mydomain.com.au/*
REFSITEEXCLUDE http://mydomain.com.au/*

Aengus

+------------------------------------------------------------------------
| TO UNSUBSCRIBE from this list:
| http://lists.meer.net/mailman/listinfo/analog-help
|
| Analog Documentation: http://analog.cx/docs/Readme.html
| List archives: http://www.analog.cx/docs/mailing.html#listarchives
| Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+------------------------------------------------------------------------

>Try
>REFREPEXCLUDE http://mydomain.com.au/*
>REFSITEEXCLUDE http://mydomain.com.au/*

Yes I tried that already to no avail - then I found the following in the
readme:

===
All directory names end in slashes, so DIRINCLUDE and DIREXCLUDE, and
REFSITEINCLUDE and REFSITEEXCLUDE, implicitly add a trailing slash even
if you don't give one. This sometimes catches people out in the
following situation.

REFSITEEXCLUDE http://my.host.com/* # probably not what you want
means not to list subdirectories of the referring site
http://my.host.com/, but to keep the site itself in the list. To exclude
the site completely, just use
REFSITEEXCLUDE http://my.host.com/
===

So I took the * out but no change.





+------------------------------------------------------------------------
| TO UNSUBSCRIBE from this list:
| http://lists.meer.net/mailman/listinfo/analog-help
|
| Analog Documentation: http://analog.cx/docs/Readme.html
| List archives: http://www.analog.cx/docs/mailing.html#listarchives
| Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+------------------------------------------------------------------------

Darren Munt <darrenm@ardex.com.au> wrote:
>> Try
>> REFREPEXCLUDE http://mydomain.com.au/*
>> REFSITEEXCLUDE http://mydomain.com.au/*
>
> Yes I tried that already to no avail - then I found the following in
> the readme:
>
> ===
> All directory names end in slashes, so DIRINCLUDE and DIREXCLUDE, and
> REFSITEINCLUDE and REFSITEEXCLUDE, implicitly add a trailing slash
> even if you don't give one. This sometimes catches people out in the
> following situation.
>
> REFSITEEXCLUDE http://my.host.com/* # probably not what you want
> means not to list subdirectories of the referring site
> http://my.host.com/, but to keep the site itself in the list. To
> exclude the site completely, just use
> REFSITEEXCLUDE http://my.host.com/
> ===
>
> So I took the * out but no change.

REFREPEXCLUDE http://www.mysite.com/*

definitely works for me. My "Referrer Report" is significantly different with that line added, because there are hundreds of "internal referrers" removed from the Referrer Report. The * is necessary to match all the different refering pages on my site.

REFSITEEXCLUDE http://www.mysite.com/ and
REFSITEEXCLUDE http://www.mysite.com both have the same small effect on my "Referring Site Report" - the top line of the report (my own site is the top referring site) disappears.

REFSITEEXCLUDE http://www.mysite.com/* doesn't work, because the * isn't part of the site name.

REFSITEEXCLUDE only effects the "Referring Site Report". REFREPEXCLUDE only effects the "Referrer Report"

Aengus

+------------------------------------------------------------------------
| TO UNSUBSCRIBE from this list:
| http://lists.meer.net/mailman/listinfo/analog-help
|
| Analog Documentation: http://analog.cx/docs/Readme.html
| List archives: http://www.analog.cx/docs/mailing.html#listarchives
| Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+------------------------------------------------------------------------

OK, so something weird is going on. Here are the top few lines of my
config file:

HOSTNAME "www.mydomain.com.au"
HOSTURL http://www.mydomain.com.au/
HOSTALIAS REGEXP:^([^.]*)$ $1.mydomain.com.au
HOSTEXCLUDE *.mydomain.com.au

# REQINCLUDE pages
REQLINKINCLUDE pages
REFLINKINCLUDE *
REDIRREFLINKINCLUDE *
FAILREFLINKINCLUDE *

REFREPEXCLUDE http://mydomain.com.au/*
REFSITEEXCLUDE http://mydomain.com.au/*

Is there perhaps something in one of the other lines that might be
causing it? I've run it several times now, checked the dates on the
config file vs the run date at the top of the report to make sure I'm
looking at the refreshed version of the HTML file. For some reason, it
is ignoring the REFREPEXCLUDE command.

Also I'm using version 6.0 if that makes a difference.


+------------------------------------------------------------------------
| TO UNSUBSCRIBE from this list:
| http://lists.meer.net/mailman/listinfo/analog-help
|
| Analog Documentation: http://analog.cx/docs/Readme.html
| List archives: http://www.analog.cx/docs/mailing.html#listarchives
| Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+------------------------------------------------------------------------

At Thursday, November 15, 2007 7:41 PM, Darren Munt <darrenm@ardex.com.au>
wrote:

> OK, so something weird is going on. Here are the top few lines of my
> config file:
>
> HOSTNAME "www.mydomain.com.au"
> HOSTURL http://www.mydomain.com.au/
> HOSTALIAS REGEXP:^([^.]*)$ $1.mydomain.com.au
> HOSTEXCLUDE *.mydomain.com.au
>
> # REQINCLUDE pages
> REQLINKINCLUDE pages
> REFLINKINCLUDE *
> REDIRREFLINKINCLUDE *
> FAILREFLINKINCLUDE *
>
> REFREPEXCLUDE http://mydomain.com.au/*
> REFSITEEXCLUDE http://mydomain.com.au/*
>
> Is there perhaps something in one of the other lines that might be
> causing it? I've run it several times now, checked the dates on the
> config file vs the run date at the top of the report to make sure I'm
> looking at the refreshed version of the HTML file. For some reason, it
> is ignoring the REFREPEXCLUDE command.
>
> Also I'm using version 6.0 if that makes a difference.

Post 4 lines from your logfile. If the Referrer field matches
http://mydomain.com.au/*, then those referrers will not be listed in your
Referrer Report. The log entries will still be included in your Request
Report, and your Host Report, because REFREPEXCLUDE just excludes them
from the Referrer Report - the actual log entries are still included for
every other relevant report.

REFSITEEXCLUDE http://mydomain.com.au/* won't work - you need to leave off
the *. It will only change the Referring Site report - the log enties are
still used for all the other relevant reports.

Aengus

+------------------------------------------------------------------------
| TO UNSUBSCRIBE from this list:
| http://lists.meer.net/mailman/listinfo/analog-help
|
| Analog Documentation: http://analog.cx/docs/Readme.html
| List archives: http://www.analog.cx/docs/mailing.html#listarchives
| Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+------------------------------------------------------------------------

>Post 4 lines from your logfile.

Here are four lines from one of the log files. The first two and the
fourth contain the referrer that I am trying to suppress.

2007-11-14 13:02:20 xxx.xxx.x.xx POST /admin/default.asp - 80 -
xxx.xxx.x.xx
Mozilla/5.0+(Macintosh;+U;+Intel+Mac+OS+X;+en-US;+rv:1.8.1.9)+Gecko/2007
1025+Firefox/2.0.0.9 http://www.turfmonthly.com.au/admin/Default.asp 200
0 0 10483 726

2007-11-14 13:02:24 xxx.xxx.x.xx POST /admin/Default.asp - 80 -
xxx.xxx.x.xx
Mozilla/5.0+(Macintosh;+U;+Intel+Mac+OS+X;+en-US;+rv:1.8.1.9)+Gecko/2007
1025+Firefox/2.0.0.9 http://www.turfmonthly.com.au/admin/default.asp 200
0 0 16426 747

2007-11-14 13:02:26 xxx.xxx.x.xx GET /article.asp id=1679 80 -
xxx.xxx.x.xx
Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.2;+.NET+CLR+1.1.4322)
http://search.live.com/results.aspx?q=monthly&mrt=en-us&FORM=LIVSOP 200
0 0 24510 393

2007-11-14 13:02:26 xxx.xxx.x.xx GET /inc/newsmain.css - 80 -
xxx.xxx.x.xx
Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.2;+.NET+CLR+1.1.4322)
http://www.turfmonthly.com.au/article.asp?id=1679 200 0 0 1570 427

The config file line is:

REFREPEXCLUDE http://turfmonthly.com.au/*

The first couple of lines of the Referrer Report:

153656: http://www.turfmonthly.com.au/article.asp
7087: http://www.turfmonthly.com.au/article.asp?id=1837
4457: http://www.turfmonthly.com.au/article.asp?id=1963


+------------------------------------------------------------------------
| TO UNSUBSCRIBE from this list:
| http://lists.meer.net/mailman/listinfo/analog-help
|
| Analog Documentation: http://analog.cx/docs/Readme.html
| List archives: http://www.analog.cx/docs/mailing.html#listarchives
| Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+------------------------------------------------------------------------

At Thursday, November 15, 2007 10:16 PM, Darren Munt
<darrenm@ardex.com.au> wrote:

>> Post 4 lines from your logfile.
>
> Here are four lines from one of the log files. The first two and the
> fourth contain the referrer that I am trying to suppress.
>
> 2007-11-14 13:02:20 xxx.xxx.x.xx POST /admin/default.asp - 80 -
> xxx.xxx.x.xx
> Mozilla/5.0+(Macintosh;+U;+Intel+Mac+OS+X;+en-US;+rv:1.8.1.9)+Gecko/2007
> 1025+Firefox/2.0.0.9 http://www.turfmonthly.com.au/admin/Default.asp
> 200 0 0 10483 726

analog: Warning F: Can't auto-detect format of logfile au2.log: ignoring
it
(For help on all errors and warnings, see docs/errors.html)

It's too late to figure out a logformat. Post the header lines from your
logfile, and I'll look at it in the morning.

And post the results of your Referrer Report against just these 4 ines
from your logfile.

Aengus

+------------------------------------------------------------------------
| TO UNSUBSCRIBE from this list:
| http://lists.meer.net/mailman/listinfo/analog-help
|
| Analog Documentation: http://analog.cx/docs/Readme.html
| List archives: http://www.analog.cx/docs/mailing.html#listarchives
| Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+------------------------------------------------------------------------

> It's too late to figure out a logformat. Post the header lines from
your
> logfile, and I'll look at it in the morning.

> And post the results of your Referrer Report against just these 4 ines

> from your logfile.

Header from the log file:

====
#Software: Microsoft Internet Information Services 6.0
#Version: 1.0
#Date: 2007-11-14 13:00:59
#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port
cs-username c-ip cs(User-Agent) cs(Referer) sc-status sc-substatus
sc-win32-status sc-bytes cs-bytes
====

I created a truncated log file with only the header lines and the four
lines I posted. The only change I made to the config file was to use a
single specific log file:

LOGFILE C:\WINDOWS\system32\LogFiles\W3SVC921719038\logfile.log

In place of:

LOGFILE C:\WINDOWS\system32\LogFiles\W3SVC921719038\ex*.log

When I ran the job, it produced a report that had suppressed
www.turfmonthly.com.au from the referrer report:

====
Referrer Report
(Go To: Top: General Summary: Weekly Report: Daily Report: Daily
Summary: Hourly Summary: Domain Report: Referrer Report: Referring Site
Report: Status Code Report: Request Report)

This report lists the referrers (where people followed links from, or
pages which included this site's images).

Listing referring URLs, sorted by the number of requests.

#reqs: URL
-----: ---
1: http://search.live.com/results.aspx
====

Appreciate the time you're spending on this. It's Friday afternoon here,
so if you need anything else, I'll be back next week.


+------------------------------------------------------------------------
| TO UNSUBSCRIBE from this list:
| http://lists.meer.net/mailman/listinfo/analog-help
|
| Analog Documentation: http://analog.cx/docs/Readme.html
| List archives: http://www.analog.cx/docs/mailing.html#listarchives
| Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+------------------------------------------------------------------------

At Thursday, November 15, 2007 11:11 PM, Darren Munt
<darrenm@ardex.com.au> wrote:

> I created a truncated log file with only the header lines and the four
> lines I posted. The only change I made to the config file was to use a
> single specific log file:
>
> LOGFILE C:\WINDOWS\system32\LogFiles\W3SVC921719038\logfile.log
>
> In place of:
>
> LOGFILE C:\WINDOWS\system32\LogFiles\W3SVC921719038\ex*.log
>
> When I ran the job, it produced a report that had suppressed
> www.turfmonthly.com.au from the referrer report:

I don't know what to tell you - if it works properly on a test log file,
and all you changed was the LOGFILE command in the analog.cfg file, then
it should work for the main logfile.

Aengus

+------------------------------------------------------------------------
| TO UNSUBSCRIBE from this list:
| http://lists.meer.net/mailman/listinfo/analog-help
|
| Analog Documentation: http://analog.cx/docs/Readme.html
| List archives: http://www.analog.cx/docs/mailing.html#listarchives
| Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+------------------------------------------------------------------------