Mailing List Archive

problem with webalizer - huge DNS cache file
Hello,

I have problem with webalizer. I want it to perform reverse dns lookups. The
problem is dns cache file is so huge! For example:

--->
globi@ornak globi $ webazolver -d -N 1
-D /var/www/webalizer/cache.db /var/log/httpd/xxxxx-access_log

Webalizer V2.01-10 (Linux 2.6.8-gentoo-r3) Polish
U¿ywam pliku logu /var/log/httpd/cms.graff.pl-access_log (clf)
Przeszukujê DNS (1): Giving 217.116.100.252 (fc6474d9) to Child 0 for
resolving
Child got work: fc6474d9(4)
Child got gprs6.idea.pl for fc6474d9(4), 13 bytes
Work requested from Child 0
Got a result (0): 217.116.100.252 -> gprs6.idea.pl
Child 0 back in task pool
Child 0 got closed input, shutting down
Reaped Child 1230
1 adres(ów) w 0.00 sekund

globi@ornak globi $ ls -l /var/www/webalizer/cache.db
-rw-r--r-- 1 globi globi 1179648 wrz 22 14:41 /var/www/webalizer/cache.db
globi@ornak globi $ ls -l /var/log/httpd/xxxxx-access_log
-rw-r--r-- 1 root root 248 wrz 21 14:02 /var/log/httpd/xxxxx-access_log
<---

As you can see file /var/log/httpd/xxxxx-access_log contains only 2 lines and
information regarding only 1 IP address. I have no idea why the DNS cache
file grows to over 1MB.

Another example:

--->
globi@ornak globi $ webazolver -N 10
-D /var/www/webalizer/cache.db /var/log/httpd/yyyyy-access_log

Webalizer V2.01-10 (Linux 2.6.8-gentoo-r3) Polish
U¿ywam pliku logu /var/log/httpd/yyyyy-access_log (clf)
Przeszukujê DNS (10): 532 adres(ów) w 37.02 sekund, 14/sec
globi@ornak globi $ ls -l /var/www/webalizer/cache.db
-rw-r--r-- 1 globi globi 183894016 wrz 22 14:47 /var/www/webalizer/cache.db
<---

This time access_log file contains 532 different IP addresses and webalizer
generates DNS cache file bigger than 180 MB!!

Have you got any ideas what's going on?
I use current version of webalizer and dnscache as a DNS resolver.
Mayby webalizer doesn't work well with dnscache??

Mayby I spoiled webalizer during installation? (But it works ok except
problems with dns). I wanted to install webalizer in polish language version
so I edited file
/usr/portage/app-admin/webalizer/webalizer-2.01.10-r4.ebuild
and added option 'with-language' so it looks like:

src_compile() {
econf --with-language=polish --enable-dns --with-db=/usr/include/db1/ || die
make || die
}

Then I made:

ebuild webalizer-2.01.10-r4.ebuild compile
ebuild webalizer-2.01.10-r4.ebuild install
ebuild webalizer-2.01.10-r4.ebuild qmerge

There was no problems during installation and as I said above webalizer works
ok.

regards,
Dariusz Zmokly

--
gentoo-user@gentoo.org mailing list
Re: problem with webalizer - huge DNS cache file [ In reply to ]
Dariusz Zmokly wrote:
> I have problem with webalizer. I want it to perform reverse dns lookups. The
> problem is dns cache file is so huge! For example:

your dnscache file isn't that big. The one on my main web server is over
300 megabytes, but it's what's called a *sparse* file, so in reality
it's only 171 megabytes. Your 1 megabyte file may *appear* to be that
large, but in reality it may only be 10k.

sparse files are files which look big, but the filesystem knows that
"blanks" in the files don't need to be stored (just yet).

To see how big it really is, do this:

du -h dns_cache.db

it should show you how big it really is.

--
gentoo-user@gentoo.org mailing list
Re: problem with webalizer - huge DNS cache file [ In reply to ]
On Wednesday 22 of September 2004 16:33, Billy wrote:
> To see how big it really is, do this:
>
> du -h dns_cache.db
>
> it should show you how big it really is.

globi@ornak globi $ du -h /var/www/webalizer/cache.db
168M /var/www/webalizer/cache.db

Hmm, it doesn't make too much difference. 180M vs 168M.
We host over 100 clients on that server so I don't think it is practice
solution to use such cache. On the other hand I wouldn't like to slow down
apache by doing reverse DNS lookups.

Dariusz Zmokly

--
gentoo-user@gentoo.org mailing list
Re: problem with webalizer - huge DNS cache file [ In reply to ]
On Thu, 2004-23-09 at 10:55 +0200, Dariusz Zmokly wrote:
> We host over 100 clients on that server so I don't think it is
> practice solution to use such cache. On the other hand I wouldn't like
> to slow down apache by doing reverse DNS lookups.

If all you really need is country-of-origin, consider not doing reverse
lookups, and instead use the GeoIP patch to Webalizer:

http://sysd.org/proj/log.php#glzr

The inside the patch itself are pretty good instructions. (You have to
download the GeoIP library from maxmind along the way). It works great
for us.

AfC
Sydney

--
Andrew Frederick Cowie

OPERATIONAL DYNAMICS
Managing massive changes and upgrades to mission critical systems

Recent presentations and conference papers posted:
operationaldynamics.com/talks