I am using analog 6.0. My site has web logs stretching back over 10 years,
amounting to a total of 3.6GB. I run analog every day using a crontab
entry.
Most of the time this is fine and analog processes the logs in minutes.
However, intermittently, it seems to ignore the cache and processes very
slowly, with the result that it is not finished 24 hours later, when a new
invocation of analog starts up. If I don¹t manually kill the running copies
of analog, after a few days the server slows to a crawl, as the processor
and i/o become devoted to running multiple copies of analog each trying to
resolve millions of IP addresses.
What could be the cause of this? And how can I solve the problem?
Here is the crontab job:
20 6 * * * nice /usr/local/bin/analog
+g/Volumes/Documents/VirtualSites/socresonline/stats/analog-sro.cfg
Here is the cache file:
-rw-r--r-- 1 root wheel 95716037 Aug 10 06:35 analog-dns-cache
I have deleted duplicate entries in the cache using a utility script called
trimdns.pl, but while this reduced the size of the cache by about 50%, it
has not cured the problem.
Thanks!
Nigel
________________________________________________________________
Professor Nigel Gilbert, ScD, FREng, AcSS, Professor of Sociology,
University of Surrey, Guildford GU2 7XH, UK. +44 (0)1483 689173
amounting to a total of 3.6GB. I run analog every day using a crontab
entry.
Most of the time this is fine and analog processes the logs in minutes.
However, intermittently, it seems to ignore the cache and processes very
slowly, with the result that it is not finished 24 hours later, when a new
invocation of analog starts up. If I don¹t manually kill the running copies
of analog, after a few days the server slows to a crawl, as the processor
and i/o become devoted to running multiple copies of analog each trying to
resolve millions of IP addresses.
What could be the cause of this? And how can I solve the problem?
Here is the crontab job:
20 6 * * * nice /usr/local/bin/analog
+g/Volumes/Documents/VirtualSites/socresonline/stats/analog-sro.cfg
Here is the cache file:
-rw-r--r-- 1 root wheel 95716037 Aug 10 06:35 analog-dns-cache
I have deleted duplicate entries in the cache using a utility script called
trimdns.pl, but while this reduced the size of the cache by about 50%, it
has not cured the problem.
Thanks!
Nigel
________________________________________________________________
Professor Nigel Gilbert, ScD, FREng, AcSS, Professor of Sociology,
University of Surrey, Guildford GU2 7XH, UK. +44 (0)1483 689173