Mailing List Archive

> I'm very new to exim and I'm just trying to be able to map my progress
> as I go along..
>
> I'm trying to run eximstats to get some output and it just keeps coming
> up with **** No valid lines read no matter what arguments I use.
>
> [/var/log/exim]# 85 > eximstats /var/log/exim/mainlog
> **** No valid log lines read
>
> [/var/log/exim]# 86 > head mainlog
> 2004-04-14 14:30:08 IPv6 socket creation failed: Protocol not supported
> 2004-04-14 14:30:08 Failed to create IPv6 socket for wildcard listening
> (No such file or directory): will use IPv4
> 2004-04-14 14:30:08 exim 4.31 daemon started: pid=29725, -q30m,
> listening for SMTP on port 25 (IPv4)
> 2004-04-14 14:30:08 Start queue run: pid=29727
> 2004-04-14 14:30:08 End queue run: pid=29727
> 2004-04-14 14:30:09 no host name found for IP address 61.82.213.150
> 2004-04-14 14:30:10 H=c-24-4-169-150.client.comcast.net [24.4.169.150]
> F=<bugrtxizbga@eurasia.msk.ru> rejected RCPT <adam.weiss@mhv.net>: found
> in dnsbl
> 2004-04-14 14:30:10 H=66.159.182.206.adsl.snet.net [66.159.182.206]
> F=<rrdtdns@tampabay.rr.com> rejected RCPT <goliath@mhv.net>: found in
> dnsbl

Are those the only entries in your logfile? eximstats looks for deliveries
and ignores such things as rejects due to the use of DNSBLs...

--Alex

Alex Lomas wrote:

> Are those the only entries in your logfile? eximstats looks for
deliveries
> and ignores such things as rejects due to the use of DNSBLs...

Ignorance is bliss :)

yes they are all dnsbl related.. there is not a successfull delivery in
the whole file.

Then let me ask a more relevant question.. is there something that will
keep stats on what exim blocked? :)

I'm trying to use exim as a filtering program, so blocking rejected
stats would be more 'of interest' at this time..

Am I at a loss?

- Brian

Brian,

Newer versions of eximstats will count the occurrences of specified patterns by use of the '-pattern' option. Eg:

eximstats -pattern "DNS Block" /dnsbl/ mainlog

However, you will get an overall failure unless one valid received or sent line is found. If you're not afraid of editing the code, you can remove this check by finding and deleting the code block :

if ($begin eq "9999-99-99 99:99:99") {
print "**** No valid log lines read\n";
exit 1;
}

Alternatively, echo a valid log line onto the end of the log before you parse it :-)

Steve Campbell

--

On Thu, 2004-04-15 at 15:25, Steve Campbell wrote:
> Newer versions of eximstats will count the occurrences of specified
> patterns by use of the '-pattern' option. Eg:
> eximstats -pattern "DNS Block" /dnsbl/ mainlog

Oooooooh.....
I had just been thinking about how to get decent figures on how much
stuff I am rejecting (actually thats most of it considering the amount
of stuff I throw on failed sender callback).

Now I guess I just need to compose a whole bacth of patterns - and
probably arrange for them to be loaded from a file.

What are other people doing for summarising reject/non-delivery data?

Nigel.
--
[ Nigel Metheringham Nigel.Metheringham@InTechnology.co.uk ]
[. - Comments in this message are my own and not ITO opinion/policy - ]

Nigel Metheringham wrote:
> I had just been thinking about how to get decent figures on how much
> stuff I am rejecting (actually thats most of it considering the amount
> of stuff I throw on failed sender callback).
>
> Now I guess I just need to compose a whole bacth of patterns - and
> probably arrange for them to be loaded from a file.
>
> What are other people doing for summarising reject/non-delivery data?

I use eximstats with the -pattern option to count spam rejected messages,
virus rejected messages, bad file extension rejected messages, bad MIME
rejected messages, sender verification defers, and sender verification
failures. The first four are exiscan related, the last two are not actual
rejects, I just use them with a custom SA rule worth 3 points.

The downside to using the -pattern options is that it only provides a count,
nothing more. I'd love an option in eximstats that outputs sender
verification failures with the failed address and the SMTP server's
response. Also, custom patterns will not transfer over when combining
eximstats reports (which I for weekly and monthly reports).

Steven
---
Steven Dickenson <sdickenson@keyschool.org>
Network Administrator
The Key School, Annapolis Maryland

On Thu, 15 Apr 2004, Nigel Metheringham wrote:

> What are other people doing for summarising reject/non-delivery data?

in response to:

> On Thu, 2004-04-15 at 15:25, Steve Campbell wrote:
> > Newer versions of eximstats will count the occurrences of specified
> > patterns by use of the '-pattern' option. Eg:
> > eximstats -pattern "DNS Block" /dnsbl/ mainlog

I do exactly this. For all my (mostly RCPT-time) ACLs that deny, I use a
very specific log_message, for example:

log_message = MSGTAG_HELONONE: HELO/EHLO name not provided for \
$sender_host_address

I then have a wrapper which runs nightly around eximstats, which runs
eximstats with:

eximstats -nr \
...
-pattern " : none provided" \
"/MSGTAG_HELONONE/" \
-pattern " : contains underscore (rejected)" \
"/MSGTAG_HELOUNDERSCOREREJECT/" \
-pattern " : contains underscore (accepted)" \
"/MSGTAG_HELOUNDERSCOREACCEPT/" \
-pattern " Rejects/multiple recipients for bounce" \
"/MSGTAG_MULTIPLENULLRCPTS/" \

and eximstats produces the handy report:

User Specified Patterns
-----------------------
Total
...
Rejects/HELO checks: total 25327
: seems to be forged 1674
: is my own domain 12434
: is my own IP 3936
: not fully qualified 6368
: none provided 2
: contains underscore (rejected) 913
: contains underscore (accepted) 0
Rejects/multiple recipients for bounce 0
...

Simple but effective.

What would be nicer is to pull these numbers into rrd/MRTG and draw pretty
graphs for some trend analysis, but I never seem to get time to do that
bit. Anyone doing that?

Jethro.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Jethro R Binks
Computing Officer, IT Services
University Of Strathclyde, Glasgow, UK

| What would be nicer is to pull these numbers into rrd/MRTG and draw pretty
| graphs for some trend analysis, but I never seem to get time to do that
| bit. Anyone doing that?

I've seen some schemes that involve re-running the whole stats job every 5
mins, which seems awfully wasteful. Better to have a long-running process
which essentially does a "tail -f" on the log and increments counters as
appropriate.

I remember ages ago Nigel suggested the perl File::Tail module. This
*almost* worked for us but, at least on solaris, seemed to occasionally
get confused and rewind back to start of file for no apparent reason -
causing a huge spike in the graphs. Never found a solution.

We now just do the "tail -f" thing ourselves in perl - something like:

for (;;)
{
while ( <LOG> )
{
# while input present, process a log line
# etc
}

# write out the stats for mrtg

# wait a bit
sleep 60;

# Clear end-of-file condition ( with a seek no-op ):
seek(LOG, 0, 1);
}

This sort of scheme has other uses. An experimental script based on this
detects local (non-MTA) systems sending large amounts of mail thru our
smarthosts. It recently alerted us where one of our windoze boxes had
been turned into a spam-zombie, long before our smarthosts got into
spamcop et al...


--
Chris Edwards, Glasgow University Computing Service

On Thu, Apr 15, 2004 at 03:55:20PM +0100, Jethro R Binks wrote:
> What would be nicer is to pull these numbers into rrd/MRTG and draw pretty
> graphs for some trend analysis, but I never seem to get time to do that
> bit. Anyone doing that?

Actually, yes. I threw together 3 Ruby scripts one evening, which chew
through the mainlog, rejectlog and mail.log, and draws graphs over
in/out counts and bytes for domains, malware counts, average spam
scores and things. I should have had a few beers, at least I'd have an
excuse for the look of the code.

I've thought about doing it proper, and while I'm at it, add in the
ability to tail the logfiles, but there's always some other project.

--
Thomas
beast@system-tnt.dk