Mailing List Archive

[Bug 3224] sa-learn failure causes locking problem, spin, and db corruption
http://bugzilla.spamassassin.org/show_bug.cgi?id=3224





------- Additional Comments From ed.randall@ingenotech.com 2004-03-28 02:33 -------
sa-learn --dump still seems to work, but spamd is hanging
on my database. It would be nice if there was an sa-learn
option to load up a new database from the output of an sa-learn --dump.

Ed



------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 3224] sa-learn failure causes locking problem, spin, and db corruption [ In reply to ]
http://bugzilla.spamassassin.org/show_bug.cgi?id=3224





------- Additional Comments From ed.randall@ingenotech.com 2004-03-28 02:39 -------
The state my database has been left in, spamd is also hanging/spinning now:
# spamd -r /var/run/spamd -u mail -a -c --debug
debug: Score set 0 chosen.
debug: Preloading modules with HOME=/tmp/spamd-79621-init
debug: ignore: test message to precompile patterns and load modules
debug: using "/usr/local/share/spamassassin" for default rules dir
debug: using "/etc/mail/spamassassin" for site rules dir
debug: bayes: 79621 tie-ing to DB file R/O /var/spamassassin/bayes_toks
debug: bayes: 79621 tie-ing to DB file R/O /var/spamassassin/bayes_seen
debug: bayes: found bayes db version 2
debug: Score set 3 chosen.
debug: Initialising learner
debug: is Net::DNS::Resolver available? yes
debug: trying (3) microsoft.com...
debug: looking up MX for 'microsoft.com'
debug: MX for 'microsoft.com' exists? 1
debug: MX lookup of microsoft.com succeeded => Dns available (set dns_available
to hardcode)
debug: is DNS available? 1
debug: all '*From' addrs: ignore@compiling.spamassassin.taint.org
debug: running header regexp tests; score so far=0
debug: running body-text per-line regexp tests; score so far=2.077
debug: bayes corpus size: nspam = 9159, nham = 567
debug: uri tests: Done uriRE
debug: tokenize: header tokens for *F = "U*ignore
D*compiling.spamassassin.taint.org D*spamassassin.taint.org D*taint.org D*org"
debug: tokenize: header tokens for *m = " 1080470186 63036 spamassassin_spamd_init "
debug: bayes token 'message' => 0.985096774193548
debug: bayes token 'N:H*m:NNNNN' => 0.0290455092126242
debug: bayes token 'somewhat' => 0.122465632233962
debug: bayes: score = 0.445067685291714
debug: Syncing Bayes journal and expiring old tokens...
debug: lock: 79621 created /var/spamassassin/bayes.lock.srv79.server4me.com.79621
debug: lock: 79621 trying to get lock on /var/spamassassin/bayes with 0 retries
debug: lock: 79621 breaking stale /var/spamassassin/bayes.lock: age=1080469440
now=1080470189
debug: lock: 79621 trying to get lock on /var/spamassassin/bayes with 1 retries
debug: lock: 79621 link to /var/spamassassin/bayes.lock: link ok
debug: bayes: 79621 tie-ing to DB file R/W /var/spamassassin/bayes_toks
debug: bayes: 79621 tie-ing to DB file R/W /var/spamassassin/bayes_seen
debug: bayes: found bayes db version 2
debug: synced Bayes databases from journal in 0 seconds: 3 unique entries (3
total entries)
debug: bayes: 79621 untie-ing
debug: bayes: 79621 untie-ing db_toks
debug: bayes: 79621 untie-ing db_seen
debug: bayes: files locked, now unlocking lock
debug: unlock: 79621 unlink /var/spamassassin/bayes.lock
debug: lock: 79621 created /var/spamassassin/bayes.lock.srv79.server4me.com.79621
debug: lock: 79621 trying to get lock on /var/spamassassin/bayes with 0 retries
debug: lock: 79621 link to /var/spamassassin/bayes.lock: link ok
debug: bayes: 79621 tie-ing to DB file R/W /var/spamassassin/bayes_toks
debug: bayes: 79621 tie-ing to DB file R/W /var/spamassassin/bayes_seen
debug: bayes: found bayes db version 2
debug: bayes: expiry check keep size, 75% of max: 112500
debug: bayes: token count: 179347, final goal reduction size: 66847
debug: bayes: First pass? Current: 1080470190, Last: 1080024097, atime:
2764800, count: 18702, newdelta: 773516, ratio: 3.57432360175382
debug: bayes: Can't use estimation method for expiry, something fishy,
calculating optimal atime delta (first pass)

--- spins forever here ---




------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
[Bug 3224] sa-learn failure causes locking problem, spin, and db corruption [ In reply to ]
http://bugzilla.spamassassin.org/show_bug.cgi?id=3224

felicity@kluge.net changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution| |WORKSFORME



------- Additional Comments From felicity@kluge.net 2004-03-28 08:28 -------
the output you've shown is completely valid. bayes is looking at expiring
tokens, it finds it can't estimate expiry, so it has to sit there and go through
your DB calculating which tokens it should get rid of. if you have a large DB,
slow CPU, or a small amount of RAM (causes thrashing), it could take a while.

from your debug output, the DB isn't large, so... as for corruption, the
expiry first pass doesn't change the DB at all, so there's no corruption.
however, if you are deleting the lock file while the expiry run is going, that
is going to cause corruption -- don't do that unless you're sure the process
which set the lock is no longer running. (which should never happen)

you can disable autoexpiry by setting "bayes_auto_expire 0". then you can run
"sa-learn --force-expire" whenever you want.



------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.