Mailing List Archive

Author: felicity
Date: Thu Aug 31 09:45:55 2006
New Revision: 438980

URL: http://svn.apache.org/viewvc?rev=438980&view=rev
Log:
merge in changes from trunk to r438979

Added:
spamassassin/branches/tvd-multi-mass-check/rules/60_whitelist_dk.cf
- copied unchanged from r438979, spamassassin/trunk/rules/60_whitelist_dk.cf
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/
- copied from r438979, spamassassin/trunk/spamd-apache2/
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/MANIFEST
- copied unchanged from r438979, spamassassin/trunk/spamd-apache2/MANIFEST
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/MANIFEST.SKIP
- copied unchanged from r438979, spamassassin/trunk/spamd-apache2/MANIFEST.SKIP
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/META.yml
- copied unchanged from r438979, spamassassin/trunk/spamd-apache2/META.yml
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/Makefile.PL
- copied unchanged from r438979, spamassassin/trunk/spamd-apache2/Makefile.PL
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/README.apache
- copied unchanged from r438979, spamassassin/trunk/spamd-apache2/README.apache
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/bin/
- copied from r438979, spamassassin/trunk/spamd-apache2/bin/
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/bin/Bench-spamd.pl
- copied unchanged from r438979, spamassassin/trunk/spamd-apache2/bin/Bench-spamd.pl
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/bin/Spamd
- copied unchanged from r438979, spamassassin/trunk/spamd-apache2/bin/Spamd
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/bin/apache-spamd.pl
- copied unchanged from r438979, spamassassin/trunk/spamd-apache2/bin/apache-spamd.pl
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/lib/
- copied from r438979, spamassassin/trunk/spamd-apache2/lib/
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/lib/Mail/
- copied from r438979, spamassassin/trunk/spamd-apache2/lib/Mail/
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/lib/Mail/SpamAssassin/
- copied from r438979, spamassassin/trunk/spamd-apache2/lib/Mail/SpamAssassin/
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/lib/Mail/SpamAssassin/Spamd/
- copied from r438979, spamassassin/trunk/spamd-apache2/lib/Mail/SpamAssassin/Spamd/
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/lib/Mail/SpamAssassin/Spamd.pm
- copied unchanged from r438979, spamassassin/trunk/spamd-apache2/lib/Mail/SpamAssassin/Spamd.pm
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/lib/Mail/SpamAssassin/Spamd/Apache2/
- copied from r438979, spamassassin/trunk/spamd-apache2/lib/Mail/SpamAssassin/Spamd/Apache2/
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/lib/Mail/SpamAssassin/Spamd/Apache2.pm
- copied unchanged from r438979, spamassassin/trunk/spamd-apache2/lib/Mail/SpamAssassin/Spamd/Apache2.pm
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/lib/Mail/SpamAssassin/Spamd/Apache2/AclIP.pm
- copied unchanged from r438979, spamassassin/trunk/spamd-apache2/lib/Mail/SpamAssassin/Spamd/Apache2/AclIP.pm
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/lib/Mail/SpamAssassin/Spamd/Apache2/AclRFC1413.pm
- copied unchanged from r438979, spamassassin/trunk/spamd-apache2/lib/Mail/SpamAssassin/Spamd/Apache2/AclRFC1413.pm
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/lib/Mail/SpamAssassin/Spamd/Apache2/Config.pm
- copied unchanged from r438979, spamassassin/trunk/spamd-apache2/lib/Mail/SpamAssassin/Spamd/Apache2/Config.pm
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/lib/Mail/SpamAssassin/Spamd/Config.pm
- copied unchanged from r438979, spamassassin/trunk/spamd-apache2/lib/Mail/SpamAssassin/Spamd/Config.pm
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/t/
- copied from r438979, spamassassin/trunk/spamd-apache2/t/
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/t/30run.t
- copied unchanged from r438979, spamassassin/trunk/spamd-apache2/t/30run.t
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/t/TEST.PL
- copied unchanged from r438979, spamassassin/trunk/spamd-apache2/t/TEST.PL
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/t/certs/
- copied from r438979, spamassassin/trunk/spamd-apache2/t/certs/
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/t/certs/Makefile
- copied unchanged from r438979, spamassassin/trunk/spamd-apache2/t/certs/Makefile
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/t/certs/server.crt
- copied unchanged from r438979, spamassassin/trunk/spamd-apache2/t/certs/server.crt
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/t/certs/server.csr
- copied unchanged from r438979, spamassassin/trunk/spamd-apache2/t/certs/server.csr
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/t/certs/server.key
- copied unchanged from r438979, spamassassin/trunk/spamd-apache2/t/certs/server.key
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/t/conf/
- copied from r438979, spamassassin/trunk/spamd-apache2/t/conf/
spamassassin/branches/tvd-multi-mass-check/spamd-apache2/t/conf/extra.last.conf.in
- copied unchanged from r438979, spamassassin/trunk/spamd-apache2/t/conf/extra.last.conf.in
Modified:
spamassassin/branches/tvd-multi-mass-check/CREDITS
spamassassin/branches/tvd-multi-mass-check/INSTALL
spamassassin/branches/tvd-multi-mass-check/MANIFEST
spamassassin/branches/tvd-multi-mass-check/MANIFEST.SKIP
spamassassin/branches/tvd-multi-mass-check/USAGE
spamassassin/branches/tvd-multi-mass-check/build/automc/run_nightly
spamassassin/branches/tvd-multi-mass-check/build/mkrules
spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/Client.pm
spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/PerMsgStatus.pm
spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/Plugin.pm
spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/Plugin/DomainKeys.pm
spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm
spamassassin/branches/tvd-multi-mass-check/masses/logdiff
spamassassin/branches/tvd-multi-mass-check/masses/rule-qa/corpus-nightly
spamassassin/branches/tvd-multi-mass-check/rules/active.list
spamassassin/branches/tvd-multi-mass-check/sa-update.raw
spamassassin/branches/tvd-multi-mass-check/spamc/libspamc.c
spamassassin/branches/tvd-multi-mass-check/t/basic_lint.t
spamassassin/branches/tvd-multi-mass-check/t/gtube.t
spamassassin/branches/tvd-multi-mass-check/t/lang_pl_tests.t
spamassassin/branches/tvd-multi-mass-check/t/lint_nocreate_prefs.t
spamassassin/branches/tvd-multi-mass-check/t/mimeheader.t
spamassassin/branches/tvd-multi-mass-check/t/prefs_include.t
spamassassin/branches/tvd-multi-mass-check/t/reportheader.t
spamassassin/branches/tvd-multi-mass-check/t/reportheader_8bit.t

Modified: spamassassin/branches/tvd-multi-mass-check/CREDITS
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/CREDITS?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/CREDITS (original)
+++ spamassassin/branches/tvd-multi-mass-check/CREDITS Thu Aug 31 09:45:55 2006
@@ -125,6 +125,10 @@

- Liam Widdowson, <liam(at)inodes.org>: shared-library use of spamc.

+ - Radoslaw Zielinski, <radek(at)pld-linux.org>:
+ Mail::SpamAssassin::Spamd::Apache2, a mod_perl2 module implementing spamd,
+ contributed as a Google Summer of Code project.
+

Translators:


Modified: spamassassin/branches/tvd-multi-mass-check/INSTALL
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/INSTALL?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/INSTALL (original)
+++ spamassassin/branches/tvd-multi-mass-check/INSTALL Thu Aug 31 09:45:55 2006
@@ -354,6 +354,22 @@
charsets and convert them into Unicode, you will need to install
this module.

+
+ - Apache::Test (from CPAN)
+
+ If you plan to run the Apache2 version of spamd in the
+ "spamd-apache2" directory, you will need to install this
+ module.
+
+
+ - Apache 2 and mod_perl
+
+ If you plan to run the Apache2 version of spamd in the "spamd-apache2"
+ directory, you will need to ensure these are installed.
+
+ Ubuntu: sudo apt-get install apache2 libapache2-mod-perl2
+
+
What Next?
----------


Modified: spamassassin/branches/tvd-multi-mass-check/MANIFEST
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/MANIFEST?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/MANIFEST (original)
+++ spamassassin/branches/tvd-multi-mass-check/MANIFEST Thu Aug 31 09:45:55 2006
@@ -167,6 +167,7 @@
masses/rule-qa/corpus-nightly
masses/rule-qa/corpus-tagtime
masses/rule-qa/corpus.example
+masses/rule-qa/get-rulemetadata-for-revision
masses/rule-qa/markup-rules-file-with-freqs
masses/rule-qa/post-bugs-for-retired-tests
masses/rule-qa/rule-hits-over-time
@@ -222,6 +223,27 @@
spamd/solaris-rc-script.sh
spamd/spamd.raw
spamd/suse-ancient-rc-script.sh
+spamd-apache2/bin/apache-spamd.pl
+spamd-apache2/bin/Bench-spamd.pl
+spamd-apache2/bin/Spamd
+spamd-apache2/lib/Mail/SpamAssassin/Spamd.pm
+spamd-apache2/lib/Mail/SpamAssassin/Spamd/Apache2.pm
+spamd-apache2/lib/Mail/SpamAssassin/Spamd/Apache2/AclIP.pm
+spamd-apache2/lib/Mail/SpamAssassin/Spamd/Apache2/AclRFC1413.pm
+spamd-apache2/lib/Mail/SpamAssassin/Spamd/Apache2/Config.pm
+spamd-apache2/lib/Mail/SpamAssassin/Spamd/Config.pm
+spamd-apache2/Makefile.PL
+spamd-apache2/MANIFEST
+spamd-apache2/MANIFEST.SKIP
+spamd-apache2/META.yml
+spamd-apache2/README.apache
+spamd-apache2/t/30run.t
+spamd-apache2/t/certs/Makefile
+spamd-apache2/t/certs/server.crt
+spamd-apache2/t/certs/server.csr
+spamd-apache2/t/certs/server.key
+spamd-apache2/t/conf/extra.last.conf.in
+spamd-apache2/t/TEST.PL
sql/README
sql/README.awl
sql/README.bayes
@@ -459,6 +481,7 @@
rules/60_awl.cf
rules/60_shortcircuit.cf
rules/60_whitelist.cf
+rules/60_whitelist_dk.cf
rules/60_whitelist_dkim.cf
rules/60_whitelist_spf.cf
rules/60_whitelist_subject.cf

Modified: spamassassin/branches/tvd-multi-mass-check/MANIFEST.SKIP
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/MANIFEST.SKIP?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/MANIFEST.SKIP (original)
+++ spamassassin/branches/tvd-multi-mass-check/MANIFEST.SKIP Thu Aug 31 09:45:55 2006
@@ -118,3 +118,12 @@
^build/buildbot/
^hack/
^patches/
+(^|/)\.
+~$
+/logs/
+/t/logs/
+\.conf$
+\.pl$
+\bCVS\b
+/t/TEST$
+

Modified: spamassassin/branches/tvd-multi-mass-check/USAGE
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/USAGE?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/USAGE (original)
+++ spamassassin/branches/tvd-multi-mass-check/USAGE Thu Aug 31 09:45:55 2006
@@ -85,7 +85,8 @@

- take a look at the notes on the Wiki website, currently at
<http://wiki.apache.org/spamassassin/UsingSiteWide>. You will probably
- want to use 'spamd' (see below).
+ want to use 'spamd' (see below). You may want to investigate the
+ new Apache mod_perl module, in the 'spamd-apache2' directory, too.

- *PLEASE* let your users know you've installed it, and how to turn it
off! This is our #1 tech support query, and the users are usually
@@ -181,6 +182,15 @@
large volumes of mail through SpamAssassin without having to
fork/exec a perl interpreter for each message. Take a look in the
'spamd' and 'spamc' directories for more details.
+
+
+ - The distribution also includes 'spamd-apache2', a mod_perl module
+ allowing the Apache HTTP server to be used as a platform for a
+ daemonized SpamAssassin, in an upwardly-compatible fashion from
+ 'spamd'. If you don't require some of the spamd features it does not
+ implement (such as switching UIDs to read per-user configuration from
+ user home directories), this may be much faster than spamd. Take a
+ look at the 'spamd-apache2' directory for details.


- spamc can now be built as a shared library for use with milters or

Modified: spamassassin/branches/tvd-multi-mass-check/build/automc/run_nightly
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/build/automc/run_nightly?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/build/automc/run_nightly (original)
+++ spamassassin/branches/tvd-multi-mass-check/build/automc/run_nightly Thu Aug 31 09:45:55 2006
@@ -16,9 +16,11 @@
nightly_trees="/home/bbmass/mc-nightly"


-# ensure we're running after 0900 UTC; complex due to daylight savings.
-# If we're running in the 08xx UTC hour-space, sleep for 1 hour; when
-# we wake, we will be in 09xx.
+# Ensure we're running after 0900 UTC; complex due to daylight savings. If
+# we're running in the 08xx UTC hour-space, sleep for 1 hour; when we wake, we
+# will be in 09xx. By doing it this way, we'll work both for (a) mass-checks
+# set to run from cron at 09xx (cron doesn't understand DST/UTC issues) and (b)
+# mass-checks starting at other times that != 09xx.
(
TIME="%e,%U,%S"; TZ=UTC; export TIME TZ
if date | egrep '^... ... .. 08:'; then

Modified: spamassassin/branches/tvd-multi-mass-check/build/mkrules
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/build/mkrules?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/build/mkrules (original)
+++ spamassassin/branches/tvd-multi-mass-check/build/mkrules Thu Aug 31 09:45:55 2006
@@ -96,6 +96,8 @@
# source files that need compilation, and their targets
my $needs_compile = { };
my $current_src;
+my $newest_src_mtime = 0;
+my $newest_out_mtime = 0;

my $default_file_header = join('', <DATA>);

@@ -111,6 +113,20 @@
}, $src);
}

+# get mtimes of output files; we can be sure that all
+# output is under the "opt_out" dir, so recurse there
+File::Find::find ({
+ wanted => \&out_wanted,
+ no_chdir => 1
+ }, $opt_out);
+
+if ($newest_src_mtime && $newest_out_mtime
+ && $newest_src_mtime < $newest_out_mtime)
+{
+ print "mkrules: no rules updated\n";
+ exit 0;
+}
+
my $rules = { };

my $file_manifest = { };
@@ -214,12 +230,28 @@
my $t;
$t = "$opt_out/$filename";

+ my @st = stat $path;
+ if ($st[9] && $st[9] > $newest_src_mtime) {
+ $newest_src_mtime = $st[9];
+ }
+
$needs_compile->{$f} = {
f => $f,
t => $t,
dir => $dir,
filename => $filename
};
+}
+
+sub out_wanted {
+ my $path = $File::Find::name;
+ return unless (-f $path);
+ return unless ($path =~ /\.cf$/i);
+
+ my @st = stat $path;
+ if ($st[9] && $st[9] > $newest_out_mtime) {
+ $newest_out_mtime = $st[9];
+ }
}

# compile all the source files found by the wanted() sub, in sorted

Modified: spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/Client.pm
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/Client.pm?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/Client.pm (original)
+++ spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/Client.pm Thu Aug 31 09:45:55 2006
@@ -143,11 +143,7 @@

return undef unless ($resp_code == 0);

- my $found_blank_line_p = 0;
-
- while (!$found_blank_line_p) {
- $line = <$remote>;
-
+ while ($line = <$remote>) {
if ($line =~ /Content-length: (\d+)/) {
$data{content_length} = $1;
}
@@ -156,8 +152,8 @@
$data{score} = $2 + 0;
$data{threshold} = $3 + 0;
}
- elsif ($line =~ /$EOL/) {
- $found_blank_line_p = 1;
+ elsif ($line =~ /^${EOL}$/) {
+ last;
}
}

@@ -254,22 +250,18 @@

return undef unless ($resp_code == 0);

- my $found_blank_line_p = 0;
-
my $did_set;
my $did_remove;

- while (!$found_blank_line_p) {
- $line = <$remote>;
-
+ while ($line = <$remote>) {
if ($line =~ /DidSet: (.*)/i) {
$did_set = $1;
}
elsif ($line =~ /DidRemove: (.*)/i) {
$did_remove = $1;
}
- elsif ($line =~ /$EOL/) {
- $found_blank_line_p = 1;
+ elsif ($line =~ /^${EOL}$/) {
+ last;
}
}

@@ -322,16 +314,14 @@
return undef unless ($resp_code == 0);

my $reported_p = 0;
- my $found_blank_line_p = 0;
-
- while (!$reported_p && !$found_blank_line_p) {
- $line = <$remote>;

+ while (($line = <$remote>)) {
if ($line =~ /DidSet:\s+.*remote/i) {
$reported_p = 1;
+ last;
}
- elsif ($line =~ /^$EOL$/) {
- $found_blank_line_p = 1;
+ elsif ($line =~ /^${EOL}$/) {
+ last;
}
}

@@ -380,16 +370,14 @@
return undef unless ($resp_code == 0);

my $revoked_p = 0;
- my $found_blank_line_p = 0;
-
- while (!$revoked_p && !$found_blank_line_p) {
- $line = <$remote>;

+ while (!$revoked_p && ($line = <$remote>)) {
if ($line =~ /DidRemove:\s+remote/i) {
$revoked_p = 1;
+ last;
}
- elsif ($line =~ /^$EOL$/) {
- $found_blank_line_p = 1;
+ elsif ($line =~ /^${EOL}$/) {
+ last;
}
}


Modified: spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/PerMsgStatus.pm
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/PerMsgStatus.pm?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/PerMsgStatus.pm (original)
+++ spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/PerMsgStatus.pm Thu Aug 31 09:45:55 2006
@@ -83,13 +83,14 @@
'subtest_names_hit' => [ ],
'spamd_result_log_items' => [ ],
'tests_already_hit' => { },
- 'hdr_cache' => { },
+ 'c' => { },
'rule_errors' => 0,
'disable_auto_learning' => 0,
'auto_learn_status' => undef,
'conf' => $main->{conf},
'async' => Mail::SpamAssassin::AsyncLoop->new($main)
};
+ #$self->{main}->{use_rule_subs} = 1;

if (defined $opts && $opts->{disable_auto_learning}) {
$self->{disable_auto_learning} = 1;
@@ -246,7 +247,7 @@

# now that we've finished checking the mail, clear out this cache
# to avoid unforeseen side-effects.
- $self->{hdr_cache} = { };
+ $self->{c} = { };

# Round the score to 3 decimal places to avoid rounding issues
# We assume required_score to be properly rounded already.
@@ -1801,11 +1802,14 @@
return;
}

+ my $use_rule_subs = $self->{main}->{use_rule_subs};
+
my $evalstr = $self->start_rules_plugin_code("header");
my $evalstr2 = '';

# hash to hold the rules, "header\tdefault value" => rulename
my %ordered = ();
+ my %testcode = ();

while (my($rulename, $rule) = each %{$self->{conf}{head_tests}->{$priority}}) {
my $def = '';
@@ -1829,16 +1833,22 @@
next if (!$self->is_user_rule_sub ($rulename.'_head_test'));
}

- $evalstr2 .= '
- sub '.$rulename.'_head_test {
- my($self,$text) = @_;
- '.$self->hash_line_for_rule($rulename).'
- while ($text '.$testtype.'~ '.$pat.'g) {
- $self->got_hit(q#'.$rulename.'#, "", ruletype => "header");
- '. $self->hit_rule_plugin_code($rulename, "header", "last") . '
- }
- }';
-
+ if ($use_rule_subs) {
+ $evalstr2 .= '
+ sub '.$rulename.'_head_test {
+ my($self,$text) = @_;
+ '.$self->hash_line_for_rule($rulename).'
+ while ($text '.$testtype.'~ '.$pat.'g) {
+ $self->got_hit(q#'.$rulename.'#, "", ruletype => "header");
+ '. $self->hit_rule_plugin_code($rulename, "header", "last") . '
+ }
+ }
+ ';
+ }
+ else {
+ # store for use below
+ $testcode{$rulename} = $testtype.'~ '.$pat.'g';
+ }
}

# setup the function to run the rules
@@ -1846,12 +1856,29 @@
my($hdrname, $def) = split(/\t/, $k, 2);
$evalstr .= ' $hval = $self->get(q#'.$hdrname.'#, q#'.$def.'#);';
foreach my $rulename (@{$v}) {
- $evalstr .= '
- if ($scoresptr->{q#'.$rulename.'#}) {
- '.$rulename.'_head_test($self, $hval); # no need for OO calling here (its faster this way)
- '.$self->ran_rule_plugin_code($rulename, "header").'
+ if ($use_rule_subs) {
+ $evalstr .= '
+ if ($scoresptr->{q#'.$rulename.'#}) {
+ '.$rulename.'_head_test($self, $hval);
+ '.$self->ran_rule_plugin_code($rulename, "header").'
+ }
+ ';
+ }
+ else {
+ my $testcode = $testcode{$rulename};
+
+ $evalstr .= '
+ if ($scoresptr->{q#'.$rulename.'#}) {
+ pos $hval = 0;
+ '.$self->hash_line_for_rule($rulename).'
+ while ($hval '.$testcode.') {
+ $self->got_hit(q#'.$rulename.'#, "", ruletype => "header");
+ '.$self->hit_rule_plugin_code($rulename, "header", "last").'
+ }
+ '.$self->ran_rule_plugin_code($rulename, "header").'
+ }
+ ';
}
- ';
}
}

@@ -1916,35 +1943,75 @@
return;
}

+ # caller can set this member of the Mail::SpamAssassin object to
+ # override this; useful for profiling rule runtimes, although I think
+ # the HitFreqsRuleTiming.pm plugin is probably better nowadays anyway
+ my $use_rule_subs = $self->{main}->{use_rule_subs};
+
# build up the eval string...
my $evalstr = $self->start_rules_plugin_code("body");
my $evalstr2 = '';
+ my $loopid = 0;

- while (my($rulename, $pat) = each %{$self->{conf}{body_tests}->{$priority}}) {
- $evalstr .= '
- if ($scoresptr->{q{'.$rulename.'}}) {
- '.$rulename.'_body_test($self,@_);
- '.$self->ran_rule_plugin_code($rulename, "body").'
+ while (my($rulename, $pat) = each %{$self->{conf}{body_tests}->{$priority}})
+ {
+ my $sub;
+ if ($self->{conf}->{tflags}->{$rulename} =~ /\bmultiple\b/)
+ {
+ # support multiple matches
+ $loopid++;
+ $sub = '
+ body_'.$loopid.': foreach my $l (@_) {
+ pos $l = 0;
+ '.$self->hash_line_for_rule($rulename).'
+ while ($l =~ '.$pat.'g) {
+ $self->got_hit(q{'.$rulename.'}, "BODY: ", ruletype => "body");
+ '. $self->hit_rule_plugin_code($rulename, "body",
+ "last body_".$loopid) . '
+ }
}
- ';
+ ';
+ }
+ else {
+ # omitting the "pos" call, "body_loopid" label, use of while()
+ # instead of if() etc., shaves off 8 perl OPs.
+ $sub = '
+ foreach my $l (@_) {
+ '.$self->hash_line_for_rule($rulename).'
+ if ($l =~ '.$pat.') {
+ $self->got_hit(q{'.$rulename.'}, "BODY: ", ruletype => "body");
+ '. $self->hit_rule_plugin_code($rulename, "body", "last") .'
+ }
+ }
+ ';
+ }
+
+ if ($use_rule_subs) {
+ $evalstr .= '
+ if ($scoresptr->{q{'.$rulename.'}}) {
+ '.$rulename.'_body_test($self,@_);
+ '.$self->ran_rule_plugin_code($rulename, "body").'
+ }
+ ';
+ }
+ else {
+ $evalstr .= '
+ if ($scoresptr->{q{'.$rulename.'}}) {
+ '.$sub.'
+ '.$self->ran_rule_plugin_code($rulename, "body").'
+ }
+ ';
+ }

if ($doing_user_rules) {
next if (!$self->is_user_rule_sub ($rulename.'_body_test'));
}

- $evalstr2 .= '
- sub '.$rulename.'_body_test {
- my $self = shift;
- foreach (@_) {
- pos = 0;
- '.$self->hash_line_for_rule($rulename).'
- while ('.$pat.'g) {
- $self->got_hit(q{'.$rulename.'}, "BODY: ", ruletype => "body");
- '. $self->hit_rule_plugin_code($rulename, "body", "return") . '
- }
- }
+ if ($use_rule_subs) {
+ $evalstr2 .= '
+ sub '.$rulename.'_body_test { my $self = shift; '.$sub.' }
+ ';
}
- ';
}

# clear out a previous version of this fn, if already defined
@@ -2329,35 +2396,67 @@
return;
}

+ my $use_rule_subs = $self->{main}->{use_rule_subs};
+
# otherwise build up the eval string...
my $evalstr = $self->start_rules_plugin_code("uri");
my $evalstr2 = '';
+ my $loopid = 0;

while (my($rulename, $pat) = each %{$self->{conf}{uri_tests}->{$priority}}) {
- $evalstr .= '
- if ($scoresptr->{q{'.$rulename.'}}) {
- '.$rulename.'_uri_test($self, @_);
- '.$self->ran_rule_plugin_code($rulename, "uri").'
+ my $sub;
+ if ($self->{conf}->{tflags}->{$rulename} =~ /\bmultiple\b/)
+ {
+ $loopid++;
+ $sub = '
+ uri_'.$loopid.': foreach my $l (@_) {
+ pos $l = 0;
+ '.$self->hash_line_for_rule($rulename).'
+ while ($l =~ '.$pat.'g) {
+ $self->got_hit(q{'.$rulename.'}, "URI: ", ruletype => "uri");
+ '. $self->hit_rule_plugin_code($rulename, "uri",
+ "last uri_".$loopid) . '
+ }
}
- ';
+ ';
+ } else {
+ $sub = '
+ foreach my $l (@_) {
+ '.$self->hash_line_for_rule($rulename).'
+ if ($l =~ '.$pat.') {
+ $self->got_hit(q{'.$rulename.'}, "URI: ", ruletype => "uri");
+ '. $self->hit_rule_plugin_code($rulename, "uri", "last") .'
+ }
+ }
+ ';
+ }
+
+ if ($use_rule_subs) {
+ $evalstr .= '
+ if ($scoresptr->{q{'.$rulename.'}}) {
+ '.$rulename.'_uri_test($self, @_);
+ '.$self->ran_rule_plugin_code($rulename, "uri").'
+ }
+ ';
+ }
+ else {
+ $evalstr .= '
+ if ($scoresptr->{q{'.$rulename.'}}) {
+ '.$sub.'
+ '.$self->ran_rule_plugin_code($rulename, "uri").'
+ }
+ ';
+ }

if ($doing_user_rules) {
next if (!$self->is_user_rule_sub ($rulename.'_uri_test'));
}

- $evalstr2 .= '
- sub '.$rulename.'_uri_test {
- my $self = shift;
- foreach (@_) {
- pos = 0;
- '.$self->hash_line_for_rule($rulename).'
- while ('.$pat.'g) {
- $self->got_hit(q{'.$rulename.'}, "URI: ", ruletype => "uri");
- '. $self->hit_rule_plugin_code($rulename, "uri", "return") .'
- }
- }
+ if ($use_rule_subs) {
+ $evalstr2 .= '
+ sub '.$rulename.'_uri_test { my $self = shift; '.$sub.' }
+ ';
}
- ';
}

# clear out a previous version of this fn, if already defined
@@ -2420,35 +2519,69 @@
return;
}

+ my $use_rule_subs = $self->{main}->{use_rule_subs};
+
# build up the eval string...
my $evalstr = $self->start_rules_plugin_code("rawbody");
my $evalstr2 = '';
+ my $loopid = 0;

while (my($rulename, $pat) = each %{$self->{conf}{rawbody_tests}->{$priority}}) {
- $evalstr .= '
- if ($scoresptr->{q{'.$rulename.'}}) {
- '.$rulename.'_rawbody_test($self, @_);
- '.$self->ran_rule_plugin_code($rulename, "rawbody").'
+ my $sub;
+ if ($self->{conf}->{tflags}->{$rulename} =~ /\bmultiple\b/)
+ {
+ # support multiple matches
+ $loopid++;
+ $sub = '
+ rawbody_'.$loopid.': foreach my $l (@_) {
+ pos $l = 0;
+ '.$self->hash_line_for_rule($rulename).'
+ while ($l =~ '.$pat.'g) {
+ $self->got_hit(q{'.$rulename.'}, "RAW: ", ruletype => "rawbody");
+ '. $self->hit_rule_plugin_code($rulename, "rawbody",
+ "last rawbody_".$loopid) . '
+ }
}
- ';
+ ';
+ }
+ else {
+ $sub = '
+ foreach my $l (@_) {
+ '.$self->hash_line_for_rule($rulename).'
+ if ($l =~ '.$pat.') {
+ $self->got_hit(q{'.$rulename.'}, "RAW: ", ruletype => "rawbody");
+ '. $self->hit_rule_plugin_code($rulename, "rawbody", "last") . '
+ }
+ }
+ ';
+ }
+
+ if ($use_rule_subs) {
+ $evalstr .= '
+ if ($scoresptr->{q{'.$rulename.'}}) {
+ '.$rulename.'_rawbody_test($self, @_);
+ '.$self->ran_rule_plugin_code($rulename, "rawbody").'
+ }
+ ';
+ }
+ else {
+ $evalstr .= '
+ if ($scoresptr->{q{'.$rulename.'}}) {
+ '.$sub.'
+ '.$self->ran_rule_plugin_code($rulename, "rawbody").'
+ }
+ ';
+ }

if ($doing_user_rules) {
next if (!$self->is_user_rule_sub ($rulename.'_rawbody_test'));
}

- $evalstr2 .= '
- sub '.$rulename.'_rawbody_test {
- my $self = shift;
- foreach (@_) {
- pos = 0;
- '.$self->hash_line_for_rule($rulename).'
- while ('.$pat.'g) {
- $self->got_hit(q{'.$rulename.'}, "RAW: ", ruletype => "rawbody");
- '. $self->hit_rule_plugin_code($rulename, "rawbody", "return") . '
- }
- }
+ if ($use_rule_subs) {
+ $evalstr2 .= '
+ sub '.$rulename.'_rawbody_test { my $self = shift; '.$sub.' }
+ ';
}
- ';
}

# clear out a previous version of this fn, if already defined

Modified: spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/Plugin.pm
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/Plugin.pm?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/Plugin.pm (original)
+++ spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/Plugin.pm Thu Aug 31 09:45:55 2006
@@ -832,13 +832,13 @@

=over 4

-=item The plugin object itself
+=item - The plugin object itself

-=item The C<Mail::SpamAssassin::PerMsgStatus> object calling the rule
+=item - The C<Mail::SpamAssassin::PerMsgStatus> object calling the rule

-=item standard arguments for the rule type in use
+=item - standard arguments for the rule type in use

-=item any and all arguments as specified in the configuration file
+=item - any and all arguments as specified in the configuration file

=back

@@ -869,13 +869,13 @@

=over 4

-=item header tests, no extra arguments
+=item - header tests: no extra arguments

-=item body tests, fully rendered message as array reference
+=item - body tests: fully rendered message as array reference

-=item rawbody tests, fully decoded message as array reference
+=item - rawbody tests: fully decoded message as array reference

-=item full tests, pristine message as scalar reference
+=item - full tests: pristine message as scalar reference

=back


Modified: spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/Plugin/DomainKeys.pm
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/Plugin/DomainKeys.pm?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/Plugin/DomainKeys.pm (original)
+++ spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/Plugin/DomainKeys.pm Thu Aug 31 09:45:55 2006
@@ -61,6 +61,8 @@
$self->register_eval_rule ("check_domainkeys_signsome");
$self->register_eval_rule ("check_domainkeys_testing");
$self->register_eval_rule ("check_domainkeys_signall");
+ $self->register_eval_rule ("check_for_dk_whitelist_from");
+ $self->register_eval_rule ("check_for_def_dk_whitelist_from");

$self->set_config($mailsaobject->{conf});

@@ -90,6 +92,86 @@
type => $Mail::SpamAssassin::Conf::CONF_TYPE_NUMERIC
});

+=item whitelist_from_dk add@ress.com [signing domain name]
+
+Use this to supplement the whitelist_from addresses with a check to make sure
+the message has been signed by a DomainKeys signature that can be verified
+against the From: domain's DomainKeys public key.
+
+In order to support signing domain names that differ from the address domain
+name, only one whitelist entry is allowed per line, exactly like
+C<whitelist_from_rcvd>. Multiple C<whitelist_from_dk> lines are allowed.
+File-glob style meta characters are allowed for the From: address, just like
+with C<whitelist_from_rcvd>. The optional signing domain name parameter must
+match from the right-most side, also like in C<whitelist_from_rcvd>.
+
+If no signing domain name parameter is specified the domain of the address
+parameter specified will be used instead.
+
+The From: address is obtained from a signed part of the message (ie. the
+"From:" header), not from envelope data that is possible to forge.
+
+Since this whitelist requires a DomainKeys check to be made, network tests must
+be enabled.
+
+Examples:
+
+ whitelist_from_dk joe@example.com
+ whitelist_from_dk *@corp.example.com
+
+ whitelist_from_dk bob@it.example.net example.net
+ whitelist_from_dk *@eng.example.net example.net
+
+=item def_whitelist_from_dk add@ress.com [signing domain name]
+
+Same as C<whitelist_from_dk>, but used for the default whitelist entries
+in the SpamAssassin distribution. The whitelist score is lower, because
+these are often targets for spammer spoofing.
+
+=cut
+
+ push (@cmds, {
+ setting => 'whitelist_from_dk',
+ code => sub {
+ my ($self, $key, $value, $line) = @_;
+ unless (defined $value && $value !~ /^$/) {
+ return $Mail::SpamAssassin::Conf::MISSING_REQUIRED_VALUE;
+ }
+ unless ($value =~ /^(\S+)(?:\s+(\S+))?$/) {
+ return $Mail::SpamAssassin::Conf::INVALID_VALUE;
+ }
+ my $address = $1;
+ my $signer = (defined $2 ? $2 : $1);
+
+ unless (defined $2) {
+ $signer =~ s/^.*@(.*)$/$1/;
+ }
+ $self->{parser}->add_to_addrlist_rcvd ('whitelist_from_dk',
+ $address, $signer);
+ }
+ });
+
+ push (@cmds, {
+ setting => 'def_whitelist_from_dk',,
+ code => sub {
+ my ($self, $key, $value, $line) = @_;
+ unless (defined $value && $value !~ /^$/) {
+ return $Mail::SpamAssassin::Conf::MISSING_REQUIRED_VALUE;
+ }
+ unless ($value =~ /^(\S+)(?:\s+(\S+))?$/) {
+ return $Mail::SpamAssassin::Conf::INVALID_VALUE;
+ }
+ my $address = $1;
+ my $signer = (defined $2 ? $2 : $1);
+
+ unless (defined $2) {
+ $signer =~ s/^.*@(.*)$/$1/;
+ }
+ $self->{parser}->add_to_addrlist_rcvd ('def_whitelist_from_dk',
+ $address, $signer);
+ }
+ });
+
$conf->{parser}->register_commands(\@cmds);
}

@@ -133,7 +215,19 @@
return $scan->{domainkeys_signall};
}

+sub check_for_dk_whitelist_from {
+ my ($self, $scan) = @_;
+ $self->_check_dk_whitelist($scan, 0) unless $scan->{dk_whitelist_from_checked};
+ $scan->{dk_whitelist_from};
+}

+sub check_for_def_dk_whitelist_from {
+ my ($self, $scan) = @_;
+ $self->_check_dk_whitelist($scan, 1) unless $scan->{def_dk_whitelist_from_checked};
+ $scan->{def_dk_whitelist_from};
+}
+
+# ---------------------------------------------------------------------------

sub _check_domainkeys {
my ($self, $scan) = @_;
@@ -167,6 +261,22 @@
return;
}

+ # get the sender address for whitelist checks
+ if (defined $message->sender()) {
+ $scan->{dk_address} = @{$message->sender()}[1];
+ dbg("dk: sender: $scan->{dk_address}");
+ } elsif (defined $message->from()) {
+ $scan->{dk_address} ||= @{$message->from()}[1];
+ dbg("dk: from: $scan->{dk_address}");
+ } else {
+ dbg("dk: could not determine sender: or from: identity");
+ }
+
+ # get the signing domain name for whitelist checks
+ $scan->{dk_signing_domain} = $self->_dkmsg_signing_domain($scan, $message);
+ dbg("dk: signing domain name: ".
+ ($scan->{dk_signing_domain} ? $scan->{dk_signing_domain} : "not found"));
+
my $timeout = $scan->{conf}->{domainkeys_timeout};

my $timer = Mail::SpamAssassin::Timeout->new({ secs => $timeout });
@@ -245,6 +355,24 @@
}
}

+# get the DK signing domain name from the Mail::DomainKeys::Message object
+sub _dkmsg_signing_domain {
+ my ($self, $scan, $message) = @_;
+ # try to use the signature() API if it exists (post-0.80)
+ if ($message->can("signature")) {
+ if (!$message->signed) {
+ return undef;
+ }
+ return $message->signature->domain;
+ } else {
+ # otherwise parse it ourself
+ if ($scan->{msg}->get_header("DomainKey-Signature") =~ /d=(\S+);/) {
+ return $1;
+ }
+ return undef;
+ }
+}
+
sub sanitize_header_for_dk {
my ($self, $ref) = @_;

@@ -310,6 +438,102 @@
# $$ref =~ s/^\n//gs; $$ref =~ s/\n$//gs;
$$ref =~ s/!nl;/\n/gs;
$$ref =~ s/!ex;/!/gs;
+}
+
+sub _check_dk_whitelist {
+ my ($self, $scan, $default) = @_;
+
+ return unless $scan->is_dns_available();
+
+ # trigger a DK check so we can get address/signer info
+ # if verification failed only continue if we want the debug info
+ unless ($self->check_domainkeys_verified($scan)) {
+ unless (would_log("dbg", "dk")) {
+ return;
+ }
+ }
+
+ unless ($scan->{dk_address}) {
+ dbg("dk: ". ($default ? "def_" : "") ."whitelist_from_dk: could not find sender or from address");
+ return;
+ }
+ unless ($scan->{dk_signing_domain}) {
+ dbg("dk: ". ($default ? "def_" : "") ."whitelist_from_dk: could not find signing domain name");
+ return;
+ }
+
+ if ($default) {
+ $scan->{def_dk_whitelist_from_checked} = 1;
+ $scan->{def_dk_whitelist_from} = 0;
+
+ # copied and butchered from the code for whitelist_from_rcvd in Evaltests.pm
+ ONE: foreach my $white_addr (keys %{$scan->{conf}->{def_whitelist_from_dk}}) {
+ my $regexp = qr/$scan->{conf}->{def_whitelist_from_dk}->{$white_addr}{re}/i;
+ foreach my $domain (@{$scan->{conf}->{def_whitelist_from_dk}->{$white_addr}{domain}}) {
+ if ($scan->{dk_address} =~ $regexp) {
+ if ($scan->{dk_signing_domain} =~ /(?:^|\.)\Q${domain}\E$/i) {
+ dbg("dk: address: $scan->{dk_address} matches def_whitelist_from_dk ".
+ "$scan->{conf}->{def_whitelist_from_dk}->{$white_addr}{re} ${domain}");
+ $scan->{def_dk_whitelist_from} = 1;
+ last ONE;
+ }
+ }
+ }
+ }
+ } else {
+ $scan->{dk_whitelist_from_checked} = 1;
+ $scan->{dk_whitelist_from} = 0;
+
+ # copied and butchered from the code for whitelist_from_rcvd in Evaltests.pm
+ ONE: foreach my $white_addr (keys %{$scan->{conf}->{whitelist_from_dk}}) {
+ my $regexp = qr/$scan->{conf}->{whitelist_from_dk}->{$white_addr}{re}/i;
+ foreach my $domain (@{$scan->{conf}->{whitelist_from_dk}->{$white_addr}{domain}}) {
+ if ($scan->{dk_address} =~ $regexp) {
+ if ($scan->{dk_signing_domain} =~ /(?:^|\.)\Q${domain}\E$/i) {
+ dbg("dk: address: $scan->{dk_address} matches whitelist_from_dk ".
+ "$scan->{conf}->{whitelist_from_dk}->{$white_addr}{re} ${domain}");
+ $scan->{dk_whitelist_from} = 1;
+ last ONE;
+ }
+ }
+ }
+ }
+ }
+
+ # if the message doesn't pass DK validation, it can't pass a DK whitelist
+ if ($default) {
+ if ($scan->{def_dk_whitelist_from}) {
+ if ($self->check_domainkeys_verified($scan)) {
+ dbg("dk: address: $scan->{dk_address} signing domain name: ".
+ "$scan->{dk_signing_domain} is in user's DEF_WHITELIST_FROM_DK and ".
+ "passed DK verification");
+ } else {
+ dbg("dk: address: $scan->{dk_address} signing domain name: ".
+ "$scan->{dk_signing_domain} is in user's DEF_WHITELIST_FROM_DK but ".
+ "failed DK verification");
+ $scan->{def_dk_whitelist_from} = 0;
+ }
+ } else {
+ dbg("dk: address: $scan->{dk_address} signing domain name: ".
+ "$scan->{dk_signing_domain} is not in user's DEF_WHITELIST_FROM_DK");
+ }
+ } else {
+ if ($scan->{dk_whitelist_from}) {
+ if ($self->check_domainkeys_verified($scan)) {
+ dbg("dk: address: $scan->{dk_address} signing domain name: ".
+ "$scan->{dk_signing_domain} is in user's WHITELIST_FROM_DK and ".
+ "passed DK verification");
+ } else {
+ dbg("dk: address: $scan->{dk_address} signing domain name: ".
+ "$scan->{dk_signing_domain} is in user's WHITELIST_FROM_DK but ".
+ "failed DK verification");
+ $scan->{dk_whitelist_from} = 0;
+ }
+ } else {
+ dbg("dk: address: $scan->{dk_address} signing domain name: ".
+ "$scan->{dk_signing_domain} is not in user's WHITELIST_FROM_DK");
+ }
+ }
}

1;

Modified: spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm (original)
+++ spamassassin/branches/tvd-multi-mass-check/lib/Mail/SpamAssassin/Plugin/URIDNSBL.pm Thu Aug 31 09:45:55 2006
@@ -76,10 +76,9 @@
in, and C<lookuptype> is the type of lookup (B<TXT> or B<A>).

C<subtest> is the sub-test to run against the returned data. The sub-test may
-either be an IPv4 dotted address for RHSBLs that return multiple A records, a
+either be an IPv4 dotted address for RHSBLs that return multiple A records or a
non-negative decimal number to specify a bitmask for RHSBLs that return a
-single A record containing a bitmask of results, or (if none of the preceding
-options seem to fit) a regular expression.
+single A record containing a bitmask of results.

Note that, as with C<urirhsbl>, you must also define a body-eval rule calling
C<check_uridnsbl()> to use this.
@@ -345,7 +344,7 @@
setting => 'urirhssub',
code => sub {
my ($self, $key, $value, $line) = @_;
- if ($value =~ /^(\S+)\s+(\S+)\s+(\S+)\s+(\S+)$/) {
+ if ($value =~ /^(\S+)\s+(\S+)\s+(\S+)\s+(\d{1,10}|\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})$/) {
my $rulename = $1;
my $zone = $2;
my $type = $3;
@@ -596,15 +595,9 @@
}
# bitmask
elsif ($subtest =~ /^\d+$/) {
- if ($rdatastr =~ m/^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$/ &&
- Mail::SpamAssassin::Util::my_inet_aton($rdatastr) & $subtest)
+ if ($rdatastr =~ m/^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$/ &&
+ Mail::SpamAssassin::Util::my_inet_aton($rdatastr) & $subtest)
{
- $self->got_dnsbl_hit($scanstate, $ent, $rdatastr, $dom, $subrulename);
- }
- }
- # regular expression
- else {
- if ($rdatastr =~ /${subtest}/) {
$self->got_dnsbl_hit($scanstate, $ent, $rdatastr, $dom, $subrulename);
}
}

Modified: spamassassin/branches/tvd-multi-mass-check/masses/logdiff
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/masses/logdiff?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/masses/logdiff (original)
+++ spamassassin/branches/tvd-multi-mass-check/masses/logdiff Thu Aug 31 09:45:55 2006
@@ -1,8 +1,10 @@
#!/usr/bin/perl

my ($l1, $l2, @diffargs);
+@diffargs = ();
+
foreach my $arg (@ARGV) {
- if (/^-/) {
+ if ($arg =~ /^-/) {
push @diffargs, $arg;
} elsif (defined $l1) {
$l2 = $arg;

Modified: spamassassin/branches/tvd-multi-mass-check/masses/rule-qa/corpus-nightly
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/masses/rule-qa/corpus-nightly?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/masses/rule-qa/corpus-nightly (original)
+++ spamassassin/branches/tvd-multi-mass-check/masses/rule-qa/corpus-nightly Thu Aug 31 09:45:55 2006
@@ -62,15 +62,24 @@
rm -rf rules # causing trouble for some reason
rm -rf rulesrc/sandbox

+ # argh svn, messiness
+ svn cleanup < /dev/null
+
if svn update -r $revision < /dev/null && \
svn update -r $revision rulesrc < /dev/null
then
break;
fi

- pkill -TERM svn
- sleep 10
- pkill -KILL svn
+ # TODO: if that failed, we should be just using
+ # a brand new "svn checkout" and forget about the incremental
+ # update niceties
+
+ # OFF: this is extremely antisocial when multiple
+ # corpus-nightly scripts run simultaneously on one CPU!
+ # pkill -TERM svn
+ # sleep 10
+ # pkill -KILL svn

if svn update -r $revision < /dev/null && \
svn update -r $revision rulesrc < /dev/null

Modified: spamassassin/branches/tvd-multi-mass-check/rules/active.list
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/rules/active.list?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/rules/active.list (original)
+++ spamassassin/branches/tvd-multi-mass-check/rules/active.list Thu Aug 31 09:45:55 2006
@@ -1,5 +1,8 @@
# active ruleset list, automatically generated from http://ruleqa.spamassassin.org/
-# with results from: bb-doc bb-zmi cthielen daf parkerm theo wtogami zmi
+# with results from: daf parkerm theo wtogami zmi
+
+# good enough
+ADDR_NUMS_AT_BIGSITE

# good enough
ADVANCE_FEE_3
@@ -14,9 +17,6 @@
AXB_FAKETZ

# good enough
-BAD_ENC_HEADER
-
-# good enough
BANG_OPRAH

# good enough
@@ -59,6 +59,9 @@
BAYES_99

# good enough
+CHINA_HEADER
+
+# good enough
CONFIRMED_FORGED

# good enough
@@ -143,9 +146,6 @@
DNS_FROM_SECURITYSAGE

# good enough
-DOS_STOCK_O_PRICE
-
-# good enough
DRUGS_ANXIETY

# good enough
@@ -158,12 +158,6 @@
DRUGS_DIET

# good enough
-DRUGS_DIET_OBFU
-
-# good enough
-DRUGS_ERECTILE
-
-# good enough
DRUGS_ERECTILE_OBFU

# good enough
@@ -179,9 +173,6 @@
DRUG_ED_GENERIC

# good enough
-DRUG_ED_ONLINE
-
-# good enough
EARN_PER_WEEK

# good enough
@@ -197,9 +188,6 @@
ENV_AND_HDR_SPF_MATCH

# good enough
-EXCUSE_24
-
-# good enough
EXCUSE_4

# good enough
@@ -359,9 +347,6 @@
FUZZY_MERIDIA

# good enough
-FU_HOODIA
-
-# good enough
GAPPY_SUBJECT

# good enough
@@ -443,12 +428,6 @@
HS_GETMEOFF

# good enough
-HS_INDEX_PARAM
-
-# good enough
-HS_MEETUP_FOR_SEX
-
-# good enough
HS_PHARMA_1

# good enough
@@ -575,10 +554,10 @@
MISSING_MIMEOLE

# good enough
-MORE_SEX
+MISSING_MIME_HB_SEP

# good enough
-MPART_ALT_DIFF
+MISSING_SUBJECT

# good enough
MSGID_DOLLARS_RANDOM
@@ -607,9 +586,6 @@
# good enough
MULTIPART_ALT_NON_TEXT

-# good enough
-NOT_ADVISOR
-
# tflags net
NO_DNS_FOR_FROM

@@ -620,9 +596,6 @@
NO_RDNS_DOTCOM_HELO

# tflags userconf
-NO_RECEIVED
-
-# tflags userconf
NO_RELAYS

# good enough
@@ -644,6 +617,9 @@
PERCENT_RANDOM

# good enough
+PORN_15
+
+# good enough
PORN_URL_SEX

# good enough
@@ -815,9 +791,6 @@
RCVD_NUMERIC_HELO

# good enough
-REMOVE_BEFORE_LINK
-
-# good enough
REPLICA_WATCH

# good enough
@@ -914,6 +887,9 @@
SUBJ_RE_NUM

# good enough
+SUB_HELLO
+
+# good enough
SUSPICIOUS_RECIPS

# good enough
@@ -950,9 +926,6 @@
TVD_EB_PHISH

# good enough
-TVD_ENHANCE
-
-# good enough
TVD_FINGER_01

# good enough
@@ -971,9 +944,6 @@
TVD_FUZZY_FIXED_RATE

# good enough
-TVD_FUZZY_MICROCAP
-
-# good enough
TVD_FUZZY_PHARMACEUTICAL

# good enough
@@ -998,9 +968,6 @@
TVD_LINK_SAVE

# good enough
-TVD_LONG_WORD5
-
-# good enough
TVD_NOT_SATISFIED

# good enough
@@ -1013,6 +980,9 @@
TVD_PH_SUBJ_ACCOUNTS_POST

# good enough
+TVD_PH_SUBJ_META
+
+# good enough
TVD_PH_SUBJ_META_ALL

# good enough
@@ -1049,9 +1019,6 @@
TVD_SINGLE_SPAN_DIV

# good enough
-TVD_SPACED_SUBJECT_WORD3
-
-# good enough
TVD_SPACED_WORDS

# good enough
@@ -1079,40 +1046,37 @@
TVD_UA_FOSTERING

# good enough
-TVD_VIS_HIDDEN
-
-# good enough
-FM_CLAIM_IPOD
+TVD_UNDER_VALUED

# good enough
-FM_CUSTOMLOGODSGNc
+TVD_VISIT_PHARMA

# good enough
-FORGED_IMS_HTML
+TVD_VIS_HIDDEN

# good enough
-FORGED_IMS_TAGS
+DOS_STOCK_O_PRICE

# good enough
-FORGED_OUTLOOK_HTML
+DOS_YOUR_PLACE

# good enough
-FORGED_OUTLOOK_TAGS
+FM_CUSTOMLOGODSGNc

# good enough
-FS_START_DOYOU
+FORGED_IMS_HTML

# good enough
-FUZZY_SPRM
+FORGED_IMS_TAGS

# good enough
-HS_EXTRA
+FORGED_OUTLOOK_HTML

# good enough
-KAM_STOCKTIP20
+FORGED_OUTLOOK_TAGS

# good enough
-KAM_STOCKTIP3
+HS_MEETUP_FOR_SEX

# good enough
RATWARE_MS_HASH
@@ -1121,22 +1085,10 @@
RATWARE_OUTLOOK_NONAME

# good enough
-TVD_FW_GRAPHIC_ID2
-
-# good enough
-TVD_FW_GRAPHIC_NAME_LONG
-
-# good enough
-TVD_FW_GRAPHIC_NAME_MID
-
-# good enough
-TVD_UNDER_VALUED
-
-# good enough
-TVD_VISIT_PHARMA
+TVD_FUZZY_MICROCAP

# good enough
-URI_L_PHP
+TVD_FW_GRAPHIC_ID1

# good enough
UNCLAIMED_MONEY
@@ -1193,9 +1145,6 @@
URI_NOVOWEL

# good enough
-URI_NO_WWW_BIZ_CGI
-
-# good enough
URI_NO_WWW_INFO_CGI

# tflags userconf
@@ -1244,10 +1193,16 @@
X_MESSAGE_INFO

# good enough
+X_MSMAIL_PRIORITY_HIGH
+
+# good enough
X_ORIG_IP_NOT_IPV4

# good enough
X_PRIORITY_CC
+
+# good enough
+X_PRIORITY_HIGH

# good enough
ZMIde_EBAYJOBSURI

Modified: spamassassin/branches/tvd-multi-mass-check/sa-update.raw
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/sa-update.raw?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/sa-update.raw (original)
+++ spamassassin/branches/tvd-multi-mass-check/sa-update.raw Thu Aug 31 09:45:55 2006
@@ -351,6 +351,7 @@

# untaint the channel listing
for(my $ind = 0; $ind < @channels; $ind++) {
+ local ($1); # bug 5061: prevent random taint flagging of $1
if ($channels[$ind] =~ /^([a-zA-Z0-9._-]+)$/) {
$channels[$ind] = $1;
}
@@ -1290,6 +1291,8 @@
--gpg and --nogpg Use (or do not use) GPG to verify updates
(--gpg is assumed by use of the above
--gpgkey and --gpgkeyfile options)
+ --import file Import GPG key(s) from file into sa-update's
+ keyring. Use multiple times for multiple files

-D, --debug [area=n,...] Print debugging messages
-V, --version Print version
@@ -1356,7 +1359,9 @@
and signing with their own key.

By default, sa-update trusts key id C<265FA05B>, which is the standard
-SpamAssassin release key. Use this option to add more trusted keys.
+SpamAssassin release key. Use this option to trust additional keys. See the
+B<--import> option for how to add keys to sa-update's keyring. For sa-update
+to use a key it must be in sa-update's keyring and trusted.

For multiple keys, use the option multiple times. i.e.:

@@ -1376,6 +1381,20 @@
keyring. By default, this is

@@LOCAL_RULES_DIR@@/sa-update-keys
+
+=item B<--import>
+
+Use to import GPG key(s) from a file into the sa-update keyring which is
+located in the directory specified by B<--gpghomedir>. Before using channels
+from third party sources, you should use this option to import the GPG key(s)
+used by those channels. You must still use the B<--gpgkey> or B<--gpgkeyfile>
+options above to get sa-update to trust imported keys.
+
+To import multiple keys, use the option multiple times. i.e.:
+
+ sa-update --import channel1-GPG.KEY --import channel2-GPG.KEY
+
+Note: use of this option automatically enables GPG verification.

=item B<-D> [I<area,...>], B<--debug> [I<area,...>]


Modified: spamassassin/branches/tvd-multi-mass-check/spamc/libspamc.c
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/spamc/libspamc.c?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/spamc/libspamc.c (original)
+++ spamassassin/branches/tvd-multi-mass-check/spamc/libspamc.c Thu Aug 31 09:45:55 2006
@@ -1573,10 +1573,13 @@
return EX_OK;
#endif
case TRANSPORT_LOCALHOST:
- /* getaddrinfo(NULL) will look up the loopback address */
- if ((origerr = getaddrinfo(NULL, port, &hints, &res)) != 0) {
+ /* getaddrinfo(NULL) will look up the loopback address.
+ * bug 5057: unfortunately, it's the IPv6 loopback address on
+ * linux! Be explicit, and force IPv4 using "127.0.0.1".
+ */
+ if ((origerr = getaddrinfo("127.0.0.1", port, &hints, &res)) != 0) {
libspamc_log(flags, LOG_ERR,
- "getaddrinfo(NULL) failed: %s",
+ "getaddrinfo(127.0.0.1) failed: %s",
gai_strerror(origerr));
return EX_OSERR;
}

Modified: spamassassin/branches/tvd-multi-mass-check/t/basic_lint.t
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/t/basic_lint.t?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/t/basic_lint.t (original)
+++ spamassassin/branches/tvd-multi-mass-check/t/basic_lint.t Thu Aug 31 09:45:55 2006
@@ -13,7 +13,7 @@
);

# override locale for this test!
-$ENV{'LC_ALL'} = 'C';
+$ENV{'LANGUAGE'} = $ENV{'LC_ALL'} = 'C';

sarun ("-L --lint", \&patterns_run_cb);
ok_all_patterns();

Modified: spamassassin/branches/tvd-multi-mass-check/t/gtube.t
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/t/gtube.t?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/t/gtube.t (original)
+++ spamassassin/branches/tvd-multi-mass-check/t/gtube.t Thu Aug 31 09:45:55 2006
@@ -22,7 +22,7 @@
endif
");

-$ENV{'LC_ALL'} = 'C'; # a cheat, but we match the description
+$ENV{'LANGUAGE'} = $ENV{'LC_ALL'} = 'C'; # a cheat, but we match the description

ok (sarun ("-L -t < data/spam/gtube.eml", \&patterns_run_cb));
ok_all_patterns();

Modified: spamassassin/branches/tvd-multi-mass-check/t/lang_pl_tests.t
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/t/lang_pl_tests.t?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/t/lang_pl_tests.t (original)
+++ spamassassin/branches/tvd-multi-mass-check/t/lang_pl_tests.t Thu Aug 31 09:45:55 2006
@@ -13,6 +13,7 @@
);

$ENV{'PERL_BADLANG'} = 0; # Sweep problems under the rug
+$ENV{'LANGUAGE'} = 'pl_PL';
$ENV{'LC_ALL'} = 'pl';
sarun ("-L -t < data/nice/004", \&patterns_run_cb);
ok_all_patterns();

Modified: spamassassin/branches/tvd-multi-mass-check/t/lint_nocreate_prefs.t
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/t/lint_nocreate_prefs.t?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/t/lint_nocreate_prefs.t (original)
+++ spamassassin/branches/tvd-multi-mass-check/t/lint_nocreate_prefs.t Thu Aug 31 09:45:55 2006
@@ -9,7 +9,7 @@
%patterns = ( q{ }, 'anything' );

# override locale for this test!
-$ENV{'LC_ALL'} = 'C';
+$ENV{'LANGUAGE'} = $ENV{'LC_ALL'} = 'C';

sarun ("-L --lint --prefspath=log/prefs", \&patterns_run_cb);
ok_all_patterns();

Modified: spamassassin/branches/tvd-multi-mass-check/t/mimeheader.t
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/t/mimeheader.t?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/t/mimeheader.t (original)
+++ spamassassin/branches/tvd-multi-mass-check/t/mimeheader.t Thu Aug 31 09:45:55 2006
@@ -4,7 +4,7 @@
use SATest; sa_t_init("mimeheader");
use Test; BEGIN { plan tests => 2 };

-$ENV{'LC_ALL'} = 'C'; # a cheat, but we need the patterns to work
+$ENV{'LANGUAGE'} = $ENV{'LC_ALL'} = 'C'; # a cheat, but we need the patterns to work

# ---------------------------------------------------------------------------


Modified: spamassassin/branches/tvd-multi-mass-check/t/prefs_include.t
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/t/prefs_include.t?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/t/prefs_include.t (original)
+++ spamassassin/branches/tvd-multi-mass-check/t/prefs_include.t Thu Aug 31 09:45:55 2006
@@ -4,7 +4,7 @@
use SATest; sa_t_init("prefs_include");
use Test; BEGIN { plan tests => 2 };

-$ENV{'LC_ALL'} = 'C'; # a cheat, but we need the patterns to work
+$ENV{'LANGUAGE'} = $ENV{'LC_ALL'} = 'C'; # a cheat, but we need the patterns to work

# ---------------------------------------------------------------------------


Modified: spamassassin/branches/tvd-multi-mass-check/t/reportheader.t
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/t/reportheader.t?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/t/reportheader.t (original)
+++ spamassassin/branches/tvd-multi-mass-check/t/reportheader.t Thu Aug 31 09:45:55 2006
@@ -4,7 +4,7 @@
use SATest; sa_t_init("reportheader");
use Test; BEGIN { plan tests => 11 };

-$ENV{'LC_ALL'} = 'C'; # a cheat, but we need the patterns to work
+$ENV{'LANGUAGE'} = $ENV{'LC_ALL'} = 'C'; # a cheat, but we need the patterns to work

# ---------------------------------------------------------------------------


Modified: spamassassin/branches/tvd-multi-mass-check/t/reportheader_8bit.t
URL: http://svn.apache.org/viewvc/spamassassin/branches/tvd-multi-mass-check/t/reportheader_8bit.t?rev=438980&r1=438979&r2=438980&view=diff
==============================================================================
--- spamassassin/branches/tvd-multi-mass-check/t/reportheader_8bit.t (original)
+++ spamassassin/branches/tvd-multi-mass-check/t/reportheader_8bit.t Thu Aug 31 09:45:55 2006
@@ -4,7 +4,7 @@
use SATest; sa_t_init("reportheader");
use Test; BEGIN { plan tests => 2 };

-$ENV{'LC_ALL'} = 'C'; # a cheat, but we need the patterns to work
+$ENV{'LANGUAGE'} = $ENV{'LC_ALL'} = 'C'; # a cheat, but we need the patterns to work

# ---------------------------------------------------------------------------