Mailing List Archive

On 2021-10-09 at 11:39:48 UTC-0400 (Sat, 9 Oct 2021 17:39:48 +0200)
Thomas Seilund <tps@netmaster.dk>
is rumored to have said:

> Dear All
>
> I see incomming mail that I would imagine that SA should classify as
> spam but mail only gets a score of 2
>
> When I run the same mail through spamc from the command line the score
> is 11.8.
>
> Why is the score not 11.8 when the mail is handled by Postfix/SA?
>
> Pls. see details below.

Based on those details, I'd say it was a matter of time.

The fact that you're running the message through spamc/spamd in both
cases with spamd running with the options "--nouser-config" and
"--username=vmail" eliminates all the more arcane sources of such
discrepancies.


> Mail server is Debian Buster running Postfix and SA 3.4.2.

You should upgrade SA. The current release is 3.4.6 and it includes
significant performance, functionality, and security improvements. You
aren't in severe danger (the security issues have no reported exploits
in the wild) but upgrading would be prudent.

> SA is called through a content filter in Postfix file master.cf
>
> I have debug on spamd set up in /etc/default/spamassassin by the line
> 'OPTIONS="--debug --max-children=5 --username=vmail --nouser-config"'
>
> After adding --debug and restarting SA with `sudo systemctl restart
> spamassassin` I see a lot of debug info in /var/log/mail.log
>
> I have run the mail through `spamc` using this command. Command and
> output shown below:
>
> sudo cat
> /srv/vmail/netmaster.dk/tps/mail/cur/1633788457.M618066P6142.linode2.netmaster.dk,S=5989,W=6185:2,Sc
> | spamc -R
> 11.8/5.0
> Spam detection software, running on the system "linode2.netmaster.dk",
> has identified this incoming email as possible spam.  The original
> message has been attached to this so you can view it or label
> similar future email.  If you have any questions, see
> the administrator of that system for details.
>
> Content preview:  The coolest and comfiest sandals to wear around the
> house,
>    or even at the office Everyone is talking about these colorful
> rubber shoes,
>    designed to ensure maximum comfort and stability. They are non-
> [...]
>
> Content analysis details:   (11.8 points, 5.0 required)
>
>  pts rule name              description
> ---- ----------------------
> --------------------------------------------------
>  2.5 URIBL_DBL_SPAM         Contains a spam URL listed in the
> Spamhaus DBL
>                             blocklist
>                             [URIs:
> nerveoil.bar]
>  1.9 URIBL_ABUSE_SURBL      Contains an URL listed in the ABUSE
> SURBL
>                             blocklist
>                             [URIs:
> nerveoil.bar]
>  3.6 RCVD_IN_SBL_CSS        RBL: Received via a relay in
> Spamhaus SBL-CSS
>                             [2.56.57.119
> listed in zen.spamhaus.org]
>  1.7 URIBL_BLACK            Contains an URL listed in the
> URIBL blacklist
>                             [URIs:
> nerveoil.bar]
> -0.0 SPF_HELO_PASS          SPF: HELO matches SPF record
>  0.1 MIME_HTML_ONLY         BODY: Message only has text/html
> MIME parts
>  0.0 HTML_MESSAGE           BODY: HTML included in message
>  0.6 HTML_MIME_NO_HTML_TAG  HTML-only message, but there is no HTML
>                             tag
>  1.3 RDNS_NONE              Delivered to internal
> network by a host with no rDNS
>  0.1 PLING_QUERY            Subject has exclamation mark
> and question mark
>
> The mail gets a score of 11.8 points.
>
> But when the mail was first received by Postfix is was only given a
> score of 2.

9.7 points of that score are due to various DNSBLs, which by their
nature tend to miss the first few instances of new spam runs.

> Why is the score different when Postfix sends mail to SA and when I do
> it manually from the command line?

22 minutes of opportunity for the shared spam-control resources of the
world to engage.


> I have lines in /var/log/mail.log that shows the two cases. Command
> and output below:

[...]

> Oct  9 16:07:37 linode2 spamd[1009]: spamd: result: . 2 -
> HTML_MESSAGE,HTML_MIME_NO_HTML_TAG,MIME_HTML_ONLY,PLING_QUERY,RDNS_NONE,SPF_HELO_PASS
[...]
> Oct  9 16:29:20 linode2 spamd[1008]: spamd: result: Y 11 -
> HTML_MESSAGE,HTML_MIME_NO_HTML_TAG,MIME_HTML_ONLY,PLING_QUERY,RCVD_IN_SBL_CSS,RDNS_NONE,SPF_HELO_PASS,URIBL_ABUSE_SURBL,URIBL_BLACK,URIBL_DBL_SPAM

When run via Postfix on arrival, the source IP was not in the Spamhaus
SBL's CSS sublist, and none of the URLs in the body were listed in the
cited URIBL lists. 22 minutes later, the sources used by those DNSBLs
caught up with that particular spam run, so the second test hit a bunch
of them. I see this all the time. The first of many targets sent a
particular spam are unprotected because the spam is new but as new
instances of the same message come in for other targets they start
showing up in shared reputation systems, hitting various RCVD_IN_* and
URIBL_* rules in SA.

There's intrinsically no good fix for this. Some systems deploy a tactic
called "greylisting" where the first message from an unknown source is
deferred a few minutes (i.e. initially rejected with a "try again later"
response code.) which mostly eliminates the issue, but it also creates
some problems naturally (legitimate mail gets delayed, which is by
design) and others that are due to unpredictable retry behaviors by
legitimate sending systems that result in mail never being delivered.

Another partial solution with Postfix is to enable its postscreen
component with the greeting delay feature enabled, which bears a slight
resemblance to greylisting, but is safer because it only ever rejects or
defers senders who violate the SMTP protocol. This at least assures that
you will get a few seconds of delay, which can be enough for the DNSBLs
to catch up with the latest spammer.

--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire