Mailing List Archive

[Bug 7909] SPF failure with correct SPF record
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7909

Bill Cole <billcole@apache.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |INVALID
Status|NEW |RESOLVED
CC| |billcole@apache.org

--- Comment #1 from Bill Cole <billcole@apache.org> ---
This is almost certainly a local configuration error. Absent any details on the
sending domain and relevant SA configuration parameters, it cannot be
considered a bug in SA.

SPF testing by SpamAssassin is done based on the trusted_networks,
internal_networks, and msa_networks settings, which are used to determine which
Received headers to trust as non-forged and which to treat as internal
handoffs. In this case, the final Received header is being treated as internal,
while an internal handoff within Microsoft is being treated as trusted but
non-internal. If you include Microsoft's external IPv4 space in your
"internal_networks" setting g, you probably should also include their public
internal-use IPv6 space to avoid this issue.

You can probably get help on how to configure the *_networks parameters for
your specific circumstances on the SpamAssassin Users mailing list, where
others with vendor relationships with Microsoft and analogous providers can
explain their solutions.


Also:

1. 2603:10a6:20b:284::13 is not "bogus" it is a perfectly valid IPv6 address in
the public 2603:1000::/24 network allocated by ARIN to Microsoft.

2. v3.4.2 is severely outdated and has known functionality and security bugs
which have been fixed in later versions. Please update to 3.4.6 if possible, or
to a packaged version which has critical updates back-ported (e.g the package
maintained by RedHat for RHEL and CentOS)

3. You appear to have locally assigned a score of 5.0 to SPF_FAIL, which is
4.99 points higher than the default score. This will reliably cause non-spam to
get scored as spam, because legitimate domain owners are less diligent about
getting their SPF records correct than are owners of domains used primarily to
spam.

This report is not actionable as a bug due to redactions and lack of config
info, The reported issue appears to be the result of configuration errors, it
is reported for any obsolete version of SpamAssassin, and the
misclassification is the result of a dangerous local score override.

--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7909] SPF failure with correct SPF record [ In reply to ]
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7909

Andrey L <sabug@lelik.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |sabug@lelik.org

--- Comment #2 from Andrey L <sabug@lelik.org> ---
Of course it is not a local config issue, there are no explicit internal or
terusted networks set, and as a matter of fact this is default installation
with default config on debian buster.

This is a clear and obivous bug on how SA infers internal networks and skips
received headers.

Your only excuse that it happens on 3.4.2 . Before getting a reply I've
installed the 3.4.4 from backports, and the bug went away:

Content analysis details: (-99.3 points, 5.0 required)

pts rule name description
---- ---------------------- --------------------------------------------------
-100 USER_IN_WHITELIST From: address is in the user's white-list
-1.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2)
[40.107.15.83 listed in wl.mailspike.net]
-0.0 SPF_PASS SPF: sender matches SPF record
-0.0 SPF_HELO_PASS SPF: HELO matches SPF record
0.5 SUBJ_ALL_CAPS Subject is all capitals
0.0 HTML_MESSAGE BODY: HTML included in message
0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily
valid
1.0 MIXED_ES Too many es are not es
0.1 DKIM_INVALID DKIM or DK signature exists, but is not valid

--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7909] SPF failure with correct SPF record [ In reply to ]
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7909

RW <rwmaillists@googlemail.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |rwmaillists@googlemail.com

--- Comment #3 from RW <rwmaillists@googlemail.com> ---
Was this run from a milter? If so are you sure that the received header from
your server was present at the time SA ran and not added afterwards.

--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7909] SPF failure with correct SPF record [ In reply to ]
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7909

--- Comment #4 from Andrey L <sabug@lelik.org> ---
Spot on, I think you are right.

It IS used via milter interface.

I did not realize that milter (spamass-milter to be preciese) constructs a fake
"received" header from hello. The log shows that SA failed to parse some
received headers, likely the one constructed by milter. So it is either a bug
in spamass-milter (likely) or in SA header parsing code.

Any hints how I can dump this constructed header, short of building a debug
version of the milter or patching SA code? For debug I'd like to dump the
constructed "received" header to something like "x-fake-received".

Thanks a lot for your input!

p.s. i wonder if tls connection played a role...


May 18 16:04:10 mail postfix/smtpd[30928]: connect from
mail-eopbgr60040.outbound.protection.outlook.com[40.107.6.40]
May 18 16:04:10 mail postfix/smtpd[30928]: Anonymous TLS connection established
from mail-eopbgr60040.outbound.protection.outlook.com[40.107.6.40]: TLSv1.2
with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
May 18 16:04:10 mail postfix/smtpd[30928]: 4Fkx625W3Mz3wYH:
client=mail-eopbgr60040.outbound.protection.outlook.com[40.107.6.40]
May 18 16:04:10 mail postfix/cleanup[30935]: 4Fkx625W3Mz3wYH:
message-id=<AM9PR03MB70289AF4DC881C6620446282A12C9@AM9PR03MB7028.eurprd03.prod.outlook.com>
May 18 16:04:11 mail spamd[2133]: spamd: connection from ::1 [::1]:58464 to
port 783, fd 5
May 18 16:04:11 mail spamd[2133]: spamd: handle_user (getpwnam) unable to find
user: '[REDACTED]'
May 18 16:04:11 mail spamd[2133]: spamd: still running as root: user not
specified with -u, not found, or set to root, falling back to nobody
May 18 16:04:11 mail spamd[2133]: spamd: processing message
<AM9PR03MB70289AF4DC881C6620446282A12C9@AM9PR03MB7028.eurprd03.prod.outlook.com>
for [REDACTED]:65534
May 18 16:04:12 mail spamd[2133]: spamd: identified spam (8.0/5.0) for
[REDACTED]:65534 in 1.1 seconds, 46161 bytes.
May 18 16:04:12 mail spamd[2133]: spamd: result: Y 8 -
DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,HTML_MESSAGE,MIXED_ES,SPF_FAIL,SPF_HELO_NONE,UNPARSEABLE_RELAY
scantime=1.1,size=46161,user=[REDACTED],uid=65534,required_score=5.0,rhost=::1,raddr=::1,rport=58464,mid=<AM9PR03MB70289AF4DC881C6620446282A12C9@AM9PR03MB7028.eurprd03.prod.outlook.com>,autolearn=no
autolearn_force=no
May 18 16:04:12 mail postfix/qmgr[12038]: 4Fkx625W3Mz3wYH:
from=<[REDACTED]@hoermann.ru>, size=46210, nrcpt=1 (queue active)
May 18 16:04:12 mail postfix/smtpd[30928]: disconnect from
mail-eopbgr60040.outbound.protection.outlook.com[40.107.6.40] ehlo=2 starttls=1
mail=1 rcpt=1 bdat=1 quit=1 commands=7
May 18 16:04:12 mail postfix/pipe[30937]: 4Fkx625W3Mz3wYH:
to=<andrey@lelik.org>, relay=dovecot, delay=1.5, delays=1.4/0.03/0/0.11,
dsn=2.0.0, status=sent (delivered via dovecot service)
May 18 16:04:12 mail postfix/qmgr[12038]: 4Fkx625W3Mz3wYH: removed

--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7909] SPF failure with correct SPF record [ In reply to ]
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7909

--- Comment #5 from Bill Cole <billcole@apache.org> ---
(In reply to Andrey L from comment #4)
> Spot on, I think you are right.
>
> It IS used via milter interface.
>
> I did not realize that milter (spamass-milter to be preciese) constructs a
> fake "received" header from hello. The log shows that SA failed to parse
> some received headers, likely the one constructed by milter. So it is either
> a bug in spamass-milter (likely) or in SA header parsing code.
>
> Any hints how I can dump this constructed header, short of building a debug
> version of the milter or patching SA code? For debug I'd like to dump the
> constructed "received" header to something like "x-fake-received".

Add '-D received-header,metadata' to the arguments of your spamd process. That
will add log entries describing the parsing and end results of Received header
parsing.


> Thanks a lot for your input!
>
> p.s. i wonder if tls connection played a role...

Unlikely.

--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7909] SPF failure with correct SPF record [ In reply to ]
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7909

--- Comment #6 from Andrey L <sabug@lelik.org> ---
Ok, we are one step closer. This is a definite bug in spamass-milter . Now, I
just have to figure why it happens...

I think it would be nice if we can track this bug here, as spamass-milter is
de-facto unmaintained...

May 19 11:36:14 mail postfix/smtpd[11268]: connect from
kot50.emails.tinkoff.ru[185.138.181.50]
May 19 11:36:14 mail postfix/smtpd[11268]: Anonymous TLS connection established
from kot50.emails.tinkoff.ru[185.138.181.50]: TLSv1.2 with cipher
ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
May 19 11:36:14 mail postfix/smtpd[11268]: 4FlR6Q4RNhz3xJc:
client=kot50.emails.tinkoff.ru[185.138.181.50]
May 19 11:36:14 mail postfix/cleanup[11275]: 4FlR6Q4RNhz3xJc:
message-id=<20210519113112.0.20210519113112_eui_@1163550.1906942035.28485779.masssending.tinkoff>
May 19 11:36:14 mail spamd[8167]: spamd: connection from ::1 [::1]:35572 to
port 783, fd 5
May 19 11:36:14 mail spamd[8167]: spamd: handle_user (getpwnam) unable to find
user: 'tattink'
May 19 11:36:14 mail spamd[8167]: spamd: still running as root: user not
specified with -u, not found, or set to root, falling back to nobody
May 19 11:36:14 mail spamd[8167]: spamd: processing message
<20210519113112.0.20210519113112_eui_@1163550.1906942035.28485779.masssending.tinkoff>
for tattink:65534
May 19 11:36:14 mail spamd[8167]: received-header: unparseable: from
kot50.emails.tinkoff.ru (unknown) by mail.lelik.us (Postfix 3.4.14/8.13.0) with
SMTP id unknown Wed, 19 May 2021 11:36:14 +0300 (envelope-from
<email-1163550-1906942035-28485779-tinkoff@emails.tinkoff.ru>);
May 19 11:36:14 mail spamd[8167]: metadata: X-Spam-Relays-Trusted:
May 19 11:36:14 mail spamd[8167]: metadata: X-Spam-Relays-Untrusted:
May 19 11:36:14 mail spamd[8167]: metadata: X-Spam-Relays-Internal:
May 19 11:36:14 mail spamd[8167]: metadata: X-Spam-Relays-External:
May 19 11:36:14 mail spamd[8167]: spamd: clean message (2.6/5.0) for
tattink:65534 in 0.2 seconds, 27671 bytes.

--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7909] SPF failure with correct SPF record [ In reply to ]
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7909

--- Comment #7 from Andrey L <sabug@lelik.org> ---
Also, this is covered here -
https://serverfault.com/questions/574255/missing-client-ip-in-postfix-received-header

--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7909] SPF failure with correct SPF record [ In reply to ]
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7909

--- Comment #8 from Bill Cole <billcole@apache.org> ---
(In reply to Andrey L from comment #6)
> Ok, we are one step closer. This is a definite bug in spamass-milter . Now,
> I just have to figure why it happens...
>
> I think it would be nice if we can track this bug here, as spamass-milter is
> de-facto unmaintained...

You are free to update this report with any new information relevant to this
issue, but spamass-milter isn't part of Apache SpamAssassin. You can find other
users of spamass-milter on the SpamAssassin Users mailing list, maybe even
people who have found solutions to this apparent problem.

There ARE milters being actively maintained (albeit also NOT by the ASF) that
use SA, e.g. Amavis & MIMEDefang, whose users also can be reached via the
SpamAssassin Users mailing list.

--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7909] SPF failure with correct SPF record [ In reply to ]
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7909

Bill Cole <billcole@apache.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC|billcole@apache.org |

--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7909] SPF failure with correct SPF record [ In reply to ]
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7909

--- Comment #9 from Andrey L <sabug@lelik.org> ---
Created attachment 5745
--> https://bz.apache.org/SpamAssassin/attachment.cgi?id=5745&action=edit
spamass-milter patch

--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7909] SPF failure with correct SPF record [ In reply to ]
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7909

--- Comment #10 from Andrey L <sabug@lelik.org> ---
Ok, I've greatly underestimated the uglyness of sendmail design.

Just to record all this crap for the history:

This is sort of complex posfix config / milter issue and the way variables from
MTA to milter are propagated.
The page http://www.postfix.org/MILTER_README.html#send-macros list macros that
postfix passes to milter. The "always available" status for macro {client_addr}
means that in theory postfix can calculate its value at any stage. IF, and only
if the name of this variable is specified in corresponding milter_macro config
setting, set in config or by default (from postfix sources). Neither "_" or
"{client_addr}" is set in sources, so milter does not get this info.

For years, postfix and milter library support API that let the milter send a
list of requested variables to MTA, instead of forcing the user to read sources
for postfix and milter, and specify these vars in config.

Also, even if no macro variable with client IP is available, the client ip
address is still presented to milter during the hello callback and can be used
as a fallback instead of "unknown".

Attached patch does exactly these 3 things:
1 - calls new API (if available) to request necessary variables
2 - falls back to client address from hello
3 - uses postfix variables (when present) to assemble a full received header.

hopefully someone would find this useful...

--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7909] SPF failure with correct SPF record [ In reply to ]
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7909

dbuergin@gluet.ch changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |dbuergin@gluet.ch

--- Comment #11 from dbuergin@gluet.ch ---
By the way, this and similar problems are what prompted me to create a new
implementation of the milter a while ago:
https://crates.io/crates/spamassassin-milter

--
You are receiving this mail because:
You are the assignee for the bug.