Mailing List Archive

Hi,

It's definitely a problem with the current spamassassin from github v4. I
went back to an old version I built on May 29th and it immediately starts
reporting DKIM normally again.

I just built the latest version and it's still exhibiting the same problem.
Based on my logs, it started happening on or around June 14th.

DMARC is not working with my version from May 29th. I wonder if I could
drop in the DMARC.pm that was updated at the end of June into the May 29th
version and see if now they both work?

btw, I previously mentioned github, but meant svn.apache.org.
svn checkout http://svn.apache.org/repos/asf/spamassassin/trunk
Mail-SpamAssassin-4.0.0







On Sat, Jun 25, 2022 at 3:07 PM Alex <mysqlstudent@gmail.com> wrote:

> Hi,
> I've been having problems with DMARC failing over the past few weeks using
> the latest SA, even on sites I know have passed. It appears to have
> coincided with an update to DMARC.pm related to timing. I just now happened
> to notice that maybe the problem is with DKIM, or there's a separate DKIM
> problem or something I simply don't understand. Installing v3.4.6 over the
> latest v4 fixes the problem instantly.
>
> It appears DKIM is loading in amavis:
> Jun 25 00:13:09 mail03 amavis[4119158]: Module Mail::DKIM::Signer
> 1.20200907
> Jun 25 00:13:09 mail03 amavis[4119158]: Module Mail::DKIM::Verifier
> 1.20200907
> Jun 25 00:13:09 mail03 amavis[4119158]: DKIM code loaded
> Jun 25 00:13:18 mail03 amavis[4119158]: SpamAssassin loaded plugins: ASN,
> AskDNS, AutoLearnThreshold, BTCBL, Bayes, BodyEval, Check, ClamAV, DCC,
> DKIM, DMARC, DNSEval, DecodeShortURLs, FreeMail, FromNameSpoof, HTMLEval,
> HTTPSMismatch, HashBL, HeaderEval, ImageInfo, Levenshtein, MIMEEval,
> MIMEHeader, OLEVBMacro, PDFInfo, PhishTag, Phishing, Pyzor, Razor2,
> RecipientMsgID, RelayCountry, RelayEval, ReplaceTags, SPF, SendGrid,
> Shortcircuit, SpamCop, TextCat, TxRep, URIDNSBL, URIDetail, URIEval,
> URILocalBL, VBounce, WLBLEval, WelcomeListSubject, iXhash2
>
> Yet it never fires. The only references to DKIM in emails are
> from DKIM_ADSP_ALL. What could I be missing, or is this possibly a bug?
>
> You might also recall from my previous reports that DKIM succeeds on an
> email where it otherwise failed when running it through SA directly.
>
> $ spamassassin --version
> SpamAssassin version 4.0.0-r1901426
> running on Perl version 5.34.1
>
> This is on fedora35. Installing the stock 3.4.6 immediately starts
> triggering DKIM hits.
>
> Is there a backport of RaciallyCharged, Esp and ExtractText (although I
> don't really use that anymore) that's available for v3.4.6, so my
> welcomelist entries work in the meantime?
>
>
>

Have you checked what debugging says?

$sa_debug = 'info,dkim,DMARC';


On Sat, Jun 25, 2022 at 03:45:48PM -0400, Alex wrote:
> Hi,
>
> It's definitely a problem with the?current spamassassin from github v4. I went
> back to an old version I built on May 29th and it immediately starts reporting
> DKIM normally again.?
>
> I just built the latest version and it's still exhibiting the same problem.
> Based on my logs, it started happening on or around June 14th.?
>
> DMARC is not working with?my version from?May 29th. I wonder if I could drop in
> the DMARC.pm that was updated at the end of June into the May 29th version and
> see if now they both work?
>
> btw, I previously mentioned github, but meant [1]svn.apache.org.
> svn checkout [2]http://svn.apache.org/repos/asf/spamassassin/trunk
> Mail-SpamAssassin-4.0.0
>
>
>
>
>
>
>
> On Sat, Jun 25, 2022 at 3:07 PM Alex <[3]mysqlstudent@gmail.com> wrote:
>
> Hi,
> I've been having problems with DMARC failing over the past few weeks using
> the latest SA, even on sites I know have passed. It appears to have
> coincided with an update to DMARC.pm related to timing. I just now happened
> to notice that maybe the problem is with DKIM, or there's a separate?DKIM
> problem or something I simply don't understand. Installing v3.4.6 over the
> latest v4 fixes the problem instantly.
>
> It appears DKIM is loading in amavis:
> Jun 25 00:13:09 mail03 amavis[4119158]: Module Mail::DKIM::Signer
> ?1.20200907
> Jun 25 00:13:09 mail03 amavis[4119158]: Module Mail::DKIM::Verifier
> 1.20200907
> Jun 25 00:13:09 mail03 amavis[4119158]: DKIM code ? ? ? ? ? ?loaded
> Jun 25 00:13:18 mail03 amavis[4119158]: SpamAssassin loaded plugins: ASN,
> AskDNS, AutoLearnThreshold, BTCBL, Bayes, BodyEval, Check, ClamAV, DCC,
> DKIM, DMARC, DNSEval, DecodeShortURLs, FreeMail, FromNameSpoof, HTMLEval,
> HTTPSMismatch, HashBL, HeaderEval, ImageInfo, Levenshtein, MIMEEval,
> MIMEHeader, OLEVBMacro, PDFInfo, PhishTag, Phishing, Pyzor, Razor2,
> RecipientMsgID, RelayCountry, RelayEval, ReplaceTags, SPF, SendGrid,
> Shortcircuit, SpamCop, TextCat, TxRep, URIDNSBL, URIDetail, URIEval,
> URILocalBL, VBounce, WLBLEval, WelcomeListSubject, iXhash2
>
> Yet it never fires. The only references to DKIM in emails are
> from?DKIM_ADSP_ALL. What could I be missing, or is this possibly a bug?
>
> You might also recall from my previous reports that DKIM succeeds on an
> email where it otherwise failed when running it through SA directly.
>
> $ spamassassin --version
> SpamAssassin version 4.0.0-r1901426
> ? running on Perl version 5.34.1
>
> This is on fedora35. Installing the stock 3.4.6 immediately starts
> triggering DKIM hits.
>
> Is there a backport of?RaciallyCharged, Esp and ExtractText (although I
> don't really use that anymore) that's available for v3.4.6, so my
> welcomelist entries work in the meantime?
>
>
>
>
> References:
>
> [1] http://svn.apache.org/
> [2] http://svn.apache.org/repos/asf/spamassassin/trunk
> [3] mailto:mysqlstudent@gmail.com

All the people that reported DKIM failures to me in SA 4.0 in the last month are using amavisd-new, could it be related to how amavisd-new is calling SA ?
Giovanni

On 6/26/22 07:55, Henrik K wrote:
>
> Have you checked what debugging says?
>
> $sa_debug = 'info,dkim,DMARC';
>
>
> On Sat, Jun 25, 2022 at 03:45:48PM -0400, Alex wrote:
>> Hi,
>>
>> It's definitely a problem with the current spamassassin from github v4. I went
>> back to an old version I built on May 29th and it immediately starts reporting
>> DKIM normally again. 
>>
>> I just built the latest version and it's still exhibiting the same problem.
>> Based on my logs, it started happening on or around June 14th. 
>>
>> DMARC is not working with my version from May 29th. I wonder if I could drop in
>> the DMARC.pm that was updated at the end of June into the May 29th version and
>> see if now they both work?
>>
>> btw, I previously mentioned github, but meant [1]svn.apache.org.
>> svn checkout [2]http://svn.apache.org/repos/asf/spamassassin/trunk
>> Mail-SpamAssassin-4.0.0
>>
>>
>>
>>
>>
>>
>>
>> On Sat, Jun 25, 2022 at 3:07 PM Alex <[3]mysqlstudent@gmail.com> wrote:
>>
>> Hi,
>> I've been having problems with DMARC failing over the past few weeks using
>> the latest SA, even on sites I know have passed. It appears to have
>> coincided with an update to DMARC.pm related to timing. I just now happened
>> to notice that maybe the problem is with DKIM, or there's a separate DKIM
>> problem or something I simply don't understand. Installing v3.4.6 over the
>> latest v4 fixes the problem instantly.
>>
>> It appears DKIM is loading in amavis:
>> Jun 25 00:13:09 mail03 amavis[4119158]: Module Mail::DKIM::Signer
>>  1.20200907
>> Jun 25 00:13:09 mail03 amavis[4119158]: Module Mail::DKIM::Verifier
>> 1.20200907
>> Jun 25 00:13:09 mail03 amavis[4119158]: DKIM code            loaded
>> Jun 25 00:13:18 mail03 amavis[4119158]: SpamAssassin loaded plugins: ASN,
>> AskDNS, AutoLearnThreshold, BTCBL, Bayes, BodyEval, Check, ClamAV, DCC,
>> DKIM, DMARC, DNSEval, DecodeShortURLs, FreeMail, FromNameSpoof, HTMLEval,
>> HTTPSMismatch, HashBL, HeaderEval, ImageInfo, Levenshtein, MIMEEval,
>> MIMEHeader, OLEVBMacro, PDFInfo, PhishTag, Phishing, Pyzor, Razor2,
>> RecipientMsgID, RelayCountry, RelayEval, ReplaceTags, SPF, SendGrid,
>> Shortcircuit, SpamCop, TextCat, TxRep, URIDNSBL, URIDetail, URIEval,
>> URILocalBL, VBounce, WLBLEval, WelcomeListSubject, iXhash2
>>
>> Yet it never fires. The only references to DKIM in emails are
>> from DKIM_ADSP_ALL. What could I be missing, or is this possibly a bug?
>>
>> You might also recall from my previous reports that DKIM succeeds on an
>> email where it otherwise failed when running it through SA directly.
>>
>> $ spamassassin --version
>> SpamAssassin version 4.0.0-r1901426
>>   running on Perl version 5.34.1
>>
>> This is on fedora35. Installing the stock 3.4.6 immediately starts
>> triggering DKIM hits.
>>
>> Is there a backport of RaciallyCharged, Esp and ExtractText (although I
>> don't really use that anymore) that's available for v3.4.6, so my
>> welcomelist entries work in the meantime?
>>
>>
>>
>>
>> References:
>>
>> [1] http://svn.apache.org/
>> [2] http://svn.apache.org/repos/asf/spamassassin/trunk
>> [3] mailto:mysqlstudent@gmail.com

Amavisd-new works fine here. Maybe $enable_dkim_verification or something is different.

On Sun, Jun 26, 2022 at 03:32:14PM +0200, giovanni@paclan.it wrote:
> All the people that reported DKIM failures to me in SA 4.0 in the last month are using amavisd-new, could it be related to how amavisd-new is calling SA ?
> Giovanni
>
> On 6/26/22 07:55, Henrik K wrote:
> >
> > Have you checked what debugging says?
> >
> > $sa_debug = 'info,dkim,DMARC';
> >
> >
> > On Sat, Jun 25, 2022 at 03:45:48PM -0400, Alex wrote:
> >> Hi,
> >>
> >> It's definitely a problem with the?current spamassassin from github v4. I went
> >> back to an old version I built on May 29th and it immediately starts reporting
> >> DKIM normally again.?
> >>
> >> I just built the latest version and it's still exhibiting the same problem.
> >> Based on my logs, it started happening on or around June 14th.?
> >>
> >> DMARC is not working with?my version from?May 29th. I wonder if I could drop in
> >> the DMARC.pm that was updated at the end of June into the May 29th version and
> >> see if now they both work?
> >>
> >> btw, I previously mentioned github, but meant [1]svn.apache.org.
> >> svn checkout [2]http://svn.apache.org/repos/asf/spamassassin/trunk
> >> Mail-SpamAssassin-4.0.0
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> On Sat, Jun 25, 2022 at 3:07 PM Alex <[3]mysqlstudent@gmail.com> wrote:
> >>
> >> Hi,
> >> I've been having problems with DMARC failing over the past few weeks using
> >> the latest SA, even on sites I know have passed. It appears to have
> >> coincided with an update to DMARC.pm related to timing. I just now happened
> >> to notice that maybe the problem is with DKIM, or there's a separate?DKIM
> >> problem or something I simply don't understand. Installing v3.4.6 over the
> >> latest v4 fixes the problem instantly.
> >>
> >> It appears DKIM is loading in amavis:
> >> Jun 25 00:13:09 mail03 amavis[4119158]: Module Mail::DKIM::Signer
> >> ?1.20200907
> >> Jun 25 00:13:09 mail03 amavis[4119158]: Module Mail::DKIM::Verifier
> >> 1.20200907
> >> Jun 25 00:13:09 mail03 amavis[4119158]: DKIM code ? ? ? ? ? ?loaded
> >> Jun 25 00:13:18 mail03 amavis[4119158]: SpamAssassin loaded plugins: ASN,
> >> AskDNS, AutoLearnThreshold, BTCBL, Bayes, BodyEval, Check, ClamAV, DCC,
> >> DKIM, DMARC, DNSEval, DecodeShortURLs, FreeMail, FromNameSpoof, HTMLEval,
> >> HTTPSMismatch, HashBL, HeaderEval, ImageInfo, Levenshtein, MIMEEval,
> >> MIMEHeader, OLEVBMacro, PDFInfo, PhishTag, Phishing, Pyzor, Razor2,
> >> RecipientMsgID, RelayCountry, RelayEval, ReplaceTags, SPF, SendGrid,
> >> Shortcircuit, SpamCop, TextCat, TxRep, URIDNSBL, URIDetail, URIEval,
> >> URILocalBL, VBounce, WLBLEval, WelcomeListSubject, iXhash2
> >>
> >> Yet it never fires. The only references to DKIM in emails are
> >> from?DKIM_ADSP_ALL. What could I be missing, or is this possibly a bug?
> >>
> >> You might also recall from my previous reports that DKIM succeeds on an
> >> email where it otherwise failed when running it through SA directly.
> >>
> >> $ spamassassin --version
> >> SpamAssassin version 4.0.0-r1901426
> >> ? running on Perl version 5.34.1
> >>
> >> This is on fedora35. Installing the stock 3.4.6 immediately starts
> >> triggering DKIM hits.
> >>
> >> Is there a backport of?RaciallyCharged, Esp and ExtractText (although I
> >> don't really use that anymore) that's available for v3.4.6, so my
> >> welcomelist entries work in the meantime?
> >>
> >>
> >>
> >>
> >> References:
> >>
> >> [1] http://svn.apache.org/
> >> [2] http://svn.apache.org/repos/asf/spamassassin/trunk
> >> [3] mailto:mysqlstudent@gmail.com
>

>
> Amavisd-new works fine here. Maybe $enable_dkim_verification or something
> is different.
>

It's good to know you're using amavisd. It's very dependent upon the SA
version you're using, though.

It appears both DKIM and DMARC worked until the May 29th version from svn
(1901385).

At some point after that, and even until yesterday's version, DKIM stopped
working. DMARC still passes with SPF, but there are no longer any
occurrences of DKIM.

Nothing changed with amavisd.

$ grep dkim amavisd.conf
$sa_debug = 'info,dkim,DMARC,dmarc';
$enable_dkim_verification = 1; # enable DKIM signatures verification
$enable_dkim_signing = 1; # load DKIM signing code, keys defined by
dkim_key

With the broken versions, DKIM still seems to be evaluated, but no DKIM
rules are triggered.
Jun 26 12:40:08 xavier amavis[752588]: (752588-04) SA dbg: dkim: signatures
provided by the caller, 2 signatures
Jun 26 12:40:08 xavier amavis[752588]: (752588-04) SA dbg: dkim: adsp:
performing lookup on _adsp._domainkey.agoda.com
Jun 26 12:40:08 xavier amavis[752588]: (752588-04) SA dbg: dkim: adsp
result: U/unknown (dns: unknown), author domain 'agoda.com'
Jun 26 12:40:08 xavier amavis[752588]: (752588-04) SA dbg: dkim: VALID
signature by agoda.com, author no-reply@agoda.com, no valid matches
Jun 26 12:40:08 xavier amavis[752588]: (752588-04) SA dbg: dkim: VALID
signature by agoda.com, author no-reply@agoda.com, no valid matches
Jun 26 12:40:08 xavier amavis[752588]: (752588-04) SA dbg: dkim: author
no-reply@agoda.com, not in any dkim welcomelist
Jun 26 12:40:09 xavier amavis[752588]: (752588-04) SA dbg: DMARC: result:
pass, disposition: none, dkim: fail, spf: pass (spf: pass, spf_helo: fail)

Here's an email from the same sender once the May 29th version was
installed. This passed both DKIM_VALID_AU and DMARC_PASS
Jun 26 12:50:42 xavier amavis[759439]: (759439-03) SA dbg: dkim: signatures
provided by the caller, 2 signatures
Jun 26 12:50:42 xavier amavis[759439]: (759439-03) SA dbg: dkim: VALID
DKIM, i=no-reply@agoda.com, d=agoda.com, s=keyx, a=rsa-sha1,
c=relaxed/relaxed, key_bits=2048, pass,matches author domain
Jun 26 12:50:42 xavier amavis[759439]: (759439-03) SA dbg: dkim: VALID DK,
i=no-reply@agoda.com, d=agoda.com, s=keyx, a=rsa-sha1, c=nofws,
key_bits=2048, pass, matches author domain
Jun 26 12:50:42 xavier amavis[759439]: (759439-03) SA dbg: dkim: signature
verification result: PASS
Jun 26 12:50:42 xavier amavis[759439]: (759439-03) SA dbg: dkim: adsp not
retrieved, author domain signature is valid
Jun 26 12:50:42 xavier amavis[759439]: (759439-03) SA dbg: dkim: adsp
result: - (valid a. d. signature), author domain 'agoda.com'
Jun 26 12:50:42 xavier amavis[759439]: (759439-03) SA dbg: dkim: VALID
signature by agoda.com, author no-reply@agoda.com, no valid matches
Jun 26 12:50:42 xavier amavis[759439]: (759439-03) SA dbg: dkim: VALID
signature by agoda.com, author no-reply@agoda.com, no valid matches
Jun 26 12:50:42 xavier amavis[759439]: (759439-03) SA dbg: dkim: author
no-reply@agoda.com, not in any dkim welcomelist
Jun 26 12:50:42 xavier amavis[759439]: (759439-03) SA dbg: DMARC: result:
pass, disposition: none, dkim: fail, spf: pass (spf: pass, spf_helo: fail)

I see the version of DMARC.pm is completely different from May 29th to
today. Should I try using the DMARC.pm from this month with the SA from
last month?

On Sun, Jun 26, 2022 at 12:57:32PM -0400, Alex wrote:
>
>
> Amavisd-new works fine here. Maybe $enable_dkim_verification or something
> is different.
>
>
> It's good to know you're using amavisd. It's very dependent upon the SA version
> you're using, though.
>
> It appears both DKIM and DMARC worked until the May 29th version from svn
> (1901385).?
>
> At some point after that, and even until yesterday's version, DKIM stopped
> working. DMARC still passes with SPF, but there are no longer any occurrences
> of DKIM.

I think Giovannis changes don't work when amavisd is passing $suppl_attrib:

https://svn.apache.org/viewvc?view=revision&revision=1901719

Sub _check_signature() isn't called at all in that case and things like tags
are not set. I'll leave it for Giovanni to fix..

Excuse my ignorance, but what is the difference between enable dkim in
amavisd-new and having it set to 0 letting spamassassin just do its thing
with loadmodule dkim?


On Mon, Jun 27, 2022 at 4:26 AM Henrik K <hege@hege.li> wrote:

> On Sun, Jun 26, 2022 at 12:57:32PM -0400, Alex wrote:
> >
> >
> > Amavisd-new works fine here. Maybe $enable_dkim_verification or
> something
> > is different.
> >
> >
> > It's good to know you're using amavisd. It's very dependent upon the SA
> version
> > you're using, though.
> >
> > It appears both DKIM and DMARC worked until the May 29th version from svn
> > (1901385).
> >
> > At some point after that, and even until yesterday's version, DKIM
> stopped
> > working. DMARC still passes with SPF, but there are no longer any
> occurrences
> > of DKIM.
>
> I think Giovannis changes don't work when amavisd is passing $suppl_attrib:
>
> https://svn.apache.org/viewvc?view=revision&revision=1901719
>
> Sub _check_signature() isn't called at all in that case and things like
> tags
> are not set. I'll leave it for Giovanni to fix..
>
>

On 6/26/22 20:26, Henrik K wrote:
> On Sun, Jun 26, 2022 at 12:57:32PM -0400, Alex wrote:
>>
>>
>> Amavisd-new works fine here. Maybe $enable_dkim_verification or something
>> is different.
>>
>>
>> It's good to know you're using amavisd. It's very dependent upon the SA version
>> you're using, though.
>>
>> It appears both DKIM and DMARC worked until the May 29th version from svn
>> (1901385). 
>>
>> At some point after that, and even until yesterday's version, DKIM stopped
>> working. DMARC still passes with SPF, but there are no longer any occurrences
>> of DKIM.
>
> I think Giovannis changes don't work when amavisd is passing $suppl_attrib:
>
> https://svn.apache.org/viewvc?view=revision&revision=1901719
>
> Sub _check_signature() isn't called at all in that case and things like tags
> are not set. I'll leave it for Giovanni to fix..
>
thanks for the hint, I've just committed a fix.
Giovanni

Hi,

>> At some point after that, and even until yesterday's version, DKIM
> stopped
> >> working. DMARC still passes with SPF, but there are no longer any
> occurrences
> >> of DKIM.
> >
> > I think Giovannis changes don't work when amavisd is passing
> $suppl_attrib:
> >
> > https://svn.apache.org/viewvc?view=revision&revision=1901719
> >
> > Sub _check_signature() isn't called at all in that case and things like
> tags
> > are not set. I'll leave it for Giovanni to fix..
> >
> thanks for the hint, I've just committed a fix.
>

That looks to have fixed it, thanks. Whew. That was very tricky. Great work.