Mailing List Archive

looks like CAPITALS-Bug in library
Hi,

after a lot of digging we think we found a bug in the python spf library..

I use postfix-policyd-spf-python , now in version 1.3.1; Jun 4, 2014
on debian wheezy (paket from wheezy-backports)
(version 1.0.1 (wheezy) showed the same behaviour)


I noticed that one specific sender-domain got rejected because
of "mailfrom not authenticated" - but the test passes on the website
http://www.openspf.org/Why?s=mfrom;id=...

strange.. It is hard to argument to the customer why his mails got rejected
when even the reference says everything looks ok ..

The manual check on the mailserver failed also:

python -m spf -v 192.168.0.10 someaddress@example.org example.org


But - a check on a different server showed a "pass" - only difference
were the
used nameservers (resolvers)

I then changed the nameservers in the resolv.conf and voila - the
check passes!

So the behaviour of the upstream DNS-servers was different - we found out
it has most probably something to do with this article from the isc:

https://deepthought.isc.org/article/AA-01113/0/Case-Insensitive-Response-Compression-May-Cause-Problems-With-Mixed-Case-Data-and-Non-Conforming-Clients.html

The DNS-server running bind 9.9.5 changed the capitalization of the
answer which made
the python library call fail.

Since bind 9.10 there is the option no-case-compress , which leads to
the same answer as the question
regarding the capitals and so the mail passes correctly.

Example:

Step one:

nslookup -q=txt example.org

Non-authoritative answer:
example.org text = "v=spf1 a mx:EXAMPLE.org ..."

So we found an spf-record.. now let's see what the mx-entry looks like:

dig EXAMPLE.org mx;; ANSWER SECTION:
example.org. 3 IN MX 10 mail.example.org.


(answer from a bind 9.9.5 - Server)

and:

;; ANSWER SECTION:
EXAMPLE.de. 60 IN MX 10 mail.EXAMPLE.org.

(answer from a bind 9.10 - server. See the Capitals?)

This appears also to be the difference for the a-record:

dig @<bind-9.9.5-nameserver> mail.EXAMPLE.org a

;; ANSWER SECTION:
mail.example.org. 60 IN A 192.168.0.10

..or...

dig @<bind-9.10-nameserver> mail.EXAMPLE.org a

;; ANSWER SECTION:
mail.EXAMPLE.de. 60 IN A 192.168.0.10



In the first variant policyd-spf will fail , the second one passes. This is
not only a bug in DNS (which has been fixed), but also a bug in the
behaviour of the policyd as dns-client; say in the spf-library. (see rfc
1034 for this)

Too bad that the option no-case-compress just does not exist
in bind 9.9.5 ..

best greetings,

Gunther




-------------------------------------------
Sender Policy Framework: http://www.openspf.net [http://www.openspf.net]
Modify Your Subscription: http://www.listbox.com/member/ [http://www.listbox.com/member/]

Archives: https://www.listbox.com/member/archive/1007/=now
RSS Feed: https://www.listbox.com/member/archive/rss/1007/1311533-9e42a648
Modify Your Subscription: https://www.listbox.com/member/?member_id=1311533&id_secret=1311533-d322f1f1
Unsubscribe Now: https://www.listbox.com/unsubscribe/?member_id=1311533&id_secret=1311533-d59c80a0&post_id=20160310105138:F78C3302-E6D7-11E5-AD88-C8E6EF10038B
Powered by Listbox: http://www.listbox.com
Re: looks like CAPITALS-Bug in library [ In reply to ]
can i suggest providing the real domain names involved so anyone can replicate/verify or possibly find alternate causation
im sure some can easily test against a range of versions of bind in different modes

At 15:51 10/03/2016 Thursday, Gunther Nitzsche wrote:
> Hi,
>
> after a lot of digging we think we found a bug in the python spf library..
>
> I use postfix-policyd-spf-python , now in version 1.3.1; Jun 4, 2014
> on debian wheezy (paket from wheezy-backports)
> (version 1.0.1 (wheezy) showed the same behaviour)
>
> I noticed that one specific sender-domain got rejected because
> of "mailfrom not authenticated" - but the test passes on the website
> http://www.openspf.org/Why?s=mfrom;id=...
>
> strange.. It is hard to argument to the customer why his mails got rejected
> when even the reference says everything looks ok ..
>
> The manual check on the mailserver failed also:
>
> python -m spf -v 192.168.0.10 someaddress@example.org example.org
>
> But - a check on a different server showed a "pass" - only difference
> were the
> used nameservers (resolvers)
>
> I then changed the nameservers in the resolv.conf and voila - the
> check passes!
>
> So the behaviour of the upstream DNS-servers was different - we found out
> it has most probably something to do with this article from the isc:
>
> https://deepthought.isc.org/article/AA-01113/0/Case-Insensitive-Response-Compression-May-Cause-Problems-With-Mixed-Case-Data-and-Non-Conforming-Clients.html
>
> The DNS-server running bind 9.9.5 changed the capitalization of the
> answer which made
> the python library call fail.
>
> Since bind 9.10 there is the option no-case-compress , which leads to
> the same answer as the question
> regarding the capitals and so the mail passes correctly.
>
> Example:
>
> Step one:
>
> nslookup -q=txt example.org
>
> Non-authoritative answer:
> example.org text = "v=spf1 a mx:EXAMPLE.org ..."
>
> So we found an spf-record.. now let's see what the mx-entry looks like:
>
> dig EXAMPLE.org mx;; ANSWER SECTION:
> example.org. 3 IN MX 10 mail.example.org.
>
> (answer from a bind 9.9.5 - Server)
>
> and:
>
> ;; ANSWER SECTION:
> EXAMPLE.de. 60 IN MX 10 mail.EXAMPLE.org.
>
> (answer from a bind 9.10 - server. See the Capitals?)
>
> This appears also to be the difference for the a-record:
>
> dig @<bind-9.9.5-nameserver> mail.EXAMPLE.org a
>
> ;; ANSWER SECTION:
> mail.example.org. 60 IN A 192.168.0.10
>
> ..or...
>
> dig @<bind-9.10-nameserver> mail.EXAMPLE.org a
>
> ;; ANSWER SECTION:
> mail.EXAMPLE.de. 60 IN A 192.168.0.10
>
> In the first variant policyd-spf will fail , the second one passes. This is
> not only a bug in DNS (which has been fixed), but also a bug in the
> behaviour of the policyd as dns-client; say in the spf-library. (see rfc
> 1034 for this)
>
> Too bad that the option no-case-compress just does not exist
> in bind 9.9.5 ..
>
> best greetings,
>
> Gunther
>


-------------------------------------------
Sender Policy Framework: http://www.openspf.net [http://www.openspf.net]
Modify Your Subscription: http://www.listbox.com/member/ [http://www.listbox.com/member/]

Archives: https://www.listbox.com/member/archive/1007/=now
RSS Feed: https://www.listbox.com/member/archive/rss/1007/1311533-9e42a648
Modify Your Subscription: https://www.listbox.com/member/?member_id=1311533&id_secret=1311533-d322f1f1
Unsubscribe Now: https://www.listbox.com/unsubscribe/?member_id=1311533&id_secret=1311533-d59c80a0&post_id=20160310170806:8E7B83A6-E70C-11E5-8030-8442BA9E19FE
Powered by Listbox: http://www.listbox.com
Re: looks like CAPITALS-Bug in library [ In reply to ]
On Thu, 10 Mar 2016, alan wrote:

> can i suggest providing the real domain names involved so anyone can
> replicate/verify or possibly find alternate causation im sure some can
> easily test against a range of versions of bind in different modes

+1

Also, the pyspf version used. For instance, 2.0.11 adds a workaround
for a DNS caching server returning random case for lookups. (Maybe
the developer wanted to make sure everyone was properly ignoring
case - I dunno.)

--
Stuart D. Gathman <stuart@gathman.org>
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.



-------------------------------------------
Sender Policy Framework: http://www.openspf.net [http://www.openspf.net]
Modify Your Subscription: http://www.listbox.com/member/ [http://www.listbox.com/member/]

Archives: https://www.listbox.com/member/archive/1007/=now
RSS Feed: https://www.listbox.com/member/archive/rss/1007/1311533-9e42a648
Modify Your Subscription: https://www.listbox.com/member/?member_id=1311533&id_secret=1311533-d322f1f1
Unsubscribe Now: https://www.listbox.com/unsubscribe/?member_id=1311533&id_secret=1311533-d59c80a0&post_id=20160310214819:B49AD16E-E733-11E5-B9D0-D16DF010038B
Powered by Listbox: http://www.listbox.com
Re: looks like CAPITALS-Bug in library [ In reply to ]
On March 10, 2016 9:48:07 PM EST, "Stuart D. Gathman" <stuart@gathman.org> wrote:
>On Thu, 10 Mar 2016, alan wrote:
>
>> can i suggest providing the real domain names involved so anyone can
>> replicate/verify or possibly find alternate causation im sure some
>can
>> easily test against a range of versions of bind in different modes
>
>+1
>
>Also, the pyspf version used. For instance, 2.0.11 adds a workaround
>for a DNS caching server returning random case for lookups. (Maybe
>the developer wanted to make sure everyone was properly ignoring
>case - I dunno.)

The repository he's installing from has 2.0.9.

Scott K


-------------------------------------------
Sender Policy Framework: http://www.openspf.net [http://www.openspf.net]
Modify Your Subscription: http://www.listbox.com/member/ [http://www.listbox.com/member/]

Archives: https://www.listbox.com/member/archive/1007/=now
RSS Feed: https://www.listbox.com/member/archive/rss/1007/1311533-9e42a648
Modify Your Subscription: https://www.listbox.com/member/?member_id=1311533&id_secret=1311533-d322f1f1
Unsubscribe Now: https://www.listbox.com/unsubscribe/?member_id=1311533&id_secret=1311533-d59c80a0&post_id=20160311020551:AE387ED8-E757-11E5-B9BD-C73F89C13CD1
Powered by Listbox: http://www.listbox.com
Re: looks like CAPITALS-Bug in library [ In reply to ]
Hi,

thanks for getting back on this issue.

On 03/11/2016 08:05 AM, Scott Kitterman wrote:
>
> On March 10, 2016 9:48:07 PM EST, "Stuart D. Gathman" <stuart@gathman.org> wrote:
>> On Thu, 10 Mar 2016, alan wrote:
>>
>>> can i suggest providing the real domain names involved so anyone can
>>> replicate/verify or possibly find alternate causation im sure some
>> can
>>> easily test against a range of versions of bind in different modes


Actually the domain in question was "wwk.de" ; sending IP was
212.144.243.129 - leading to:

python -m spf -v 212.144.243.129 someaddress@wwk.de wwk.de
top: wwk.de "v=spf1 a mx:WWK.de include:spf.mandrillapp.com
include:_spf.retarus.com -all"
include: spf.mandrillapp.com "v=spf1 ip4:198.2.128.0/24
ip4:198.2.132.0/22 ip4:198.2.136.0/23 ip4:198.2.186.0/23
ip4:205.201.131.128/25 ip4:205.201.134.128/25 ip4:205.201.136.0/23
ip4:205.201.139.0/24 ip4:198.2.180.0/24 ip4:198.2.179.0/24
ip4:198.2.178.0/24 ip4:198.2.177.0/24 ~all"
include: _spf.retarus.com "v=spf1 ip4:94.199.88.0/22 ip4:82.135.18.0/23
ip4:62.245.148.0/26 ip4:62.134.45.32/28 ip4:217.111.120.0/27
ip4:94.199.92.0/23 ip4:185.93.140.0/24 ~all"
(('fail', 550, 'SPF fail - not authorized'), '-all')

when asking a resolver using bind 9.95

The behaviour was like described.

pyspf used is :
VERSION
2.0.9: Apr 29, 2014

kind regards,

Gunther



>> +1
>>
>> Also, the pyspf version used. For instance, 2.0.11 adds a workaround
>> for a DNS caching server returning random case for lookups. (Maybe
>> the developer wanted to make sure everyone was properly ignoring
>> case - I dunno.)
> The repository he's installing from has 2.0.9.
>
> Scott K
>
>




-------------------------------------------
Sender Policy Framework: http://www.openspf.net [http://www.openspf.net]
Modify Your Subscription: http://www.listbox.com/member/ [http://www.listbox.com/member/]

Archives: https://www.listbox.com/member/archive/1007/=now
RSS Feed: https://www.listbox.com/member/archive/rss/1007/1311533-9e42a648
Modify Your Subscription: https://www.listbox.com/member/?member_id=1311533&id_secret=1311533-d322f1f1
Unsubscribe Now: https://www.listbox.com/unsubscribe/?member_id=1311533&id_secret=1311533-d59c80a0&post_id=20160311033147:AFA7964E-E763-11E5-9704-94A5F010038B
Powered by Listbox: http://www.listbox.com
Re: looks like CAPITALS-Bug in library [ In reply to ]
Hi,

>The repository he's installing from has 2.0.9.
>
>Scott K

Scott helped me out - Installing the fresh 2.0.11 .deb Module he offered
(including upgrading the python3-ipaddr and python3-dns) got things
running.


# /usr/bin/python3 -m spf -v 212.144.243.129 someaddress@wwk.de wwk.de
top: wwk.de "v=spf1 a mx:WWK.de include:spf.mandrillapp.com
include:_spf.retarus.com -all"
('pass', 250, 'sender SPF authorized') mx:WWK.de

Looks like this puzzle is solved .. :-) We will upgrade the resolver
software anyhow..:-/

Thank you again,

best greetings,

Gunther



-------------------------------------------
Sender Policy Framework: http://www.openspf.net [http://www.openspf.net]
Modify Your Subscription: http://www.listbox.com/member/ [http://www.listbox.com/member/]

Archives: https://www.listbox.com/member/archive/1007/=now
RSS Feed: https://www.listbox.com/member/archive/rss/1007/1311533-9e42a648
Modify Your Subscription: https://www.listbox.com/member/?member_id=1311533&id_secret=1311533-d322f1f1
Unsubscribe Now: https://www.listbox.com/unsubscribe/?member_id=1311533&id_secret=1311533-d59c80a0&post_id=20160311091800:0CCD48CA-E794-11E5-8B41-7911F010038B
Powered by Listbox: http://www.listbox.com
Re: looks like CAPITALS-Bug in library [ In reply to ]
On Friday, March 11, 2016 03:17:50 PM Gunther Nitzsche wrote:
> Hi,
>
> >The repository he's installing from has 2.0.9.
> >
> >Scott K
>
> Scott helped me out - Installing the fresh 2.0.11 .deb Module he offered
> (including upgrading the python3-ipaddr and python3-dns) got things
> running.
>
>
> # /usr/bin/python3 -m spf -v 212.144.243.129 someaddress@wwk.de wwk.de
> top: wwk.de "v=spf1 a mx:WWK.de include:spf.mandrillapp.com
> include:_spf.retarus.com -all"
> ('pass', 250, 'sender SPF authorized') mx:WWK.de
>
> Looks like this puzzle is solved .. :-) We will upgrade the resolver
> software anyhow..:-/

For completeness:

I've just uploaded pyspf 2.0.11 to the Debian wheezy-backports suite so it
will shortly be available for all users of that release.

Scott K


-------------------------------------------
Sender Policy Framework: http://www.openspf.net [http://www.openspf.net]
Modify Your Subscription: http://www.listbox.com/member/ [http://www.listbox.com/member/]

Archives: https://www.listbox.com/member/archive/1007/=now
RSS Feed: https://www.listbox.com/member/archive/rss/1007/1311533-9e42a648
Modify Your Subscription: https://www.listbox.com/member/?member_id=1311533&id_secret=1311533-d322f1f1
Unsubscribe Now: https://www.listbox.com/unsubscribe/?member_id=1311533&id_secret=1311533-d59c80a0&post_id=20160311092716:58372870-E795-11E5-B2D0-AB22B94FA8A8
Powered by Listbox: http://www.listbox.com