Mailing List Archive

non-ascii SPF policy
I just committed a test to the test suite for non-ascii-policy. Pyspf
flunks this test, so I am running the expected result by the community
in the hopes of discovering that pyspf is right after all. :-)

The test:
non-ascii-policy:
spec: 3.1.1/1
helo: hosed
host: 1.2.3.4
mailfrom: "foobar@hosed.example.com"
result: permerror
zonedata:
hosed.example.com:
- SPF: "v=spf1 a:\xEF\xBB\xBFgarbage.example.net -all"

The spec states that SPF policies are USASCII. What should an
implementation do when a policy is not, in fact, USASCII? My
reading is that this is a syntax error and the result is PermError.

This situation came up in production. The domain name was preceded
by a Unicode BOM coded as utf-8 (as in the test case). The
domain really existed, utf-8 coded BOM and all. The test case doesn't
reproduce this aspect (domain actually existing) as I wasn't immediately sure
how to code this in YAML. The important thing is that non-ascii
bytes in the SPF policy are a syntax error.

For speculation:

While I'm pretty sure rfc4408 says non-ascii domains are a syntax error,
a related question is whether spfv3 should continue that policy. Since
DNS allows any sequence of 8-bit bytes for label, and there are
implementations by a large software vendor that can store labels as
unicode preceded by BOM and encoded as utf-8, perhaps SPF policies
should allow arbitrary bytes for domain-spec. (The macro mechanism
already provides for quoting special chars.) Alternatively, macro-literal
could be expanded to include %7f-ff so that non-ascii chars within a
domain-spec could be quoted.

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.


-------------------------------------------
Sender Policy Framework: http://www.openspf.org [http://www.openspf.org]
Modify Your Subscription: http://www.listbox.com/member/ [http://www.listbox.com/member/]

Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/1311532-17d8a1ba
Modify Your Subscription: https://www.listbox.com/member/?member_id=1311532&id_secret=1311532-f2ea6ed9
Unsubscribe Now: https://www.listbox.com/unsubscribe/?member_id=1311532&id_secret=1311532-bdbb122a&post_id=20111005165041:AE030642-EF93-11E0-A9B1-BB3AC529DFB4
Powered by Listbox: http://www.listbox.com
Re: non-ascii SPF policy [ In reply to ]
On Wed, 5 Oct 2011, Stuart D. Gathman wrote:

> how to code this in YAML. The important thing is that non-ascii bytes in the
> SPF policy are a syntax error.

Hmm. What about USACII control chars? These will trigger other syntax errors
if used anywhere outside domain-spec. Macro-literal cannot represent
control chars, but they could be included verbatim and the policy
still be "usascii". For instance, what should be the result of this
policy (using YAML 8-bit string escapes):

SPF: "v=spf1 a:\x01ctrl.example.com -all"

In this case, domain-spec is defined to consist of (in macro-literal) "visible
chars except %", and so the result is again PermError (whether
or not the domain actually exists). I will add another test case when
we all agree on this result.

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.


-------------------------------------------
Sender Policy Framework: http://www.openspf.org [http://www.openspf.org]
Modify Your Subscription: http://www.listbox.com/member/ [http://www.listbox.com/member/]

Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/1311532-17d8a1ba
Modify Your Subscription: https://www.listbox.com/member/?member_id=1311532&id_secret=1311532-f2ea6ed9
Unsubscribe Now: https://www.listbox.com/unsubscribe/?member_id=1311532&id_secret=1311532-bdbb122a&post_id=20111005172958:2BDB141A-EF99-11E0-9BC4-A94352CCF860
Powered by Listbox: http://www.listbox.com
Re: non-ascii SPF policy [ In reply to ]
On 5 October 2011 22:50, Stuart D. Gathman wrote:

>    mailfrom: "foobar@hosed.example.com"
>    result: permerror
> zonedata:
>  hosed.example.com:
>    - SPF:  "v=spf1 a:\xEF\xBB\xBFgarbage.example.net -all"

> The spec states that SPF policies are USASCII.  What should an
> implementation do when a policy is not, in fact, USASCII?  My
> reading is that this is a syntax error and the result is PermError.

+1 IDN (internationalized domain names) would be encoded with
the IDNA2008 rules (punycode introduced by xn--) in SPF policies.

Caveat: Local parts in the first or soon second set of EAI RFCs
can be UTF-8, in that case you should treat the "binary" UTF-8
output of the local part macro "as is". But that doesn't affect
what you find directly (before macro expansion) in SPF policies.

And you won't care about "raw UTF-8 octets" in obscure EAI local
parts before you can handle UTF8SMTP and IDNA forward, sideward,
and backward; always punycoding U-labels into A-labels, where "U"
stands for IDNA2008 UTF-8, and "A" for ASCII compatible xn--LDH.

> The important thing is that non-ascii bytes in the SPF policy
> are a syntax error.

Yes, at least not directly. And indirectly only as U-label (=>
A-label) or as EAI local part (=> raw UTF-8 treated as a binary
DNS label, still with the known length limit for all DNS labels)

> related question is whether spfv3 should continue that policy.

Sure, folks will manage to punycode their U-labels. For normal
SPF business (= host names) the corresponding A-labels will do.

If receivers start to support UTF8SMTP they'd also need an SPF
implementation supporting punycode "somewhere". Ideally that's
done outside of check_host():

As the name says check_host() is for host names, IOW it expects
LDH including A-labels. Converting U-labels to A-labels can be
done before check_host(), and any invalid cruft is a PermError.

The only difference of an EAI check_host() would be to permit
EAI local parts for the purposes of the local part macro in an
SPF policy.

> DNS allows any sequence of 8-bit bytes for label, and there are
> implementations by a large software vendor that can store labels
> as unicode preceded by BOM and encoded as utf-8, perhaps SPF
> policies should allow arbitrary bytes for domain-spec.

NAK, "binary labels" are binary labels, using octets that happen
to be a BOM if interpreted as UTF-8 does not change this. For
starters this would be a syntax error even in UTF8SMTP and EAI.
It is no valid IDNA2008 host name (A-label), and it is also no
valid U-label (never used directly in DNS -- the "A" in IDNA is
for IDN in Applications, there is no "IDNB" with BOMs in DNS ;-)

-Frank


-------------------------------------------
Sender Policy Framework: http://www.openspf.org [http://www.openspf.org]
Modify Your Subscription: http://www.listbox.com/member/ [http://www.listbox.com/member/]

Archives: https://www.listbox.com/member/archive/735/=now
RSS Feed: https://www.listbox.com/member/archive/rss/735/1311532-17d8a1ba
Modify Your Subscription: https://www.listbox.com/member/?member_id=1311532&id_secret=1311532-f2ea6ed9
Unsubscribe Now: https://www.listbox.com/unsubscribe/?member_id=1311532&id_secret=1311532-bdbb122a&post_id=20111005224116:A829D8A0-EFC4-11E0-9274-FDCC0922F49A
Powered by Listbox: http://www.listbox.com