Mailing List Archive

On Sunday 02 December 2007 21:43, Frank Ellermann wrote:

> -----------------------------------------------------------------
> The last pending erratum: What do real implementations if the
> <target-name> has a form unsuited for DNS queries, e.g. adjacent
> dots ? Just ignore the <target-name> as "no match" ?
>
> For the test suite: With "v=spf1 exists:%{l}.%{d}" and mail from
> "ugly..dots"@valid.domain.test what are implementations supposed
> to do ? A dot (like any other octet) is permitted in a label, of
> course that's no host name, and it might be tricky to construct
> a DNS query, but maybe "ugly\.\.dots.valid.domain.test" works.
>
> It's not as esoteric as you might think, for UTF8SMTP local parts
> can contain "raw" UTF-8 octets.

pyspf will just no-match for adjacent dots.

Scott K

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/1007/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/1007/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311533&id_secret=71437655-42b33d
Powered by Listbox: http://www.listbox.com

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Frank Ellermann wrote:
> with 2821bis in Last Call I removed "<target-name> is a valid TLD"
> as potential erratum, if 2821bis permits it we'll follow suite.
>
> Syntactically SPF policies can't talk directly about this case,
> a <domain-spec> ends with a dot-separated <toplabel>, plus the
> optional trailing dot (that's certainly no 2821bis construct ;-)
>
> IOW we can't have "v=spf1 a:museum" or similar, this won't kill
> e-mail. I think that deserves two test cases in the test suite:
>
> 1 - "v=spf1 a:museum" is a PERMERROR (missing dot)

Not sure whether this is already being implicitly tested by some other
test in the test suite. If not, this would do it:

tests:
a-only-top-label:
description: >-
Domain-spec may not consist of only a top-label without a leading dot.
spec: 8.1/2
helo: mail.example.com
host: 1.2.3.4
mailfrom: foo@e5a.example.com
result: permerror
zonedata:
e5a.example.com:
- SPF: v=spf1 a:museum

... probably to be inserted in the "A mechanism syntax" scenario.

> 2 - "v=spf1 -a:%{h} all" for h=museum or similar should be PASS,
> if the address doesn't exist or at least doesn't match. I
> forgot how the test suite arranges this. Maybe these tests
> already exist.

This one is already covered by the "require-valid-helo" test.

> We could note that future SPF versions (4408bis or otherwise)
> need a syntax where "v=spf1 a:museum" is no syntax error, but
> IMO that's no 4408 erratum, it wasn't predictable that 2821bis
> changes the rules.

Pardon my ignorance, but how does 2821bis change the rules with regard to
this?

> -----------------------------------------------------------------
> The last pending erratum: What do real implementations if the
> <target-name> has a form unsuited for DNS queries, e.g. adjacent
> dots ? Just ignore the <target-name> as "no match" ?

- From a design theoretical PoV, that should be a PermError because it won't
go away on retrying. However, since RFC 4408 does not specify behavior in
this case, there's little point in retroactively codifying observed
behavior in the spec or in an erratum.

As for real implementation behavior, Mail::SPF currently treats
"a:foo..bar" as a simple mismatch.

> For the test suite: With "v=spf1 exists:%{l}.%{d}" and mail from
> "ugly..dots"@valid.domain.test what are implementations supposed
> to do ? A dot (like any other octet) is permitted in a label, of
> course that's no host name, and it might be tricky to construct
> a DNS query, but maybe "ugly\.\.dots.valid.domain.test" works.

Following RFC 4408, behavior would have to be the same as for a literal
"exists:ugly..dots.valid.domain.test".

I have thought about defining (in a new specification -- SPFv3, 4408bis,
whatever) macro expansion to treat any expanded macro as an atomic and
opaque value when constructing a <target-name>, which would enable
"ugly..dots" to be considered as a domain label of its own in "%{l}.%{d}".
That, however, would break "%{ir}._spf.%{d}" and similar cases.

> It's not as esoteric as you might think, for UTF8SMTP local parts
> can contain "raw" UTF-8 octets.

We can support that in our next spec easily simply by allowing 8-bit bytes
in SPF records and defining that localparts have to be encoded as UTF-8
before %{l} expansion (handling of other non-ASCII input data can be
defined analogously).

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFHVCXdwL7PKlBZWjsRAvsCAJsEVsFmaNCOgtCeaL0HRecf88OLTACfa1qO
7IJfuKGvLiSsEnmyXdXvr4c=
=XQgx
-----END PGP SIGNATURE-----

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/1007/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/1007/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311533&id_secret=71464015-0f84a5
Powered by Listbox: http://www.listbox.com

Julian Mehnle wrote:

> how does 2821bis change the rules with regard to this?

Back in 2004 IIRC the USEFOR WG had an issue about legacy
(UUCP) names like "demon" vs. FQDNs like "example.com",
and RFC 2821 has a rule that a <domain> contains at least
one dot.

Later (February 2004) RFC 3696 confirmed this, and before
I sent a mail to the author of RFC 2821 and asked if the
"1" in domain = subdomain 1*( "." subdomain ) is a typo,
or intentional against "bare" (unqualified) host names.

He wrote it was intentional - at that time I had no idea
that "author of RFC 2821" was one of the Internet gods ;-)

I-D.klensin-2821bis removes the "dot required" syntax, in
essence it strips the "1", allowing TLDs, and with MUSTard
against "bare" hosts in prose.

RFC 4408 (and the approved Netnews RFC) ended up with the
"old" 2821 and 3696 one-dot-only rules. That's no serious
problem, let alone an erratum. If say TLD "cat" decides
to start a news server they'll have to fight it out with
any legacy (UUCP) "cat", and nobody is in trouble if a:cat
won't fly in an SPF policy.

>> -----------------------------------------------------------------
>> The last pending erratum: What do real implementations if the
>> <target-name> has a form unsuited for DNS queries, e.g. adjacent
>> dots ? Just ignore the <target-name> as "no match" ?

> - From a design theoretical PoV, that should be a PermError
> because it won't go away on retrying.

Partially I agree: A TempError certainly won't cut it. You could
however treat it like a (literal) "invalid", a reserved pseudo-TLD
without IP or MX, nowhere, never (unlike "localhost" or "test").

> since RFC 4408 does not specify behavior in this case, there's
> little point in retroactively codifying observed behavior in the
> spec or in an erratum.

Makes sense. But we normally _try_ that PermError is predictable,
mainly used for syntax errors. If Scott picks "invalid" (ignore),
and you pick PermError, it's hard to find out what's wrong for an
affected user.

> As for real implementation behavior, Mail::SPF currently treats
> "a:foo..bar" as a simple mismatch.

Wait a moment, I was talking about <target-name> foo..bar, are you
talking about a <directive> a:foo..bar ? Adjacent dots before the
macro expansion step directly written in a policy are a PermError,
aren't they ?

>> For the test suite: With "v=spf1 exists:%{l}.%{d}" and mail from
>> "ugly..dots"@valid.domain.test what are implementations supposed
>> to do ? A dot (like any other octet) is permitted in a label, of
>> course that's no host name, and it might be tricky to construct
>> a DNS query, but maybe "ugly\.\.dots.valid.domain.test" works.

> Following RFC 4408, behavior would have to be the same as for a
> literal "exists:ugly..dots.valid.domain.test".

Interesting, you (kind of) ignored the quoted-string "ugly..dots".
You could also treat it as "ugly\.\.dots", AFAIK semantically the
same for RFC 2822: Unnecessary quoted-pairs \. in this context,
but for DNS queries <liaden> necessity exists </liaden>.

> I have thought about defining (in a new specification -- SPFv3,
> 4408bis, whatever) macro expansion to treat any expanded macro
> as an atomic and opaque value when constructing a <target-name>,
> which would enable "ugly..dots" to be considered as a domain
> label of its own in "%{l}.%{d}".

Yes, I'm in the same position wrt EAI, I kind of promised that I
submit an "UTF8SMTP considerations for SPF" draft. After I lost
the Last Call battle against the weird "experimental MIME update"
in EAI will be soon enough... <eg>

> That, however, would break "%{ir}._spf.%{d}" and similar cases.

Not necessarily, you could transform dots within a <quoted-string>
local part to <quoted-pair>, and leave a sound <dot-atom-text> as
is. Of course you'd also keep <quoted-pair> \. in <quoted-string>,
we don't want \\\. in this case.

> We can support that in our next spec easily simply by allowing
> 8-bit bytes in SPF records and defining that localparts have to
> be encoded as UTF-8

It should work directly with RFC 4408, nobody needs to spell out
local parts, ASCII or otherwise, in a policy, %{l} is good enough.

JFTR, RFC 4408 also doesn't allow to spell out "embedded dots" in
a policy, we only can do VCHAR + SP - "." directly or with macros.

For IDN (internationalized domain names) in a policy we can use a
form with A-labels ("punycode"). For U-labels ("unencoded UTF-8")
in an UTF8SMTP envelope sender address we need the draft mentioned
above, that would also explain some fine print about <alt-address>:

Inconsistent policies for an "I18N address" and the corresponding
(optional) <alt-address> won't work as expected from the POV of a
sender trying this stunt. No SPF issue, only a new caveat.

The DKIM folks are in far worse troubles with the EAI experiment.

Frank

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/1007/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/1007/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311533&id_secret=71591909-f57ae3
Powered by Listbox: http://www.listbox.com

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Frank Ellermann wrote:
> Julian Mehnle wrote:
> > how does 2821bis change the rules with regard to this?
>
> [...]

Thanks for the explanation.

> > > -----------------------------------------------------------------
> > > The last pending erratum: What do real implementations if the
> > > <target-name> has a form unsuited for DNS queries, e.g. adjacent
> > > dots ? Just ignore the <target-name> as "no match" ?
> >
> > - From a design theoretical PoV, that should be a PermError
> > because it won't go away on retrying.
>
> Partially I agree: A TempError certainly won't cut it. You could
> however treat it like a (literal) "invalid", a reserved pseudo-TLD
> without IP or MX, nowhere, never (unlike "localhost" or "test").

RFC 4408 -- as it is -- doesn't forbid this behavior. As I said, I was
speaking from a purely design theoretical PoV.

> [...] But we normally _try_ that PermError is predictable, mainly used
> for syntax errors. If Scott picks "invalid" (ignore), and you pick
> PermError, it's hard to find out what's wrong for an affected user.

Well, yes, PermError being used only for syntax error is an unwritten
axiom of SPFv1, however it is an unwritten one, which (thankfully!)
allows us to extend the PermError semantics in a future SPF revision.
Again, as I said, I was speaking from a purely design theoretical PoV.

> > As for real implementation behavior, Mail::SPF currently treats
> > "a:foo..bar" as a simple mismatch.
>
> Wait a moment, I was talking about <target-name> foo..bar, are you
> talking about a <directive> a:foo..bar ? Adjacent dots before the macro
> expansion step directly written in a policy are a PermError, aren't
> they ?

I was talking about <target-name> = "foo..bar", but it's all the same,
really. "a:foo..bar" is not a syntax error. In any case, according to
RFC 4408, a PermError should not be thrown for that reason.

> > > For the test suite: With "v=spf1 exists:%{l}.%{d}" and mail from
> > > "ugly..dots"@valid.domain.test what are implementations supposed
> > > to do ? A dot (like any other octet) is permitted in a label, of
> > > course that's no host name, and it might be tricky to construct
> > > a DNS query, but maybe "ugly\.\.dots.valid.domain.test" works.
> >
> > Following RFC 4408, behavior would have to be the same as for a
> > literal "exists:ugly..dots.valid.domain.test".
>
> Interesting, you (kind of) ignored the quoted-string "ugly..dots".

I think the quoting of localparts only serves to enable the use of
additional characters that are otherwise reserved. The quotes themselves
are not part of the localpart's value if I understand it correctly.

> > I have thought about defining (in a new specification -- SPFv3,
> > 4408bis, whatever) macro expansion to treat any expanded macro
> > as an atomic and opaque value when constructing a <target-name>,
> > which would enable "ugly..dots" to be considered as a domain
> > label of its own in "%{l}.%{d}".
>
> [...]
>
> > That, however, would break "%{ir}._spf.%{d}" and similar cases.
>
> Not necessarily, you could transform dots within a <quoted-string>
> local part to <quoted-pair>, and leave a sound <dot-atom-text> as
> is. Of course you'd also keep <quoted-pair> \. in <quoted-string>,
> we don't want \\\. in this case.

I think we should stay as far away as possible from EVER introducing
sophisticated (de)quoting/(un)escaping rules into SPF. Stuff like this
has caused generations of broken implementations (see RFC 2047 MIME
encoded words for a prime example).

> JFTR, RFC 4408 also doesn't allow to spell out "embedded dots" in
> a policy, we only can do VCHAR + SP - "." directly or with macros.

I don't think RFC 4408 allows dots-embedded-in-domain-labels even
_implicitly_, through macro expansion or otherwise. "%{h}.foo" with
HELO = "quux..bar" will always get expanded to "quux..bar.foo", which is
likely to be rejected, rather than actually parsed and looked up, by very
most resolver libraries.

> For IDN (internationalized domain names) in a policy we can use a
> form with A-labels ("punycode"). For U-labels ("unencoded UTF-8")
> in an UTF8SMTP envelope sender address we need the draft mentioned
> above, that would also explain some fine print about <alt-address>:
>
> Inconsistent policies for an "I18N address" and the corresponding
> (optional) <alt-address> won't work as expected from the POV of a
> sender trying this stunt. No SPF issue, only a new caveat.

IDNA, in all its unbelievable ugliness, was invented for exactly this
reason: network/transport-layer applications NOT having to care about it.
Thus it's not our problem.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFHVHYzwL7PKlBZWjsRAowrAKDncmIhbRJoBjBNnV0OirXZ4SMu4gCeOcfL
CvPCFtcLsrf3alPeQPc5evY=
=pZtE
-----END PGP SIGNATURE-----

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/1007/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/1007/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311533&id_secret=71616655-2859fd
Powered by Listbox: http://www.listbox.com

Julian Mehnle wrote:

> "a:foo..bar" is not a syntax error.

Ugh, right, it's 2*<macro-literal> "." <toplabel>

If we ever try "SSP" (simple sender policy) let's kill those
annoying macros, their only real purpose outside of theoretical
SES and "per user policy" discussions is trolling for Doug's
"SPF script" flames :-(

We could even do that "by definition", a v=spf1 policy without
macros and with at most one mx mechanism is "by definition" a
simple sender policy.

>>>> For the test suite: With "v=spf1 exists:%{l}.%{d}" and mail from
>>>> "ugly..dots"@valid.domain.test what are implementations supposed
>>>> to do ? A dot (like any other octet) is permitted in a label, of
>>>> course that's no host name, and it might be tricky to construct
>>>> a DNS query, but maybe "ugly\.\.dots.valid.domain.test" works.

>>> Following RFC 4408, behavior would have to be the same as for a
>>> literal "exists:ugly..dots.valid.domain.test".

>> Interesting, you (kind of) ignored the quoted-string "ugly..dots".

> I think the quoting of localparts only serves to enable the use
> of additional characters that are otherwise reserved. The quotes
> themselves are not part of the localpart's value if I understand
> it correctly.

Yes, you have to quote anything that is no <dot-atom-text>. That's
as you said anything containing non-<atext> characters, including
any <quoted-pair>, necessary or otherwise, down to <quoted-pair>
with NO-WS-CTL and worse 2822 obs-cenities. But it also covers any
leading / trailing / adjacent dots, that's also no <dot-atom-text>.

> we should stay as far away as possible from EVER introducing
> sophisticated (de)quoting/(un)escaping rules into SPF. Stuff
> like this has caused generations of broken implementations (see
> RFC 2047 MIME encoded words for a prime example).

We should, but we can't without ruling out certain kinds of local
parts. Or radically killing %{l} as bad idea. EAI even kept the
<quoted-pair> construct in conjunction with UTF-8 characters, one
of the three EAI lunacies I hope to address in the IETF Last Call.

There are more issues with %{l}, local parts are by definition
case sensitive, but LDH-labels in something that could be a host
name are not. Or rather they are, but you can have at most one
spelling of a host name in DNS, and that's supposed to match any
other spelling. Adding this %[l} can of worms to SPF wasn't a
smart move, putting it mildly. What does it really buy outside
of Doug's DDoS drafts ?

> "%{h}.foo" with HELO = "quux..bar" will always get expanded to
> "quux..bar.foo", which is likely to be rejected, rather than
> actually parsed and looked up, by very most resolver libraries.

Maybe a %{l} example with this effect is more compelling, as HELO
it's an SMTP syntax error justifying GiGo results. As local part
implementations could look up quux\.\.bar.foo (two labels) to get
it right. Similar "space me" could be space\ me.foo (two labels).

Or whatever makes the API accept embedded spaces, dots, NO-WS-CTL,
the works. We should have some test cases in this direction, for
"DNS can do" and "SPF doesn't say DON'T" I get a "permitted", no
matter how much sh*t real SPF implementations are willing to eat.

> IDNA, in all its unbelievable ugliness, was invented for exactly
> this reason: network/transport-layer applications NOT having to
> care about it. Thus it's not our problem.

It's a problem for users wishing to use SPF and participate in the
EAI experiment, what their UTF8SMTP MTA sends / gets as MAIL FROM
can use U-labels (UTF-8) in the right hand side.

IOW SPF implementations wishing to support EAI have to know how to
transform U-labels into A-labels, or they need an EAI layer between
UTF8SMTP and SPF with this effect, or they need a DNS resolver API
doing that on the fly.

Frank

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/1007/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/1007/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311533&id_secret=71640151-0827b0
Powered by Listbox: http://www.listbox.com