Mailing List Archive

Test suite v1.0 ?
We have a candidate for test suite 1.0 posted to SVN. Please review. We
are looking for:

1) touches upon all relevant sections of RFC (exhaustive testing is not
practical).

2) correct. Best way to test is to run against your implementation.

One area not covered is joining text segments. For example, I get a *lot*
permerrors for records like "v=spf1 ip4:1.2.3.4a:example.com-all". That
will required another feature in the test driver to check, however.q

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com
Re: Test suite v1.0 ? [ In reply to ]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Stuart D. Gathman wrote:
> We have a candidate for test suite 1.0 posted to SVN. Please review.

I applied some cosmetical clean-ups in [r61] & [r62].

I also deliberately added two test cases in r62 for CNAME support and CNAME
loop detection, and removed them again in r63 after I had tested Mail::
SPF's CNAME handling. These two tests are now in the history of the
Subversion repo, in case we need them in the future. (The same goes for
Mail::SPF's CNAME handling code, which I have removed again, too.)

And finally, I changed occurrances of multiple spec references per test
case into sequences (as opposed to comma-separated scalars) in order to
provide structure. This implies a change in the test-suite schema!


Now on to the real meat...

> We are looking for:
>
> 1) touches upon all relevant sections of RFC (exhaustive testing is not
> practical).
>
> 2) correct. Best way to test is to run against your implementation.
>
> One area not covered is joining text segments. For example, I get a
> *lot* permerrors for records like "v=spf1 ip4:1.2.3.4a:example.com-all".
> That will required another feature in the test driver to check,
> however.

In the Mail::SPF test-suite driver, I am currently skipping the
"cidr6-0-ip4" and "cidr6-ip4" test cases. They assume that IPv4 connec-
tions must never match the "ip6" mechanism, even if the IPv4 connection
comes in through an IPv6 interface. I discussed this with Wayne and Scott
on #spf extensively, and it seems we cannot agree on a common interpreta-
tion of RFC 4408. Here's my rationale:

Nowhere does RFC 4408 say that the "ip6" mechanism must not match IPv4
addresses. 5/9 saying "Even if the SMTP connection is via IPv6, an
IPv4-mapped IPv6 IP address [...] MUST still be considered an IPv4
address" doesn't change that.

Wayne thinks that the IPv4 and IPv6 address spaces are distinct and that
thus this restriction can be extrapolated from the spec, but I disagree.
The IPv4 address space is NOT distinct from the IPv6 one. Rather, it is
embedded in IPv6 address space with the ::FFFF/96 prefix (the ::/96 prefix
has been deprecated a while ago). Further, it would be totally un-DWIM
(do what I mean) if "ip6:::/0" or "ip6:::FFFF:0/96" did not match in
incoming connection from ::FFFF:1.2.3.4. It would even make "ip6:::FFFF:
n.n.n.n" essentially meaningless.

IMO, the idea of "ip6" principally never matching IPv4 addresses runs
contrary to the concept of IPv4-and-IPv6 addressing.


Beyond that, Mail::SPF is now 100% compliant with the test suite. However,
I'm going to sanity check all of the test cases and how Mail::SPF meets
them before I make the Mail::SPF "gold" release. That may take some time,
though, and we may not want to wait for that.

About the joining of text segments, Mail::SPF::Test (the implementation
independent test-suite parser) supports both plan scalars ("v=spf1 -all")
as well as sequences (["v=spf1", " -all"]) for SPF/TXT records, so it
would in fact be able to grok such test cases.

Finally, the test-suite specification[1] is slightly out of date and should
be revised before it is released with the test-suite as "1.0" or
something. Stuart, can you do that?


References:
r61. http://new.openspf.org/source/project?rev=61&view=rev
r62. http://new.openspf.org/source/project?rev=62&view=rev
1. http://new.openspf.org/Test_Suite/Schema

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFFW2EuwL7PKlBZWjsRAt3HAJ9dUle8g1QWE43cenDzHla9thlTegCdFx4d
w+uyYDStropKqLScxHmXwnk=
=swuT
-----END PGP SIGNATURE-----

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?list_id=1007
Re: Test suite v1.0 ? [ In reply to ]
Julian Mehnle wrote:

> I discussed this with Wayne and Scott on #spf extensively, and it
> seems we cannot agree on a common interpretation of RFC 4408.

Some reference implementation passing the test suite would be nice,
it doesn't need to be Mail:SPF, anything that works as a base for
why.html is fine.

> "Even if the SMTP connection is via IPv6, an IPv4-mapped IPv6 IP
> address [...] MUST still be considered an IPv4 address" doesn't
> change that.

It's actually pretty simple, your logic is just inverted at the
wrong place, because you look at it from the default-IPv6 POV.

Of course IPv6-aware implementations MUST understand ip4 mechanisms
and interpret them for the what's it, ::FFFF:ip4 (?)

But the opposite isn't true, and IPv4-only implementations are free
to ignore ip6 mechanisms completely. They never get an IPv6 address
as input, because they are IPv4-only. Therefore you can't expect
that they look at ip6-mechanisms, which happen to talk about IPv4
addresses in a twisted way.

Of course the spec. can't rule out such cases in any consistent way
in syntax. Maybe 4408bis could add that as MUST, if the connecting
IP is an IPv4 (or the IPv6 version of an IPv4) all ip6-mechanisms
have to be ignored. And the IPv4 CIDRs have to be used (translated
to IPv6 if that's how you need this in your code), but not the IPv6
CIDR of a dual CIDR. IPv6 CIDRs, ip6 mechanisms, and AAAA are for
"real" IPv6 IPs, not the "upgraded" IPv4 IPs.

Frank


-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?list_id=1007
Re: Test suite v1.0 ? [ In reply to ]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Frank Ellermann wrote:
> Julian Mehnle wrote:
> > I discussed this with Wayne and Scott on #spf extensively, and it
> > seems we cannot agree on a common interpretation of RFC 4408.
>
> Some reference implementation passing the test suite would be nice,
> it doesn't need to be Mail:SPF, anything that works as a base for
> why.html is fine.

I agree that Mail::SPF doesn't have to be "the" or "a" reference
implementation.

However, you have misunderstood me. I am disputing the test-suite with
regard to the cited test cases.

> Of course IPv6-aware implementations MUST understand ip4 mechanisms
> and interpret them for the what's it, ::FFFF:ip4 (?)
>
> But the opposite isn't true, and IPv4-only implementations are free
> to ignore ip6 mechanisms completely.

I think this is a baseless assumption.

> They never get an IPv6 address as input, because they are IPv4-only.
> Therefore you can't expect that they look at ip6-mechanisms, which happen
> to talk about IPv4 addresses in a twisted way.

Look, I already told this to Wayne on #spf. There is little point in not
implementing the "ip6" mechanism. Yes, IPv6 address syntax checks aren't
exactly trivial, but they aren't hard either, and doing IPv6 address math
_is_ trivial. There's hardly a reason for an implementation not to
support the "ip6" mechanism, and I think the old Mail::SPF::Query (for
instance) not having supported it was a joke.

Besides, the RFC requires you to detect syntax errors in "ip6" mechs
anyway, even if you are on an IPv4-only system.

> Of course the spec. can't rule out such cases in any consistent way
> in syntax. Maybe 4408bis could add that as MUST, if the connecting
> IP is an IPv4 (or the IPv6 version of an IPv4) all ip6-mechanisms
> have to be ignored. [...]

That just doesn't make any sense from an IPv6 perspective, which is a super
set of the IPv4 world. Do you really want to define "ip6:::/0" to cover
all IPv6 addresses EXCEPT for ::FFFF:0/96? Pardon me, but WTF? :-)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFFW39AwL7PKlBZWjsRAtWAAKCj+WLpLwk6/d4M3wFlxW20UyalyACg+RF/
GZGHKEaF9sc0Gf7bhnCKO50=
=ZAyZ
-----END PGP SIGNATURE-----

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?list_id=1007