Mailing List Archive

Status: The test suite
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Some definite progress has been made on the test suite:

http://new.openspf.org/Test_Suite#roadmap

All of the old test data from Wayne's test suite, the libspf2 Subversion
repository, and the spf1-test.mailzone.com. and spftest.org. DNS zones has
been gathered and stored in the test suite's dedicated repository.

The next step is to define a formal schema for the test data, which will be
edited in YAML format and converted to, and provided to implementors in,
other formats (including XML) automatically. The current state of the
schema discussion can be found in the following spf-devel thread:

http://thread.gmane.org/gmane.mail.spam.spf.devel/934/

Then we can actually start creating the new, RFC-compliant test suite.

There is no time frame for the completion of the new test suite yet.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)

iD8DBQFEx/hIwL7PKlBZWjsRAveLAKCrEOpu39i6WYz3UKmzWNxkQFxx2ACg6CJ9
NVoIjKt8PbiA0ZiIkyaizFc=
=bIAf
-----END PGP SIGNATURE-----

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com
Re: Status: The test suite [ In reply to ]
Am Mittwoch, den 26.07.2006, 23:18 +0000 schrieb Julian Mehnle:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Some definite progress has been made on the test suite:
>
> http://new.openspf.org/Test_Suite#roadmap
>
> All of the old test data from Wayne's test suite, the libspf2 Subversion
> repository, and the spf1-test.mailzone.com. and spftest.org. DNS zones has
> been gathered and stored in the test suite's dedicated repository.
>
> The next step is to define a formal schema for the test data, which will be
> edited in YAML format and converted to, and provided to implementors in,
> other formats (including XML) automatically. The current state of the
> schema discussion can be found in the following spf-devel thread:
>
> http://thread.gmane.org/gmane.mail.spam.spf.devel/934/
>
> Then we can actually start creating the new, RFC-compliant test suite.
>
> There is no time frame for the completion of the new test suite yet.

Thx for the info.. We will start to write an "yaml" adapter for the new
files to use them in junit tests.

bye
Norman

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com
Re: Status: The test suite [ In reply to ]
On Thu, 27 Jul 2006, Norman Maurer wrote:

> Thx for the info.. We will start to write an "yaml" adapter for the new
> files to use them in junit tests.

An informal schema description has been added to
http://new.openspf.org/Test_Suite/Schema

Test scenarios from the pyspf package have been added to svn as
pyspf.yaml.

Question:
How should paragraphs be referenced? For example, section 8.1
is large. A trailing dot test should refer to just the paragraph
about trailling dots. Paragraph number is somewhat ambiguous since
paragraph first lines are not clearly distinguished from other elements,
and there is no paragraph indent.

Question:
I don't think it is correct for an SPF library to return PASS for
127.* ips (unless sender policy says so, of course). ScottK proposed the
result LOCAL, which is understood to not be an official SPF result,
and should not go in a Received-SPF header field.

I have been using the X-Guessed-SPF header field to record unofficial results
used for receiver policy. However, this name is not appropriate for
LOCAL (which is a definite policy and not guessed). Any naming
suggestions?

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com
Re: Status: The test suite [ In reply to ]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

> On Thu, 27 Jul 2006, Norman Maurer wrote:
> Question:
> I don't think it is correct for an SPF library to return PASS for
> 127.* ips (unless sender policy says so, of course). ScottK proposed the
> result LOCAL, which is understood to not be an official SPF result,
> and should not go in a Received-SPF header field.
>
> I have been using the X-Guessed-SPF header field to record unofficial results
> used for receiver policy. However, this name is not appropriate for
> LOCAL (which is a definite policy and not guessed). Any naming
> suggestions?

I use X-LOCAL-SPF-Policy.

- --
Boyd Gerber <gerberb@zenez.com>
ZENEZ 1042 East Fort Union #135, Midvale Utah 84047
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: For info see http://quantumlab.net/pine_privacy_guard/

iD8DBQFEyRZrVtBjDid73eYRAm6UAJwNA2HtKRb+XsIFXyzcYqwr7WFSmACbBbB+
DsYBNDXgnlvp49g5KlBTDLo=
=7URc
-----END PGP SIGNATURE-----

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com
Re: Status: The test suite [ In reply to ]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Stuart D. Gathman wrote:
> An informal schema description has been added to
> http://new.openspf.org/Test_Suite/Schema

Good stuff, thanks! I applied some minor improvements earlier today.

> Test scenarios from the pyspf package have been added to svn as
> pyspf.yaml.

Can you `svn move` them to old-data/pyspf-<release>/?

> Question:
> How should paragraphs be referenced? For example, section 8.1
> is large. A trailing dot test should refer to just the paragraph
> about trailling dots. Paragraph number is somewhat ambiguous since
> paragraph first lines are not clearly distinguished from other elements,
> and there is no paragraph indent.

| Stuart D. Gathman wrote:
| > Julian Mehnle wrote:
| > > Should we define a formal structure for [the "spec"] field? Should
| > > multiple spec references per test be allowed?
| >
| > No. Yes.

I think the format I had proposed earlier, "section/paragraph/sentence",
actually _is_ useful. We just need to define clearly how paragraphs are
counted.

I'd say RFC4408.txt is the reference and anything separated by empty lines
from the surrounding text (modulo page breaks) counts as a single block.
Consecutive ABNF rules count as a single block, regardless of page breaks.

3.1.5 has 7 paragraphs, 4.1 has 6, 4.6.1 has 6, 4.6.2 has 8, 5 has 10,
5.2 has 10, 6.2 has 13, 7 has 17.

> Question:
> I don't think it is correct for an SPF library to return PASS for
> 127.* ips (unless sender policy says so, of course).

Agreed.

> ScottK proposed the result LOCAL, which is understood to not be an
> official SPF result, and should not go in a Received-SPF header field.
>
> I have been using the X-Guessed-SPF header field to record unofficial
> results used for receiver policy. However, this name is not appropriate
> for LOCAL (which is a definite policy and not guessed). Any naming
> suggestions?

Am I missing something or does this question _not_ relate to the test
suite?

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)

iD8DBQFEyUHpwL7PKlBZWjsRAkq4AJ0R1qskxC576teNWL83aaWEgMEfxgCghwG5
wTWUPXOx6oFnj7n5bnZHNJ8=
=HI9Z
-----END PGP SIGNATURE-----

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com
Re: Re: Status: The test suite [ In reply to ]
On Thu, 27 Jul 2006, Julian Mehnle wrote:
>> Test scenarios from the pyspf package have been added to svn as pyspf.yaml.

> Can you `svn move` them to old-data/pyspf-<release>/?

When there is an actual RFC based test suite. The pyspf suite
is not old, it is current - just ad hoc for a specific implementation
(relected in the name). The name should indicate it is not
the intended suite. But it is useful for people to look at
as an example.

> > Question:
> > I don't think it is correct for an SPF library to return PASS for
> > 127.* ips (unless sender policy says so, of course).
>
> Agreed.
>
> > ScottK proposed the result LOCAL, which is understood to not be an
> > official SPF result, and should not go in a Received-SPF header field.
> >
> > I have been using the X-Guessed-SPF header field to record unofficial
> > results used for receiver policy. However, this name is not appropriate
> > for LOCAL (which is a definite policy and not guessed). Any naming
> > suggestions?
>
> Am I missing something or does this question _not_ relate to the test
> suite?

The header naming doesn't relate, but I am adding a test for
*not* always returning pass for 127.*.

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.


-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com
Re: Status: The test suite [ In reply to ]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Stuart D. Gathman wrote:
> On Thu, 27 Jul 2006, Julian Mehnle wrote:
> > > Test scenarios from the pyspf package have been added to svn as
> > > pyspf.yaml.
> >
> > Can you `svn move` them to old-data/pyspf-<release>/?
>
> When there is an actual RFC based test suite. The pyspf suite is not
> old, it is current - just ad hoc for a specific implementation (reflected
> in the name). The name should indicate it is not the intended suite.
> But it is useful for people to look at as an example.

OK. I am thinking of old-data/ more as contrib/, perhaps we should rename
it so?

> > > Question:
> > > I don't think it is correct for an SPF library to return PASS for
> > > 127.* ips (unless sender policy says so, of course).
> >
> > Agreed.
> >
> > > ScottK proposed the result LOCAL, which is understood to not be an
> > > official SPF result, and should not go in a Received-SPF header
> > > field.
> > >
> > > I have been using the X-Guessed-SPF header field to record
> > > unofficial results used for receiver policy. However, this name is
> > > not appropriate for LOCAL (which is a definite policy and not
> > > guessed). Any naming suggestions?
> >
> > Am I missing something or does this question _not_ relate to the test
> > suite?
>
> The header naming doesn't relate, but I am adding a test for
> *not* always returning pass for 127.*.

Good idea.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)

iD8DBQFEyckBwL7PKlBZWjsRAsF8AKCPD3f1DMRw8AKReTyNGyJnxbCvjwCfbdYI
W6uZM1yfc7kDtFDCwUquOCg=
=YOND
-----END PGP SIGNATURE-----

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com
Re: Re: Status: The test suite [ In reply to ]
On Thu, 27 Jul 2006, Julian Mehnle wrote:

> I'd say RFC4408.txt is the reference and anything separated by empty lines
> from the surrounding text (modulo page breaks) counts as a single block.
> Consecutive ABNF rules count as a single block, regardless of page breaks.

What about examples? By the above, the following:

A '%' character not followed by a '{', '%', '-', or '_' character is
a syntax error. So

-exists:%(ir).sbl.spamhaus.example.org

is incorrect and will cause check_host() to return a "PermError".
Instead, say

-exists:%{ir}.sbl.spamhaus.example.org

would be 4 paragraphs. Is that what you intended?

> 3.1.5 has 7 paragraphs, 4.1 has 6, 4.6.1 has 6, 4.6.2 has 8, 5 has 10,
> 5.2 has 10, 6.2 has 13, 7 has 17.

How many paragraphs does 8.1 have?

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com
Re: Status: The test suite [ In reply to ]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Stuart D. Gathman wrote:
> On Thu, 27 Jul 2006, Julian Mehnle wrote:
> > I'd say RFC4408.txt is the reference and anything separated by empty
> > lines from the surrounding text (modulo page breaks) counts as a
> > single block. Consecutive ABNF rules count as a single block,
> > regardless of page breaks.
>
> What about examples? By the above, the following:
>
> A '%' character not followed by a '{', '%', '-', or '_' character is
> a syntax error. So
>
> -exists:%(ir).sbl.spamhaus.example.org
>
> is incorrect and will cause check_host() to return a "PermError".
> Instead, say
>
> -exists:%{ir}.sbl.spamhaus.example.org
>
> would be 4 paragraphs. Is that what you intended?

Yes.

> > 3.1.5 has 7 paragraphs, 4.1 has 6, 4.6.1 has 6, 4.6.2 has 8, 5 has
> > 10, 5.2 has 10, 6.2 has 13, 7 has 17.
>
> How many paragraphs does 8.1 have?

29.

I noted that the xml2rfc-generated rfc4480.txt[1] has slightly different
formatting in a few places. I guess we'll have to refer strictly to the
IETF-official rfc4408.txt[2] in order to be unambiguous (even though its
formatting is seriously messed up in a few -- very few -- other places).

References:
1. http://new.openspf.org/svn/project/specs/rfc4408.txt
2. http://www.ietf.org/rfc/rfc4408.txt

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)

iD8DBQFEygKewL7PKlBZWjsRAlBXAKCOcADu5UIfpTe4TaNbK3VAG4ZnbACgpg1s
rYqyHOOQrIeTryAqCKQawZ8=
=UAse
-----END PGP SIGNATURE-----

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com
Re: Re: Status: The test suite [ In reply to ]
On Thursday 27 July 2006 21:40, Stuart D. Gathman wrote:
> On Thu, 27 Jul 2006, Julian Mehnle wrote:
> >> Test scenarios from the pyspf package have been added to svn as
> >> pyspf.yaml.
> >
> > Can you `svn move` them to old-data/pyspf-<release>/?
>
> When there is an actual RFC based test suite. The pyspf suite
> is not old, it is current - just ad hoc for a specific implementation
> (relected in the name). The name should indicate it is not
> the intended suite. But it is useful for people to look at
> as an example.
>
> > > Question:
> > > I don't think it is correct for an SPF library to return PASS for
> > > 127.* ips (unless sender policy says so, of course).
> >
> > Agreed.
> >
> > > ScottK proposed the result LOCAL, which is understood to not be an
> > > official SPF result, and should not go in a Received-SPF header field.
> > >
> > > I have been using the X-Guessed-SPF header field to record unofficial
> > > results used for receiver policy. However, this name is not
> > > appropriate for LOCAL (which is a definite policy and not guessed).
> > > Any naming suggestions?
> >
> > Am I missing something or does this question _not_ relate to the test
> > suite?
>
> The header naming doesn't relate, but I am adding a test for
> *not* always returning pass for 127.*.

For the test suite that's fine, but for implementation in pySPF, I think it's
different. It's an incompatible change with pySPF 1.6 and so it ought to be
one of the changes associated with the API compatibility flag we've
discussed.

I know that there are implementations out there that use pySPF and use that
functionality for locally submitted mail.

Scott K

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com
Re: Re: Status: The test suite [ In reply to ]
On Fri, 28 Jul 2006, Scott Kitterman wrote:

> For the test suite that's fine, but for implementation in pySPF, I think it's
> different. It's an incompatible change with pySPF 1.6 and so it ought to be
> one of the changes associated with the API compatibility flag we've
> discussed.
>
> I know that there are implementations out there that use pySPF and use that
> functionality for locally submitted mail.

It is ok to accept mail from 127.* (or any other local policy).
However, that does not make the SPF result 'pass'. There needs to
be a distinction between local policy decisions and SPF result.

It should really not be the default to return incorrect SPF results.

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com
Re: Re: Status: The test suite [ In reply to ]
On Friday 28 July 2006 09:39, Stuart D. Gathman wrote:
> On Fri, 28 Jul 2006, Scott Kitterman wrote:
> > For the test suite that's fine, but for implementation in pySPF, I think
> > it's different. It's an incompatible change with pySPF 1.6 and so it
> > ought to be one of the changes associated with the API compatibility flag
> > we've discussed.
> >
> > I know that there are implementations out there that use pySPF and use
> > that functionality for locally submitted mail.
>
> It is ok to accept mail from 127.* (or any other local policy).
> However, that does not make the SPF result 'pass'. There needs to
> be a distinction between local policy decisions and SPF result.
>
> It should really not be the default to return incorrect SPF results.

I agree in general. That's why I think we should change it going forward. We
need to be 100% compatible with pySPF 1.6 for the backward compatible version
(when I have time, I'll double check that 127.* exists in 1.6, IIRC it does).

When we release the updated pySPF we need to do two things:

1. Deliver a fully backward compatible version that implements as much of RFC
4408 as possible. To date we've got using the old result names
(error/unknown). I think that 127.* goes into this pile of items for
compatibility reasons.

2. Deliver a fully compliant pySPF that supports RFC 4408.

I think we can do both in one library with a compatibility flag, but the
default flag setting has to be for compatibility. If that's not the default,
then changes will have to be made to integrate it.

Scott K

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com