Mailing List Archive: Test suite schema release candidate

Test suite schema release candidate

Jul 8, 2006, 2:23 PM

Post #1 of 13 (4972 views)

After discussion on IRC and here, we have selected YAML as the 'source'
format for test data. YAML is directly convertible to XML, so you can
consider it XML with a more readable syntax. The important thing, of
course, is the schema. Here is the the pyspf test suite again as YAML
with the latest schema. Please comment if there is a problem, because it
is time to start adding to the test suite. Oh, Julian wants to rename
zonedata to zonerecords. I don't like it, but I thought I'd throw it out.

Pyspf can read BIND and YAML (or JSON) test format, and run the tests or
write YAML. It creates SPFScenario objects with the zonedata and a
map of SPFTest objects. If the C lib people (libspf,libspf2) don't like
YAML, it can be easily converted to any XML equivalent syntax,
including XML.

---
comment: |
check basic exists with macros
tests:
exists-pass:
helo: mail.example.net
host: 1.2.3.5
mailfrom: lyme.eater@example.co.uk
result: pass
exists-fail:
helo: mail.example.net
host: 1.2.3.4
mailfrom: lyme.eater@example.co.uk
result: fail
zonedata:
lyme.eater.example.co.uk.1.2.3.5.spf.example.net:
- A: 127.0.0.1
example.co.uk:
- SPF: v=spf1 mx/26 exists:%{l}.%{d}.%{i}.spf.example.net -all

---
comment: |
include loop
tests:
incloop:
helo: mail.examplea.com
host: 66.150.186.79
mailfrom: chuckvsr@examplea.com
result: permerror
zonedata:
examplea.com:
- SPF: v=spf1 a mx include:b.com
exampleb.com:
- SPF: v=spf1 a mx include:a.com

---
tests:
nospace1:
comment: |
test no space
test multi-line comment
helo: mail.example1.com
host: 1.2.3.4
mailfrom: foo@example2.com
result: none
empty:
comment: |
test empty
helo: mail1.example1.com
host: 1.2.3.4
mailfrom: foo@example1.com
result: neutral
nospace2:
helo: mail.example1.com
host: 1.2.3.4
mailfrom: foo@example3.com
result: pass
zonedata:
example3.com:
- SPF: v=spf1mx
- SPF: v=spf1 mx
- MX: [0, mail.example1.com]
example1.com:
- SPF: v=spf1
example2.com:
- SPF: v=spf1mx
mail.example1.com:
- A: 1.2.3.4

---
comment: |
check trailing dot with redirect
tests:
traildot1:
helo: msgbas2x.cos.example.com
host: 192.168.218.40
mailfrom: test@example.com
result: pass
zonedata:
example.com.d.spf.example.com:
- SPF: v=spf1 redirect=a.spf.example.com
a.spf.example.com:
- SPF: v=spf1 mx:example.com include:o.spf.example.com -exists:%{s}.S.bl.spf.example.com
exists:%{s}.S.%{i}.AI.spf.example.com ~all
o.spf.example.com:
- SPF: v=spf1 ip4:192.168.144.41 ip4:192.168.218.40 ip4:192.168.218.41
msgbas1x.cos.example.com:
- A: 192.168.240.36
example.com:
- A: 192.168.90.76
- SPF: v=spf1 redirect=%{d}.d.spf.example.com.
- MX: [10, msgbas1x.cos.example.com]

---
comment: |
test empty MX
tests:
emptyMX:
helo: mail.example.com
host: 1.2.3.4
mailfrom: ''
result: neutral
zonedata:
mail.example.com:
- MX: [0, '']
- SPF: v=spf1 mx

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com

Re: Test suite schema release candidate [ In reply to ]

stuart at bmsi

Jul 8, 2006, 2:40 PM

Post #2 of 13 (4899 views)

Permalink

On Sat, 8 Jul 2006, Stuart D. Gathman wrote:

> with the latest schema. Please comment if there is a problem, because it
> is time to start adding to the test suite. Oh, Julian wants to rename
> zonedata to zonerecords. I don't like it, but I thought I'd throw it out.

Here is something important missing. Tests need an optional
explanation or exp field, to test the exp= modifier. I will call the
field "exp", unless someone insists on a longer name.

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com

Re: Test suite schema release candidate [ In reply to ]

stuart at bmsi

Jul 8, 2006, 2:50 PM

Post #3 of 13 (4901 views)

Permalink

On Sat, 8 Jul 2006, Stuart D. Gathman wrote:

> On Sat, 8 Jul 2006, Stuart D. Gathman wrote:
>
> > with the latest schema. Please comment if there is a problem, because it
> > is time to start adding to the test suite. Oh, Julian wants to rename
> > zonedata to zonerecords. I don't like it, but I thought I'd throw it out.
>
> Here is something important missing. Tests need an optional
> explanation or exp field, to test the exp= modifier. I will call the
> field "exp", unless someone insists on a longer name.

In fact, my pyspf tests also don't give an example of the 'spec' field.

The Schlitt tests have a 'default-explanation' field that is used like
an exp= modifier unless overriden by an actual exp= modifier. Do we
want this? Or should the zonedata for the test simply include an
exp modifier? I'm not sure how many libraries will support
default-explanation.

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com

Re: Test suite schema release candidate [ In reply to ]

stuart at bmsi

Jul 8, 2006, 3:04 PM

Post #4 of 13 (4931 views)

Permalink

On Sat, 8 Jul 2006, Stuart D. Gathman wrote:

> In fact, my pyspf tests also don't give an example of the 'spec' field.
>
> The Schlitt tests have a 'default-explanation' field that is used like
> an exp= modifier unless overriden by an actual exp= modifier. Do we
> want this? Or should the zonedata for the test simply include an
> exp modifier? I'm not sure how many libraries will support
> default-explanation.

The Schlitt tests also have a received-spf field. Since that is part
of the spec, should we handle that?

How would suite deal with the free form and optional parts of header?
Here is one idea, break out the parts:

tests:
testrcvspf:
host: 1.2.3.4
mailfrom: postmaster@115.spf1-test.mailzone.com
helo: 115.spf1-test.mailzone.com
result: permerror
received-spf:
result: permerror
client-ip: 1.2.3.4
envelope-from: postmaster@115.spf1-test.mailzone.com
helo: 115.spf1-test.mailzone.com

But this seems kind of redundant.

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com

Re: Test suite schema release candidate [ In reply to ]

julian at mehnle

Jul 8, 2006, 4:27 PM

Post #5 of 13 (4922 views)

Permalink

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Stuart D. Gathman wrote:
> After discussion on IRC and here, we have selected YAML as the 'source'
> format for test data. YAML is directly convertible to XML, so you can
> consider it XML with a more readable syntax. The important thing, of
> course, is the schema. Here is the the pyspf test suite again as YAML
> with the latest schema. Please comment if there is a problem, because
> it is time to start adding to the test suite.

My intent was to take a top-down approach, i.e. go through the spec, think
about what tests would be needed for each part, and then select suitable
tests from existing test data or invent new tests.

You seem to prefer a bottom-up approach, i.e. go through the existing test
data after having converted it, and assign each test to one or more parts
of the spec.

Is that a correct assessment?

- From the pySPF test cases you quoted I see that they already have basic
comments, right? If that's the case, we could take a middle way and
rudimentarily classify the existing tests before switching to top-down.
What I really would like to avoid is covering any part of the spec
significantly more than necessary.

> Oh, Julian wants to rename zonedata to zonerecords.

No, my proposal actually was "records", because the structure being named
is a list of records, not a list of zonedata or data.

> In fact, my pyspf tests also don't give an example of the 'spec' field.

True. Should we define a formal structure for that field? Should multiple
spec references per test be allowed?

The annotatable spec[1] says: "Spec is the section, paragraph, and sentence
in the specification that is the subject of the test (section/paragraph/
sentence). More than one Spec per test is allowed."

Examples:

spec: 4.5/2

spec:
- 5.4
- 10.1/6/2
- 10.1/7
# (should be equivalent to: spec: ["5.4", "10.1/6/2", "10.1/7"])

> The Schlitt tests have a 'default-explanation' field that is used like
> an exp= modifier unless overriden by an actual exp= modifier. Do we
> want this? Or should the zonedata for the test simply include an
> exp modifier? I'm not sure how many libraries will support
> default-explanation.

The point of such a field should not be to test whether implementations
correctly use a default explanation, but to test macro expansion. So the
field should probably be _two_ fields called "default-exp" and "exp",
containing an explanation template (when the test records don't explicitly
specify one) and the expected explanation string. (Oh, right, Python
doesn't cope well with dashes in field names. Can't PyYAML be instructed
to automatically map them to underscores or something? Or we could use
underscores right away, like "default_exp".)

> The Schlitt tests also have a received-spf field. Since that is part
> of the spec, should we handle that?

No. The free form is not fixed so it can't be tested (and the only thing
being tested would be the implementation's header field formatting logic),
and a structured form would be mostly redundant to the other test data
fields, as your example shows.

References:
1. http://new.openspf.org/auth/Test_Suite/Annotatable_Specification

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)

iD8DBQFEsD9cwL7PKlBZWjsRAsRfAKDtuyWEnDbgyFiCHzc5sTNZZSekCACdEFtv
9XQsZUyeW6XlJcevAyfJvhY=
=wxjZ
-----END PGP SIGNATURE-----

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com

Re: Re: Test suite schema release candidate [ In reply to ]

stuart at bmsi

Jul 8, 2006, 7:25 PM

Post #6 of 13 (4914 views)

Permalink

On Sat, 8 Jul 2006, Julian Mehnle wrote:

> My intent was to take a top-down approach, i.e. go through the spec, think
> about what tests would be needed for each part, and then select suitable
> tests from existing test data or invent new tests.
>
> You seem to prefer a bottom-up approach, i.e. go through the existing test
> data after having converted it, and assign each test to one or more parts
> of the spec.
>
> Is that a correct assessment?

No, I just didn't want to miss things Schlitt et al already thought of.
I think the next step is to go through the spec and convert all examples
to YAML. Ad hoc tests based on actual bugs encountered in the field
(all the current pyspf tests) can go in a separate file.

> From the pySPF test cases you quoted I see that they already have basic
> comments, right? If that's the case, we could take a middle way and
> rudimentarily classify the existing tests before switching to top-down.
> What I really would like to avoid is covering any part of the spec
> significantly more than necessary.

> > Oh, Julian wants to rename zonedata to zonerecords.
>
> No, my proposal actually was "records", because the structure being named
> is a list of records, not a list of zonedata or data.

data means records.

> True. Should we define a formal structure for that field? Should multiple
> spec references per test be allowed?

No. Yes.

>
> > The Schlitt tests have a 'default-explanation' field that is used like
> > an exp= modifier unless overriden by an actual exp= modifier. Do we
> > want this? Or should the zonedata for the test simply include an
> > exp modifier? I'm not sure how many libraries will support
> > default-explanation.
>
> The point of such a field should not be to test whether implementations
> correctly use a default explanation, but to test macro expansion. So the
> field should probably be _two_ fields called "default-exp" and "exp",
> containing an explanation template (when the test records don't explicitly

If the point is simply to test expansion, then the zonedata can include
an exp= modifier, and there is no need for default-exp.

> specify one) and the expected explanation string. (Oh, right, Python
> doesn't cope well with dashes in field names. Can't PyYAML be instructed
> to automatically map them to underscores or something? Or we could use
> underscores right away, like "default_exp".)

If hyphens are allowed in variable names, how are you supposed to
represent subtraction? Hyphens in dictionary keys, or even attributes
are no problem. But then you have to use subscripting or function
notation to reference the value. I wanted to have objects with normal
member syntax for the tests. PyYAML has no problem with hyphenated names,
they end up as dictionary keys. Of course, I can map the YAML names
to a legal variable name when converting the dictionaries to objects. But I
would prefer to just have simple words for the YAML names and avoid such
mappings.

But I suppose mapping '-' to '_' is easy enough, so if you *really* want
those hyphens you can have them. Is XML fine with hyphenated tags?
We want to be able to trivially convert the YAML to XML.

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com

Re: Test suite schema release candidate [ In reply to ]

julian at mehnle

Jul 9, 2006, 2:02 AM

Post #7 of 13 (4914 views)

Permalink

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Stuart D. Gathman wrote:
> I think the next step is to go through the spec and convert all examples
> to YAML. Ad hoc tests based on actual bugs encountered in the field
> (all the current pyspf tests) can go in a separate file.

OK.

> > > Oh, Julian wants to rename zonedata to zonerecords.
> >
> > No, my proposal actually was "records", because the structure being
> > named is a list of records, not a list of zonedata or data.
>
> data means records.

But "data" is not a plural form, so "zonedata" can't _accurately_
denominate a list of things. But I won't make a fuss about it, so let's
stick to "zonedata" if you want.

> > True. Should we define a formal structure for that field? Should
> > multiple spec references per test be allowed?
>
> No.

Not even "section[/paragraph[/sentence]]"?

> Yes.

OK. Should the "spec" field always contain a list, or just if there are
multiple entries? I'd prefer the latter.

> > > The Schlitt tests have a 'default-explanation' field that is used
> > > like an exp= modifier unless overriden by an actual exp= modifier.
> > > Do we want this? Or should the zonedata for the test simply include
> > > an exp modifier? I'm not sure how many libraries will support
> > > default-explanation.
> >
> > The point of such a field should not be to test whether
> > implementations correctly use a default explanation, but to test macro
> > expansion. So the field should probably be _two_ fields called
> > "default-exp" and "exp", containing an explanation template (when the
> > test records don't explicitly
>
> If the point is simply to test expansion, then the zonedata can include
> an exp= modifier, and there is no need for default-exp.

OK. _Is_ there another point?

> > (Oh, right, Python doesn't cope well with dashes in field names. Can't
> > PyYAML be instructed to automatically map them to underscores or
> > something? Or we could use underscores right away, like
> > "default_exp".)
>
> If hyphens are allowed in variable names, how are you supposed to
> represent subtraction?

I didn't talk of variable names.

> Hyphens in dictionary keys, or even attributes are no problem. But then
> you have to use subscripting or function notation to reference the value.
> I wanted to have objects with normal member syntax for the tests. [...]
> Of course, I can map the YAML names to a legal variable name when
> converting the dictionaries to objects.

That shouldn't be a big problem, right?

> But I would prefer to just have simple words for the YAML names and avoid
> such mappings.

Well, I can see having "mailfrom" instead of "mail-from", but "defaultexp"
or something looks really ugly and should be avoided if possible. So
depending on whether we have multiword field names besides "mailfrom", I
think converting '-' to '_' is the lesser evil.

> Is XML fine with hyphenated tags?

Yes:

http://www.w3.org/TR/REC-xml/#sec-starttags
http://www.w3.org/TR/REC-xml/#NT-NameChar

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)

iD8DBQFEsMYPwL7PKlBZWjsRAuT9AKDhDRvoId49D5XLexJ75rIiBzc7hgCgqfnT
D/R+gghH3c7DyPDwenNqM/k=
=WiW9
-----END PGP SIGNATURE-----

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com

Re: Test suite schema release candidate [ In reply to ]

monte.hansen at stashsoft

Jul 10, 2006, 12:32 AM

Post #8 of 13 (4902 views)

Permalink

Is there any yaml or xml consumable test data available? That is, derived
from qualified RFC test cases. I'd like to start testing my COMponent and
would love some xpath-able test data.

Also, does anyone have any jscript/vbscript/windozscript to create the zone
recs to test against?

Thanks in advance,

Monte Hansen

"Stuart D. Gathman" <stuart@bmsi.com> wrote in message
news:Pine.LNX.4.44.0607081712130.27366-100000@bmsred.bmsi.com...
> After discussion on IRC and here, we have selected YAML as the 'source'
> format for test data. YAML is directly convertible to XML, so you can
> consider it XML with a more readable syntax. The important thing, of
> course, is the schema. Here is the the pyspf test suite again as YAML
> with the latest schema. Please comment if there is a problem, because it
> is time to start adding to the test suite. Oh, Julian wants to rename
> zonedata to zonerecords. I don't like it, but I thought I'd throw it out.
>
> Pyspf can read BIND and YAML (or JSON) test format, and run the tests or
> write YAML. It creates SPFScenario objects with the zonedata and a
> map of SPFTest objects. If the C lib people (libspf,libspf2) don't like
> YAML, it can be easily converted to any XML equivalent syntax,
> including XML.
>
> ---
> comment: |
> check basic exists with macros
> tests:
> exists-pass:
> helo: mail.example.net
> host: 1.2.3.5
> mailfrom: lyme.eater@example.co.uk
> result: pass
> exists-fail:
> helo: mail.example.net
> host: 1.2.3.4
> mailfrom: lyme.eater@example.co.uk
> result: fail
> zonedata:
> lyme.eater.example.co.uk.1.2.3.5.spf.example.net:
> - A: 127.0.0.1
> example.co.uk:
> - SPF: v=spf1 mx/26 exists:%{l}.%{d}.%{i}.spf.example.net -all
>
> ---
> comment: |
> include loop
> tests:
> incloop:
> helo: mail.examplea.com
> host: 66.150.186.79
> mailfrom: chuckvsr@examplea.com
> result: permerror
> zonedata:
> examplea.com:
> - SPF: v=spf1 a mx include:b.com
> exampleb.com:
> - SPF: v=spf1 a mx include:a.com
>
> ---
> tests:
> nospace1:
> comment: |
> test no space
> test multi-line comment
> helo: mail.example1.com
> host: 1.2.3.4
> mailfrom: foo@example2.com
> result: none
> empty:
> comment: |
> test empty
> helo: mail1.example1.com
> host: 1.2.3.4
> mailfrom: foo@example1.com
> result: neutral
> nospace2:
> helo: mail.example1.com
> host: 1.2.3.4
> mailfrom: foo@example3.com
> result: pass
> zonedata:
> example3.com:
> - SPF: v=spf1mx
> - SPF: v=spf1 mx
> - MX: [0, mail.example1.com]
> example1.com:
> - SPF: v=spf1
> example2.com:
> - SPF: v=spf1mx
> mail.example1.com:
> - A: 1.2.3.4
>
> ---
> comment: |
> check trailing dot with redirect
> tests:
> traildot1:
> helo: msgbas2x.cos.example.com
> host: 192.168.218.40
> mailfrom: test@example.com
> result: pass
> zonedata:
> example.com.d.spf.example.com:
> - SPF: v=spf1 redirect=a.spf.example.com
> a.spf.example.com:
> - SPF: v=spf1 mx:example.com
> include:o.spf.example.com -exists:%{s}.S.bl.spf.example.com
> exists:%{s}.S.%{i}.AI.spf.example.com ~all
> o.spf.example.com:
> - SPF: v=spf1 ip4:192.168.144.41 ip4:192.168.218.40 ip4:192.168.218.41
> msgbas1x.cos.example.com:
> - A: 192.168.240.36
> example.com:
> - A: 192.168.90.76
> - SPF: v=spf1 redirect=%{d}.d.spf.example.com.
> - MX: [10, msgbas1x.cos.example.com]
>
> ---
> comment: |
> test empty MX
> tests:
> emptyMX:
> helo: mail.example.com
> host: 1.2.3.4
> mailfrom: ''
> result: neutral
> zonedata:
> mail.example.com:
> - MX: [0, '']
> - SPF: v=spf1 mx
>
>
> --
> Stuart D. Gathman <stuart@bmsi.com>
> Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
> "Confutatis maledictis, flammis acribus addictis" - background song for
> a Microsoft sponsored "Where do you want to go from here?" commercial.
>
> -------
> To unsubscribe, change your address, or temporarily deactivate your
> subscription,
> please go to
> http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com
>

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com

Re: Test suite schema release candidate [ In reply to ]

julian at mehnle

Jul 10, 2006, 2:52 AM

Post #9 of 13 (4920 views)

Permalink

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Monte Hansen wrote:
> Is there any yaml or xml consumable test data available? That is,
> derived from qualified RFC test cases. I'd like to start testing my
> COMponent and would love some xpath-able test data.

As you may be aware, we're working on compiling such a test suite right
now.

There is some previous data available that we'll be converting, but
personally I wouldn't vouch for the correctness of any of it. Stuart
would probably vouch for the correctness of his pySPF tests, though, and
I'd trust him on that. So if you need some test data _soon_, I'd go with
the pySPF tests.

As soon as we have finalized the test data YAML schema, we'll convert the
old data to this format and provide conversion scripts for more formats,
including XML.

> Also, does anyone have any jscript/vbscript/windozscript to create the
> zone recs to test against?

I don't get what you mean.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)

iD8DBQFEsiNBwL7PKlBZWjsRAtPHAJ9AXUHxWId8LVv5VIibWtuW2JlCDACgraeu
xIEVAkHwbxu3eIklbd2oaMs=
=Geba
-----END PGP SIGNATURE-----

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com

Re: Test suite schema release candidate [ In reply to ]

info at stashsoft

Jul 10, 2006, 7:56 AM

Post #10 of 13 (4913 views)

Permalink

> There is some previous data available that we'll be converting, but
> personally I wouldn't vouch for the correctness of any of it. Stuart
> would probably vouch for the correctness of his pySPF tests, though, and
> I'd trust him on that. So if you need some test data _soon_, I'd go with
> the pySPF tests.

Thanks Julian. I got that, but the datestamp on the underlying files were
2004 so I wasn't sure if I should trust it. I'll run with that data, but
could you confirm Stuart?

>> Also, does anyone have any jscript/vbscript/windozscript to create the
>> zone recs to test against?
>
> I don't get what you mean.

I hadn't realized that the tests worked against live domains/dns records.
Further examination of the data set me straight.

Kudos for the test data. It's going to make my life much easier =).

Monte Hansen

"Julian Mehnle" <julian@mehnle.net> wrote in message
news:200607100952.01836.julian@mehnle.net...
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Monte Hansen wrote:
>> Is there any yaml or xml consumable test data available? That is,
>> derived from qualified RFC test cases. I'd like to start testing my
>> COMponent and would love some xpath-able test data.
>
> As you may be aware, we're working on compiling such a test suite right
> now.
>
> There is some previous data available that we'll be converting, but
> personally I wouldn't vouch for the correctness of any of it. Stuart
> would probably vouch for the correctness of his pySPF tests, though, and
> I'd trust him on that. So if you need some test data _soon_, I'd go with
> the pySPF tests.
>
> As soon as we have finalized the test data YAML schema, we'll convert the
> old data to this format and provide conversion scripts for more formats,
> including XML.
>
>> Also, does anyone have any jscript/vbscript/windozscript to create the
>> zone recs to test against?
>
> I don't get what you mean.
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.3 (GNU/Linux)
>
> iD8DBQFEsiNBwL7PKlBZWjsRAtPHAJ9AXUHxWId8LVv5VIibWtuW2JlCDACgraeu
> xIEVAkHwbxu3eIklbd2oaMs=
> =Geba
> -----END PGP SIGNATURE-----
>
> -------
> To unsubscribe, change your address, or temporarily deactivate your
> subscription,
> please go to
> http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com
>

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com

Re: Re: Test suite schema release candidate [ In reply to ]

stuart at bmsi

Jul 10, 2006, 2:10 PM

Post #11 of 13 (4906 views)

Permalink

On Mon, 10 Jul 2006, Monte Hansen wrote:

> Thanks Julian. I got that, but the datestamp on the underlying files were
> 2004 so I wasn't sure if I should trust it. I'll run with that data, but
> could you confirm Stuart?

The pyspf tests are correct, but only represent a handful of pyspf specific
bugs that came up since 2004. The point of the SPF test project is to go
through the current RFC and come up with new tests that cover the RFC without
a lot of bloat. The schema is close enough that it is safe to start working on
that. There is still some argument about naming (sender vs mfrom vs mailfrom
vs mail-from), but nothing that couldn't be fixed with a search and replace
when the dust settles.

> I hadn't realized that the tests worked against live domains/dns records.
> Further examination of the data set me straight.

The dns records do not have to be live. It should be possible to
substitute a dummy DNSLookup function in an SPF library. Each test
scenario has an independent set of DNS records. The old test data depended
on live DNS data, but that was unreliable. If you need to test with
live DNS, we suggest a DNSLookup that tacks on a domain including scenario
or test id.

> Kudos for the test data. It's going to make my life much easier =).

Yes, we're working on it.

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com

Re: Re: Test suite schema release candidate [ In reply to ]

monte.hansen at stashsoft

Jul 10, 2006, 5:37 PM

Post #12 of 13 (4914 views)

Permalink

> that. There is still some argument about naming (sender vs mfrom vs
> mailfrom
> vs mail-from), but nothing that couldn't be fixed with a search and
> replace
> when the dust settles.

FWIW, I find that whatever is closest to the RFC context references to work
best when trying to work against the RFC. It's easy to get confused
sometimes when jumping between implementations that use different
references. Particularly, when a reference can change in context.

Monte Hansen

"Stuart D. Gathman" <stuart@bmsi.com> wrote in message
news:Pine.LNX.4.44.0607101703410.12565-100000@bmsred.bmsi.com...
> On Mon, 10 Jul 2006, Monte Hansen wrote:
>
>> Thanks Julian. I got that, but the datestamp on the underlying files were
>> 2004 so I wasn't sure if I should trust it. I'll run with that data, but
>> could you confirm Stuart?
>
> The pyspf tests are correct, but only represent a handful of pyspf
> specific
> bugs that came up since 2004. The point of the SPF test project is to go
> through the current RFC and come up with new tests that cover the RFC
> without
> a lot of bloat. The schema is close enough that it is safe to start
> working on
> that. There is still some argument about naming (sender vs mfrom vs
> mailfrom
> vs mail-from), but nothing that couldn't be fixed with a search and
> replace
> when the dust settles.
>
>> I hadn't realized that the tests worked against live domains/dns records.
>> Further examination of the data set me straight.
>
> The dns records do not have to be live. It should be possible to
> substitute a dummy DNSLookup function in an SPF library. Each test
> scenario has an independent set of DNS records. The old test data
> depended
> on live DNS data, but that was unreliable. If you need to test with
> live DNS, we suggest a DNSLookup that tacks on a domain including scenario
> or test id.
>
>> Kudos for the test data. It's going to make my life much easier =).
>
> Yes, we're working on it.
>
> --
> Stuart D. Gathman <stuart@bmsi.com>
> Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
> "Confutatis maledictis, flammis acribus addictis" - background song for
> a Microsoft sponsored "Where do you want to go from here?" commercial.
>
> -------
> To unsubscribe, change your address, or temporarily deactivate your
> subscription,
> please go to
> http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com
>

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com

Re: Re: Test suite schema release candidate [ In reply to ]

stuart at bmsi

Jul 12, 2006, 7:55 PM

Post #13 of 13 (4898 views)

Permalink

On Sun, 9 Jul 2006, Julian Mehnle wrote:

> But "data" is not a plural form, so "zonedata" can't _accurately_

Actually, "data" *is* a plural form:
"The Collaborative International Dictionary of English v.0.48"
data Da"ta, n. pl. L. pl. of datum.

1. See Datum.
1913 Webster

2. a collection of facts, observations, or other information
related to a particular question or problem; as, the
historical data show that the budget deficit is only a
small factor in determining interest rates.

...
3. { defines data in opposition to executable "code" }

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?listname=spf-devel@v2.listbox.com