Mailing List Archive

On Mon, Mar 08, 2004 at 06:34:23PM -0800, Bart Schaefer wrote:
> So ... SPF is supposed to apply to the envelope-sender address, not the
> From: header address. But SA generally doesn't have access to the former.
> How can SPF be applied as a reliable test within SA?

Isn't this line available to SPF ?

In the headers of your message:
From spamassassin-us......@incubator.apache.org
Received: from mail.apache.org (daedalus.apache.org [208.185.179.12])

No SPF record for incubator.apache.org, the current SPF library
makes an educated guess.

Fake a record "v=spf1 mx/24 ~all"
(or something similar; not important for the discussion)

MX record for incubator.apache.org is mail.apache.org
Address of mail.apache.org is 208.185.179.12
Address of sending host is 208.185.179.12 (confirmed)

208.185.179.12 is in the same /24 as 208.185.179.12 is.
Faked SPF returns OK.

HTH
Alex
--
begin sig
http://www.googlism.com/index.htm?ism=alex+van+den+bogaerdt&type=1
This message was produced without any <iframe tags

> On Mon, Mar 08, 2004 at 06:34:23PM -0800, Bart Schaefer wrote:
> > So ... SPF is supposed to apply to the envelope-sender address, not the
> > From: header address. But SA generally doesn't have access to
> the former.
> > How can SPF be applied as a reliable test within SA?
>
> Isn't this line available to SPF ?
>
> In the headers of your message:
> From spamassassin-us......@incubator.apache.org
> Received: from mail.apache.org (daedalus.apache.org [208.185.179.12])
>
> No SPF record for incubator.apache.org, the current SPF library
> makes an educated guess.
>
> Fake a record "v=spf1 mx/24 ~all"
> (or something similar; not important for the discussion)
>
> MX record for incubator.apache.org is mail.apache.org
> Address of mail.apache.org is 208.185.179.12
> Address of sending host is 208.185.179.12 (confirmed)
>
> 208.185.179.12 is in the same /24 as 208.185.179.12 is.
> Faked SPF returns OK.
>
The problem with that is that it takes no account of forwarding.

Domain A, B and C.

If I send a mail from domain A to domain B, that is then forwarded to domain
C it will appear to be spoofed as domain B is not in domain A's SPF records.
This is what SRS (http://spf.pobox.com/srs.html) was designed to fix.
However SRS only works on the envelope sender NOT the from address in the
headers. So if SA does SPF on the From header it's going to break forwarding
and mail redirection.... even if you are using the SPF methodology of
forwarding (SRS).

I have to say that I'm still much more into the Microsoft idea of putting
all this in the headers and keeping with the RFCs. Has anyone else read
their plan? I've seen to talk about it. Can anyone here poke the same huge
hole in the MS plan as with the SPF plan? I can't. If anyone would like to
read their plan and tell me why it's not better than SPF I would love to
hear the argument. You can find the details here:
http://www.microsoft.com/mscorp/twc/privacy/spam_callerID.mspx

Nick

At 08:14 AM 3/9/2004, Nick Fisher wrote:
>However SRS only works on the envelope sender NOT the from address in the
>headers. So if SA does SPF on the From header it's going to break
>forwarding and mail redirection.... even if you are using the SPF
>methodology of forwarding (SRS).

No, the From line listed in the example *is* the envelope sender. Headers
from this list look like:

From spamassassin...@incubator.apache.org
Received: from mail.apache.org (daedalus.apache.org [208.185.179.12])
...
From: "Nick Fisher" <nick@itsinteractive.com>

The first From line may show up as Return-Path or not at all in your mail
client, but it's there at the point SA is called by procmail, milters, etc.

>If anyone would like to read their plan and tell me why it's not better
>than SPF I would love to hear the argument. You can find the details here:
>http://www.microsoft.com/mscorp/twc/privacy/spam_callerID.mspx

It's patented. IANAL, so I don't know what to look for to figure out
whether it would be legal to write a GPL/BSD/Artistic/whatever-licensed
implementation of the scheme. I'll leave that to those more versed in
legalese.


Kelson Vibber
SpeedGate Communications <www.speed.net>

Hi,

On Tue, 9 Mar 2004, Nick Fisher wrote:

> > On Mon, Mar 08, 2004 at 06:34:23PM -0800, Bart Schaefer wrote:

> I have to say that I'm still much more into the Microsoft idea of putting
> all this in the headers and keeping with the RFCs. Has anyone else read
> their plan? I've seen to talk about it. Can anyone here poke the same huge
> hole in the MS plan as with the SPF plan? I can't. If anyone would like to
> read their plan and tell me why it's not better than SPF I would love to
> hear the argument. You can find the details here:
> http://www.microsoft.com/mscorp/twc/privacy/spam_callerID.mspx

Ask that question on SPAM-L and see what kind of answer you get.

But to respond, briefly:

1) It's patent-encumbered and therefore a non-starter. The license
agreement reads, in part:

"Microsoft and its Affiliates hereby grant you ("Licensee") a fully paid,
royalty-free, non-exclusive worldwide license under Microsoft's Necessary
Claims to make, use, sell, offer to sell, import, and otherwise distribute
Licensed Implementations, provided, Licensee, on behalf of itself and its
Affiliates, hereby grants Microsoft and all other Specification Licensees,
a reciprocal, fully paid, royalty-free, non-exclusive worldwide
nontransferrable, non-sublicenseable, license under Necessary Claims of
Licensee to to make, use, sell, offer to sell, import, and otherwise
distribute Licensed Implementations."

Explain what rights you cede to MS by accepting that license and what
assurances we have that they won't revise the spec and start charging
royalties once it gets wide adoption ("embrace and extend".)

2) XML is too much overhead for DNS.

Compare the simplest useful SPF record:

"v=spf1 mx -all"
(14 characters)

to the MS equivalent:

"<ep xmlns='http://ms.net/1'><out><m><mx/></m></out></ep>"
(56 characters)

The overhead just gets worse with the more complicated policies.

2.a) Explain the "HashedPuzzle" element in their schema. (See "embrace and
extend" above...)

3) It requires a lot of message header rewriting, breaking a lot of MUAs
and MTAs. Unworkable at present due to its reliance on parsable,
spec-compliant Received headers. Meaning as-is, the proposal breaks
forwarding, mobile mail, and mailing lists just like SPF.

4) In the MS proposal, judgment cannot be made on the message envelope
alone, making this a non-starter. Judgment must be performed prior to the
SMTP data phase, first to save bandwidth of the receiving system, and
second, so the mail can be rejected prior to taking responsibility for
message delivery. Once an MTA has accepted the mail it either must deliver
it or bounce it back to the sender with an explanation of why it can't be
delivered. That's fine for Exchange and qmail which (IIRC) only
accept-then-bounce; it does nothing for Sendmail, Postfix, and Exim users.

I only see more overhead and IP encumbrance in the MS proposal with little
(if any) additional advantage to show for it. Overall, I don't see how
this is less broken than SPF.

Regardless, the SATalk list is probably not the right forum to discuss
this.

-- Bob

> At 08:14 AM 3/9/2004, Nick Fisher wrote:
> >However SRS only works on the envelope sender NOT the from
> address in the
> >headers. So if SA does SPF on the From header it's going to break
> >forwarding and mail redirection.... even if you are using the SPF
> >methodology of forwarding (SRS).
>
> No, the From line listed in the example *is* the envelope sender.
> Headers
> from this list look like:
>
> From spamassassin...@incubator.apache.org
> Received: from mail.apache.org (daedalus.apache.org [208.185.179.12])
> ...
> From: "Nick Fisher" <nick@itsinteractive.com>
>
> The first From line may show up as Return-Path or not at all in your mail
> client, but it's there at the point SA is called by procmail,
> milters, etc.
Ah... my bad. Sorry... I should have looked closer.

Nick

> > I have to say that I'm still much more into the Microsoft idea
> of putting
> > all this in the headers and keeping with the RFCs. Has anyone else read
> > their plan? I've seen to talk about it. Can anyone here poke
> the same huge
> > hole in the MS plan as with the SPF plan? I can't. If anyone
> would like to
> > read their plan and tell me why it's not better than SPF I would love to
> > hear the argument. You can find the details here:
> > http://www.microsoft.com/mscorp/twc/privacy/spam_callerID.mspx
>
> Ask that question on SPAM-L and see what kind of answer you get.
I only mentioned it as we do seem to be talking alot about this sorta stuff.

> 1) It's patent-encumbered and therefore a non-starter. The license
> agreement reads, in part:
Agreed. I think this is the No1 reason that it's going to get nowhere. We
can always pick the corpse for ideas though.

> 2) XML is too much overhead for DNS.
I thought the XML was kinda heavy too. It does allow for alot of flexability
though.

> 3) It requires a lot of message header rewriting, breaking a lot of MUAs
> and MTAs. Unworkable at present due to its reliance on parsable,
> spec-compliant Received headers. Meaning as-is, the proposal breaks
> forwarding, mobile mail, and mailing lists just like SPF.
Now this is where it's a bit more intresting. I frankly thought that doing
all this in headers rather than in the envelop sender would make this
easyer. Alot of MUAs and MTAs are already setup to add/remove headers, there
would just need to be an expansion of that.... rather than changing the way
SMTP works as you need to with SRS.
I'm also not sure why you make a point of saying that you need
'spec-compliant Received headers'. As far as I can see they are just normall
Received headers.... arn't they in the RFCs and implemented already?

> 4) In the MS proposal, judgment cannot be made on the message envelope
> alone, making this a non-starter. Judgment must be performed prior to the
> SMTP data phase, first to save bandwidth of the receiving system, and
> second, so the mail can be rejected prior to taking responsibility for
> message delivery.
You say this like it's a bad thing.... frankly it seems to me that it makes
more sence to take the burden of determining where the mail is actually from
away from the MTA. It seems to me that the MTA should be there to transport
mail not process it.
Yes it would require that more bandwidth is used. However (IIRC) most MTAs
do not reject mail untill the DATA phase of the mail is over, thus even if
the MTA does reject the only bandwidth saved is that of the bounce message.

> Once an MTA has accepted the mail it either must deliver
> it or bounce it back to the sender with an explanation of why it can't be
> delivered. That's fine for Exchange and qmail which (IIRC) only
> accept-then-bounce; it does nothing for Sendmail, Postfix, and Exim users.
Err..... I know that postfix can bounce mail after it's recived.... I would
be amazed if the other MTAs you mention cannot.

> I only see more overhead and IP encumbrance in the MS proposal with little
> (if any) additional advantage to show for it. Overall, I don't see how
> this is less broken than SPF.
I would agree that both are broken in some respects. However it still seems
to me that trying to auth the message is much easyer to do in headers than
through the envelope sender. In my mind it breaks less and is easyer to deal
with. Having said that I think it would need to be reinvented without the
legal crud tying it down to be worth anything.

> Regardless, the SATalk list is probably not the right forum to discuss
> this.
Fair enough... but as there seems to be a SPF thread rolling I thought I
could throw it in without being too OT.

Nick

At 10:14 AM 3/9/2004, Nick Fisher wrote:
>>Once an MTA has accepted the mail it either must deliver it or bounce it
>>back to the sender with an explanation of why it can't be delivered.
>>That's fine for Exchange and qmail which (IIRC) only accept-then-bounce;
>>it does nothing for Sendmail, Postfix, and Exim users.
>
>Err..... I know that postfix can bounce mail after it's recived.... I
>would be amazed if the other MTAs you mention cannot.

Sendmail, at least, can do even better: It can read the entire message,
process it, and then issue a rejection saying "Whoops, sorry, we don't want
this after all!" - all within the SMTP transaction.

This means you can analyze the heck out of a message and reject it, and you
don't need to worry about whether the bounce notice is going to the right
place. No risk of undeliverable bounces waiting around in your outgoing
mail queue, plus you don't contribute to the flood of invalid "You sent us
a virus!" notices going to people whose computers aren't infected.


Kelson Vibber
SpeedGate Communications <www.speed.net>

Nick Fisher wrote:
> Err..... I know that postfix can bounce mail after it's recived.... I
> would be amazed if the other MTAs you mention cannot.
[ sendmail and Exim ]

The problem with accept-then-bounce is that the message ends up stuck on
the *receiving* server. This is NOT a good way to handle email-
especially considering quite a bit of spam doesn't have a valid return
address of ANY kind ANYWHERE in the message, and viruses may technically
have a valid return address... but not a *correct* one.

On high-load mail systems (ie, multi-million messages/day/server), this
can mean a LOT of dead-end messages stuck in the queue for days on end.

-kgd
--
"Sendmail administration is not black magic. There are legitimate
technical reasons why it requires the sacrificing of a live chicken."
- Unknown

> > Err..... I know that postfix can bounce mail after it's recived.... I
> > would be amazed if the other MTAs you mention cannot.
> [ sendmail and Exim ]
>
> The problem with accept-then-bounce is that the message ends up stuck on
> the *receiving* server. This is NOT a good way to handle email-
> especially considering quite a bit of spam doesn't have a valid return
> address of ANY kind ANYWHERE in the message, and viruses may technically
> have a valid return address... but not a *correct* one.
>
> On high-load mail systems (ie, multi-million messages/day/server), this
> can mean a LOT of dead-end messages stuck in the queue for days on end.
>
Unless you are proposing to move all spam/virus/malware checks into the MTA
to be used during incomming SMTP sessions, this is a problem you are always
going to have. There is nothing stoppping the MTA from checking the headers
and bouncing during a SMTP session (though it does require a peek in the
headers), but unlike SPF it isn't required.

Nick

On Tue, Mar 09, 2004 at 11:14:39AM -0500, Nick Fisher wrote:

> > In the headers of your message:
> > From spamassassin-us......@incubator.apache.org
> > Received: from mail.apache.org (daedalus.apache.org [208.185.179.12])
> >
[snip]
> >
> The problem with that is that it takes no account of forwarding.
>
> Domain A, B and C.
>
> If I send a mail from domain A to domain B, that is then forwarded to domain
> C it will appear to be spoofed as domain B is not in domain A's SPF records.

It does not *appear* to be spoofed, it *is* spoofed. Yes, SPF stops that.

You were already pointed at the difference between "From " (note the space)
and "From:". As you can see, SPF works perfectly for the spamassassin
mailing list.

Now consider a "mailing list" with just one subscriber.

domain A: home of the author of a message
domain B: hosting said "list"
domain C: home of the sole subscriber to that list

userA@domainA sends a message to userB@domainB. userB@domainB setup
forwarding. All domainB has to do when it transfers the message to
domainC is:

1: make sure the envelope sender address is at domainB, not domainA
2: there is no #2.

No SRS is needed. No other special tricks are needed. SPF does not
break forwarding. SPF breaks spoofing; and does not care if spoofing
is done by someone with good intentions or someone with bad intentions.

SRS is just a smart way _for_domainB_ to generate a *local* address that
has the address of userA@domainA encoded in it. DomainB could keep a
database instead.

But what about bounces? Well, if userB@domainB screws up, admin@domainB
will have to educate userB. If domainC screws up, that's between domains
B and C. DomainA succesfully delivered the message to its destination.

cheers,
Alex
--
begin sig
http://www.googlism.com/index.htm?ism=alex+van+den+bogaerdt&type=1
This message was produced without any <iframe tags

Nick Fisher wrote:

> Unless you are proposing to move all spam/virus/malware checks into the MTA
> to be used during incomming SMTP sessions, this is a problem you are always
> going to have. There is nothing stoppping the MTA from checking the headers
> and bouncing during a SMTP session (though it does require a peek in the
> headers), but unlike SPF it isn't required.

This is what I and many others do using Exim and either Exiscan or
SA-Exim. I believe the Postfix people are also working on integrating
this into their next release. It's really the best way to bounce mail,
particularly with all the address forging going on.

Steven
--
Steven Dickenson <steven@mrchuckles.net>
http://www.mrchuckles.net