Mailing List Archive

On Wed, 28 Apr 2004, Bill Moseley wrote:

> I currently forward mail. Does that mean that right now if someone
> sends mail from, say, AOL (that publishes SPF) to an address I host
> and I forward that mail on to a host that verifies SPF records that the
> mail will be rejected?

For your example of AOL, then no, AOL's SPF record never results in
FAIL. You get PASS or NEUTRAL (or ERROR). However, in the more general
case:

If you are forwarding for yourself, then no, not if you whitelist your
forwarding machine on the recipient system. If you are forwarding for lots of
end users who don't know what SPF is and don't know how or why to whitelist
the forwarder, then yes, you need to rewrite the sender in case any of them
check SPF.

> BTW -- when forwarding can rewriting of the sender be skipped if the
> sender's domain doesn't publish SPF txt record?

Yes - although it's *much* simpler to just always rewrite - even when
not forwarding. This allows you to reject those annoying bounces of
spam and worms you didn't send - plus those worms that try to "worm" their
way past spam defences by pretending to be a DSN. If you always rewrite,
then any bounces (MAIL FROM: <>) that come to your machine are spams
and worms.

In fact, always rewriting is often called SES - and if receivers
implement CBV, you don't need SPF. However, SPF is better because
TCP connections are comparatively expensive. With only SES, anyone could
easily DOS your system by propagating lots of spam with your return address.
Millions of recipient systems calling your system to check whether the
sender is valid is not something you want to suffer through.

(I had this happen to me. My solution: I changed the MX of the domain
to point to some spammer machines. Kind of drastic, but it worked. No,
the domain was no longer used, so no legit mail got sent to the spammers.)


--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

On Wed, 28 Apr 2004, Stuart D. Gathman wrote:

> > BTW -- when forwarding can rewriting of the sender be skipped if the
> > sender's domain doesn't publish SPF txt record?
>
> Yes - although it's *much* simpler to just always rewrite - even when
> not forwarding. This allows you to reject those annoying bounces of
> spam and worms you didn't send - plus those worms that try to "worm" their
> way past spam defences by pretending to be a DSN. If you always rewrite,
> then any bounces (MAIL FROM: <>) that come to your machine are spams
> and worms.

Ooops, I meant any bounces that are not rewritten with a valid (crypto-
graphic or database backed) rewritten sender are spams and worms.

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

On Wed, 28 Apr 2004, Bill Moseley wrote:

> I'm reposting to the SRS list:
>
> So for exim4 there's:
>
> http://www.infradead.org/rpr.html
>
> But, I'm not really clear if that's *the* recommend solution for
> rewriting. I need more time to grok the concepts -- is there an up to
> date primer around anyplace?

The up to date information is all at www.anarres.org/projects/srs/

There is an approximate exim4 implementation in the Mail::SRS package
but I have not yet had time to test it. I am working on this. I will
sort this out RSN.

S.

--
Shevek http://www.anarres.org/
I am the Borg. http://www.gothnicity.org/

On Wed, 28 Apr 2004, Stuart D. Gathman wrote:
>
> In fact, always rewriting is often called SES - and if receivers
> implement CBV, you don't need SPF.

You should distinguish between routine sender rewriting and only signing
the sender address at the source. Both of these are enough to detect
forgeries without assistance from SPF. However, SPF+SRS implements a
hop-by-hop security chain which is inherently weaker than the end-to-end
model.

It would be good to add a mechanism to SPF that allows a domain to state
that it signs its sender addresses; these addresses should not then be
rewritten when forwarded and should be verified by the recipient using
callouts or perhaps public key crypto (if that can be made to work).

> However, SPF is better because TCP connections are comparatively
> expensive. With only SES, anyone could easily DOS your system by
> propagating lots of spam with your return address. Millions of recipient
> systems calling your system to check whether the sender is valid is not
> something you want to suffer through.

A nice way to solve this problem would be to use asymmetric cryptography
to sign sender addresses and publish the public key in the DNS.
Unfortunately there's only space for about 100-200 bits of signature in an
email address, which is too small for most asymmetric cryptosystems.

--
Tony Finch <dot@dotat.at> http://dotat.at/

> From: Stuart D. Gathman
> Sent: Wednesday, April 28, 2004 8:53 PM

<...>

> Yes - although it's *much* simpler to just always rewrite - even when
> not forwarding. This allows you to reject those annoying bounces of
> spam and worms you didn't send - plus those worms that try to "worm" their
> way past spam defences by pretending to be a DSN. If you always rewrite,
> then any bounces (MAIL FROM: <>) that come to your machine are spams
> and worms.

Very true.

>
> In fact, always rewriting is often called SES - and if receivers
> implement CBV, you don't need SPF. However, SPF is better because
> TCP connections are comparatively expensive. With only SES, anyone could
> easily DOS your system by propagating lots of spam with your
> return address.
> Millions of recipient systems calling your system to check whether the
> sender is valid is not something you want to suffer through.

Actually, you are no more immune to a DDoS attack with SPF than with SES.
The attack method is slightly different, but the bandwidth and CPU
consequences are virtually identical. If you have an SPF-enabled MTA, an
attacker could use a number of compromised machines on broadband lines where
the provider does not do outbound port 25 blocking and rate limiting (in
plentiful supply). Let's say the attacker has 1,000 such zombies under his
control (a conservative number, based on DDoS attack descriptions I have
seen) and each machine sends 100 messages to your MX per hour. That is 100K
TCP connections per hour, and the attacker can scale up the number of
zombies very easily and reduce the number of messages coming from each one
making it virtually impossible to identify them. Your MX will reject every
one of those messages based on SPF, but each one was still a TCP connection,
so your overhead is about the same as receiving a CBV that fails. If you
can reject the connection earlier than the SPF check by means of black lists
or IP heuristics, you could do the same for a CBV, so the two cases are
quite similar in terms of overhead.

For the case of 1,000 zombies each sending you 100 messages per hour (or
10,000 zombies each sending you 10 messages per hour), that is 100K TCP
connections per hour or 2.4 million TCP connections in a 24 hour period.
Depending on your bandwidth, that could be just a minor annoyance or a big
problem.

By its very nature, there is no reasonable way to stop a DDoS attack, as
long as not too many connections come from any single source and there is
nothing "unusual" about the packets used that you can write a router rule to
null route (assuming that you control your own border router). Since the
attack "appears to be coming from everywhere at once", blacklisting the
zombies one at a time is pointless since the attacker can find new ones
faster than you can blacklist them.

The only real defense is having enough bandwidth to weather the storm until
the attacker gets bored. If you have the good fortune to have a domain that
sends no mail attacked, as Stuart did, you can then take the satisfying
action that he did, but that is not the general case.


>
> (I had this happen to me. My solution: I changed the MX of the domain
> to point to some spammer machines. Kind of drastic, but it worked. No,
> the domain was no longer used, so no legit mail got sent to the spammers.)

Payback! A rare opportunity but no doubt very satisfying.

--

Seth Goodman

On Thu, 29 Apr 2004, Seth Goodman wrote:

> Actually, you are no more immune to a DDoS attack with SPF than with SES.
> The attack method is slightly different, but the bandwidth and CPU
> consequences are virtually identical.

The difference though is that the DDOS attack caused by all the CBVs from
a joe job is unintentional in a way that a direct DDOS attack is not --
it's a side-effect of a spam run. At least CBVs do not magnify the attack
capacity of a spammer's zombie farm...

--
Tony Finch <dot@dotat.at> http://dotat.at/

> From: Tony Finch
> Sent: Thursday, April 29, 2004 5:33 AM
>
>
> On Wed, 28 Apr 2004, Stuart D. Gathman wrote:
> >
> > In fact, always rewriting is often called SES - and if receivers
> > implement CBV, you don't need SPF.
>
> You should distinguish between routine sender rewriting and only signing
> the sender address at the source. Both of these are enough to detect
> forgeries without assistance from SPF. However, SPF+SRS implements a
> hop-by-hop security chain which is inherently weaker than the end-to-end
> model.

A very good point which is often overlooked.

Another related point is that SPF+SRS also can only validate the domain, or
at best that a given local part is valid at that domain. It cannot validate
that the sender had the rights to use that local part. SES can do that when
implemented properly. If the hash secret is different for each user
(ideally the user's login password), the MSA can check the claimed return
path to match the local login account and then sign the return path using
the user's hash secret.

>
> It would be good to add a mechanism to SPF that allows a domain to state
> that it signs its sender addresses; these addresses should not then be
> rewritten when forwarded and should be verified by the recipient using
> callouts or perhaps public key crypto (if that can be made to work).

This is an excellent idea that Meng should seriously consider. In addition,
since SES addresses can be much simpler than SRS addresses, we would ideally
allow for the alternate address format in the case that the domain declares
in it's SPF record that it SES signs all outgoing mail. That one sounds
like a win-win. Let the domain owners decide which verification mechanism
they wish to use.


> > However, SPF is better because TCP connections are comparatively
> > expensive. With only SES, anyone could easily DOS your system by
> > propagating lots of spam with your return address. Millions of recipient
> > systems calling your system to check whether the sender is valid is not
> > something you want to suffer through.
>
> A nice way to solve this problem would be to use asymmetric cryptography
> to sign sender addresses and publish the public key in the DNS.
> Unfortunately there's only space for about 100-200 bits of signature in an
> email address, which is too small for most asymmetric cryptosystems.

Yes, this is an issue. I'll work on it to see if there is some way to deal
with it that is cryptographically "strong enough" for the purpose. However,
this would shift the burden of validation to the recipient rather than the
sender and that deserves some comment.

Meng has produced and published slides which correctly, IMHO, argue that the
some of the cost burden of email needs to be shifted from the recipients to
the senders. At the same time, he has argued that CBV's are too costly and
that major providers would likely not buy into any scheme that relies on
CBV's. Since the incremental cost to senders of a system relying on CBV's
is much higher than the incremental costs to recipients, this seems at odds
with the idea of cost shifting to senders. The position is further
confounded by the fact that Meng's organization, Pobox.com, does sender
callouts. If sender callouts are so expensive, why would PoBox do them,
even in the absence of SES where you don't get much information by doing
one? I respectfully ask Meng to clarify his position on this issue (and I
do mean respectfully, as my knowledge of email is puny compared to Meng's
and yet he has always replied to my questions in a helpful manner). If I
have misconstrued or misrepresented Meng's position, I apologize in advance
and ask that he point out any errors in my thinking.

--

Seth Goodman

On Thu, Apr 29, 2004 at 12:42:04PM -0500, Seth Goodman wrote:
| > It would be good to add a mechanism to SPF that allows a domain to state
| > that it signs its sender addresses; these addresses should not then be
| > rewritten when forwarded and should be verified by the recipient using
| > callouts or perhaps public key crypto (if that can be made to work).
|
| This is an excellent idea that Meng should seriously consider. In addition,
| since SES addresses can be much simpler than SRS addresses, we would ideally
| allow for the alternate address format in the case that the domain declares
| in it's SPF record that it SES signs all outgoing mail. That one sounds
| like a win-win. Let the domain owners decide which verification mechanism
| they wish to use.

i would be happy to do that. how about a modifier like srs=decline or
would ses=always be more appropriate?

On Thu, 29 Apr 2004, Seth Goodman wrote:
>
> > A nice way to solve this problem would be to use asymmetric cryptography
> > to sign sender addresses and publish the public key in the DNS.
> > Unfortunately there's only space for about 100-200 bits of signature in an
> > email address, which is too small for most asymmetric cryptosystems.
>
> Yes, this is an issue. I'll work on it to see if there is some way to deal
> with it that is cryptographically "strong enough" for the purpose.

I'm told by people who know more about crypto than myself that the
most likely candidate is using DSA with ECC. Some of the cryptographic
weakness can be addressed by re-keying more frequently than usual.

> However, this would shift the burden of validation to the recipient
> rather than the sender and that deserves some comment.
>
> Meng has produced and published slides which correctly, IMHO, argue that the
> some of the cost burden of email needs to be shifted from the recipients to
> the senders. At the same time, he has argued that CBV's are too costly and
> that major providers would likely not buy into any scheme that relies on
> CBV's. Since the incremental cost to senders of a system relying on CBV's
> is much higher than the incremental costs to recipients, this seems at odds
> with the idea of cost shifting to senders.

The problem with CBVs is that when widely implemented they can magnify the
bad effects of a joe-job on the servers for the domain that was the
subject of forgery.

However they do have advantages in the context of SSAs: for example if a
spammer or a virus gets hold of an SSA and uses it in forged messages, the
address can be revoked so that it no longer verifies with CBV. With public
keys in the DNS the key would have to be revoked which will have a wider
effect than is desired. This could be mitigated by having a key per user,
say.

Other things to do to protect SSAs against replay attacks include:

(1) Timestamps in the address to limit their lifetime, as for SRS. However
this has to be a fairly loose limit to allow for delivery delays.

(2) The address should be unsigned on final delivery, e.g. in the case of
the MAIL FROM address, before it is put into the Return-Path header. This
protects it from leaking onto the web in mailing list archives, and from
being found by viruses that grovel through mailboxes. Unfortunately this
requires changes to third-party software.

[.BTW, I'm calling them "signed sender addresses" because they can be used
to verify addresses in the 822 header as well as the envelope.]

--
Tony Finch <dot@dotat.at> http://dotat.at/

On Thu, 29 Apr 2004, Meng Weng Wong wrote:

> | > It would be good to add a mechanism to SPF that allows a domain to state
> | > that it signs its sender addresses
> |
> | This is an excellent idea that Meng should seriously consider.
>
> i would be happy to do that. how about a modifier like srs=decline or
> would ses=always be more appropriate?

I would have thought that it should be a mechanism rather than a modifier,
though I'm a bit murky on where the boundary between them falls. For
example why is include a mechanism and redirect a modifier? Perhaps some
more explanatory text could be added to the specification.

A slight problem is that this would make SPF dependent on another
specification that hasn't been written yet...

--
Tony Finch <dot@dotat.at> http://dotat.at/

> From: Tony Finch
> Sent: Thursday, April 29, 2004 1:12 PM

<...>

>
>
> On Thu, 29 Apr 2004, Seth Goodman wrote:
> >
> > > A nice way to solve this problem would be to use asymmetric
> > > cryptography
> > > to sign sender addresses and publish the public key in the DNS.
> > > Unfortunately there's only space for about 100-200 bits of
> > > signature in an
> > > email address, which is too small for most asymmetric cryptosystems.
> >
> > Yes, this is an issue. I'll work on it to see if there is some
> > way to deal
> > with it that is cryptographically "strong enough" for the purpose.
>
> I'm told by people who know more about crypto than myself that the
> most likely candidate is using DSA with ECC. Some of the cryptographic
> weakness can be addressed by re-keying more frequently than usual.

Thanks for the helpful lead. I'll definitely look into it.


> However they do have advantages in the context of SSAs: for example if a
> spammer or a virus gets hold of an SSA and uses it in forged messages, the
> address can be revoked so that it no longer verifies with CBV. With public
> keys in the DNS the key would have to be revoked which will have a wider
> effect than is desired. This could be mitigated by having a key per user,
> say.

Replay attacks are a point of exploit for any envelope signing scheme, and
this includes normal SRS rewriting at forwarders. Anyone who harvests an
SRS addresses can inflict some pain as well.

>
> Other things to do to protect SSAs against replay attacks include:
>
> (1) Timestamps in the address to limit their lifetime, as for SRS. However
> this has to be a fairly loose limit to allow for delivery delays.

Good call. My original proposal on this list for SES includes a timestamp
field for exactly this purpose. It was meant to be used exactly as the SRS
timestamp and would be set to whatever value a local site desires, probably
a week or two, though some sites may wish to allow longer for DSN's to
arrive.

>
> (2) The address should be unsigned on final delivery, e.g. in the case of
> the MAIL FROM address, before it is put into the Return-Path header. This
> protects it from leaking onto the web in mailing list archives, and from
> being found by viruses that grovel through mailboxes. Unfortunately this
> requires changes to third-party software.

SRS requires the exact same changes. You don't ever want an SRS rewritten
address to be exposed. Fortunately, this change is in the MTA, not the MUA.
In fact, we could consider leveraging off the SPF implementations so that
one package would ideally do both protocols.

>
> [.BTW, I'm calling them "signed sender addresses" because they can be used
> to verify addresses in the 822 header as well as the envelope.]

That's reasonable, but from the perspective of SPF, the only thing of
interest is MAIL FROM:, so any recognition of an alternate protocol in SPF
could only relate to that field.

--

Seth Goodman

> From: Meng Weng Wong
> Sent: Thursday, April 29, 2004 12:46 PM
>
>
> On Thu, Apr 29, 2004 at 12:42:04PM -0500, Seth Goodman wrote:
> | > It would be good to add a mechanism to SPF that allows a
> | > domain to state
> | > that it signs its sender addresses; these addresses should not then be
> | > rewritten when forwarded and should be verified by the recipient using
> | > callouts or perhaps public key crypto (if that can be made to work).
> |
> | This is an excellent idea that Meng should seriously consider.
> | In addition,
> | since SES addresses can be much simpler than SRS addresses, we
> | would ideally
> | allow for the alternate address format in the case that the
> | domain declares
> | in it's SPF record that it SES signs all outgoing mail. That one sounds
> | like a win-win. Let the domain owners decide which
> | verification mechanism
> | they wish to use.
>
> i would be happy to do that. how about a modifier like srs=decline or
> would ses=always be more appropriate?

I would personally prefer ses=always, but either would accomplish the
purpose. Thank you for offering. That does put the onus on SES advocates
to write up the proposal as an RFC draft, but that's fine.

--

Seth Goodman

In <Pine.SOL.4.58.0404291912140.17469@yellow.csi.cam.ac.uk> Tony Finch <dot@dotat.at> writes:

> On Thu, 29 Apr 2004, Meng Weng Wong wrote:
>
>> | > It would be good to add a mechanism to SPF that allows a domain to state
>> | > that it signs its sender addresses
>> |
>> | This is an excellent idea that Meng should seriously consider.
>>
>> i would be happy to do that. how about a modifier like srs=decline or
>> would ses=always be more appropriate?
>
> I would have thought that it should be a mechanism rather than a modifier,
> though I'm a bit murky on where the boundary between them falls. For
> example why is include a mechanism and redirect a modifier? Perhaps some
> more explanatory text could be added to the specification.

BTFOOM why redirect is a modifier instead of a mechanism like
include. They both change the SPF result. Once upon a time, the
"all" mechanism was a modifier too.



> A slight problem is that this would make SPF dependent on another
> specification that hasn't been written yet...

Well, if it is a modifier, it doesn't change the SPF result. Email
receivers are free to recognize the ses=always modifier if they want
and take the correct action.

-wayne

> From: wayne
> Sent: Thursday, April 29, 2004 4:59 PM

<...>

> BTFOOM why redirect is a modifier instead of a mechanism like
> include. They both change the SPF result. Once upon a time, the
> "all" mechanism was a modifier too.

So for general understanding, what determines whether something should be a
mechanism or a modifier?

--

Seth Goodman

In <MHEGIFHMACFNNIMMBACAOEPHHMAA.sethg@GoodmanAssociates.com> "Seth Goodman" <sethg@GoodmanAssociates.com> writes:

>> From: wayne
>> Sent: Thursday, April 29, 2004 4:59 PM
>
> <...>
>
>> BTFOOM why redirect is a modifier instead of a mechanism like
>> include. They both change the SPF result. Once upon a time, the
>> "all" mechanism was a modifier too.
>
> So for general understanding, what determines whether something should be a
> mechanism or a modifier?


Well, from the SPF spec section 3.2:

There are two types of directives: mechanisms and modifiers. A given
mechanism type may always appear multiple times in a record.
Modifiers may be constrained to appear at most once per record,
depending on the definition of the modifier. Unknown mechanisms
cause processing to abort with the result "unknown". Unknown
modifiers are ignored by clients.


Now, the "all" mechanism is allowed to appear more than once because
it is a mechanism, but anything after the first all mechanism doesn't
do anything. Modifiers may be constrainted to appear only once but,
as in the accred= modifier, it doesn't have to.

Mechanisms can have prefixes, modifiers can't.


-wayne

In <Pine.SOL.4.58.0404301047250.17469@yellow.csi.cam.ac.uk> Tony Finch <dot@dotat.at> writes:

> On Thu, 29 Apr 2004, wayne wrote:
>>
>> >> BTFOOM why redirect is a modifier instead of a mechanism like
>> >> include. They both change the SPF result. Once upon a time, the
>> >> "all" mechanism was a modifier too.
>
>> Mechanisms can have prefixes, modifiers can't.
>
> Perhaps this is the key. What would the prefix on a redirect mean?

In my opinion: If no (valid) SPF record is found at the target, the
redirect mechanism would act like the all mechanism and the prefix
would be the default result. Currently, the SPF spec is silent on
what should happen in this case.

So, if you had "v=spf1 ~redirect:_spf.other-domain.tld", and there
wasn't a valid SPF record under _spf, the result would be softfail.



-wayne

On Fri, Apr 30, 2004 at 06:48:44AM -0500, wayne wrote:
|
| So, if you had "v=spf1 ~redirect:_spf.other-domain.tld", and there
| wasn't a valid SPF record under _spf, the result would be softfail.
|

no-no, the above would indicate the redirect mechanism, which doesn't
exist.

redirect modifier you have to use =

In <20040430210156.GE5369@dumbo.pobox.com> Meng Weng Wong <mengwong@dumbo.pobox.com> writes:

> On Fri, Apr 30, 2004 at 06:48:44AM -0500, wayne wrote:
> |
> | So, if you had "v=spf1 ~redirect:_spf.other-domain.tld", and there
> | wasn't a valid SPF record under _spf, the result would be softfail.
> |
>
> no-no, the above would indicate the redirect mechanism, which doesn't
> exist.

Umm, yes, that is correct.

The subject of this badly drifted and off-topic sub-thread is "why is
redirect= a modifer when include: is a mechanism, and what would a
redirect: modifier do if one existed?"


-wayne