Mailing List Archive

Yet another attempt to fix forwarding
This is the latest description of the tentative solution I assembled in my
mind after browsing old posts about TENBOX and more recent discussions.
Re-reading it, I see that it is not tightly coupled with SPF. However, it
fixes forwarding also for SPF concerns. Thus, it's not off topic for this
list. Actually, it may play a nice stereo effect with the latest TENBOX
draft. If we are still alive, that is.


The tentative solution provides for a local forwarding policy. It relies
upon a database keyed on <forwarder, envelope recipient> pairs; where
"forwarder" is either the IP address or its FQDN, provided it got an SPF
"pass". When a message is being forwarded (i.e. it already has a
Return-Path), a mailer should look up the policy in order to determine
what envelope sender replacement, if any, it should use.

Not replacing the envelope sender is fine if the mailer can be
authenticated and/or whitelisted by the target MTA. Such whitelisting is
granted on a per-recipient basis. The forwarder might use existing
extensions to authenticate, e.g. the AUTH parameter idea. Either the
mailer or the local policy interface implementation should be able to
determine if the target MTA supports this kind of authorization, even if
the current recipient has not been authorized yet. In the latter case the
current message should remain in the queue (or be rejected with a 4xx
response) until the required authorization is granted. If the target MTA
does not support this solution, messages may still be forwarded using an
SPF-compliant sender replacement, as ruled by the local policy. For
example, a sender replacement is needed anyway when the final recipient's
addresses are not to be disclosed to the message originator.

A forwarder may advertise some capabilities, such as querying some DNSBLs,
performing SPF checks, signing some headers, and similar. Zero or more of
the available capabilities may be enabled for each specific recipient.
After it has been whitelisted, a forwarder will not be deemed responsible
for relayed spam. If both hosts support spam reporting, that activity may
be coordinated by sending spam reports back to the forwarder when they
belong there.

In return for a forwarding authorization, a forwarder grants the final
recipient permissions to directly inquiry, update, and delete the relevant
record of its local forwarding policy. To delete here means to not forward
any further message, possibly responding "551 user not local". That way a
recipient can control and recurse through a chain of forwarders, e.g. via
web services. Update and delete actions should be handled differently
according to the kind of alias to alias-expansion relationship (1-1 or
1-many). This feature is not usually available, thus users currently need
a special page in their address books, maybe a sticky note on their
computer, whatever. Since it provides direct access to user's data on
remote system, it might increase the sexyness of this solution.

Authorizations should be granted automatically by target MTAs. Different
sites may implement different strategies in order to check a specific
relay is trustworthy. For example, it may consist of a three-way handshake
where a target MTA (i) grants a pre-auth and wants an https url to get
capabilities from, (ii) checks the forwarder's certificate against a list
of trusted CAs, and (iii) grants full auth or redirects the forwarder
toward a different protocol which may imply manual operations. This step
is important, as after a forwarder has been registered, it will be able to
obtain authorizations for forwarding to each new user in a fully automated
fashion.

A target MTA must keep track of the authorizations it granted. For each
authorization, some properties may be relevant, e.g. if the forwarding
happens after subscribing to a list, if the address has been
double-checked, what was the requesting IP, if an alias is meant for
concealing its expansion rather than as an address update, etcetera. The
access credentials gained from forwarders are also stored, providing an
opportunity for verifying opt-in data and to enable end users to maintain
their forwarding chains.


More or less, that's it. Comments will be highly appreciated.

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=92171479-00257b
Powered by Listbox: http://www.listbox.com
Re: Yet another attempt to fix forwarding [ In reply to ]
At 04:57 PM 1/31/2008 +0100, Alessandro Vesely wrote:

>This is the latest description of the tentative solution I assembled in my mind after browsing old posts about TENBOX and more recent discussions. Re-reading it, I see that it is not tightly coupled with SPF. However, it fixes forwarding also for SPF concerns. Thus, it's not off topic for this list. Actually, it may play a nice stereo effect with the latest TENBOX draft. If we are still alive, that is.

I for one welcome any discussion relevant to SPF, and IP-based authentication in general, but I am especially interested in how to make IP-based authentication deliver on its original promise - an end to email forgery. If SPF can solve the Forwarding Problem, the last excuse for senders will disappear, and we may at last have universal, robust authentication.

>The tentative solution provides for a local forwarding policy. It relies upon a database keyed on <forwarder, envelope recipient> pairs; where "forwarder" is either the IP address or its FQDN, provided it got an SPF "pass". When a message is being forwarded (i.e. it already has a Return-Path), a mailer should look up the policy in order to determine what envelope sender replacement, if any, it should use.

Once again, I'm confused with terminology. Which FQDN? (Reverse IP, Helo, Mail From, IPwhois, ...) Which mailer? (I assume the Receiver/Forwarder since that would be the right place to re-write the Return Address.) What policy? (I assume the SPF policy of a verified Return Address, but why would this affect how you re-write the Return Address?)

>Not replacing the envelope sender is fine if the mailer can be authenticated and/or whitelisted by the target MTA.

Which mailer? Which target MTA? (Receiver, Forwarder, MDA, ...)

I'm worried that not having clear and concise terminology and a consensus on what problems we are trying to solve will lead to yet another non-productive discussion. I'll try to keep up, however. I've posted the latest terminology at http://open-mail.org/Forwarding.html, and I'll add more when possible (e.g. when two people seem to be using the same term for the same thing). I guess that's the best we can do.

I would really like to see one more step before discussing the details of various solutions - a discussion on requirements. Here are my suggestions. Others are welcome. Try to keep them general and high-level. If you want to say something like "must use SPF", try to relate that to requirements that users will understand (e.g. no cost to senders who have already published SPF records).

Requirements for Solution to Forwarding Problems
1) Use IP-based authentication (signatures are a separate topic)
2) No cost or risk to Agents on Sender's side
3) Small cost or risk to Agents on Recipient's side
4) No lost mail
5) Effective
6) Minimum vulnerability to new attacks

-- Dave

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=92464603-0561c2
Powered by Listbox: http://www.listbox.com
Re: Yet another attempt to fix forwarding [ In reply to ]
David MacQuigg wrote:
> At 04:57 PM 1/31/2008 +0100, Alessandro Vesely wrote:
>> The tentative solution provides for a local forwarding policy. It
>> relies upon a database keyed on <forwarder, envelope recipient>
>> pairs; where "forwarder" is either the IP address or its FQDN,
>> provided it got an SPF "pass". When a message is being forwarded
>> (i.e. it already has a Return-Path), a mailer should look up the
>> policy in order to determine what envelope sender replacement, if
>> any, it should use.
>
> Once again, I'm confused with terminology. Which FQDN? (Reverse IP,
> Helo, Mail From, IPwhois, ...)

The description I posted is nowhere near a (draft) specification. I
described a trustworthy network of forwarding links that can be traversed
forward and backward with the possibility for users to modify their remote
settings directly. Many details are vague at this time. For the record, I
had the helo name in mind when I wrote that phrase; however, those details
require more discussion for being worked out.

Please note that this token (FQDN or IP) is used to retrieve an entry in
the local forwarding policy, not for authentication.

Although I believe it is worth defining a sound terminology, we don't need
it for rough descriptions as the one I posted. We can run the risk of
misunderstanding some concepts. In return, such rough discussions may
enable spotting difficult points much earlier. In addition, good-faith
misunderstandings quite often originate from alternative views of some
problem, and may eventually result in valuable alternative solutions, as
in this case.

> Which mailer? (I assume the
> Receiver/Forwarder since that would be the right place to re-write the
> Return Address.)

Yes, the forwarder. I used the term "mailer" to stress that the local
forwarding policy should affect all SMTP clients that may be invoked in a
given MTA framework - typically, the sendmail executable. Actually, if the
given message contains no Return-Path header, that process cannot be
consistently called a Forwarder.

> What policy?

The one I've been talking about. The mailer looks up the local forwarding
policy to get a replacement for the envelope sender. As a side feature,
that may lead to a way to inject an SRS implementation into mailers that
don't provide one natively.

> I would really like to see one more step before discussing the details
> of various solutions - a discussion on requirements. Here are my
> suggestions. Others are welcome. Try to keep them general and
> high-level. If you want to say something like "must use SPF", try to
> relate that to requirements that users will understand (e.g. no cost to
> senders who have already published SPF records).
>
> Requirements for Solution to Forwarding Problems
> 1) Use IP-based authentication (signatures are a separate topic)

In this fix-forwarding tentative solution (hereafter FF) we might use
user/password authentication, where "user" is the above FQDN and
"passwords" are generated by the granting host. IP-based authentication
has the disadvantage of being difficult to maintain and is not suitable
for a global network of forwarding links, just like IP address won't work
as the domain part of an email address.

FF can use an IP instead of a FQDN, but I'm not sure it is a good idea and
would limit it to cases where it is not possible to do otherwise.

> 2) No cost or risk to Agents on Sender's side

OK

> 3) Small cost or risk to Agents on Recipient's side

FF requires a database with <forwarder, envelope recipient> tuples.
Obviously, the Recipient already has an envelope-recipient database: It
will be multiplied by a number of Forwarders who have the corresponding
forwarding configuration. On average, the resulting figure could match the
number of forwarding recipes that a Forwarder already stores for the
envelope recipients in its own database. However, that may be a problem in
some cases.

Even if it represents an additional cost, that data is needed for
privacy-compliant forwarding - the problem I state below and that is
missing from http://open-mail.org/Forwarding.html

Problem P - It is a mess to keep track of forwarding chains, let alone
updating any of those nodes.

The reason I call its solution "privacy-compliant" is because allowing
users to query, update, delete their data is mandated by European privacy
directives. For example, assume a former employee wants to delete a
forwarding recipe that has been configured years ago, when she left, as a
favor to her. Does she have to get in touch with her former boss?

> 4) No lost mail

OK

> 5) Effective
> 6) Minimum vulnerability to new attacks

I hope the latter two points will be the discussed on this list for FF as
well as for TENBOX, possibly comparing them.

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=92535246-d152ac
Powered by Listbox: http://www.listbox.com
Re: Yet another attempt to fix forwarding [ In reply to ]
On Thu, 31 Jan 2008, David MacQuigg wrote:

> I for one welcome any discussion relevant to SPF, and IP-based authentication
> in general, but I am especially interested in how to make IP-based
> authentication deliver on its original promise - an end to email forgery. If
> SPF can solve the Forwarding Problem, the last excuse for senders will
> disappear, and we may at last have universal, robust authentication.

There is no forwarding problem for senders. Any problems are caused
by the receiver, and while most forwarding problems receivers cause have
nothing to do with SPF, those that are related to SPF are easily avoided
by not rejecting due solely to SPF until the receiver resolves its problems.

That said, TENBOX and friends are good attempts to help large receivers
manage forwarders efficiently. My question is, since the receivers
with problems are the big ones with millions of users, why would they
listen to any proposed solutions from us?

> Requirements for Solution to Forwarding Problems
> 1) Use IP-based authentication (signatures are a separate topic)
> 2) No cost or risk to Agents on Sender's side
> 3) Small cost or risk to Agents on Recipient's side
> 4) No lost mail
> 5) Effective
> 6) Minimum vulnerability to new attacks

+1 good list

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=92661958-da964f
Powered by Listbox: http://www.listbox.com
Re: Yet another attempt to fix forwarding [ In reply to ]
At 12:16 PM 2/1/2008 +0100, Alessandro Vesely wrote:
>David MacQuigg wrote:
>
>>Requirements for Solution to Forwarding Problems
>>1) Use IP-based authentication (signatures are a separate topic)
>
>In this fix-forwarding tentative solution (hereafter FF) we might use user/password authentication, where "user" is the above FQDN and "passwords" are generated by the granting host.

Good point. As a Recipient, my own forwarding arrangements involve only password authentication. My Forwarders have no direct relationship with my MDA. To make changes at any of these Agents, I sign in with a password.

I should have said something like - if authentication is necessary, use something simple. I deleted this requirement from the list, since the valid parts are implicit in the other requirements. What I was really trying to say was let's not get involved in discussing solutions far outside the scope of this group, as the details of signature-based methods would certainly be.

>FF requires a database with <forwarder, envelope recipient> tuples. Obviously, the Recipient already has an envelope-recipient database: It will be multiplied by a number of Forwarders who have the corresponding forwarding configuration. On average, the resulting figure could match the number of forwarding recipes that a Forwarder already stores for the envelope recipients in its own database. However, that may be a problem in some cases.
>
>Even if it represents an additional cost, that data is needed for privacy-compliant forwarding - the problem I state below and that is missing from http://open-mail.org/Forwarding.html
>
>Problem P - It is a mess to keep track of forwarding chains, let alone updating any of those nodes.

Maybe we could re-state this more clearly as: Problem P - Recipients have difficulty keeping track of and updating their forwarding arrangements. I added that to our Statement of Forwarding Problems at the link above, with a note "still needing discussion".

-- Dave



-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=92939999-3d6085
Powered by Listbox: http://www.listbox.com
Re: Yet another attempt to fix forwarding [ In reply to ]
At 11:08 AM 2/1/2008 -0500, you wrote:

>On Thu, 31 Jan 2008, David MacQuigg wrote:
>
>> I for one welcome any discussion relevant to SPF, and IP-based authentication
>> in general, but I am especially interested in how to make IP-based
>> authentication deliver on its original promise - an end to email forgery. If
>> SPF can solve the Forwarding Problem, the last excuse for senders will
>> disappear, and we may at last have universal, robust authentication.
>
>There is no forwarding problem for senders. Any problems are caused
>by the receiver, and while most forwarding problems receivers cause have
>nothing to do with SPF, those that are related to SPF are easily avoided
>by not rejecting due solely to SPF until the receiver resolves its problems.

Maybe what we really need is not a new protocol, but a BCP (Best Current Practices document) for Forwarders and downstream Agents. I can see that as a good solution if the recommendations are so simple and easy that there is no excuse for not following them. http://tools.ietf.org/html

>That said, TENBOX and friends are good attempts to help large receivers
>manage forwarders efficiently. My question is, since the receivers
>with problems are the big ones with millions of users, why would they
>listen to any proposed solutions from us?

Big ESPs won't listen to us, but they will listen to their customers. After a few thousand complaints, or even questions like - "Hey Mr. Yahoo, how come all my forwarded mail goes into the [bulk] folder? Why can't I whitelist my forwarders?" After hearing that, they will eventually modify their whitelisting option, so it works with a domain name, not necessarily a full address.

Maybe we should add a requirement to our list.
6) At every stage of adoption, benefits must exceed costs to each Agent
that must take some action.

Participation is now motivated only by self interest. The urgency to solve the spam problem is gone. Everyone has their own solution, and is no longer motivated by "doing the right thing" or solving the remaining common problems.

Dick St. Peters had an unique term to describe this requirement. He called it "adiabatic expansion". I guess only a fellow physicist would appreciate the analogy. I hope Dick is still on the list.

>> Requirements for Solution to Forwarding Problems
>> 1) Use IP-based authentication (signatures are a separate topic)
>> 2) No cost or risk to Agents on Sender's side
>> 3) Small cost or risk to Agents on Recipient's side
>> 4) No lost mail
>> 5) Effective
>> 6) Minimum vulnerability to new attacks
>
>+1 good list

Thanks. I was starting to worry the old-timers were not happy with this discussion.

-- Dave

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=92952984-ba8d13
Powered by Listbox: http://www.listbox.com
Re: Yet another attempt to fix forwarding [ In reply to ]
David MacQuigg wrote:
> What I was really
> trying to say was let's not get involved in discussing solutions far
> outside the scope of this group, as the details of signature-based
> methods would certainly be.

IMHO the requirement may indicate to resolve the issue before the DATA
command, much like SPF. (Actually, SPF can already make a decision after
MAIL FROM; FF also needs the RCPT TO.)

After a message has been accepted, a wrong signature may permit to detect
that the message was a fraud. In that case we should question if the
forwarder is trustworthy. This kind of activity is important to properly
maintain a database of trusted forwarders - those who can get an
authorization automatically. No matter how well one examines a forwarder,
it may still become a spammer after having been examined. A forwarder who
states *each and every* message it forwards is signed by the original
sender, makes it possible to detect any attempt to originate spam
pretending it is being forwarded.

If the original sender provides no signature, it is difficult to discover
that a trusted forwarder has become a spammer. We should devise a
technique for detecting if a message has not actually been forwarded.
Checking a random sample may suffice. Any idea?

>> missing from http://open-mail.org/Forwarding.html
> I added that to our Statement of Forwarding Problems at
> the link above, with a note "still needing discussion".

Thanks!

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=92967279-4511d8
Powered by Listbox: http://www.listbox.com
Re: Yet another attempt to fix forwarding [ In reply to ]
At 10:49 AM 2/2/2008 +0100, Alessandro Vesely wrote:
>David MacQuigg wrote:
>> What I was really
>>trying to say was let's not get involved in discussing solutions far
>>outside the scope of this group, as the details of signature-based
>>methods would certainly be.
>
>IMHO the requirement may indicate to resolve the issue before the DATA command, much like SPF.

Good point. Why didn't I think of this. :>( How about:
7) Forwarder authentication must be resolved before the DATA command.
This is related to cost (requirement 2), but can be an independent requirement.

>After a message has been accepted, a wrong signature may permit to detect that the message was a fraud. In that case we should question if the forwarder is trustworthy. This kind of activity is important to properly maintain a database of trusted forwarders - those who can get an authorization automatically. No matter how well one examines a forwarder, it may still become a spammer after having been examined. A forwarder who states *each and every* message it forwards is signed by the original sender, makes it possible to detect any attempt to originate spam pretending it is being forwarded.
>
>If the original sender provides no signature, it is difficult to discover that a trusted forwarder has become a spammer. We should devise a technique for detecting if a message has not actually been forwarded. Checking a random sample may suffice. Any idea?

Seems to me that once a forwarding relationship has been properly established between two Agents, the only responsibility of the downstream Agent is to not allow forgery of a specific connection ( Forwarder ID, RCPT Address ). If the authorized Forwarder itself is bad, the Recipient must take action.

|-------- Recipient's Network ---------|
/
--> / --> Receiver/Forwarder ~~> MDA ==> Recipient
/
Border

If the downstream Agent does not accept this responsibility (my MDA does not), the same result can be achieved by just keeping your mailbox address secret. My MDA address has a few random digits known only to my Forwarders. It has been operating for months with no problem.

-- Dave

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=93005594-2883ea
Powered by Listbox: http://www.listbox.com
Re: Yet another attempt to fix forwarding [ In reply to ]
David MacQuigg wrote:
> At 10:49 AM 2/2/2008 +0100, Alessandro Vesely wrote:
>> If the original sender provides no signature, it is difficult to
>> discover that a trusted forwarder has become a spammer. We should
>> devise a technique for detecting if a message has not actually been
>> forwarded. Checking a random sample may suffice. Any idea?
>
> Seems to me that once a forwarding relationship has been properly
> established between two Agents, the only responsibility of the
> downstream Agent is to not allow forgery of a specific connection (
> Forwarder ID, RCPT Address ). If the authorized Forwarder itself is
> bad, the Recipient must take action.

So you're basically saying that it's up to recipients to delete any
forwarding that starts delivering just spam. Probably that's better than
any complicated cross checking that might be devised. Checking that the
deletion of a forwarding is being honored is one stringent test that can
be carried out to maintain a forwarder's trustworthiness.

In addition, a forwarder may reserve a different name (or IP) for
forwarding messages signed by some previous hop, thereby allowing the
"MDA" to check them while the forwarding relationship is operative. Such
possibility may or may not be relevant or not for the downstream agent.

> |-------- Recipient's Network ---------|
> /
> --> / --> Receiver/Forwarder ~~> MDA ==> Recipient
> /
> Border
>
> If the downstream Agent does not accept this responsibility (my MDA
> does not), the same result can be achieved by just keeping your mailbox
> address secret. My MDA address has a few random digits known only to
> my Forwarders. It has been operating for months with no problem.

I know what you mean. However, that only works for a particular setup.
Even in your case, a good FF implementation might provide the added
functionality of a centralized forwarders management, seamlessly
integrating forwarders and mailing lists, irrespective of the number of
hops they are far from the final recipient.

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=93141256-183df4
Powered by Listbox: http://www.listbox.com
Re: Yet another attempt to fix forwarding [ In reply to ]
On Fri, 1 Feb 2008, David MacQuigg wrote:

> Big ESPs won't listen to us, but they will listen to their customers. After
> a few thousand complaints, or even questions like - "Hey Mr. Yahoo, how come
> all my forwarded mail goes into the [bulk] folder? Why can't I whitelist my
> forwarders?" After hearing that, they will eventually modify their
> whitelisting option, so it works with a domain name, not necessarily a full
> address.

Customers of a big ESP have no clue what "forwarding" is, and will therefore
never ask such questions. I've tried explaining forwarding to yahoo
and aol users, complete with postal analogies, but without much success.

It is worth taking note of what big ESP customers expect (at least
from my conversations). They expect their ESP account to act like a
mail store ("where I check my mail"). They expect to be to register/purchase
email addresses that end up in their "mail store" when sent to.
They do *not* expect to have to keep track of all their email addresses
(i.e. forwarders in most cases). Computers are supposed to keep track of stuff
for them. When I try to explain "forwarding", they want to know where
they can see the list of all their email addresses. The suggestion
that they should have to keep track of the list in order for email to work
properly is met with disbelief. This is a pretty reasonable set of end-user
expectations, actually.

So there might be more traction in educating end-users enough to demand
"a list of all my email addresses" from their ESP. Any ESP able to provide
that, will have the information needed to handle forwarders properly.
Especially if forwarders publish SPF records for the forwarded domains
(receivers can validate an alias forwarder by comparing the IP against the SPF
records of valid forwarded domains for that recipient - a technique I've
called checking the "pretend domain").

I'm not sure how such a "list of all my email addresses" would get
transferred when a customer decides to switch ESPs.

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=93492651-935ffb
Powered by Listbox: http://www.listbox.com
Re: Yet another attempt to fix forwarding [ In reply to ]
On Fri, 1 Feb 2008, David MacQuigg wrote:
> Big ESPs won't listen to us, but they will listen to their customers.
> After a few thousand complaints, or even questions like - "Hey Mr. Yahoo,
> how come all my forwarded mail goes into the [bulk] folder? Why can't I
> whitelist my forwarders?"

Or, "I want to consolidate my mail, at your mailstore for [whatever
reason], but [Other ISP] is refusing to do it due to some weird technical
problem they say is at your end. Their explanation is at [URL], maybe you
can sort it out..."

> After hearing that, they will eventually modify their whitelisting
> option, so it works with a domain name, not necessarily a full address.

That's not the right direction.

One advantage SWK-SPF has over the other hacks I've seen proposed is that
the token used to identify a forwarder can be exactly the same as the
input address to the forward. This makes things more luser-resistant.

If we develop a TENBOX/O solution, then tokens that don't match the
forwarding address could be added to the whitelist of end-users with
little trouble, if the ISPs involved are on the ball.

But most users will probably look into their whitelist once in a while,
and "garbage collect" entries that they don't understand. For example, a
user might kill a mysterious-to-him "forward-co.example" entry, not
knowing that it is needed to support his alias
<jdoe@bunny-lovers.example>.

---- Michael Deutschmann <michael@talamasca.ocis.net>

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=93707497-f47ff4
Powered by Listbox: http://www.listbox.com
Re: Yet another attempt to fix forwarding [ In reply to ]
Stuart D. Gathman wrote:
> It is worth taking note of what big ESP customers expect (at least
> from my conversations). They expect their ESP account to act like a
> mail store ("where I check my mail"). They expect to be to register/purchase
> email addresses that end up in their "mail store" when sent to.
> They do *not* expect to have to keep track of all their email addresses
> (i.e. forwarders in most cases). Computers are supposed to keep track of stuff
> for them.

That's exactly what FF provides. In return for being whitelisted, a
forwarder gives the recipient credentials for accessing the relevant
record in its local forwarding database. Accessing the database through a
web service delivers any relevant information about the forwarding recipe.
In addition, it recursively delivers a list of credentials for any
forwarders one hop apart along that reverse path.

By credentials, I mean both the user/password pair and the url where they
may be applied.

A forwarding recipe may contain any data and flags. Any privacy concern is
assumed to concern hiding info to upstream senders: downstream recipients
are regarded as the owners of that data.

> When I try to explain "forwarding", they want to know where
> they can see the list of all their email addresses. The suggestion
> that they should have to keep track of the list in order for email to work
> properly is met with disbelief. This is a pretty reasonable set of end-user
> expectations, actually.

I agree, and consider those expectations a chance to fix forwarding.

> Especially if forwarders publish SPF records for the forwarded domains
> (receivers can validate an alias forwarder by comparing the IP against the SPF
> records of valid forwarded domains for that recipient - a technique I've
> called checking the "pretend domain").

While that is a possible solution, I don't 100% agree that changing the
envelope sender perfectly suits all cases. If a forwarder can be
whitelisted, in many cases the original envelope sender can be fine.

> I'm not sure how such a "list of all my email addresses" would get
> transferred when a customer decides to switch ESPs.

If both ESPs support FF, writing a recipe to forward from the older to the
newer ESP is the first occasion for the older to ask the newer for an
authorization to be whitelisted. An additional occasion will be presented
whenever an actual message is being forwarded. If that authorization is
eventually granted, accessing the older ESP's list will be possible by
means of the exchanged credentials.

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=93720039-87df3d
Powered by Listbox: http://www.listbox.com
Re: Yet another attempt to fix forwarding [ In reply to ]
At 01:37 PM 2/4/2008 -0500, Stuart Gathman wrote:

>On Fri, 1 Feb 2008, David MacQuigg wrote:
>
>> Big ESPs won't listen to us, but they will listen to their customers. After
>> a few thousand complaints, or even questions like - "Hey Mr. Yahoo, how come
>> all my forwarded mail goes into the [bulk] folder? Why can't I whitelist my
>> forwarders?" After hearing that, they will eventually modify their
>> whitelisting option, so it works with a domain name, not necessarily a full
>> address.
>
>Customers of a big ESP have no clue what "forwarding" is, and will therefore
>never ask such questions. I've tried explaining forwarding to yahoo
>and aol users, complete with postal analogies, but without much success.

Maybe you have a different situation in mind, but what I am thinking of is a setup the *Recipient* has initiated, perhaps by filling out a form on a webmail server. e.g. Yahoo's webmail: Mail Options --> POP & Forwarding -->
Forward incoming Yahoo! Mail messages to a different email address
Email address:
[ ]
(e.g. user@company.com)
There is also a help link in case this is not self-explanatory. The help window has a few FAQs at the top, followed by an interactive section where users ask each other for help. The top-rated answers appear at the top of this section.

>It is worth taking note of what big ESP customers expect (at least
>from my conversations). They expect their ESP account to act like a
>mail store ("where I check my mail"). They expect to be to register/purchase
>email addresses that end up in their "mail store" when sent to.
>They do *not* expect to have to keep track of all their email addresses
>(i.e. forwarders in most cases). Computers are supposed to keep track of stuff
>for them. When I try to explain "forwarding", they want to know where
>they can see the list of all their email addresses. The suggestion
>that they should have to keep track of the list in order for email to work
>properly is met with disbelief. This is a pretty reasonable set of end-user
>expectations, actually.

The "big ESPs" I'm familiar with offer very little individual help, certainly not keeping track of an individual's forwarding arrangements.

>So there might be more traction in educating end-users enough to demand
>"a list of all my email addresses" from their ESP. Any ESP able to provide
>that, will have the information needed to handle forwarders properly.
>Especially if forwarders publish SPF records for the forwarded domains
>(receivers can validate an alias forwarder by comparing the IP against the SPF
>records of valid forwarded domains for that recipient - a technique I've
>called checking the "pretend domain").
>
>I'm not sure how such a "list of all my email addresses" would get
>transferred when a customer decides to switch ESPs.

When you say "ESP", I assume you are referring to the MDA in this setup:

|-------- Recipient's Network ---------|
/
--> / --> Receiver/Forwarder ~~> MDA ==> Recipient
/
Border

Currently, MDAs do not keep track of Recipients' forwarding arrangements, and there is no way they can know the original RCPT TO address, since it gets re-written by the Forwarder.

If we are thinking of a new protocol or BCP, with reasonable expectations of each of the Actors involved, I can envision some SHOULD requirements for Forwarders and for MDAs. Maybe something like "Forwarders should send a monthly reminder that the forwarding arrangement is still active, and how to cancel it." I get a few of these each month for mailing lists that I subscribe to. Forwarding is very similar to a mailing list in this situation.

I'm all for minimizing burdens on the Recipient, but we need to be careful that doing so does not put an unreasonable expectation on other Actors. Ultimately, it is the Recipient (or his secretary) that must keep track of his own email addresses, his own passwords at various services, etc.

-- Dave


-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=93804186-14290a
Powered by Listbox: http://www.listbox.com
Re: Yet another attempt to fix forwarding [ In reply to ]
At 07:50 PM 2/4/2008 -0800, Michael Deutschmann wrote:

>On Fri, 1 Feb 2008, David MacQuigg wrote:
>> Big ESPs won't listen to us, but they will listen to their customers.
>> After a few thousand complaints, or even questions like - "Hey Mr. Yahoo,
>> how come all my forwarded mail goes into the [bulk] folder? Why can't I
>> whitelist my forwarders?"
>
>Or, "I want to consolidate my mail, at your mailstore for [whatever
>reason], but [Other ISP] is refusing to do it due to some weird technical
>problem they say is at your end. Their explanation is at [URL], maybe you
>can sort it out..."
>
>> After hearing that, they will eventually modify their whitelisting
>> option, so it works with a domain name, not necessarily a full address.
>
>That's not the right direction.

I don't understand what you mean. What I was talking about is a very common forwarding arrangement, one where the Recipient fills out a web form at <forwarder.org> to have all his mail currently addressed to <recipient@forwarder.org> forwarded to his new MDA. The problem was that the MDA (yahoo.com in my example) allowed whitelisting of mail from specific individual Senders, but not of all mail forwarded by <forwarder.org>. This is a common limitation I have seen at services like yahoo.com.

|-------- Recipient's Network ---------|
/
--> / --> Receiver/Forwarder ~~> MDA ==> Recipient
/
Border

Perhaps if you explain the situation you have in mind, using terms that everyone understands, I might be able to understand your proposed solution.

-- Dave


-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=93808993-6961fe
Powered by Listbox: http://www.listbox.com
Re: Yet another attempt to fix forwarding [ In reply to ]
"Stuart D. Gathman" <stuart@bmsi.com> writes:

> It is worth taking note of what big ESP customers expect (at least
> from my conversations). They expect their ESP account to act like a
> mail store ("where I check my mail"). They expect to be to register/purchase
> email addresses that end up in their "mail store" when sent to.
> They do *not* expect to have to keep track of all their email addresses
> (i.e. forwarders in most cases). Computers are supposed to keep track of stuff
> for them. When I try to explain "forwarding", they want to know where
> they can see the list of all their email addresses. The suggestion
> that they should have to keep track of the list in order for email to work
> properly is met with disbelief. This is a pretty reasonable set of end-user
> expectations, actually.

I disagree about it being reasonable. To my mind there seems absolutely
no point in having these additional email addresses which all forward
to the one ESP account if replies are sent (both rfc2821 and 2822) from
the ESP account rather than the forwarded email address. If someone
sends an email to user@example.com which is forwarded to user1@esp.com
and the response comes back from user1@esp.com, the sender is likely to
update his address book and in future send directly to user1@esp.com
(thinking that the recipient has changed his email address).

Which is another problem with this type of forwarding (rather than
forwarding an email to someone else to handle or for information etc),
that it is often not simple for the recipient to send any reply as
though from the forwarded account rather than the final destination
(where it is actually being read).

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=93800392-bbcb7d
Powered by Listbox: http://www.listbox.com
Re: Yet another attempt to fix forwarding [ In reply to ]
On Tue, 5 Feb 2008, Alessandro Vesely wrote:

> > Especially if forwarders publish SPF records for the forwarded domains
> > (receivers can validate an alias forwarder by comparing the IP against the SPF
> > records of valid forwarded domains for that recipient - a technique I've
> > called checking the "pretend domain").
>
> While that is a possible solution, I don't 100% agree that changing the
> envelope sender perfectly suits all cases. If a forwarder can be
> whitelisted, in many cases the original envelope sender can be fine.

The "pretend domain" does *not* change the envelope sender. Instead,
the receiver ignores the envelope sender while checking the IP against
a list of authorized forwarder domains, using each forwarder domain in
turn in place of envelope sender for an SPF check. As suggested by
others, the AUTH= ESMTP keyword provided by the forwarder could be used to
provide a "hint", and avoid having to check the entire list. But that is an
optimization, and not required for proper operation.

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=93836537-6afbf8
Powered by Listbox: http://www.listbox.com
Re: Yet another attempt to fix forwarding [ In reply to ]
Graham Murray wrote:
> "Stuart D. Gathman" <stuart@bmsi.com> writes:
>> The suggestion
>> that they should have to keep track of the list in order for email to work
>> properly is met with disbelief. This is a pretty reasonable set of end-user
>> expectations, actually.
>
> I disagree about it being reasonable. To my mind there seems absolutely
> no point in having these additional email addresses which all forward
> to the one ESP account if replies are sent (both rfc2821 and 2822) from
> the ESP account rather than the forwarded email address.

Even if replies are sent from the final ESP account, From and Reply-To
headers may indicate otherwise. Of course, that is a problem if forwarding
is meant to conceal the true user's identity.

However, if many users do so, then it must be reasonable!

As far as FF is concerned, mailing lists can be treated like forwarding,
which is an example we all are making use of.

> If someone
> sends an email to user@example.com which is forwarded to user1@esp.com
> and the response comes back from user1@esp.com, the sender is likely to
> update his address book and in future send directly to user1@esp.com
> (thinking that the recipient has changed his email address).
>
> Which is another problem with this type of forwarding (rather than
> forwarding an email to someone else to handle or for information etc),

Do you mean wrapping inside a mime rfc822 message?

> that it is often not simple for the recipient to send any reply as
> though from the forwarded account rather than the final destination
> (where it is actually being read).

Some clients are quite good at handling several identities on a single
account, automatically using the correct From header.

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=93891377-57fa29
Powered by Listbox: http://www.listbox.com
Re: Yet another attempt to fix forwarding [ In reply to ]
On Tue, 5 Feb 2008, David MacQuigg wrote:
> >> After hearing that, they will eventually modify their whitelisting
> >> option, so it works with a domain name, not necessarily a full address.
> >
> >That's not the right direction.
>
> I don't understand what you mean. What I was talking about is a very
> [...]
> yahoo.com.

Right direction -- extend things so that the end-user can add the
forwarding input address to their whitelist and things will Just Work.
Only SWK-SPF does this.

Wrong direction -- extend things so that the whitelist can hold
non-email-address tokens, like hostnames from rDNS or HELO. This is
dangerous because whitelist entries to support a given forward may have no
obvious connection to the forwarding, tempting lusers to "garbage collect"
them out of their lists.

Experts can use such techniques, and often have to because their
forwarders have no support for anything safer, but it could get ugly if
people who are not 100% clued-in attempt them.

---- Michael Deutschmann <michael@talamasca.ocis.net>

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=94114184-690e99
Powered by Listbox: http://www.listbox.com
Re: Yet another attempt to fix forwarding [ In reply to ]
David MacQuigg wrote:
> At 01:37 PM 2/4/2008 -0500, Stuart Gathman wrote:
>> They do *not* expect to have to keep track of all their
>> email addresses (i.e. forwarders in most cases). Computers are
>> supposed to keep track of stuff for them.
>
> [...] Ultimately, it is the Recipient (or his secretary) that must
> keep track of his own email addresses, his own passwords at various
> services, etc.

Fortunately, files management is fully automated, nowadays :-)

There are users who don't bother to register their own domain, they may
need forwarding for years, for email based registrations and because it is
not possible to know who still has one's old address. You mentioned
Yahoo!, who has managed to make forwarding recipes accessible from the
web: A proprietary solution not universally available. (My former employer
is still missing it, although most MTAs provide for forwarding recipes.)

FF gets more complicated by including forwarding recipes management. It
adds the cost of maintaining databases at both forwarders and recipients.
A wsdl, if we'll stick to using web services, will have to be specified.
IMHO, the resulting solution is somewhat cleaner than tricky expediences
that, as cleverly designed as they may be, fail to properly define a chain
of trust on the recipient's side of a message path.

Does the added stuff balance the costs? The question is threefold. One
aspect concerns the final runtime costs. That is not the same as the
installation costs at each site. The third point is about the increased
design, development, and testing costs. In all cases, if the solution will
be really awesome, costs will be happily dealt with.

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=94115695-316aea
Powered by Listbox: http://www.listbox.com
Re: Yet another attempt to fix forwarding [ In reply to ]
At 10:08 PM 2/5/2008 -0800, Michael Deutschmann wrote:

>On Tue, 5 Feb 2008, David MacQuigg wrote:
>> >> After hearing that, they will eventually modify their whitelisting
>> >> option, so it works with a domain name, not necessarily a full address.
>> >
>> >That's not the right direction.
>>
>> I don't understand what you mean. What I was talking about is a very
>> [...]
>> yahoo.com.
>
>Right direction -- extend things so that the end-user can add the
>forwarding input address to their whitelist and things will Just Work.
>Only SWK-SPF does this.
>
>Wrong direction -- extend things so that the whitelist can hold
>non-email-address tokens, like hostnames from rDNS or HELO. This is
>dangerous because whitelist entries to support a given forward may have no
>obvious connection to the forwarding, tempting lusers to "garbage collect"
>them out of their lists.

OK, I understand you mean "right direction" as "right thing to do". I read it as something to do with the direction of mail flow. English is a terrible language when we need clear, unambiguous communication! :>(

I'm still not understanding something. Maybe it's what problem are we trying to solve. I'm thinking about Problem S, which I would state even more succinctly as "Forwarder not recognized by downstream Agent". The solution I suggested (MDA allowing the Recipient to whitelist the Forwarder's domain) works for me as a Recipient, but may not be a good solution in all situations.

|-------- Recipient's Network ---------|
/
--> / --> Receiver/Forwarder ~~> MDA ==> Recipient
/
Border

My MDA is yahoo.com. They allow me to whitelist individual Senders (e.g. tom@aol.com), but not all mail coming from box67.com (my Forwarder). I have accomplished the desired result by setting up a private account at yahoo.com, and turning OFF all spam filtering on that account. As long as I can keep the address of my private account secret, it's as good as a secure channel from my Forwarder to my MDA.

What situation are you thinking of?

-- Dave


-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=94626847-b96eab
Powered by Listbox: http://www.listbox.com