Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. SPF>
  3. Discuss
Perils of reputation
stuart at bmsi
Feb 6, 2007, 5:06 PM
Post #1 of 11 (2781 views)
Permalink
Consider the case of this sender:

2007Feb06 02:07:11 [1324] connect from cmn1lsm3.beliefnet.com at
('129.33.230.137', 43757) EXTERNAL
2007Feb06 02:07:12 [1324] hello from cmn1lsm3.beliefnet.com
2007Feb06 02:07:12 [1324] mail from <listadmin4@partner.beliefnet.com> ()
2007Feb06 02:07:12 [1324] Received-SPF: pass (smtp.example.com: domain of
partner.beliefnet.com designates 129.33.230.137 as permitted sender)
client_ip=129.33.230.137;
envelope_from="listadmin4@partner.beliefnet.com";
helo=cmn1lsm3.beliefnet.com; receiver=smtp.example.com;
mechanism="a:cmn1lsm3.beliefnet.com"; identity=mailfrom
2007Feb06 02:07:12 ham: 0, spam: 25
2007Feb06 02:07:12 ID partner.beliefnet.com:SPF reputation:
-76.159416,2.014513
2007Feb06 02:07:12 [1324] X-GOSSiP: uqaWJvNVWKgzP7TsOY9.Jg,-76,2
2007Feb06 02:07:12 [1324] rcpt to <jackiel@example.com> ()
2007Feb06 02:07:12 [1324] REJECT: REPUTATION

They are not actually spamming. They have a very nice SPF record. Users
at this company actually signed up for their mailings. Their mailings
*are* laden with advertising. That is, after all, how their operation
is funded. This similarity with actual spam causes a message or two
to be quarantined. The user doesn't actually care that much about reading
the messages, and doesn't bother releasing them from the quarantine. They
never send any email to the domain, so no auto-whitelisting takes place.
The stats snowball until all messages are quarantined. The reputation
takes a nosedive, and the system starts rejecting all messages. Quite
reasonable, since they were just sitting in quarantine until deleted anyway.

This is an example of practical spam. Stuff that users sign up for, but
don't actually have time to read. Kind of like those magazine subscriptions
that pile up in the bathroom, or those newpapers sitting in your recycling
bin that you never get around to reading. It is a good thing that
the system eventually learns to refuse delivery. However, I feel like
there should be a different kind of demerit for this kind of "spam", because
the company is not actually doing anything wrong. The reputation should
have a high "lost interest" score, that is distinguished from a
high "criminal spammer" score. But I am not sure how to capture that
distinction from end users.

Certainly, the best way to do this is to charge recipients for the
subscription. That will certainly motivate them to whitelist the sender.
And if they never read it, they don't have to renew.

However, advertising funded content is very popular. I suppose that
messages actually reported as spam or sent to a honeypot mailbox should
get a different kind of demerit than messages that are simply left in
quarantine. So there would be three counts: ham, spam, cageliner. The
last two would count together for purposes of quarantine and rejection,
but only the spam stat would determine the "evilness" of the sender.

Which might affect how the system GOSSiPs about senders.
When responding to a reputation query, the cageliner messages should
count as ham, rather than spam.

Comments? Insights?

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-------
Sender Policy Framework: http://www.openspf.org/
Archives at http://archives.listbox.com/spf-discuss/current/
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?list_id=735
Re: Perils of reputation [ In reply to ]
paddy at panici
Feb 7, 2007, 4:30 AM
Post #2 of 11 (2741 views)
Permalink
On Tue, Feb 06, 2007 at 08:06:26PM -0500, Stuart D. Gathman wrote:
> Consider the case of this sender:
>
> 2007Feb06 02:07:11 [1324] connect from cmn1lsm3.beliefnet.com at
> ('129.33.230.137', 43757) EXTERNAL
> 2007Feb06 02:07:12 [1324] hello from cmn1lsm3.beliefnet.com
> 2007Feb06 02:07:12 [1324] mail from <listadmin4@partner.beliefnet.com> ()
> 2007Feb06 02:07:12 [1324] Received-SPF: pass (smtp.example.com: domain of
> partner.beliefnet.com designates 129.33.230.137 as permitted sender)
> client_ip=129.33.230.137;
> envelope_from="listadmin4@partner.beliefnet.com";
> helo=cmn1lsm3.beliefnet.com; receiver=smtp.example.com;
> mechanism="a:cmn1lsm3.beliefnet.com"; identity=mailfrom
> 2007Feb06 02:07:12 ham: 0, spam: 25
> 2007Feb06 02:07:12 ID partner.beliefnet.com:SPF reputation:
> -76.159416,2.014513
> 2007Feb06 02:07:12 [1324] X-GOSSiP: uqaWJvNVWKgzP7TsOY9.Jg,-76,2
> 2007Feb06 02:07:12 [1324] rcpt to <jackiel@example.com> ()
> 2007Feb06 02:07:12 [1324] REJECT: REPUTATION
>
> They are not actually spamming. They have a very nice SPF record. Users
> at this company actually signed up for their mailings. Their mailings
> *are* laden with advertising. That is, after all, how their operation
> is funded. This similarity with actual spam causes a message or two
> to be quarantined. The user doesn't actually care that much about reading
> the messages, and doesn't bother releasing them from the quarantine. They
> never send any email to the domain, so no auto-whitelisting takes place.
> The stats snowball until all messages are quarantined. The reputation
> takes a nosedive, and the system starts rejecting all messages. Quite
> reasonable, since they were just sitting in quarantine until deleted anyway.
>
> This is an example of practical spam. Stuff that users sign up for, but
> don't actually have time to read. Kind of like those magazine subscriptions
> that pile up in the bathroom, or those newpapers sitting in your recycling
> bin that you never get around to reading. It is a good thing that
> the system eventually learns to refuse delivery. However, I feel like
> there should be a different kind of demerit for this kind of "spam", because
> the company is not actually doing anything wrong. The reputation should
> have a high "lost interest" score, that is distinguished from a
> high "criminal spammer" score. But I am not sure how to capture that
> distinction from end users.
>
> Certainly, the best way to do this is to charge recipients for the
> subscription. That will certainly motivate them to whitelist the sender.
> And if they never read it, they don't have to renew.
>
> However, advertising funded content is very popular. I suppose that
> messages actually reported as spam or sent to a honeypot mailbox should
> get a different kind of demerit than messages that are simply left in
> quarantine. So there would be three counts: ham, spam, cageliner. The
> last two would count together for purposes of quarantine and rejection,
> but only the spam stat would determine the "evilness" of the sender.
>
> Which might affect how the system GOSSiPs about senders.
> When responding to a reputation query, the cageliner messages should
> count as ham, rather than spam.
>
> Comments? Insights?


I see two distinct issues here.

if you base your reputation scores on things like textual similarity
or similarity of SA scores, rather than whether the email is actually
wanted by the recipient, then you are going to see anomalies.


The quarantine delays user involvement in feedback, the system feeds back
on itself until it comes to the attention of a human being. if you delay
human feedback, when it becomes necessary the mechanical feedback will
have had longer to work.

Arguably, since quarantines mostly work (sort of), this is a feature of
a quarantine system and a trade-off to be considered when using one.

I really like the term cageliner :-)

An interesting question is the relationship individual users have with
cageliner material. One user may go around signing up for stuff and not
really realising they have, and not wanting it, or even reacting to it
as spam (reporting it), while another user might sign up for the same
material and really want it and be annoyed if it is filtered
inappropriately.

I also question whether there is really a clear
black and white boundary between ham and spam or whether when you look
closely it isn't more continuous - shades of grey. I think that the
fact that you might treat the exact same circular differently for
different users is in fact the extra dimension you are looking for with
your three counts, and it's a real question mark as to whether three
counts will turn out to be a useful model of that. I'm sorry I can't
offer more help with what might make a good model, but I would be
looking in the direction of statistics, eg: mean and sd, so instead of a
count of ham/spam scoring you get a distribution.

Regards,
Paddy

-------
Sender Policy Framework: http://www.openspf.org/
Archives at http://archives.listbox.com/spf-discuss/current/
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?list_id=735
RE: Perils of reputation [ In reply to ]
sethg at goodmanassociates
Feb 7, 2007, 9:59 AM
Post #3 of 11 (2739 views)
Permalink
paddy@panici.net wrote on Wednesday, February 07, 2007 6:30 AM -0600:

> I would be looking in the direction of statistics, eg: mean and sd, so
instead of a
> count of ham/spam scoring you get a distribution.

This is very interesting, as it addresses what a reputation score means
for different compositions of message flows, and similar question just
came up for the Spambayes classifier. I imagine that Stuart converts
message spam scores into sender reputation the same way everyone else
does:

1) compute a real-valued spam score for each received message

2) "quantize" the score into a binary result (ham/spam)

3) count how many results are in each class

4) use those two counts to compute an overall reputation score

5) make a decision


You can see that there are two decision steps where real-valued numbers
are quantized into a binary result, steps 2 and 5. The final
quantization in step 5 is unavoidable, but the first one in step 2 is
done out of convenience and for historical reasons. Each time you do
this you add "quantization noise", exactly analogous to that in an A/D
converter.

If the messages classify definitively as ham or spam, quantizing
individual message scores into a binary result and counting them
preserves the important information. However, with Stuart's cageliner
material, the individual message spam scores are not definitive.
Quantizing these scores into two classes in step 2 introduces a lot of
noise that the subsequent averaging in step 3 may not adequately remove,
and the resulting overall reputation score may be biased. Since the
expected value for the overall score is somewhere near the decision
threshold for the cageliner class of material, quantization noise can
have a large impact on the final decision.

I suspect that Paddy's suggestion will result in better categorization
of cageliner material without causing problems for strongly classifying
ham/spam. There are a couple of ways to go about this. One way is to
replace steps 2, 3 and 4 above with a computation of statistics, i.e.
mean and variance, as Paddy suggests. In step 5, you compare the mean
to a threshold to produce a binary result (make a decision), and if you
care to assume a distribution shape, use the variance to compute a
real-valued confidence indicator. Another approach is to use a
combining algorithm, i.e. Fischer combining, to produce a real-valued
result that you then compare to a threshold.

--
Seth Goodman

-------
Sender Policy Framework: http://www.openspf.org/
Archives at http://archives.listbox.com/spf-discuss/current/
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?list_id=735
Re: Perils of reputation [ In reply to ]
macquigg at box67
Feb 7, 2007, 12:30 PM
Post #4 of 11 (2738 views)
Permalink
Interesting problem! The example is very helpful in defining the category
of email we are talking about. How about we call it "Solicited Commercial
Email" or SCE, instead of "cageliner". This would include mail from
charitable and political organizations asking for donations, ads from
Walgreens, and stock tips that might be something your broker signed you up
for, but you would like to see an authentication header to help you decide
whether to un-subscribe or report it as spam.

One way of dealing with SCE is an accreditation service like Goodmail. The
sender pays a fee, and the accreditation service handles all complaints,
including the inevitable complaints from recipients who forgot they signed
up. Receivers who trust Goodmail can whitelist a sender based on that
sender's rating with Goodmail.

I wonder if a really good reputation system can eliminate the need for
accreditation services and special processing of SCE. Reputable senders of
SCE should make sure their recipients know why they are on the list. They
should respond promptly to any complaints, and leave very few
unresolved. Maybe by providing special processing of SCE, we are
encouraging senders to spend money on accreditation services instead of
resolving complaints. The goal is no unwanted mail, even if that mail
somehow fits the legal requirements of SCE. Blurring the line on
reputation may mean fewer senders of SCE will make the effort to get
clearly on the right side of the line. I want that broker to send me a
subscription request, not just put me on a list after some rambling phone
conversation.

-- Dave

At 08:06 PM 2/6/2007 -0500, Stuart D. Gathman wrote:

>Consider the case of this sender:
>
>2007Feb06 02:07:11 [1324] connect from cmn1lsm3.beliefnet.com at
>('129.33.230.137', 43757) EXTERNAL
>2007Feb06 02:07:12 [1324] hello from cmn1lsm3.beliefnet.com
>2007Feb06 02:07:12 [1324] mail from <listadmin4@partner.beliefnet.com> ()
>2007Feb06 02:07:12 [1324] Received-SPF: pass (smtp.example.com: domain of
>partner.beliefnet.com designates 129.33.230.137 as permitted sender)
>client_ip=129.33.230.137;
>envelope_from="listadmin4@partner.beliefnet.com";
>helo=cmn1lsm3.beliefnet.com; receiver=smtp.example.com;
>mechanism="a:cmn1lsm3.beliefnet.com"; identity=mailfrom
>2007Feb06 02:07:12 ham: 0, spam: 25
>2007Feb06 02:07:12 ID partner.beliefnet.com:SPF reputation:
>-76.159416,2.014513
>2007Feb06 02:07:12 [1324] X-GOSSiP: uqaWJvNVWKgzP7TsOY9.Jg,-76,2
>2007Feb06 02:07:12 [1324] rcpt to <jackiel@example.com> ()
>2007Feb06 02:07:12 [1324] REJECT: REPUTATION
>
>They are not actually spamming. They have a very nice SPF record. Users
>at this company actually signed up for their mailings. Their mailings
>*are* laden with advertising. That is, after all, how their operation
>is funded. This similarity with actual spam causes a message or two
>to be quarantined. The user doesn't actually care that much about reading
>the messages, and doesn't bother releasing them from the quarantine. They
>never send any email to the domain, so no auto-whitelisting takes place.
>The stats snowball until all messages are quarantined. The reputation
>takes a nosedive, and the system starts rejecting all messages. Quite
>reasonable, since they were just sitting in quarantine until deleted anyway.
>
>This is an example of practical spam. Stuff that users sign up for, but
>don't actually have time to read. Kind of like those magazine subscriptions
>that pile up in the bathroom, or those newpapers sitting in your recycling
>bin that you never get around to reading. It is a good thing that
>the system eventually learns to refuse delivery. However, I feel like
>there should be a different kind of demerit for this kind of "spam", because
>the company is not actually doing anything wrong. The reputation should
>have a high "lost interest" score, that is distinguished from a
>high "criminal spammer" score. But I am not sure how to capture that
>distinction from end users.
>
>Certainly, the best way to do this is to charge recipients for the
>subscription. That will certainly motivate them to whitelist the sender.
>And if they never read it, they don't have to renew.
>
>However, advertising funded content is very popular. I suppose that
>messages actually reported as spam or sent to a honeypot mailbox should
>get a different kind of demerit than messages that are simply left in
>quarantine. So there would be three counts: ham, spam, cageliner. The
>last two would count together for purposes of quarantine and rejection,
>but only the spam stat would determine the "evilness" of the sender.
>
>Which might affect how the system GOSSiPs about senders.
>When responding to a reputation query, the cageliner messages should
>count as ham, rather than spam.
>
>Comments? Insights?


-------
Sender Policy Framework: http://www.openspf.org/
Archives at http://archives.listbox.com/spf-discuss/current/
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?list_id=735
Re: Perils of reputation [ In reply to ]
alex at ergens
Feb 9, 2007, 6:36 AM
Post #5 of 11 (2719 views)
Permalink
On Wed, Feb 07, 2007 at 01:30:43PM -0700, David MacQuigg wrote:
> Interesting problem! The example is very helpful in defining the category
> of email we are talking about. How about we call it "Solicited Commercial
> Email" or SCE, instead of "cageliner".

no problem here, but ...

> This would include mail from
> charitable and political organizations asking for donations,

SPAM!

> ads from
> Walgreens,

SPAM!

> and stock tips that might be something your broker signed you up
> for, but you would like to see an authentication header to help you decide
> whether to un-subscribe or report it as spam.

Someone else signs me up without me asking? Obviously not a properly
managed mailing list (confirmed opt-in). Who says it was my broker signing
me up?

Also: SPAM!


If I leave my email address with somebody because they need to inform
me about delivery of my new computer, and they then use it so send me
ads for their new and improved car insurance product: SPAM!

All examples you mentioned, and the extra one I did, have one thing in
common: they are UNsollicited. I added the extra example to show that
knowing each other does not necessarily mean it is OK to send ads.

Alex

-------
Sender Policy Framework: http://www.openspf.org/
Archives at http://archives.listbox.com/spf-discuss/current/
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?list_id=735
Re: Perils of reputation [ In reply to ]
dtaylor at vocalabs
Feb 9, 2007, 8:00 AM
Post #6 of 11 (2745 views)
Permalink
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

There are value judgments here, and society as a whole is more forgiving
than yourself.

Alex van den Bogaerdt wrote:
> On Wed, Feb 07, 2007 at 01:30:43PM -0700, David MacQuigg wrote:
>> Interesting problem! The example is very helpful in defining the category
>> of email we are talking about. How about we call it "Solicited Commercial
>> Email" or SCE, instead of "cageliner".
>
> no problem here, but ...
>
>> This would include mail from
>> charitable and political organizations asking for donations,
>
> SPAM!
>
Noncommercial. UNE?
This is a particular category that, as a society, we have decided is
"good" enough to allow interrupting your dinner with a phone call. I
expect that if it comes down to legislation, your inbox will not be
considered more sacred than your dinner hour.

>> ads from
>> Walgreens,
>
> SPAM!
On the contrary, I get the Walgreen's ads, but it is because I signed up
for them (with verification in the loop and permission to send e-mail
advertisements). Definitely solicited, and they seem to be rather
careful about how they use their list, as does my local newspaper who
also sends me commercial e-mails.

>
>> and stock tips that might be something your broker signed you up
>> for, but you would like to see an authentication header to help you decide
>> whether to un-subscribe or report it as spam.
>
> Someone else signs me up without me asking? Obviously not a properly
> managed mailing list (confirmed opt-in). Who says it was my broker signing
> me up?
>
> Also: SPAM!
>
Here I would side with you Alex. Stock tips? There lies more scams and
trickery (and lies...) than any area of commerce since impotence treatments.


>
> If I leave my email address with somebody because they need to inform
> me about delivery of my new computer, and they then use it so send me
> ads for their new and improved car insurance product: SPAM!
>
> All examples you mentioned, and the extra one I did, have one thing in
> common: they are UNsollicited. I added the extra example to show that
> knowing each other does not necessarily mean it is OK to send ads.
>
Interestingly enough, I have never encountered this sort of situation.
Of course, I pay attention to the checkbox (that is nearly always there)
that asks if it's OK for them to send "related" e-mail. For 90% of
businesses the answer is *no*. If there isn't such a checkbox, I use a
tagged address. I haven't gotten spam to *any* of those tagged addresses
yet.

Maybe I'm just lucky, but it seems to me that the bad actors in this
game are more than ready to identify themselves as such by going around
the rules to begin with rather than trying to appear like they are
following the rules. This is why I am a big fan of technical compliance
checks (such as SPF) as a first level of spam filtering.

- --
Daniel Taylor VP Operations Vocal Laboratories, Inc.
dtaylor@vocalabs.com http://www.vocalabs.com/ (952)941-6580x203
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFzJqR8/QSptFdBtURApqNAJ4iaS1n1hBoGglFOeu5S1IlF5EZygCfWgyP
L0zeZaM0PGSsQi6yiSK+nGc=
=bFtt
-----END PGP SIGNATURE-----

-------
Sender Policy Framework: http://www.openspf.org/
Archives at http://archives.listbox.com/spf-discuss/current/
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?list_id=735
Re: Perils of reputation [ In reply to ]
alex at ergens
Feb 9, 2007, 8:23 AM
Post #7 of 11 (2738 views)
Permalink
On Fri, Feb 09, 2007 at 10:00:17AM -0600, Daniel Taylor wrote:

> There are value judgments here, and society as a whole is more forgiving
> than yourself.

Maybe so, maybe not. You yourself are using a value judgement, one which
could be wrong itself. Perhaps you are more forgiving than society as a
whole, and I am on par.

> Alex van den Bogaerdt wrote:
> > On Wed, Feb 07, 2007 at 01:30:43PM -0700, David MacQuigg wrote:
> >> Interesting problem! The example is very helpful in defining the category
> >> of email we are talking about. How about we call it "Solicited Commercial
> >> Email" or SCE, instead of "cageliner".
> >
> > no problem here, but ...
> >
> >> This would include mail from
> >> charitable and political organizations asking for donations,
> >
> > SPAM!
> >
> Noncommercial. UNE?
> This is a particular category that, as a society, we have decided is
> "good" enough to allow interrupting your dinner with a phone call. I
> expect that if it comes down to legislation, your inbox will not be
> considered more sacred than your dinner hour.

Which society? This is 'not done' over here where I live. And illegal
when using email (not sure about phone).

> >> ads from Walgreens,
> >
> > SPAM!
> On the contrary, I get the Walgreen's ads, but it is because I signed up
> for them (with verification in the loop and permission to send e-mail
> advertisements). Definitely solicited, and they seem to be rather
> careful about how they use their list, as does my local newspaper who
> also sends me commercial e-mails.

Ack. If you signed up, then it's solicited. That wasn't clear in the
message I replied to.

But if you gave your email address to your local newspaper in order to
receive "important messages" related to your subscription, I do not
consider their advertisements to be part of the deal.

Anyway, we seem to drift further and further away from the true topic
of this list: fighting email forgery using SPF. I think we should not
continue to define semantics of spam, uce, ube, "une" and such here.

cheers
alex

-------
Sender Policy Framework: http://www.openspf.org/
Archives at http://archives.listbox.com/spf-discuss/current/
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?list_id=735
Re: Perils of reputation [ In reply to ]
dtaylor at vocalabs
Feb 9, 2007, 9:42 AM
Post #8 of 11 (2743 views)
Permalink
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Alex van den Bogaerdt wrote:
> On Fri, Feb 09, 2007 at 10:00:17AM -0600, Daniel Taylor wrote:
>
>> There are value judgments here, and society as a whole is more forgiving
>> than yourself.
>
> Maybe so, maybe not. You yourself are using a value judgement, one which
> could be wrong itself. Perhaps you are more forgiving than society as a
> whole, and I am on par.
>
You'd be amazed the stuff that sets me off.

>> Alex van den Bogaerdt wrote:
>>> On Wed, Feb 07, 2007 at 01:30:43PM -0700, David MacQuigg wrote:
>>>> Interesting problem! The example is very helpful in defining the category
>>>> of email we are talking about. How about we call it "Solicited Commercial
>>>> Email" or SCE, instead of "cageliner".
>>> no problem here, but ...
>>>
>>>> This would include mail from
>>>> charitable and political organizations asking for donations,
>>> SPAM!
>>>
>> Noncommercial. UNE?
>> This is a particular category that, as a society, we have decided is
>> "good" enough to allow interrupting your dinner with a phone call. I
>> expect that if it comes down to legislation, your inbox will not be
>> considered more sacred than your dinner hour.
>
> Which society? This is 'not done' over here where I live. And illegal
> when using email (not sure about phone).
>
The United States national "do not call" list legislation specifically
excludes calls for political or charity fundraising purposes. To me that
sends a very clear message on the priorities of our legislators at a
minimum, and by extension their advisors and many of their constituents.

I disagree with this myself, so I continue to avoid the national list.

>>>> ads from Walgreens,
>>> SPAM!
>> On the contrary, I get the Walgreen's ads, but it is because I signed up
>> for them (with verification in the loop and permission to send e-mail
>> advertisements). Definitely solicited, and they seem to be rather
>> careful about how they use their list, as does my local newspaper who
>> also sends me commercial e-mails.
>
> Ack. If you signed up, then it's solicited. That wasn't clear in the
> message I replied to.
>
> But if you gave your email address to your local newspaper in order to
> receive "important messages" related to your subscription, I do not
> consider their advertisements to be part of the deal.
>
In both cases the checkboxes related to the different message types were
very clear. I let my local newspaper send me a limited subset of
advertisements as an experiment. So far they are sticking to what they
said they would.

> Anyway, we seem to drift further and further away from the true topic
> of this list: fighting email forgery using SPF. I think we should not
> continue to define semantics of spam, uce, ube, "une" and such here.
>
Ah, but there is a relevant point to this.
SPF as a technical measure helps separate good actors from bad actors up
front. In fact I consider SPF or similar sender authorisation to be a
minimal necessary precondition to building a reputation system, and
obvious attempts to bypass or dilute such a system to be just cause for
refusing to accept delivery of e-mail.

Mind you, business considerations prevent me from being so draconian at
work as I might prefer, but on my personal domain....

- --
Daniel Taylor VP Operations Vocal Laboratories, Inc.
dtaylor@vocalabs.com http://www.vocalabs.com/ (952)941-6580x203
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFzLJr8/QSptFdBtURAs+pAJwMsBlXrkbUhD81qaDqO7anq6s3dACePBV7
4hK2G/1R5KeyoYfgk5iNsAA=
=R7R1
-----END PGP SIGNATURE-----

-------
Sender Policy Framework: http://www.openspf.org/
Archives at http://archives.listbox.com/spf-discuss/current/
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?list_id=735
Re: Perils of reputation [ In reply to ]
nobody at xyzzy
Feb 9, 2007, 10:29 AM
Post #9 of 11 (2739 views)
Permalink
Daniel Taylor wrote:

>>> This would include mail from charitable and political
>>> organizations asking for donations,

>> SPAM!

> Noncommercial. UNE?
> This is a particular category that, as a society, we have decided is
> "good" enough to allow interrupting your dinner with a phone call.

AFAIK not in my society, but I'm not sure. In mail I'd "spamcop" it.

Frank


-------
Sender Policy Framework: http://www.openspf.org/
Archives at http://archives.listbox.com/spf-discuss/current/
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?list_id=735
Re: Perils of reputation - refocus [ In reply to ]
macquigg at box67
Feb 9, 2007, 12:04 PM
Post #10 of 11 (2725 views)
Permalink
At 05:23 PM 2/9/2007 +0100, Alex wrote:

>Anyway, we seem to drift further and further away from the true topic
>of this list: fighting email forgery using SPF. I think we should not
>continue to define semantics of spam, uce, ube, "une" and such here.

I'll agree with that, but I'm would like to hear more discussion on the
proposal that we should have a special category for SCE (or whatever we
want to call it). Discussion of reputation systems is a bit outside the
scope of SPF, but I believe the topic is relevant to SPF, because the true
value of SPF will only be realized when it is used in conjunction with a
reputation system, and the needs of reputation systems should influence
further development of SPF.

My initial opinion is that opening up a special category for SCE will blur
the line between spam and ham, and that *any* legitimate sender can avoid
getting near that line by simply sending a clearly-worded confirmation
email before adding an address to their list. That will take care of the
broker who sincerely understood that I had requested more "information".

There is still the practical problem that Stuart detailed at the start of
this thread. How can an automated reputation system deal with legitimate
senders of SCE (e.g. amazon.com)? My initial thoughts are that it can't be
automated, at least not for senders just getting started. Established
senders of SCE should have no difficulty *maintaining* a good reputation,
because any lowering of that reputation should be based on human feedback,
and there will be plenty.

New senders of SCE will probably need the help of an accreditation service,
or perhaps a "voucher" from their ESP or trade association. Or they can
simple relay their mail through their ESP's server, and rely on that ESP's
reputation to ensure delivery. Any of these options can be implemented in
a reputation system without establishing a separate category or procedures
for SCE. The accreditation service is treated the same as any rating
service, and will lose its credibility if it is not careful. Vouchers
simply say "treat this email as if it came from us". The vouching ID can
then be held responsible. Relaying requires the least change in status
quo. I am not aware of any deliver-ability problems in mail I have sent
via my two relays - yahoo.com and controlledmail.com. (I have not tried
any mass mailings, however).

-- Dave


-------
Sender Policy Framework: http://www.openspf.org/
Archives at http://archives.listbox.com/spf-discuss/current/
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?list_id=735
Re: Perils of reputation - refocus [ In reply to ]
william at elan
Feb 9, 2007, 1:41 PM
Post #11 of 11 (2728 views)
Permalink
On Fri, 9 Feb 2007, David MacQuigg wrote:

> At 05:23 PM 2/9/2007 +0100, Alex wrote:
>
>> Anyway, we seem to drift further and further away from the true topic
>> of this list: fighting email forgery using SPF. I think we should not
>> continue to define semantics of spam, uce, ube, "une" and such here.
>
> I'll agree with that, but I'm would like to hear more discussion on the
> proposal that we should have a special category for SCE (or whatever we want
> to call it). Discussion of reputation systems is a bit outside the scope of
> SPF, but I believe the topic is relevant to SPF, because the true value of
> SPF will only be realized when it is used in conjunction with a reputation
> system, and the needs of reputation systems should influence further
> development of SPF.

I agree that reputation should be take into account when talking
about and how to extend SPF but I'd disagree about "true value"
though obviously true value is in the eye of the beholder...

> My initial opinion is that opening up a special category for SCE will blur
> the line between spam and ham, and that *any* legitimate sender can avoid
> getting near that line by simply sending a clearly-worded confirmation email
> before adding an address to their list. That will take care of the broker
> who sincerely understood that I had requested more "information".

Reputation data request about certain email is basically request about
some data where you do not recognize the sender. So instead of
talking about "SCE" as you name and some special arrangements for
known ones (which in the end will depend on reputation vendor)
what you should have is that if you know that email came from
known entity that you requested to receive emails from, then
this email should be whitelisted based on user preferences.

Though this is more of the topic for ASRG (which is due to current
management is largely not productive and as a result sees very
little discussion on reputation schemes or ideas for anti-spam)
what you should really have is mechanism that user could confirm
his request to be subscribed to email vendor which then allows for
easy whitelisting on the user side. One option is using encrypted
confirmation response. Lets assume that user 'david@example.com'
has private key KEY for use with symmetric key encryption system.
The user receives a request for confirmation from 'ESP@example.net'
and what he does is create encrypted token for example:
TOKEN=sha1(KEY,'david@example.com','ESP@example.com')
and replies to the provider with new email address for subscription
from new provider-specific address
TOKEN#david@example.com
Now if ESP@example.com sends new email to
TOKEN#david@example.com
then mail system could at the SMTP session time verify that request
came from confirmed ESP by doing same SHA1 calculation using given
MAIL FROM and RCPT TO addresses.

This is not the only way to do something like this obviously but
if you do it as above you can easily whitelist known good senders
without having to maintain large list of those senders and so after
you verified using SPF or that true sender is really ESP@example.net
you avoid having the email subjected to further reputation checks.

> There is still the practical problem that Stuart detailed at the start of
> this thread. How can an automated reputation system deal with legitimate
> senders of SCE (e.g. amazon.com)? My initial thoughts are that it can't be
> automated, at least not for senders just getting started. Established
> senders of SCE should have no difficulty *maintaining* a good reputation,
> because any lowering of that reputation should be based on human feedback,
> and there will be plenty.
>
> New senders of SCE will probably need the help of an accreditation service,
> or perhaps a "voucher" from their ESP or trade association. Or they can
> simple relay their mail through their ESP's server, and rely on that ESP's
> reputation to ensure delivery. Any of these options can be implemented
> in a reputation system without establishing a separate category or
> procedures for SCE.

You're mixing some concepts & systems above. Reputation system is one
concept but what you're really talking about above is comprehensive
anti-spam system. Such comprehensive system would use results from
both accreditation/whitelisting and reputation checks.

> The accreditation service is treated the same as any rating service,
> and will lose its credibility if it is not careful.

Yes, reputation rating of accreditation providers is concept that
has been mentioned. I suspect that is way in the future though.

> Vouchers simply say "treat this email as if it came from us". The
> vouching ID can then be held responsible. Relaying requires the
> least change in status quo. I am not aware of any deliver-ability
> problems in mail I have sent via my two relays - yahoo.com and
> controlledmail.com. (I have not tried any mass mailings, however).

I suspect that vouchers would still have to be reputation database
maintained in different way then normal reputation database of
identified senders even though as you point out when you
"treat this email as if it came from us" you can maintain together.
My view of this is that if you base reputation not on just one
name but on (name, identity_type) pair where 'identity_type'
is "MAIL FROM" or "EHLO", etc - then the reputation of the
accreditor is in fact type of identity with name being replaced
with "accreditation provider name", i.e. you'd have pair
("accreditor.example", "ACR") in your reputation database.

--
William Leibzon
Elan Networks
william@elan.net

-------
Sender Policy Framework: http://www.openspf.org/
Archives at http://archives.listbox.com/spf-discuss/current/
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?list_id=735

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal