Mailing List Archive

from the trenches: sender authentication relevance fading, DNSBLs and SpamAssassin rule
Hello All,

A little update from the trenches. I last was seen here in
mid-2004 (search the archive for "lawless"). At the time, like
everyone else, I was drowning in spam and desperate for a
solution. I was able to establish a private 'sendmail' MTA
running a customized version of the 'perl' SPF milter. The
customizations had mainly to do with increasing the
aggressiveness of the milter's rejection policies.

Since then the customized SPF-derived blocker has stopped tens
of thousands of UCE messages from entering my in-box. Hurray!

Around December of 2004 I also started adding DNSBLs to the
'sendmail' configuration. The DNSBLs were and are good at
blocking spam from senders with relatively stationary IP
addresses--precisely the sort of spammers that might pass even
an aggressivized SPF check. SPF continued to block garbage
emanating from zombie PCs since the operators of botnets rarely
bother to arrange SPF compliance.

In general the SORBS lists have not been useful as they
frequently block legitimate mail relays. However the SORBS DHUL
list has been most helpful. It lists dynamic IP DOCSIS and DSL
address as well as dial-up IPs and blocks much spam originating
from zombie PCs. So 'dul.dnsbl.sorbs.net' has been an important
block list.

Earlier this year Spamhaus added a similar list. The quality of
Spamhaus lists has always been excellent, and the new "Policy
Block List" is no exception. It's more comprehensive than the
SORBS DUHL list by far.

So the combined Spamnhaus lists (zen.spamhaus.org), the SORBS
DUHL and the SpamCop blocklist form the first line of my spam
defense. SPF is the second barricade against the unwashed hords
after the DNSBLs. Finally SpamAssassin managed by a third-party
forms the rearguard.

What's interesting is that since the Spamhaus PBL went into
place and especially over the last two months or so, the amount
of spam escaping the DNSBL layer has dropped to almost
zero--with Spamhaus stopping the vast majority. SpamAssassin
picks off what's left and SPF nets zilch. A couple of weeks
back I switched the SPF component to tag failed messages rather
than bounce them as most of the SPF hits were false positives.

I'm not quite ready to declare sender authentication dead;
who knows what the next year will bring? However it's looking
bleak. Sender authentication requires substantial effort.
I don't see people (myself included) putting in that effort if
a handful of easily configured high-quality DNSBLs block 99.9%
of the spam.

Critical mass has gone to SPAMHAUS. Now so may MTAs use this
list that anyone on it in error *has* to go to whatever effort
it takes to be delisted. With SPF so few MTAs make use of it
that mis-configured SPF records abound. I had accidently missed
adding a new relay to my domain's SPF record for months before
catching it. No e-mail bounces arrived to clue me in, though
perhaps some messages were grey-listed. Without crisp feedback
about errors, incorrect SPF records are likely to continue to
accumulate further weakening the benefits of SPF in a viscous
rather than virtuous cycle.

Regards,

David

-------------------------------------------
-----------------------------------------------------------------------
Sender Policy Framework: http://www.openspf.org/
Archives at http://archives.listbox.com/spf-discuss/current/
To unsubscribe, change your address, or temporarily deactivate your
subscription,
please go to http://v2.listbox.com/member/?list_id=735
Powered by Listbox: http://www.listbox.com
Re: from the trenches: sender authentication relevance fading, DNSBLs and SpamAssassin rule [ In reply to ]
On Jun 17, 2007, at 1:55 PM, David Lawless wrote:

>
> I'm not quite ready to declare sender authentication dead;
> who knows what the next year will bring? However it's looking
> bleak. Sender authentication requires substantial effort.
> I don't see people (myself included) putting in that effort if
> a handful of easily configured high-quality DNSBLs block 99.9%
> of the spam.

Thanks for the insightful anecdote. I'm back, too, from being away
since around the same time.

And you're right that DNSBL quality has improved significantly.
Spamhaus are doing a fantastic job. There's an impressive amount of
coverage out there. Interestingly, though, there is also less
overlap than one might expect.

https://my.karmasphere.com/app/visibl/composite?
composite_id=karmasphere.email-sender


Now, blocking the bad stuff has always been the primary goal of the
antispam cause.

But in reaching that goal, new problems have been created.

I've talked to people both inside and outside the email world, and
their biggest complaint is now false positives.

Lots of legitimate messages aren't getting through when they should.
The very existence of the "deliverability" industry is testament to
this.

The picture that comes to mind is the poor person scrolling through
the spam folder once a day or once a week.

That picture has a caption on it that reads YOU'RE DOING IT WRONG. Heh.

How do we fix this?

In the last few years SPF adoption has done very well; DKIM is coming
along nicely.

I want to harness authentication to fight false positives.

That's why I've been on a campaign lately to acquire as many domain
whitelists as possible.

I'll put them together as a feedset on Karmasphere; once that's
ready, maybe we can find a way to integrate SPF checks with domain
whitelist lookups. I know there are Perl plugins for both. Postfix
is my favourite MTA, so I might start by patching whatever postfix
policyd is current...

What do you think?

-------------------------------------------
-----------------------------------------------------------------------
Sender Policy Framework: http://www.openspf.org/
Archives at http://archives.listbox.com/spf-discuss/current/
To unsubscribe, change your address, or temporarily deactivate your
subscription,
please go to http://v2.listbox.com/member/?list_id=735
Powered by Listbox: http://www.listbox.com
Re: from the trenches: sender authentication relevance fading, DNSBLs and SpamAssassin rule [ In reply to ]
On Jun 17, 2007, at 3:43 PM, Meng Weng Wong wrote:
>> bleak. Sender authentication requires substantial effort.
>> I don't see people (myself included) putting in that effort if
>> a handful of easily configured high-quality DNSBLs block 99.9%
>> of the spam.
>
> And you're right that DNSBL quality has improved significantly.
> Spamhaus are doing a fantastic job. There's an impressive amount
> of coverage out there. Interestingly, though, there is also less
> overlap than one might expect.
>

Oh, and by the way, from what I've seen of the backchannels in the
DNSBL world where folks decide which IP ranges go get blacklisted,
SPF records are often very useful: many spammers publish them, and it
makes the DNSBL maintainer's job a whole lot simpler when they do.

So SPF is, in some way, actually contributing to the quality of
DNSBLs that you're witnessing. In a way they are just doing what
you're doing ahead of time.

-------------------------------------------
-----------------------------------------------------------------------
Sender Policy Framework: http://www.openspf.org/
Archives at http://archives.listbox.com/spf-discuss/current/
To unsubscribe, change your address, or temporarily deactivate your
subscription,
please go to http://v2.listbox.com/member/?list_id=735
Powered by Listbox: http://www.listbox.com
Re: from the trenches: sender authentication relevance fading, DNSBLs and SpamAssassin rule [ In reply to ]
On Sun, 17 Jun 2007 15:43:44 -0700 Meng Weng Wong <mengwong@pobox.com>
wrote:

>I'll put them together as a feedset on Karmasphere; once that's
>ready, maybe we can find a way to integrate SPF checks with domain
>whitelist lookups. I know there are Perl plugins for both. Postfix
>is my favourite MTA, so I might start by patching whatever postfix
>policyd is current...
>
>What do you think?
>
The current version of the Perl policy server (we have a Python one too now) has a very
rudimentary whitelist handler that I think would be appropriate to build this out from.

http://www.openspf.org/Software

Scott K

-------------------------------------------
-----------------------------------------------------------------------
Sender Policy Framework: http://www.openspf.org/
Archives at http://archives.listbox.com/spf-discuss/current/
To unsubscribe, change your address, or temporarily deactivate your
subscription,
please go to http://v2.listbox.com/member/?list_id=735
Powered by Listbox: http://www.listbox.com