Mailing List Archive

Google's SPF Record
At 01:04 AM 12/22/2007 +0000, you wrote:
>WebMaster@commerco.net wrote:
>> Getting back to the original point of the thread, why Google
>> apparently wants folks to specify "~all" rather than "-all", perhaps
>> in their case (because they offer a huge email service), they don't
>> wish to reveal all the possible outgoing SMTP servers to avoid some
>> type of attack on GMail. Personally, I think there are better ways
>> of handling such things even in huge scale email service environments.
>
>An interesting theory of yours, but I think if that was their motivation,
>they would have been bright enough to use an "exists:" mechanism to hide
>their infrastructure. :-)

Actually, I was only half joking in my last post. While there may be some "propaganda" motivation in these huge SPF records we see from some large companies, I don't think Google is playing that game. They want to avoid abuse of their servers, but they don't want to make it difficult for new users to sign up for their email service. Putting restrictions on new accounts, like rate-limits which would discourage spammers, is not something they are willing to do. The only thing they require for a new account is a "word verification", i.e. being able to type the word you see on their signup screen.

Originally, their SPF record showed: 'v=spf1 ptr ?all', which had the effect of authorizing all of their netblocks. That resulted in a huge amount of spam from their authorized servers. Sending them spam reports would only get an automated denial - "someone is forging our name". Following up with "No its not a forgery. It came from a server authorized by the PTR method you specified." got only a discussion with a robot whose job was apparently to sandbag complaints. Twice we had to take google.com off our whitelist, and they have remained in that status since July 07. A check of our recent log files shows we are still getting lots of spam from their servers.

I see now they have changed their SPF record from ptr to an explicit list of netblocks. 9 blocks totaling 147456 IP addresses is still way too much, but we need to encourage them in this effort. If I were Google, and really wanting to be a good citizen, I would authorize maybe 8 blocks of 8 addresses each to say HELO this is google.com. This should be more than enough for a worldwide network of well-connected servers. Then I would move the new accounts to a different HELO name, limit the new accounts to maybe 100 recipients per day, and watch for patterns of abuse. After a few weeks of normal use, a new account could be migrated to one of the regular servers.

Google, are you listening?

-- Dave
************************************************************ *
* David MacQuigg, PhD email: macquigg at open-mail.org * *
* President, Open-Mail dot org phone: USA 520-721-4583 * * *
* Postmaster, Box67 dot com * * *
* 9320 East Mikelyn Lane * * *
* http://purl.net/macquigg Tucson, Arizona 85710 *
************************************************************ *

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=78780266-1a1ea7
Powered by Listbox: http://www.listbox.com
Re: Google's SPF Record [ In reply to ]
David MacQuigg wrote:

> Google, are you listening?

Maybe not :-) I think they reject FAIL, that's IMO far more
worth than publishing FAIL, or talking about FAIL in an FAQ
for new google apps users: The decision to publish FAIL is
not trivial, it's too complex for a simple Google apps FAQ.

See also <http://www.openspf.org/Frank_Ellermann/Google>, if
it's generally interesting I could move it to an "ordinary"
openspf page.

Frank

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=78841658-c7dc9b
Powered by Listbox: http://www.listbox.com
Re: Google's SPF Record [ In reply to ]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Frank Ellermann wrote:
> I think [Google] reject FAIL, [...]
>
> See also <http://www.openspf.org/Frank_Ellermann/Google>, if it's
> generally interesting I could move it to an "ordinary" openspf page.

Before we consider that, how can we be sure that the rejection demons-
trated on that page is actually due to an SPF Fail?

| Apparently Gmail rejects SPF FAIL after DATA:
|
| 20071104 21:21:55 TCP connection with GMAIL-SMTP-IN.L.GOOGLE.COM:25
| 21:55.97 220 mx.google.com ESMTP y37si6730238iky
| 21:55.97 ehlo xyzzy.dnsalias.org
| 21:56.12 250-mx.google.com at your service, [62.134.88.223]
| [...]
| 21:56.12 mail from:<nobody@xyzzy.claranet.de>
| 21:56.26 250 2.1.0 OK
| 21:56.26 rcpt to:<hmdmhdfmhdjmzdtjmzdtzktdkztdjz@gmail.com>
| 21:59.88 250 2.1.5 OK
| 21:59.88 data
| 21:59.99 354 Go ahead
| 22:00.29 .
| 22:01.99 550-5.7.1 [62.134.88.223] The IP you're using to send email is not authorized
| 22:01.99 550-5.7.1 to send email directly to our servers. Please use
| 22:01.99 550 5.7.1 the SMTP relay at your service provider instead. [...]

Yes, their SMTP and DSN status codes match what RFC 4408 recommends for
rejections due to SPF Fail, but I looked into the definitions of these
status codes and it's very well possible that they and the SMTP response
text are due to some other IP-based policy (e.g., a dynamic IP address
blacklist).

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFHbXz9wL7PKlBZWjsRAmVoAJ9Xjobj8Ii03R0Gcyk74/J92+rIYwCeOZgR
uOkSshCcoPx23jDFVsbKIm4=
=URAP
-----END PGP SIGNATURE-----

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=78857372-954cd7
Powered by Listbox: http://www.listbox.com
Re: Google's SPF Record [ In reply to ]
Julian Mehnle wrote:

> how can we be sure that the rejection demons- trated on that
> page is actually due to an SPF Fail?


Well, "apparently" isn't the same "surely" :-) Obviously they
looked at the DATA for their decision, so they likely use more
than only one indicator, otherwise they'd be wasting bandwidth.

And "more than one indicator" could include Dyn IP lists, or
looking for a DKIM PASS in the DATA overruling SPF FAIL, or any
combination up to sets where SPF FAIL plays no role at all.

BUT. But check out the first link on my SPF "Google" page:
<http://www.google.com/mail/help/fightspam/spamexplained.html>

| Gmail supports multiple authentication systems, including SPF
| (Sender Policy Framework), DomainKeys, and DKIM (DomainKeys
| Identified Mail), so we can be more certain that your mail is
| from who it says it's from. Also, unlike many other providers
| that automatically let through all mail from certain senders,
| making it possible for their messages to bypass spam filters,
| Gmail puts all senders through the same rigorous checks.

*They* claim that they use SPF (among others), and what I got
confirms it. Admittedly my evidence doesn't prove it, in fact
they obviously don't reject SPF FAIL a.s.a.p., they go to the
trouble to look at the DATA.

Frank

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=78862944-ce201e
Powered by Listbox: http://www.listbox.com
Re: Google's SPF Record [ In reply to ]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Frank Ellermann wrote:
> Julian Mehnle wrote:
> > how can we be sure that the rejection demonstrated on that page
> > is actually due to an SPF Fail?
> >
> > | Apparently Gmail rejects SPF FAIL after DATA:
>
> Well, "apparently" isn't the same "surely" :-) Obviously they looked at
> the DATA for their decision, so they likely use more than only one
> indicator, otherwise they'd be wasting bandwidth.

For the record: It's possible that they simple always reject only after
DATA in order to collect as much information as possible on all messages,
e.g. for statistics. If they rejected SPF Fails before DATA, their spam
stats would get distorted.

Anyway, I think we can only be about 90% certain that they actually reject
due to SPF Fail (be it before or after DATA). There _might_ be other
reasons for the rejection you observed. We could ask them to know it for
sure, though.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFHbYqIwL7PKlBZWjsRAvfTAKC8Ej0mJr3+uU3O8Vm8Pt3aefsmigCfcMZH
FqYUmdcUqpYe0L9AO7r3aM8=
=kA1b
-----END PGP SIGNATURE-----

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=78865358-52182f
Powered by Listbox: http://www.listbox.com
Re: Google's SPF Record [ In reply to ]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Julian Mehnle wrote:
> Frank Ellermann wrote:
> > I think [Google] reject FAIL, [...]
> >
> > See also <http://www.openspf.org/Frank_Ellermann/Google>, if it's
> > generally interesting I could move it to an "ordinary" openspf page.
>
> Before we consider that, how can we be sure that the rejection demons-
> trated on that page is actually due to an SPF Fail?

Here's an indication to the contrary from 2007-12-03:

http://groups.google.com/group/Gmail-ABCs/browse_thread/thread/86ce42bd8ddf2bd0/dbc3e8c800be5cbe?lnk=gst&q=

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFHbZQIwL7PKlBZWjsRAqOVAKDXNQDZyD6K5ywDAENRUp+9aS/Q1ACdFoCZ
j3Grug7gsF/dETfjmbIWZMQ=
=IvZM
-----END PGP SIGNATURE-----

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=78867981-6d2418
Powered by Listbox: http://www.listbox.com
Re: Google's SPF Record [ In reply to ]
Julian Mehnle wrote:

> Here's an indication to the contrary from 2007-12-03:

From the test description it's *possibly* a "DKIM PASS
overrules SPF FAIL" case. I've posted a question about
this in the Google group.

Frank

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Archives: http://v2.listbox.com/member/archive/735/=now
RSS Feed: http://v2.listbox.com/member/archive/rss/735/
Modify Your Subscription: http://v2.listbox.com/member/?member_id=1311532&id_secret=78903303-7c1a4b
Powered by Listbox: http://www.listbox.com