Mailing List Archive

On Tuesday 02 December 2008 00:37, Jeff Mills wrote:
> Firstly, the reason I'm having to whitelist a domain is because python
> policyd times out when checking an email sent from aol. Not sure why
> this is - possibly because of a very long SPF record?

aol.com has a record big enough that it won't generally fit in a UDP packet
and so it has to fall back to TCP. This is legal, but not recommended by RFC
4408.

Some things to check:

Make sure you have the current pyspf (python-spf) and pydns (python-dns). You
need pyspf 2.0.5 (or 2.0.4-4 in Debian/Ubuntu) and pydns at least 2.3.1 (and
you really want 2.3.3 for security reasons). Earlier versions did not
support TCP fallback.

If those are current, then consider using tcpdump or wireshark to capture the
DNS traffic. Some firewalls block DNS on TCP by default. If it's just AOL,
then I'm reasonably certain it's something to do with TCP fallback and not
the Python policy server.

Scott K


-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/1020/=now
RSS Feed: https://www.listbox.com/member/archive/rss/1020/
Powered by Listbox: http://www.listbox.com

On Tuesday 02 December 2008 00:37, Jeff Mills wrote:
> Because of this, I switched to perl policyd only to find out that it
> times out when checking an email sent from citigroup.com
>
> My guess with the timeout on citigroup is because they have
> underscores in the hostnames of their SPF records:
> ;; ANSWER SECTION:
> citigroup.com.          3347    IN      TXT     "v=spf1
> a:1._spf.citigroup.com a:2._spf.citigroup.com
> redirect=ext1._spf.citigroup.com"

Works here with the Perl policy server. My prime suspect is a firewall that's
trying to help you because it thinks the underscores aren't valid (they are
valid domain names, but not valid hostnames). Once again, I suggest
wireshark or tcpdump to capture some data off the wire.

Scott K


-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/1020/=now
RSS Feed: https://www.listbox.com/member/archive/rss/1020/
Powered by Listbox: http://www.listbox.com

On Tue, Dec 2, 2008 at 6:46 PM, Scott Kitterman <scott@kitterman.com> wrote:
> On Tuesday 02 December 2008 00:37, Jeff Mills wrote:
>> Firstly, the reason I'm having to whitelist a domain is because python
>> policyd times out when checking an email sent from aol. Not sure why
>> this is - possibly because of a very long SPF record?
>
> aol.com has a record big enough that it won't generally fit in a UDP packet
> and so it has to fall back to TCP. This is legal, but not recommended by RFC
> 4408.
>
> Some things to check:
>
> Make sure you have the current pyspf (python-spf) and pydns (python-dns). You
> need pyspf 2.0.5 (or 2.0.4-4 in Debian/Ubuntu) and pydns at least 2.3.1 (and
> you really want 2.3.3 for security reasons). Earlier versions did not
> support TCP fallback.
>
> If those are current, then consider using tcpdump or wireshark to capture the
> DNS traffic. Some firewalls block DNS on TCP by default. If it's just AOL,
> then I'm reasonably certain it's something to do with TCP fallback and not
> the Python policy server.
>
> Scott K
>

Thanks Scott,
Looks like I have the right versions installed, so unfortunately its
time to go digging.

[ Searching for package 'pydns' in all categories among: ]
* installed packages
[I--] [ ~] dev-python/pydns-2.3.3 (0)

[ Searching for package 'pyspf' in all categories among: ]
* installed packages
[I--] [ ~] dev-python/pyspf-2.0.5 (0)


--
-----
regards,
Jeff Mills
http://www.australianpipebands.com
http://www.rightlanehogs.com


-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/1020/=now
RSS Feed: https://www.listbox.com/member/archive/rss/1020/
Powered by Listbox: http://www.listbox.com

On Wed, 3 Dec 2008 08:52:32 +1100 "Jeff Mills" <hybiepoo@gmail.com> wrote:
>On Tue, Dec 2, 2008 at 6:46 PM, Scott Kitterman <scott@kitterman.com>
wrote:
>> On Tuesday 02 December 2008 00:37, Jeff Mills wrote:
>>> Firstly, the reason I'm having to whitelist a domain is because python
>>> policyd times out when checking an email sent from aol. Not sure why
>>> this is - possibly because of a very long SPF record?
>>
>> aol.com has a record big enough that it won't generally fit in a UDP
packet
>> and so it has to fall back to TCP. This is legal, but not recommended
by RFC
>> 4408.
>>
>> Some things to check:
>>
>> Make sure you have the current pyspf (python-spf) and pydns
(python-dns). You
>> need pyspf 2.0.5 (or 2.0.4-4 in Debian/Ubuntu) and pydns at least 2.3.1
(and
>> you really want 2.3.3 for security reasons). Earlier versions did not
>> support TCP fallback.
>>
>> If those are current, then consider using tcpdump or wireshark to
capture the
>> DNS traffic. Some firewalls block DNS on TCP by default. If it's just
AOL,
>> then I'm reasonably certain it's something to do with TCP fallback and
not
>> the Python policy server.
>>
>> Scott K
>>
>
>Thanks Scott,
>Looks like I have the right versions installed, so unfortunately its
>time to go digging.
>
>[ Searching for package 'pydns' in all categories among: ]
> * installed packages
>[I--] [ ~] dev-python/pydns-2.3.3 (0)
>
>[ Searching for package 'pyspf' in all categories among: ]
> * installed packages
>[I--] [ ~] dev-python/pyspf-2.0.5 (0)

I assume you looked in your mail logs and there are no errors of note?

If you can capture the attempt with tcpdump or wireshark, I'll have a look.
Send it off list if you prefer.

At this point my money is on router/firewall confguration.

Scott K


-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/1020/=now
RSS Feed: https://www.listbox.com/member/archive/rss/1020/
Powered by Listbox: http://www.listbox.com

>
> I assume you looked in your mail logs and there are no errors of note?
>
> If you can capture the attempt with tcpdump or wireshark, I'll have a look.
> Send it off list if you prefer.
>
> At this point my money is on router/firewall confguration.
>
> Scott K
>
>

The mail logs show nothing except for the spf timeout.
I just did some more manual testing.
If I use the following, the query can take up to 10 mins:
request=smtpd_access_policy
protocol_state=RCPT
protocol_name=SMTP
helo_name=mail.citigroup.com
queue_id=8045F2AB23
instance=12345.6789
sender=someuser@dinersclub.com.au
recipient=someuser@somedomain.com
client_address=192.193.221.103
client_name=dinersclub.com.au

10:13:00.510856 IP 10.100.2.240.63370 > exchange.domain: 60215+ TXT?
mail.citigroup.com. (36)
10:13:00.512634 IP 10.100.2.240.56945 > exchange.domain: 3832+ A?
mail.citigroup.com. (36)
10:13:19.815489 IP 10.100.2.240.15441 > exchange.domain: 7567+ TXT?
mail.citigroup.com. (36)
10:13:19.816997 IP 10.100.2.240.2577 > exchange.domain: 17004+ A?
mail.citigroup.com. (36)
10:15:00.944531 IP 10.100.2.240.61705 > exchange.domain: 28011+ TXT?
mail.citigroup.com. (36)
10:15:00.953546 IP 10.100.2.240.24226 > exchange.domain: 63835+ A?
mail.citigroup.com. (36)

If I change helo_name to citigroup.com instead of mail.citigroup.com,
the response is immediate

proxy2 millsj # policyd-spf
request=smtpd_access_policy
protocol_state=RCPT
protocol_name=SMTP
helo_name=citigroup.com
queue_id=8045F2AB23
instance=12345.6789
sender=someuser@dinersclub.com.au
recipient=someuser@somedomain.com
client_address=192.193.221.103
client_name=dinersclub.com.au

action=prepend Received-SPF: Pass (sender SPF authorized)
identity=helo; client-ip=192.193.221.103; helo=citigroup.com;
envelope-from=someuser@dinersclub.com.au;
receiver=someuser@somedomain.com

10:00:43.768132 IP 10.100.2.240.21745 > exchange.domain: 2978+ TXT?
citigroup.com. (31)
10:00:43.769410 IP 10.100.2.240.5208 > exchange.domain: 62770+ A?
1._spf.citigroup.com. (38)
10:00:43.769942 IP exchange.domain > 10.100.2.240.5208: 62770 18/0/0 A
smtp8.citigroup.com[|domain]


In the tcpdump I'm only requesting port 53.
Should I be looking at more data than that?


> -------------------------------------------
> Sender Policy Framework: http://www.openspf.org
> Modify Your Subscription: http://www.listbox.com/member/
> Archives: https://www.listbox.com/member/archive/1020/=now
> RSS Feed: https://www.listbox.com/member/archive/rss/1020/
> Powered by Listbox: http://www.listbox.com
>



--
-----
regards,
Jeff Mills
http://www.australianpipebands.com
http://www.rightlanehogs.com


-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/1020/=now
RSS Feed: https://www.listbox.com/member/archive/rss/1020/
Powered by Listbox: http://www.listbox.com

>
> I assume you looked in your mail logs and there are no errors of note?
>
> If you can capture the attempt with tcpdump or wireshark, I'll have a look.
> Send it off list if you prefer.
>
> At this point my money is on router/firewall confguration.
>
> Scott K
>

Ok, well I solved the issue by removing our internal DNS servers from
my mail host.
I'm now querying external servers and both of my issues went away.

So either Microsoft DNS has issues with SPF, or our PIX is blocking something.
I have dns-guard enabled on the firewall as well as a dns packet size
limit of 4096.



>
> -------------------------------------------
> Sender Policy Framework: http://www.openspf.org
> Modify Your Subscription: http://www.listbox.com/member/
> Archives: https://www.listbox.com/member/archive/1020/=now
> RSS Feed: https://www.listbox.com/member/archive/rss/1020/
> Powered by Listbox: http://www.listbox.com
>



--
-----
regards,
Jeff Mills
http://www.australianpipebands.com
http://www.rightlanehogs.com


-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/1020/=now
RSS Feed: https://www.listbox.com/member/archive/rss/1020/
Powered by Listbox: http://www.listbox.com

On Tuesday 02 December 2008 18:52, Jeff Mills wrote:
> > I assume you looked in your mail logs and there are no errors of note?
> >
> > If you can capture the attempt with tcpdump or wireshark, I'll have a
> > look. Send it off list if you prefer.
> >
> > At this point my money is on router/firewall confguration.
> >
> > Scott K
>
> Ok, well I solved the issue by removing our internal DNS servers from
> my mail host.
> I'm now querying external servers and both of my issues went away.
>
> So either Microsoft DNS has issues with SPF, or our PIX is blocking
> something. I have dns-guard enabled on the firewall as well as a dns packet
> size limit of 4096.

Glad to hear it.

It's hard for me to pick. Generally if there's PIX involved, I'm going to
blame it. If you're up for a test it'd be worthwhile to switch back and then
disable dns-guard. If that's the problem, I'd guess Cisco would be
interested in a bug report. Because of their interest in DKIM, I expect
they'd actually be responsive on fixing TXT record related issues.

What platform are you running the policy server on and did you get it through
your distro/OS packaging system or did your install from source?

Scott K


-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/1020/=now
RSS Feed: https://www.listbox.com/member/archive/rss/1020/
Powered by Listbox: http://www.listbox.com

>
> Glad to hear it.
>
> It's hard for me to pick. Generally if there's PIX involved, I'm going to
> blame it. If you're up for a test it'd be worthwhile to switch back and then
> disable dns-guard. If that's the problem, I'd guess Cisco would be
> interested in a bug report. Because of their interest in DKIM, I expect
> they'd actually be responsive on fixing TXT record related issues.
>
> What platform are you running the policy server on and did you get it through
> your distro/OS packaging system or did your install from source?
>
> Scott K
>

Ok, so I installed pypolicyd on another two servers.
The first one was on the inside network, and it queried the internal
DNS servers fine.
This still supported the theory of the firewall blocking the queries.
I installed pypolicyd on another host in the DMZ and set it to query
the internal DNS servers.
It also had no problems, so I'm not sure this is a firewall issue.

I'm running Gentoo and I have installed via the Gentoo Portage System,
but I have also tried installing pypolicyd-spf from source.


--
-----
regards,
Jeff Mills
http://www.australianpipebands.com
http://www.rightlanehogs.com


-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/1020/=now
RSS Feed: https://www.listbox.com/member/archive/rss/1020/
Powered by Listbox: http://www.listbox.com

On Wed, Dec 3, 2008 at 11:12 AM, Scott Kitterman <scott@kitterman.com> wrote:
> On Tuesday 02 December 2008 18:52, Jeff Mills wrote:
>> > I assume you looked in your mail logs and there are no errors of note?
>> >
>> > If you can capture the attempt with tcpdump or wireshark, I'll have a
>> > look. Send it off list if you prefer.
>> >
>> > At this point my money is on router/firewall confguration.
>> >
>> > Scott K
>>
>> Ok, well I solved the issue by removing our internal DNS servers from
>> my mail host.
>> I'm now querying external servers and both of my issues went away.
>>
>> So either Microsoft DNS has issues with SPF, or our PIX is blocking
>> something. I have dns-guard enabled on the firewall as well as a dns packet
>> size limit of 4096.
>
> Glad to hear it.
>
> It's hard for me to pick. Generally if there's PIX involved, I'm going to
> blame it. If you're up for a test it'd be worthwhile to switch back and then
> disable dns-guard. If that's the problem, I'd guess Cisco would be
> interested in a bug report. Because of their interest in DKIM, I expect
> they'd actually be responsive on fixing TXT record related issues.
>
> What platform are you running the policy server on and did you get it through
> your distro/OS packaging system or did your install from source?
>
> Scott K
>
>

One final update.
The firewall was only allowing UDP DNS requests for the host in
question. TCP DNS requests were not allowed.
Fixed!

Thanks for your help Scott.


> -------------------------------------------
> Sender Policy Framework: http://www.openspf.org
> Modify Your Subscription: http://www.listbox.com/member/
> Archives: https://www.listbox.com/member/archive/1020/=now
> RSS Feed: https://www.listbox.com/member/archive/rss/1020/
> Powered by Listbox: http://www.listbox.com
>



--
-----
regards,
Jeff Mills
http://www.australianpipebands.com
http://www.rightlanehogs.com


-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/1020/=now
RSS Feed: https://www.listbox.com/member/archive/rss/1020/
Powered by Listbox: http://www.listbox.com