Mailing List Archive

Dec 8, 2008, 1:23 PM

Post #2 of 9 (4967 views)

On Mon, 8 Dec 2008 14:45:07 +0000 "Simon Jones" <simonmjones@gmail.com>
wrote:
>Hi all, I've enabled spf policy checking in postfix but it's maxing
>out the server - to the point that it's unable to process any mail.
>is there a way to set the limit on the number of perl processes called
>or a way to tweak the settings to stop it maxing out?
>
It's a function of the number of Postfix smtpd processes.

What's the output of postconf -n?

Scott K

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/1020/=now
RSS Feed: https://www.listbox.com/member/archive/rss/1020/
Powered by Listbox: http://www.listbox.com

Re: perl processes [ In reply to ]

Dec 9, 2008, 4:04 AM

Post #3 of 9 (4991 views)

2008/12/8 Scott Kitterman <scott@kitterman.com>:
> On Mon, 8 Dec 2008 14:45:07 +0000 "Simon Jones" <simonmjones@gmail.com>
> wrote:
>>Hi all, I've enabled spf policy checking in postfix but it's maxing
>>out the server - to the point that it's unable to process any mail.
>>is there a way to set the limit on the number of perl processes called
>>or a way to tweak the settings to stop it maxing out?
>>
> It's a function of the number of Postfix smtpd processes.
>
> What's the output of postconf -n?
>
> Scott K
>
>
Hi Scott,

address_verify_map = btree:/etc/postfix/verify
command_directory = /usr/sbin
config_directory = /etc/postfix
daemon_directory = /usr/libexec/postfix
debug_peer_level = 5
default_process_limit = 80
disable_vrfy_command = yes
fast_flush_domains = $relay_domains
header_checks = regexp:/etc/postfix/header_checks
html_directory = no
in_flow_delay = 1s
local_recipient_maps =
luser_relay = /dev/null
mail_spool_directory = /var/spool/mail
mailbox_size_limit = 1024000000
mailq_path = /usr/bin/mailq
manpage_directory = /usr/local/man
message_size_limit = 80000000
mynetworks = /etc/postfix/network_table
mynetworks_style = host
myorigin = smtprev.saqnet.co.uk
newaliases_path = /usr/bin/newaliases
queue_minfree = 120000000
readme_directory = no
relay_domains = hash:/etc/postfix/backup_domains
relay_recipient_maps = hash:/etc/postfix/relay_recipients
sample_directory = /etc/postfix
sendmail_path = /usr/sbin/sendmail
setgid_group = postdrop
smtp_connect_timeout = 5s
smtp_data_done_timeout = 20s
smtp_data_init_timeout = 30s
smtp_data_xfer_timeout = 60s
smtp_helo_timeout = 30s
smtp_mail_timeout = 30s
smtp_quit_timeout = 10s
smtp_rcpt_timeout = 60s
smtpd_banner = $myhostname NO UCE ESMTP
smtpd_etrn_restrictions = reject_unknown_client
smtpd_hard_error_limit = 5
smtpd_helo_required = yes
smtpd_helo_restrictions = permit_mynetworks, reject_invalid_hostname,
regexp:/etc/postfix/helo.regxp, permit
smtpd_recipient_limit = 100
smtpd_recipient_restrictions = reject_non_fqdn_recipient,
reject_non_fqdn_sender, reject_unknown_client,
reject_unknown_recipient_domain, reject_unauth_pipelining,
permit_mynetworks, reject_unauth_destination, check_sender_access
hash:/etc/postfix/sender_checks, permit
smtpd_sender_restrictions = permit_mynetworks,
reject_unauth_destination, reject_invalid_hostname,
reject_non_fqdn_sender, reject_non_fqdn_recipient,
hash:/etc/postfix/access, hash:/etc/postfix/network_table,
hash:/etc/postfix/backup_domains, reject_rbl_client bl.spamcop.net,
reject_rbl_client list.dsbl.org, reject_rbl_client
sbl.spamhaus.org,reject_rbl_client cbl.abuseat.org,
reject_rbl_client dul.dnsbl.sorbs.net, reject_rhsbl_sender
dsn.rfc-ignorant.org, reject_rbl_client
SBDQTL8DJ25VYNLHD2BM2TC8Q8ZQKQE.r.mail-abuse.com, reject_rbl_client
sbl-xbl.spamhaus.org, reject_rbl_client zen.spamhaus.org,
reject_rbl_client spamsources.fabel.dk, reject_rbl_client
dnsbl.ahbl.org, reject_rbl_client dnsbl.njabl.org,
reject_rbl_client zombie.dnsbl.sorbs.net, reject_rhsbl_client
rhsbl.sorbs.net, reject_rhsbl_sender rhsbl.sorbs.net
smtpd_soft_error_limit = 2
transport_maps = hash:/etc/postfix/transport
unknown_local_recipient_reject_code = 550
unverified_recipient_reject_code = 550

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/1020/=now
RSS Feed: https://www.listbox.com/member/archive/rss/1020/
Powered by Listbox: http://www.listbox.com

Re: perl processes [ In reply to ]

Dec 9, 2008, 8:28 AM

Post #4 of 9 (4972 views)

On Tuesday 09 December 2008 07:04, Simon Jones wrote:
> 2008/12/8 Scott Kitterman <scott@kitterman.com>:
> > On Mon, 8 Dec 2008 14:45:07 +0000 "Simon Jones" <simonmjones@gmail.com>
> >
> > wrote:
> >>Hi all, I've enabled spf policy checking in postfix but it's maxing
> >>out the server - to the point that it's unable to process any mail.
> >>is there a way to set the limit on the number of perl processes called
> >>or a way to tweak the settings to stop it maxing out?
> >
> > It's a function of the number of Postfix smtpd processes.
> >
> > What's the output of postconf -n?
> >
> > Scott K
>
> Hi Scott,
>
snip ...

> smtpd_recipient_restrictions = reject_non_fqdn_recipient,
> reject_non_fqdn_sender, reject_unknown_client,
> reject_unknown_recipient_domain, reject_unauth_pipelining,
> permit_mynetworks, reject_unauth_destination, check_sender_access
> hash:/etc/postfix/sender_checks, permit
snip ...

You aren't actually using the policy service what what I can see in your
postconf -n. Somewhere in here (recommended right after
reject_unauth_destination in smtpd_recipient_restrictions) you need to have
check_policy_service in your smtpd_*_restrictions and I don't see it. See:

http://www.postfix.org/postconf.5.html#check_policy_service

and the README provided with the policy server for details.

In general note that anything that is shown in the output of postconf -n is a
non-default value. You (or your distro packager) have adjusted these. In
many cases some of these values seem rather large to me.

It isn't apparent to me that you're actually using the policy server at all
currently. Is check_policy_service actually used somewhere and I'm not
seeing it in postconf -n?

Scott K

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/1020/=now
RSS Feed: https://www.listbox.com/member/archive/rss/1020/
Powered by Listbox: http://www.listbox.com

Re: perl processes [ In reply to ]

Dec 9, 2008, 9:23 AM

Post #5 of 9 (4968 views)

2008/12/9 Scott Kitterman <scott@kitterman.com>:
> On Tuesday 09 December 2008 07:04, Simon Jones wrote:
>> 2008/12/8 Scott Kitterman <scott@kitterman.com>:
>> > On Mon, 8 Dec 2008 14:45:07 +0000 "Simon Jones" <simonmjones@gmail.com>
>> >
>> > wrote:
>> >>Hi all, I've enabled spf policy checking in postfix but it's maxing
>> >>out the server - to the point that it's unable to process any mail.
>> >>is there a way to set the limit on the number of perl processes called
>> >>or a way to tweak the settings to stop it maxing out?
>> >
>> > It's a function of the number of Postfix smtpd processes.
>> >
>> > What's the output of postconf -n?
>> >
>> > Scott K
>>
>> Hi Scott,
>>
> snip ...
>
>> smtpd_recipient_restrictions = reject_non_fqdn_recipient,
>> reject_non_fqdn_sender, reject_unknown_client,
>> reject_unknown_recipient_domain, reject_unauth_pipelining,
>> permit_mynetworks, reject_unauth_destination, check_sender_access
>> hash:/etc/postfix/sender_checks, permit
> snip ...
>
> You aren't actually using the policy service what what I can see in your
> postconf -n. Somewhere in here (recommended right after
> reject_unauth_destination in smtpd_recipient_restrictions) you need to have
> check_policy_service in your smtpd_*_restrictions and I don't see it. See:
>
> http://www.postfix.org/postconf.5.html#check_policy_service
>
> and the README provided with the policy server for details.
>
> In general note that anything that is shown in the output of postconf -n is a
> non-default value. You (or your distro packager) have adjusted these. In
> many cases some of these values seem rather large to me.
>
> It isn't apparent to me that you're actually using the policy server at all
> currently. Is check_policy_service actually used somewhere and I'm not
> seeing it in postconf -n?
>
> Scott K
>
>
>
Hi Scott,

yes I am (was) using the policy server and did have
check_policy_service unix:private/policy, permit in my
smtp_recipient_restrictions but removed it because the who system was
grinding to a halt. I've enabled it again today but after an hour the
server was back on it's knees so have taken it out of main.cf while I
figure out why it's maxing out with all those perl processes.

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/1020/=now
RSS Feed: https://www.listbox.com/member/archive/rss/1020/
Powered by Listbox: http://www.listbox.com

Re: perl processes [ In reply to ]

Dec 9, 2008, 10:06 AM

Post #6 of 9 (4969 views)

On Tuesday 09 December 2008 12:23, Simon Jones wrote:
> 2008/12/9 Scott Kitterman <scott@kitterman.com>:
> > On Tuesday 09 December 2008 07:04, Simon Jones wrote:
> >> 2008/12/8 Scott Kitterman <scott@kitterman.com>:
> >> > On Mon, 8 Dec 2008 14:45:07 +0000 "Simon Jones"
> >> > <simonmjones@gmail.com>
> >> >
> >> > wrote:
> >> >>Hi all, I've enabled spf policy checking in postfix but it's maxing
> >> >>out the server - to the point that it's unable to process any mail.
> >> >>is there a way to set the limit on the number of perl processes called
> >> >>or a way to tweak the settings to stop it maxing out?
> >> >
> >> > It's a function of the number of Postfix smtpd processes.
> >> >
> >> > What's the output of postconf -n?
> >> >
> >> > Scott K
> >>
> >> Hi Scott,
> >
> > snip ...
> >
> >> smtpd_recipient_restrictions = reject_non_fqdn_recipient,
> >> reject_non_fqdn_sender, reject_unknown_client,
> >> reject_unknown_recipient_domain, reject_unauth_pipelining,
> >> permit_mynetworks, reject_unauth_destination, check_sender_access
> >> hash:/etc/postfix/sender_checks, permit
> >
> > snip ...
> >
> > You aren't actually using the policy service what what I can see in your
> > postconf -n. Somewhere in here (recommended right after
> > reject_unauth_destination in smtpd_recipient_restrictions) you need to
> > have check_policy_service in your smtpd_*_restrictions and I don't see
> > it. See:
> >
> > http://www.postfix.org/postconf.5.html#check_policy_service
> >
> > and the README provided with the policy server for details.
> >
> > In general note that anything that is shown in the output of postconf -n
> > is a non-default value. You (or your distro packager) have adjusted
> > these. In many cases some of these values seem rather large to me.
> >
> > It isn't apparent to me that you're actually using the policy server at
> > all currently. Is check_policy_service actually used somewhere and I'm
> > not seeing it in postconf -n?
> >
> > Scott K
>
> Hi Scott,
>
> yes I am (was) using the policy server and did have
> check_policy_service unix:private/policy, permit in my
> smtp_recipient_restrictions but removed it because the who system was
> grinding to a halt. I've enabled it again today but after an hour the
> server was back on it's knees so have taken it out of main.cf while I
> figure out why it's maxing out with all those perl processes.

OK. What OS/distribution is this running on?

What happens if you run the policy server by hand (how to do this is described
in the README that came with the package (or the man page for Debian/Ubuntu)?

Scott K

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/1020/=now
RSS Feed: https://www.listbox.com/member/archive/rss/1020/
Powered by Listbox: http://www.listbox.com

Re: perl processes [ In reply to ]

Dec 10, 2008, 2:50 AM

Post #7 of 9 (4970 views)

2008/12/9 Scott Kitterman <scott@kitterman.com>:
> On Tuesday 09 December 2008 12:23, Simon Jones wrote:
>> 2008/12/9 Scott Kitterman <scott@kitterman.com>:
>> > On Tuesday 09 December 2008 07:04, Simon Jones wrote:
>> >> 2008/12/8 Scott Kitterman <scott@kitterman.com>:
>> >> > On Mon, 8 Dec 2008 14:45:07 +0000 "Simon Jones"
>> >> > <simonmjones@gmail.com>
>> >> >
>> >> > wrote:
>> >> >>Hi all, I've enabled spf policy checking in postfix but it's maxing
>> >> >>out the server - to the point that it's unable to process any mail.
>> >> >>is there a way to set the limit on the number of perl processes called
>> >> >>or a way to tweak the settings to stop it maxing out?
>> >> >
>> >> > It's a function of the number of Postfix smtpd processes.
>> >> >
>> >> > What's the output of postconf -n?
>> >> >
>> >> > Scott K
>> >>
>> >> Hi Scott,
>> >
>> > snip ...
>> >
>> >> smtpd_recipient_restrictions = reject_non_fqdn_recipient,
>> >> reject_non_fqdn_sender, reject_unknown_client,
>> >> reject_unknown_recipient_domain, reject_unauth_pipelining,
>> >> permit_mynetworks, reject_unauth_destination, check_sender_access
>> >> hash:/etc/postfix/sender_checks, permit
>> >
>> > snip ...
>> >
>> > You aren't actually using the policy service what what I can see in your
>> > postconf -n. Somewhere in here (recommended right after
>> > reject_unauth_destination in smtpd_recipient_restrictions) you need to
>> > have check_policy_service in your smtpd_*_restrictions and I don't see
>> > it. See:
>> >
>> > http://www.postfix.org/postconf.5.html#check_policy_service
>> >
>> > and the README provided with the policy server for details.
>> >
>> > In general note that anything that is shown in the output of postconf -n
>> > is a non-default value. You (or your distro packager) have adjusted
>> > these. In many cases some of these values seem rather large to me.
>> >
>> > It isn't apparent to me that you're actually using the policy server at
>> > all currently. Is check_policy_service actually used somewhere and I'm
>> > not seeing it in postconf -n?
>> >
>> > Scott K
>>
>> Hi Scott,
>>
>> yes I am (was) using the policy server and did have
>> check_policy_service unix:private/policy, permit in my
>> smtp_recipient_restrictions but removed it because the who system was
>> grinding to a halt. I've enabled it again today but after an hour the
>> server was back on it's knees so have taken it out of main.cf while I
>> figure out why it's maxing out with all those perl processes.
>
> OK. What OS/distribution is this running on?
>
> What happens if you run the policy server by hand (how to do this is described
> in the README that came with the package (or the man page for Debian/Ubuntu)?
>
> Scott K
>
Hi Scott,

not sure if this is right - what should be output look like?

[root@smtp postfix-policyd-spf-perl-2.001]# perl
/usr/libexec/postfix/policyd-spf-perl

action=DUNNO

request=smtpd_access_policy

action=DUNNO

OS is Centos 5.2 with kernel 2.6.18-92.1.18.el5

Simon

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/1020/=now
RSS Feed: https://www.listbox.com/member/archive/rss/1020/
Powered by Listbox: http://www.listbox.com

Re: perl processes [ In reply to ]

Dec 10, 2008, 9:01 AM

Post #8 of 9 (4961 views)

On Wed, 10 Dec 2008 10:50:54 +0000 "Simon Jones" <simonmjones@gmail.com>
wrote:
>2008/12/9 Scott Kitterman <scott@kitterman.com>:
>> On Tuesday 09 December 2008 12:23, Simon Jones wrote:
>>> 2008/12/9 Scott Kitterman <scott@kitterman.com>:
>>> > On Tuesday 09 December 2008 07:04, Simon Jones wrote:
>>> >> 2008/12/8 Scott Kitterman <scott@kitterman.com>:
>>> >> > On Mon, 8 Dec 2008 14:45:07 +0000 "Simon Jones"
>>> >> > <simonmjones@gmail.com>
>>> >> >
>>> >> > wrote:
>>> >> >>Hi all, I've enabled spf policy checking in postfix but it's maxing
>>> >> >>out the server - to the point that it's unable to process any mail.
>>> >> >>is there a way to set the limit on the number of perl processes
called
>>> >> >>or a way to tweak the settings to stop it maxing out?
>>> >> >
>>> >> > It's a function of the number of Postfix smtpd processes.
>>> >> >
>>> >> > What's the output of postconf -n?
>>> >> >
>>> >> > Scott K
>>> >>
>>> >> Hi Scott,
>>> >
>>> > snip ...
>>> >
>>> >> smtpd_recipient_restrictions = reject_non_fqdn_recipient,
>>> >> reject_non_fqdn_sender, reject_unknown_client,
>>> >> reject_unknown_recipient_domain, reject_unauth_pipelining,
>>> >> permit_mynetworks, reject_unauth_destination, check_sender_access
>>> >> hash:/etc/postfix/sender_checks, permit
>>> >
>>> > snip ...
>>> >
>>> > You aren't actually using the policy service what what I can see in
your
>>> > postconf -n. Somewhere in here (recommended right after
>>> > reject_unauth_destination in smtpd_recipient_restrictions) you need to
>>> > have check_policy_service in your smtpd_*_restrictions and I don't see
>>> > it. See:
>>> >
>>> > http://www.postfix.org/postconf.5.html#check_policy_service
>>> >
>>> > and the README provided with the policy server for details.
>>> >
>>> > In general note that anything that is shown in the output of postconf
-n
>>> > is a non-default value. You (or your distro packager) have adjusted
>>> > these. In many cases some of these values seem rather large to me.
>>> >
>>> > It isn't apparent to me that you're actually using the policy server
at
>>> > all currently. Is check_policy_service actually used somewhere and
I'm
>>> > not seeing it in postconf -n?
>>> >
>>> > Scott K
>>>
>>> Hi Scott,
>>>
>>> yes I am (was) using the policy server and did have
>>> check_policy_service unix:private/policy, permit in my
>>> smtp_recipient_restrictions but removed it because the who system was
>>> grinding to a halt. I've enabled it again today but after an hour the
>>> server was back on it's knees so have taken it out of main.cf while I
>>> figure out why it's maxing out with all those perl processes.
>>
>> OK. What OS/distribution is this running on?
>>
>> What happens if you run the policy server by hand (how to do this is
described
>> in the README that came with the package (or the man page for
Debian/Ubuntu)?
>>
>> Scott K
>>
>Hi Scott,
>
>not sure if this is right - what should be output look like?
>
>[root@smtp postfix-policyd-spf-perl-2.001]# perl
>/usr/libexec/postfix/policyd-spf-perl
>
>action=DUNNO
>
>request=smtpd_access_policy
>
>action=DUNNO
>
>OS is Centos 5.2 with kernel 2.6.18-92.1.18.el5

I found some references via Google to Perl performance problems on Centos 5.2. It seems
likely you are hitting those. I don't think you have a problem specific to this policy server.

There is also a Python implementation (on http://www.openspf.org/Software
if not through your packaging system). I'd recommend trying that instead.
It has a many more options than the Perl implementation, so review the
documentation that comes with the package (the defaults are reasonably sane
for common situations).

Scott K

-------------------------------------------
Sender Policy Framework: http://www.openspf.org
Modify Your Subscription: http://www.listbox.com/member/
Archives: https://www.listbox.com/member/archive/1020/=now
RSS Feed: https://www.listbox.com/member/archive/rss/1020/
Powered by Listbox: http://www.listbox.com

Re: perl processes [ In reply to ]

Dec 11, 2008, 1:14 AM

Post #9 of 9 (4978 views)