Mailing List Archive

On Thursday 01 March 2007 10:12, John A. Martin wrote:
> It appears to me that, when the policy daemon is called during the
> RCPT protocol state, the queue_id attribute is empty only for the
> first recipient of a message with more than one recipient but has a
> value for each recipient after the first. Is this is generally true
> and can it be relied upon, as it seems it may be a fundamental
> consequence of the Postfix architecture?
>
> If the absence and presence of a value for the queue_id attribute can
> be relied upon, would using something like
>
> return "PREPEND $...-spf_header"
> unless ($attr->{queue_id} ne '');
>
> instead of
>
> return "PREPEND $..._spf_header"
> unless $cache->{added_spf_header}++;
>
> and doing away with the cache be simpler and better?
>
> Please pardon my poor Perl, I am not a native Perl speaker. :)
>
The purpose of the cache is larger than just making sure we prepend the single
header. It is also done for performance and scalability. Since the entire
object is cached, redundant queries and processing for multiple recipients
are avoided.

We would actually key off of instance and not queue_id per the
SMTPD_POLICY_README if we didn't use the cache approach:

http://www.postfix.org/SMTPD_POLICY_README.html

"The "instance" attribute value can be used to correlate different requests
regarding the same message delivery."

This is what version 0.3 of the Python policy server does. I intend to
implement a similar object cache there too, but haven't got it done yet.

One thing I intend to do with the Python server (my Perl is even more limited
than yours, so I will take patches/requests or beg help from Julian for the
Perl server, but am not intending to add features to it on my own initiative)
is to also cache what has been prepended so that I can call the policy server
twice from Postfix and the first time through get SPF-Recieved for HELO and
the second time through get SPF-Recieved for Mail From. This should support
SpamAssassin 3.2 SPF rules better than a single SPF-Recieved.

I agree with simpler, but not better.

Scott K

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?list_id=1007

>>>>> "Scott" == Scott Kitterman
>>>>> "Re: postfix-policyd-spf-perl cache"
>>>>> Thu, 1 Mar 2007 10:29:24 -0500

Thank you for your explanations.

>> Please pardon my poor Perl, I am not a native Perl speaker. :)

Scott> The purpose of the cache is larger than just making sure we
Scott> prepend the single header. It is also done for performance
Scott> and scalability. Since the entire object is cached,
Scott> redundant queries and processing for multiple recipients
Scott> are avoided.

My Perl reading is so lame that I failed to see what is being cached
and how it is used. It remains opaque to me.

Scott> We would actually key off of instance and not queue_id per
Scott> the SMTPD_POLICY_README if we didn't use the cache
Scott> approach:

Scott> http://www.postfix.org/SMTPD_POLICY_README.html

Scott> "The "instance" attribute value can be used to correlate
Scott> different requests regarding the same message delivery."

Yes, but it seemed to me that the eq/ne on one attribute was simpler
than "correlating different requests" but that is moot in view of what
is actually being done.

Scott> I will take patches/requests or beg help from Julian for
Scott> the Perl server, but am not intending to add features to it
Scott> on my own initiative)

Please do not short change the Perl server which is invaluable for
dedicated MTA boxen because it adds fewer and lighter dependencies.

One of difficulties I see, and basically what seems to me to require
ugly hacking to treat SPF pass messages differently than others before
DATA, is that the Postfix policy protocol, as I understand it, has the
'prepend' action effectively only as a 'dunno' action with a prepend
side effect. If a prepend side effect could be specified for other
actions, particularly for restriction actions (such as restriction
classes), it would be easy to control which restrictions are applied
after a SPF pass, for example, as distinguished from those applied
after other SPF results.

jam

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?list_id=1007

On Friday 02 March 2007 10:05, John A. Martin wrote:
> >>>>> "Scott" == Scott Kitterman
> >>>>> "Re: postfix-policyd-spf-perl cache"
> >>>>> Thu, 1 Mar 2007 10:29:24 -0500
>
> Thank you for your explanations.
>
> >> Please pardon my poor Perl, I am not a native Perl speaker. :)
>
> Scott> The purpose of the cache is larger than just making sure we
> Scott> prepend the single header. It is also done for performance
> Scott> and scalability. Since the entire object is cached,
> Scott> redundant queries and processing for multiple recipients
> Scott> are avoided.
>
> My Perl reading is so lame that I failed to see what is being cached
> and how it is used. It remains opaque to me.

If you really want it explained, Julian will have to do it, it's his work.
What is being cached is the Mail::SPF (I think) result object.

> Scott> We would actually key off of instance and not queue_id per
> Scott> the SMTPD_POLICY_README if we didn't use the cache
> Scott> approach:
>
> Scott> http://www.postfix.org/SMTPD_POLICY_README.html
>
> Scott> "The "instance" attribute value can be used to correlate
> Scott> different requests regarding the same message delivery."
>
> Yes, but it seemed to me that the eq/ne on one attribute was simpler
> than "correlating different requests" but that is moot in view of what
> is actually being done.

Agreed. Just for clarification.

> Scott> I will take patches/requests or beg help from Julian for
> Scott> the Perl server, but am not intending to add features to it
> Scott> on my own initiative)
>
> Please do not short change the Perl server which is invaluable for
> dedicated MTA boxen because it adds fewer and lighter dependencies.

It's not a question of short changing the Perl one. I have essentially no
Perl, so my ability to deal with it is limited. I'm willing to invest time
in improving it, but am very limited what I can do on my own.

I'm also willing to argue fewer/lighter depencies for Perl versus Python. The
add-ons required for pyspf and the associated Python policy server are very
limited.

> One of difficulties I see, and basically what seems to me to require
> ugly hacking to treat SPF pass messages differently than others before
> DATA, is that the Postfix policy protocol, as I understand it, has the
> 'prepend' action effectively only as a 'dunno' action with a prepend
> side effect. If a prepend side effect could be specified for other
> actions, particularly for restriction actions (such as restriction
> classes), it would be easy to control which restrictions are applied
> after a SPF pass, for example, as distinguished from those applied
> after other SPF results.

That is a limitation of the Postfix policy server protocol. For the Python
server in order to add Received SPF for both HELO and Mail From results, it
looks like I'm going to have to call the policy server more than once.

Scott K

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?list_id=1007

>>>>> "Scott" == Scott Kitterman
>>>>> "Re: Re: postfix-policyd-spf-perl cache"
>>>>> Fri, 2 Mar 2007 16:41:43 -0500

Scott> I'm also willing to argue fewer/lighter depencies for Perl
Scott> versus Python. The add-ons required for pyspf and the
Scott> associated Python policy server are very limited.

The next time I build a dedicated Postfix MTA Debian box I will
compare the total additional burden of each. Also, IIRC there have
been issues on Debian with long lived Python processes during package
upgrades, no? Maybe those issues have gone away.

>> One of difficulties I see, and basically what seems to me to
>> require ugly hacking to treat SPF pass messages differently
>> than others before DATA, is that the Postfix policy protocol,
>> as I understand it, has the 'prepend' action effectively only
>> as a 'dunno' action with a prepend side effect. If a prepend
>> side effect could be specified for other actions, particularly
>> for restriction actions (such as restriction classes), it would
>> be easy to control which restrictions are applied after a SPF
>> pass, for example, as distinguished from those applied after
>> other SPF results.

Scott> That is a limitation of the Postfix policy server protocol.

Exactly.

Scott> For the Python server in order to add Received SPF for both
Scott> HELO and Mail From results, it looks like I'm going to have
Scott> to call the policy server more than once.

If you are willing to sacrifice the matched mechanism only in the
'Received-SPF pass' header field then you might be able to do
something similar to my modifications to your Perl solution. NB. the
matched mechanism remains in the logs where IMHO debugging information
belongs.

On the other hand a better Python solution might show the way for a
better Perl solution.

jam

-------
To unsubscribe, change your address, or temporarily deactivate your subscription,
please go to http://v2.listbox.com/member/?list_id=1007