Mailing List Archive

Stuart D. Gathman wrote:

> On Tue, 23 Mar 2004, Alain Knaff wrote:
>
>> It is sendmail that is smart enough to do the merging. If there are
>> several S lines with the same ruleset name, all rules are merged
>> within that same ruleset.
>
> OK, the SRS line for ruleset EnvFromSMTP comes before the rest,
> whereas Mark's instructions have me add the SRS line *after* the rest.

The last line of EnvFromSMTP is:

R$+ $: $>MasqEnv $1 do masquerading

There are two good reasons to place SRS encoding at the end of EnvFromSMTP.
For one, since SRS encoding may alter the domain name (!), doing it earlier
might break masquerading in those cases. For two, it simply makes sense to
only make_srs after everything else with the address is already done.

> Similary, the SRS line for ruleset ParseLocal comes *after*, whereas
> Mark has me add it before.

Conversely, before we have sendmail do any sort of checks on the recipient,
we do well to first revert the address back to a regular recipient. My
ParseLocal, for instance, normally starts with code that does REDIRECT for
users who have moved. It is, of course, important that sendmail enters the
standard part of ParseLocal as it normally would, with a regular recipient.

> Does this make any difference?

I believe it does.

> I have hardwired the envfrom2srs.py script to alwaysrewrite=True,
> since the sendmail.cf now skips calling it as required.

I will add that too.

Cheers,

- Mark

System Administrator Asarian-host.org

---
"If you were supposed to understand it,
we wouldn't call it code." - FedEx

On Tue, 23 Mar 2004, Mark wrote:

> The last line of EnvFromSMTP is:
>
> R$+ $: $>MasqEnv $1 do masquerading
>
> There are two good reasons to place SRS encoding at the end of EnvFromSMTP.
> For one, since SRS encoding may alter the domain name (!), doing it earlier
> might break masquerading in those cases. For two, it simply makes sense to
> only make_srs after everything else with the address is already done.
>
> > Similary, the SRS line for ruleset ParseLocal comes *after*, whereas
> > Mark has me add it before.
>
> Conversely, before we have sendmail do any sort of checks on the recipient,
> we do well to first revert the address back to a regular recipient. My
> ParseLocal, for instance, normally starts with code that does REDIRECT for
> users who have moved. It is, of course, important that sendmail enters the
> standard part of ParseLocal as it normally would, with a regular recipient.

So how do we fix the m4 HACK? I tried moving the HACK(`pysrs`) to
after the mailers, but it didn't help. It is still backward! The
problem seems to be that the HACK gets included in the middle,
with ParseLocal coming before, and EnvFromSMTP coming after.
So how do you get stuff included near the beginning? Near the end?

It is really important to have the changes encapsulated as a macro for
ease of installation. The package so far, installed on 4 production
mail gateways (boy am I getting tired of monitoring logs for problems):

http://bmsi.com/python/pysrs.html

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Very few of our customers are going to have a pure Unix
or pure Windows environment." - Dennis Oldroyd, Microsoft Corporation

On Tue, 23 Mar 2004, Mark wrote:

> Stuart D. Gathman wrote:
>
> > I have hardwired the envfrom2srs.py script to alwaysrewrite=True,
> > since the sendmail.cf now skips calling it as required.
>
> I will add that too.

Why?

It won't speed anything up, and in the case that Mail::SRS detects that
rewriting is optional, you probably still don't want to perform it.
Mail::SRS only skips the rewrite in the case where SPF would pass in
either case.

S.

--
Shevek http://www.anarres.org/
I am the Borg. http://www.gothnicity.org/

Stuart D. Gathman wrote:

> On Tue, 23 Mar 2004, Mark wrote:
>
>> The last line of EnvFromSMTP is:
>>
>> R$+ $: $>MasqEnv $1 do masquerading
>>
>> There are two good reasons to place SRS encoding at the end of
>> EnvFromSMTP. For one, since SRS encoding may alter the domain name
>> (!), doing it earlier might break masquerading in those cases. For
>> two, it simply makes sense to only make_srs after everything else
>> with the address is already done.
>>
>>> Similary, the SRS line for ruleset ParseLocal comes *after*, whereas
>>> Mark has me add it before.
>>
>> Conversely, before we have sendmail do any sort of checks on the
>> recipient, we do well to first revert the address back to a regular
>> recipient. My ParseLocal, for instance, normally starts with code
>> that does REDIRECT for users who have moved. It is, of course,
>> important that sendmail enters the standard part of ParseLocal as it
>> normally would, with a regular recipient.
>
> So how do we fix the m4 HACK? I tried moving the HACK(`pysrs`) to
> after the mailers, but it didn't help. It is still backward! The
> problem seems to be that the HACK gets included in the middle,
> with ParseLocal coming before, and EnvFromSMTP coming after.
> So how do you get stuff included near the beginning? Near the end?

Good question. ;) Seriously, though, as I suspected, m4 does not provide
such fine-grained control. I asked, to be sure, and got word from Claus
Assman today, saying this is, indeed, not possible (if proto.m4 does does
provide the hook either; but editing proto.m4 leaves us exactly where
editing sendmail.cf leaves us). He said that when he himself was faced with
such a situation, that he just uses a patch file against the newly created
cf.

And it may just really come to that. So, we can provide m4 files; but then
the user may be required to move a line or two.

Regards,

- Mark

System Administrator Asarian-host.org

---
"If you were supposed to understand it,
we wouldn't call it code." - FedEx

On Wed, 24 Mar 2004, Mark wrote:

> Good question. ;) Seriously, though, as I suspected, m4 does not provide
> such fine-grained control. I asked, to be sure, and got word from Claus
> Assman today, saying this is, indeed, not possible (if proto.m4 does does
> provide the hook either; but editing proto.m4 leaves us exactly where
> editing sendmail.cf leaves us). He said that when he himself was faced with
> such a situation, that he just uses a patch file against the newly created
> cf.

I fixed the EnvFromSMTP - and I'm not even an m4 expert! m4 uses redirections,
so I changed pysrs.m4 like this:

diff -c -r1.2 pysrs.m4
*** pysrs.m4 24 Mar 2004 01:03:21 -0000 1.2
--- pysrs.m4 24 Mar 2004 22:03:44 -0000
***************
*** 32,41 ****
F{noSrsMailers}-o NO_SRS_FILE %[^\#]
')dnl

! LOCAL_RULESETS

SEnvFromSMTP
R$+ $: $>MakeSrs $1 make SRS

SIsSrs
# Answers YES or NO whether the address in parameter is srs or not
--- 32,43 ----
F{noSrsMailers}-o NO_SRS_FILE %[^\#]
')dnl

! MAILER_DEFINITIONS

SEnvFromSMTP
R$+ $: $>MakeSrs $1 make SRS
+
+ LOCAL_RULESETS

SIsSrs
# Answers YES or NO whether the address in parameter is srs or not


As a finishing touch, I can check that HACK(pysrs) comes after the smtp mailer
by checking that _MAILER_DEFINED_ is defined.

This, however, leaves ParseLocal as follows:

SParseLocal=98

# addresses sent to foo@host.REDIRECT will give a 551 error code
R$* < @ $+ .REDIRECT. > $: $1 < @ $2 . REDIRECT . > < ${opMode} >
R$* < @ $+ .REDIRECT. > <i> $: $1 < @ $2 . REDIRECT. >
R$* < @ $+ .REDIRECT. > < $- > $#error $@ 5.1.1 $: "551 User has moved; please try " <$1@$2>

SParseLocal=98
R$* $: $>ReverseSrs $1

This can be fixed by providing a feature which does

LOCAL_RULESET_0
R$* $: $>ReverseSrs $1

and must come before redirect.

However, is this really necessary? It looks like adding REDIRECT has
already been done before entering ParseLocal. And the ReverseSrs would
not do anything in that case. On the other hand, if REDIRECT has *not*
been added, then ReverseSrs will work normally.

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Very few of our customers are going to have a pure Unix
or pure Windows environment." - Dennis Oldroyd, Microsoft Corporation

I have attached a cdiff for changes to support adding SRS to EnvFromSMTP
in the right place.

If it is determined that the ParseLocal rule will not work as it stands,
then I can create a HACK(reverse_srs) which will come before
FEATURE(redirect).

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Very few of our customers are going to have a pure Unix
or pure Windows environment." - Dennis Oldroyd, Microsoft Corporation

Stuart D. Gathman wrote:

> On Wed, 24 Mar 2004, Mark wrote:
>
>> Good question. ;) Seriously, though, as I suspected, m4 does not
>> provide such fine-grained control. I asked, to be sure, and got word
>> from Claus Assman today, saying this is, indeed, not possible (if
>> proto.m4 does does provide the hook either; but editing proto.m4
>> leaves us exactly where editing sendmail.cf leaves us). He said that
>> when he himself was faced with such a situation, that he just uses a
>> patch file against the newly created cf.
>
> I fixed the EnvFromSMTP - and I'm not even an m4 expert! m4 uses
> redirections, so I changed pysrs.m4 like this:

Brilliant work! :)

> This can be fixed by providing a feature which does
>
> LOCAL_RULESET_0
> R$* $: $>ReverseSrs $1
>
> and must come before redirect.

Yes, I figured out the LOCAL_RULESET_0 too -- a very fortunate hook (one
that exists for ruleset 1, 2, and 3 too). I am impressed you solved the
EnvFromSMTP thingy. With your permission, I'd like to create an m4 of my
own, including your trick (for which I will accredite you, of course).

You did good! :)

- Mark

System Administrator Asarian-host.org

---
"If you were supposed to understand it,
we wouldn't call it code." - FedEx

On Wed, 24 Mar 2004, Mark wrote:

> Yes, I figured out the LOCAL_RULESET_0 too -- a very fortunate hook (one
> that exists for ruleset 1, 2, and 3 too). I am impressed you solved the
> EnvFromSMTP thingy. With your permission, I'd like to create an m4 of my
> own, including your trick (for which I will accredite you, of course).

No problem. I am shamelessly incorporating stuff into pysrs. That's
what open source is for.

One more m4 change. To reduce places where it is configured, I am
passing the forwarding domain as a command line argument to envfrom2srs.py:

ifelse(defn(`_ARG_'),`',,`define(`SRS_DOMAIN',_ARG_)')

LOCAL_CONFIG
# Forward SRS program map
Kmake_srs program /usr/bin/envfrom2srs.py ifdef(`SRS_DOMAIN',SRS_DOMAIN)
# Reverse SRS program map
Kreverse_srs program /usr/bin/srs2envtol.py

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Very few of our customers are going to have a pure Unix
or pure Windows environment." - Dennis Oldroyd, Microsoft Corporation

I am pretty dense when it comes to sendmail-cf. I can't get
the NO_SRS_FILE to work.

It generates a class:
# Class of destination mailers not needing SRS
F{noSrsMailers}-o /etc/mail/no-srs-mailers %[^\#]

And MakeSrs looks like this:

SMakeSrs
#
# Prevent SRS encapsulation if "To" address is SRS
R$* $: $1 $>IsSrs $&u
R$* YES $@ $1
R$* NO $: $1
#
# If destination mailer is in non-SRS list, do not apply SRS
# This is intended for handling communication between secondary MX and
# primary MX
R$* $: $&h $| $1
R$={noSrsMailers} $| $* $@ $2
R$* $| $* $: $2
R$* $: $(make_srs $1 $)

The no-srs-mailers file has hostnames in it (changed to protect the innocent):

# no-srs-mailers - list domains we should not SRS encode for when we
# send to them. E.g. primary MX servers for which we are a secondary.
#
mycust.com
anothercust.com

When I send mail to user@mycust.com, the message is still SRS encoded.
It looks like $&h would be mycust.com, and it should exit the
subrout early - but somehow make_srs is still called.

Do I have the format wrong? Or does $&h mean the *source* host?
Why would I want to not SRS encode certain source hosts? (That
would be covered by NO_SRS_FROM_LOCAL, right?)

How do you trace cf macros to try and debug them?

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Very few of our customers are going to have a pure Unix
or pure Windows environment." - Dennis Oldroyd, Microsoft Corporation

On Thu, 25 Mar 2004, Stuart D. Gathman wrote:

> I am pretty dense when it comes to sendmail-cf. I can't get
> the NO_SRS_FILE to work.
...
> When I send mail to user@mycust.com, the message is still SRS encoded.
> It looks like $&h would be mycust.com, and it should exit the
> subrout early - but somehow make_srs is still called.

By trial and error, I have discovered that $&h means the target host,
as in the RHS of mailertable or a hostname found in an MX record.

It works now.

--
Stuart D. Gathman <stuart@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Very few of our customers are going to have a pure Unix
or pure Windows environment." - Dennis Oldroyd, Microsoft Corporation

Stuart D. Gathman wrote:

> On Thu, 25 Mar 2004, Stuart D. Gathman wrote:
>
>> I am pretty dense when it comes to sendmail-cf. I can't get
>> the NO_SRS_FILE to work.
> ....
>> When I send mail to user@mycust.com, the message is still SRS
>> encoded. It looks like $&h would be mycust.com, and it should exit
>> the subrout early - but somehow make_srs is still called.
>
> By trial and error, I have discovered that $&h means the target host,
> as in the RHS of mailertable or a hostname found in an MX record.

Yes. That is why, in an earlier post, I parsed $&u as parameter to make_srs:

"R$* $: $(make_srs $&u $1 $)"

Alain does the same; MakeSrs checks on the domain of $&u, not $&h; and on
$&h, not $&u, to exempt the noSrsMailers, when it means "destination
mailer". Which is exactly right.

- Mark

System Administrator Asarian-host.org

---
"If you were supposed to understand it,
we wouldn't call it code." - FedEx