Mailing List Archive

better support for macros
at our site, we use Exim on a number of different kinds of servers. to
keep complexity down, we have separated the common bits into files which
can be included in each exim.conf. at this time, we have 14 different
exim.conf, so I'm sure you understand that this is essential to be able
to maintain the files consistently.

however, the current macro handling in Exim is quite restrictive: they
have to be set in the main section, and they can't be redefined. this
means that most macro definitions must be duplicated across each
exim.conf as we can't put the default values in a common include file
and redefine them where needed.

the enclosed patch adds support for macro definitions in the sections
"acls", "authenticators", "routers" and "transports". I wasn't sure if
the syntax could be added to the sections "local_scans", "retrys" and
"rewrites" without breaking backwards compatibility, so I didn't change
those. a macro definition can not appear in the middle of a
router/acl/transport stanza.

the patch also adds support for redefining a macro using the "==" token.
it is an error to redefine a non-existing macro, but you can use this
syntax to gather data, e.g.

OPTIONS = an_option
OPTIONS == OPTIONS new_option
OPTIONS == OPTIONS another_option

note that

MACRO ==foo

would previously set MACRO to "=foo", but it will now cause a syntax
panic as "MACRO" is undefined. the fix is to add a space:

MACRO = =foo

although backwards compatibility is broken somewhat, I think it will be
rare, and it will be discovered very quickly indeed by the admin ;-)

the patch also fixes a small inconsistency in that

MACRO1 = foo
MACRO = bar

is allowed in today's Exim, whereas

MACRO = bar
MACRO1 = foo

causes a syntax panic due to MACRO1 containing the name of an already
defined macro.

I would love to see this patch included upstream. thank you for your
consideration.
--
Kjetil T.
postmaster, University of Oslo
Re: better support for macros [ In reply to ]
On 3/15/2005 16:45, "Kjetil Torgrim Homme" <kjetilho@ifi.uio.no> wrote:

> at our site, we use Exim on a number of different kinds of servers. to
> keep complexity down, we have separated the common bits into files which
> can be included in each exim.conf. at this time, we have 14 different
> exim.conf, so I'm sure you understand that this is essential to be able
> to maintain the files consistently.

We have a similar but smaller problem at our site.

We keep the configuration under source code control, and have an install
script which actually puts them in place for Exim's use.

The install script includes a step in which server-specific macros--grouped
together at the top of the original file are adjusted using the standard
patch
program. Seems to work well.

However, we have not broken our config down into a bunch of individual files
(we do have a few such, to avoid repeated code). We haven't had a need for
macros in those.

--John
Re: better support for macros [ In reply to ]
On Tue, 2005-03-15 at 18:03 -0800, John W. Baxter wrote:
> On 3/15/2005 16:45, "Kjetil Torgrim Homme" <kjetilho@ifi.uio.no> wrote:
>
> > at our site, we use Exim on a number of different kinds of servers. to
> > keep complexity down, we have separated the common bits into files which
> > can be included in each exim.conf. at this time, we have 14 different
> > exim.conf, so I'm sure you understand that this is essential to be able
> > to maintain the files consistently.
>
> We have a similar but smaller problem at our site.
>
> We keep the configuration under source code control, and have an install
> script which actually puts them in place for Exim's use.

so do we. each of the eleven "flavours" has a file detailing which
files from the repository should go where.

> The install script includes a step in which server-specific macros--grouped
> together at the top of the original file are adjusted using the standard
> patch
> program. Seems to work well.

I guess we could do something like that, but editing the diff sounds
like a pain ;-)

> However, we have not broken our config down into a bunch of individual files
> (we do have a few such, to avoid repeated code). We haven't had a need for
> macros in those.

one example from our ACL file, this is not currently allowed:

begin acls
NOT_POSTMASTER = acl_m0

accept local_parts = postmaster
domains = +local_domains

warn set NOT_POSTMASTER = true

we need to define all these macros in the main section, in a different
file, which makes it a bit harder to keep in sync. the other
alternative is to sprinkle acl_m0, acl_m1 etc. throughout the rules, but
that's definitely unreadable :-)
--
Kjetil T.
postmaster, University of Oslo
Re: better support for macros [ In reply to ]
On Wed, 16 Mar 2005, Kjetil Torgrim Homme wrote:

> the enclosed patch adds support for macro definitions in the sections
> "acls", "authenticators", "routers" and "transports".

<snip>

> I would love to see this patch included upstream. thank you for your
> consideration.

While I am always happy to look at patches - I'll look at this one - I
am wary about adding anything that increases the time spent processing
the configuration file. Remember that Exim re-reads its configuration
file *very frequently*. It is only the daemon that reads it once and
then runs for ever. Other Exim processes are short-lived, and they all
have to read the configuration when they start up.

For maximum performance, it is always better to use a fixed config file
(as short as possible); you can always set up some external process to
generate such files if necessary.

Having said that:

(1) Over the lifetime of Exim, I have given in to people's requests and
added stuff like macros and .include to the config, so I suppose this is
no different. If you don't use it, it won't impact (much).

(2) Processors get faster and faster and maybe the resources needed to
process the config file are, in absolute terms, pretty trivial. So we
might as well use them to make the sysadmin's life easier...

--
Philip Hazel University of Cambridge Computing Service,
ph10@cus.cam.ac.uk Cambridge, England. Phone: +44 1223 334714.
Get the Exim 4 book: http://www.uit.co.uk/exim-book
Re: better support for macros [ In reply to ]
On Wed, 2005-03-16 at 10:48 +0000, Philip Hazel wrote:
> On Wed, 16 Mar 2005, Kjetil Torgrim Homme wrote:
>
> > the enclosed patch adds support for macro definitions in the sections
> > "acls", "authenticators", "routers" and "transports".
>
> <snip>
>
> > I would love to see this patch included upstream. thank you for your
> > consideration.
>
> While I am always happy to look at patches - I'll look at this one - I
> am wary about adding anything that increases the time spent processing
> the configuration file. Remember that Exim re-reads its configuration
> file *very frequently*. It is only the daemon that reads it once and
> then runs for ever. Other Exim processes are short-lived, and they all
> have to read the configuration when they start up.

fair point, but this time is dwarfed by SpamAssassin at most sites these
days. the patch I sent purposely scans over the macro name twice as I
don't think a loop of isalnum() over a dozen characters will show up on
anyone's profile, and I think the simplified code is worth that cost.
(I'll fix it if you want me to.) for users who don't use the added
functionality, the increased processing is only an extra isupper() for
each router/transport/acl.

> Having said that:
>
> (1) Over the lifetime of Exim, I have given in to people's requests and
> added stuff like macros and .include to the config, so I suppose this is
> no different. If you don't use it, it won't impact (much).
>
> (2) Processors get faster and faster and maybe the resources needed to
> process the config file are, in absolute terms, pretty trivial. So we
> might as well use them to make the sysadmin's life easier...

sounds like we are in total agreement :-) btw, let me know if you
accept the patch so that I can prepare a documentation patch to go with
it.

thanks again,
--
Kjetil T.
postmaster, University of Oslo
Re: better support for macros [ In reply to ]
On Wed, 16 Mar 2005, Kjetil Torgrim Homme wrote:

> sounds like we are in total agreement :-) btw, let me know if you
> accept the patch so that I can prepare a documentation patch to go with
> it.

It will be some time. I am not spending very much time on the code at
present (other - Exim related - things need to be done) and you are not
first in the queue. :-)

--
Philip Hazel University of Cambridge Computing Service,
ph10@cus.cam.ac.uk Cambridge, England. Phone: +44 1223 334714.
Get the Exim 4 book: http://www.uit.co.uk/exim-book