Mailing List Archive

[Bug 139] Dynamically loadable lookup modules
------- You are receiving this mail because: -------
You are the QA contact for the bug.

http://bugs.exim.org/show_bug.cgi?id=139




--- Comment #20 from Tony Finch <dot@dotat.at> 2008-05-14 14:57:21 ---
Some comments based on a fairly cursory read-through of the patch...

I'm not sure that configuring the available lookups by installing files in
a library directory is very nice. I can see it makes sense from a Linux
distribution point of view, but when building from source or creating a
unified package it makes sense to install all the lookups then load only
the necessary ones at run time. I'm not sure if this would lead to a
chicken/egg problem.

drtables.c:

The #include <dlfcn.h> should be removed for portability to
systems without that header. Instead add defined(LOOKUP_MODULE_DIR) to
the #if on line 463 of exim.h which guards another #include <dlfcn.h>.

lookupapi.h:

This file is missing the copyright and end-of-file rubric. The LMM1
comment should be less cryptic.

lookups/Makefile:

This file is gmake-only which is a nasty regression. Exim's build system
has stuff in the scripts directory that constructs the actual Makefile
based on EDITME. I would greatly prefer it if that mechanism were adapted
to support the new functionality so that we don't have to add another
build dependency.

The .c.so rule is non-portable. The shared-library command line options
need to be made into an EDITME option.

Tony.


--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at http://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 139] Dynamically loadable lookup modules [ In reply to ]
------- You are receiving this mail because: -------
You are the QA contact for the bug.

http://bugs.exim.org/show_bug.cgi?id=139




--- Comment #21 from Tony Finch <dot@dotat.at> 2008-05-14 16:18:16 ---
On Wed, 14 May 2008, David Woodhouse wrote:
>
> When you're building from source, why would you ever build modules
> dynamically? You can just turn off the ones you don't want (or don't
> have the necessary development environment for).

I was thinking of situations where you want one build for a few
different machines that use different sets of lookups. It saves time to
only load the lookups that are required.

> When you say "creating a unified package", you mean a package such as
> the existing distro packages, with all the lookups present -- except
> that they're as modules, not built-in? That kind of defeats the object
> of having them modular, surely?

Not entirely. Exim could ship with the ldap lookup and would work even if
the ldap libraries are not installed, and could support ldap when they
are. (I'm imagining feeble package managers with weak dependency
handling.)

> The reason I _don't_ like it is because it means you have to edit your
> config when you (or your distro) switch to building with
> dynamically-loaded modules.

A valid point.

I suppose I'm wondering if this feature can usefully be generalized to
non-linux-packagey situations, but I'm happy to be convinced that there's
no point.

Your packaging plan is exactly what I thought you had in mind, and I
expect it would suit Debian fine as well.

> > lookupapi.h:
> >
> > This file is missing the copyright and end-of-file rubric.
>
> University of Cambridge 1995-2007?

/* $Cambridge$ */

/*************************************************
* Exim - an Internet mail transport agent *
*************************************************/

/* Copyright (c) University of Cambridge 1995 - 2007 */
/* See the file NOTICE for conditions of use and distribution. */

CODE GOES HERE

/* End of lookupapi.h */

On the other hand, why not include the contents of lookupapi.h in
structs.h?

> I suspect that the end-of-file rubric might be something we could start
> to dispense with.

I quite like it :-)

> (One of these days I might even be brave enough to suggest running the
> whole thing through indent, too. It makes my brain hurt :)

No, the history is too useful to screw up for trivial reasons.

> > The LMM1 comment should be less cryptic.
>
> Can you be a little more specific about what you want it to say?

/* This magic number is used by the following lookup_module_info structure
for checking API compatibility. It's equivalent to the string "LMM1". */

> Will play with that; thanks. Which bits are gmake-only?

All the conditional stuff, and the magic $(LOOKUP_$*_INCLUDE) expansions.
(I might have missed something else.) It's probably easy to find most of
the make portability bugs by trying to build with BSD make. (Dunno what
Red Hat calls it.)

Tony.


--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at http://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[Bug 139] Dynamically loadable lookup modules [ In reply to ]
------- You are receiving this mail because: -------
You are the QA contact for the bug.

http://bugs.exim.org/show_bug.cgi?id=139




--- Comment #22 from David Woodhouse <dwmw2@infradead.org> 2008-05-14 17:09:38 ---
On Wed, 2008-05-14 at 16:18 +0100, Tony Finch wrote:
> I was thinking of situations where you want one build for a few
> different machines that use different sets of lookups. It saves time to
> only load the lookups that are required.

True. On the other hand, in such a manual installation surely you might
as well just delete the lookup modules you don't want, rather than
configuring Exim to load the ones you do?

> > When you say "creating a unified package", you mean a package such as
> > the existing distro packages, with all the lookups present -- except
> > that they're as modules, not built-in? That kind of defeats the object
> > of having them modular, surely?
>
> Not entirely. Exim could ship with the ldap lookup and would work even if
> the ldap libraries are not installed, and could support ldap when they
> are. (I'm imagining feeble package managers with weak dependency
> handling.)

Actually I think that works right now -- you can have the module
present, but it'll fail to load. Assuming your package manager lets you
install it without its dependency, of course :)

> > The reason I _don't_ like it is because it means you have to edit your
> > config when you (or your distro) switch to building with
> > dynamically-loaded modules.
>
> A valid point.
>
> I suppose I'm wondering if this feature can usefully be generalized to
> non-linux-packagey situations, but I'm happy to be convinced that there's
> no point.

I'm having trouble seeing the benefits of listing modules explicitly in
the config file, certainly. Although all we really need to overcome my
_objection_ to it is set the default behaviour to be loading all
available modules, but still add a config option which lets you list
them by name. That way, upgrades would still work fine without modifying
the config.

One other potential solution _might_ be to load lookup modules only on
demand -- so when Exim encounters '${lookup{...}FOO ...}' it'll go
looking for a FOO.so if it doesn't already recognise the lookup type in
question. Not entirely sure if that's easily extendable to all the ways
you can invoke lookups (like 'lsearch;...' etc.)

You'd also need to cope with the fact that certain modules provide more
than one type of lookup. A translation table or symbolic links could
achieve that, but makes it all seem a bit more Heath Robinson and a bit
less attractive...


--
Configure bugmail: http://bugs.exim.org/userprefs.cgi?tab=email

--
## List details at http://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##