Mailing List Archive

Yes, that works, and I?ve used that on a couple of projects.

Another alternative is defining an Abstract Base Class, https://docs.python.org/3/library/abc.html, and having an institution-specific implementation passed into your module.

From: Python-list <python-list-bounces+gweatherby=uchc.edu@python.org> on behalf of Loris Bennett <loris.bennett@fu-berlin.de>
Date: Wednesday, March 15, 2023 at 1:03 PM
To: python-list@python.org <python-list@python.org>
Subject: Implementing a plug-in mechanism
*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

Hi,

I have written a program which, as part of the non-core functionality,
contains a module to generate email. This is currently very specific
to my organisation, so the main program contains

import myorg.mailer

This module is specific to my organisation in that it can ask an
internal server to generate individualised salutations for a given UID
which is known within the organisation.

I want to share the code with other institutions, so I would like to

1. replace the organisation-specific mailer with a generic one
2. allow an organisation-specific mailer to be used instead of the
generic one, if so desired

Is importlib the way to go here or is there another approach?

Cheers,

Loris

--
This signature is currently under constuction.
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!izCZs2X5PFeGpCF4TrtdDABzPqCFFT5i89Zsu-msRJAIpyWZYybdHDOFdxno9J3JvpNsRRQK9w72qgYj0MjlB2L-LsVXW1o$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!izCZs2X5PFeGpCF4TrtdDABzPqCFFT5i89Zsu-msRJAIpyWZYybdHDOFdxno9J3JvpNsRRQK9w72qgYj0MjlB2L-LsVXW1o$>
--
https://mail.python.org/mailman/listinfo/python-list

On 16/03/2023 01.47, Loris Bennett wrote:
> I have written a program which, as part of the non-core functionality,
> contains a module to generate email. This is currently very specific
> to my organisation, so the main program contains
>
> import myorg.mailer
>
> This module is specific to my organisation in that it can ask an
> internal server to generate individualised salutations for a given UID
> which is known within the organisation.
>
> I want to share the code with other institutions, so I would like to
>
> 1. replace the organisation-specific mailer with a generic one
> 2. allow an organisation-specific mailer to be used instead of the
> generic one, if so desired

This may call for the plug-in pattern, ie the user will choose whether
to plug-in the specific, or the generic, module.

In Python, we would tend to use a Dependency Injection approach (one of
Uncle Bob's SOLID principles).

There's a rather abstract description of the plugin pattern at
https://martinfowler.com/eaaCatalog/plugin.html

OpenClassrooms has a more practical discussion at
https://openclassrooms.com/en/courses/6397806-design-your-software-architecture-using-industry-standard-patterns/6896171-plug-in-architecture

There is a PyPi library called pluggy (not used it). I've used informal
approaches using an ABC as a framework/reminder (see @George's response).

--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list

On 3/15/2023 2:45 PM, dn via Python-list wrote:
> On 16/03/2023 01.47, Loris Bennett wrote:
>> I have written a program which, as part of the non-core functionality,
>> contains a module to generate email.  This is currently very specific
>> to my organisation, so the main program contains
>>
>>    import myorg.mailer
>>
>> This module is specific to my organisation in that it can ask an
>> internal server to generate individualised salutations for a given UID
>> which is known within the organisation.
>>
>> I want to share the code with other institutions, so I would like to
>>
>>    1. replace the organisation-specific mailer with a generic one
>>    2. allow an organisation-specific mailer to be used instead of the
>>       generic one, if so desired
>
> This may call for the plug-in pattern, ie the user will choose whether
> to plug-in the specific, or the generic, module.
>
> In Python, we would tend to use a Dependency Injection approach (one of
> Uncle Bob's SOLID principles).
[snip]

Here is (slightly modified) plugin code I'm using in one project. You
could use a naming convention to see if there is a plugin for a specific
organizations, or each module could contain a UID variable which you
could inspect to find the desired one. This code is under the MIT
License, so feel free to adapt it if you like.

def import_all_plugins(plugins_import_list, plugin_dir):
"""Import modules from the plugins directory and return a list of them.

If plugins_import_list is not empty or None, only import the ones
listed there. Otherwise import all ".py" files.

RETURNS
a list of successfully imported modules.
"""
modules = []

if not plugins_import_list:
plugins_import_list = []
for root, dirs, files in os.walk(plugin_dir):
if root == plugin_dir:
break
for f in files:
f, ext = os.path.splitext(f)
if ext == '.py':
plugins_import_list.append(f)

for f in plugins_import_list:
try:
mod = importlib.import_module(f'plugins.{f}')
modules.append(mod)
except ImportError as e:
print(f'{__name__}: {f} plugin: {e}')
continue
return modules


--
https://mail.python.org/mailman/listinfo/python-list

I do something similar to Thomas. (Also MIT licensed). I like objects. I like type hints.

Each plugin needs to have check and purpose functions and accepts either PluginSpec (by default) or AddonSpec if it defines addon = True

This requires a single-level plugin directory with no extra files in it (unless they start with _, like __init__.py)


And I should use os.path.splitext but I forget what?s it called and find it easier just to split.

# noinspection PyUnresolvedReferences
@dataclass
class NamedModule:
"""Datacheck module and its name"""
mod: object
name: str

def __post_init__(self):
"""Validate attributes"""
assert hasattr(self.mod, 'check')
assert hasattr(self.mod, 'purpose')

def check(self, inspec: Union[PluginSpec, AddonSpec]) -> PluginResult:
return self.mod.check(inspec)

@property
def purpose(self) -> str:
return self.mod.purpose()

@property
def addon(self) -> bool:
"""Return true if this module uses AddonSpec"""
return getattr(self.mod, 'addon', False)


class Integrity:

@property
def plugins(self) -> List[NamedModule]:
"""Get list of plugins by scanning plugin directory"""
modules = []
us = os.path.abspath(__file__)
plugin_dir = os.path.join(os.path.dirname(us), 'plugins')
de: os.DirEntry
for de in os.scandir(plugin_dir):
if not de.name.startswith('_'):
n = de.name.split('.')[0]
mod = importlib.import_module(f'.plugins.{n}', 'dataintegrity')
modules.append(NamedModule(mod, n))
return modules
--
https://mail.python.org/mailman/listinfo/python-list

On 3/15/2023 6:06 PM, Weatherby,Gerard wrote:
> I do something similar to Thomas. (Also MIT licensed). I like objects. I like type hints.
>
> Each plugin needs to have check and purpose functions and accepts either PluginSpec (by default) or AddonSpec if it defines addon = True

I omitted the checks because they specific to the use, so I thought they
would just be a bit confusing.

--
https://mail.python.org/mailman/listinfo/python-list

On Thu, Mar 16, 2023 at 07:45:18AM +1300, dn via Python-list wrote:
>There is a PyPi library called pluggy (not used it). I've used informal
>approaches using an ABC as a framework/reminder (see @George's
>response).

typing.Protocol is also useful here as the plugin interface can be
defined separately not requiring inheriting from an ABC.

Simon
--
A complex system that works is invariably found to have evolved from a
simple system that works.—John Gall
--
https://mail.python.org/mailman/listinfo/python-list

Simon Ward <simon+python@bleah.co.uk> writes:

> On Thu, Mar 16, 2023 at 07:45:18AM +1300, dn via Python-list wrote:
>> There is a PyPi library called pluggy (not used it). I've used
>> informal approaches using an ABC as a framework/reminder (see
>> @George's response).
>
> typing.Protocol is also useful here as the plugin interface can be
> defined separately not requiring inheriting from an ABC.

Thanks to all for the helpful suggestions. I realise that I don't
actually need to be able to load a bunch of arbitrary plugins, but
merely to be able to override one (or, perhaps later, more) piece of
default behaviour.

Therefore I think the following very simple scheme will work for me:

$ tree -L 3
.
??? myproj
??? __init__.py
??? mailer.py
??? main.py
??? plugins
??? normanmailer.py

Where main.py is

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

if __name__ == "__main__":

try:
import plugin.mailer as mailer
print("Found plugin.mailer")
except ModuleNotFoundError:
import mailer
print("Found mailer")

m = mailer.Mailer('abc')
m.run()

mailer.py is

class Mailer():

def run(self):
print("This is a generic Mailer object!")

and plugins/normanmailer.py is

class Mailer():

def run(self):
print("This is a customized Mailer object!")

This then gives me

$ poetry run myproj\/main.py
Found mailer
This is a generic Mailer object!

$ mv myproj/plugins/{norman,}mailer.py

$ poetry run myproj\/main.py
Found plugins.mailer
This is a customized Mailer object!

I suspect I was using slightly incorrect/misleading terminology. I
don't want to be able to load arbitrary functionality via plugins,
e.g. sending an email, dumping to a database, uploading to a cloud.
That would, I far as I can tell, necessitate having some mechanism to
select the functionality.

Instead I just want to modify the behaviour of a piece of fixed
functionality. e.g. sending a mail. So am I really talking about
customisation here.

Cheers,

Loris

--
This signature is currently under constuction.
--
https://mail.python.org/mailman/listinfo/python-list