Mailing List Archive

Polymorphic imports
I guess this is kind of like mocking for testing. I have a simple module that's imported in a number of other spots in my program. There's a condition in the OS/filesystem where I'd like to import a polymorphically compatible variant of the same module. Can this be accomplished in a sort of once-and-only once spot?

For example, consider something like this:

client/
module_a
module_a_prime
lib/
paths
lib_a
lib_b
...
model/
model_a
model_b
...
top_level_a
top_level_b
...


I have a number of imports of module_a. I have a paths module that isolates all of my file system access, and that's where the determination can be made which one to use, so I tried to do something like:

def dynamic_client_module():
return client.module_a_prime if the_condition_occurs else client.module_a


Hoping that I could do something like

from lib import paths
import paths.dynamic_client_module()

But this seems to not work. Import can only take real modules? Not programatic ones?

Is there a Not-Too-Evil-Way(tm) to add a level of programmatic indirection in the import declarations? Or some other trick from a different angle?
--
https://mail.python.org/mailman/listinfo/python-list
Re: Polymorphic imports [ In reply to ]
On Wed, Sep 22, 2021 at 4:59 AM Travis Griggs <travisgriggs@gmail.com> wrote:
>
> I guess this is kind of like mocking for testing. I have a simple module that's imported in a number of other spots in my program. There's a condition in the OS/filesystem where I'd like to import a polymorphically compatible variant of the same module. Can this be accomplished in a sort of once-and-only once spot?
>
> For example, consider something like this:
>
> client/
> module_a
> module_a_prime
> lib/
> paths
> lib_a
> lib_b
> ...
> model/
> model_a
> model_b
> ...
> top_level_a
> top_level_b
> ...
>
>
> I have a number of imports of module_a. I have a paths module that isolates all of my file system access, and that's where the determination can be made which one to use, so I tried to do something like:
>
> def dynamic_client_module():
> return client.module_a_prime if the_condition_occurs else client.module_a
>
>
> Hoping that I could do something like
>
> from lib import paths
> import paths.dynamic_client_module()
>
> But this seems to not work. Import can only take real modules? Not programatic ones?
>
> Is there a Not-Too-Evil-Way(tm) to add a level of programmatic indirection in the import declarations? Or some other trick from a different angle?

You can dynamically import modules using importlib.import_module(),
but an easier way might just be a conditional import:

# client/__init__.py
if some_condition:
import module_a_default as module_a
else:
import module_a_prime as module_a

Now everything that refers to client.module_a.whatever will get the
appropriate one, either the original or the alternate.

Alternatively, since you are talking about paths, it might be easiest
to give everything the same name, and then use sys.path to control
your import directories. Not sure which would work out best.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Polymorphic imports [ In reply to ]
On 2021-09-22 at 05:10:02 +1000,
Chris Angelico <rosuav@gmail.com> wrote:

> You can dynamically import modules using importlib.import_module(),
> but an easier way might just be a conditional import:
>
> # client/__init__.py
> if some_condition:
> import module_a_default as module_a
> else:
> import module_a_prime as module_a
>
> Now everything that refers to client.module_a.whatever will get the
> appropriate one, either the original or the alternate.

+1

> Alternatively, since you are talking about paths, it might be easiest
> to give everything the same name, and then use sys.path to control
> your import directories. Not sure which would work out best.

-1

Please don't do that. Mutable shared and/or global state (i.e.,
sys.paths) is the root of all evil. And homegrown crypto and date
libraries. And those funny red hats.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Polymorphic imports [ In reply to ]
On Wed, Sep 22, 2021 at 6:05 AM <2QdxY4RzWzUUiLuE@potatochowder.com> wrote:
>
> On 2021-09-22 at 05:10:02 +1000,
> Chris Angelico <rosuav@gmail.com> wrote:
>
> > You can dynamically import modules using importlib.import_module(),
> > but an easier way might just be a conditional import:
> >
> > # client/__init__.py
> > if some_condition:
> > import module_a_default as module_a
> > else:
> > import module_a_prime as module_a
> >
> > Now everything that refers to client.module_a.whatever will get the
> > appropriate one, either the original or the alternate.
>
> +1
>
> > Alternatively, since you are talking about paths, it might be easiest
> > to give everything the same name, and then use sys.path to control
> > your import directories. Not sure which would work out best.
>
> -1
>
> Please don't do that. Mutable shared and/or global state (i.e.,
> sys.paths) is the root of all evil. And homegrown crypto and date
> libraries. And those funny red hats.

All depends on whether this is a script/application or a library. If
it's a library, then I agree, don't mutate sys.path, don't change the
working directory, etc, etc, etc. But applications are free to do
those sorts of things. I don't know what the OP's purpose here is, and
it's entirely possible that sys.path switching is the cleanest way to
do it.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Polymorphic imports [ In reply to ]
On Tue, 21 Sep 2021 18:58:31 +0000, Travis Griggs <travisgriggs@gmail.com>
declaimed the following:


>from lib import paths
>import paths.dynamic_client_module()
>
>But this seems to not work. Import can only take real modules? Not programatic ones?

Consider "import" to be equivalent to a compile-time operation.

https://docs.python.org/3/using/cmdline.html#environment-variables cf
PYTHONPATH

https://docs.python.org/3/library/sys.html#sys.path
"""
sys.path

A list of strings that specifies the search path for modules.
Initialized from the environment variable PYTHONPATH, plus an
installation-dependent default.

As initialized upon program startup, the first item of this list,
path[0], is the directory containing the script that was used to invoke the
Python interpreter. If the script directory is not available (e.g. if the
interpreter is invoked interactively or if the script is read from standard
input), path[0] is the empty string, which directs Python to search modules
in the current directory first. Notice that the script directory is
inserted before the entries inserted as a result of PYTHONPATH.

A program is free to modify this list for its own purposes. Only
strings and bytes should be added to sys.path; all other data types are
ignored during import.
"""

So... Putting the module variants into separate directories, and
modifying the import path to reference the directory of a specific variant,
might do what you want -- as long as the module name itself remains fixed.

The other alternative may be
https://docs.python.org/3/library/functions.html#__import__


--
Wulfraed Dennis Lee Bieber AF6VN
wlfraed@ix.netcom.com http://wlfraed.microdiversity.freeddns.org/

--
https://mail.python.org/mailman/listinfo/python-list
Re: Polymorphic imports [ In reply to ]
On Thu, Sep 23, 2021 at 4:20 AM Dennis Lee Bieber <wlfraed@ix.netcom.com> wrote:
>
> The other alternative may be
> https://docs.python.org/3/library/functions.html#__import__
>

I wouldn't recommend calling a dunder. If you just want to pass a text
string and get back a module, importlib is a better choice.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list