Mailing List Archive

Relative package imports
Although David has already copy-posted a message regarding this issue
to the list, I would like to restate the problem to get a discussion
going (and then maybe take it to c.l.p for general flaming ;).

The problem we have run into on starship is that some well-known
packages have introduced naming conflicts leading to the unfortunate
situation that they can't be all installed on the same default
path:

1. Zope has a module named DateTime which also is the base name of the
package mxDateTime.

2. Both Zope and PIL have a top-level module named ImageFile.py
(different ones of course).

Now the problem is how to resolve these issues. One possibility
is turning Zope and PIL into proper packages altogether. To
ease this transition, one would need a way to specify relative
intra-package imports and a way to tell pickle where to look
for modules/packages.

The next problem we'd probably run into sooner or later is that
there are quite a few useful top-level modules with generic
names that will conflict with package names and other modules
with the same name.

I guess we'd need at least three things to overcome this situation once
and for all ;-):

1. Provide a way to do relative imports, e.g. a single dot could
be interpreted as "parent package":

modA.py
modD.py
[A]
modA.py
modB.py
[B]
modC.py
modD.py

In modC.py:

from modD import * (works as usual: import A.B.modD)
from .modA import * (imports A.modA)
from ..modA import * (import the top-level modA)

2. Establish a general vendor based naming scheme much like the one
used in the Java world:

from org.python.core import time,os,string
from org.zope.core import *
from com.lemburg import DateTime
from com.pythonware import PIL

3. Add a way to prevent double imports of the same file.

This is the mayor gripe I have with pickle currently, because intra-
package imports often lead to package modules being imported
twice leading to many strange problems (e.g. splitting class
hierarchies, problems with isinstance() and issubclass(), etc.), e.g.

from org.python.core import UserDict
u = UserDict.UserDict()
import UserDict
v = UserDict.UserDict()

Now u and v will point to two different classes:
>>> u.__class__
<class org.python.core.UserDict.UserDict at 80d3b48>
>>> v.__class__
<class UserDict.UserDict at 80aed18>

4. Add some kind of redirection or lookup hook to pickle et al.
so that imports done during unpickling can be redirected to the
correct (possibly renamed) package.

--
Marc-Andre Lemburg
______________________________________________________________________
Y2000: 196 days left
Business: http://www.lemburg.com/
Python Pages: http://www.lemburg.com/python/
Re: Relative package imports [ In reply to ]
> 2. Both Zope and PIL have a top-level module named ImageFile.py
> (different ones of course).
>
> Now the problem is how to resolve these issues. One possibility
> is turning Zope and PIL into proper packages altogether. To
> ease this transition, one would need a way to specify relative
> intra-package imports and a way to tell pickle where to look
> for modules/packages.

fwiw, PIL 1.0b1 can already be used as a package, but you
have to explicitly import the file format handlers you need:


from PIL import Image
import PIL.GifImagePlugin
import PIL.PngImagePlugin
import PIL.JpegImagePlugin

etc. this has been fixed in PIL 1.0 final.

</F>
Re: Relative package imports [ In reply to ]
It seems that there is not much interest in the topic...

I'll be offline for the next two weeks -- maybe someone could
pick the thread up and toss it around a bit while I'm away.

Thanks,
--
Marc-Andre Lemburg
______________________________________________________________________
Y2000: 193 days left
Business: http://www.lemburg.com/
Python Pages: http://www.lemburg.com/python/
RE: Relative package imports [ In reply to ]
> It seems that there is not much interest in the topic...
>
> I'll be offline for the next two weeks -- maybe someone could
> pick the thread up and toss it around a bit while I'm away.

OK - here are my 2c on it:

Unless I am mistaken, this problem could be solved with 2 steps:
* Code moves to Python packages.
* The standard Python library move to a package.

If all non-trivial Python program used packages, and some agreement on a
standard namespace could be met, I think it would be addressed. There was
a thread on the newsgroup about the potential naming of the standard
library.

You did state as much in your proposal - indeed, you state "to ease the
transition". Personally, I dont think it is worth it, mainly because we
end up with a half-baked scheme purely for the transition, but one that can
never be removed.

To me, the question is one of:

* Why arent Zope/PIL capable of being used as packages.
* If they are (as I understand to be the case) why do people choose not to
use them as such, or why do the authors not recommend this?
* Is there a deficiency in the package scheme that makes it hard to use?
Eg, should "__" that ni used for the parent package be reinstated?

Mark.
Re: Relative package imports [ In reply to ]
Mark Hammond wrote:
> * Why arent Zope/PIL capable of being used as packages.

PIL can be used as a package ("from PIL import Image"), assuming
that it's installed under a directory in your path. there's one pro-
blem in 1.0b1, though: you have to explicitly import the file format
handlers you need:

import PIL.JpegImagePlugin
import PIL.PngImagePlugin

this has been fixed in 1.0 final.

> * If they are (as I understand to be the case) why do people choose not to
> use them as such, or why do the authors not recommend this?

inertia, and compatibility concerns. we've decided that all
official material related to PIL 1.0 will use the old syntax (and
all 1.X releases will be possible to install using the PIL.pth
approach). too many users out there...

now, PIL 2.0 is a completely different thing...

> * Is there a deficiency in the package scheme that makes it hard to use?

not that I'm aware...

</F>
Re: Relative package imports [ In reply to ]
Mark Hammond wrote:
>
> > It seems that there is not much interest in the topic...
> >
> > I'll be offline for the next two weeks -- maybe someone could
> > pick the thread up and toss it around a bit while I'm away.
>
> OK - here are my 2c on it:
>
> Unless I am mistaken, this problem could be solved with 2 steps:
> * Code moves to Python packages.
> * The standard Python library move to a package.
>
> If all non-trivial Python program used packages, and some agreement on a
> standard namespace could be met, I think it would be addressed. There was
> a thread on the newsgroup about the potential naming of the standard
> library.
>
> You did state as much in your proposal - indeed, you state "to ease the
> transition". Personally, I dont think it is worth it, mainly because we
> end up with a half-baked scheme purely for the transition, but one that can
> never be removed.

With "easing the transition" I ment introducing a way to do relative
package imports: you don't need relative imports if you can be
sure that the package name will never change (with a fixed naming
scheme, a la com.domain.product.package...). The smarter import
mechanism is needed to work-around the pickle problems you face
(because pickle uses absolute package names).

> To me, the question is one of:
>
> * Why arent Zope/PIL capable of being used as packages.
> * If they are (as I understand to be the case) why do people choose not to
> use them as such, or why do the authors not recommend this?
> * Is there a deficiency in the package scheme that makes it hard to use?
> Eg, should "__" that ni used for the parent package be reinstated?

I guess this would help a great deal; although I'd personally
wouldn't like yet another underscore in the language. Simply leave
the name empty as in '.submodule' or '..subpackage.submodule'.

Cheers,
--
Marc-Andre Lemburg
______________________________________________________________________
Y2000: 193 days left
Business: http://www.lemburg.com/
Python Pages: http://www.lemburg.com/python/