Mailing List Archive: Corporate installations

Corporate installations

Oct 15, 1999, 12:21 AM

Post #1 of 5 (1243 views)

Python is getting used widely enough in my place of employment that people
are agitating for a common installation (personal variants are screwing
people too often). Various machines are running Win95, Win98, NT,
Windows2000, assorted flavors of Linux, Solaris, Irix and MacOS. I'll choke
on that whole banana when it's shoved down my throat.

For starters, I just need to get it running across Windows platforms. Never
looked into this before, and it seems to be exceedingly complicated right
out of the box <0.5 wink>:

D:\Python>python
Python 1.5.2 (#0, Apr 13 1999, 10:51:12) [MSC 32 bit (Intel)] on win32
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> import sys
[.'',
'D:\\Python\\win32',
'D:\\Python\\win32\\lib',
'D:\\Python',
'D:\\Python\\Pythonwin',
'D:\\Python\\Lib\\plat-win',
'D:\\Python\\Lib',
'D:\\Python\\DLLs',
'D:\\Python\\Lib\\lib-tk',
'D:\\PYTHON\\DLLs',
'D:\\PYTHON\\lib',
'D:\\PYTHON\\lib\\plat-win',
'D:\\PYTHON\\lib\\lib-tk',
'D:\\PYTHON']
>>>

Where does all that stuff come from? The first is apparently the current
directory; cool. The next 4 appear to come from MarkH's stuff, via the
registry. The next 4 appear to come from Python's own registry PythonPath
key. The 4 after that are a mystery, and duplicate the preceding 4 (except
for meaningless-- it's Windows <wink> -- case differences). The last is
another mystery. site.py isn't implicated in any of them (same thing when
running with -S). I suppose this is a clue:

D:\Python>set PYTHONHOME=ick;fooey

D:\Python>python
Python 1.5.2 (#0, Apr 13 1999, 10:51:12) [MSC 32 bit (Intel)] on win32
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> import sys
>>> sys.path
[.'',
'D:\\Python\\win32',
'D:\\Python\\win32\\lib',
'D:\\Python',
'D:\\Python\\Pythonwin',
'D:\\Python\\Lib\\plat-win',
'D:\\Python\\Lib',
'D:\\Python\\DLLs',
'D:\\Python\\Lib\\lib-tk',
'ick',
'fooey\\DLLs',
'ick',
'fooey\\lib',
'ick',
'fooey\\lib\\plat-win',
'ick',
'fooey\\lib\\lib-tk',
'D:\\PYTHON']
>>>

Doesn't appear to be sensible, but at least it's predictable <wink>.

I was surprised to find that setting PYTHONPATH can't be used to override
any of this -- it just inserts even more stuff, into sys.path[1:1].

Do I really care? Not much -- the complexity just makes it hard to get a
mental model of what Python *thinks* it's trying to accomplish here, and so
harder to figure out what needs to be done. Which is pretty severe:
"Dragon std" apps must run the same version of Python (which can be local)
with the same libraries (which must be network-mounted), regardless of
whatever version of Python each user may have built on their own, and
regardless of their registry settings or envar values.

I think this means I need to distribute a python15.dll and python.exe (via
our source control system) into a directory very early on the PATH (other
common .exe's are distributed this way, so no big deal), and add a
sitecustomize.py that replaces all of sys.path with the Big Four
(Lib\plat-win, Lib, DLLs, and Lib\lib-tk) expressed as UNC paths, + the
common Dragon dir.

Network name mangling being what it is, I suppose I'll also need to force
PYTHONCASEOK. There's no way to do that from within Python, is there? If
not, everyone going through "unfortunate" paths in the network will have to
set that themselves.

Anyone have better ideas? That shouldn't be hard <wink>.

the-difference-between-one-user-and-two-is-infinite-ly y'rs - tim

RE: Corporate installations [ In reply to ]

mhammond at skippinet

Oct 15, 1999, 1:02 AM

Post #2 of 5 (1233 views)

Permalink

> The 4 after that are a mystery, and duplicate the
> preceding 4 (except
> for meaningless-- it's Windows <wink> -- case differences).

Python makes a brave attempt to deduce the PYTHONHOME itself by
looking for "well-known" files in the lib directory. Python15.dll
uses argv[0] or the result of GetModuleFileName(NULL) (whichever is
deemed "better" by the existance of a slash) as the basis for a search
for the landmark file. If found, those 4 entries are added and if not
they are added relatively (ie ".\lib" in the vain hope they will
suddenly become meaningful (which they almost certainly wont)

So, when running a .EXE that lives in the main Python directory (such
as Python.exe), we get duplicated entries - the ones we deduced, and
the ones explicitely in the registry.

The problem arises when Python15.dll is used by a .exe that doesnt
live in this nice directory - eg Pythonwin, or any other embedding,
such as COM objects or peoples custom apps. As argv[0] and
GetModuleFileName(NULL) both return a directory where "\lib" appended
will not find the landmark file (string.py for windows), you get the
useless relative paths added.

So, what this means is that Python's strategy for deducing the
PythonPath only works for Python.exe and Pythonw.exe - typically the
only .exes in the Python directory. So we have the situation where
the paths must also be registered in the registry if any other .exe
wants to work.

> I was surprised to find that setting PYTHONPATH can't be used
> to override
> any of this -- it just inserts even more stuff, into sys.path[1:1].

Once upon a time Guido stated that PYTHONPATH should override the
registry completely. I cant recall the history of this tho, and
wouldnt object is that was the case. You are then faced with how to
set this env variable (and if you are doing that, why not just set the
registry?)

> I think this means I need to distribute a python15.dll and
> python.exe (via
> our source control system) into a directory very early on the
> PATH (other
> common .exe's are distributed this way, so no big deal), and add a
> sitecustomize.py that replaces all of sys.path with the Big Four
> (Lib\plat-win, Lib, DLLs, and Lib\lib-tk) expressed as UNC
> paths, + the
> common Dragon dir.

This should be fine - as long as no one needs to run any other .EXE
that needs Python15.dll (or unless that other .exe is also in that
same directory).

> Network name mangling being what it is, I suppose I'll also
> need to force
> PYTHONCASEOK. There's no way to do that from within Python,
> is there? If

While my views on this have temepered from illogically ranting to
Guido how much I hate it, I still believe this is a mis-feature for
Windows.

The other alternative is to put Python.exe on the network path, and
have your library in a ".\lib" directory under that. Doesnt really
solve the problem for your "DragonLib" stuff - Im sure you dont want
to stuff _everything_ into a single .lib directory.

Also, IMO Python shouldnt bother going sniffing for its library if the
registry exists. It just adds time to startup, and if it works is
almost guaranteed to be redundant, and if it doesnt work will add
useless entries that slow down module searches. However, that still
doesnt solve your problem.

The final suggestion is to put some of the win32 extensions
(specifically, win32api) on a network share along with Python.exe and
the standard lib. Then provide a .bat file that actually sets up the
registry, and off you go. Sure, people hate the registry, but setting
an environment variable is worse. Or an alternative - provide a .reg
file with the necessary registry keys - users can "execute" it
(double-clicking from explorer, typing the name in start->run, execute
"start whatever.reg in their logon script, etc) to have their registry
setup.

I wish I had a better answer, but can't for the life of me work out
what it should be even if MS supported it!

Mark.

Re: Corporate installations [ In reply to ]

mal at lemburg

Oct 15, 1999, 1:34 AM

Post #3 of 5 (1233 views)

Permalink

Tim Peters wrote:
>
> Python is getting used widely enough in my place of employment that people
> are agitating for a common installation (personal variants are screwing
> people too often). Various machines are running Win95, Win98, NT,
> Windows2000, assorted flavors of Linux, Solaris, Irix and MacOS. I'll choke
> on that whole banana when it's shoved down my throat.
>
> For starters, I just need to get it running across Windows platforms. Never
> looked into this before, and it seems to be exceedingly complicated right
> out of the box <0.5 wink>:
>
> D:\Python>python
> Python 1.5.2 (#0, Apr 13 1999, 10:51:12) [MSC 32 bit (Intel)] on win32
> Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
> >>> import sys
> [.'',
> 'D:\\Python\\win32',
> 'D:\\Python\\win32\\lib',
> 'D:\\Python',
> 'D:\\Python\\Pythonwin',
> 'D:\\Python\\Lib\\plat-win',
> 'D:\\Python\\Lib',
> 'D:\\Python\\DLLs',
> 'D:\\Python\\Lib\\lib-tk',
> 'D:\\PYTHON\\DLLs',
> 'D:\\PYTHON\\lib',
> 'D:\\PYTHON\\lib\\plat-win',
> 'D:\\PYTHON\\lib\\lib-tk',
> 'D:\\PYTHON']
> >>>
>
> Where does all that stuff come from? The first is apparently the current
> directory; cool. The next 4 appear to come from MarkH's stuff, via the
> registry. The next 4 appear to come from Python's own registry PythonPath
> key. The 4 after that are a mystery, and duplicate the preceding 4 (except
> for meaningless-- it's Windows <wink> -- case differences). The last is
> another mystery. site.py isn't implicated in any of them (same thing when
> running with -S).

On Linux I get:

['',
'/home/lemburg/bin',
'/home/lemburg/lib',
'/home/lemburg/projects',
'/home/lemburg/python/Lib/',
'/home/lemburg/python/Lib/test',
'/home/lemburg/python/Lib/plat-linux2',
'/home/lemburg/python/Lib/lib-tk',
'/home/lemburg/python/Modules',
'/usr/local/lib/python1.5/site-packages']

It seems that the last entry refers to what Python thinks is the standard
location for system wide extensions. The first 3 come from my PYTHONPATH
setting, the next few are inserted by the Python startup code based
on what Python found as landmark to determine PYTHONHOME.

> I think this means I need to distribute a python15.dll and python.exe (via
> our source control system) into a directory very early on the PATH (other
> common .exe's are distributed this way, so no big deal), and add a
> sitecustomize.py that replaces all of sys.path with the Big Four
> (Lib\plat-win, Lib, DLLs, and Lib\lib-tk) expressed as UNC paths, + the
> common Dragon dir.

Sounds reasonable ;-)

> Network name mangling being what it is, I suppose I'll also need to force
> PYTHONCASEOK. There's no way to do that from within Python, is there? If
> not, everyone going through "unfortunate" paths in the network will have to
> set that themselves.

If Windows allows writing the process' environment you can set PYTHONCASEOK
dynamically: the check queries the environment for the flag prior
to every check (instead of doing going the usual internal Python flag
way as many of the others do). See Python/import.c:check_case() for details.

--
Marc-Andre Lemburg
______________________________________________________________________
Y2000: 77 days left
Business: http://www.lemburg.com/
Python Pages: http://www.lemburg.com/python/

Re: Corporate installations [ In reply to ]

jim at interet

Oct 15, 1999, 8:22 AM

Post #4 of 5 (1231 views)

Permalink

Tim Peters wrote:
> [.'',
> 'D:\\Python\\win32',
> 'D:\\Python\\win32\\lib',
> 'D:\\Python',
> 'D:\\Python\\Pythonwin',
> 'D:\\Python\\Lib\\plat-win',
> 'D:\\Python\\Lib',
> 'D:\\Python\\DLLs',
> 'D:\\Python\\Lib\\lib-tk',
> 'D:\\PYTHON\\DLLs',
> 'D:\\PYTHON\\lib',
> 'D:\\PYTHON\\lib\\plat-win',
> 'D:\\PYTHON\\lib\\lib-tk',
> 'D:\\PYTHON']
> >>>
>
> Where does all that stuff come from?

Good question. I don't know, and every time I start to
think I do, I am wrong. Certainly PYTHONPATH is full of
junk on Windows.

The point is that PYTHONPATH just doesn't work for commercial
installations. Please see these for some discussion:

ftp://ftp.interet.com/bootmodule.html
ftp://ftp.interet.com/pylib.html

I think the solution is to put the library
files and packages into archive files (in some format yet
to be determined) and find them by looking in the
directory of python15.dll and/or python.exe. On Unix
there is $prefix, which actually seems to work.

> I think this means I need to distribute a python15.dll and python.exe (via
> our source control system) into a directory very early on the PATH

Yes, that is a good start. Just understand that *.pyd and and DLL's
which are imported
(instead of linked against) are found from PYTHONPATH, not PATH. Also
you will need a valid PATH on all PC's which is quite a bother. An
alternative is to use the Wise installer to set up icons for your
apps. Real Windows apps don't use PATH. We use Wise for internal
as well as external apps.

Note that if you import a module "import foo", and foo
lives in "foo.dll", then on Windows, PYTHONPATH is searched, and
the standard Windows DLL path is not searched. That means you
can't use standard Windows locations for DLL's which are dynamically
imported. This is an error and should be fixed.

> and add a
> sitecustomize.py that replaces all of sys.path with the Big Four
> (Lib\plat-win, Lib, DLLs, and Lib\lib-tk)

The problem is that you already need a valid PYTHONPATH to
find sitecustomize.py. Better is to create a custom python15.dll
with sitecustomize.py, exceptions.py and site.py (all needed during
boot)
frozen in as frozen modules. In our office I freeze in other library
files too.

> Network name mangling being what it is, I suppose I'll also need to force
> PYTHONCASEOK.

I don't seem to need that on 95/98/NT. Network names shouldn't
get mangled on these systems. I guess you mean lack of case
sensitivity, but just make sure the case is correct in the Lib
directory. This is another reason to prefer archive files anyway.
But no, you shouldn't need PYTHONCASEOK.

> Anyone have better ideas? That shouldn't be hard <wink>.

Yes, this is a major problem with Python's acceptance for
commercial apps (and I include internal production code as
an app) so lets figure out what to do and fix it.

Jim Ahlstrom

Re: Corporate installations [ In reply to ]

jim at interet

Oct 15, 1999, 11:19 AM

Post #5 of 5 (1236 views)

Permalink

"James C. Ahlstrom" wrote:
Sorry, correct URL is:

ftp://ftp.interet.com/pub/bootmodule.html
ftp://ftp.interet.com/pub/pylib.htm

Jim Ahlstrom