Mailing List Archive

Linux why not fully dynamic?
Why doesn't ./configure set up for a libpython.so.1.5.2 or whatever?
Also does anyone understand how the RH setup knows where the .py & .so
files will be stored?
--
Robin Becker
Linux why not fully dynamic? [ In reply to ]
Gordon McMillan wrote:
>
> Robin Becker asks:
>
> > Why doesn't ./configure set up for a libpython.so.1.5.2 or whatever?
>
> Don't know. Maybe because this kind of scheme doesn't work on
> some other Unixen?
>

Tcl manages it and AFAIK Tcl runs on the same systems as Python will.
Also Python already has support for loadable modules and backlinking
so adding support for a shared library is actually very easy.

--
Paul Duffin
DT/6000 Development Email: pduffin@hursley.ibm.com
IBM UK Laboratories Ltd., Hursley Park nr. Winchester
Internal: 7-246880 International: +44 1962-816880
Linux why not fully dynamic? [ In reply to ]
Robin Becker asks:

> Why doesn't ./configure set up for a libpython.so.1.5.2 or whatever?

Don't know. Maybe because this kind of scheme doesn't work on
some other Unixen?

> Also does anyone understand how the RH setup knows where the .py &
> .so files will be stored? -- Robin Becker

It tries all kinds of things (and it varies by platform). For *nix,
read the comments at the top of Modules/getpath.c. The common case
is ${PREFIX}/lib/python${VERSION} and <ditto>/lib-dynload. There are
completely different getpath.c modules for Windows and OS/2.

- Gordon
Linux why not fully dynamic? [ In reply to ]
Robin Becker writes:
> Why doesn't ./configure set up for a libpython.so.1.5.2 or whatever?

This is an FAQ, see:

http://www.python.org/doc/FAQ.html#3.30
Linux why not fully dynamic? [ In reply to ]
Gordon McMillan wrote:
>
> Robin Becker asks:
>
> > Why doesn't ./configure set up for a libpython.so.1.5.2 or whatever?
>
> Don't know. Maybe because this kind of scheme doesn't work on
> some other Unixen?

This question (or something similar to it) is answered in the Python
FAQ:
http://www.python.org/doc/FAQ.html#3.30

-----------------------------------------------------------------------
3.30. Why is the Python interpreter not built as a shared
library?

(This is a Unix question; on Mac and Windows, it is a shared library.)

It's just a nightmare to get this to work on all different
platforms. Shared library portability is a pain. And yes, I know about
GNU libtool -- but it requires me to use its conventions for filenames
etc, and it would require a complete and utter rewrite of all the
makefile and config tools I'm currently using.

In practice, few applications embed Python -- it's much more common to
have Python extensions, which already are shared libraries. Also,
serious embedders often want total control over which Python version
and configuration they use so they wouldn't want to use a standard
shared library anyway. So while the motivation of saving space when
lots of apps embed Python is nice in theory, I doubt that it will save
much in practice. (Hence the low priority I give to making a shared
library.)
-----------------------------------------------------------------------

--
Mitch Chapman
chapman@bioreason.com
Linux why not fully dynamic? [ In reply to ]
In message <14250.62239.580394.67647@buffalo.fnal.gov>, Charles G
Waldman <cgw@fnal.gov> writes
>Robin Becker writes:
> > Why doesn't ./configure set up for a libpython.so.1.5.2 or whatever?
>
>This is an FAQ, see:
>
> http://www.python.org/doc/FAQ.html#3.30
>
Thanks.

--
Robin Becker
Linux why not fully dynamic? [ In reply to ]
Mitch Chapman wrote:
>
> Gordon McMillan wrote:
> >
> > Robin Becker asks:
> >
> > > Why doesn't ./configure set up for a libpython.so.1.5.2 or whatever?
> >
> > Don't know. Maybe because this kind of scheme doesn't work on
> > some other Unixen?
>
> This question (or something similar to it) is answered in the Python
> FAQ:
> http://www.python.org/doc/FAQ.html#3.30
>
> -----------------------------------------------------------------------
> 3.30. Why is the Python interpreter not built as a shared
> library?
>
> (This is a Unix question; on Mac and Windows, it is a shared library.)
>
> It's just a nightmare to get this to work on all different
> platforms. Shared library portability is a pain. And yes, I know about
> GNU libtool -- but it requires me to use its conventions for filenames
> etc, and it would require a complete and utter rewrite of all the
> makefile and config tools I'm currently using.
>
> In practice, few applications embed Python -- it's much more common to
> have Python extensions, which already are shared libraries. Also,
> serious embedders often want total control over which Python version
> and configuration they use so they wouldn't want to use a standard
> shared library anyway. So while the motivation of saving space when
> lots of apps embed Python is nice in theory, I doubt that it will save
> much in practice. (Hence the low priority I give to making a shared
> library.)
> -----------------------------------------------------------------------

AFAIK this is a quote from GvR which was given a long time ago, I would
be interested to know if he has changed his mind on this.

Whenever anyone asks this question they "eventually" get pointed to this
quote. I think that there have been enough people asking about this to
invalidate the primary reason for this being given such a low profile.

Considering the fact that Python already has support for dynamic
extensions and that it uses backlinking which does not care whether
the symbols are in a library or an executable it is not as difficult
as Guido thinks.

Unfortunately because of this statement people are reluctant to do
the necessary work to support this because they do not feel that it
will be added into the code. Instead they workaround it or do enough
for themselves and so the problem never gets fixed.

--
Paul Duffin
DT/6000 Development Email: pduffin@hursley.ibm.com
IBM UK Laboratories Ltd., Hursley Park nr. Winchester
Internal: 7-246880 International: +44 1962-816880
Linux why not fully dynamic? [ In reply to ]
In article <37AB003C.28D9@mailserver.hursley.ibm.com>, Paul Duffin
<pduffin@mailserver.hursley.ibm.com> writes
...
>>
>> (This is a Unix question; on Mac and Windows, it is a shared library.)
>>
>> It's just a nightmare to get this to work on all different
>> platforms. Shared library portability is a pain. And yes, I know about
>> GNU libtool -- but it requires me to use its conventions for filenames
>> etc, and it would require a complete and utter rewrite of all the
>> makefile and config tools I'm currently using.
...
>> In practice, few applications embed Python -- it's much more common to
>> have Python extensions, which already are shared libraries. Also,
>> serious embedders often want total control over which Python version
>> and configuration they use so they wouldn't want to use a standard
>> shared library anyway. So while the motivation of saving space when
>> lots of apps embed Python is nice in theory, I doubt that it will save
>> much in practice. (Hence the low priority I give to making a shared
>> library.)
>> -----------------------------------------------------------------------
The windows version uses DLLs and there is a mechanism already for
extensions to know what version of python they're built for. Surely most
unices are just as good as if not better than win32 when it comes to
preventing DLL hell.

My view on python is that much of the success in win32 comes from the
great work done by Mark Hammond & buddies which allows python to be used
easily as a com object generator/interfacer etc etc. Some of the
problems come from the fact that python15.dll doesn't work exactly the
same as expected ie it doesn't know where python.exe is etc. PythonCOM
makes use of DLL's for inproc servers.

Can CORBA and the like make use of python in the same kind of way and
would this be embedding?
>
>AFAIK this is a quote from GvR which was given a long time ago, I would
>be interested to know if he has changed his mind on this.
>
>Whenever anyone asks this question they "eventually" get pointed to this
>quote. I think that there have been enough people asking about this to
>invalidate the primary reason for this being given such a low profile.
>
>Considering the fact that Python already has support for dynamic
>extensions and that it uses backlinking which does not care whether
>the symbols are in a library or an executable it is not as difficult
>as Guido thinks.
>
>Unfortunately because of this statement people are reluctant to do
>the necessary work to support this because they do not feel that it
>will be added into the code. Instead they workaround it or do enough
>for themselves and so the problem never gets fixed.
>
Paul was a major force behind the stubs mechanism in Tcl (I believe
because of failings in AIX). The stubs mechanism allows versions to be
more independent and overcomes deficiencies in inter extension linking.
There is certainly less of this in python as extensions tend to
implement specific python objects and normally don't refer across
directly to the internals of other extensions.

However, I see no reason why python shouldn't be split into smaller bits
if only to reduce the footprint. The parser module is unwanted for many
applications and certainly under Linux the executable size came down by
40% when the *shared* Setup option was used.
--
Robin Becker
Linux why not fully dynamic? [ In reply to ]
Robin Becker wrote:
>
> In article <37AB003C.28D9@mailserver.hursley.ibm.com>, Paul Duffin
> <pduffin@mailserver.hursley.ibm.com> writes
> > [snip]
> Paul was a major force behind the stubs mechanism in Tcl (I believe
> because of failings in AIX). The stubs mechanism allows versions to be
> more independent and overcomes deficiencies in inter extension linking.
> There is certainly less of this in python as extensions tend to
> implement specific python objects and normally don't refer across
> directly to the internals of other extensions.
>

I cannot remember exactly why I did it now but I seem to remember
something about Jean-Claude Wippler holding a swirling disk in front
of my face .....@-|

--
Paul Duffin
DT/6000 Development Email: pduffin@hursley.ibm.com
IBM UK Laboratories Ltd., Hursley Park nr. Winchester
Internal: 7-246880 International: +44 1962-816880