Mailing List Archive

1.5.2 broke IRIX module loading
--wac7ysb48OaltWcw
Content-Type: text/plain; charset=us-ascii

I built and tried 1.5.2 this morning, and it failed to load a module
that worked fine on 1.5.1. See below for the errors.

This is a C library I don't have source for which has references to a
number of zeroed-out internal symbols which aren't used. In this case (see
errors), "currstepc" is the exported C interface wrapper, and "currstep" is
the internal FORTRAN routine which has been zeroed-out.

To convince Python to load this again, I found I had to reverse one of
the 1.5.1 changes to importdl.c (patch attached). I believe this news post
describes the problem I'm running into:

http://www.dejanews.com/[ST_rn=ps]/getdoc.xp?AN=289722207

Could we consider reversing this change to importdl.c in the Python dist?

Randall

-------------------------------------------------------------------------------

> python
Python 1.5.2 (#1, Apr 14 1999, 08:42:47) [C] on irix646-o32
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> import Util
Traceback (innermost last):
File "<stdin>", line 1, in ?
File "Util.py", line 2, in ?
import Utilc
ImportError: 1380391:python-o32: rld: Fatal Error: unresolvable symbol in /home/rhh/work/libUtil.so: currstep

-------------------------------------------------------------------------------

> nm -Bo libUtil.so | grep currstep
libModels3.so: 5ffccda0 B currstep
libModels3.so: 5fe1e6c0 T currstep_
libModels3.so: 5fe1e880 T currstepc

-------------------------------------------------------------------------------


--wac7ysb48OaltWcw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="Python-1.5.2-IRIX-dload.patch"

--- Python/ORIG/importdl.c Wed Jan 27 12:53:10 1999
+++ Python/importdl.c Wed Apr 14 10:41:40 1999
@@ -441,7 +441,7 @@
#ifdef RTLD_NOW
/* RTLD_NOW: resolve externals now
(i.e. core dump now if some are missing) */
- void *handle = dlopen(pathname, RTLD_NOW);
+ void *handle = dlopen(pathname, RTLD_NOW | RTLD_GLOBAL);
#else
void *handle;
if (Py_VerboseFlag)

--wac7ysb48OaltWcw--
1.5.2 broke IRIX module loading [ In reply to ]
Randall Hopper <aa8vb@vislab.epa.gov> said:
> I built and tried 1.5.2 this morning, and it failed to load a module
> that worked fine on 1.5.1.

We had the same problem on our SGIs. In our case, our vendors
accidentally shipped .a files with functions which reference other
functions of theirs which they didn't ship.

We used SWIG to create a .so files from the .a, and with the
removal of RTLD_GLOBAL, this means we can no longer import our
module with 1.5.2 .

This change was discuessed on the newsgroup some months back (sorry,
don't have a DejaNews reference handy) and the outcome was that
libraries shipped in this form are broken, and it must be done this
way so different shared libraries don't get namespace collisions.

For us there were two ways around it:
1) upgrade to our vendor's newest libraries
2) create dummmy C functions which resolve the functions

> "currstep" is the internal FORTRAN routine which has been zeroed-out.

But I don't know what zeroed-out means in this context. For me
I found which were missing with "nm library.so | grep UNDEFINED"


I mentioned this problem to Guido when I tested 1.5.2c1 on our
SGIs; not that it had to be changed back but that people were going
to find this change unexpected. Here are his comments:

Guido said:
| Unfortunately the conclusion of the debate about this was unanimously
| that using RTLD_GLOBAL is evil; even though it solves some real
| practical problems in some cases (like yours), it can break other
| things for which the only fix is to remove RTLD_GLOBAL. The cases
| where it is needed are always caused by bad programming (e.g. your
| vendor's). Sorry.
|
| [....]
|
| > And, if not, I'll just put "| RTLD_GLOBAL" into my dlopen.
|
| That would be the easiest solution for you, as long as you don't run
| into the other problem (conflicting externals defined in two
| libraries).

Andrew Dalke
dalke@bioreason.com
1.5.2 broke IRIX module loading [ In reply to ]
Thanks for the reply.

|> "currstep" is the internal FORTRAN routine which has been zeroed-out.
|
| But I don't know what zeroed-out means in this context. For me
|I found which were missing with "nm library.so | grep UNDEFINED"

I don't know exactly how they built this library, but somehow they
internalized the FORTRAN function names so that they are not exported by
the FORTRAN objects (e.g. currstep in currstep.o), while a dangling
reference exists to them in the C wrapper object (e.g. currstepc.o):

Symbols from currstep.o:

[Index] Value Size Class Type Section Name

[0] | 0| |File |ref=16 |Text | /tmp_mnt/pub/storage/xcc/work/m3io/currstep.f
[1] | 0| |Proc |end=15 unsigned long |Text | currstep_
->[2] | -8| |Local |unsigned long |Abs | currstep

Symbols frstepc

[Index] Class Type Section Name

[0] | 0| |File |ref=15 |Text | /tmp_mnt/pub/storage/xcc/work/m3io/currstepc.c
[1] | 0| |Proc |end=14 int |Text | currstepc
[13] | 420| |End |ref=1 |Text | currstepc
->[101] | 0| |Proc | |Undefined| currstep


If one links with and calls any of these wrappers (e.g. currstepc()), it
works fine.

Randall
1.5.2 broke IRIX module loading [ In reply to ]
Thanks for the reply.

|> "currstep" is the internal FORTRAN routine which has been zeroed-out.
|
| But I don't know what zeroed-out means in this context. For me
|I found which were missing with "nm library.so | grep UNDEFINED"

I don't know exactly how they built this library, but somehow they
internalized the FORTRAN function names so that they are not exported by
the FORTRAN objects (e.g. currstep in currstep.o), while a dangling
reference exists to them in the C wrapper object (e.g. currstepc.o):

Symbols from currstep.o:

[Index] Value Size Class Type Section Name

[0] | 0| |File |ref=16 |Text | /tmp_mnt/pub/storage/xcc/work/m3io/currstep.f
[1] | 0| |Proc |end=15 unsigned long |Text | currstep_
->[2] | -8| |Local |unsigned long |Abs | currstep

Symbols frstepc

[Index] Class Type Section Name

[0] | 0| |File |ref=15 |Text | /tmp_mnt/pub/storage/xcc/work/m3io/currstepc.c
[1] | 0| |Proc |end=14 int |Text | currstepc
[13] | 420| |End |ref=1 |Text | currstepc
->[101] | 0| |Proc | |Undefined| currstep


If one links with and calls any of these wrappers (e.g. currstepc()), it
works fine.

Randall
1.5.2 broke IRIX module loading [ In reply to ]
In article <37161C26.C121CA1E@bioreason.com>,
Andrew Dalke <dalke@bioreason.com> wrote:
> Randall Hopper <aa8vb@vislab.epa.gov> said:
> > I built and tried 1.5.2 this morning, and it failed to load a module
> > that worked fine on 1.5.1.
>

Does this failure occur for all modules including standard
Python Library modules or is it limited to some specialized module?

Scott Rowland

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/ Search, Read, Discuss, or Start Your Own
1.5.2 broke IRIX module loading [ In reply to ]
srowland@biocryst.com:
|> Randall Hopper:
|> > I built and tried 1.5.2 this morning, and it failed to load a module
|> > that worked fine on 1.5.1.
|
|Does this failure occur for all modules including standard
|Python Library modules or is it limited to some specialized module?

The latter. It's an unresolved symbol issue that before didn't make any
difference. The new dlopen flags basically force every symbol to resolve
at link time whether they are used or not.

Randall