Mailing List Archive

Cross-references between dynamically loaded modules under AIX
I am tuning a C program into a python module. The program contains
some very large arrays of doubles, that I would like to become NumPy
arrays in Python. I wrote a small function to convert a C pointer to
a NumPy array (I am using SWIG), the program is appended below. It
works fine on an alpha processor running OSF1 V4.0, but it coredumps
under AIX version 4.1.5. The coredump occurs when PyArray_FromDims is
called. It looks like the dynamical loader under AIX cannot resolve
symbols in one module that refers to another module. It is probably
related to the primitive way dynamic loading is working under AIX.
The modules are created by the ld_so_aix, which is using a text file
with the symbols defined in the python executable (created when the
python executable was linked). I assume that the linker cannot link
the references to symbols in the NumPy modules without similar
information.

Does anyone have a solution for this problem? Repeating the relevant
code from the NumPy source would not be very nice, as it is quite a
bit that would have to be repeated.


Best regards,

Jakob Schiotz


Here is the interface file for SWIG. Vector is typedef'ed to
double[3] in main.h.
------------------------------------------------------------
/* Emacs: Use -*- C -*- mode for this stuff */
%module fast

%title "FAST - a molecular dynamics module",keep

%{
#include "main.h"
#include "arrayobject.h"
%}

%include fastcommon.i

/* Initialize FAST, reading input files */
CmFile *initfast(char *potfilename, char *infilename, int wantclass,
int dynamics, int frame, int hmode);


%init %{
setdefaults();
fprintf(stderr, "FAST: %s %s\n", __DATE__, __TIME__);
%}

/* Access functions for global arrays of Vector */
%{
PyObject *my_Vectors_As_NumPy(PyObject *self, PyObject *args)
{
char *vectorp;
double *cdata;
PyArrayObject *numpy;
int dims[2];

/* Read the argument, it should be a string encoding a pointer */
if (!PyArg_ParseTuple(args, "s", &vectorp))
return NULL;

/* Decode the pointer */
if (SWIG_GetPtr(vectorp, (void **) &cdata, "_Vector_p"))
{
PyErr_SetString(PyExc_TypeError, "Not a pointer to Vector");
return NULL;
}

/* Create an empty numpy array of dimension nAtoms * 3 */
dims[0] = nAtoms;
dims[1] = 3;
numpy = (PyArrayObject *) PyArray_FromDims(2, dims, PyArray_DOUBLE);

/* Copy the data to the numpy array */
memcpy(numpy->data, cdata, dims[0]*dims[1]*sizeof(double));

return (PyObject*) numpy;
}
%}

%native(Vectors_As_NumPy) extern PyObject *my_Vectors_As_NumPy(PyObject *self,
PyObject *args);

--
Jakob Schiotz, CAMP and Department of Physics, Tech. Univ. of Denmark,
DK-2800 Lyngby, Denmark. http://www.fysik.dtu.dk/~schiotz/
This email address is used for newsgroups and mailing lists
(spam protection). Official email: schiotz @ fysik . dtu . dk

-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/ Search, Read, Discuss, or Start Your Own
Cross-references between dynamically loaded modules under AIX [ In reply to ]
Jakob Schiotz:
|I wrote a small function to convert a C pointer to a NumPy array (I am
|using SWIG), ... it coredumps under AIX ... the dynamical loader under
|AIX cannot resolve symbols in one module that refers to another module.

Not knowing NumPy, my assumption is you are failing on a .so-to-.so module
dynamic link. If so...

I did something like this with SWIG recently myself, albeit on IRIX. Does
AIX have the equivalent of an RPATH (check the ld man page, and search for
rpath)? If so, just augment RPATH for the wraper module (.so) to point to
the directory where your wrapped C shared libraries (.so's) live.

FWIW, here is what I use to build my SWIG wrapper libraries. Note that the
core C libraries they wrap live (or rather, are linked) to the current
directory, so adding the current directory ($PWD) to the rpath is
sufficient for my case:

LINK_SO = ld -shared -rdata_shared -w -o32 -KPIC -rpath $(PWD) -L $(PWD)

You can also use LD_LIBRARY_PATH, but for a number of reasons, it's better
to use RPATH when you can.

Randall
Cross-references between dynamically loaded modules under AIX [ In reply to ]
Jakob Schiotz <jschiotz@hotmail.com> writes:

> a NumPy array (I am using SWIG), the program is appended below. It
> works fine on an alpha processor running OSF1 V4.0, but it coredumps
> under AIX version 4.1.5. The coredump occurs when PyArray_FromDims is
> called. It looks like the dynamical loader under AIX cannot resolve
> symbols in one module that refers to another module. It is probably
> related to the primitive way dynamic loading is working under AIX.

Not really. Most Unix systems let you choose whether symbols in shared
libraries are globally visible or not, AIX just does it in a different
way. Python (at least the most recent versions) defaults to *not*
making these symbols global on systems that allow this. So your problem
is not specific to AIX.

The preferred way to get access to symbols from another shared library
is importing this library (via Python's module import mechanism) and
retrieving the addresses via CObjects. NumPy does this for you
automatically (unless you have a really old version); all you have to
do is to add a call to import_array() in the initialization function
of your module.


I am not a SWIG expert, but I suppose all you have to do is to add one
line here:

> %init %{
> setdefaults();
--> import_array();
> fprintf(stderr, "FAST: %s %s\n", __DATE__, __TIME__);
> %}

--
-------------------------------------------------------------------------------
Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr
Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.55.69
Rue Charles Sadron | Fax: +33-2.38.63.15.17
45071 Orleans Cedex 2 | Deutsch/Esperanto/English/
France | Nederlands/Francais
-------------------------------------------------------------------------------