Hey folks,
I was getting consistent but inexplicable segmentation faults with the
following code, using NumPy's C API to move data between Python and C
(for sparse matrix multiplication; the data structure is from Meschach,
in case that's relevant). I found the problem to be caused by gcc's
optimization options.
static PyObject * mul(PyObject *self, PyObject *args) {
...Snip declarations...
...Snip working code. Function ends with:
return MakeFromMes(product);
}
which is defined as:
PyObject * MakeFromMes(SPMAT *in) {
int i, j, m, n, nzs, count, pos, therow, thecol;
int dummy[0];
double elem;
PyArrayObject *pr, *ir, *jc;
m = in->m; n = in->n; nzs = 0;
/* Count the number of nonzero elements of in
ISZERO is a macro to test for "good enough" floating point 0 */
for(i = 0; i < m; i++) {
for(j = 0; j < n; j++) {
if(!ISZERO(sp_get_val(in, i, j))) nzs++;
}
}
dummy[0] = nzs;
if(!( (pr = PyArray_FromDims(1, dummy, PyArray_DOUBLE)) &&
(ir = PyArray_FromDims(1, dummy, PyArray_INT)) &&
(jc = PyArray_FromDims(1, dummy, PyArray_INT)) )) {
fprintf(stderr, "MakeFromMes: Could not create output arrays.\n");
return Py_BuildValue("O", Py_None);
}
count = pos = 0;
/* Fill column-wise */
for(j = 0; j < n; j++) {
for(i = 0; i < n; i++) {
elem = sp_get_val(in, i, j);
if(!ISZERO(elem)) {
therow = pos % n;
thecol = (int)(pos / n);
*(double *)(pr->data + count * pr->strides[0]) = elem;
*(int *)(ir->data + count * ir->strides[0]) = therow;
*(int *)(jc->data + count * jc->strides[0]) = thecol;
count++;
}
pos++;
}
}
return Py_BuildValue("OOO", pr, ir, jc);
}
The segmentation fault occured immediately after MakeMes() returned.
Were I to change the mul() function to end with
PyObject *out = MakeFromMes(prod);
printf("Hello.");
return out;
}
the error would occur even before execution passed to the printf.
The problem occurs only with an -O option in compilation. Other
specific -f optimization options do not cause the seg fault. So my
question is -- because I've let gcc optimize other extensions to Python
I've written without incident -- what was it here that caused this
behavior? I'd appreciate any ideas and speculations, because I'm still
baffled by this, though the problem seems to be solved.
Thanks,
John
Sent via Deja.com http://www.deja.com/
Share what you know. Learn what you don't.
I was getting consistent but inexplicable segmentation faults with the
following code, using NumPy's C API to move data between Python and C
(for sparse matrix multiplication; the data structure is from Meschach,
in case that's relevant). I found the problem to be caused by gcc's
optimization options.
static PyObject * mul(PyObject *self, PyObject *args) {
...Snip declarations...
...Snip working code. Function ends with:
return MakeFromMes(product);
}
which is defined as:
PyObject * MakeFromMes(SPMAT *in) {
int i, j, m, n, nzs, count, pos, therow, thecol;
int dummy[0];
double elem;
PyArrayObject *pr, *ir, *jc;
m = in->m; n = in->n; nzs = 0;
/* Count the number of nonzero elements of in
ISZERO is a macro to test for "good enough" floating point 0 */
for(i = 0; i < m; i++) {
for(j = 0; j < n; j++) {
if(!ISZERO(sp_get_val(in, i, j))) nzs++;
}
}
dummy[0] = nzs;
if(!( (pr = PyArray_FromDims(1, dummy, PyArray_DOUBLE)) &&
(ir = PyArray_FromDims(1, dummy, PyArray_INT)) &&
(jc = PyArray_FromDims(1, dummy, PyArray_INT)) )) {
fprintf(stderr, "MakeFromMes: Could not create output arrays.\n");
return Py_BuildValue("O", Py_None);
}
count = pos = 0;
/* Fill column-wise */
for(j = 0; j < n; j++) {
for(i = 0; i < n; i++) {
elem = sp_get_val(in, i, j);
if(!ISZERO(elem)) {
therow = pos % n;
thecol = (int)(pos / n);
*(double *)(pr->data + count * pr->strides[0]) = elem;
*(int *)(ir->data + count * ir->strides[0]) = therow;
*(int *)(jc->data + count * jc->strides[0]) = thecol;
count++;
}
pos++;
}
}
return Py_BuildValue("OOO", pr, ir, jc);
}
The segmentation fault occured immediately after MakeMes() returned.
Were I to change the mul() function to end with
PyObject *out = MakeFromMes(prod);
printf("Hello.");
return out;
}
the error would occur even before execution passed to the printf.
The problem occurs only with an -O option in compilation. Other
specific -f optimization options do not cause the seg fault. So my
question is -- because I've let gcc optimize other extensions to Python
I've written without incident -- what was it here that caused this
behavior? I'd appreciate any ideas and speculations, because I'm still
baffled by this, though the problem seems to be solved.
Thanks,
John
Sent via Deja.com http://www.deja.com/
Share what you know. Learn what you don't.