I am transferring my code from Python/C interfaced using ctypes to Python/C++ interfaced using Cython. The new interface will give me an easier to maintain code, because I can exploit all the C++ features and need relatively few lines of interface-code.
The interfaced code works perfectly with small arrays. However it encounters a segmentation fault when using large arrays. I have been wrapping my head around this problem, but have not gotten any closer to a solution. I have included a minimal example in which the segmentation fault occurs. Please note that it consistently occurs on Linux and Mac, and also valgrind did not give insights. Also note that the exact same example in pure C++ does work without problems.
The example contains (part of) a Sparse matrix class in C++. An interface is created in Cython. As a result the class can be used from Python.
C++ side
sparse.h
#ifndef SPARSE_H
#define SPARSE_H
#include <iostream>
#include <cstdio>
using namespace std;
class Sparse {
public:
int* data;
int nnz;
Sparse();
~Sparse();
Sparse(int* data, int nnz);
void view(void);
};
#endif
sparse.cpp
#include "sparse.h"
Sparse::Sparse()
{
data = NULL;
nnz = 0 ;
}
Sparse::~Sparse() {}
Sparse::Sparse(int* Data, int NNZ)
{
nnz = NNZ ;
data = Data;
}
void Sparse::view(void)
{
int i;
for ( i=0 ; i<nnz ; i++ )
printf("(%3d) %d
",i,data[i]);
}
Cython interface
csparse.pyx
import numpy as np
cimport numpy as np
# UNCOMMENT TO FIX
#from cpython cimport Py_INCREF
cdef extern from "sparse.h":
cdef cppclass Sparse:
Sparse(int*, int) except +
int* data
int nnz
void view()
cdef class PySparse:
cdef Sparse *ptr
def __cinit__(self,**kwargs):
cdef np.ndarray[np.int32_t, ndim=1, mode="c"] data
data = kwargs['data'].astype(np.int32)
# UNCOMMENT TO FIX
#Py_INCREF(data)
self.ptr = new Sparse(
<int*> data.data if data is not None else NULL,
data.shape[0],
)
def __dealloc__(self):
del self.ptr
def view(self):
self.ptr.view()
setup.py
from distutils.core import setup, Extension
from Cython.Build import cythonize
setup(ext_modules = cythonize(Extension(
"csparse",
sources=["csparse.pyx", "sparse.cpp"],
language="c++",
)))
Python side
import numpy as np
import csparse
data = np.arange(100000,dtype='int32')
matrix = csparse.PySparse(
data = data
)
matrix.view() # --> segmentation fault
To run:
$ python setup.py build_ext --inplace
$ python example.py
Note that data = np.arange(100,dtype='int32')
does work.
See Question&Answers more detail:
os