Python C / C ++ Extension Pattern-Pass data to Python as np.array

View C / C ++ data as np.array from Python.

struct Particle {
  float x[3], v[3];
};

vector<Particle> v;

And suppose data like Particle * is on the C / C ++ side.

numpy.array from C/C++ First, include numpy / arrayobject.h and initialize it. This seems to initialize static variables, and if you have more than one source code, you have to call ʻimport_array ()` in all the source code that uses numpy. Otherwise, you will get a segmentation fault.

#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
#include "numpy/arrayobject.h"

PyMODINIT_FUNC
PyInit_cext07(void) {
  import_array();
  return PyModule_Create(&module);
}

Set ʻinclude_dirs in setup.py` to read the NumPy headers.

import numpy as np
from distutils.core import setup, Extension

setup(name='cext07',
      version='0.0.1',
      description='c_ext07 read data as np.array',
      author='Jun Koda',
      url='https://github.com/junkoda/python_c_ext',
      ext_modules=[
          Extension('cext07', ['cext07.c'],
                    include_dirs = [np.get_include()]),
      ],
)

Where the data is in Python

A function that passes a pointer to vector <Particle> so that it can be referenced from the Python side:

static void py_particles_free(PyObject *obj);

static PyObject* py_particles_alloc(PyObject *self, PyObject *args)
{
  // Allocate a new C++ pointer as a Python object "_Particles"
  vector<Particle>* const p= new vector<Particle>();

  p->resize(10); //Assuming that the data has 10 particles
  
  return PyCapsule_New(p, "_Particles", py_particles_free);
}

void py_particles_free(PyObject *obj)
{
  // Destructor function for _Particles pointer, called automatically from Python
  vector<Particle>* const p=
    (vector<Particle>*) PyCapsule_GetPointer(obj, "_Particles");
    
  delete p;
}

Pass the data as np.array to Python

Functions that pass vector <Particle> as np.array:

PyObject* py_as_array(PyObject *self, PyObject *args)
{
  PyObject* py_particles;
  if(!PyArg_ParseTuple(args, "O", &py_particles))
    return NULL;

  // Get vector<Particle>* pointer back from _Particles
  vector<Particle>* const p=
    (vector<Particle>*) PyCapsule_GetPointer(py_particles, "_Particles");
  if(!p) return NULL;

  int nd=2; //Two-dimensional array with rows and columns
  int ncol= sizeof(Particle)/sizeof(float); // ncol=6 x[3], v[3]6 rows
  npy_intp dims[]= {(npy_intp) p->size(), ncol};

  return PyArray_SimpleNewFromData(nd, dims, NPY_FLOAT, &(p->front()));
}

Usual C / C ++ extension functions

static PyMethodDef methods[] = {
  {"_particles_alloc", py_particles_alloc, METH_VARARGS, "new Particles"},
  {"_as_array", py_as_array, METH_VARARGS, "vector<Particle> as an array"},
  {NULL, NULL, 0, NULL}
};

static struct PyModuleDef module = {
  PyModuleDef_HEAD_INIT,
  "cext07",                // name of this module
  "Use vector<Particle>",  // Doc String
  -1,
  methods
};

Wrapping with a Python class

import numpy as np
import cext07

class Particles:
    def __init__(self):
        self._p= cext07._particles_alloc() # _Particle pointer

    def __repr__(self):
        return cext07._as_array(self._p).__repr__()
    
    def __getitem__(self, index):
        return cext07._as_array(self._p)[index]
    
    def asarray(self):
        return cext07._as_array(self._p)        
    #etc

import cext07
p = Particles()
p        # __repr__Sees array by
p[0]     # __getitem__;First particle
p[2:5,2] #slice is also np.array handles it
p[:,0]   # x[0]
p[:,4]   # v[1]
p[:]     #All data

Postscript

In this example, I didn't need a Python class, just the as_array function. I will simplify it later.

All chords

The full code can be found on GitHub.

Recommended Posts

Python C / C ++ Extension Pattern-Pass data to Python as np.array
Python C / C ++ Extensions: Pass some of the data as np.array to Python (set stride)
Python C / C ++ Extension Pattern-Pointer
I tried Python C extension
Pass OpenCV data from the original C ++ library to Python
[Python] How to FFT mp3 data
Numba to speed up as Python
Introduction to Protobuf-c (C language ⇔ Python)
How to wrap C in Python
[Python] How to store a csv file as one-dimensional array data
[Introduction to Python3, Day 17] Chapter 8 Data Destinations (8.1-8.2.5)
[Introduction to Python3, Day 17] Chapter 8 Data Destinations (8.3-8.3.6.1)
Convert Excel data to JSON with python
[Introduction to Python3 Day 19] Chapter 8 Data Destinations (8.4-8.5)
Convert FX 1-minute data to 5-minute data with Python
[Introduction to Python3 Day 18] Chapter 8 Data Destinations (8.3.6.2 to 8.3.6.3)
How to use "deque" for Python data
Compress python data and write to sqlite
[Introduction to Data Scientists] Basics of Python ♬
Exchange encrypted data between Python and C #
An introduction to Python for C programmers
Convert csv, tsv data to matrix with python --using MovieLens as an example
How to use the C library in Python
Output formatted output in Python, such as C / C ++ printf.
How to generate permutations in Python and C ++
Just add the python array to the json data
Writing logs to CSV file (Python, C language)
[Python] How to read data from CIFAR-10 and CIFAR-100
I tried to get CloudWatch data with Python
[Introduction to Python] How to handle JSON format data
I'm addicted to Kintone as a data store
[Python] Flow from web scraping to data analysis
Write CSV data to AWS-S3 with AWS-Lambda + Python
How to output "Ketsumaimo" as standard output in Python
I want to make C ++ code from Python code!
Run Python on Apache to view InfluxDB data
I want to get custom data attributes of html as elements using Python Selenium
Data analysis python
python C ++ notes
python, openFrameworks (c ++)
"Backport" to python 2
[python] Read data
Summary of tools needed to analyze data in Python
Try to make a Python module in C language
[For beginners] How to study Python3 data analysis exam
How to scrape image data from flickr with python
Reading Note: An Introduction to Data Analysis with Python
[Introduction to python] A high-speed introduction to Python for busy C ++ programmers
Send data from Python to Processing via socket communication
Write data to KINTONE using the Python requests module
Specify MinGW as the compiler to use with Python
I felt that I ported the Python code to C ++ 98.
[Introduction to Udemy Python3 + Application] 68. Import statement and AS
I tried to analyze J League data with Python
[Python] How to call a c function from python (ctypes)
[Technical book] Introduction to data analysis using Python -1 Chapter Introduction-