Tips for calling Python from C

Many articles have summarized how to call a function written in C from Python, but I was mainly addicted to it because there were few articles that described in detail how to call a function from C to Python. ・ Put together a memo.

Command line arguments

An error occurs when someone tries to reference a command line argument. If you get caught in the above error, define it.

pytest.py


import sys
    if not hasattr(sys, 'argv'):
        sys.argv = ['']  #Add command line arguments here if needed

Loading python module

In C, I handwrite the parts that are normally used without being aware of, such as the handling of reference counters.

callPy.c


#include <Python.h>

//Add current directory to path
void set_path() {
    PyObject *sys      = PyImport_ImportModule("sys");
    PyObject *sys_path = PyObject_GetAttrString(sys, "path");
    PyList_Append(sys_path, PyUnicode_DecodeFSDefault("."));
}

void main(){
    PyObject *pModule, *pFunc, *pName;
    Py_Initialize();
  set_path();
    pName = PyUnicode_DecodeFSDefault("pytest");//file name
    pModule = PyImport_Import(pName);
    Py_DECREF(pName);//References that are used up decrement the reference count.
    if(pModule != NULL){
        pFunc = PyObject_GetAttrString(pModule, "pyMethod");//Function name
    }
}

For some reason, if you put the Python file of the module next to the executable file, it cannot be read, and it seems that you have to add the current folder to the path.

Structure definition

callpy.c


typedef struct{
    int val1;
    float val2;
    float val3;
    float val4;
} STRUCT2;
#define ARRAY_MAX 100
typedef struct{
    int structNo;
    STRUCT2 results[ARRAY_MAX];
} STRUCT1;

If you want to define a C structure like the one above on the Python side, In Python, it is defined using ctypes.

pytest.py


class STRUCT2(Structure):
        _fields_ = [
            ('val1', c_int32),
            ('val2', c_float),
            ('val3', c_float),
            ('val4', c_float),
        ]
class STRUCT1(Structure):
        ARRAY_MAX = 100
        _fields_ = [
            ('dataNo', c_int32),
            ('dataArray', STRUCT2 * ARRAY_MAX)
        ]

If it is a pointer array instead of an array entity, it can be defined as a pointer as shown below.

POINTER(STRUCT2) * ARRAY_MAX

I'm dissatisfied with having to define the array length as the initial value even though it's Python. Is there a way to define it dynamically when calling from C?

Calling Python from C

Add PyObject to Tuple in the argument list and call the function with CallObject

callpy.c


PyObject *pValue; //Pointer for return value
Pyobject *pArgArray[3];//Take three arguments
Pyobject *pArgs = PyTuple_New(3);//Argument list
//Convert the value of each argument to PyObject type
pArg[0] = PyUnicode_FromString("hogehoge");//String
pArg[1] = PyLong_FromLong(1000);//Numerical value,int
pArg[2] = PyFloat_FromDouble(0.998);//Floating point
for(int i = 0; i < 3 ; i++){
   PyTuple_SetItem(pArgs, i, pArg[i]);//Set arguments in the argument list
}
pValue = PyObject_CallObject(pFunc, pArgs);

If you want to pass a structure or multidimensional array to Python

For example, if you want to pass a 640 * 480 RGB raw image, use PyArray_SimpleNewFromData to make it a 3D array type object.

callpy.c


import_array();

npy_intp dims[3] = {480, 640, 3};
pArg = PyArray_SimpleNewFromData(3, dimensions, NPY_UINT8, image);

Return value to C side (normal value type or character string)

If it is a PyObject type return value, you can return it like a Python function. Of course, multiple return values are OK

pytest.py


return val1, val2 #int, string

callpy.c


PyObject* pValue;
int num;
float decimal;
---Abbreviation---
pValue = PyObject_CallObject(pFunc, pArgs);
int result = PyArg_ParseTuple(pValue, "is", &num, &decimal);

There are several ways to receive the return value in C, but ParseTuple seems to be good for the time being. Confirm the format of the return type in advance.

Return value to C side (structure or array of structures)

If it was a ctypes type like C, I was addicted to trial and error thinking that I should just pass the pointer from Python to C.

pytest.py


memoryview(mystruct).tobytes()

When I made a byte array like this, it seems that it is necessary to make it a PyObject type once because it is an array type of ctypes, and it can not be passed as a pointer of C as it is. However, even if I passed it as PyObject type, I didn't know how to parse it with c, so I implemented it by ** passing it using numpy.darray type **.

pytest.py


return np.frombuffer(struct, c_int8)

Just convert the structure directly to darray type with numpy.frombuffer and receive it as a numpy array with C.

callpy.c


PyArrayObject *pArray;
PyArg_ParseTuple(pValue, "O", &pArray );
STRUCT1* struct_ptr = (STRUCT1*)PyArray_DATA(pArray);

After all

At first, I didn't know how to pass huge data at all, so I tried various things such as using memory mapping, but in the end, passing as a byte array with numpy.darray seems to be the quickest and easiest to understand. ..

Recommended Posts

Tips for calling Python from C
Note for Pyjulia calling Julia from Python
Wrap C with Cython for use from Python
~ Tips for Python beginners from Pythonista with love ① ~
Wrap C ++ with Cython for use from Python
~ Tips for Python beginners from Pythonista with love ② ~
Tips for Python beginners to use the Scikit-image example for themselves 9 Use from C
[Python + Selenium] Tips for scraping
~ Tips for beginners to Python ③ ~
python tips
Call popcount from Ruby / Python / C #
Python cheat sheet (for C ++ experienced)
[TouchDesigner] Tips for for statements using python
Execute Python code from C # GUI
Run Python scripts synchronously from C #
Call C / C ++ from Python on Mac
Python Tips
Python tips
Call c language from python (python.h)
"Python AI programming" starting from 0 for windows
Tips for dealing with binaries in Python
Tips for using python + caffe with TSUBAME
Try calling Python from Ruby with thrift
Python> Output numbers from 1 to 100, 501 to 600> For csv
Generate C language from S-expressions in Python
Tips for making small tools in python
Use C ++ functions from python with pybind11
An introduction to Python for C programmers
2016-10-30 else for Python3> for:
Python Conda Tips
python [for myself]
Tips for manipulating numpy.ndarray from c ++ -I want to use an iterator-
Python click tips
sql from python
Unexpectedly (?) Python tips
python C ++ notes
python, openFrameworks (c ++)
Run a Python script from a C # GUI application
Create a C array from a Python> Excel sheet
A memorandum of calling Python from Common Lisp
Tips for hitting the ATND API in Python
Boost.NumPy Tutorial for Extending Python in C ++ (Practice)
Call Python library for text normalization from MATLAB
Call a Python script from Embedded Python in C ++ / C ++
I want to make C ++ code from Python code!
Call Polly from the AWS SDK for Python
Notes on oct2py calling Octave scripts from Python
About Python for loops
Use thingsspeak from python
Touch MySQL from Python 3
Python and numpy tips
Operate Filemaker from Python
Python basics ② for statement
Use fluentd from python
Python C / C ++ Extension Pattern-Pointer
Access bitcoind from python
Changes from Python 3.0 to Python 3.5
Changes from Python 2 to Python 3.0
Benchmark for C, Java and Python with prime factorization
Python from or import
Use MySQL from Python