Use shared memory with shared libraries

I wanted to share the data on the shared library between the processes that are attaching it, so I made a lot of trial and error, but I was a little addicted to it, so I will write it as an article.

Global variables on shared libraries

Unlike Windows DLLs, UNIX-like shared libraries are placed in the memory space of each process even if variables are placed in the global space, so interprocess communication cannot be performed simply by using the shared library. Shared memory is the easiest way to share the data used by a shared library. So here is the sample code.

lib_shared_memory.c


#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <sys/shm.h>

static void *sharedmemory; //Pointer to shared memory
static int seg_id;         //Shared memory ID
static int* pnumattachproc;//Number of processes attached to shared memory(Placed on shared memory)

__attribute__ ((constructor))
void attach_shmem(void)
{
    int isfirstproc = 0;
    key_t key = 10;
    size_t size = 1000;
    //Create shared memory
    seg_id = shmget(key, size, IPC_CREAT | IPC_EXCL | 0660);
    //Failed to create,And if shared memory with the same key exists
    if ((seg_id == -1) && (errno == EEXIST)){
        //Get the id of that shared memory
        seg_id = shmget(key, size, 0);
    }else if(seg_id > 0){
        //If the shared memory is created successfully, set a flag to perform initialization processing.
        isfirstproc = 1;
    }
    if (seg_id == -1)
        exit(1);
    //Attach shared memory
    sharedmemory = shmat(seg_id, NULL, 0);
    if(isfirstproc){
        pnumattachproc = (int*)sharedmemory;
        //Initialize the number of attached processes
        *pnumattachproc = 0;
    }
    (*pnumattachproc)++; //Increment the number of attached processes
}

__attribute__ ((destructor))
void detach_shmem(void)
{
    (*pnumattachproc)--; //Decrement the number of attached processes
    if(*pnumattachproc == 0){
        //If the number of attached processes reaches 0,
        //Give the shared memory the release attribute
        shmctl(segid, IPC_RMID, NULL);    
    }
    //Shared memory detach
    (void) shmdt(sharedmemory);
}

int get_attachproc_num(void){
    return *pnumattachproc;
}

The build uses the options for shared libraries.

gcc -fPIC -shared -o libshm.so lib_shared_memory.c

As a caveat

shmctl(segid, IPC_RMID, NULL); 

Let's execute it after the attached process becomes 0 and the release is confirmed. If the attached process is executed when it exists, when a new process tries to attach to the shared memory, it will create a new shared memory even though it has the same key. To prevent this, the number of processes attached to the shared memory is managed by placing it as a variable on the shared memory so that the process to detach the shared library at the end can be released.

I didn't understand this, I was adding a release request when creating shared memory, and I wasn't able to attach to shared memory.

that's all.

Recommended Posts

Use shared memory with shared libraries
Debug shared libraries with VScode
[IOS] Use a shared sheet with Pythonista3.
Use mecab-ipadic-neologd with igo-python
Use RTX 3090 with PyTorch
Use ansible with cygwin
Use pipdeptree with virtualenv
Save memory with `` __slots__``
[Python] Use JSON with Python
Use Mock with pytest
Use indicator with pd.merge
Use Gentelella with django
Use mecab with Python3
Use tensorboard with Chainer
Interprocess communication ~ shared memory ~
Use DynamoDB with Python
Use pip with MSYS2
Use Python 3.8 with Anaconda
Use pyright with Spacemacs
Use python with docker
Use TypeScript with django-compressor
Use LESS with Django
Shared memory between processes
Use MySQL with Django
Use Enums with SQLAlchemy
Use tensorboard with NNabla
Use GPS with Edison
Use nim with Jupyter
Use Trello API with python
Use "$ in" operator with mongo-go-driver
Use custom tags with PyYAML
Use directional graphs with networkx
Use TensorFlow with Intellij IDEA
Use Twitter API with Python
Use pip with Jupyter Notebook
error while loading shared libraries
Use DATE_FORMAT with SQLAlchemy filter
Use TUN / TAP with Python
Use sqlite3 with NAO (Pepper)
Use sqlite load_extensions with Pyramid
Use Windows 10 fonts with WSL
Install external libraries with Python
Use chainer with Jetson TK1
Use SSL with Celery + Redis
Use Cython with Jupyter Notebook
Use Maxout + CNN with Pylearn2
Use WDC-433SU2M2 with Manjaro Linux
Use OpenBLAS with numpy, scipy
Investigate memory leaks with objgraph
Use subsonic API with python3
Use Sonicwall NetExtener with Systemd
Use prefetch_related conveniently with Django
Use AWS interpreter with Pycharm
Use Bokeh with IPython Notebook
Use Python-like range with Rust
Use pyright with CentOS7, emacs lsp-mode
Python: How to use async with
Use Azure SQL Database with SQLAlchemy
Use PointGrey camera with Python (PyCapture2)
Use vl53l0x with Raspberry Pi (python)
Use PX-S1UD / PX-Q1UD with Jetson nano