Interprocess communication between Ruby and Python (POSIX message queue)

Overview

When I was thinking about passing data between ruby and python daemon using redis (Pub / Sub) [^ Streams], I heard a heavenly voice saying that it would be easier to use interprocess communication (POSIX IPC), so I tried it. I tried it.

In addition, since it runs on the raspberry pi this time, we are considering a strategy to reduce the number of processes as much as possible due to power supply problems.

[^ Streams]: From redis5, there is an additional feature called Streams that is more reliable than the Pub / Sub feature.

What is a POSIX message queue?

One of the interprocess communication available through system calls / library functions in POSIX. Asynchronous communication protocol, like a super-simplified version of messaging middleware such as RabbitMQ, without fear of misunderstanding. Since it is POSIX compliant, it can be easily used with a simple API.

Please refer to the following for details.

https://linuxjm.osdn.jp/html/LDP_man-pages/man7/mq_overview.7.html

Whole system

• OS: Raspbian GNU/Linux 10 (buster)
• ruby: ruby 2.5.7p206 (2019-10-01 revision 67816) [armv7l-linux-eabihf] --Library used: posix-mqueue
  • https://github.com/sirupsen/posix-mqueue
• python: Python 2.7.16 --Library used: posix_ipc
  • https://pypi.org/project/posix_ipc/

Ruby side implementation (sender side)

It doesn't matter which one you send / receive, but this time I tried Ruby as the sender. I used posix-mqueue as the library to use. Very simple.

require 'posix/mqueue'
require 'json'

def main
  m = POSIX::Mqueue.new("/whatever", msgsize: 8192, maxmsg: 10)
  10.times do |i|
    m.send({ messsage: "hello python world #{i}" }.to_json)
    sleep 1
  end
  m.send ({ message: 'end'}.to_json)
ensure
  m.unlink if m
end

if __FILE__ == $PROGRAM_NAME
  main
end

Python side implementation (receiver side)

This is also horribly simple. The library used is posix_ipc

# -*- coding: utf-8 -*-
import time
import posix_ipc
import json

def main () :
    mq = posix_ipc.MessageQueue("/whatever", posix_ipc.O_CREAT)
    while True:
        data = mq.receive()
        j = json.loads(data[0]) # ->  (message, priority)Come back with tuples
        print j
        if j.get('message') == "end":
            break

if __name__ == "__main__" : 
    main()

result

It was output on the Python side as expected!

$ python main.py 
{u'messsage': u'hello python world 0'}
{u'messsage': u'hello python world 1'}
{u'messsage': u'hello python world 2'}
{u'messsage': u'hello python world 3'}
{u'messsage': u'hello python world 4'}
{u'messsage': u'hello python world 5'}
{u'messsage': u'hello python world 6'}
{u'messsage': u'hello python world 7'}
{u'messsage': u'hello python world 8'}
{u'messsage': u'hello python world 9'}
{u'message': u'end'}

It's a simple source, but I've posted it on github for the time being. https://github.com/y-amadatsu/posix-mq-with-raspberry-pi-os

Digression

By the way, if you use POSIX messages, the message Queue will be displayed under / dev / mqueue. You can also mount it and operate it with ls or rm. This area is very UNIX! It feels like: relaxed:

cat /dev/mqueue/whatever 
QSIZE:350        NOTIFY:0     SIGNO:0     NOTIFY_PID:0