MessagePack-Call Python (or Python to Ruby) methods from Ruby using RPC

MessagePack-A summary of how to call Python (or vice versa) methods from Ruby using RPC.

--I mainly use Ruby, but I want to use scientific computing libraries such as Python's numpy and matplotlib. --I mainly use Python, but I want to operate some Rails apps.

If you use it for such purposes, you may be able to take advantage of the good points of each language.

Premise

• Ruby 2.3.1
  • msgpack-rpc 0.5.4
• Python 3.5.2
  • msgpack-rpc-python 0.4

What is MessagePack-RPC?

Remote Procedure Call protocol. Serialize using MessagePack.

See here for details on the specifications.

• https://github.com/msgpack-rpc/msgpack-rpc

Libraries that implement MessagePack-RPC in various languages are available. The following two are implemented in Ruby and Python. Implementations of various other languages are open to the public.

• https://github.com/msgpack-rpc/msgpack-rpc-ruby
• https://github.com/msgpack-rpc/msgpack-rpc-python

By the way, it seems that neovim plugin can be implemented using MessagePack-RPC. There is an advantage that the plug-in can be implemented in various languages as long as it conforms to the MessagePack-RPC specifications.

The Ruby version has almost no information in the README even if you look at the official github page, so here is a brief summary of how to use it.

Minimal example

First, an example of a Ruby version of the server

server.rb

require 'msgpack/rpc'

class MyHandler
  def add(x,y)
    return x+y
  end
end

svr = MessagePack::RPC::Server.new
svr.listen('localhost', 18800, MyHandler.new)
svr.run

Ruby version client example

client.rb

require 'msgpack/rpc'
c = MessagePack::RPC::Client.new('localhost',18800)
result = c.call(:add, 1, 2)
puts result

Execute as follows.

ruby server.rb &   #start server
ruby client.rb     #The result of the operation is displayed

It can be written in Python in exactly the same way.

server.py

import msgpackrpc

class MyHandler(object):

    def add(self, x, y):
        return x+y

svr = msgpackrpc.Server( MyHandler(), unpack_encoding='utf-8' )
svr.listen( msgpackrpc.Address('localhost',18800) )
svr.start()

client.py

import msgpackrpc

client = msgpackrpc.Client(msgpackrpc.Address("localhost", 18800), unpack_encoding='utf-8')
result = client.call('add', 1, 2)
print( result )

Of course, running a Ruby server and a Python client (or vice versa) gives exactly the same result.

In the case of Python, it should be noted that when the Client or Server is initialized, the default is ʻunpack_encoding ='None', and the transferred data is interpreted as a byte string. In the above sample, there is no problem because it only sends and receives numerical values, but if you want to send and receive character strings, you need to specify ʻunpack_encoding ='utf-8'. Otherwise, the received data will be a string of bytes and you will need to explicitly call .decode ('utf-8') in your program. Unless you want to send binary data, it seems good to specify ʻunpack_encoding ='utf-8'` by default.

Asynchronous execution

The processing on the server side is not limited to the case where it is completed in an instant, and there may be cases where you want to execute a processing that takes time. Asynchronous execution mechanism is also provided for such cases.

The method called by call_async returns the process immediately and returns the future object. The Future object returns the value if the result is returned from the server when # get is called. If no result is returned from the server, wait until the result is obtained from the server.

It's quick to see the sample code below.

async_client.rb

require 'msgpack/rpc'
c = MessagePack::RPC::Client.new('localhost',18800)

puts "async call"
future1 = c.call_async(:delayed_add, 1, 1, 2)
future2 = c.call_async(:delayed_add, 1, 2, 3)
puts future2.get     #The order does not necessarily have to be the order of calling
puts future1.get

async_server.rb

require 'msgpack/rpc'

class MyHandler
  def delayed_add(t,x,y)
    puts "delayed_add is called"
    as = MessagePack::RPC::AsyncResult.new
    Thread.new do
      sleep t
      as.result(x+y)
    end
    as
  end
end

You can get the result with future.get () for Python as well.

async_client.py

import msgpackrpc

client = msgpackrpc.Client(msgpackrpc.Address("localhost", 18800), unpack_encoding='utf-8')
future = client.call_async('delayed_add', 3, 1, 2)
print( future.get() )

async_server.py

import msgpackrpc
import threading, time

class MyHandler(object):
    def delayed_add(self, t, x, y):
        print("delayed_add is called")
        ar = msgpackrpc.server.AsyncResult()
        def sleep_add():
            time.sleep(t)
            ar.set_result(x+y)
        thread = threading.Thread(target=sleep_add)
        thread.start()
        return ar

svr = msgpackrpc.Server( MyHandler(), unpack_encoding='utf-8' )
svr.listen( msgpackrpc.Address('localhost',18800) )
svr.start()

Call matplotlib from Ruby

As a more practical example, I will show an example of drawing a Ruby numeric array with Python's matplotlib.

plot_server.py

import msgpackrpc
import matplotlib.pyplot as plt

svr = msgpackrpc.Server( plt, unpack_encoding='utf-8' )
svr.listen( msgpackrpc.Address('localhost',18800) )
svr.start()

plot_client.rb

require 'msgpack/rpc'
c = MessagePack::RPC::Client.new('localhost',18800)
c.timeout = Float::INFINITY

xs = -3.14.step(3.14, 0.1).to_a
ys = xs.map {|x| Math.sin(x) }
c.call( :scatter, xs, ys )
c.call( :show )

The following window will appear and the plot will be drawn.

Note that the server does not return any processing until the drawn window is closed. If it is left as it is, it will time out and an exception will occur on the client side, so c.timeout = Float :: INFINITY is set.

Here, the Ruby and Python processes are started manually from the shell, but starting Python as an external process in the Ruby program seems to be a neat interface as if plotting directly from Ruby. ..

Call Rails methods from Python

As an example in the opposite direction, get the information in your Rails app from Python

rails_server.rb

require 'msgpack/rpc'
require File.join( ENV['RAILS_ROOT'], 'config/environment' )

Object.class_eval do
  def to_msgpack(*args)  # to_If msgpack is not defined, to_Make it a specification to convert to msgpack after calling s
    to_s.to_msgpack(*args)
  end
end

class MyRailsHandler
  def get_book(name)
    Book.where(name: name).first.as_json
    # as_Hash with json.
  end
end

svr = MessagePack::RPC::Server.new
svr.listen('localhost', 18800, MyRailsHandler.new)
svr.run

rails_client.py

import msgpackrpc

client = msgpackrpc.Client(msgpackrpc.Address("localhost", 18800), unpack_encoding='utf-8')
result = client.call('get_book', 'echo')   #result is dictionary
print( result )

The point is to define to_msgpack (* args) for Object. For a class for which to_msgpack is not defined, ʻObject # to_msgpack (* args)` defined here is called. This makes it possible to serialize Time objects and the like.

Processing that is difficult to realize with RPC

RPC can handle simple processing that only calls one method as we have seen so far. However, in the following cases, RPC does not seem to solve it easily.

--If you want to pass a block to a function -For examplearray.map {|x| f(x) }Can't pass f in RPC when you want to run --In principle, it is impossible because it cannot be serialized with msgpack. --When chaining methods --For example, processing such as Books.where (author:" foo "). Asc (: price) .first that is often seen in Ruby. If you want to call Rails API from Python, there are likely to be many restrictions. --You can send the request several times, but it's unpleasant to have to keep the intermediate state on the server side.