GRPC starting with Python

Overview

gRPC is a modern, high-performance open source RPC (Remoto Protocol Call) framework that can be run in any environment. It has the advantage of being able to serialize data using Protocol Buffers for high-speed communication. It is compatible with various languages and platforms, and bidirectional streaming utilizing http / 2 is possible. Services (data and functions to communicate) can be simply defined using Protocol Buffers, and API specifications can be clearly stated.

git: k-washi/stereophonic-Sound-System/proto

Golang versin's gRPC article can be found in Starting with Golang gRPC.

Contents

This section describes how gRPC responds to a request from a client with the position (x, y, z) of an appropriate object calculated by the server. Originally, it is a system that generates stereophonic sound according to the position of the character, and was created to transmit position information.

Library installation

pip install grpcio-tools==1.26.0

Protocol definition

Define positions x, y, z as Pos in the .proto file as shown below. Then, Position becomes a protocol (communication protocol) including location information. On the other hand, the Msg protocol is defined as a result of issuing a message requesting location information to the server side or location information. In addition, PositionReq and PositionPub are defined as functions used for communication. PositionReq is a function that sends Msg to the server and receives Position information from the server, and PositionPub is a function that sends Position information to the server and receives Msg. In this article, we will explain using an example using PositionReq.

proto/position.proto


syntax = "proto3";

package posXYZ;
option go_package="posXYZpb";

message Pos {
  float x = 1;
  float y = 2;
  float z = 3;
}

//client streaming
message Position{
  Pos position = 1;
  int32 status = 2;
  string msg = 3;
}

message Msg{
  int32 status = 1;
  string msg = 2;
}


service PositionService{
  
  rpc PositionReq(Msg) returns (Position) {};
  rpc PositionPub(Position) returns (Msg) {};
}

Conversion to protocol buffer for python

As described in Golang versin's gRPC article gRPC starting with Golang, even if it is converted to protocol buffer for golang, it looks like this article. Even if you convert to protocol buffer for python, the conversion will be done based on the above protocol definition. By this conversion to protoco buffer, the structure of each message defined in each language and the definition document (program) including the service function are output.

Python can be converted by executing the following program.

proto/codegen.py


from grpc.tools import protoc

protoc.main(
    (
        '',
        '-I.',
        '--python_out=.',
        '--grpc_python_out=.',
        './proto/position.proto',
    )
)
python ./proto/codegen.py

As a result of the above command, position_pb2.py that defines the message defined in the protocol file and position_pb2_grpc.py that defines the function used for gRPC communication are generated.

Server-side implementation

Here, implement a server that receives request Msg and issues Position (location information). The position information issued here is a position that rotates while maintaining a distance of 1 m from (x, y, z) = (0, 0, 0).

Regarding configInit for settings not related to gRPC and logger for logging, my past article Parameter setting by python configparser, [Python logging Please refer to How to use the module.

The PositionServer class overloads the PositionServerServer defined in position_pb2_grpc.py and defines the PositionReq function that returns the position information in response to the request. This return value, Position, uses the one defined in position_pb2.py. In addition, this class also defines functions that store and output location information x, y, and z, and manages location information.

The Server class is a class that summarizes the processing performed by the server side of gRPC. Therefore, it has an instance of PosotionServer class as a variable. The start function defines the process to start the gRPC server, and the stop function defines the process to stop the gRPC server. Since these start and stop processes are fixed phrases, the basic flow does not change in any program.

In the main process, the server is opened by executing the start function, and the server-side process of gRPC is over. Here, in order to change the location information every time, the location information issued by posServer.pubPos (x, y, z) is overwritten.

proto/server.py


import os
import sys

sys.path.append(os.path.join(os.path.dirname(__file__), '..'))

# ------------

from utils.config import configInit
Conf = configInit()
logger = Conf.setLogger(__name__)

# ------------
#grpc

import grpc
from proto.position_pb2 import *
from proto import position_pb2_grpc

# ------------

from concurrent import futures

class PositionService(position_pb2_grpc.PositinServiceServicer):
  def __init__(self):
    self.x = 1.
    self.y = 0.
    self.z = 0.

  def PositionReq(self, request, context):
    try:
      is_success = 0
    except:
      is_success = -1
    return Position(
      position = Pos(
        x = self.x, y = self.y, z = self.z
      ),
      status = is_success,
      msg = "character position"
    )

  def pubPos(self, x, y, z):
    self.x, self.y, self.z = x, y, z
  
  def getPos(self, x, y, z):
    return self.x, self.y, self.z

class Server():
  def __init__(self):
    self.posServer = PositionService()

  def start(self):
    
    self.server = grpc.server(futures.ThreadPoolExecutor(max_workers=3))
    position_pb2_grpc.add_PositinServiceServicer_to_server(
      self.posServer, self.server
    )

    self.server.add_insecure_port(Conf.PosServer)
    self.server.start()
    logger.info("Start server {0}".format(Conf.PosServer))

  def stop(self):
    self.server.stop(0)

if __name__ == "__main__":
  import time
  import numpy as np
  server = Server()
  server.start()
  
  z = 0.
  azimuth = 0.
  aziShift = 5* np.pi / 180.

  def azi2pos(azimuth):
    x = np.cos(azimuth)
    y = np.sin(azimuth)
    return x, y

  try:
    while True:
      time.sleep(0.1)
      azimuth += aziShift
      x,y = azi2pos(azimuth)
      server.posServer.pubPos(x,y,z)

  except Exception as e:
    logger.critical(e)
    server.stop()

Client-side implementation

The client side implementation is implemented in the posClient class. Start client processing with the open function. Note that position_pb2_grpc.PositinServiceStub contains the functions that the client executes. Therefore, in the posRequest function, Msg can be sent by the self.stub.PositionReq function and the information from the server can be obtained as the return value. After that, every time you execute posRequest () in the main process, you can communicate with the server side and get the location information.

proto/client.py


mport os
import sys

sys.path.append(os.path.join(os.path.dirname(__file__), '..'))

# ------------

from utils.config import configInit
Conf = configInit()
logger = Conf.setLogger(__name__)

# ------------
#grpc

import grpc
from proto.position_pb2 import *
from proto import position_pb2_grpc

class posClient():
  def __init__(self):
    self.x = 0.
    self.y = 0.
    self.z = 0.

  def posRequest(self):
    request = Msg(
      status = 0,
      msg = "request pos"
    )
    res = self.stub.PositionReq(request)
    if res.status == 0:
      logger.info("PositionRes {0}, {1}, x:{2}, y:{3}, z:{4}".format(res.status, res.msg, res.position.x, res.position.y, res.position.z))
      self.x, self.y, self.z = res.position.x, res.position.y, res.position.z
      return True
    
    logger.error("Position Response Error")
    return False

  
  def open(self):
    self.channel = grpc.insecure_channel(Conf.PosClient)
    self.stub = position_pb2_grpc.PositinServiceStub(self.channel)
    logger.info("Open position client channel: {0}".format(Conf.PosClient))

  def close(self):
    self.channel.close()
  
  def getPos(self):
    return self.x, self.y, self.z
  
if __name__ == "__main__":
  import time
  posCl = posClient()
  posCl.open()
  while True:
    time.sleep(1)
    try:
      ok = posCl.posRequest()
      if ok:
        x, y, z = posCl.getPos()
        logger.info("{0}, {1}, {2}".format(x, y, z))
    except Exception as e:
      logger.error("client error {0}".format(e))
  posCl.close()

Summary

With the above, location information can be exchanged with gRPC using python. I think gRPC will be a technology that will be used in many situations in the future because it has advantages such as making it easier to build microservices written in multiple languages. Please give it a try.

Recommended Posts

GRPC starting with Python
Python starting with Windows 7
Reinforcement learning starting with Python
I tried gRPC with Python
Python starting with Hello world!
Data analysis starting with python (data visualization 1)
Data analysis starting with python (data visualization 2)
FizzBuzz with Python3
Scraping with Python
Scraping with Python
Python with Go
Twilio with Python
Integrate with Python
Play with 2016-Python
AES256 with python
Tested with Python
python starts with ()
with syntax (Python)
Bingo with python
Zundokokiyoshi with python
Excel with Python
Microcomputer with Python
Cast with python
Communicate between Elixir and Python with gRPC
System trading starting with Python3: long-term investment
"System trade starting with Python3" reading memo
Business efficiency starting from scratch with Python
Data analysis starting with python (data preprocessing-machine learning)
"First Elasticsearch" starting with a python client
Serial communication with Python
Zip, unzip with python
Django 1.11 started with Python3.6
Python with eclipse + PyDev.
Socket communication with Python
Data analysis with python 2
Scraping with Python (preparation)
Try scraping with Python.
Learning Python with ChemTHEATER 03
Sequential search with Python
Machine learning starting with Python Personal memorandum Part2
"Object-oriented" learning with python
Handling yaml with python
Solve AtCoder 167 with python
Serial communication with python
[Python] Use JSON with Python
Learning Python with ChemTHEATER 05-1
Learn Python with ChemTHEATER
Run prepDE.py with python3
1.1 Getting Started with Python
Collecting tweets with Python
Binarization with OpenCV / Python
3. 3. AI programming with Python
Kernel Method with Python
Machine learning starting with Python Personal memorandum Part1
Scraping with Python + PhantomJS
Drive WebDriver with python
[Python] Redirect with CGIHTTPServer
Voice analysis with python
Think yaml with python
Operate Kinesis with Python
Use DynamoDB with Python