The pain of gRPC using Python. November 2019. (Personal memo)

It may be normal in the gRPC world, but I'll note that a beginner was unexpectedly addicted to writing a gRPC client in Python.

Work based on Python Quick Start. First, download the sample code. Now use the slightly older v1.19.0 instead of the latest v1.25.0. The reason will be described later.

git clone -b v1.19.0 https://github.com/grpc/grpc

Let's change the taste from the document and create a Python project in a different directory. Again, we'll use v1.19.0.

mkdir python-grpc
cd python-grpc
pipenv --python 3.7
pipenv install grpcio~=1.19.0
pipenv install --dev grpcio-tools
pipenv shell
mkdir pb

Spicy 1: The grpc_tools.protoc option is difficult.

Create a py from the proto file using the grpc_tools.protoc command and write it to the pb directory.

python -m grpc_tools.protoc \
    -I../grpc/examples/protos/ \
    --python_out=pb \
    --grpc_python_out=pb \
    ../grpc/examples/protos/helloworld.proto

First, the -I option is required. Without it, you will get the error File does not reside within any path specified using --proto_path (or -I) .. It is unreasonable why I can not find it because I have specified the position of proto properly.

Also, the -I option has a trick: if you specify a higher directory, for example -I ../grpc/examples/, the file will be generated in pb / protos instead of pb. If you don't know, you'll be at a loss as to where the file went.

Also, the description of the --help option does not mention the difference between --python_out and --grpc_python_out. Although there are differences as described above, it is unreasonable to have another option because it is not enough to just fail to import if you put it in another directory.

Spicy 2: The generated file does not have a relative path.

When I try to read the created one from python, I get an error.

$ python
>>> import pb.helloworld_pb2_grpc
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/tyamamiya/tmp/python-grpc/pb/helloworld_pb2_grpc.py", line 4, in <module>
    import helloworld_pb2 as helloworld__pb2
ModuleNotFoundError: No module named 'helloworld_pb2'

The relative path of the generated code is wrong (it seems to be Python2 style). It can't be helped, so add pb / __ init__.py according to python: use relative imports in generated modules.

import sys
from pathlib import Path

sys.path.append(str(Path(__file__).parent))

Tip: How to debug

If it doesn't work because there are no errors, it is better to set environment variables and execute. Now start the server and client and check the operation.

$ python ../grpc/examples/python/helloworld/greeter_server.py & 
$ export GRPC_TRACE=all
$ export GRPC_VERBOSITY=DEBUG
$ python ../grpc/examples/python/helloworld/greeter_client.py
...A lot of things come out.
Greeter client received: Hello, you!
$ unset GRPC_TRACE
$ unset GRPC_VERBOSITY

Good.

Spicy 3: google.api

Occasionally there is a proto file that includes google.api. The proto file itself is located at https://github.com/googleapis/googleapis/tree/master/google/api, so you can download it and specify it with -I, but I included it in the converted code. The contents of proto are not included. I felt that it would be imported with the option --include_imports, but that is not the case, and I need to install the converted one separately.

pipenv install googleapis-common-protos

Well, if you know this, it doesn't matter.

Spicy 4: missing selected ALPN property.

When connecting to an existing gRPC server over TLS with the latest Python grpcio library, you may get missing selected ALPN property..

D1125 19:20:57.313482000 4619453888 security_handshaker.cc:186]        Security handshake failed: {"created":"@1574677257.313465000","description":"Cannot check peer: missing selected ALPN property.","file":"src/core/lib/security/security_connector/ssl_utils.cc","file_line":118}
I1125 19:20:57.313703000 4619453888 subchannel.cc:1000]                Connect failed: {"created":"@1574677257.313465000","description":"Cannot check peer: missing selected ALPN property.","file":"src/core/lib/security/security_connector/ssl_utils.cc","file_line":118}

Searching with this keyword reveals a lot of interesting facts, but from https://github.com/grpc/grpc/issues/18710, it seems that you can't access a server that doesn't support the ALPN check feature.

It can't be helped, so use the old version v1.19.0 to make the tea muddy. https://stackoverflow.com/questions/57397723/grpc-client-failing-to-connect-to-server-with-tls-certificates

Conclusion

The combination of Python and gRPC feels unfamiliar.

Recommended Posts

The pain of gRPC using Python. November 2019. (Personal memo)
[Personal memo] julia --Using Python library with julia using PyCall
Try using the collections module (ChainMap) of python3
Find the geometric mean of n! Using Python
Get the update date of the Python memo file.
Cut a part of the string using a Python slice
Explanation of the concept of regression analysis using Python Part 1
Explanation of the concept of regression analysis using Python Extra 1
Study from the beginning of Python Hour8: Using packages
Ssh connection memo using ProxyCommand of ssh_config in Python
Towards the retirement of Python2
[Personal memo] Auto-completion of bash
About the ease of Python
python: Basics of using scikit-learn ①
Python memo using perl --join
About the features of Python
The Power of Pandas: Python
[Memo] The mystery of cumulative assignment statements in Python functions
View using the python module of Nifty Cloud mobile backend
A memo of writing a basic function in Python using recursion
[Python] I tried collecting data using the API of wikipedia
The story of Python and the story of NaN
Image capture of firefox using python
[Python] The stumbling block of import
First Python 3 ~ The beginning of repetition ~
[Python] I wrote the route of the typhoon on the map using folium
[Personal memo] Python virtual environment command memo
Confirmation of impulse response (personal memo)
Removal of haze using Python detailEnhanceFilter
[Python] LINE notification of the latest information using Twitter automatic search
pyenv-change the python version of virtualenv
Change the Python version of Homebrew
Python3 compatible memo of "python start book"
[Python] Understanding the potential_field_planning of Python Robotics
Review of the basics of Python (FizzBuzz)
[Personal memo] Python sequence type / mapping type
Implementation of desktop notifications using Python
Get and set the value of the dropdown menu using Python and Selenium
Extract the targz file using python
Separate display of Python graphs (memo)
Try using the Python Cmd module
About the basics list of Python basics
Learn the basics of Python ① Beginners
[Python] Visualize the heat of Tokyo and XX prefectures (DataFrame usage memo)
Organize the flow when running Django using NGINX and gunicorn (personal memo)
Get and estimate the shape of the head using Dlib and OpenCV with python
Python learning memo for machine learning by Chainer until the end of Chapter 2
Python: Basics of image recognition using CNN
[Python] Extension using inheritance of matplotlib (NavigationToolbar2TK)
Automatic collection of stock prices using python
About building GUI using TKinter of Python
Change the length of Python csv strings
Try using the Wunderlist API in Python
Check the behavior of destructor in Python
(Bad) practice of using this in Python
[Learning memo] Basics of class by python
[Python3] Understand the basics of Beautiful Soup
Try using the Kraken API in Python
Behind the flyer: Using Docker with Python
Pass the path of the imported python module
The story of making Python an exe