A story about trying a (Golang +) Python monorepo with Bazel

This article is the third day of OpenSaaS Studio Advent Calendar 2019.

Introduction

The former is a mono-repo configuration using Bazel in Golang and Python microservices. I wondered if Python could be mixed with the mono repo if I used Bazel, which supports multiple languages, but it was difficult, so I will leave a difficult point.

Premise

• Microservices consist of Golang + Python
• There is no Golang story in this article.
• Each service is built on a Docker image and runs on Kubernetes.
• Build process before trying
• Golang runs source code build, Docker image creation, push of created image with Bazel
• Python creates a Docker image with docker build and pushes that image
• Bazel and Monorepo
  • Bazel
• Build tool that supports multiple languages. Build rules for multiple languages can be defined in the same notation.
• Mono repo
• Manage multiple services in one repository. It's easy to refactor and code review, and it doesn't complicate the dependencies between libraries.
• mixi's [this article](https://medium.com/mixi-developers/bazel%E3%81%A8%E3%83%A2%E3%83%8E%E3%83%AC%E3% 83% 9D-b901ffba61ce) was easy to understand.
• I have almost no experience with Python
• I made an app for slack notification with AWS Lambda while searching the net.

motivation

• I want to make it easier to manage the version of the proto file.
• If you change the proto file, you need to reflect the change in each of the separate repositories.
• If there is only one repository, you can change the proto file in it.
• I want to improve the visibility of build definition / process.
• As I wrote in the premise, the build tools and processes were different depending on the language.
• It would be nice to be able to reduce the context switching related to builds by consolidating them into one build tool / process.
• I want to reduce CI / CD jobs and manage less.
• If you can build transparently with Bazel, you may be able to organize jobs without having to build each language so much.

Clogged point

Both Python 2 and 3 are required

I used only Python3 series in the service, but I needed Python2 series to push the created image. Since I used Container Registry as the image registry of the container, Bazel needs Google Cloud SDK to push the image and it is linked to it. I think that Python2 system was needed in the form of a sequel. Description of Google Cloud SDK

Recent versions of macOS include the appropriate version of Python required for the Google Cloud SDK. The Cloud SDK requires Python 2 with a release number of Python 2.7.9 or later. If you install an additional Python interpreter, it must not interfere with the installation of the Google Cloud SDK.

What happened

Bazel has a mechanism that allows you to specify the Python interpreter to use when building, so I used that.

• Install Python 2.7 and Python 3.6 with pyenv.
• By defaulting Python2.7, Python2 system will be used when pushing images with Bazel.
• Define the Python3 interpreter to be used at build time.
• Specify the version for the target you want to build with Python3 series.

The interpreter definition and build definition look like this.

• BUILD

load("@rules_python//python:defs.bzl", "py_runtime_pair")

py_runtime(
    name = "py2_local_bin_runtime",
    interpreter_path = "/usr/local/bin/python2",
    python_version = "PY2",
)

py_runtime(
    name = "py3_6_local_bin_runtime",
    interpreter_path = "/usr/local/bin/python3.6",
    python_version = "PY3",
)

py_runtime_pair(
    name = "py_local_bin_runtime_pair",
    py2_runtime = ":py2_local_bin_runtime",
    py3_runtime = ":py3_6_local_bin_runtime",
)

toolchain(
    name = "py_mac_toolchain",
    exec_compatible_with = [
        "@bazel_tools//platforms:osx",
        "@bazel_tools//platforms:x86_64",
    ],
    toolchain = "py_local_bin_runtime_pair",
    toolchain_type = "@rules_python//python:toolchain_type",
)

• WORKSPACE

register_toolchains(
    "//:py_mac_toolchain",
    "//:py_linux_toolchain",
)

• BUILD.bazel

py_binary(
    name = "mysite",
    srcs = ["manage.py"],
    deps = [
        "//mysite/mysite:py_default_library",
        "//mysite/polls:py_default_library",
    ],
    main = "manage.py",
    python_version = "PY3",
)

I can't get the library corresponding to Python3

When using Bazel rules for Python, it seems that the pip associated with the python command is executed when downloading the library. As I wrote above, the default python was 2 series, so I could download the Python 3 series library. The code is around here

What happened

Changed to python-> python3.6 so that you can use 3 series by forking .. However, when I checked the latest version, it was Pip3 compatible by default, so there may be no problem now.

Suffering from the entry point of the created image

Bazel has a default rule (py3_image) for creating a Docker image of Python3 series, and when I built it in combination with python: 3.6-slim, the created image could not be started. The build definition looks like this

• WORKSPACE

container_pull(
    name = "py3_base",
    registry = "index.docker.io",
    repository = "library/python",
    tag = "3.6-slim",
)

• BUILD.bazel

py3_image(
    name = "mysite_image",
    srcs = ["manage.py"],
    deps = [
        "//mysite/mysite:py_default_library",
        "//mysite/polls:py_default_library",
    ],
    main = "manage.py",
    base = "@py3_base//image",
)

Actually, the entry point of the image created by py3_image is / usr / bin / python by default, but in python: 3.6-slim, the python command is not set in that path and it can not be started. did. The code is around here

What happened

I used this image because it started normally with python: 3.6. (Image optimization is assumed to be performed later if necessary)

Python version is different for each service

I had three Python services and the versions of Python I was using were all different from 3.5, 3.6 and 3.7. The method of using the Python interpreter described above can only separate the 2nd and 3rd systems, so this mechanism cannot handle it.

What happened

I decided to move all Python versions used in the service to 3.6. It took a long time to verify so far, and I thought that complicating the configuration around the build would deviate from the original policy, so I decided to reduce the number of things to manage. If you want to coexist with multiple versions of Python3, you may be able to separate WORKSPACE and specify the required Python interpreter for each. (unconfirmed)

• Configuration image

├── service_for_python3.6
│   └── WORKSPACE <-Python3 here.Set up 6 interpreters
├── service_for_python3.7
│   └── WORKSPACE <-Python3 here.Set up 7 interpreters
└── WORKSPACE

Import of __init__.py is broken

(Probably) The same event as this issue occurred in the google-cloud-XXX series library. It seems that the problem was caused by the different directory structure when installing the library with pip install and when building with Bazel.

What happened

The issue also has a comment, but it can now be executed by specifying the attribute legacy_create_init.

Docker image and build environment platform are different

I'm using tensorflow-gpu for some services and couldn't download the library because the build environment didn't have a GPU.

What happened

It is unsolved. It's been a while since I started the verification, so I rounded it up here. Bazel may be able to handle it by specifying a platform associated with the GPU (if it exists) or by building this service on a build farm that has a GPU.

Summary

• If you don't need a special platform such as GPU, you can use Bazel to build Python.
• It is unclear if build and test can be compatible in both local environment and CI environment when GPU is required.
• However, in addition to the learning cost of Bazel itself, there is a habit of rules for Python, so it seems that there is little merit to adopt it in a Python-only environment.
• If it is composed of multiple languages, there may be a merit to adopt it because build definitions and processes can be integrated.
• Bazel is fast build, so it takes a long time to build with mono repo In some cases, there may be an advantage to adopting Bazel.
• It will be a little easier if the Google Cloud SDK no longer depends on Python2. It's been less than a month until EOL, but will Python3 alone be enough by then?