Build a Docker image containing the private repository Python library on GitHub Actions

background purpose

I've summarized how to bring an internal library from another private repository when building a Python image on Github Actions.

I already had an article by my colleague @ elyunim26, "Securely browse GitHub private repository in a container built with GitHub Actions", but I'm not a machine user, but a [deploy key]( Since I was using https://docs.github.com/ja/free-pro-team@latest/developers/overview/managing-deploy-keys), I needed a secure method for ssh authentication. You can also safely use the ssh key to build an image by using Build-time secrets.

Implementation method

Briefly, it can be achieved by the following procedure.

Deploy key settings

First, you need to set the deploy key in the repository on the library side. Please refer to Official Github documentation for how to set the deploy key.

Github Actions settings

You need to be aware of the following factors: Installing a node module in a private repository with GitHub Actions was helpful for troubleshooting (although there is a difference between Python and node.js).

The following is a setting example. Since our team uses AWS, we also include the code to push to ECR later.

github_actions_ecr.yml


      - name: Build, tag, and push image to Amazon ECR
        id: build-image
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          ECR_REPOSITORY: ${{ steps.extract_repository.outputs.repository }}
          IMAGE_TAG: ${{ github.sha }}
          DEPLOY_KEY_GITHUB: ${{ secrets.DEPLOY_KEY_GITHUB }}
        run: |
          echo ${DEPLOY_KEY_GITHUB} > .deploy_key
          sed -i -e "s#\\\\n#\n#g" .deploy_key
          chmod 600 .deploy_key
          DOCKER_BUILDKIT=1 docker build --secret id=ssh,src=.deploy_key \
            -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG \
            -t $ECR_REGISTRY/$ECR_REPOSITORY:latest \
            .
          rm .deploy_key
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:latest

Dockerfile settings

Our team often uses poetry, so we use pyproject.toml, but if you use requirements.txt, you should be able to use the same settings.

Dockerfile


# syntax=docker/dockerfile:experimental
FROM python:3.7

# ssh-Due to keyscan execution, openssh depending on the original image-client needs to be installed
RUN apt-get install -y openssh-client git

COPY ./pyproject.toml /app/pyproject.toml
COPY ./src /app/src
WORKDIR /app

RUN mkdir -m 700 $HOME/.ssh
RUN ssh-keyscan -H github.com > $HOME/.ssh/known_hosts
RUN --mount=type=secret,id=ssh,dst=$HOME/.ssh/id_rsa \
  pip install --upgrade pip && \
  pip install . && rm pyproject.toml

In pyproject.toml, it is described as follows. This is created by specifying the repository with the poetry add command, as described in the official poetry documentation.

pyproject.toml


[tool.poetry.dependencies]
{Library name} = {git = "ssh://[email protected]/{username}/{Library name}.git", rev = "main"}

Referenced articles

Recommended Posts

Build a Docker image containing the private repository Python library on GitHub Actions
I have a private Python package but I want to install pipenv on GitHub Actions and build a Docker Image
How to build a Django (python) environment on docker
docker build python based on alpine
Run TensorFlow Docker Image on Python3
Build a python3 environment on CentOS7
Build a python environment on MacOS (Catallina)
Build a Python + bottle + MySQL environment with Docker on RaspberryPi3! [Easy construction]
Build CGI Server running on Python 3 on Docker
Use cryptography library cryptography with Docker Python image
Clone the github repository on jupyter notebook
[Mac] Build a Python 3.x environment at the fastest speed using Docker
[Golang] Create docker image with Github Actions
Build a Python + OpenCV environment on Cloud9
[Python] A progress bar on the terminal
Get the size of the image file on the web (Python3, no additional library required)
Build a Python + bottle + MySQL environment with Docker on RaspberryPi3! [Trial and error]
Build a command line app in Python to understand setup.py, argparse, GitHub Actions
A procedure manual for quickly publishing a C ++ Python library using pybind11 on Github.
Build a LAMP environment on your local Docker
Simply build a Python 3 execution environment on Windows
Build a python environment with ansible on centos6
What is wheezy in the Docker Python image?
Build a Python environment on Mac (Mountain Lion)
[Python] Build a Django development environment with Docker
Pip install a private repository when building Docker
Build a Python development environment on your Mac
Try CIing the pushed python code on GitHub.
A memorandum about the Python tesseract wrapper library
Build a Python development environment on Raspberry Pi
A note on the library implementation that explores hyperparameters using Bayesian optimization in Python
[Python] A notebook that translates and downloads the ipynb file on GitHub into Japanese.
Build a basic Data Science environment (Jupyter, Python, R, Julia, standard library) with Docker.
Build a GVim-based Python development environment on Windows 10 (3) GVim8.0 & Python3.6
# 2 Build a Python environment on AWS EC2 instance (ubuntu18.04)
Build a machine learning Python environment on Mac OS
Build a Python extension for E-Cell 4 on Windows 7 (64bit)
Install the 3rd party python library on Cinema 4D
Make a breakpoint on the c layer with python
Build a GVim-based Python development environment on Windows 10 (1) Installation
Build a Python development environment on Mac OS X
Write a log-scale histogram on the x-axis in python
Build a Python development environment using pyenv on MacOS
[Python] Mask the image into a circle using Pillow
How to build a Python environment on amazon linux 2
Steps to create a Job that pulls a Docker image and tests it with Github Actions
build Python on Ubuntu
Install Python 3.6 on Docker
Build a data analysis environment with Kedro + MLflow + Github Actions
Build a python machine learning study environment on macOS sierra
[python] Change the image file name to a serial number
How to build a new python virtual environment on Ubuntu
Try a similar search for Image Search using the Python SDK [Search]
Introducing Kaggle's Docker Image on Windows to build an environment
Build a python data analysis environment on Mac (El Capitan)
Build a GVim-based Python development environment on Windows 10 (2) Basic settings
Build a Python environment and transfer data to the server
Build a Python extension for E-Cell 4 on Mac OSX (Yosemite)
Build a python environment on CentOS 7.7 for your home server
The story of releasing a Python text check tool on GitHub x CircleCI for the first time