I built a Wheel for Windows using Github Actions

I maintain a client library called mysqlclient that uses libmysqlclient for Python.

This library provides a statically linked Binary Wheel for libmysqlclient for easy use by Windows users. I tried to automate the build of this Binary Wheel using Github Actions.

I'm new to Github Actions, so there may be a better way, but I hope it helps someone.

Impressions

I will write from the impression first. I've already used AppVeyor to auto-build Wheels for Windows in other projects, and it's far more comfortable than that.

image.png

(Some of these may actually be the same with AppVeyor. It's just a personal impression.)

Prepare a repository for build

Instead of using the repository of the library itself, I created another repository mysqlclient-build for build and tried and errored there. This allows you to do trial and error without polluting the library repository.

.github/workflows/windows.yaml


name: Build windows wheels
on:
  push:
    branches:
      - master
  create:

As I will explain later, this affected the directory to be checked out and it was a little troublesome, so I feel that it was okay to make a trial and error with a pull request of the repository of the library itself.

Build dependent libraries

mysqlclient uses libmysqlclient, but Wheel for Windows uses a compatible MariaDB Connector / C instead of MySQL Connector / C.

This is because it can be built to use the Windows API instead of OpenSSL for SSL connections to MySQL, sha256 calculations for caching_sha2_password, etc., which is suitable for statically linked binary distribution.

However, MariaDB Connector / C, which is distributed in binary, has plugins such as authentication as DLLs. I want to make it a single binary if possible, so I build MariaDB Connector / C myself.

Use the cache action because it takes time to build this dependent library every time you build mysqlclient. The cache action caches the specified directory when the job succeeds, so push it once until the MariaDB Connector / C build, make it successful, and cache it. Now I was able to take advantage of the pre-built MariaDB Connector / C when doing the trial and error to build mysqlclient.

.github/workflows/windows.yaml


jobs:
  build:
    runs-on: windows-latest
    env:
      CONNECTOR_VERSION: "3.1.5"

    steps:
      - name: Cache Connector
        id: cache-connector
        uses: actions/cache@v1
        with:
          path: c:/mariadb-connector
          key: mariadb-connector-3.1.5-win

      - name: Download and Unzip Connector
        if: steps.cache-connector.outputs.cache-hit != 'true'
        shell: bash
        run: |
          curl -LO "https://downloads.mariadb.com/Connectors/c/connector-c-${CONNECTOR_VERSION}/mariadb-connector-c-${CONNECTOR_VERSION}-src.zip"
          unzip "mariadb-connector-c-${CONNECTOR_VERSION}-src.zip" -d c:/
          mv "c:/mariadb-connector-c-${CONNECTOR_VERSION}-src" c:/mariadb-connector-src

      - name: Build Connector
        if: steps.cache-connector.outputs.cache-hit != 'true'
        shell: cmd
        working-directory: c:/mariadb-connector-src
        run: |
          mkdir build
          cd build
          cmake -A x64 .. -DCMAKE_BUILD_TYPE=Release -DCLIENT_PLUGIN_DIALOG=static -DCLIENT_PLUGIN_SHA256_PASSWORD=static -DCLIENT_PLUGIN_CACHING_SHA2_PASSWORD=static
          cmake --build . -j 8 --config Release
          cmake -DCMAKE_INSTALL_PREFIX=c:/mariadb-connector -DCMAKE_INSTALL_COMPONENT=Development -DCMAKE_BUILD_TYPE=Release -P cmake_install.cmake

Build mysqlclient

First, check out mysqlclient with the checkout action. By default, the repository with workflow (mysqlclient-build in this case) is checked out to the working directory, but you can use with: to check out another repository to another path.

There was one pitfall here, but if you check out with path: mysqlclient, the checkout destination will be ../ mysqlclient instead of ./mysqlclient. This is because the default working directory is the directory for checking out that repository, and when checking out another repository, the directory is created next to it instead of under it and checked out. So, in the steps after checking out, working-directory: ../mysqlclient is specified.

After building, use the action upload-artifact to upload the wheel to Github, and then make sure you can actually install and import the wheel.

I wanted to test it to work if possible, but I haven't done it this time because windows-latest doesn't have MySQL Server installed. It seems that Docker can be used, so I will try to set up MySQL Server using Docker in the future.

.github/workflow/windows.yml


      - name: Checkout mysqlclient
        uses: actions/checkout@v1
        with:
          repository: PyMySQL/mysqlclient-python
          ref: master
          fetch-depth: 10
          path: mysqlclient

      - name: Site Config
        shell: bash
        working-directory: ../mysqlclient
        run: |
          pwd
          find .
          cat <<EOF >site.cfg
          [options]
          static = True
          connector = C:/mariadb-connector
          EOF
          cat site.cfg

      - name: Build wheels
        shell: cmd
        working-directory: ../mysqlclient
        run: |
          py -3.8 -m pip install -U setuptools wheel pip
          py -3.8 setup.py bdist_wheel
          py -3.7 -m pip install -U setuptools wheel pip
          py -3.7 setup.py bdist_wheel
          py -3.6 -m pip install -U setuptools wheel pip
          py -3.6 setup.py bdist_wheel

      - name: Upload Wheel
        uses: actions/upload-artifact@v1
        with:
          name: win-wheels
          path: ../mysqlclient/dist

      - name: Check wheels
        shell: bash
        working-directory: ../mysqlclient/dist
        run: |
          ls -la
          py -3.8 -m pip install mysqlclient-1.4.6-cp38-cp38-win_amd64.whl
          py -3.8 -c "import MySQLdb"
          py -3.7 -m pip install mysqlclient-1.4.6-cp37-cp37m-win_amd64.whl
          py -3.7 -c "import MySQLdb"
          py -3.6 -m pip install mysqlclient-1.4.6-cp36-cp36m-win_amd64.whl
          py -3.6 -c "import MySQLdb"

Recommended Posts

I built a Wheel for Windows using Github Actions
I built a TensorFlow environment on windows10
Prepare a pseudo API server using GitHub Actions
I made a list site of Kindle Prime Reading using Scrapy and GitHub Actions
I tried using Tensorboard, a visualization tool for machine learning
I made a Line-bot using Python!
I created a visualization site for GDP (Gross Domestic Product) using DASH!
I tried using eval (a, b) for Fibonacci, but it wasn't fast
Notes for using OpenCV on Windows10 Python 3.8.3.
I made a dash docset for Holoviews
[50 counts] Key transmission using Python for Windows
I tried drawing a line using turtle
Beginner: I made a launcher using dictionary
I tried using pipenv, so a memo
I made a library for actuarial science
A procedure manual for quickly publishing a C ++ Python library using pybind11 on Github.
I made a python dictionary file for Neocomplete
I checked the library for using the Gracenote API
I made a spare2 cheaper algorithm for uWSGI
I created a Dockerfile for Django's development environment
I made a downloader for word distributed expression
I tried using Pythonect, a dataflow programming language.
I tried reading a CSV file using Python
Notes for using TensorFlow on Bash on Ubuntu on Windows
What I did before installing Docker for Windows
I tried using firebase for Django's cache server
Tips for using ElasticSearch in a good way
I wrote a Japanese parser in Japanese using pyparsing.
Let's make a module for Python using SWIG
I made a peeping prevention product for telework.
I tried using a database (sqlite3) with kivy
Convert binary packages for windows to wheel format
I tried to make a ○ ✕ game using TensorFlow
I installed Chainer, a framework for deep learning
A real way for people using python 3.8.0-2 from windows to work with multibyte characters
I want to set up a mock server for python-flask in seconds using swagger-codegen.