If you are a Han, you want to install a prebuilt wheel that includes C ++ modules on Windows, macOS, and Linux with Python pip!
Let's do it with cibuildwheel!
https://github.com/joerick/cibuildwheel
It's a script that builds in various environments and python versions. It's great. You can incorporate this into your own project CI (travis, GitHub Actions, etc).
It's a little difficult to understand,
$ python3 -m cibuildwheel --output-dir wheelhouse
By executing, python setup.py bdist_wheel
equivalent will be built and executed for each python version environment.
(For Linux and macOS, use Docker).
In cibuildwheel, Linux and macOS are executed in the Docker container, so when performing CI build with Travis etc., something like git submodule update
must be executed inside the docker container.
You can set some in the environment variables of cibuildwheel.
https://cibuildwheel.readthedocs.io/en/latest/options/
Notes on uploading packages in Linux binaries to PyPI https://qiita.com/syoyo/items/6185380b8d9950b25561
Please refer to
With Azure pipeline, upload automation after build with twine etc. seems to be troublesome at present, but is the setting of vispy helpful?
https://github.com/vispy/vispy/blob/master/azure-pipelines.yml
Templates have been proposed for the time being, but it seems that there are some restrictions.
https://github.com/joerick/cibuildwheel/issues/229
I actually tried using it with tinyobjloader, but the online documentation was hard to understand and it felt more troublesome than Travis. (It would be nice if the yaml description could be syntax checked with an online editor)
If you can use Travis, we recommend using travis (or Github actions).
CopyFiles @ 2
to copy the build to the staging folder (disk space left after the job is finished?)$ (Build.ArtifactStagingDirectory)
is the staging folderPublishBuildArtifacsts @ 0
- task: PublishBuildArtifacts@1
inputs:
path: $(Build.ArtifactStagingDirectory)
artifactName: tinyobjDeployWindows
In deploy (upload to e.g. PyPI),
First check that the dependent task is complete (dependsOn
) and that it is a tag commit.
- job: deployPyPI
# Based on vispy: https://github.com/vispy/vispy/blob/master/azure-pipelines.yml
pool: {vmImage: 'Ubuntu-16.04'}
condition: and(succeeded(), startsWith(variables['Build.SourceBranch'], 'refs/tags/v'))
dependsOn:
- linux
- macos
- windows
Then use DownloadBuildArtifacts
to download the job artifacts (actually folders).
- task: DownloadBuildArtifacts@0
inputs:
artifactName: 'tinyobjDeployWindows'
downloadPath: $(Pipeline.Workspace)
At this time, a folder is created with ʻartifactName (
$ (Pipeline.Workspace) / tinyobjDepoloyWinwdows`), and the build is placed in it.
For debugging, it seems good to output it to the log with ls etc. so that you can check it.
Then publish it on twine and you're done!
# Publish to PyPI through twine
- bash: |
cd $(Pipeline.Workspace)
find .
python -m pip install --upgrade pip
pip install twine
echo tinyobjDeployLinux/python/dist/*
echo tinyobjDeployLinux/python/wheelhouse/* tinyobjDeployMacOS/python/wheelhouse/* tinyobjDeployWindows/python/wheelhouse/*
twine upload -u "__token__" --skip-existing tinyobjDeployLinux/python/dist/* tinyobjDeployLinux/python/wheelhouse/* tinyobjDeployMacOS/python/wheelhouse/* tinyobjDeployWindows/python/wheelhouse/*
env:
TWINE_PASSWORD: $(pypiToken2)
For the TWINE password (secret token), register the secret token with variable on the Azure Pipeline website with the name pypiToken2
.
I do git push every time, trigger the build, wait for about 10 minutes, fix it if an error occurs & attach the tag again and push is troublesome, so I want an environment where I can try it at hand.
Twine token
Azure Pipeline also has a secret variables function that can handle Twine (PyPi) secret tokens, so you can use that.
https://docs.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#secret-variables
The Secret variable is specified via an environment variable in the build script.
PublishPipelineArtifact
only prints logs that aren't even the error message ʻOne or more errors occurred` (no error reason is displayed) and it doesn't work.
https://stackoverflow.com/questions/58841733/how-to-debug-azure-devops-task-publishpipelineartifact-when-one-or-more-errors-o
Use PublishBuildArtifacts
.
cibuildwheel, pypi upload,
pytinyexr (PyEXR) uses Travis.
https://github.com/syoyo/PyEXR
tinyobjloader uses Azure Pipeline.
https://github.com/tinyobjloader/tinyobjloader/blob/master/azure-pipelines.yml
Py2.7(or pypy) + Windows + pybind11
At least pypy will cause a build error, so be sure to include pp27-win32 and pp36-win32 in CIBW_SKIP
and do not build.
(Is anyone using pypy?)
Also, python2.7 + C ++ 11 is basically deprecated.
https://cibuildwheel.readthedocs.io/en/stable/cpp_standards/
Unless you have a strong reason to use py27, you shouldn't support python 2.7.
Recommended Posts