Python 2.7, 3.4, 3.5 extension module build environment on Windows

Here's a summary of some basic knowledge and useful information you should know when creating a Python binary package for Windows.

About the importance of binary packages on Windows

Users of Python on Windows are very unlikely to have an environment to build Python extensions. There are fewer users who originally installed the compiler than on Mac, and while the Mac version has a fixed compiler for each OS version, on Windows the version of the compiler is determined by the version of Python instead of the OS. For some reason, it's almost zero chance that someone other than someone who has researched and prepared a Python development environment has set up the right compiler.

On the other hand, in Windows, the official binary provided by python.org has become a strong standard, and the Python environment distributed by third parties is almost binary compatible, so users can distribute the binary package by themselves. You can use Python extension modules without building. The fact that the compiler version is not tied to the OS is also an advantage for the binary package provider that it is not necessary to prepare multiple OS versions.

Therefore, it is recommended that the provider of the package containing the extension module prepare the Windows binary package if possible.

Nowadays, even if you don't want to install heavy tools like Windows SDK or Visual Studio on your Windows, you can use EC2, Azure, GCP, etc. to use a Windows environment with a stronger CPU such as Atom on an hourly basis. I will.

About installing multiple versions of Python and the py command

The standard installation directory is C: \\ Python34 for Python 3.4. You can change it, but avoid directories that contain spaces to avoid trouble.

If you install both the 64-bit version and the 32-bit version, the default installation destination is the same, so change one to C: \\ Python34_amd64. It will be a little lighter if you remove IDLE, TK, tests, documentation, etc. in the installation options.

When you install Python 3.3 or later, a Python launcher called py is installed. This launcher launches the 64-bit version of Python 3.4 with py -3.4 and the 32-bit version of Python 2.7 with py -2.7-32. This is useful because you don't have to enter long full paths multiple times or switch PATH settings from build to build. Not only when you run setup.py, but also when you run the pip command, you can run it with the -m option like py -2.7-32 -m pip.

About Wheel

Python can build binary packages in installer (exe or msi) format by default. However, packages of this form cannot be pip installed or installed inside virtualenv. (If you look at that, it's a good distribution method for users familiar with Windows.)

The currently recommended binary package for pip install is wheel.

To build a wheel, use pip install -U wheel setuptools to get the latest version of wheel and setuptools, then use the command python setup.py bdist_wheel.

Although wheel is a binary package, even if you distribute a pure Python library, it has advantages such as faster installation than .tar.gz, so let's actively use it.

About MSVCRT

When you run a program written in C or C ++, you need a library called its runtime library. This library contains standard library functions such as fopen and malloc.

The runtime library used by the Visual C ++ compiler is MSVCRT. It is provided for each compiler version. Note that the VC version does not match the VS version (year).

It is possible for a process to use multiple versions of MSVCRT at the same time, but it is safer to unify them if possible. Therefore, when building Python extensions, you should use the VC that was used to build the official version of Python for Windows.

VS version VC version Python
2008 9 2.7
2010 10 3.4
2012 11
2013 12
2015 14 3.5

Python 2.7

The VC for Python 2.7 is five years old, so it was a hassle to prepare now.

Fortunately, this year Microsoft released Microsoft Visual C ++ Compiler for Python 2.7. Now, just install it and you will have a build environment for Python 2.7 extension modules.

When building, it will be searched from the registry and used without worrying about environment variables, so just execute the command from the command prompt normally. In the following example, wheels are created for 32bit version and 64bit version respectively. The created wheel is under the dists / directory and can be pip install.

> py -2.7-32  setup.py bdist_wheel
> py -2.7-x64 setup.py bdist_wheel

Python 3.4

There are Express Edition of Visual Studio and the recently released Community Edition as a way to get VC for free.

Unfortunately, there are 11 VCs available in the Community Edition, so you'll have to wait for Python 3.5. It's getting harder to find a way to get Visual Studio 2010 Express Edition. Even if you go through it lightly, there are only 2012 sites at the top. In addition, VS 2010 Express Edition has a limitation that the 64-bit compiler cannot be used. This makes it impossible to prepare a 64-bit version of Python binary package.

Therefore, install Windows SDK 7.1 instead of Visual Studio. You can choose which parts to install during installation, so choose only the x86 and amd64 C ++ compilers.

When using the Windows SDK, unlike Visual Studio, Python does not automatically detect the compiler in the registry, so you need to set environment variables. When you install the SDK, a command prompt shortcut called "Windows SDK 7.1 Command Prompt" will be created in the start menu, so copy it and prepare a shortcut for the Python extension build. Set the "link destination" of the shortcut as follows, and specify the working folder as your working folder.

Link destination (If you change the installation destination of Windows SDK, please rewrite that part):

> #32bit version
> C:\Windows\System32\cmd.exe /E:ON /V:ON /T:0E /K "C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.cmd" /Release /x86
> #64bit version
> C:\Windows\System32\cmd.exe /E:ON /V:ON /T:0E /K "C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.cmd" /Release /x64

All you have to do now is launch from each shortcut and run the command as you did in 2.7.

> #32bit version
> py -3.4-32  setup.py bdist_wheel
> #64bit version
> py -3.4-x64 setup.py bdist_wheel

Python 3.5

You can use Visual Studio Community Edition. As with Python 2.7, you can build extension modules without any hassle.

Also, [Visual C ++ Build tools](http://blogs.msdn.com/b/vcblog/archive/2015/11/02/announcing-visual-c-build-tools-2015-standalone-c-tools-for -build-environments.aspx) has been announced in beta. It's unverified, but you'll probably get the compiler and other tools you need without having to install the heavyweight Visual Studio, just like Python 2.7.

Operation check and upload

Install and test your wheel with pip install before uploading it to your PyPI. You can also use setup.py to upload to PyPI, like py -3.4-32 setup.py bdist_wheel upload, but you'll have to recreate the wheel after testing.

According to the Pytohn Packaging User Guide, the currently recommended package upload method is [twine](https://pypi.python. org / pypi / twine). You only need to install twine on one Python and you can upload packages for all Python.

> py -3.4 -m pip install -U twine
> py -3.4 -m twine upload dist/yourpackage-1.2.3-*.whl

Summary

Recommended Posts

Python 2.7, 3.4, 3.5 extension module build environment on Windows
Build Python environment on Windows
Build python environment on windows
Simply build a Python 3 execution environment on Windows
Build a Python extension for E-Cell 4 on Windows 7 (64bit)
Build a GVim-based Python development environment on Windows 10 (1) Installation
Python environment construction memo on Windows 10
Anaconda python environment construction on Windows 10
Install python2.7 on windows 32bit environment
Install Python development environment on Windows 10
Build Python 3.8 + Pipenv environment on Ubuntu 18.04
Build a python3 environment on CentOS7
Python on Windows
python windows environment
Build a GVim-based Python development environment on Windows 10 (2) Basic settings
Build Python3 and OpenCV environment on Ubuntu 18.04
Build a python environment on MacOS (Catallina)
Creating a python virtual environment on Windows
Build Python environment with Anaconda on Mac
Build a Python + OpenCV environment on Cloud9
Build a 64-bit Python 2.7 environment with TDM-GCC and MinGW-w64 on Windows 7
python windows environment construction
python basic on windows ②
Install python on windows
Build TensorFlow on Windows
Build XGBoost on Windows
build Python on Ubuntu
Build python environment with pyenv on EC2 (ubuntu)
Build Python3.5 + matplotlib environment on Ubuntu 12 using Anaconda
Build a python environment with ansible on centos6
Build a Python environment on Mac (Mountain Lion)
Build a Python development environment on your Mac
Build Python3 + flask environment on GCP Compute Engine
Create a Python virtual development environment on Windows
Build a Python development environment on Raspberry Pi
Run Openpose on Python (Windows)
Install watchdog on Windows + Python 3.3
# 2 Build a Python environment on AWS EC2 instance (ubuntu18.04)
Build a machine learning Python environment on Mac OS
Python + Kivy development on Windows
Prepare Chainer environment on Windows
Python environment construction (Windows10 + Emacs)
Create a comfortable Python 3 (Anaconda) development environment on windows
Sphinx-autobuild (0.5.2) on Windows7, Python 3.5.1, Sphinx 1.3.5
Build python3 environment with ubuntu 16.04
Fastest Python installation on Windows
How to build a Django (python) environment on docker
Build python environment with direnv
Build a Python development environment on Mac OS X
Django environment development on Windows 10
Build a Python environment on your Mac using pyenv
[ev3dev × Python] Build ev3dev development environment
Procedure for building a CDK environment on Windows (Python)
Python environment construction under Windows7 environment
Build a Python development environment using pyenv on MacOS
Build a Python environment offline
I ran python on windows
[Tensorflow] Tensorflow environment construction on Windows 10
Create a decent shell and python environment on Windows
[Python] [Chainer] [Windows] Install Chainer on Windows
Use Python on Windows (PyCharm)