What is this

-~~ Still working ~~ A memorandum of work (including advance preparation) when updating a Python 2 series script to 3 series

Premise

--Using pyenv for Python version control --Docker, virtualenv, Pipenv are unused

Overview

Decide which Python to migrate
Verify locally
Raise the version in production (stg)

procedure

--The following is a brief migration procedure.

preliminary survey

Check the current operating environment

--The version of Python to update is limited depending on the OS you are using, so investigate in advance. --Do pyenv install --list on the server to see which version of Python you can migrate --The following is an example when the Linux OS is Linux: Amazon Linux AMI release 2016.03

[ec2-user@app]$ pyenv install --list
Available versions:
#Omission
  3.6.0
  3.6-dev
  3.6.1
  3.6.2
  3.6.3
  3.6.4
  3.7.0b2
  3.7-dev
  3.8-dev
#Omission

Determine the Python version to migrate to

--Select the Python version from the versions that can be updated with pyenv install --list earlier. --Avoid those with dev orb *after the patch version number because they are not stable versions. -Python Developer ’s Guide has the maintenance deadline for each version, so choose the one with the longest EOL possible. --In the previous example of Linux: Amazon Linux AMI release 2016.03, it is appropriate to set it to 3.6.4 for the above reason.

Python Developer’s Guide: https://devguide.python.org/#status-of-python-branches

Local operation verification

Identifying logic that cannot be used in Python 3 series

-Try updating Python code with futurize --We will verify the files and methods that were not applied. --It seems to be the method recommended by the official Python

--Porting from Python 2 to Python 3: https://docs.python.org/ja/3/howto/pyporting.html#porting-python-2-code-to-python-3

futurize　Py2 to Py2/3: http://python-future.org/automatic_conversion.html#stage-1-safe-fixes

Try automatic conversion to Python3 notation

-[2to3](2to3 --Automatic code conversion from Python 2 to 3) automatically converts the notation that is valid only in Python 2 to the 3 system notation (to some extent). ――If the range of influence on the 3rd system is large, you may try converting it at once here (overconfidence is prohibited, operation verification should be done) --This is also the methodology described in the official Python3 documentation.

--2to3 Python 2 to 3 automatic code conversion: https://docs.python.org/ja/3/library/2to3.html#module-lib2to3

Python version switching & script execution

--Once you have some idea of migrating the code and logic in the above library, you can actually run it locally. --The main order is "version switching with pyenv "," library installation ", and" script execution ". --If there is logic that cannot be completed locally, such as the part that needs to be connected to the DB, skip it, and execute only the executable part.

Make a note of the backup method in case of moss

--If it works locally, you will need to update stg and production environment. --Before that, let's describe the switchback operation of "if there is a problem with the operation in Python 3 code" in the document or something. --Specifically expected switchback work --Switch to an older version of Python with pyenv --Revert the commit of the updated code --Reinstall the library with the old version

Implementation of transition

――After that, we will finally change the Python version of the production script. --If you have a stg environment, implement it first in the stg environment

Switching Python version

--Switch the Python version with pyenv

# 3.6.Install 4
$ pyenv install 3.6.4
$ pyenv versions  
  system  
* 2.7 (set by /home/ec2-user/.pyenv/version)
  3.6.4 #Should have been added
$ pyenv local 3.6.4
$ pyenv rehash
$ pyenv versions  
  system  
　 2.7 (set by /home/ec2-user/.pyenv/version)
＊ 3.6.4 #Should have changed
$ python --version
Python 3.6.4 #Should have changed

Reinstall library

--Install the library in Python3 environment --The following is an example of installing using requirements.txt

$ pip install -r requirements.txt
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests->-r requirements.txt (line 5))
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/e1/e5/df302e8017440f111c11cc41a6b432838672f5a70aa29227bf58149dc72f/urllib3-1.25.9-py2.py3-none-any.whl (126kB)
    100% |████████████████████████████████| 133kB 9.1MB/s
Collecting certifi>=2017.4.17 (from requests->-r requirements.txt (line 5))
  Cache entry deserialization failed, entry ignored
  Cache entry deserialization failed, entry ignored
  Downloading https://files.pythonhosted.org/packages/57/2b/26e37a4b034800c960a00c4e1b3d9ca5d7014e983e6e729e33ea2f36426c/certifi-2020.4.5.1-py2.py3-none-any.whl (157kB)
    100% |████████████████████████████████| 163kB 7.3MB/s
#The following is omitted

Checking the operation of the script

--If the script can be moved manually, try running the script. --If it works as expected without throwing an error, the migration is successful. ――It is best to get the operation log and spit it out to a file, or paste it in a document.


[ec2-user@app]$ python main.py

Work memo to migrate and update Python 2 series scripts on the cloud to 3 series