Switch the module to be loaded for each execution environment in Python

Overview

Note that it was necessary to switch the PYD module to be read by 32bitOS and 64bitOS. If the module you want to load is xxx.yyy.zzz and your environment is Win64, Make an import hook to read xxx.yyy. ** win64 **. zzz.

from xxx.yyy import zzz
#If this import statement is a win64 environment
# from xxx.yyy.win64 import zzz
#To be the same process as

5. Import System — Python 3.4.3 Document # import-hooks

Get the execution environment

import sys

def get_env():
    platform = ''
    if sys.platform.startswith('win') or sys.platform.startswith('cygwin'):
        platform = 'win'
    elif sys.platform.startswith('darwin'):
        platform = 'mac'
    elif sys.platform.startswith('linux'):
        platform = 'lnx'
    return platform + ('32' if sys.maxsize < 2**31 else '64')

The OS is determined by ** sys.platform **. You can tell whether it is 32bit or 64bit with ** sys.maxsize **. Return'win64'for Win64 and'mac64' for Mac.

Make an import hook

import os.path
from importlib import import_module
from importlib.abc import MetaPathFinder
from importlib.machinery import ModuleSpec, ExtensionFileLoader

#Get the execution environment
env = get_env()

class PydMetaPathFinder(MetaPathFinder):
    def find_spec(fullname, path, target=None):
        if path is None:
            return None
        #Scan the entire list of paths
        for p in path:
            #Module existence confirmation
            pac_mod = fullname.rsplit('.', maxsplit=1)
            pac,mod = pac_mod[0],pac_mod[-1]
            origin = os.path.join(p, env, mod+'.pyd')
            if os.path.exists(origin):
                #The parent package must be imported
                import_module('.'.join([pac,env]))
                name = '.'.join([pac,env,mod])
                return ModuleSpec(
                    name=name,
                    loader=ExtensionFileLoader(name, origin),
                    origin=origin,
                )

This time, we will create a Finder to be registered in ** sys.meta_path **, so make it a subclass of ** MetaPathFinder **. The MetaPathFinder abstract base class wants to override the find_spec abstract method, so we'll implement it.

find_spec About abstract methods

  1. ** fullname **: * str * --Full module name (such as'xxx.yyy.zzz')
  2. ** path **: * [str] *-List of paths to the package containing the module (for example, ['xxx \ yyy'])
  3. ** target ** --Ignored because I didn't use it this time

The return value will be an instance of ** ModuleSpec **. If the module cannot be loaded, return None and the process will be delegated to another Finder.

Implementation points

--The parent package of the module to be loaded is not automatically imported. ** importlib.import_module ** will automatically import the parent package, so import the parent package before loading the target module. --This time, I used ExtensionFileLoader because it is an import of PYD file, but I think that if it is a normal PY file, it will be read by SourceFileLoader.

31.5. importlib – The implementation of import — Python 3.5.0 documentation #import_module 31.5. importlib – The implementation of import — Python 3.5.0 documentation #MetaPathFinder 31.5. importlib – The implementation of import — Python 3.5.0 documentation #SourceFileLoader 31.5. importlib – The implementation of import — Python 3.5.0 documentation #ExtensionFileLoader 31.5. importlib – The implementation of import — Python 3.5.0 documentation #ModuleSpec

Import hook registration

import sys

sys.meta_path.append(PydMetaPathFinder)

** sys.meta_path ** is a list of Finder, so you can append a subclass of Finder. After append, when the import statement is executed, the registered class will be inquired.

Supplement

If you can use cython's pyximport, we recommend that.

Recommended Posts

Switch the module to be loaded for each execution environment in Python
Switch the package to be installed for each environment with poetry
Check the operation of Python for .NET in each environment
Output the specified table of Oracle database in Python to Excel for each file
How to set the development environment for each project with VSCode + Python extension + Miniconda
How to switch the configuration file to be read by Python
Video cannot be loaded with Spyder in Python development environment
In the python command python points to python3.8
Python unittest module execution in vs2017
How to find the cumulative sum / sum for each group using DataFrame in Spark [Python version]
Master the weakref module in Python
How to set the output resolution for each keyframe in Blender
Python environment construction 2016 for those who aim to be data scientists
[Introduction to Python] How to use the in operator in a for statement?
Jupyter Notebook 6.0.2 cannot be installed in the Python 2.7 environment created in Anaconda
Method to build Python environment in Xcode 6
Module to generate word N-gram in Python
MongoDB for the first time in Python
To reference environment variables in Python in Blender
How to switch python versions in cloud9
How to add python module to anaconda environment
Install the Python module in any directory
Prepare Python development environment for each project in Windows environment (VSCode + virtualEnvWrapper + Pylint)
Approach commentary for beginners to be in the top 1.5% (0.83732) of Kaggle Titanic_3
Add syntax highlighting for the Kv language to Spyder in the Python IDE
Approach commentary for beginners to be in the top 1.5% (0.83732) of Kaggle Titanic_1
Notify using Notification Center when the execution environment is macOS in Python
Approach commentary for beginners to be in the top 1.5% (0.83732) of Kaggle Titanic_2
Solution to the problem that Ctrl + z cannot be used in Powershell in Docker for windows environment (provisional)
What to do if you get the error Target WSGI script'/var/www/xxx/xxx.wsgi' cannot be loaded as python module
How to pass the execution result of a shell command in a list in Python
How to use the C library in Python
CERTIFICATE_VERIFY_FAILED in Python 3.6, the official installer for macOS
The fastest way for beginners to master Python
++ and-cannot be used for increment / decrement in python
Introduction to Python Let's prepare the development environment
A solution to the problem that the Python version in Conda cannot be changed
Prepare the execution environment of Python3 with Docker
Create execution environment for each language with boot2docker
Match the distribution of each group in Python
Try adding an external module to pepper. For the time being, in requests.
To dynamically replace the next method in python
Install the python package in an offline environment
Tips for hitting the ATND API in Python
Use cryptography module to handle OpenSSL in Python
The trick to write flatten concisely in python
Try to calculate RPN in Python (for beginners)
To add a module to python put in Julialang
I made a module in C language to filter images loaded by Python
How to get the files in the [Python] folder
I searched for the skills needed to become a web engineer in Python
Introduction to Python "Re" 1 Building an execution environment
Instructions for connecting Google Colab. To the local runtime in a Windows environment
I want to display the progress in Python!
Use os.getenv to get environment variables in Python
Tips for Python beginners to use the Scikit-image example for themselves 7 How to make a module
Run the output code with tkinter, saying "A, pretending to be B" in python
I just want to find the 95% confidence interval for the difference in population ratios in Python
[For beginners] Web scraping with Python "Access the URL in the page to get the contents"
How to count the number of occurrences of each element in the list in Python with weight
You have to be careful about the commands you use every day in the production environment.