Dynamically import / reload modules that have changed in Python

This is an example of monitoring a specific directory and importing / reloading when a Python module is created / updated.

manner

File monitoring

File monitoring can be implemented by yourself, but it's easy to do with existing libraries.

Here we use a module called watchdog. You can use watchdog to detect events such as creating / updating / deleting files in a specific directory.

Dynamic import of modules

You usually use the import statement to import a module, but you cannot specify the module name as a string.

If you want to dynamically specify the module you want to import with a character string, use the standard module impotlib.import_module (). To do.

Module name cannot be specified as a character string in the import statement


>>> import 'sys'
  File "<stdin>", line 1
    import 'sys'
               ^
SyntaxError: invalid syntax

importlib.import_module()Then specify the module name as a character string


>>> import importlib
>>> importlib.import_module('sys')
>>> sys = importlib.import_module('sys')
>>> sys
<module 'sys' (built-in)>

Module reload

Updating the Python file of an imported module will not be reflected in the running program. Even if you execute the import statement or ʻimpotlib.import_module ()` again, it will not be reflected.

Use importlib.reload () to reload the imported module.

python


>>> with open('a.py', 'w') as f:
...     f.write('def f():\n    print("1")')
...
23
>>> import a
>>> a.f()
1
>>> with open('a.py', 'w') as f:
...     f.write('def f():\n    print("2")')
...
23
>>> a.f()
1
>>> import a
>>> a.f()
1
>>> import importlib
>>> a = importlib.import_module('a')
>>> a.f()
1
>>> a = importlib.reload(a)
>>> a.f()
2

Sample code

Works with Python 3.4 and above.

plugin_manager.py


import sys
import time
from importlib import import_module, reload
from pathlib import Path

from watchdog.events import FileSystemEvent, PatternMatchingEventHandler
from watchdog.observers import Observer


class PluginManager:

    class Handler(PatternMatchingEventHandler):

        def __init__(self, manager: 'PluginManager', *args, **kwargs):
            super().__init__(*args, **kwargs)
            self.manager = manager

        def on_created(self, event: FileSystemEvent):
            print(event)
            if event.src_path.endswith('.py'):
                self.manager.load_plugin(Path(event.src_path))

        def on_modified(self, event):
            print(event)

    def __init__(self, path: str):
        self.plugins = {}
        self.path = path
        self.observer = Observer()

        sys.path.append(self.path)

    def start(self):

        self.scan_plugin()

        self.observer.schedule(self.Handler(self, patterns='*.py'), self.path)
        self.observer.start()

    def stop(self):
        self.observer.stop()
        self.observer.join()

    def scan_plugin(self):
        for file_path in Path(self.path).glob('*.py'):
            self.load_plugin(file_path)

    def load_plugin(self, file_path):
        module_name = file_path.stem
        if module_name not in self.plugins:
            self.plugins[module_name] = import_module(module_name)
            print('{} loaded.'.format(module_name))
        else:
            self.plugins[module_name] = reload(self.plugins[module_name])
            print('{} reloaded.'.format(module_name))


def main():

    plugin_manager = PluginManager('plugins/')
    plugin_manager.start()

    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        plugin_manager.stop()

if __name__ == '__main__':
    main()

Execution result

plugins/To a.Start with py


a loaded.

plugins/To b.Create py


<FileCreatedEvent: src_path='plugins/b.py'>
b loaded.
<DirModifiedEvent: src_path='plugins/'>
<DirModifiedEvent: src_path='plugins/__pycache__'>

b.To pyprint('bbb')Write and save


<FileCreatedEvent: src_path='plugins/b.py'>
bbb
b reloaded.
<DirModifiedEvent: src_path='plugins/'>
<DirModifiedEvent: src_path='plugins/__pycache__'>

Supplement

--I think the watchdog handler should call ʻon_modified () when it detects a file update, but ʻon_created () was called.

Recommended Posts

Dynamically import / reload modules that have changed in Python
Dynamically import scripts in Python
Import your own modules in Grasshopper's Python development
In Python, create a decorator that dynamically accepts arguments Create a decorator
Modules that may go through the shell in Python
Dynamically call methods in Python
Working with LibreOffice in Python: import
Dynamically load json type in python
Import modules that are often used when starting the python interpreter
Dynamically define functions (methods) in Python
Features of regular expression modules that I often use personally in Python
[Python3] Dynamically define global variables in functions
Modules and packages in Python are "namespaces"
How to dynamically define variables in Python
Note that Python decorators should have wraps
How to dynamically zero pad in Python
Write python modules in fortran using f2py
Module import and exception handling in python
Import Error in Python3: No module named'xxxxx'
A solution to the problem that the Python version in Conda cannot be changed
One liner that outputs multiplication tables in Python
Draw contour lines that appear in textbooks (Python)
[Python] A tool that allows intuitive relative import
Building an environment that uses Python in Eclipse
Summary of how to import files in Python 3
Note that the Pandas loc specifications have changed
A program that removes duplicate statements in Python
To dynamically replace the next method in python
Static type checking that starts loosely in Python
Testing methods that return random values in Python
Difference in object ID due to import in Python
The one that displays the progress bar in Python
Import cv2 ModuleNotFoundError: No module named'cv2' in python3
Formulas that appear in Doing Math with Python
Note that the method of publishing modules to PyPI has changed in various ways.