[Reintroduction to python] How to import via the parent directory

I'm not sure what to do when importing my own module with python, so make a note

background

If you are programming, not limited to python, you will want to divide the process described in one file into multiple files. And then you want to manage the divided files by dividing them into directories. It took some time to import the files that were started to be managed by dividing them into these directories, so I will leave the corresponding method.

In addition, although it is fine, the pattern can be divided into the following two. In this article, I will describe the one.

1. If you want to import a module in the path via the parent directory in the executable file

.
├package1
│  ├__init__.py
│  └module1.py
└package2
   ├__init__.py
   └module2.py

↑ When the executable file is module1.py and you want to call module2.py from module1.py

2. If you want to import a module in the path via the parent directory among the modules imported by the executable file

.
├package1
│  ├__init__.py
│  └module1.py
├package2
│  ├__init__.py
│  └module2.py
└module3.py

↑ When the executable file is module3.py, module3.py wants to import module1.py, and module1.py wants to import module2.py.

A little more concrete

Until cutting out common processing

When developing a function with python, you will want to cut out common processing and modularize it as the number of files increases from 1 file to 2 files and 3 files at first.

initial state

python_import_test
├functionA.py
├functionB.py
└functionC.py

↓ Cut out common processing into common package

python_import_test
├common
│  ├util.py
│  └__init__.py # <= python3.After 3__init__.Can be imported without py
├functionA.py
├functionB.py
└functionC.py

Contents of common / util.py

common/util.py

import os


def test():
    print("this method is in %s" % os.path.basename(__file__))

Contents of functionA.py

functionA.py

from common import util


print("this is %s" % __file__)
util.test()

Run

$pwd
/python_import_test

$python functionA.py
this is functionA.py
this method is in util.py

Main subject: I want to manage directories separately

I don't think there is any particular problem so far, but after this, the number of files will increase, and you will want to manage the directories separately for each function as shown below.

Before directory management

.
├common
│  ├db.py
│  └__init__.py
├functionA_1.py
├functionA_2.py
├functionA_3.py
├functionB_1.py
├functionB_2.py
└functionB_3.py

↓ After directory management

.
├common
│  ├util.py
│  └__init__.py
├functionA
│ ├functionA_1.py
│ ├functionA_2.py
│ └functionA_3.py
└functionB
  ├functionB_1.py
  ├functionB_2.py
  └functionB_3.py

When I try to execute it by writing the following, an error occurs.

functionA/functionA_1.py

from ..common import util

print("this is %s" % __file__)
util.test()

$pwd
/python_import_test/functionA

$python functionA_1.py
Traceback (most recent call last):
  File "functionA_1.py", line 1, in <module>
    from ..common import util
ImportError: attempted relative import with no known parent package

It seems that python initializes sys, built-in, and main at runtime, but when initializing main, the upper directory of the executable file is not included in the module search path, so import It seems that you will not be caught in the search when you do.

solution

There were two major solutions in the range examined.

1. Add a module search path
2. Pass the parent directory through the environment variable PYTHONPATH

1. Add module search path

In python import is done in the order of module search => load The path to be searched for the module is stored in sys.path, so if you add the path of the parent directory there, import will be executed normally. However, this seems to violate Python's coding convention PEP8 (because the convention says from and import are at the top of the file), and I'm angry that I'm using a grammar check tool. (Although it does not matter much because it works)

functionA/functionA_1.py

import os
import sys
sys.path.append(os.pardir)
from common import util

print("this is %s" % __file__)
util.test()

$pwd
/python_import_test/functionA

$python functionA_1.py
this is functionA_1.py
this method is in util.py

2. Pass the parent directory through the environment variable PYTHONPATH

If you find it awkward to mess with sys.path in your .py file, here's how to do it (Ichiou is supposed to be the royal road) For local execution only, you can write it in .bashrc, but if you want to run the .py file inside the Docker container, you need to set it in the ENV command.

.bashrc

export PYTHONPATH="<python_import_Full path to test>:$PYTHONPATH"

Dockerfile

ENV PYTHONPATH "<python_import_Full path to test>:$PYTHONPATH"

functionA/functionA_1.py

from common import util
print("this is %s" % __file__)
util.test()

Setting environment variables is a hassle, so I thought I would use 2 when making it rigidly, and 1 otherwise.