Please stop printing and import logging for log output --Qiita
I was reading while thinking, "Well, I see." But at the same time, I thought like this.
** So how do you write logging? ** ** ** Even for free, I'm not good at logging in Python and it takes time, but I can't do logger !! **
So I decided to write this article. The purpose is as follows.
――I want to share my own understanding that I found as a result of desperately facing logger --Don't waste your time worrying about logging when everyone makes apps in Python
Official docs and the view from reading the Qiita article above.
logging has a hierarchical structure.
└── rootLogger
├── scanLogger
│ ├── scanLogger.txt
│ ├── scanLogger.html
│ └── scanLogger.pdf
└── anotherLogger
It has (or is created) multiple child loggers with a logger called rootLogger as the parent.
For example ...
--Module called ʻurllib uses logger called ʻurllib
--ʻApscheduler.scheduler module uses ʻapscheduler.scheduler
logger
The logger for each of these packages and apps is called rootLogger
.
bad_sample.py
import logging
logging.info("Taking log...")
This is synonymous with playing with rootLogger
directly, so let's stop.
It's similar to the concept of playing with an instance instead of playing with a class.
Even if you look at the official document or the sample, you do not know how to write it ... sweat How much fixed wording and how much can I change by myself? (Range that you can name yourself)
That's why I will present the configuration whose operation has been confirmed. If you have a problem, please copy and use it.
As much as possible, I will include a comment in the comments so that you can easily customize it.
** I am writing the logging config file in toml. ** **
You can refer to the toml package with pip install toml
.
The notation method in yaml is here.
terminal
$ tree
.
├── app
│ ├── logger
│ │ ├── __init__.py
│ │ └── logger.py
│ ├── logging.config.toml
│ └── main.py
└── log
├── application.log
└── error.log
$ cd app/
$ python app.py # app/I am writing a sample assuming that this command is executed in the directory
Also put the Github sample code.
main.py
#The previous logger is a directory. The logger at the back is a file.
from logger.logger import get_logger
logger = get_logger()
def main():
logger.info("start = main()")
# do something
logger.error("oops! something wrong!!")
# do something
logger.info("end = main()")
if __name__ == "__main__":
main()
logging.config.toml
is set to output logs to the console screen and file.
toml:logging.config.toml
version = 1
disable_existing_loggers = false #Do not disable loggers on other modules
[formatters]
[formatters.basic]
#Set the log format. Details of how to write will be described later
format = "%(asctime)s [%(name)s][%(levelname)s] %(message)s (%(filename)s:%(module)s:%(funcName)s:%(lineno)d)"
datefmt = "%Y-%m-%d %H:%M:%S"
[handlers]
[handlers.console]
#Set console output here
class = "logging.StreamHandler" #Fixed wording
level = "DEBUG" #log level is your choice
formatter = "basic" #Select the log format listed in formatters
stream = "ext://sys.stdout" #Fixed wording
[handlers.file]
class = "logging.handlers.TimedRotatingFileHandler" #Fixed wording. Details of TimeRotatingFileHandler will be described later. There are notes
formatter = "basic"
filename = "../log/application.log" #If specified by relative path$Write from a directory that runs python
when = 'D' #log A unit for switching files. D= day。
interval = 1 #Since day is selected above, a new file is created every day.
backupCount = 31 #Since day is selected above, the log file for 31 days is retained.
[handlers.error]
class = "logging.handlers.TimedRotatingFileHandler"
level = "ERROR"
formatter = "basic"
filename = "../log/error.log"
when = 'D'
interval = 1
backupCount = 31
[loggers]
[loggers.app_name] # app_name is the name of the logger called from python. Arbitrarily named
level = "INFO"
handlers = [
"console",
"file",
"error",
] #Set which of the handlers set above to use
propagate = false
[root]
level = "INFO"
handlers = [
"console",
"file",
"error"
] #Setting of logger's parent root Logger. I also want to keep the rootLogger in console and file
logger/__init__.py
from . import logger
logger.init_logger('logging.config.toml', 'app_name')
#First argument: configfile =The name of the config file. Relative path or absolute path of the directory executed by the python command
#Second argument: loggername =The name of the logger set in the config file
logger/logger.py
import os
import toml
from logging import getLogger
from logging.config import dictConfig
CONFIGFILE = "logging.config.toml" #Treat as a global variable. The default value is as shown on the left. Any
LOGGERNAME = "root" #Treat as a global variable. The default value is root. Any
def init_logger(configfile, loggername):
global CONFIGFILE
global LOGGERNAME
CONFIGFILE = configfile
LOGGERNAME = loggername
dictConfig(toml.load(CONFIGFILE))
def get_logger():
return getLogger(LOGGERNAME)
This is more flexible than logging in the Config file.
In the Config file, the parent directory is ../
, which makes it more OS-dependent.
If you set it with python dict, you can specify it with ʻos.path.pardir`.
terminal
$ tree
.
├── app
│ ├── logger
│ │ ├── __init__.py
│ │ └── logger.py
│ └── main.py
└── log
├── application.log
└── error.log
$ cd app/
$ python main.py
Also put the Github sample code.
main.py
is [main.py above](https://qiita.com/uANDi/items/9a2b980262bc43455f2e#3-1-2-%E3%83%95%E3%82%A1%E3%82 Same as% A4% E3% 83% AB).
Also, the contents of CONFIG
are the same as the above logging.config.toml
.
logger/__init__.py
from . import logger
logger.init_logger()
logger/logger.py
import os
from logging import getLogger
from logging.config import dictConfig
APP_NAME = os.getenv('APP_NAME', default='app_name')
CONFIG = {
'version': 1,
'disable_existing_loggers': False,
'formatters': {
'basic': {
'format': '%(asctime)s [%(name)s][%(levelname)s] %(message)s (%(module)s:%(filename)s:%(funcName)s:%(lineno)d)',
'datefmt': '%Y-%m-%d %H:%M:%S'
}
},
'handlers': {
'console': {
'class': 'logging.StreamHandler',
'level': 'DEBUG',
'formatter': 'basic',
'stream': 'ext://sys.stdout'
},
'file': {
'class': 'logging.handlers.TimedRotatingFileHandler',
'formatter': 'basic',
'filename': os.path.join(os.path.pardir, 'log', 'application.log'),
'when': 'D',
'interval': 1,
'backupCount': 31
},
'error': {
'class': 'logging.handlers.TimedRotatingFileHandler',
'level': 'ERROR',
'formatter': 'basic',
'filename': os.path.join(os.path.pardir, 'log', 'error.log'),
'when': 'D',
'interval': 1,
'backupCount': 31
}
},
'loggers': {
APP_NAME: {
'level': 'INFO',
'handlers': [
'console',
'file',
'error'
],
'propagate': False
}
},
'root': {
'level': 'INFO',
'handlers': [
'console',
'file',
'error'
]
}
}
def init_logger():
dictConfig(CONFIG)
def get_logger():
return getLogger(APP_NAME)
The output format is the same for both Pattern 1. and Pattern 2.
[app_name]
is the logger name.
terminal
$ cd app/
$ python main.py
2019-12-21 15:43:28 [app_name][INFO] start = main() (main:main.py:main:8)
2019-12-21 15:43:28 [app_name][ERROR] oops! something wrong!! (main:main.py:main:10)
2019-12-21 15:43:28 [app_name][INFO] end = main() (main:main.py:main:12)
$ cat ../log/application.log
2019-12-21 15:43:28 [app_name][INFO] start = main() (main:main.py:main:8)
2019-12-21 15:43:28 [app_name][ERROR] oops! something wrong!! (main:main.py:main:10)
2019-12-21 15:43:28 [app_name][INFO] end = main() (main:main.py:main:12)
$ cat ../log/error.log
2019-12-21 15:43:28 [app_name][ERROR] oops! something wrong!! (main:main.py:main:10)
3-4-1. TimedRotatingFileHandler
What is set in ʻinterval and
when` in Official Document TimedRotatingFileHandler Can you do it? "Is written!
Outputs log to a new file at the set time.
In the above example, 'D'
, that is, the file is rewritten every day.
For past log files, the date surfix is added to the file name, such as ʻapplication.log.2019-12-21`.
**Caution! ** **
TimedRotatingFileHandler
will not work for applications where app.py does not start all day.
It works for web apps like Django and Flask, but may not work properly with cron.
3-4-2. Format
The Official Documentation LogRecord Attribute (https://docs.python.org/en/3/library/logging.html#id2) has a list of each format. Let's arrange the items you need!
I introduced two types of how to make logging! Let's look back on each! (If you think it's perfect, skip it !!)
By the way, Takeaways means "points, points". take away = Take away → Take this away with today's presentation! It is such a nuance.
A number of loggers have been created with rootLogger as the parent. An image like rootLogger → Class, logger → instance.
logging.info ()
will directly mess with the rootLogger, so let's create your own logger.
Then how do you make it concretely?
--Read logging settings from Config file -Toml sample -Yaml sample --Write the settings in the Python dictionary -Sample
--If you do not set disable_existing_loggers = False
, the logger of gunicorn
will be broken.
--When creating a Config file
--Note the OS dependency of the relative path
--The relative path of the log folder is from the directory where you run $ python
TimedRotatingFileHandler
--It is meaningless if the program execution time is short for the interval
--For web apps such as Django and FlaskI don't think this is the correct way to write logging. You can also add a handler to your python file with ʻaddHandler`. We would be grateful if you could teach us how to set up your recommended logging.