Python logging standard library for file output by log level

background

-There was talk of wanting to separate log files for each log level in Python. ・ In the first place, I used the logging library in a hurry and didn't understand it properly.

Let's solve the problem while moving our hands and organizing.

review

import logging

logging.basicConfig(format='%(asctime)s   %(message)s', datefmt='%m/%d/%Y', filename = './basic.log')
logging.warning('basic.Recorded in log with date.')
logging.basicConfig(filename = './not-appeared.log')
logging.warning('basicConfig only works the first time after booting, so not-appeared.No log is generated.')

root_logger = logging.getLogger('')
root_logger.setLevel(logging.ERROR)
root_logger.error('A root logger has been set up to record ERROR levels and above. basic.It will continue to appear in the log. I haven't specified a handler, so I can't find anything else.')

root_logger.addHandler(logging.StreamHandler())

root_logger.error('This will appear on the console. The format specified in basicConfig is ignored.')
root_logger.warning('Since the log level is low, it does not appear on the console. If ignored, basic.It does not appear in the log.')

critical_handler = logging.FileHandler('./critical.log')
critical_handler.setLevel(logging.CRITICAL)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
critical_handler.setFormatter(formatter)
root_logger.addHandler(critical_handler)

root_logger.critical('Added a handler that records only CRITICAL. Added date and level name.')
root_logger.error('Although this appears on the console, it is critical because the log level is low.It does not appear in the log.')

second_logger = logging.getLogger('second')
second_logger.setLevel(logging.WARNING)
second_logger.addHandler(logging.StreamHandler())

second_logger.warning('Add a logger and put it on the console. basic.In addition to log, it appears twice on the console with the root logger.')

second_logger.propagate = False

second_logger.warning('If you reject propagate, it will not be transmitted upstream, this is second_Only one logger is available.')

Looking back and examining

-Can be inherited in multiple stages based on the root logger. -Multiple handlers can be set for each. -Log levels can be set for loggers and handlers respectively. -However, since both are lower thresholds, it is not possible to filter only specific logs as required.

solution

If you use a filter, you can filter by calling the log record (instance of the log message) as an argument and returning the boolean value each time you log.

I referred to this Stack Overflow post. As # 1 says, I agree that extracting only a specific level does not make it readable by humans.

python logging specific level only

log_file_by_level.py


import logging

class LoggingFilter(object):
    """A filter that leaves only a specific log level"""
    def __init__(self, level):
        self.__level = level

    def filter(self, logRecord):
        #Remove the level setting to handler and here== self.__It should be possible to set it to level.
        return logRecord.levelno <= self.__level

def set_handler(loglevel, filename):
    #Common log format
    log_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s', datefmt='%Y-%m-%d %H:%M:%S')
    handler = logging.FileHandler(filename)
    handler.setLevel(loglevel)
    handler.setFormatter(log_format)
    handler.addFilter(LoggingFilter(loglevel))
    logger.addHandler(handler)

logger = logging.getLogger(__name__)
#Specify the minimum level to be recorded in the app.
logger.setLevel(logging.DEBUG)
set_handler(logging.WARN, './warning.log')
set_handler(logging.INFO, './info.log')

main.py


from log_file_by_level import logger

logger.warning('I warn you.')
logger.info('Please be informed that this is just a test.')
logger.error('Found error on your smile.')

reference

-Python logging specific level only -Logging HOWTO — Python 3.6.1 documentation

Recommended Posts

Python logging standard library for file output by log level
Output log in JSON format with Python standard logging
Notes for Python file input / output
Logging settings for daily log rotation in python
Output python log to both console and file
[CentOS8] How to output Python standard output to systemd log
Change the standard output destination to a file in Python
[GCP] How to output Cloud Functions log to Cloud Logging (Stackdriver Logging) (Python)
How to change the log level of Azure SDK for Python
Read the standard output of a subprocess line by line in Python
Output log file with Job (Notebook) of Cloud Pak for Data
[python3] Implement debug log output function easily with logging and Click
Python3 standard input for competitive programming
Tips on Python file input / output
Output to csv file with Python
Write standard output to a file
Make standard output non-blocking in Python
Unit test log output with python
<For beginners> python library <For machine learning>
Add TRACE log level to Python ...?
[google-oauth] [python] Google APIs Client Library for Python
Output elapsed time for data logging (for yourself)
Output Python log to console with GAE
Read the file line by line in Python
Read the file line by line in Python
UnicodeEncodeError struggle with standard output of python3
Pandas of the beginner, by the beginner, for the beginner [Python]
Python> Output numbers from 1 to 100, 501 to 600> For csv
Python standard library: second half (Python learning memo ⑨)
Atcoder standard input set for beginners (python)
Overwrite download file for python selenium Chrome
Python standard library: First half (Python learning memo ⑧)
A textbook for beginners made by Python beginners
Output timing is incorrect when standard (error) output is converted to a file in Python
I tried searching for files under the folder with Python by file name