Summary of logrotate software logrotate

Very major software installed by default in recent Linux. It can be used not only for server log files but also for rotation of web application log files.

file organization

path role
/etc/logrotate.conf Configuration file for the entire log rotation
/etc/logrotate.d/* Configuration file for each service
/var/lib/logrotate.status or
/var/lib/logrotate/status or
/var/lib/logrotate/logrotate/status or
/var/lib/logrotate/logrotate.status
A file that manages the date and time of the last rotation. The path differs depending on the OS and version.

/etc/logrotate.conf contains the default settings. Since /etc/logrotate.d and below are also included, Create a configuration file for each service here.

List of setting items

Settings related to rotation conditions

Directive option Option example Explanation
hourly - - Perform rotation every hour. However, if you do not set it to hourly with cron, it will be daily set with cron.
daily - - Perform rotation daily.
weekly - - Perform weekly rotation.
monthly - - Perform rotation every month.
yearly - - Perform rotation every year.
size [file size] 1M ログファイルが指定したfile size以上になったらローテーションする。サイズの後ろに kKilobytes,MWith megabytes,GIf you add, it is interpreted as gigabyte. Time interval(daily, weekly, monthly, yearly) Setting is ignoredlogrotateRotate only log files that are larger than the specified size when the command is executed.
maxsize [file size] 1M ログファイルが指定したfile size以上になったらローテーションする。時間間隔(daily, weekly, monthly, yearly)も設定されている場合はfile size以上になったときも時間が経過したときも両方ローテーションされる。ただし、logrotateIf you haven't started it on an hourly basis, it's set to cron daily by default, so daily is the shortest.logrotateVersion 3.Features implemented from 10.
minsize [file size] 1M ログファイルが指定したfile size以上になったらローテーションする。時間間隔(daily, weekly, monthly, yearlyIf) is also set, it will not be rotated until both conditions are met.
ifempty - - Rotate even if the log file is empty.
notifempty - - Do not rotate if the log file is empty. When this setting is madeifemptyOverride the settings of.

Settings related to log file generation

Directive option Option example Explanation
create [Permissions username group name] 0644 nginx nginx Create a new empty log file after rotation. It is used when the program that outputs the log file cannot generate the file. You can specify file permissions, user name, and group name. For attributes that are not specified, the attributes of the original file are inherited.postrotateExecuted before the script is executed.
nocreate - - Do not create a new empty log file. When this setting is madecreateOverride the settings of.
copytruncate - - Copy the log file and empty the original file. In the case of a program that cannot close the log file, if the default is used, the log will continue to be written to the archive that should have been rotated. Use when you want to rotate with the log file open. * There is a slight time lag between copying and emptying, and there is a risk that the log will be lost during that time.
nocopytruncate - - Do not empty the original file after copying the log file. When this setting is madecopytruncateOverride the settings of.
copy - - Copy the log file, but leave the original file as is. It is used to take a snapshot of the log file, when other utilities require file truncation, or when parsing the file. When this setting is madecreateOverride the settings of.
nocopy - - Do not copy log files. When this setting is madecopyOverride the settings of.

Settings related to the file name of the log file

Directive option Option example Explanation
dateext - - 8 digits date instead of numbering the suffixes of the rotated logs(-YYYYMMDD)Turn on.
dateformat Format string _%Y-%m-%d Specify the date format.%Y,%m,%d,%H,%sCan be used. * 3.9.In versions prior to 0%H:%M:%SCannot be used.
dateyesterday - - Use yesterday's date instead of today for the date added by dateext. As a result, the time stamp of the rotated file matches the date of the file name.
extension extension .log The specified extension is maintained after log rotation. If you use compression, the extension is followed by the compression extension. For examplemylog.logI have a log file namedmylog.log.1.gznot,mylog.1.log.gzYou can rotate it by adding an extension after the number like.
start Numbers 5 The beginning of the number to be added to the end of the rotation file. The default is1.. For example5If you specifya.log => a.log.5 => a.log.6 =>... and the log file name5start from.
tabooext [+] ext[,ext,…] + .test Files with the extension specified by this option will not be processed. The default is.rpmsave, .rpmorig, ~, .disabled, .dpkg-old, .dpkg-dist, .dpkg-new, .cfsaved, .ucf-old, .ucf-dist, .ucf-new, .rpmnew, .swp, .cfsaved, .rhn-cfg-tmp-*Is excluded.+If you write, the specified extension will be added to the existing extension list. If not, replace the existing list.

Log file directory settings

Directive option Option example Explanation
olddir Directory name /var/backup/log Moved to the directory specified during rotation. The destination must be on the same device as the original. Both absolute path and relative path can be specified. When this setting is madenoolddirOverride the settings of.
noolddir - - Rotate the logs in the same directory. ※Default
createolddir [Permissions username group name] 0644 nginx nginx olddirIf the directory specified in does not exist, a new directory will be created.
nocreateolddir - - olddirIf the directory specified in does not exist, the directory will not be created.

Settings related to log compression

Directive option Option example Explanation
compress - - Compress the rotated log with gzip.
compresscmd Compression command /usr/bin/bzip2 Specify the command to be used for compression when you want to compress with something other than gzip.compresscmdIf you change the compression program withcompressextIf you do not change the value of, there is a high possibility that an error will occur.
compressext extension .bz2 Specifies the extension of the compressed log file. By default, it follows the set compression command.
compressoptions Compression options --best Specifies command line options to pass to the compression program. With the default gzip-6(Compression rate priority) is specified.
nocompress - - Do not compress rotated logs.
delaycompress - - The first generation log is not compressed. In the case of a program that cannot close the log file, if the default is used, the log will continue to be written to the archive that should have been rotated, but if it is compressed, it cannot be written, so it is delayed assuming that case.compressIs ignored if is not specified.
delaycompress - - Logs are compressed from the first generation.compressIs ignored if is not specified.

Settings related to the number of log file rotations and deletion

Directive option Option example Explanation
rotate Number of times 7 Reserves the specified number of logs and deletes older log files.
maxage Days 30 Reserves logs for the specified number of days and deletes older log files.
shred - - unlink()Instead of,shred -uDelete the log file using. *shredFills the file to be deleted with random numbers so that the data cannot be recovered, and then deletes it.
shredcycles Number of times 5 How many times to overwrite the log file before deletingshredInstruct. If this setting is omittedshredThe default value of is adopted.
noshred - - When deleting old log filesshredDo not use. * Default setting

Settings related to email during log rotation

Directive option Option example Explanation
mail mail address [email protected] If the log exceeds the existence period due to rotation, it will be sent to the specified email address.
mailfirst - - mailIf you have set a directive, the file that has just been rotated is sent instead of the file that expires.
maillast - - mailIf you have set a directive, the expired file will be sent instead of the file that was just rotated. *mailDefault settings when using directives
nomail - - Do not email log files.

Settings related to errors during log rotation

Directive Explanation
missingok Continues processing without issuing an error even if the log file does not exist.
nomissingok Gives an error if the log file does not exist. * Default setting

Script settings to be executed during log rotation

Directive Explanation
prerotate/endscript Execute the described command before log rotation.sharedscriptsIf is specified, the absolute path to all files is passed to the script.sharedscriptsIf is not specified, the command will be executed for each target log file.
postrotate/endscript Execute the described command after log rotation.sharedscriptsIf is specified, the absolute path to all files is passed to the script.sharedscriptsIf is not specified, the command will be executed for each target log file.
firstaction/endscript Execute the described command at the beginning of log rotation. If the log file path is a wildcard, the absolute path of the wildcard is passed to the script.
lastaction/endscript Execute the described command at the end of log rotation. If the log file path is a wildcard, the absolute path of the wildcard is passed to the script.
preremove/endscript Executed before deleting the log file. The name of the file to be deleted after execution is passed to the script.
sharedscripts By defaultprerotateWhenpostrotateThe script is run for each rotated log file and the absolute path to that log file is passed as the first argument to the script.sharedscriptsIf is specified, the script will only be executed once, no matter how many files match the wildcard specification.
nosharedscripts prerotateWhenpostrotateExecute the script for each rotated log file. The absolute path to that log file is passed as the first argument to the script.

--/ Bin/sh is used as the command to be executed. --The script will not be executed if none of them require rotation.

Other settings

Directive option Option example Explanation
su [Username Group name] nginx nginx Rotate the log file with the specified users and groups. The default is root.logrotateVersion 3.8.Features introduced from 0.
include File or directory - The file given as an argument,includeRead the directive at the location. If a directory is specified, most files in the directory (directories and named pipes,tabooextFiles ending with the name specified in the directive are excluded) are read in alphabetical order.

Log rotation execution order

  1. Execute first action/end script
  2. Rotation of rotated files (eg log.1.gz-> log.2.gz)
  3. Execute prerotate/endscript
  4. Rotation of target file (example: log-> log.1)
  5. Execute postrotate/endscript
  6. File compression (eg log.1-> log.1.gz)
  7. Execute preremove/endscript *
  8. Delete the compression source file (example: delete log.1) *
  9. Execute preremove/endscript *
  10. Delete expired files (example: delete log.3.gz) *
  11. Execute lastaction/endscript

Setting Example

apache configuration example

/var/log/httpd/*log {   #The log file path is specified here. You can use wildcards.
    missingok           #Processing continues even if the specified log file does not exist.
    notifempty          #Skip if the log file is empty.
    sharedscripts       #Execute the command described in postrotate or prerotate only once for multiple specified log files.
    delaycompress       #The first generation log is not compressed.
    postrotate          #Execute the described command after log rotation. Here, apache is forced to reload the settings.
        /sbin/service httpd reload > /dev/null 2>/dev/null || true
    endscript
}

Syslog configuration example

#Set up 4 log files at once
/var/log/cron
/var/log/messages
/var/log/secure
/var/log/spooler
{
    missingok           #Processing continues even if the specified log file does not exist.
    sharedscripts       #Execute the command described in postrotate or prerotate only once for multiple specified log files.
    postrotate          #Execute the described command after log rotation. If you move a file while syslogd is running, syslogd will lose track of the log file it should output, so it sends a HUP signal to syslogd to reload the settings.
        /bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true
    endscript
}

WEB application log rotation setting example

#Log file path
/var/www/html/application/logs/log.php {
    compress             #Compress rotated logs with gzip
    daily                #Run daily
    dateext              #8 digits date instead of numbering the suffixes of the rotated logs(-YYYYMMDD)Turn on.
    dateformat -%Y-%m-%d #Specify the date format.
    delaycompress        #The first generation log is not compressed.
    extension .php       #Match the extension to the log.
    missingok            #Processing continues even if the specified log file does not exist.
    rotate 60            #Hold for 60 days
    su www-data www-data #Change users and groups to suit your environment.
    create 666 www-data www-data
    lastaction
        #Grant read and write permissions to the owner
        chmod u+rw /var/www/html/application/logs/*.php
    endscript
}

command

Format

logrotate [-dv] [-f|--force] [-s|--state status file]setting file..

option

option Explanation
-? or –help Display a help message.
-d or –debug Turn on debug mode and-Implicitly specify v. Logs and logrotate status files are unchanged in debug mode.
-f or –force Forcibly execute log rotation.
-m command or
–Mail command
Specify the command to use when sending the log by e-mail.
-s status file or
–State status file
Instruct to use an alternative status file.
–usage Show concise usage.
+-v or –verbose Turn on detailed display mode.

Command example

A command to check the operation in debug mode after setting log rotation.

# logrotate -d /etc/logrotate.conf

A command to check the operation in debug mode when a log file that does not meet the conditions is forcibly rotated.

# logrotate -fd /etc/logrotate.conf

A command to actually execute and check.

# logrotate -v /etc/logrotate.conf

A command for forcibly rotating log files that do not meet the conditions.

# logrotate -f /etc/logrotate.conf

Check the date of the rotated log file.

# cat /var/lib/logrotate.status

important point

About execution authority

Since logrotate is run in cron, it will run with root privileges unless you change the user in the cron configuration and the logrotate config file.

If the parent directory of the log file is o + w or the group is other than root and is g + w, it will not rotate.

Also, if you execute a script with preremove/endscript or postremove/endscript, it will be executed with root privileges, so if it is an inappropriate script, unexpected files will be deleted, or in the worst case, it will be deleted. The server may be damaged.

In this case, change the authority with the su option or check the operation in the verification environment.

About wildcards

When the log file is specified by a wildcard, if the rotated log is included, the log file will increase in a mouse-like manner.

Wildcards should have an extension like ~/*. Log instead of~/*so that logs and archives can be distinguished.

About copytruncate

The copytruncate option has a slight time lag between copying and emptying, and there is a risk of losing logs during that time.

It is best not to use it unless you have a specific reason.

About date format

If the log file name is in date format, logrotate will sort by file name to determine which file is the oldest during rotation, so the date format will be % d-% m-% Y. If you do, you may delete new logs.

Make sure to order by date, such as -% Y-% m-% d. Well, if you're Japanese, no one would reverse it.

About hourly

logrotate is set to run in cron.daily by default. If it is left as it is, even if hourly is specified, it will be rotated daily.

If you want to rotate with hourly, make sure that logrotate is executed with cron.hourly.

Moved logrotate cron settings from daily to hourly.

# mv /etc/cron.daily/logrotate /etc/cron.hourly/

reference

https://hackers-high.com/linux/man-jp-logrotate/#nocopy https://isleofhoso.com/linux-logrotate-size/ https://www2.filewo.net/wordpress/2013/03/31/logrotate%E3%82%B9%E3%82%AF%E3%83%AA%E3%83%97%E3%83%88%E3%81%AE%E8%AA%BF%E6%9F%BB/ https://qiita.com/hdtkkj/items/85d8d3bdfc45ffeaba89 https://mogu2itachi.hatenablog.com/entry/2020/01/26/145929 http://1000k.github.io/2010/05/20/usage-of-logrotate/

Recommended Posts

Summary of logrotate software logrotate
Numerical summary of data
Summary of pyenv usage
Summary of string operations
Summary of Python arguments
Summary of test method
Summary of python file operations
Summary of Python3 list operations
2017.3.6 ~ 3.12 Summary of what we did
Convenient usage summary of Flask
Summary of Linux distribution types
A brief summary of Linux
Summary of Proxy connection settings
Summary of how to use pandas.DataFrame.loc
Summary of basic knowledge of PyPy Part 1
Summary of basic implementation by PyTorch
Summary of scraping relations (selenium, pyautogui)
A brief summary of Python collections
Summary of Stack Overflow Developer Survey 2020
Summary of how to use pyenv-virtualenv
Machine learning ③ Summary of decision tree
Summary of various operations in Tensorflow
A rough summary of OS history
Summary of go json conversion behavior
A Tour of Go Learning Summary
Summary of "nl command Advent Calendar 2020"
Summary of how to use csvkit
Construction of Ceph (Octopus) (preparation software)
[For competition professionals] Summary of doubling
Summary of Python indexes and slices
Summary of multi-process processing of script language
Summary of restrictions by file system
[OpenCV; Python] Summary of findcontours function
[Python] Summary of how to use pandas
2014/02/28 Summary of contents demoed at #ssmjp, part 1
Software emulation of deprecated SWP instruction (CONFIG_SWP_EMULATE)
Summary of frequently used commands of django (beginner)
Summary of methods for automatically determining thresholds
Summary of frequently used commands in matplotlib
[Linux] Summary of middleware version confirmation commands
Summary of study done this week (H29.2.13 ~ 2.24)
Summary of various for statements in Python
[Python] Summary of array generation (initialization) time! !! !!
Summary of library hosting pages by language
Summary of numpy functions I didn't know
[Python2.7] Summary of how to use unittest
Summary of snippets when developing with Go
Summary of study done this week (H29.1.30 ~ 2.5)
Summary of stumbling blocks in installing CaboCha
Summary of petit techniques for Linux commands
Async / Await syntax Summary of available languages
Summary of procedures up to PyPI registration
Summary of SQLAlchemy connection method by DB
Summary of modules and classes in Python-TensorFlow2-
Summary of useful techniques for Python Scrapy
A brief summary of Pinax overview #djangoja
Summary of how to use Python list
[Python2.7] Summary of how to use subprocess
Axis option specification summary of Python "numpy.sum (...)"
[AWS] Summary of CLI installation error countermeasures
Summary of how to write AWS Lambda