Aggregate the number of hits per second for one day from the web server log with Python

Reason for making

I wanted to know the number of accesses per second for one day, but the seconds that were not accessed with "uniq -c" etc. were missing, so I decided to try Nantoka.

hits.py


#!/usr/bin/env python
import sys

def main():
	
	dict = {}
	
	#init dict
	for hour in range(24):
		for min in range(60):
			for sec in range(60):
				hhmmdd = '%02d:%02d:%02d' % (hour, min, sec,)
				dict[hhmmdd] = 0
	
	
	#read stdin
	for line in sys.stdin:
		text = line.rstrip('\n')
		dict[text] = dict[text] + 1
	
	
	#print hit/s
	for hour in range(24):
		for min in range(60):
			for sec in range(60):
				hhmmdd = '%02d:%02d:%02d' % (hour, min, sec,)
				print '%s %d' % (hhmmdd, dict[hhmmdd])
	
	
if __name__ == '__main__':
	main()

How to use

Pass the hour, minute, and second portion of the web server log

$ cat /var/log/httpd/access_log | cut -d" " -f4 | cut -d":" -f2- | python hits.py

Chat

When I was organizing SSDs, I found a fake script that I made a little while ago, so I put it on Qiita. I wanted to know the number of accesses per second from the Apache log, so I used the "uniq -c" command to add up, but when I thought about it carefully, the seconds without access were missing instead of "0". It's quite natural ... (I started to notice that I'm going to do 86400, even though I counted the number of accesses per second for one day)

So, I made the following script appropriately. It's pretty rough, but I remembered that I was able to do it.

Recommended Posts

Aggregate the number of hits per second for one day from the web server log with Python
The second night of the loop with for
Calculate the total number of combinations with python
Let Code Day10 Starting from Zero "1431. Kids With the Greatest Number of Candies"
Align the number of samples between classes of data for machine learning with Python
A python script that gets the number of jobs for a specified condition from indeed.com
Learn Nim with Python (from the beginning of the year).
Build API server for checking the operation of front implementation with python3 and Flask
Second half of the first day of studying Python Try hitting the Twitter API with Bottle
How to know the number of GPUs from python ~ Notes on using multiprocessing with pytorch ~
[Homology] Count the number of holes in data with Python
How to change the log level of Azure SDK for Python
March 14th is Pi Day. The story of calculating pi with python
[python, ruby] fetch the contents of a web page with selenium-webdriver
Studying web scraping for the purpose of extracting data from Filmarks # 2
The story of making a standard driver for db with python.
Seaborn basics for beginners ① Aggregate graph of the number of data (Countplot)
From the introduction of JUMAN ++ to morphological analysis of Japanese with Python
Existence from the viewpoint of Python
[Completed version] Try to find out the number of residents in the town from the address list with Python
Get the number of searches with a regular expression. SeleniumBasic VBA Python
Try using the Python web framework Django (1)-From installation to server startup
Hit a method of a class instance with the Python Bottle Web API
Get the number of articles accessed and likes with Qiita API + Python
I measured the speed of list comprehension, for and while with python2.7.
Get the key for the second layer migration of JSON data in python
The story of migrating from home server (MariaDB + Java) to AWS (DynamoDB + Python + PHP) with reduced monthly cost
WEB scraping with Python (for personal notes)
Web server for browser testing with Mocha
Learning notes from the beginning of Python 1
Check the existence of the file with python
Download files on the web with Python
The third night of the loop with for
Pandas of the beginner, by the beginner, for the beginner [Python]
Learning notes from the beginning of Python 2
Count the number of characters with echo
[For beginners] Try web scraping with Python
Data acquisition from analytics API with Google API Client for python Part 2 Web application
[Python] Number of integer sequences of length n for which the sum is m
Get one day of the week from Zeller's joint ceremony ~ Incidentally, the perpetual calendar ~
Deep Learning from scratch The theory and implementation of deep learning learned with Python Chapter 3
SSH login to the target server from Windows with a click of a shortcut
Note: How to get the last day of the month with python (added the first day of the month)
[Introduction to Python] How to get the index of data with a for statement