How to calculate the sum or average of time series csv data in an instant

TL;DR

Thing you want to do

input.csv


#date(Up to minutes),USD rate,JPY rate,EUR rate
#There are 144 rows of data per day every 10 minutes
# 2020/2/24 18:00 ~ 2020/6/11 23:10
yyyy/mm/dd HH:MM,NN.N(data),NN.N,NN.N
yyyy/mm/dd HH:MM,NN.N(data),NN.N,NN.N
yyyy/mm/dd HH:MM,NN.N(data),NN.N,NN.N

Trying to reinvent the wheel

First, write the code and ask (try to ask)

  1. Open the file
  2. Consider the first column as a date and time and convert it to date type
  3. Accumulate values until the boundary between the same month and day is crossed
  4. Divide the accumulated data by the number of accumulated data to calculate the average
  5. File output

Elements that seem to be troublesome

Find the invented wheel

csvcalc.py


import pandas as pd
df = pd.read_csv("input.csv",index_col=0,parse_dates=True)
df.resample('D').mean().to_csv("output.csv")

out.csv


2020/2/25,13.99777778,4.343472222,13.19909722
2020/2/26,13.55291667,3.572986111,7.519791667
2020/2/27,12.31902778,4.954513889,9.513611111
・ ・ ・

The wheels even output the graph

Conclusion

Recommended Posts

How to calculate the sum or average of time series csv data in an instant
How to read time series data in PyTorch
How to generate exponential pulse time series data in python
How to get an overview of your data in Pandas
How to extract features of time series data with PySpark Basics
How to know the internal structure of an object in Python
<Pandas> How to handle time series data in a pivot table
How to handle time series data (implementation)
How to change multiple columns of csv in Pandas (Unixtime-> Japan Time)
Plot CSV of time series data with unixtime value in Python (matplotlib)
How to calculate the volatility of a brand
I want to store the result of% time, %% time, etc. in an object (variable)
Part 1 I wrote an example of the answer to the reference problem of how to write offline in real time in Python
How to get the number of digits in Python
How to measure processing time in Python or Java
How to change python version of Notebook in Watson Studio (or Cloud Pak for Data)
How to find the optimal number of clusters in k-means
Various ways to calculate the similarity between data in python
I made a program in Python that changes the 1-minute data of FX to an arbitrary time frame (1 hour frame, etc.)
How to manipulate the DOM in an iframe with Selenium
How to calculate the amount of calculation learned from ABC134-D
[Question] How to get data of textarea data in real time using Python web framework bottle
Example of how to aggregate a large amount of time series data using Python at a reasonable speed in a small memory environment
What you should not do in the process of time series data analysis (including reflection)
How to plot the distribution of bacterial composition from Qiime2 analysis data in a box plot
How to get rid of the "Tags must be an array of hashes." Error in the qiita api
Put the process to sleep for a certain period of time (seconds) or more in Python
Part 1 I wrote the answer to the reference problem of how to write offline in real time in Python
How to intentionally issue an error in the shell During testing
How to handle multiple versions of CUDA in the same environment
How to determine the existence of a selenium element in Python
I compared the calculation time of the moving average written in Python
How to implement Java code in the background of RedHat (LinuxONE)
How to make a string into an array or an array into a string in Python
How to check the memory size of a variable in Python
[TensorFlow 2] How to check the contents of Tensor in graph mode
How to get the vertex coordinates of a feature in ArcPy
How to create a large amount of test data in MySQL? ??
How to calculate "xx time" in one shot with Python timedelta
How to calculate the autocorrelation coefficient
Differentiation of time series data (discrete)
Time series analysis 3 Preprocessing of time series data
How to read standard input or variable files at the same time like paste command in Python
Power of forecasting methods in time series data analysis Semi-optimization (SARIMA) [Memo]
Instantly illustrate the predominant period in time series data using spectrum analysis
How to get the date and time difference in seconds with python
How to compare if the contents of the objects in scipy.sparse.csr_matrix are the same
Hit the Rakuten Ranking API to save the ranking of any category in CSV
How to store CSV data in Amazon Kinesis Streams with standard input
How to put a line number at the beginning of a CSV file
I want to leave an arbitrary command in the command history of Shell
How to format a list of dictionaries (or instances) well in Python
An example of the answer to the reference question of the study session. In python.
[Python] How to open two or more files at the same time
The story of an error in PyOCR
How to compare time series data-Derivative DTW, DTW-
How to check the version of Django
How to set the server time to Japanese time
How to calculate Use% of df command
[Python] Accelerates loading of time series CSV
How to save the feature point information of an image in a file and use it for matching