I know? Data analysis using Python or things you want to use when you want with numpy
Codes that you want to reach when you want them, mainly in Pandas
** I tried to make it like Advent Calender, but I'm sorry I couldn't make it in time **
I will add it soon
Below, give an alias as follows
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
Do not omit columns and rows when displaying
pd.set_option('display.max_columns', 100)
pd.set_option('display.max_rows', 500)
Use describe () for lines other than numbers
DF.desctibe(include='all')
Data in chronological order
I referred to here https://note.nkmk.me/python-pandas-time-series-resample-asfreq/
Create date data
df = pd.DataFrame({'data' : range(0, 30, 2)},
index=pd.date_range(start='2018/1/1', end='2018/1/30', freq='2D'))
Extract data at regular intervals
For example, if you want to extract data every 3 days, use asfreq (). Dates that originally have no data will be None.
df.asfreq(freq='3D')
| before | after |
|---|---|
|
 |
If you want to extract every Wednesday and fill the None part with the previous data, use asfreq (freq ='W-WED', method ='pad').
df.asfreq(freq='W-WED', method='pad')
| before | after |
|---|---|
|
 |
Conversely, to fill with the trailing value, use asfreq (method ='bfill').
df.asfreq(freq='W-WED', method='bfill')
| before | after |
|---|---|
|
 |
Have date data and get totals weekly
Use resample ()
df.resample('W').sum()
| before | after |
|---|---|
|
 |
Data with the same date, total for each date
The data looks like this
df2 = pd.DataFrame({'data': range(17)},
index=pd.date_range('2018-08-01', '2018-08-05', freq='6H'))
Use resample ('D'). Sum () to get the daily sum.
df2.resample('D').sum()
| before | after |
|---|---|
|
 |
For the mean, use resample ('D'). Mean ().
| before | after |
|---|---|
|
 |
Visualization (Sun)
Prepare virtual data from 2018/1/1 to 2018/12/31
np.random.seed(0)
df3 = pd.DataFrame({'data':
np.maximum(0, np.minimum(np.arange(0, -365, -1) + 182, 0) + np.minimum(np.arange(365), 182) + np.floor(np.random.normal(50, 50, 365)))},
index=pd.date_range('2018-01-01', '2018-12-31'))
If you visualize this,
plt.figure(figsize=(18, 6), facecolor='white')
plt.plot(df3.index, df3['data'], marker='o')
plt.xticks(rotation=90)
plt.show()
| graph | data |
|---|---|
 |
 |
Visualization (weekly)
Visualize every Monday
np.random.seed(0)
df3 = pd.DataFrame({'data':
np.maximum(0, np.minimum(np.arange(0, -365, -1) + 182, 0) + np.minimum(np.arange(365), 182) + np.floor(np.random.normal(50, 50, 365)))},
index=pd.date_range('2018-01-01', '2018-12-31'))
_df = df3.asfreq('W-MON')
plt.figure(figsize=(18, 6), facecolor='white')
plt.plot(_df.index, _df['data'], marker='o')
plt.xticks(rotation=90)
plt.show()
| graph | data |
|---|---|
 |
 |
Visualization (monthly)
Visualized at the end of the month
np.random.seed(0)
df3 = pd.DataFrame({'data':
np.maximum(0, np.minimum(np.arange(0, -365, -1) + 182, 0) + np.minimum(np.arange(365), 182) + np.floor(np.random.normal(50, 50, 365)))},
index=pd.date_range('2018-01-01', '2018-12-31'))
_df = df3.asfreq('1M')
plt.figure(figsize=(18, 6), facecolor='white')
plt.plot(_df.index, _df['data'])
plt.xticks(rotation=90)
plt.show()
| graph | data |
|---|---|
 |
 |
asfreq ('1M') is the last day of the month
df3.asfreq('1M')
The first day of the month
np.random.seed(0)
df3 = pd.DataFrame({'data':
np.maximum(0, np.minimum(np.arange(0, -365, -1) + 182, 0) + np.minimum(np.arange(365), 182) + np.floor(np.random.normal(50, 50, 365)))},
index=pd.date_range('2018-01-01', '2018-12-31'))
_df = df3.asfreq('1MS')
plt.figure(figsize=(18, 6), facecolor='white')
plt.plot(_df.index, _df['data'])
plt.xticks(rotation=90)
plt.show()
| graph | data |
|---|---|
 |
 |
asfreq ('1MS') is the first day of the month
df3.asfreq('1MS')
Visualization (1 quarter)
np.random.seed(0)
df3 = pd.DataFrame({'data':
np.maximum(0, np.minimum(np.arange(0, -365, -1) + 182, 0) + np.minimum(np.arange(365), 182) + np.floor(np.random.normal(50, 50, 365)))},
index=pd.date_range('2018-01-01', '2018-12-31'))
_df = df3.asfreq('1QS')
plt.figure(figsize=(18, 6), facecolor='white')
plt.plot(_df.index, _df['data'], marker='o')
plt.xticks(rotation=90)
plt.show()
| graph | data |
|---|---|
 |
 |
Visualization (total for one week)
np.random.seed(0)
df3 = pd.DataFrame({'data':
np.maximum(0, np.minimum(np.arange(0, -365, -1) + 182, 0) + np.minimum(np.arange(365), 182) + np.floor(np.random.normal(50, 50, 365)))},
index=pd.date_range('2018-01-01', '2018-12-31'))
_df = df3.resample('W').sum()
plt.figure(figsize=(18, 6), facecolor='white')
plt.plot(_df.index, _df['data'], marker='o')
plt.xticks(rotation=90)
plt.show()
| graph | data |
|---|---|
 |
 |
Visualization (total for one month)
np.random.seed(0)
df3 = pd.DataFrame({'data':
np.maximum(0, np.minimum(np.arange(0, -365, -1) + 182, 0) + np.minimum(np.arange(365), 182) + np.floor(np.random.normal(50, 50, 365)))},
index=pd.date_range('2018-01-01', '2018-12-31'))
_df = df3.resample('MS').sum()
plt.figure(figsize=(18, 6), facecolor='white')
plt.plot(_df.index, _df['data'], marker='o')
plt.xticks(rotation=90)
plt.show()
| graph | data |
|---|---|
 |
 |
Time series interpolation
Time series interpolation is often used for occupational patterns
This can be done by using asfreq () well.
df = pd.DataFrame({'data' : range(0, 30, 2)},
index=pd.date_range(start='2018/1/1', end='2018/1/30', freq='2D'))
df2 = df.asfreq('1D')
| before | after |
|---|---|
 |
 |
Date → day of the week
Use weekday or dayofweek for datatime Series
0 is Monday and 6 is Sunday
https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DatetimeIndex.weekday.html
np.random.seed(0)
df3 = pd.DataFrame({'data':
np.maximum(0, np.minimum(np.arange(0, -365, -1) + 182, 0) + np.minimum(np.arange(365), 182) + np.floor(np.random.normal(50, 50, 365)))},
index=pd.date_range('2018-01-01', '2018-12-31'))
df3.index.weekday
Int64Index([0, 1, 2, 3, 4, 5, 6, 0, 1, 2,
...
5, 6, 0, 1, 2, 3, 4, 5, 6, 0],
dtype='int64', length=365)
Recommended Posts