python pandas study recent summary

Aggregation by element

I tried various things, I calmed down like this.

for sp in set(data['element']):
	df_op = data[data['element'] == sp]
	com_plot(df_op,sp)  #Move to the function to aggregate.

From one file to multiple files to be aggregated

I had selected the file name,

import glob

print("Start reading transaction data")
csv_name	=	r"Folder with input data\*.csv"
csv_files	=	glob.glob(csv_name)
list = []

for f in csv_files:
    list.append(pd.read_csv(f, index_col=0).drop(['Payment Number','Refund Number'], axis=1))
df = pd.concat(list)

Since I was able to select multiple, let me enter the date and change the way of movement

Change the processing content by entering any of "YYMMDD", "YYMM", and "all".

def main():
	print("Specify the aggregation date "YYMMDD" or "YYMM" or "all"")
	print("Change the acquired contents in the format of the aggregated date")
	day = input("Specify the aggregation date-> ")
	print("")
	
	if (len(day) == 6):
		print("Date and time data aggregation")
		df = day_data(day)
		data_plot(df,day,True)
		top_20(df,day)
	elif (len(day) == 4):
		df = o_mon_data(day)
		data_plot(df,day,True)
		top_20(df,day)
	elif (day == all):
		df = month_data()
		data_plot(df,day,False)
	else:
		print("Redo")
		main()

if __name__ == '__main__':
	main()


def day_data(day):
	print("Extract with this →:", day)
	yy = day[:2]
	mm = day[2:4]

Creating top20

Add TOTAL → sort → delete TOTAL In my environment, I get a little angry when deleting TOTAK rows and columns. .. ..

#Descending sort by TOTAL
data.sort_values(['TOTAL'],ascending=False,inplace=True)

#Swap the matrix using the top 20
pvtop20=data[:21]

#TOTAL row and column deletion
drop_idx = ['TOTAL']
pvtop20.drop(drop_idx, inplace=True)

drop_col = ['TOTAL']
pvtop20.drop(drop_col, axis=1, inplace=True)

Additional information linking

Linked from data with the same key. I was supposed to use a dictionary, The function of pandas was amazing. .. ..

print(Daily data + alpha)
data = pd.merge(df, tenbun_meisai, on=["Key"])

Although csv aggregation in perl was the basis of the idea I was able to do what I wanted to do.

I forgot to write tkinter, so I added it


import os, tkinter, tkinter.filedialog, tkinter.messagebox

#Display file selection dialog
root = tkinter.Tk()
root.withdraw()

#I want to extract csv, so the extension is displayed as csv
fTyp = [("","*.csv")]

#Specify the folder where the input file exists
iDir = r'hoge' 

#Input file selection
tkinter.messagebox.showinfo('Data extraction target selection','Select a processing file!')

file = tkinter.filedialog.askopenfilenames(filetypes = fTyp,initialdir = iDir)

list = []
for f in file:
	list.append(pd.read_csv(f, index_col=0).drop(['Payment Number','Refund Number'], axis=1))
df = pd.concat(list)

Recommended Posts

python pandas study recent summary
Python summary
Python study note_002
[Python] Summary of how to use pandas
Python study notes _000
Python tutorial summary
Python study notes_006
Python study note_004
Python study note_003
Python study notes _005
python related summary
python pandas notes
Python study notes_001
Python basics summary
Python study day 1
Summary about Python scraping
Python Django tutorial summary
Pandas Personal Notes Summary
Programming study site summary
Summary about Python3 + OpenCV3
Python function argument summary
Python directory operation summary
Python AI framework summary
Python iteration related summary
Summary of Python arguments
Python application: Pandas # 3: Dataframe
Python Basic --Pandas, Numpy-
Summary of pre-processing practices for Python beginners (Pandas dataframe)
[Python] Summary of table creation method using DataFrame (pandas)
pandas Matplotlib Summary by usage
Read csv with python pandas
Python application: Pandas Part 2: Series
Python3 programming functions personal summary
Summary of python file operations
Summary of Python3 list operations
[Python] Convert list to Pandas [Pandas]
What's new in Python 3.10 (Summary)
Python pandas strip header space
Study Python with Google Colaboratory
Study from Python Hour4: Object-oriented ②
Standard input / summary / python, ruby
Study from Python Hour3: Functions
Python class member scope summary
Python web programming article summary
[Python] Change dtype with pandas
Install pandas 0.14 on python3.4 [on Mac]
Python Competitive Programming Site Summary
[WIP] Fluent Python Study Note
Python data type summary memo
Study from Python Hour4: Object-oriented ①
Face detection summary in Python
Basic usage of Pandas Summary
Memorandum @ Python OR Seminar: Pandas
100 Pandas knocks for Python beginners
What's new in Python 3.9 (Summary)
Data analysis using python pandas
The Power of Pandas: Python
Python Crawling & Scraping Chapter 4 Summary
Python hand play (Pandas / DataFrame beginning)
[Python] Loading csv files using pandas
[Python] Operation memo of pandas DataFrame