Do you handle ** multiple files ** when analyzing business data?
For example, consider the case where 12 files are prepared with ** 1 year's worth of data ** and ** (2019_01.csv, 2019_02.csv, .. .., 2019_12.csv) **. ..
It's a hassle to read and combine one by one. ..
Here's some useful code. (** This time, it is assumed that there are multiple files in the current directory. ** If they are in other directories, apply the specified directory to each argument.)
import os
#Get current directory
current_dir = os.getcwd()
current_dir
#You can get the current directory in the following form
>>> /Users/user_name/Date_science
#Get the folder inside the current directory
os.listdir(current_dir)
#You can get all the files in the current directory in the following form
>>>["2019_01.csv","2019_02.csv","2019_03.csv","2019_04.csv",....,"2019_12.csv"]
tbl_order_file = os.path.join(current_dir,"2019_*.csv")
tbl_order_file
#"Wildcard" in the part of the file name corresponding to "~ month"*By specifying ", only the specified file can be displayed.
#See 3 for details.
>>>/Users/user_name/Date_science/2019_*.csv
#In the glob module, you can get the file pathname that matches the pattern specified in the argument.
import glob
tbl_order_files = glob.glob(tbl_order_file)
tbl_order_files
#I was able to collectively operate the specified files in the following form.
>>>['/Users/user_name/Date_science/2019_01.csv',
'/Users/user_name/Date_science/2019_02.csv',
'/Users/user_name/Date_science/2019_03.csv',
..................,
..................,
'/Users/user_name/Date_science/2019_12.csv']
all_df = pd.DataFrame()
for file in tbl_order_files:
all_df = pd.read_csv(file)
print(f"{file}:{len(order_data)}")
all_df = pd.concat([all_df, order_data],ignore_index = True)
#Data frames can be combined vertically while outputting "which file" and "number of data" in the following form.
>>>
2019_01.csv:233301
2019_02.csv:233260
2019_03.csv:241139
.................,
.................,
2019_12.csv:241135
import os
import glob
current_dir = os.getcwd()
tbl_order_file = os.path.join(current_dir,"2019_*.csv")
tbl_order_files = glob.glob(tbl_order_file)
all_df = pd.DataFrame()
for file in tbl_order_files:
all_df = pd.read_csv(file)
print(f"{file}:{len(order_data)}")
all_df = pd.concat([all_df, order_data],ignore_index = True)