Remove headings from multiple format CSV files with python

Introduction

--I want to delete all the heading lines of a large number of csv files ――I'm using Jupyter Notebook

Advance preparation:

--This file is placed in the current directory --Put the csv file you want to header process in the Machine11_trd folder in the current directory.

import csv, os

#Create the path of the folder you want to process(1)Reference
csv_folder_path = os.path.join(".","InputFolder")

#Create the path of the output destination folder(2)Destination
headerRemoved_path = os.path.join(csv_folder_path,"headerRemoved")

#Create a new folder to contain the header-deleted files.
os.makedirs(headerRemoved_path, exist_ok=True)

#Loop all files in the specified folder under the current directory
#Extract the file name list in the specified folder
for csv_filename in os.listdir(csv_folder_path):
    if not csv_filename.endswith(".csv"):
        continue #Skip if it is not a csv file
    #print("Heading is being deleted"+ csv_filename + "...")
    
    #Read line by line from CSV file(Skip the first line)
    csv_rows=[]
    csv_file_obj = open(os.path.join(csv_folder_path, csv_filename))
    reader_obj = csv.reader(csv_file_obj)
    
    for row in reader_obj:
        #Do not add on the first or second line,Others will be added(3)Header condition
        if reader_obj.line_num == 1 or reader_obj.line_num == 2:
            continue #Skip the target line
        csv_rows.append(row)
    csv_file_obj.close()
    
    #Export CSV file to specified folder
    csv_file_obj = open(os.path.join(headerRemoved_path, csv_filename),"w",newline="")
    csv_writer = csv.writer(csv_file_obj)
    
    for row in csv_rows:
        csv_writer.writerow(row)
    csv_file_obj.close() 

What you are doing

--Skip the first and second rows and save all rows in csv_rows [](change as needed) --Create a new CSV file in the headerRemove folder created in the current directory and write it out. --Do these with all CSVs in the target folder.

Consideration

--At first, I tried to do it with datafram, but I couldn't read the ones with different numbers of columns well, so I read the list line by line and converted the list to dataframe type.

Recommended Posts

Remove headings from multiple format CSV files with python
Handle Excel CSV files with Python
Decrypt files encrypted with openssl from python with openssl
Csv output from Google search with [Python]! 【Easy】
Generate an insert statement from CSV with Python.
Convert multiple proto files at once with python
Csv tinkering with python
[Python] Reading CSV files
Combine multiple csv files into one csv file with python (assuming only one line of header)
Read csv with python pandas
Sorting image files with Python (2)
Sort huge files with python
Sorting image files with Python (3)
Sorting image files with Python
Integrate PDF files with Python
Decompress multiple compressed files (Python)
Format json with Vim (with python)
With skype, notify with skype from python!
Download csv file with python
[Python] Send gmail with python: Send one by one with multiple image files attached
Notes on importing data from MySQL or CSV with Python
[R] [Python] Memo to read multiple csv files in multiple zip files
Create Heroku, Flask, Python, Nyanko bulletin boards with "csv files"
Transpose CSV files in Python Part 1
[Python] Loading csv files using pandas
Call C from Python with DragonFFI
Make apache log csv with python
Recursively unzip zip files with python
Using Rstan from Python with PypeR
[Python] Write to csv file with Python
Manipulating EAGLE .brd files with Python
Create folders from '01' to '12' with python
Handle csv files with Django (django-import-export)
How to import CSV and TSV files into SQLite with Python
Output to csv file with Python
Load multiple JavaScript files with PyWebView
[Python] POST wav files with requests [POST]
Read and use Python files from Python
Decrypt files encrypted with OpenSSL with Python 3
Post multiple Twitter images with python
Run Aprili from Python with Orange
Animate multiple still images with Python
Read files in parallel with Python
Call python from nim with Nimpy
Reading and writing CSV with Python
Multiple integrals with Python and Sympy
Read fbx from python with cinema4d
[Python] Creating multiple windows with Tkinter
Extract database tables with CSV [ODBC connection from R and python]
[Python] Format when to_csv with pandas
Extract strings from files in Python
[Data science basics] I tried saving from csv to mysql with python
Prepare an environment to touch grib2 format files with python (Docker edition)
Preprocessing with Python. Convert Nico Nico Douga tag search results to CSV format
Collecting information from Twitter with Python (Twitter API)
[AWS] Using ini files with Lambda [Python]
Receive textual data from mysql with python
Get html from element with Python selenium
[Note] Get data from PostgreSQL with Python
Create wordcloud from your tweet with python3
Read CSV file with python (Download & parse CSV file)