Convert the character code of the file with Python3

Here are two ways to convert the character code of a file with Python3.

This time, I have a CSV file with the character code "shift-jis", and I will write the code to convert it to "utf-8".

Operating environment

Convert character code using codecs

You can read and write files by specifying the character code using codecs.

utf8_test1.py


# -*- coding:utf-8 -*-
import codecs

def main():
    # Shift_JIS file path
    shiftjis_csv_path = './download/shift_jis_data.csv'
    # UTF-8 File path
    utf8_csv_path = './download/utf8_data.csv'

    #Character code utf-Convert to 8 and save
    fin = codecs.open(shiftjis_csv_path, "r", "shift_jis")
    fout_utf = codecs.open(utf8_csv_path, "w", "utf-8")
    for row in fin:
        fout_utf.write(row)
    fin.close()
    fout_utf.close()

if __name__ == '__main__':
    main()

Convert character code using nkf

You can convert the character code by calling the nkf command from Python.

utf8_test2.py


# -*- coding:utf-8 -*-
import subprocess

def main():
    # Shift_JIS file path
    shiftjis_csv_path = './download/shift_jis_data.csv'
    # UTF-8 File path
    utf8_csv_path = './download/utf8_data.csv'

    cmd = "nkf -w %s > %s" % (shiftjis_csv_path, utf8_csv_path)
    subprocess.call(cmd, shell=True)

if __name__ == '__main__':
    main()

However, nkf is not a Python feature, so you need to have nkf installed in advance.

$ brew install nkf

Recommended Posts

Convert the character code of the file with Python3
[Python] Get the character code of the file
Check the existence of the file with python
[Python] [chardet] Automatic detection of character code of file
2.x, 3.x character code of python
Extract the xz file with python
Mass generation of QR code with character display by Python
python character code
Handling of character code of file in IronPython
[Python3] Rewrite the code object of the function
Let's read the RINEX file with Python ①
Read the file by specifying the character code.
[Python3] Understand the basics of file operations
Let's summarize the degree of coupling between modules with Python code
The idea of feeding the config file with a python file instead of yaml
[Python] Read the source code of Bottle Part 2
Prepare the execution environment of Python3 with Docker
2016 The University of Tokyo Mathematics Solved with Python
[Note] Export the html of the site with python.
Calculate the total number of combinations with python
Convert svg file to png / ico with Python
the zen of Python
[Automation] Convert Python code into an exe file
Check the date of the flag duty with Python
Edit the file of the SSH connection destination server on the server with VS Code
I tried to get the authentication code of Qiita API with Python.
About Python3 character code
[Python] Read the source code of Bottle Part 1
Links and memos of Python character code strings
[Python] Summary of S3 file operations with boto3
Code for checking the operation of Python Matplotlib
Static analysis of Python code with GitLab CI
[Python] Determine the type of iris with SVM
[Blender x Python] Think of code with symbols
Get the update date of the Python memo file.
Extract the table of image files with OneDrive & Python
Template of python script to read the contents of the file
Learn Nim with Python (from the beginning of the year).
[Python] Get the official file path of the shortcut file (.lnk)
Try touching the micro: bit with VS Code + Python
I tried to touch the CSV file with Python
Destroy the intermediate expression of the sweep method with Python
Let's break down the basics of TensorFlow Python code
Visualize the range of interpolation and extrapolation with python
Get the return code of the Python script from bat
# Function that returns the character code of a string
Convert the image in .zip to PDF with Python
Calculate the regression coefficient of simple regression analysis with python
Read QR code from image file with Python (Mac)
How to convert JSON file to CSV file with Python Pandas
Summary of the basic flow of machine learning with Python
Get the operation status of JR West with Python
Extract the band information of raster data with python
Calculate the product of matrices with a character expression?
Python with VS Code (Windows 10)
Towards the retirement of Python2
<Python> A quiz to batch convert file names separated by a specific character string as part of the file name
Download the file in Python
Draw netCDF file with python
Explain the code of Tensorflow_in_ROS
Debug Python with VS Code