CRLF becomes LF when reading a Python file

LF, CR, CRLF are mixed in the received text file, When you need to replace CRLF with CR.

I will leave it as a memorandum because it did not come out even if I googled other than.

Python3.7

Case study

CRLF exists at the end of the second line (^ M is CR). I want to replace this with another character.

$ cat crlf.txt
1	2	3
4	5	6^M
7	8	9

(×) Open as a text file

If you open it as a normal text file in Python, that…? Will it be replaced?


with open('crlf.test') as f:
    for line in f:
        line = line.replace('\r\n','XXX') #Replace CRLF
        print(line)

# 1       2       3
#
# 4       5       6
#
# 7       8       9

Apparently Python kindly removes the CR. Go and C # running on CentOS were the same, so it seems that it depends on the OS environment.

(〇) Open as a binary file

If you open it in binary mode, it won't be converted without permission. I thought, I tried it. Since the read data is a byte string, the step of decoding and converting it to a character string is necessary. Since it is binary, line breaks can only be identified after character string conversion ... I thought, it's easy because it recognizes the binary LF as a line break and returns one line.


with open('crlf.test','rb') as f:
    for line in f:
        line = line.decode().replace('\r\n','XXX')
        print(line)

# 1       2       3
#
# 4       5       6XXX
# 7       8       9

Since CRLF is now XXX, there is no newline.

What happens if only CR is replaced?


with open('crlf.test','rb') as f:
    for line in f:
        line = line.decode().replace('\r','XXX')
        print(line)

# 1       2       3
# 
# 4       5       6XXX
# 
# 7       8       9

Only CR is replaced properly, and the LF part of CRLF remains.

Conclusion

When dealing with text files with mixed line feed codes Open a text file in binary mode

Recommended Posts

CRLF becomes LF when reading a Python file
[Note] File reading ~ Python ~
Reading pyc file (Python 3.5.2)
I tried reading a CSV file using Python
python external file reading
A story when a Python user passes a JSON file
When reading a csv file with read_csv of pandas, the first column becomes index
Create a binary file in Python
Python CSV file reading and writing
Precautions when creating a Python generator
When writing a program in Python
Creating a simple PowerPoint file with Python
Precautions when pickling a function in python
Two rules when reading Python (slice notation)
Create a deb file from a python package
[GPS] Create a kml file in Python
Error due to UnicodeDecodeError when reading CSV file with Python [For beginners]
I made a configuration file with Python
Output timing is incorrect when standard (error) output is converted to a file in Python
Error when installing a module with Python pip
How to read a CSV file with Python 2/3
I made a python dictionary file for Neocomplete
Create a GIF file using Pillow in Python
Speaking Japanese with OpenJtalk (reading a text file)
[Python] Split a large Flask file using Blueprint
Read a file containing garbled lines in Python
Speaking Japanese with gTTS (reading a text file)
How to create a JSON file in Python
Python variadic memorandum when inheriting a defined class
Python Note: When assigning a value to a string
A memo when creating a python environment with miniconda
Export Python3 version OpenCV KeyPoint to a file
Run a Python file from html using Django
Create a Photoshop format file (.psd) with python
Trap trapped when running a Python Windows executable
Create a MIDI file in Python using pretty_midi
Read line by line from a file with Python
I want to write to a file with Python
Open a file dialog with a python GUI (tkinter.filedialog)
Use pydantic when reading environment variables in Python
Script python file
Python file processing
Writing notes when running a normal python executable (file containing argparse) with Jupyter notebook
[Grasshopper] When creating a data tree on Python script
Try creating a compressed file using Python and zlib
I tried running python etc. from a bat file
A memorandum when writing experimental code ~ Logging in python
What to do when gdal_merge creates a huge file
A memorandum to run a python script in a bat file
I want to randomly sample a file in Python
Problems when creating a csv-json conversion tool with python
Things to note when initializing a list in Python
What's in that variable (when running a Python script)
Use communicate () when receiving output in a Python subprocess
I tried reading data from a file using Node.js.
Python script to create a JSON file from a CSV file
Run a Python file with relative import in PyCharm
[Python] Create a Tkinter program distribution file with cx_Freeze
Create a 2d CAD file ".dxf" with python [ezdxf]
[Python] Start a batch file from Python and pass variables.