Python / numpy> Read the data file with the item name line> Use genfromtxt ()

Operating environment


GeForce GTX 1070 (8GB)
ASRock Z170M Pro4S [Intel Z170chipset]
Ubuntu 14.04 LTS desktop amd64
TensorFlow v0.11
cuDNN v5.1 for Linux
CUDA v8.0
Python 2.7.6
IPython 5.1.0 -- An enhanced Interactive Python.
gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4
GNU bash, version 4.3.8(1)-release (x86_64-pc-linux-gnu)

Target file

I want to read the value of a file like the following.

$ head IntField-Y 
x y z |E|^2 Ex.r Ex.i Ey.r Ey.i Ez.r Ez.i
-0.2094395102 -1.466076572 -5.235987756 1.117036835 -0.02405501404 0.01851210121 0.4265089085 0.9619413614 0.0789243352 -0.05143356113
0.2094395102 -1.466076572 -5.235987756 1.117036835 0.02405501406 -0.01851210123 0.4265089085 0.9619413614 0.0789243352 -0.05143356113
-1.047197551 -1.047197551 -5.235987756 1.080829273 -0.03701212691 0.09467516163 0.4467058558 0.9303997369 0.0629824475 -0.0365979255
-0.6283185307 -1.047197551 -5.235987756 0.8888575386 -0.01687903824 -0.0712841231 0.4031262231 0.8443906708 0.08152662539 -0.0365822794
-0.2094395102 -1.047197551 -5.235987756 1.355786123 0.000532696637 -0.07408332312 0.2568592009 1.131470229 0.05970715325 -0.02304404718
0.2094395102 -1.047197551 -5.235987756 1.355786123 -0.0005326966253 0.0740833231 0.256859201 1.131470229 0.05970715325 -0.02304404718
0.6283185307 -1.047197551 -5.235987756 0.8888575386 0.01687903825 0.07128412309 0.4031262231 0.8443906708 0.08152662538 -0.0365822794
1.047197551 -1.047197551 -5.235987756 1.080829273 0.03701212691 -0.09467516164 0.4467058558 0.9303997368 0.06298244747 -0.03659792549
-1.047197551 -0.6283185307 -5.235987756 1.699276704 -0.02108930862 -0.06464253993 0.3399972729 1.255779421 0.04140882396 -0.01893399186

The delimiter is a space, not a comma.

reference

http://www.mwsoft.jp/programming/numpy/csv.html

loadtxt ()> Failure

test_read_170418a.py


import numpy as np

data = np.loadtxt('IntField-Y', delimiter=' ')
print(data[:,0])  # 1st columne
print(data[:,1])  # 2nd columne

run


$ python test_read_170418a.py 
Traceback (most recent call last):
  File "test_read_170418a.py", line 9, in <module>
    data = np.loadtxt('IntField-Y', delimiter=' ')
  File "/home/user/tensorflow-GPU/local/lib/python2.7/site-packages/numpy/lib/npyio.py", line 930, in loadtxt
    items = [conv(val) for (conv, val) in zip(converters, vals)]
  File "/home/user/tensorflow-GPU/local/lib/python2.7/site-packages/numpy/lib/npyio.py", line 659, in floatconv
    return float(x)
ValueError: could not convert string to float: x
[1]+End subl test_read_170418a.py

Item name line(x y z |E|^2 Ex.r Ex.i Ey.r Ey.i Ez.r Ez.i)I get an error.

genfromtxt ()> Success

test_read_170419a.py


import numpy as np

#data = np.loadtxt('IntField-Y', delimiter=' ')
data = np.genfromtxt('IntField-Y', delimiter=' ')
print(data[:,0])  # 1st columne
print(data[:,1])  # 2nd columne

run


$ python test_read_170419a.py 
[        nan -0.20943951  0.20943951 ...,  1.04719755 -0.20943951
  0.20943951]
[        nan -1.46607657 -1.46607657 ...,  1.04719755  1.46607657
  1.46607657]

I was able to read a space-separated data file with a line of item names by using genfromtxt (). The character string in the line of the item name is nan, but there is no problem because it is not treated as data.

Recommended Posts

Python / numpy> Read the data file with the item name line> Use genfromtxt ()
Let's read the RINEX file with Python ①
Read the file line by line in Python
Read a character data file with numpy
[Python] Read the specified line in the file
Read table data in PDF file with Python
Read the file with python and delete the line breaks [Notes on reading the file]
Read line by line from a file with Python
[Implementation example] Read the file line by line with Cython (Python) from the last line
Python --Read data from a numeric data file and find the multiple regression line.
Read json data with python
[Python] Read the csv file and display the figure with matplotlib
[Python] Read command line arguments from file name or stdin
Try to decipher the garbled attachment file name with Python
Extract the xz file with python
Read a file in Python with a relative path from the program
[Python] Read a csv file with a large data size using a generator
Read CSV file with python (Download & parse CSV file)
[Python] Get the variable name with str
[Python] Use string data with scikit-learn SVM
[Automation] Read mail (msg file) with Python
[Python] How to make the file name of the output data unique (use year, month, day, hour, second)
Various ways to read the last line of a csv file in Python
How to read a CSV file with Python 2/3
Read data with python / netCDF> nc.variables [] / Check data size
Read Python csv data with Pandas ⇒ Graph with Matplotlib
Specify the Python executable to use with virtualenv
[Python] How to read excel file with pandas
Use logger with Python for the time being
Read Excel name / cell range with Python VBA
The easiest way to use OpenCV with python
Extract zip with Python (Japanese file name support)
Draw a line / scatter plot on the CSV file (2 columns) with python matplotlib
Convert the character code of the file with Python3
I want to get the file name, line number, and function name in Python 3.4
[python] Read data
I tried searching for files under the folder with Python by file name
Python2 / numpy> Replace only a specific column in a file with column data from another file> numpy.c_
[Python] Create structured array (store heterogeneous data with NumPy)
I tried to touch the CSV file with Python
Read the xml file by referring to the Python tutorial
Read QR code from image file with Python (Mac)
Python program that looks for the same file name
Read json file with Python, format it, and output json
Specify MinGW as the compiler to use with Python
Let's use the open data of "Mamebus" in Python
Extract the band information of raster data with python
Use mecab with Python3
Use DynamoDB with Python
Use Python 3.8 with Anaconda
Use python with docker
Read Python csv file
Data analysis with Python
Python --Read data from a numeric data file to find the covariance matrix, eigenvalues, and eigenvectors
I know? Data analysis using Python or things you want to use when you want with numpy
Try scraping the data of COVID-19 in Tokyo with Python
Why can I use the module by importing with python?
[python] Change the image file name to a serial number
How to switch the configuration file to be read by Python
[python] A note when trying to use numpy with Cython
Read the GRIB2 file of the Japan Meteorological Agency with pygrib