As the title says, I will write a memorandum of notes when inputting from CSV with Python and outputting to json to make it an exe. The implementation example is a completely personal tool, so I'll just give you an overview. The important thing is the error message and the remedy.
First, load csv. Prepare the data in csv called ʻinput.csv` and read it.
import csv
csv_file = open("./input.csv", "r")
c = csv.DictReader(csv_file)
Suddenly, when I try to look at the contents of csv here, I get the following error (probably).
Exception has occurred: UnicodeDecodeError
'cp932' codec can't decode byte 0xef in position 0: illegal multibyte sequence
File "C:\Users\hoge\fuga.py", line 19, in <module>
for row in c:
You will be scolded for not being able to decode. This is because python tries to make the ʻutf-8file
shift-jis`. It needs to be modified as follows.
csv_file = open("input.csv", "r",encoding="utf-8-sig")
The part that was the most annoying this time.
#Excerpt of only parameters from csv
params = []
for row in c:
points.append(row)
n = len(params)
#Read json and secure the parameter frame
import json
json_open = open('sample.json', 'r')
j = json.load(json_open)
for i in range(n-1):
j["Movements"].append(j["Movements"][0])
I did something awkward in the bottom row, which was the source of the trouble, but sample.json
has a data template, which is extended to the number of parameter lines n
. Therefore, I took the method of copying the template n-1 times. As a result, n
frames have been successfully created in the Movements
key of the json data j
, so after that, we will just turn the for
statement and insert the data.
…… But after I finished putting in the data, I was troubled by the mysterious result that ** for some reason all the rows have the same data **.
After a bit of debugging, I found that every input rewrites every row, which means that ** every row references the same data **. Even if the apparent data is in the n
line, it only refers to one data, and if you modify any one of them, all the data will be affected.
Why did this happen?
For types like ʻintand
str, even if you copy a variable with
= , it will be considered as" another variable "and assigned an ID when you make any changes, but
list If you edit the variable after copying the variable with =
(actually, it could not be copied), the original data will be edited. I knew it as knowledge, but so was dict
. The only solution is to copy the variables using a technique called hard copy.
#Excerpt of only parameters from csv
params = []
for row in c:
points.append(row)
n = len(params)
#Read json and secure the parameter frame
import json
import copy
json_open = open('sample.json', 'r')
j = json.load(json_open)
for i in range(n-1):
new_j = copy.deepcopy(j["Movements"][0])
j["Movements"].append(new_j)
In this case, it had to be deepcopy
(deep copy) instead of justcopy
(shallow copy).
There is a module called pyinstaller
that makes it an exe so that even people who do not have Python can run it. When I tried to pip
this, I couldn't install it due to an error, and if I went around, I should downgrade pip (20
→ 18
), so I succeeded. When I tried to reproduce it now, for some reason even the latest pip succeeded, so the error cannot be reproduced. I'm sorry.
So I made it into an exe using pyinstaller
safely, but even if I execute it, it ends immediately. It doesn't even spit out the first print
that I put in for debugging, so it seems that there is a problem at the time of ʻimport`. So, I will hunt witches one by one for the following modules that I actually put in.
import json
import pprint
import csv
import cmath
import math
import numpy as np
import copy
import os
As a result, it was caused by numpy
. Fortunately, I didn't use the part that needed numpy in the program (I used np.pi
, but I don't need it anymore), so I just deleted the module. If you have to exe a program that uses numpy to calculate rigorously, think again ...
I'm sorry that there are many contents that "I managed to do it, but I didn't understand", but I will forget more and more if I do not accumulate such things, so I wrote it for myself. I've been afraid to skip this kind of thing until now, so from now on I'll actively write program records like this diary.
It's a completely personal tool, so I don't think it's useful to anyone, but for the time being.
import json
import pprint
import csv
import cmath
import math
import copy
import os
json_open = open('./sample.json', 'r')
j = json.load(json_open)
print("sample.loaded json")
csv_file = open("./input.csv", "r",encoding="utf-8-sig")
c = csv.DictReader(csv_file)
points = []
for row in c:
points.append(row)
n = len(points)
for i in range(n-1):
new_j = copy.deepcopy(j["Movements"][0])
j["Movements"].append(new_j)
for i,p in enumerate(points):
x = float(p['X'])
y = float(p['Y'])
z = float(p['Z'])
duration = float(p['Duration'])
j["Movements"][i]["Duration"] = duration
j["Movements"][i]["StartPos"]["x"] = x
j["Movements"][i]["StartPos"]["y"] = y
j["Movements"][i]["StartPos"]["z"] = z
j["Movements"][i-1]["EndPos"]["x"] = x
j["Movements"][i-1]["EndPos"]["y"] = y
j["Movements"][i-1]["EndPos"]["z"] = z
y -= 1.5
c = complex(-x,-z)
rad = cmath.phase(c)
deg = -math.degrees(rad)+90
#print(deg)
j["Movements"][i]["StartRot"]["y"] = deg
j["Movements"][i-1]["EndRot"]["y"] = deg
c2 = complex(abs(c),y)
rad2 = cmath.phase(c2)
deg2 = math.degrees(rad2)
#print(-deg2)
j["Movements"][i]["StartRot"]["x"] = deg2
j["Movements"][i-1]["EndRot"]["x"] = deg2
pprint.pprint(j)
with open('output.json', 'w') as f:
json.dump(j, f, indent=4)
print("Completed the work")
end = input()
Recommended Posts