A story when a Python user passes a JSON file

Fringe81 Advent Calendar 2019, 12th day article.

This time, I will write a small story that people who have only touched Python (like me) will need in practice. As the title suggests, it is a story when passing data as a JSON file to a front engineer after squeezing the data with Python.

Data preparation

For example, I want to create a screen for introducing myself to new employees! At that time, suppose you are asked to have data in the following form (each data is like a CSV file, so it is assumed that they are individual).

interface Member {
    id: string;
    name: string;
    profile: {
        age: number;
        height: number;
        weight: number:
    };
    likes: {
        name: string;
        reason: string;
    }[];
}[];

I think that newcomers who can only use Python can organize this with pandas and output it. First, prepare the data.

import pandas as pd

#The following data assumes that there is a separate file and it matches the state read by pandas
df1 = pd.DataFrame(
    {
        "id": ["01", "02", "03"],
        "name": ["Mr. A", "Mr. B", "Mr. C"]
    }
)
df2 = pd.DataFrame(
    {
        "id": ["01", "02", "03"],
        "age": [22, 24, 27],
        "height": [177, 171, 167],
        "weight": [64, 57, 62]
    }
)
df3 = pd.DataFrame(
    {
        "id": ["01", "02", "03"],
        "name": ["Sushi / golf", "Baseball / camera", "Ducks, villagers, warriors"],
        "reason": ["Because it's delicious and it's refreshing", "Because it's exciting and I want to keep it in my memory", "Because it's cute, it's kind, it's cool"]
    }
)

The contents of the data look like this.

df1

df2

df3

Data shaping

Next, we will format the data. For the time being, shape it and combine it into one data frame.

#df2 shaping
# "profile"Enter the dictionary summary in the column
df2["profile"] = df2.apply(lambda row: {"age": row["age"], "height": row["height"], "weight": row["weight"]}, axis=1)
df2 = df2[["id", "profile"]]

#df3 shaping
#Divide the text separated by "・" and include it in the list type
# "likes"Enter the dictionary summary in the column
df3["name"] = df3["name"].str.split("・")
df3["reason"] = df3["reason"].str.split("・")
df3["likes"] = df3.apply(lambda row: [{"name": name, "reason": reason} for name, reason in zip(row["name"], row["reason"])], axis=1)
df3 = df3[["id", "likes"]]

#Data combination
df = df1.merge(df2, on="id").merge(df3, on="id")

The contents are like this.

df

Data output

Finally, write the data to a JSON file.

import json

#Create a dict for output
#orient is set so that the nested structure in the list can be saved as it is (arguments here are important)
output = df.to_dict(orient="records")

#Open the save destination file and enter the data.
with open("introduction.json", "w") as f:
    #indent is set so that the contents of the file are easy to see
    # ensure_ascii is set to prevent garbled Japanese characters
    json.dump(output, f, indent=4, ensure_ascii=False)

By the way, the contents should look like this.

output

[{'id': '01',

'name':'Mr. A', 'profile': {'age': 22, 'height': 177, 'weight': 64}, 'likes': [{'name':'sushi','reason':'because it's delicious'}, {'name':'Golf',' reason':'Because I'm crazy'}]}, {'id': '02', 'name':'Mr. B', 'profile': {'age': 24, 'height': 171, 'weight': 57}, 'likes': [{'name':'baseball','reason':'because it gets excited'}, {'name':'camera','reason':'because I want to remember'}]}, {'id': '03', 'name':'C-san', 'profile': {'age': 27, 'height': 167, 'weight': 62}, 'likes': [{'name':'duck','reason':'because it's cute'}, {'name':'villager',' reason':'because it's kind'}, {'name':'Warrior',' reason':'Because it's cool'}]}]

that's all. Apart from this, it may be easier to put together only with the dict type, but this time I wrote a method to convert what was read in the pandas data frame as it is.

Finally

Thank you for reading until the end. Next time is the designer mirinrin! Stay tuned for Fringe81 Advent Calendar 2019.