Visualize your pocket money files with the Python web framework Dash

Preface

There is often a need for quick visualization of your files. In such a quick environment, non-programmers can also use the data, and the world of unfounded intuition and guts can be transformed into the world of intuition and guts based on data.

This time, as a familiar example, I created a (pic1) Dash application that can upload the following affiliated pocket money files. The date column contains the date, the variable column contains the expense item, and the value column contains the amount. Unfortunately, I don't have a pocketbook, so I'm using Japanese household survey data.

pic1.png

The final application I created looks like this: You can select a file by clicking the file upload tool, and if it is pocket data with the above three items, a graph will be created and you can also select and draw the item.

Image from Gyazo

You have to write a lot of code to make something like this! You might think that. However, the code length of the application itself is 92 lines, formatted with black.

URL of the actual app: https://okodukai.herokuapp.com/ (It takes a little time to start because it runs for free) File used: https://github.com/mazarimono/okodukai_upload/blob/master/data/kakei_chosa_long.csv

Creation environment

The creation environment is as follows.

Windows10 Python 3.8.3 dash 1.13.4 plotly 4.8.2 pandas 1.0.4

How to make

See the previous article for how to create a Dash application.

https://qiita.com/OgawaHideyuki/items/6df65fbbc688f52eb82c

Dash itself is a package like React's Python wrapper, and one of its features is that it makes it easy to visualize data using a graph package called Plotly.

Combine the components to create a layout and use callbacks to make them work interactively.

Important components in this application

The following two points are important for the pocket money upload app.

--Where to upload the file --Data retention

It leverages the Upload and Store components, respectively.

Upload component

The Upload component is a component for uploading files and is displayed in the application as follows.

pic2.png

The code looks like this:

upload.py


dcc.Upload(
            id="my_okodukai",
            children=html.Div(["Pocket money file", html.A("Please use csv or excel!")]),
            style=upload_style,
        ),

Store component

The Store component does not appear in the layout, but it is used to hold the uploaded data. When the application uploads a file, it is passed to the Store component, which is called to update the display item selection in the dropdown and the graph is updated.

** When the app starts ** pic3.png

** Upload the file ** pic4.png

Callback

Callbacks connect each component and make each work dynamically. In the case of this app, we will utilize the following two callbacks.

--If the data uploaded to the Upload component is a CSV file or an Excel file, it will be a data frame. Take advantage of it to create dropdown options. Then deposit the data in the Store component (1) --Using the data read from the Store component, create the data according to the dropdown selection and draw it on the graph (2)

The callback code of (1) is as follows. prevent_initial_call prevents the callback from being triggered at startup. This will only trigger this callback when the file is uploaded.

@app.callback(
    [
        Output("my_dropdown", "options"),
        Output("my_dropdown", "value"),
        Output("tin_man", "data"),
    ],
    [Input("my_okodukai", "contents")],
    [State("my_okodukai", "filename")],
    prevent_initial_call=True,
)
def update_dropdown(contents, filename):
    df = parse_content(contents, filename)
    options = [{"label": name, "value": name} for name in df["variable"].unique()]
    select_value = df["variable"].unique()[0]
    df_dict = df.to_dict("records")
    return options, [select_value], df_dict

The parse_content function looks like this: Briefly, if the file name ends with .csv, it will be read as a CSV file, and if xls is included, it will be read as an Excel file. We do not support other than utf-8. Also, it seems that there are xls and xlsx in the Excel file, so I made it like this.


def parse_content(contents, filename):
    content_type, content_string = contents.split(",")
    decoded = base64.b64decode(content_string)

    try:
        if filename.endswith(".csv"):
            df = pd.read_csv(io.StringIO(decoded.decode("utf-8")))
        elif "xls" in filename:
            df = pd.read_excel(io.BytesIO(decoded))
    except Exception as e:
        print(e)
        return html.Div(["There was an error reading the file"])

    return df

The callback of (2) is as follows. This also prevents the callback from being triggered when the app starts, and when the dropdown is updated, it calls the data and updates the graph.


@app.callback(
    Output("my_okodukai_graph", "figure"),
    [Input("my_dropdown", "value")],
    [State("tin_man", "data")],
    prevent_initial_call=True,
)
def update_graph(selected_values, data):
    df = pd.DataFrame(data)
    df_selected = df[df["variable"].isin(selected_values)]
    return px.line(df_selected, x="date", y="value", color="variable")

Look at a little Japanese household survey

It may end up like this, so let's take a look at the Japanese household survey. At the moment, the data is up to May.

First, looking at consumer spending, which represents overall consumption, it fell sharply in April and May.

pic5.png

No, when I encouraged the consumption of milk, I saw that it was effective, thank you, but when I looked at how it was, it seemed to be true. I went back to my childhood and drank a lot of milk. Milk with abstinence! !! !!

pic6.png

Well, if you have any interesting trends in the various coronaviruses, I would appreciate it if you could leave them in the comments section. It's a little slow, probably because it's a free frame, but please be patient. I was interested in rice or bread, whichever I was spending money on. I'm definitely American, but my daughters only eat bread. I was wondering why ...

pic7.png

It seems that I was the one who was left behind. Fish and meat have more meat like that, but I understand that feeling because I am a meat pie.

Summary

With the above feeling, you can easily create a tool that immediately visualizes the standard files you have. There are also requests for more support, but I wonder if it is necessary to make cuts in that area from the perspective of data storage.

DX your own life! !! Is it like that?

Application URL: https://okodukai.herokuapp.com/ github: https://github.com/mazarimono/okodukai_upload

The code on github is what app_heroku.py gives to Heroku.

Application code

app.py



import base64
import io

import dash
import dash_core_components as dcc
import dash_html_components as html
import pandas as pd
import plotly.express as px

from dash.dependencies import Input, Output, State

upload_style = {
    "width": "50%",
    "height": "120px",
    "lineHeight": "60px",
    "borderWidth": "1px",
    "borderStyle": "dashed",
    "borderRadius": "5px",
    "textAlign": "center",
    "margin": "10px",
    "margin": "3% auto",
}
external_stylesheets = ["https://codepen.io/chriddyp/pen/bWLwgP.css"]
app = dash.Dash(__name__, external_stylesheets=external_stylesheets)
app.config.suppress_callback_exceptions = True

app.layout = html.Div(
    [
        dcc.Upload(
            id="my_okodukai",
            children=html.Div(["Pocket money file", html.A("Please use csv or excel!")]),
            style=upload_style,
        ),
        dcc.Dropdown(
            id="my_dropdown", multi=True, style={"width": "75%", "margin": "auto"}
        ),
        dcc.Graph(id="my_okodukai_graph"),
        dcc.Store(id="tin_man", storage_type="memory"),
    ]
)


def parse_content(contents, filename):
    content_type, content_string = contents.split(",")
    decoded = base64.b64decode(content_string)

    try:
        if filename.endswith(".csv"):
            df = pd.read_csv(io.StringIO(decoded.decode("utf-8")))
        elif "xls" in filename:
            df = pd.read_excel(io.BytesIO(decoded))
    except Exception as e:
        print(e)
        return html.Div(["There was an error reading the file"])

    return df

@app.callback(
    [
        Output("my_dropdown", "options"),
        Output("my_dropdown", "value"),
        Output("tin_man", "data"),
    ],
    [Input("my_okodukai", "contents")],
    [State("my_okodukai", "filename")],
    prevent_initial_call=True,
)
def update_dropdown(contents, filename):
    df = parse_content(contents, filename)
    options = [{"label": name, "value": name} for name in df["variable"].unique()]
    select_value = df["variable"].unique()[0]
    df_dict = df.to_dict("records")
    return options, [select_value], df_dict

Recommended Posts

Visualize your pocket money files with the Python web framework Dash
Download files on the web with Python
Let's make a web framework with Python! (1)
Let's make a web framework with Python! (2)
[Python] Get the files in a folder with Python
Introduction to Tornado (1): Python web framework started with Tornado
Django python web framework
Parse and visualize JSON (Web application ⑤ with Python + Flask)
Try using the Python web framework Django (2) --Look at setting.py
Visualize the results of decision trees performed with Python scikit-learn
Improve your productivity by processing huge Excel files with Python
The first API to make with python Djnago REST framework
Run the intellisense of your own python library with VScode.
[Part.2] Crawling with Python! Click the web page to move!
[CleanArchitecture with Python] Part2: Frameworks & Drivers Layer: Introducing the Web
Save images on the web to Drive with Python (Colab)
Operate your browser using the Selenium Web Driver Python bindings
Web scraping with python + JupyterLab
Sorting image files with Python (3)
Web API with Python + Falcon
Sorting image files with Python
Integrate PDF files with Python
Reading .txt files with Python
Call the API with python3.
Web scraping beginner with python
Streamline web search with python
Web application with Python + Flask ④
[Python] I tried to visualize the prize money of "ONE PIECE" over 100 million characters with matplotlib.
[Python] A program that finds the maximum number of toys that can be purchased with your money
The first artificial intelligence. Challenge web output with python. ~ Flask introduction
Get started with the Python framework Django on Mac OS X
[python, ruby] fetch the contents of a web page with selenium-webdriver
Visualize the frequency of word occurrences in sentences with Word Cloud. [Python]
Extract the xz file with python
Recursively unzip zip files with python
Web scraping with Python ① (Scraping prior knowledge)
Manipulating EAGLE .brd files with Python
Getting Started with Python Web Applications
Web scraping with Python First step
I tried web scraping with python.
[Python] POST wav files with requests [POST]
Get the weather with Python requests
Get web screen capture with python
Get the weather with Python requests 2
Decrypt files encrypted with OpenSSL with Python 3
Find the Levenshtein Distance with python
Hit the Etherpad-lite API with Python
Install the Python plugin with Netbeans 8.0.2
Visualize python package dependencies with graphviz
Install the data files with setup.py
Handle Excel CSV files with Python
I liked the tweet with python. ..
Master the type with Python [Python 3.9 compatible]
[Python] I tried to visualize the night on the Galactic Railroad with WordCloud!
Created a new corona app in Kyoto with Python's web framework Dash
Try using the Python web framework Django (1)-From installation to server startup
Hit a method of a class instance with the Python Bottle Web API
I tried using "Streamlit" which can do the Web only with Python
Implement a simple application with Python full scratch without using a web framework.