Overview

Describes the procedure for creating a graph that displays an image of the target data with a mouse hover using Dash, a data visualization library. Dash allows you to create such dynamic graphs with only python and very little code. The created graph looks like the following.

background

I think it is common to use T-SNE or UMAP to reduce the dimensions of high-dimensional features extracted from image data and create a graph as a scatter plot to check the distribution of the data. At that time, I wanted to easily create a dynamic graph so that I could check the image of the data I was interested in.

environment

It is done by Google Colab, but it can also be done by local Jupyter. The graph is displayed in Jupyter, but it can be started as a single application by changing a few lines. I have attached the code for launching the application in Supplement at the bottom of this article.

procedure

Install the library
Import the library
Define auxiliary function
Create a graph
Display with Dash

1. Install the library

Install dash and jupyter_dash.

!pip install dash
!pip install jupyter_dash

2. Import the library

Import the library to be used.

from jupyter_dash import JupyterDash 
import dash_core_components as dcc 
import dash_html_components as html 
import plotly.express as px
from dash.dependencies import Input, Output
from sklearn.datasets import load_digits
from sklearn.manifold import TSNE
import numpy as np
from PIL import Image
from io import BytesIO
import base64

3. Define auxiliary function

Defines a function that converts a numpy array to base64.

def numpy_to_b64(array):
    # Convert from 0-16 to 0-255
    array = np.uint8(255 - 255/16 * array)

    im_pil = Image.fromarray(array)
    buff = BytesIO()
    im_pil.save(buff, format='png')
    im_b64 = base64.b64encode(buff.getvalue()).decode('utf-8')
    return im_b64

4. Create a graph

MNist data is reduced to two dimensions with T-SNE. Create a graph that displays the results as a scatter plot.

digits = load_digits()
tsne = TSNE(n_components=2, random_state=0)
projections = tsne.fit_transform(digits.data)
fig = px.scatter(
    projections, x=0, y=1,
    color=digits.target
)

5. Display with Dash

Define a Callback to display the image with the mouse hover.

app = JupyterDash(__name__)

app.layout = html.Div([
                       html.Div(id="output"),
                       dcc.Graph(id="fig1", figure=fig)
])

@app.callback(
    Output('output', 'children'),
    [Input('fig1', 'hoverData')])
def display_image(hoverData):
    if hoverData:
        idx = hoverData['points'][0]['pointIndex']
        im_b64 = numpy_to_b64(digits.images[idx])
        value = 'data:image/png;base64,{}'.format(im_b64)
        return html.Img(src=value, height='100px')
    return None

app.run_server(mode="inline")

After executing the above, the graph will be displayed.

Supplement

If you run it as an application, it will be something like app.py. Run the program like python app.py and open http://127.0.0.1:8050/ in your browser to see the graph.

`app.py`


import dash
import dash_core_components as dcc 
import dash_html_components as html 
import plotly.express as px
from dash.dependencies import Input, Output
from sklearn.datasets import load_digits
from sklearn.manifold import TSNE
import numpy as np
from PIL import Image
from io import BytesIO
import base64


def numpy_to_b64(array):
    # Convert from 0-16 to 0-255
    array = np.uint8(255. - 255./16. * array)

    im_pil = Image.fromarray(array)
    buff = BytesIO()
    im_pil.save(buff, format="png")
    im_b64 = base64.b64encode(buff.getvalue()).decode("utf-8")
    return im_b64


digits = load_digits()
tsne = TSNE(n_components=2, random_state=0)
projections = tsne.fit_transform(digits.data)
fig = px.scatter(
    projections, x=0, y=1,
    color=digits.target
)

app = dash.Dash(__name__)

app.layout = html.Div([
                       html.Div(id="output"),
                       dcc.Graph(id="fig1", figure=fig)
])

@app.callback(
    Output('output', 'children'),
    [Input('fig1', 'hoverData')])
def display_image(hoverData):
    if hoverData:
        idx = hoverData['points'][0]['pointIndex']
        im_b64 = numpy_to_b64(digits.images[idx])
        value = 'data:image/png;base64,{}'.format(im_b64)
        return html.Img(src=value, height='100px')
    return None

app.run_server(debug=True)

reference

Use Python visualization library Dash 3 Utilize mouse hover https://qiita.com/OgawaHideyuki/items/b4e0c4f134c94037fd4f

You can use Dash on Jupyter jupyter_dash https://qiita.com/OgawaHideyuki/items/725f4ffd93ffb0d30b6c

Create a graph that displays an image with a mouse hover using the data visualization library Dash