[DOCKER] Dynamically display epidemic data using the Grafana Dashboard

Learn how Alibaba Cloud dynamically displayed information about COVID-19 outbreaks using the Grafana Dashboard.

Project preparation

I want to customize the dashboard that displays the epidemic status to clarify the purpose. Since I am currently in Shanghai, I would like to display the number of confirmed cases, suspected cases, recoverers, and deaths nationwide, and at the same time, display the case curve to observe the trend of epidemic development, and each province and district in Shanghai. I want to display the status of.

Grafana is just a tool for viewing data, so you need to get the data source first. Currently, the available plague data sources cannot be integrated directly into Grafana. Therefore, you need to do the following:

--Prepare the ** Grafana program ** on your laptop or Kubernetes cluster. We recommend using a Docker container to run Grafana.

--Install the ** SimpleJson plugin ** that can convert JSON format data to a data source for Grafana.

Data source development

Here, I am developing a data source using Python Bottle. You can also choose Flask (https://flask.palletsprojects.com/en/1.1.x/?spm=a2c65.11461447.0.0.2e7418faOcjQf4), which has the same functionality. I decided to use Bottle because I used it to develop my previous Grafana data source. Not only debug settings, but also Dockerfile for building Docker images and deploy.yaml file for deploying Kubernetes are prepared, and they are ready to use. Developing a Grafana data source using Python is very easy. All you have to do is make sure your data meets the format requirements of SimpleJson. Oz Nahum Tiram's blog on how to use Python to develop data sources for Grafana. Article "[Visualize Almost Anything with Grafana and Python](http://oz123.github.io/writings/2019-06-16-Visualize-almost-anything-with-Grafana-and-Python/index.html?spm=" Please read a2c65.11461447.0.0.2e7418faOcjQf4) ".

There are two types of data available for data source customization:

Time series type data

To view epidemic trends in China, especially Shanghai, in real time, you can view confirmed cases, suspected cases, collections, deaths, and changes in data compared to previous data. You can also draw curves to compare confirmed and suspected cases, and the number of recoveries and deaths.

For confirmed cases, simply return the data by combining the number of confirmed cases nationwide gntotal with the current time stamp. Other metrics can be processed in the same way.

@app.post('/query')
def query():
    print(request.json)
    body = []
    all_data = getDataSync()
    time_stamp = int(round(time.time() * 1000))
    for target in request.json['targets']:
    name = target['target']
    if name == 'gntotal':
        body.append({'target': 'gntotal', 'datapoints': [[all_data['gntotal'], time_stamp]]})
    body = dumps(body)
    return HTTPResponse(body=body, headers={'Content-Type': 'application/json'})

Table type data

You can use a table that maps the number of confirmed cases, suspected cases, collections, and deaths in each province of China, and another table that shows the same data for each district in Shanghai.

You can extract names, confirmed cases, suspected cases, recoverers, and dead from the data and add them to the row rows.

@app.post('/query')
def query():
    print(request.json)
    body = []
    all_data = getDataSync()
    sh_data = getShDataSync()
    if request.json['targets'][0]['type'] == 'table':
        rows = []
        for data in all_data['list']:
            row = [data['name'], data['value'], data['susNum'], data['cureNum'], data['deathNum']]
            rows.append(row)
        sh_rows = []
        for data in sh_data['city']:
            row = [data['name'], data['conNum'], data['susNum'], data['cureNum'], data['deathNum']]
            sh_rows.append(row)
        bodies = {'all': [{
            "columns": [
                {"text": "Province", "type": "name"},
                {"text": "确 诊", " type": "conNum"},
                {"text": "pseudo", " type": "susNum"},
                {"text": "Cure", "type": "cureNum"},
                {"text": "death", "type": "deathNum"}
            ],
            "rows": rows,
            "type": "table"
        }],
            'sh': [{
                "columns": [
                    {"text": "Province", "type": "name"},
                    {"text": "确 诊", " type": "value"},
                    {"text": "pseudo", " type": "susNum"},
                    {"text": "Cure", "type": "cureNum"},
                    {"text": "death", "type": "deathNum"}
                ],
                "rows": sh_rows,
                "type": "table"
            }]}

        series = request.json['targets'][0]['target']
        body = dumps(bodies[series])
  return HTTPResponse(body=body, headers={'Content-Type': 'application/json'})

Panel type selection

Data is generally displayed using four panels. --Single stat is used to display the case number.

--The graph displays a comparison curve.

--Table is used to display the table.

--The text is used for the title of the text.

Data source settings

Now let's set up the data source.

Case number panel

Since there is only one value here, select First.

image.png

Case number graph

These graphs compare the number of confirmed and suspected cases, the number of collections and the number of deaths.

image.png

Data table

image.png

result

The overall finish is fine and is used as the company's trendy dashboard. Here, we are using a relatively small screen Xiaomi TV, so we have expanded the fonts and display panel to enhance the demo.

image.png

building

After packaging your code into a Docker image, you can run your code in any environment or in a Kubernetes cluster. The image has been uploaded to Docker Hub and can be pulled from there for immediate use.

# Dockerfile
FROM python:3.7.3-alpine3.9

LABEL maintainer="[email protected]"

COPY . /app

RUN echo "https://mirrors.aliyun.com/alpine/v3.9/main/" > /etc/apk/repositories \
    && apk update \
    && apk add --no-cache gcc g++ python3-dev python-dev linux-headers libffi-dev openssl-dev make \
    && pip3 install -r /app/requestments.txt -i http://mirrors.aliyun.com/pypi/simple --trusted-host mirrors.aliyun.com

WORKDIR /app

ENTRYPOINT ["uwsgi","--ini","uwsgi.ini"]

Running it

-** Pull the image. ** **

docker pull guoxudongdocker/feiyan-datasource

-** Execute the image. ** **

docker run -d --name datasource -p 8088:3000 guoxudongdocker/feiyan-datasource

-** Add a data source. ** **

Select the SimpleJson data source, click Add and enter the address of the data source.

image.png

-** Import dashboard ** Click on the ʻUpload.json file and select wuhan2020-grafana / dashboard.json`. image.png

-(Optional) Use Kubernetes for deployment.

kubectl apply -f deploy.yaml

Overview

As of mid-February, when I wrote this, the number of confirmed cases in China is still increasing rapidly, but fortunately the growth rate of suspected cases is beginning to decline. It can be seen that the rate of increase in the number of confirmed cases is steadily increasing, and the number of collections is also increasing. Compared to other regions, the number of cases in Shanghai has not increased significantly, even though many people have returned to work. As part of this, strict crackdowns on residential areas are beginning to take effect. Currently, only one person has died in Shanghai, as it recorded China's first recovery. In general, as long as we pay attention to prevention and stay at home, there is no doubt that we will overcome the epidemic and overcome this difficult time.

The JSON file for importing dashboards and the YMAL file for deploying Kubernetes are on GitHub. The address of the project is here. https://github.com/sunny0826/wuhan2020-grafana

While continuing to fight the coronaviruses that are occurring around the world, Alibaba Cloud wants to do its part and do everything it can to help others in the fight against the coronavirus. I am. For information on how to support your business continuity, see https://www.alibabacloud.com/campaign/fight-coronavirus-covid-19 Please check with -covid-19).

Recommended Posts

Dynamically display epidemic data using the Grafana Dashboard
Let's display the map using Basemap
Execute raw SQL using python data source with redash and display the result
Check the status of your data using pandas_profiling
Scraping the winning data of Numbers using Docker
Display the image after Data Augmentation with Pytorch
Display the edge
Big data analysis using the data flow control framework Luigi
I tried clustering ECG data using the K-Shape method
Write data to KINTONE using the Python requests module
I tried using the API of the salmon data project