You can see the transition of points in the J League on the graph!

Overview

Article summary

While introducing the service created this time, I will introduce the development flow and stumbling points.

App overview

――You can see the transition of the points of the J League team in the graph. -You can jump to the service from here. --Scraping from the J League data site, data is collected, distributed as an API, and displayed from the front. スクリーンショット 2021-01-08 19.01.23.png

Reason for creating the app

――Because I wanted to see the overwhelming results of Kawasaki Frontale visually.

Overall composition

The source code for Front is on GitHub.

Diagram

J勝ち点推移サービス構成図.png

Technology used

Creation flow & stumbling point

flow

  1. Scraping J League Data Site with AWS Lambda and storing it in AWS Daynamo DB
  2. Create an API for AWS Daynamo DB data with API Gateway & Lambda
  3. Receive API with Nuxt.js and display it with Chart.js
  4. Deploy on GitHub pages

Flow details

I will explain the flow of development while introducing the sites that I referred to.

1. Scraping with Lambda

Scraping uses requests and beautifulsoup. To use a third party library with Lambda, I referred to the following site

Use Docker image for scraping with Lambda [Python]

To send scraped data to Daynamo DB, you can use the official library called Python Boto3.

·reference Official Document I tried working with DynamoDB from AWS Lambda using Python Boto 3

2. Create API with Lambda + API Gateway

Create a Lambda function and API Gateway that fetches data from DaynamoDB. It's pretty easy. It was difficult to sort and get the data from DaynamoDB, so I decided to sort again with Lambda. Sorting the list of dictionaries is also easy with the library.

·reference API Gateway + Lambda + DynamoDB Sort the list of Python dictionaries

3.Nuxt.js ✖️ Chart.js I referred to the following to handle chart.js in Nuxt. There is an official sample to handle the API with chart.js, so refer to it! ·reference

Easy graph creation using vue-chart.js with Nuxt.js Official

Impressions

Write your impressions lazily.

――After all Lambda is fun and easy to develop and convenient ~ ――It's nice to be able to easily display graphs with Chart.js! ――I want to improve JS power!

Future outlook

I will list what I want to do in the future.

--Strengthening security (such as inserting an API key) --API test --API documentation --UI improvements

Recommended Posts

You can see the transition of points in the J League on the graph!
Introducing sites where you can see trends in the 2019 framework
Display the graph of tensorBoard on jupyter
You can call the method of the class directly in Python3 without adding @staticmethod
You can see it! Comparison of optimization methods (2020)
Visualize the timeline of the number of issues on GitHub assigned to you in Python
[Mathematics] If you understand the meaning of "inner product" graphically, you can see various things.
Omit the decimal point of the graph scale in matplotlib
Looking back on the transition of the Qiita Advent calendar
Difference in results depending on the argument of multiprocess.Process
I want to see the graph in 3D! I can make such a dream come true.
A note on the default behavior of collate_fn in PyTorch
Survey on the use of machine learning in real services
Analyzing data on the number of corona patients in Japan
Have the equation graph of the linear function drawn in Python
Count the number of characters in the text on the clipboard on mac
[Python] Calculate the angle consisting of three points on the coordinates
Connected components of the graph
Can you delete the file?
Read the coordinates of the plot on the graph with Python-matplotlib (super beginner)
Verification of the spread of hoaxes in the "State of Emergency Declaration on April 1"
[TensorFlow 2] How to check the contents of Tensor in graph mode
Find the rank of a matrix in the XOR world (rank of a matrix on F2)
Get the number of readers of a treatise on Mendeley in Python
Check the type of the variable you are using
How to get the notebook name you are currently using in Google Colab
If you think that the person you put in with pip doesn't work → Maybe you are using python3?
You can see the transition of points in the J League on the graph!
About the development environment you are using
Check the status of your data using pandas_profiling
Let's get notified of the weather in your favorite area from yahoo weather on LINE!