About a year ago, I used Django to develop a simple web app.
A story about data collection, AI development, and Web application publishing all done in Python
Get data by scraping from the entered URL → Infer with a trained machine learning model → Return the answer
It is a very simple thing. Click here for the app ↓
Key discrimination AI
After using it for the first time in a long time, I couldn't get the correct result due to the specification change of the scraping site. I thought this was bad and decided to repair it.
I took a peek at the code for the first time in a while, but I'm not sure ... At the time of development, I was aiming to make it work anyway, so the code is dirty and I don't even know how to deploy it ...
So, considering that it will be repaired irregularly in the future, we built an environment with ** Docker ** to make it easier to develop and deploy. The goal is to be able to develop by cloning and docker-compose, and still be able to deploy from a container to heroku.
Also, since this app itself does not use DB and is very simple, obviously there is no need to use Django, so I will rewrite it to flask at this opportunity.
I have published the code on Github. https://github.com/hatena-hanata/KJA_app
Here is the final directory structure.
KJA_APP
├── Dockerfile
├── Procfile
├── app.py
├── docker-compose.yml
├── requirements.txt
└── src
├── modules
│ ├── module.py
│ └── music_class.py
├── static
│ └── model
│ ├── le.pkl
│ └── model.pkl
└── templates
docker-compose.yml
version: '3'
services:
web:
build: .
ports:
- "8080:8080"
volumes:
- .:/home/KJA_APP
tty: true
environment:
TZ: Asia/Tokyo
command: flask run --host 0.0.0.0 --port 8080
Dockerfile
FROM python:3.8.0
USER root
# install google chrome
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
RUN sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list'
RUN apt-get -y update && apt-get install -y google-chrome-stable && apt-get install -yqq unzip
# install chromedriver
RUN wget -O /tmp/chromedriver.zip http://chromedriver.storage.googleapis.com/`curl -sS chromedriver.storage.googleapis.com/LATEST_RELEASE`/chromedriver_linux64.zip
RUN unzip /tmp/chromedriver.zip chromedriver -d /usr/local/bin/
# install heroku cli
RUN curl https://cli-assets.heroku.com/install.sh | sh
# set display port to avoid crash
ENV DISPLAY=:99
# upgrade pip
RUN apt-get update && apt-get install -y \
&& pip install --upgrade pip
# change dir
WORKDIR /home/KJA_APP
# install module
COPY requirements.txt /home
RUN pip install -r /home/requirements.txt
# flask setting
ENV FLASK_APP '/home/KJA_APP/app.py'
ENV FLASK_DEBUG 1
This time, in order to run chrome and scrape with selenium, I installed chrome and chrome driver first. I referred to the dockerfile of here. I also have heroku cli installed for deployment to heroku.
The last two lines, flask setting, are the settings for flask.
In FLASK_APP
, by specifying the py file that has the main function of flask, that file will be executed by the flask run
command.
Setting FLASK_DEBUG
to 1 puts you in debug mode and allows file updates to be reflected in real time.
Now you can develop immediately with git clone → docker-compose. Since we are running the flask run
command during docker-compose, you can see that the app is running by accessinghttp: // localhost: 8080 /
.
You have already registered as a member of Heroku.
gunicorn
to deploy, so don't forget to install it with pip.Procfile
web: gunicorn app:app --log-file -
You can do it from the terminal, but it's easier to understand from the browser, so I'll do it with the browser.
heroku / python
from Settings-> Buildpacks on the page of the created app.
(It seems that python may be added automatically at the time of deployment, but it was a problem because it was not recognized as a python project probably because my directory structure was bad, so it is recommended to add it manually in advance. .)
Also, this time we will scrape using selenium and chrome, so chrome and chrome driver are required. This can also be handled by adding a build pack.
https://qiita.com/nsuhara/items/76ae132734b7e2b352dd#chrome%E3%81%A8driver%E3%81%AE%E8%A8%AD%E5%AE%9AFrom here, we will deploy in a Docker container.
root@a59194fe1893:/home/KJA_APP# heroku login
https://git.heroku.com/ [app name created in 2. above] .git
. (You can also check it in your browser Settings).
The command below means to push the contents of the current branch to the master branch of heroku's repository.root@a59194fe1893:/home/KJA_APP# git push https://git.heroku.com/[Above 2.App name created in].git master
If there are no errors, the deployment is complete.
docker-compose is convenient.
Recommended Posts