[Note] Build a Python3 environment with Docker in EC2

Overview

Contents

--Build a Python environment on AWS EC2 using Docker --This time, containerize including the package of automatic browser operation with selenium

environment

• OS:AWS Linux2 --Service: AWS (EC2) --Type: t2.micro

Premise

--The EC2 of the above environment is set up from your account

Reference material

-Basics of Docker in Amazon ECS -docker docker-compose command -How to register Docker Image to Docker Hub -How to install docker and docker-compose on AWS EC2 instance and launch a simple web service

Partially extracted from this area and fulfilled.

Required files

• Dockerfile
• docker-compose.yml

Process

--Connect to EC2 instance --Update the packages and package cache installed on your instance

$ sudo yum update -y

--Install the latest Docker Community Edition package

$ sudo amazon-linux-extras install docker

--Next, install Docker Compose with curl.

$ sudo curl -L https://github.com/docker/compose/releases/download/1.21.0/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose

$ sudo chmod +x /usr/local/bin/docker-compose

$ docker-compose --version

Success if version is displayed

--Started Docker service

$ sudo service docker start

--Add ec2-user to docker group and omit sudo to be entered in future commands

$ sudo usermod -a -G docker ec2-user

--Log out once to reflect, and log in to EC2 again --Verify that ec2-user can execute Docker commands without using sudo

$ docker info

2. Register with Dockerhub

--Account registration on Docker Hub - https://hub.docker.com/

--Create a repository

--Log in to Docker Hub on EC2

$ docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
Username:account name
Password:password
Login Succeeded

3. Create a file to create a Docker image

--Required files - Dockerfile - docker-compose.yml

--Create a Dockerfile.

$ touch Dockerfile

--Edit contents with vim

$ vim Dockerfile

--This time, create an image container that contains a package for automatic operation of python3 and the browser.

Dockerfile

FROM python:3
USER root

RUN apt-get update
RUN apt-get -y install locales && \
localedef -f UTF-8 -i ja_JP ja_JP.UTF-8
ENV LANG ja_JP.UTF-8
ENV LANGUAGE ja_JP:ja
ENV LC_ALL ja_JP.UTF-8
ENV TZ JST-9
ENV TERM xterm

#Install various packages
RUN apt-get install -y vim less
RUN pip install --upgrade pip
RUN pip install --upgrade beautifulsoup4
RUN pip install --upgrade chromedriver
RUN pip install --upgrade chromedriver-binary
RUN pip install --upgrade datetime
RUN pip install --upgrade pandas
RUN pip install --upgrade pyyaml
RUN pip install --upgrade requests
RUN pip install --upgrade selenium

--Copy the above: Save as wq

--Next, create docker-compose.yml

touch docker-compose.yml

--This also edits the contents with vim.

vim docker-compose.yml

docker-compose.yml

version: '3'
services:
  python3:
    restart: always
    build: .
    container_name: 'scrayping'
    working_dir: '/root/'
    tty: true
    volumes:
      - ./opt:/root/opt

--Copy the above: Save as wq

3. Create a Docker image

--Create an image based on the contents of docker-compose.yml --At this time, you cannot push to Docker Hub unless you name the account name / image name given on docker hub with "-t". Note

$ docker-compose build -t {dockerhub account name/Image name}

--Check if you could create an image

$ docker images

REPOSITORY                  TAG                 IMAGE ID            CREATED             SIZE
shunsukenashiki/scrayping   latest              ed6df28c956e        About an hour ago   1.15GB

It's OK if it's made like this

4. Create a container based on the Docker image

--Create a container based on the created Docker image --The reason for adding "-d" is that the container will not start in the background without -d.

$ docker run -d --name #{Any container name} -d -it {REPOSITORY name}

--Check if the container has been created and started. If you can create it, you're done

$ docker ps

CONTAINER ID        IMAGE                       COMMAND             CREATED             STATUS              PORTS               NAMES
b42460c71e9c        shunsukenashiki/scrayping   "python3"           5 seconds ago       Up 4 seconds                            scrayping

5. Run the created container

--Execute the created container

$ docker exec -it #{Any container name} bash

--Complete if the specified version is included in the Dockerfile

$ pip3 list

Package         Version
--------------- ---------
beautifulsoup4  4.9.1
certifi         2020.6.20
chardet         3.0.4
chromedriver    2.24.1
DateTime        4.3
idna            2.10
numpy           1.19.1
pandas          1.1.0
pip             20.2.1
python-dateutil 2.8.1
pytz            2020.1
requests        2.24.0
selenium        3.141.0
setuptools      49.2.1
six             1.15.0
soupsieve       2.0.1
urllib3         1.25.10
wheel           0.34.2
zope.interface  5.1.0

Upload to Doekerhub

--Once you can confirm the operation, push it to Dockerhub

$ docker push shunsukenashiki/{dockerhub account name/Image name}:latest

--Complete if it is properly linked to Dockerhub. After that, if you pull the image from here, you can reuse it