Analytical environment construction with Docker (jupyter notebook + PostgreSQL)

In this article

--Building a data analysis environment using Docker --Checking the connection between python and postgresql with docker-compose

Is doing

About image

For postgresql, we use the official Docker image. The anaconda environment creates a Dockerfile based on your own image.

FROM kyobad/anaconda3-alpine

MAINTAINER K.Kato

RUN conda install -y seaborn psycopg2 networkx\
&& pip install chainer

WORKDIR /root/notebook

CMD ["/bin/sh", "-c", "jupyter notebook --no-browser --port=8888 --ip=0.0.0.0"]

This is an image of installing an analysis environment such as seaborn and psycopg2 based on the anaconda environment.

About compose files

Create a docker-compose configuration file to connect this python environment to postgresql.

docker-compose.yml


version: '2'
services:
    db:
        image:
            postgres
        restart:
            always
        environment:
            POSTGRES_USER:
                pythonist
            POSTGRES_PASSWORD:
                password
            POSTGRES_DB:
                mypyenv
        ports:
            - "5432"
        container_name:
            postgres-db
        volumes:
            - ./postgres-db:/var/lib/postgresql/data
            - ./docker-entrypoint-initdb.d:/docker-entrypoint-initdb.d
    python:
        build: .
        volumes:
            - ./workdir:/root/notebook/
        ports:
            - "8888:8888"
        links:
            - db
        depends_on:
            - db

The user name, password, etc. are set in the db environment. Please change these as appropriate. postgres-db / set in volume mounts postgres data locally and backs it up (is it really desirable to create a dataonly container?) docker-entrypoint-initdb.d is the default setting when starting the container. If you put sql files such as table creation here, they will be done when the container is started. The python volume is a mount for saving the launched notebook locally.

starting method

In the directory where you put these,

docker-compose up

By executing the command, the postgres container and the jupyter notebook container will be launched.

From your browser, you can access localhost: 8888 and access the db with the db name and password written in the compose file.

スクリーンショット 2016-12-26 14.36.13.png

Conclusion

--By using Docker, you can easily build and reconfigure your analysis environment. --Backup & portable by mounting notebook and postgres data locally

Recommended Posts

Analytical environment construction with Docker (jupyter notebook + PostgreSQL)
Virtual environment construction with Docker + Flask (Python) + Jupyter notebook
Pepper-kun remote control environment construction with Docker + IPython Notebook
Easy Jupyter environment construction with Cloud9
[MEMO] [Development environment construction] Jupyter Notebook
Data science environment construction with Docker
Build Jupyter Lab (Python) environment with Docker
How to use jupyter notebook without polluting your environment with Docker
Build Django + NGINX + PostgreSQL development environment with Docker
Environment construction: GCP + Docker
I wanted to use jupyter notebook with docker in pip environment (opticspy)
Try using conda virtual environment with Jupyter Notebook
Python local development environment construction template [Flask / Django / Jupyter with Docker + VS Code]
R environment construction with Jupyter (formerly IPython notebook) (on OS X El Capitan 10.11.3)
[Python] OpenCV environment construction with Docker (cv2.imshow () also works)
From Kafka to KSQL --Easy environment construction with docker
Data analysis environment construction with Python (IPython notebook + Pandas)
EC2 provisioning with Vagrant + Jupyter (IPython Notebook) on Docker
Using Graphviz with Jupyter Notebook
Use pip with Jupyter Notebook
ML environment construction with Miniconda
Docker + Django + React environment construction
Use Cython with Jupyter Notebook
Prepare python3 environment with Docker
Play with Jupyter Notebook (IPython Notebook)
Switch virtual environment with jupyter
python3.8 venv environment jupyter notebook
Easily prepare a Jupyter Notebook environment with Docker (Tensorflow and Graphviz are also available)
Build a Django development environment with Docker! (Docker-compose / Django / postgreSQL / nginx)
Machine learning with Jupyter Notebook in OCI Always Free environment (2019/12/17)
Reflect the virtual environment created with Miniconda in Jupyter notebook
From environment construction to deployment for flask + Heroku with Docker
I want to use a virtual environment with jupyter notebook!
Allow external connections with jupyter notebook
Formatting with autopep8 on Jupyter notebook
Get started with Python! ~ ① Environment construction ~
Until you start Jupyter with Docker
ruby environment construction with aws EC2
Visualize decision trees with jupyter notebook
Make a sound with Jupyter notebook
Build Mysql + Python environment with docker
Build PyPy execution environment with Docker
Use markdown with jupyter notebook (with shortcut)
[Linux] Docker environment construction on Redhat
Add more kernels with Jupyter Notebook
Build IPython Notebook environment with boot2docker
Convenient analysis with Pandas + Jupyter notebook
Automate environment construction with Shell Script
Python3 environment construction with pyenv-virtualenv (CentOS 7.3)
Using Chainer with CentOS7 [Environment construction]
pytorch @ python3.8 environment construction with pipenv
Rebuild Django's development environment with Docker! !! !! !!
[docker] python3.5 + numpy + matplotlib environment construction
Environment construction with pyenv and pyenv-virtualenv
Build a comfortable psychological experiment / analysis environment with PsychoPy + Jupyter Notebook
Postgres environment construction with Docker I struggled a little, so note
Create an arbitrary machine learning environment with GCP + Docker + Jupyter Lab
Build jupyter notebook environment with Visual Studio Code (VS Code) Mac version
Comfortable Jupyter Lab (Python) analysis environment created with Docker + VSCode + Remote Container
[Ubuntu 18.04] Python environment construction with pyenv + pipenv
Use nb extensions with Anaconda's Jupyter notebook