Here are some frequently used commands when developing with Python/Django.
I would like to take this opportunity to thank you for solving the problem by borrowing the wisdom of our predecessors, and I am very sorry to say that I will summarize it here as my own memo.
(Production environment)
(Development environment)
Build a WEB server, AP server, and DB server on Amazon EC2 (Elastic Computing Cloud): Amazon Linux 2.
(Diagram)
--Execute the following commands with the SSH connection to the EC2 instance.
Please also refer to [Python/Django] Summary of frequently used commands (4) -Part 1- <Production operation: Amazon EC2 (Amazon Linux 2)>.
Terminal
$ vi ~/.bash_profile
You can edit it by pressing the "i" key. After editing, press the "esc" key and enter ": wq" to save the contents and finish editing. (Enter ": q!" To exit without saving.)
~/.bash_profile
(...Abbreviation...)
export DJANGO_SECRET_KEY='<SECRET given when creating a Djnago project_KEY(50 digits)>'
export DJANGO_SETTINGS_MODULE=<Project name>.settings
export DB_USER=<Arbitrary user name>
export DB_PASSWORD=<Arbitrary password>
export ALLOWED_HOSTS=<Elastic IP> (or <unique domain name>) *
export AWS_SES_ACCESS_KEY_ID=<Access key ID obtained when setting up Amazon SES usage>
export AWS_SES_SECRET_ACCESS_KEY=<Secret Access key obtained when setting up Amazon SES usage>
Terminal
$ source ~/.bash_profile
Terminal
$ env
(...Abbreviation...)
XDG_SESSION_ID=514
HOSTNAME=ip-XXX-XXX-XXXX-XXX.ap-northeast-1.compute.internal
AWS_SES_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
TERM=xterm
SHELL=/bin/bash
HISTSIZE=1000
SSH_CLIENT=XXX.XXX.XXX.XXX 57002 22
DJANGO_SECRET_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
SSH_TTY=/dev/pts/0
USER=<Arbitrary user name>
AWS_SES_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXX
(...Omission...)
ALLOWED_HOSTS=<Original domain name>
MAIL=/var/spool/mail/<Arbitrary user name>
PATH=/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/<Arbitrary user name>/.local/bin:/home/<Arbitrary user name>/bin
PWD=/home/<Arbitrary user name>
DJANGO_SETTINGS_MODULE=<Project name>.settings
DB_PASSWORD=<Arbitrary password>
LANG=ja_JP.UTF-8
HISTCONTROL=ignoredups
SHLVL=1
HOME=/home/<Arbitrary user name>
LOGNAME=<Arbitrary user name>
DB_USER=<Arbitrary user name>
SSH_CONNECTION=XXX.XXX.XXX.XXX 57002 XXX.XXX.XXX.XXX 22
LESSOPEN=||/usr/bin/lesspipe.sh %s
XDG_RUNTIME_DIR=/run/user/1001
_=/usr/bin/env
To reduce the load on the AP server, static files are returned from the WEB server (Nginx).
Terminal
$ sudo mkdir -p /usr/share/nginx/html/static For static files
$sudo chown <any user>/usr/share/nginx/html/static
$ sudo mkdir /usr/share/nginx/html/media for media files
$sudo chown <any user>/usr/share/nginx/html/media
(Premise):
settings.py
STATIC_ROOT = 'usr/share/nginx/html/static'
By executing the following, static files such as css and jpg will be placed in the / usr/share/nginx/static
directory.
Terminal
(venv_<Project name>)$ python manage.py collectstatic
236 static files copied to '/usr/share/nginx/html/static'.
Enter the Python virtual environment and execute the following in the directory where manage.py exists.
Terminal
(venv_<Project name>)$ python manage.py migrate
Terminal
$ amazon-linux-extras list | grep nginx
38 nginx1=latest enabled [ =stable ]
We have found that nginx1 is the highest version that can be installed. Execute the following to install.
Terminal
$ sugo amazon-linux-extras install nginx1
Edit the configuration file for Nginx and configure the environment settings.
You can edit it by pressing the "i" key. After editing, press the "esc" key and enter ": wq" to save the contents and finish editing. (Enter ": q!" To exit without saving.)
Terminal
$ sudo vi /etc/nginx/nginx.conf
nginx.conf
(...Abbreviation...)
include /etc/nginx/conf.d/*.conf;
server {
listen 80;
listen [::]:80;
server_name <Elastic IP or your own domain name>;
return 301 https://$host/$request_uri;
}
# Setting for a TLS enabled server.
server {
listen 443;
listen [::]:443;
server_name <Elastic IP or your own domain name>;
root /usr/share/nginx/html;
ssl_certificate "/etc/letsencrypt/live/<Own domain>/fullchain.pem";
ssl_certificate_key "/etc/letsencrypt/live/<Own domain>/privkey.pem";
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
location /static {
alias /usr/share/nginx/html/static;
}
location /media {
alias /usr/share/nginx/html/media;
}
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-proto $scheme;
proxy_pass http://127.0.0.1:8000;
}
location /.well-known/acme-challenge {
root /usr/share/nginx/html;
}
error_page 404 /404.html;
location = /40x.html {
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
}
}
}
(...Abbreviation...)
Terminal
$ sudo systemctl enable nginx.service
Terminal
$ sudo systemctl start nginx.service
$ systemctl status nginx.service
● nginx.service - The nginx HTTP and reverse proxy server
Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; vendor preset: disabled)
Active: active (running)since tree 2020-12-10 08:20:46 JST; 2 days ago
Process: 13919 ExecStart=/usr/sbin/nginx (code=exited, status=0/SUCCESS)
Process: 13916 ExecStartPre=/usr/sbin/nginx -t (code=exited, status=0/SUCCESS)
Process: 13915 ExecStartPre=/usr/bin/rm -f /run/nginx.pid (code=exited, status=0/SUCCESS)
Main PID: 13922 (nginx)
CGroup: /system.slice/nginx.service
├─13922 nginx: master process /usr/sbin/nginx
└─13923 nginx: worker process
If you want to restart Nginx, do the following:
Terminal
$ sudo systemctl reload nginx.service
If you want to stop Nginx, do the following:
Terminal
$ sudo systemctl stop nginx.service
Enter the Python virtual environment and execute the following in the directory where manage.py exists.
Terminal
(venv_Project name)$ gunicorn --bind 127.0.0.1:8000 <Project name>.wsgi -D
(venv_Project name)$ ps ax | grep gunicorn
671 pts/0 S+ 0:00 grep --color=auto gunicorn
13937 ? S 0:26 /home/<Arbitrary user name>/venv_<Project name>/bin/python3 /home/<Arbitrary user name>/venv_<Project name>/bin/gunicorn --bind 127.0.0.1:8000 <Project name>.wsgi -D
13940 ? S 0:03 /home/<Arbitrary user name>/venv_<Project name>/bin/python3 /home/<Arbitrary user name>/venv_<Project name>/bin/gunicorn --bind 127.0.0.1:8000 <Project name>.wsgi -D
If you want to stop Gunicorn, do the following:
Terminal
(venv_Project name)$ pkill gunicorn
Please refer to [Nginx] What to do if you cannot get the SSL certificate from Let ’s Encrypt with certbot-auto.
backup_db.py
import csv
import datetime
import os
from django.conf import settings
from django.core.management.base import BaseCommand
from ...models import Post
class Command(BaseCommand):
help = "Backup Database"
def handle(self, *args, **options):
date = datetime.date.today().strftime("%Y%m%d")
file_path = settings.BACKUP_PATH + 'database_' + date + '.csv'
os.makedirs(settings.BACKUP_PATH, exist_ok=True)
with open(file_path, 'w') as file:
writer = csv.writer(file)
header = [field.name for field in Post._meta.fields]
writer.writerow(header)
blogs = Post.objects.all()
for customer in customers:
writer.writerow([str(customer.id,
customer.name,
customer.address,
str(customer.phone_number),
str(customer.created_at),
str(customer.updated_at)])
files = os.listdir(settings.BACKUP_PATH)
if len(files) >= settings.NUM_SAVED_BACKUP:
files.sort()
os.remove(settings.BACKUP_PATH + files[0])
settings.py
BACKUP_PATH ="backup/"
NUM_SAVED_BACKUP = 30
Cron is a program that automatically executes registered commands on a regular basis. A feature built into Amazon Linux 2 from the beginning.
You can edit it by pressing the "i" key. After editing, press the "esc" key and enter ": wq" to save the contents and finish editing. (Enter ": q!" To exit without saving.)
Terminal
$ crontab -e
Contents added by the crontab command
0 4 * * * source ~/venv_<Project name>/bin/activate; cd ~/venv_<Project name>/<Project name>; python manage.py backup_db > ~/cron.log 2>&1
** Basic cron format **
<minute> <hour> <day> <month> <day of the week> <command>
The above example is an example of executing a command at 4:00 AM (UTC) every day.
Notes
The specification in cron format cannot be changed because the time zone is UTC only.
To set in Japan Standard Time (JST), specify a value of "-9 hours" (minus 9 hours) for UTC.
Therefore, the above example is an example of executing the command at 1 pm (13:00) (JST) every day.
It will be done after HTTPS conversion is completed. Enter the Python virtual environment and execute the following in the directory where manage.py exists.
settings.py
# security.W004
SECURE_HSTS_SECONDS = 31536000
SECURE_HSTS_INCLUDE_SUBDOMAINS = True
# security.W006
SECURE_CONTENT_TYPE_NOSNIFF = True
# security.W007
SECURE_BROWSER_XSS_FILTER = True
# security.W008
SECURE_SSL_REDIRECT = True
# security.W012
SESSION_COOKIE_SECURE = True
# security.W016
CSRF_COOKIE_SECURE = True
# security.W019
X_FRAME_OPTIONS = 'DENY'
# security.W021
SECURE_HSTS_PRELOAD = True
Terminal
(venv_<Project name>)$ python manage.py check --deploy
If the WARNING message is displayed, it means that some security measures are insufficient. Normally, WARNING is not displayed in the above settings in the project settings file (settings.py).
Reference: Security in Django
[Python/Django] Summary of frequently used commands (1) <Creating virtual environment, project, application>
[Python/Django] Summary of frequently used commands (2)
You can use EC2 as a development environment as well as during production operation, so I would like to consider using Docker as well.
Recommended Posts