I tried standalone deployment of play with fabric [AWS operation with boto] [Play deployment]

Automating play application deployment with fabric I operated Amazon EC2 and ELB in the fabric, so I would like to give you a brief introduction.

What is fabric?

http://fabric-ja.readthedocs.org/en/latest/tutorial.html

What is Fabric:

-A tool that executes arbitrary Python functions via the command line. -A library of subroutines (built on top of a low-level library) Easily execute shell commands via SSH in a Python-like way.

It's hard to understand if it's an official document, By combining the two, you can automate the process to the remote server. Production deployment Manual work is too scary and troublesome, so let's automate it more and more.

For comparison with Capistrano, I referred to the following article. http://dekokun.github.io/posts/2013-05-21.html

Premise

○ The flow of deployment looks like this Source update ↓ Remove EC2 from ELB ↓ play stop ↓ Deploy with play ↓ play start ↓ Add EC2 to ELB ↓ End of deployment

environment

Play framework 2.3.x activator 1.2.10 The OS to deploy to is Amazon Linux Fabric 1.8.1

fabric installation

centOS sudo yum -y install python python-devel python-setuptools gcc sudo easy_install pip sudo pip install fabric==1.8.1

Mac sudo easy_install fabric

What you want to know in advance

call local-> local shell command env-> env variable (global singleton), used for information sharing between tasks lcd-> local current directory specification execute-> execute task

Decorator @task-> Load as a valid task @runs_once-> Prevent wrapped functions from being executed multiple times

For the time being, I will put in a fabfile. Partially described below. fabfile

# -*- coding: utf-8 -*-
from fabric.api import *
from fabric.decorators import runs_once
import boto.ec2.elb

env.hosts = ['localhost']
env.user = "username"

env.project_root = "path/to/project"
env.build_path = "target/universal/stage/bin/project_name"
env.config_path = env.project_root + '/conf/application.conf'
env.branch = 'master'

env.region = "ap-northeast-1"
env.elb_name = 'your_elb_name'
env.aws_access_key = 'your_aws_access_key'
env.aws_secret_key = 'your_aws_secret_key'


##Build
@task
@runs_once
def build_play():
    """ activator clean stage"""
    with lcd(env.project_root):
        local("activator clean stage")


##activator activation system
@task
def play_start():
    """Launch activator in production mode"""
    with lcd(env.project_root):
        local("%(bin)s -Dconfig.file=%(conf)s &" % {"bin":env.build_path, "conf":env.config_path})

@task
def play_stop():
    """Kill a process running in production mode"""
    if local('ps -ef | grep "%(root)s/target/universal/stage" | grep -v grep | wc -l' % {"root":env.project_root}, capture=True) == "1":
        local("kill `cat %(root)s/target/universal/stage/RUNNING_PID`" % {"root":env.project_root})

@task
def play_restart():
    """Reboot in production mode"""
    execute(play_stop)
    execute(play_start)


##AWS ELB operation
def get_ec2_id():
    """Get instanceID for ec2"""
    env.ec2_id = local("curl http://169.254.169.254/latest/meta-data/instance-id", capture=True)

def conn_elb():
    env.conn_elb = boto.ec2.elb.connect_to_region(
        env.region,
        aws_access_key_id = env.aws_access_key,
        aws_secret_access_key = env.aws_secret_key)

def get_elb():
    execute(conn_elb)
    env.elb = env.conn_elb.get_all_load_balancers(env.elb_name)[0]

def remove_ec2():
    """ ec2(myself)From ELB"""
    execute(get_elb)
    execute(get_ec2_id)
    env.elb.deregister_instances(env.ec2_id)

def add_ec2():
    """ ec2(myself)To ELB"""
    execute(get_elb)
    execute(get_ec2_id)
    env.elb.register_instances(env.ec2_id)


@task
@runs_once
def update(branch=env.branch):
    """ env.Synchronize with branch. When specifying a branch-> fab update:branch=branch_name"""
    with lcd(env.project_root):
        local("git fetch")
        local("git reset --hard HEAD")
        local("git checkout " + branch)
        local("git pull")


##Deploy
@task
@runs_once
def deploy(branch=env.branch):
    """Branch specification-> fab deploy:branch=branch_name"""
    execute(remove_ec2)
    execute(play_stop)
    execute(update,branch=branch)
    execute(build_play)
    execute(play_start)
    execute(add_ec2)

Play deploy

Start the play process in the background

@task
def play_start():
    """Launch activator in production mode"""
    with lcd(env.project_root):
        local("%(bin)s -Dconfig.file=%(conf)s &" % {"bin":env.build_path, "conf":env.config_path})

play process stop

Delete the process using the RUNNING_PID file that manages the process in the play application.

@task
def play_stop():
    """Kill a process running in production mode"""
    if local('ps -ef | grep "%(root)s/target/universal/stage" | grep -v grep | wc -l' % {"root":env.project_root}, capture=True) == "1":
        local("kill `cat %(root)s/target/universal/stage/RUNNING_PID`" % {"root":env.project_root})

ELB operation

boto I use a library called boto (AWS SDK for Python) to operate ELB from fabric. API document: http://boto.readthedocs.org/en/latest/ https://github.com/boto/boto

Import boto

import boto.ec2.elb

Connect to AWS and get an ELB named env.elb_name

def conn_elb():
    env.conn_elb = boto.ec2.elb.connect_to_region(
        env.region,
        aws_access_key_id = env.aws_access_key,
        aws_secret_access_key = env.aws_secret_key)

def get_elb():
    execute(conn_elb)
    env.elb = env.conn_elb.get_all_load_balancers(env.elb_name)[0]

Get the instance ID of EC2 (yourself)

EC2 seems to be able to get various meta information of itself. Benri to know http://d.hatena.ne.jp/rx7/20100605/p1

Note that in the case of local, it will not be entered in the variable unless capture = True.

def get_ec2_id():
    """Get instanceID for ec2"""
    env.ec2_id = local("curl http://169.254.169.254/latest/meta-data/instance-id", capture=True)

Remove EC2 (yourself) from ELB

def remove_ec2():
    """ ec2(myself)From ELB"""
    execute(get_elb)
    execute(get_ec2_id)
    env.elb.deregister_instances(env.ec2_id)

Attach EC2 (self) from ELB

def add_ec2():
    """ ec2(myself)To ELB"""
    execute(get_elb)
    execute(get_ec2_id)
    env.elb.register_instances(env.ec2_id)

demo

Execute the fab command in the current directory where the fabfile is installed. I think it can be placed in the project route or under / home / user.

The list with the @task decorator is displayed. $ fab --list スクリーンショット 2015-02-24 20.46.00.png

$ fab deploy

スクリーンショット 2015-02-24 20.55.23.png

スクリーンショット 2015-02-24 20.53.54.png

in conclusion

I touched both fabric and boto for the first time this time, but I thought that the benefits were high for the low learning cost. I also made a deploy version from the deploy server to the remote host, so when I have time, that too I would like to blog.

Thank you for reading until the end.

Recommended Posts

I tried standalone deployment of play with fabric [AWS operation with boto] [Play deployment]
I tried to automatically post to ChatWork at the time of deployment with fabric and ChatWork Api
I want to play with aws with python
I tried handwriting recognition of runes with scikit-learn
[Python / DynamoDB / boto3] List of operations I tried
I tried hundreds of millions of SQLite with python
I tried image recognition of CIFAR-10 with Keras-Learning-
I tried image recognition of CIFAR-10 with Keras-Image recognition-
I tried Flask with Remote-Containers of VS Code
I tried connecting AWS Lambda with other services
I tried to extract features with SIFT of OpenCV
I tried Amazon Comprehend sentiment analysis with AWS CLI.
[OpenCV / Python] I tried image analysis of cells with OpenCV
I tried AWS CDK!
I tried AWS Iot
[Introduction to AWS] I tried playing with voice-text conversion ♪
I tried "morphology conversion" of images with Python + OpenCV
I tried to find the entropy of the image with python
I tried to find the average of the sequence with TensorFlow
I tried starting Django's server with VScode instead of Pycharm
I tried running Movidius NCS with python of Raspberry Pi3
I tried to implement ListNet of rank learning with Chainer
I tried handwriting recognition of runes with CNN using Keras
I tried a stochastic simulation of a bingo game with Python
I wrote the basic operation of matplotlib with Jupyter Lab
I tried fp-growth with python
I tried scraping with Python
I tried Learning-to-Rank with Elasticsearch!
S3 operation with python boto3
I tried clustering with PyCaret
I tried gRPC with Python
I tried scraping with python
I tried using AWS Chalice
I tried to unlock the entrance 2 lock sesame with a single push of the AWS IoT button
I tried scraping the ranking of Qiita Advent Calendar with Python
[AWS / Tello] I tried operating the drone with my voice Part2
I tried to automate the watering of the planter with Raspberry Pi
I tried to create a list of prime numbers with python
I tried to fix "I tried stochastic simulation of bingo game with Python"
I tried to get a list of AMI Names using Boto3
I wrote the basic operation of Pandas with Jupyter Lab (Part 1)
[AWS / Tello] I tried operating the drone with my voice Part1
I tried to expand the size of the logical volume with LVM
I tried running the DNN part of OpenPose with Chainer CPU
I tried to improve the efficiency of daily work with Python
I tried to automatically collect images of Kanna Hashimoto with Python! !!
I wrote the basic operation of Pandas with Jupyter Lab (Part 2)
I tried to make a mechanism of exclusive control with Go
Play with numerical calculation of magnetohydrodynamics
I tried trimming efficiently with OpenCV
I tried summarizing sentences with summpy
Get AWS account ID with boto3
I tried machine learning with liblinear
I tried web scraping with python.
I tried moving food with SinGAN
I tried using GrabCut of OpenCV
I tried implementing DeepPose with PyTorch
Automation of remote operations with Fabric
I tried face detection with MTCNN
I couldn't daemonize gunicorn with Fabric
I tried running prolog with python 3.8.2.