Use additional Python packages with Serverless Framework (v1.x)

Serverless (2) Advent Calendar This is the article on the 17th day. To be honest, it's not a big story, but ... (^_^;) Since there is not much information on Python in Serverless Framework (v1.x) and I was worried about it myself, I tried this. (Added at 6:17 AM on December 17) Due to another matter, I changed the input data from CSV to JSON.

background

Originally, in our "[IoT Easy Pack](http://dc.ogis-ri.co.jp/iotpack/" IoT Easy Pack ")", the only message broker that can be selected is MQTT, and it must support HTTPS. It was because I couldn't do it. (It can be published even with REST of AWS IoT, but the authentication is too special and this time the target device is a microcomputer chip, so I had to make the API as simple as possible ... ^^;)

Last time, I tried to run the sample as it is with API Gateway with Serverless Framework (v1.1). [I tried using Serverless Framework](http://qiita.com/daikunjp/items/2b8aa35ecd104001f4e5 I tried using Serverless Framework)

This time, I would like to create an API that inserts CSV data PUT into Web API into MySQL.

Web API construction procedure using Serverless Framework

Serverless Framework version upgrade

Since it was updated to v1.3 before I knew it, I will upgrade the created environment [last time](http://qiita.com/daikunjp/items/2b8aa35ecd104001f4e5 I tried using Serverless Framework).

% npm install -g serverless

Check version

% sls --version
1.3.0

Project creation

Create and initialize the project directory

% mkdir iot-api
% cd iot-api
% sls create --template aws-python --name iot-api

Added MySQL library for Python

I have to put the MySQL driver in Python on lambda ... When I look at the manual, it says that pip can be used for Python, so I installed mysql-python with pip normally, but it doesn't work. Apparently, if you put it in the project directory, it will be put in the zip that uploads to lambda as it is, so install it in the project directory by the following procedure.

% pip install mysql-python -t .

Copy Shared Object to current directory

% cp /usr/lib64/mysql/libmysqlclient.so.18 .

Edit serverless.yml

Edit serverless.yml as follows. This time, I also added VPC settings to access RDS MySQL from lambda 【reference】 https://serverless.com/framework/docs/providers/aws/guide/functions/ http://dev.classmethod.jp/etc/serverless-framework-lambda-function-run-in-vpc/

serverless.yml


service: iot-api
provider:
  name: aws
  runtime: python2.7
  vpc:
   securityGroupIds:
     - sg-xxxxxxxx ← Security group ID
   subnetIds:
     - subnet-xxxxxxxx ← Subnet ID
     - subnet-xxxxxxxx ← Subnet ID
  stage: dev
  region: ap-northeast-1
  iamRoleStatements:
  - Effect: "Allow"
    Action:
      - "ec2:CreateNetworkInterface"
      - "ec2:DescribeNetworkInterfaces"
      - "ec2:DeleteNetworkInterface"
    Resource:
      - "*"
functions:
  iotput:
    handler: handler.iotput
    events:
      - http:
          path: iotput
          method: put

Python code

Click here for the code that actually works Since it is for demo, I do not check the arguments at all (^ _ ^;)

handler.py


# -*- coding: utf-8 -*-
import MySQLdb,json

HOST = "Hostname of the RDS to connect to"
DBNAME = "DB name"
DBUSER = "DB username"
PASSWD = "password"
CHARSET = "utf8"

def iotput(event, context):

  # MySQL Connect
    connect = MySQLdb.connect(host=HOST, db=DBNAME, user=DBUSER, passwd=PASSWD, charset=CHARSET)
    connect.cursorclass = MySQLdb.cursors.DictCursor
    cursor = connect.cursor()

  # BODY Data split for CSV
  #  data1 = event['body'].split(",")
  # BODY Data split for JSON
    data1 = json.loads(event['body'])
    data1[0] = data1['data1']
    data1[1] = data1['data2']
    data1[2] = data1['data3']

  # MySQL Data insert
    cursor.execute("insert into sensor01(data1,data2,data3) values(%s,%s,%s)",
        (data1[0],data1[1],data1[2]))
    connect.commit()
    cursor.close()
    connect.close()

    body = {
        "message": "input data is %s %s %s" % (data1[0],data1[1],data1[2]),
    }

    response = {
        "statusCode": 200,
        "body": json.dumps(body)
    }

    return response

Deploy

% sls deploy
Serverless: Packaging service...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading service .zip file to S3 (916.94 KB)...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
..............
Serverless: Stack update finished...

Service Information
service: iot-api
stage: dev
region: ap-northeast-1
api keys:
  None
endpoints:
  PUT - https://xxxxxxxx.execute-api.ap-northeast-1.amazonaws.com/dev/iotput
functions:
  iot-api-dev-iotput: arn:aws:lambda:ap-northeast-1:xxxxxxxxxxxxxxx:function:iot-api-dev-iotput

test

Try testing with curl from another machine.

% curl -X PUT  -d '{"data1":"10","data2":"20","data3":"40"}' https://xxxxxxxxxx.execute-api.ap-northeast-1.amazonaws.com/dev/iotput
{"message": "input data is 10 20 40"}
% curl -X PUT  -d '{"data1":"1.1","data2":"5.4","data3":"2.7"}' https://xxxxxxxxxx.execute-api.ap-northeast-1.amazonaws.com/dev/iotput
{"message": "input data is 1.1 5.4 2.7"}

Check the result

mysql> select * from sensor01;
+-------+-------+-------+
| data1 | data2 | data3 |
+-------+-------+-------+
|    10 |    20 |    40 |
|   1.1 |   5.4 |   2.7 |
+-------+-------+-------+
2 rows in set (0.01 sec)

Recommended Posts

Use additional Python packages with Serverless Framework (v1.x)
Touch AWS with Serverless Framework and Python
[Python] Use JSON with Python
Use mecab with Python3
Use DynamoDB with Python
Use Python 3.8 with Anaconda
Use python with docker
Create Python version Lambda function (+ Lambda Layer) with Serverless Framework
Deploy Python3 function with Serverless Framework on AWS Lambda
Use Trello API with python
Easily serverless with Python with chalice
Use Twitter API with Python
Use TUN / TAP with Python
Use subsonic API with python3
Effortlessly with Serverless Python Requirements
[Package cloud] Manage python packages with package cloud
Python: How to use async with
Use PointGrey camera with Python (PyCapture2)
Use vl53l0x with Raspberry Pi (python)
Proper use of Python visualization packages
[Python] Use Basic/Digest authentication with Flask
Use NAIF SPICE TOOLKIT with Python
Use rospy with virtualenv in Python3
Use Python in pyenv with NeoVim
How to use FTP with Python
Use Windows 10 speech synthesis with Python
[Django] Use MessagePack with Django REST framework
Use OpenCV with Python 3 in Window
Use PostgreSQL with Lambda (Python + psycopg2)
How to use Serverless Framework & Python environment variables and manage stages
Install pip in Serverless Framework and AWS Lambda with Python environment
Install Python as a Framework with pyenv
Use smbus with python3 under pyenv environment
Use DeepL with python (for dissertation translation)
Use Amazon Simple Notification Service with Python
[Python] Use string data with scikit-learn SVM
[Introduction to Python] Let's use foreach with Python
Use PIL and Pillow with Cygwin Python
Use cryptography library cryptography with Docker Python image
Use Application Insights with Python 3 (including bottles)
Let's make a web framework with Python! (1)
Use C ++ functions from python with pybind11
Serverless face recognition API made with Python
Use selenium phantomjs webdriver with python unittest
Until you can use opencv with python
Use Python and MeCab with Azure Functions
Let's make a web framework with Python! (2)
Create a simple reception system with the Python serverless framework Chalice and Twilio
If you want to make a discord bot with python, let's use a framework
I want to use MATLAB feval with python
Introduction to Tornado (1): Python web framework started with Tornado
Specify the Python executable to use with virtualenv
Use logger with Python for the time being
Python> Run with run-time arguments> Use import argparse
Use multiple versions of python environment with pyenv
Wrap C with Cython for use from Python
Serverless application with AWS SAM! (APIGATEWAY + Lambda (Python))
I made a LINE Bot with Serverless Framework!
The easiest way to use OpenCV with python
Use various rabbimq features with pika in python
I want to use Temporary Directory with Python2