[JAVA] POST the image selected on the website with multipart / form-data and save it to Amazon S3! !!

What I wanted to do

After selecting an image file on the website, I wanted to POST the image with multipart / form-data with the upload button and save it to S3 on AWS, so I figured out how to do it. It may be helpful if you want to use AWS services, including saving images on your website.

Overall configuration diagram

スクリーンショット 2020-03-06 13.25.15.png

Website image

スクリーンショット 2020-03-06 11.11.42.png

Development language and service used

We use ** HTML, CSS, JavaScript ** as the development language on the web side and ** Python 3.7 ** as the language on the Lambda side. The following are the services used.

** ・ AWS Lambda ** An AWS service that allows you to execute function code triggered by an event. **-Amazon S3 (used to save uploaded images and to publish websites) ** Abbreviation for Simple Storage Service, AWS online storage service

After POSTing the image in the form of multipart / form-data on the Web side using ajax, start Lambda by hitting API Gateway and save the image in S3 with the function in Lambda. As a bonus, store the file name of the saved image in the website's local storage so that you can browse the saved image.

Overall creation procedure

  1. [Create Website](#web-Create Site)
  2. [Create Bucket for image storage in S3](-bucket-Create for image storage in # s3-)
  3. Create Lambda (including API Gateway) (Create # lambda-including api-gateway)

It becomes the procedure.

Creating a website

Full code on the website side

This time, I took the form of POSTing to the API in the form of multipart / form-data from the website side. We also perform processing to prevent transition to another screen when the submit button is clicked, and change the style of various buttons.

index.html


<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <!--jquery library is not fixed-->
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script> 
    <title>Title</title>
    <style>
        label > input {
            display:none; /*Disable file button style*/
        }
        label > button {
            display:none; /*Disable button style*/
        }

        label {
            display: inline-block;
            padding: 0.5em 1em;
            text-decoration: none;
            background: #fd9535;/*Button color*/
            color: #FFF;
            border-bottom: solid 4px #627295;
            border-radius: 3px;
        }
        button{
            display: inline-block;
            padding: 0.5em 1em;
            text-decoration: none;
            background: #fd9535;/*Button color*/
            color: #FFF;
            border-bottom: solid 4px #627295;
            border-radius: 3px;
        }
        label:active {
            /*When you press the button*/
            -webkit-transform: translateY(4px);
            transform: translateY(4px);/*Move down*/
            border-bottom: none;/*Erase the line*/
        }
        button:active{
            /*When you press the button*/
            -webkit-transform: translateY(4px);
            transform: translateY(4px);/*Move down*/
            border-bottom: none;/*Erase the line*/
        }
    </style>
</head>
<body>
<form action="xxx Where to enter the URL of API Gateway xxx" method="post" enctype="multipart/form-data" id="imgForm"  target="sendPhoto">
    <p>
        <label for ="upfile">
Please select a file
            <input type="file" name="fileName" id="upfile" accept="image/*" capture="camera">
        </label>
        <span style="color: #ff0000;" data-mce-style="color: #ff0000;"><div><img id="thumbnail" src=""></div></span>
    </p>
    <p>
        <label for="upload">
upload
            <button type="submit" action="" name="save" id="upload" >upload</button>
        </label>
    </p>
    <p id="compUpload"></p>
</form>
<iframe name="sendPhoto" style="width:0px;height:0px;border:0px;"></iframe>
<script>
$('#upfile').change(function(){
if (this.files.length > 0) {
//Get selected file information
var file = this.files[0];
//Store the file data encoded as a data URL in the result property of reader
var reader = new FileReader();
reader.readAsDataURL(file);
reader.onload = function() {
$('#thumbnail').attr('src', reader.result );
}
}
});

//Upload image
    $('#imgForm').on('submit', function(e) {
        e.preventDefault();
        var formData = new FormData($('#imgForm').get(0));
        $.ajax($(this).attr('action'), {
            type: 'post',
            processData: false,
            contentType: false,
            data: formData,
            success: document.getElementById('compUpload').innerHTML = 'Uploading' //When the transmission is successful

    }).done(function(response){
        let jsonbody = JSON.parse(response.body);
            console.log('succes!');  //When there is a response
            //Store response file name in local storage
            var array = [];
            var obj = {
                'Key 1': 'Value 1',
                'Key 2': 'Value 2'
            };
            array.push(obj);
            var setjson = JSON.stringify(obj);
            localStorage.setItem('Key', jsonbody.message);
            document.getElementById('compUpload').innerHTML = 'Upload completed'

    }).fail(function() {
            console.log('error!'); //When an error occurs
        });
        return false;
    });
</script>
</body>
</html>

Copy and paste the above code to create an html file. Place the file on your favorite server or S3 and view it. This completes the creation on the Web side except for "Specify the URL of API Gateway".

Specifying the URL of API Gateway

The code below is the specified part

python


<body>
<form action="xxx Where to enter the URL of API Gateway xxx" method="post" enctype="multipart/form-data" id="imgForm"  target="sendPhoto">

I'll add this when I know the URL of the API Gateway in a later step to create a Lambda. Also, the default button was dull, so I changed the design with the style tag at the top of the code. The default display is hidden with display: none; and then changed to a new design.

python


<style>
        label > input {
            display:none; /*Disable file button style*/
        }
        label > button {
            display:none; /*Disable button style*/
        }

        label {
            display: inline-block;
            padding: 0.5em 1em;
            text-decoration: none;
            background: #fd9535;/*Button color*/
            color: #FFF;
            border-bottom: solid 4px #627295;
            border-radius: 3px;
        }
        button{
            display: inline-block;
            padding: 0.5em 1em;
            text-decoration: none;
            background: #fd9535;/*Button color*/
            color: #FFF;
            border-bottom: solid 4px #627295;
            border-radius: 3px;
        }
        label:active {
            /*When you press the button*/
            -webkit-transform: translateY(4px);
            transform: translateY(4px);/*Move down*/
            border-bottom: none;/*Erase the line*/
        }
        button:active{
            /*When you press the button*/
            -webkit-transform: translateY(4px);
            transform: translateY(4px);/*Move down*/
            border-bottom: none;/*Erase the line*/
        }
    </style>

Description of the image upload part

I'm using ajax to upload images. If the POST to the server side is successful, the web side will display "Uploading".

python


<script>
//Upload image
    $('#imgForm').on('submit', function(e) {
        e.preventDefault();
        var formData = new FormData($('#imgForm').get(0));
        $.ajax($(this).attr('action'), {
            type: 'post',
            processData: false,
            contentType: false,
            data: formData,
            success: document.getElementById('compUpload').innerHTML = 'Uploading' //When the transmission is successful

    }).done(function(response){
        let jsonbody = JSON.parse(response.body);
            console.log('succes!');  //When there is a response
            //Store response file name in local storage
            var array = [];
            var obj = {
                'Key 1': 'Value 1',
                'Key 2': 'Value 2'
            };
            array.push(obj);
            var setjson = JSON.stringify(obj);
            localStorage.setItem('Key', jsonbody.message);
            document.getElementById('compUpload').innerHTML = 'Upload completed'

    }).fail(function() {
            console.log('error!'); //When an error occurs
        });
        return false;
    });
</script>

Create a bucket for saving images on S3

Use S3 as the location for uploading image files from the web. Select S3 from the AWS Management Console and create a bucket. It is recommended that the region specified at this time be the same as the region specified when Lambda is created later. (Because it may take time to reflect the settings in another region) ⇨ Reference link There is no problem with the setting at the time of creation by default, but in this article, uncheck'Block all public access'in the permission setting and set it to allow public access. (To display the response in Lambda on the web) If you want to set the access permission individually, please set it from the bucket policy after creation. スクリーンショット 2020-03-09.png

If you specify the path of the bucket created here in Lambda later, the uploaded image will be dropped.

Create Lambda (including API Gateway)

Creating a Lambda function

In Lambda, the image POST on the Web side is triggered to run a function that saves the image in S3. To do this, first select Lambda from the AWS Management Console and click Create Function. At the time of creation, this article was created in the northern Virginia region. (It is recommended to make it the same as the region created in S3)

スクリーンショット 2020-03-09-2.png

On the function creation screen, ・ Create from scratch -Function name (Enter your favorite function name) ・ Runtime (Python 3.7 is used this time) · Execution role (choose create new role with basic Lambda permissions)

After deciding these, click Create Function. Here, the function name is myFunctionName. The Run role allows access to S3 and CloudWatch, but first create it with basic Lambda permissions and add it later. スクリーンショット 2020-03-09 11.18.30.png

Creating an API Gateway

When the function creation is completed, the function edit screen opens. Click ** Add Trigger ** in the Designer window. The window will switch, so select ** API Gateway ** and make detailed settings.

スクリーンショット 2020-03-09 11.46..png

On the Add Trigger screen, use the respective drop-down box and radio button choices. ・ API ⇨ Create new API -Select a template ⇨ REST API ・ Security ⇨ Open ・ Click additional settings ⇨ Added ** image / png ** and ** multipart / form-data ** to binary media types After that, click ** Add ** at the bottom right of the screen.

スクリーンショット 2020-03-11 17.30.25.png スクリーンショット 2020-03-11 17.32.21.png

Then you should see API Gateway added to Designer as shown in the image below. This is a mechanism in which API Gateway receives a POST on the Web side and triggers the Lambda function to start.

スクリーンショット 2020-03-09 13.54..png

When you click API Gateway in Designer, you will see a ** API endpoint ** that starts with https. Enter this URL as [Uncertain part in the website creation procedure](# The code below is -url-). Enter in (Part).

スクリーンショット 2020-03-09 14.50.2.png

Next, click the "Function name-API" link displayed directly above the API endpoint to create an API method. スクリーンショット 2020-03-09 14.50.26.mo.png

API Gateway POST method configuration

Click the "Function Name-API" link to open the API Gateway settings page. Now configure the API POST method to receive json-formatted files from your website.

  1. Click ** Create Method ** from the action to display the select box (click the collapsible arrow to the right of the function name)

  2. Select ** POST ** from the select box and click the checkmark

  3. Select POST, select the Lambda region you are using on the screen that appears, then enter the function name you created earlier in the ** Lambda function ** input field and click Save.

  4. Click OK if a window appears to add permissions to your Lambda function

  5. Select Enable CORS from Actions, enter'true'in Advanced Access-Control-Allow-Credentials (including''), enable CORS and click Replace existing CORS header to value To add.

  6. Select ** POST ** again, click ** Integration Request ** on the method execution screen, and map template ⇨ if no template is defined (recommended) ⇨ add mapping template ⇨ ** multipart / Enter form-data ** ⇨ Select ** Method request passthrough ** in Template generation and save

  7. After clicking the Save button, select ** Function Name ** above ANY, select Deploy API from Actions, select default for the stage to be deployed, then click the Deploy button

This completes the API settings. Please note that there are many steps and it is easy to make mistakes. Next, we will move on to the function code settings, so please return to the Lambda function setting screen. You can return by service ⇨ Lambda ⇨ function name.

スクリーンショット 2020-03-10 10.20.46.png

スクリーンショット 2020-03-10 10.38.02.png

スクリーンショット 2020-03-10 10.33.35.png

スクリーンショット 2020-03-10 10.43.59.mo.png

スクリーンショット 2020-03-12 9.07.02.png

スクリーンショット 2020-03-10 11.05.43.mo.png

スクリーンショット 2020-03-12 9.21.23.png

スクリーンショット 2020-03-12 9.26.57.png

Lambda function code description

If you click the function name in Designer and scroll, there is an item to write the function code, so write the code in Python here. Here, the decoding of the image, the setting of the storage location, the response for acquiring the image name on the Web side, etc. are described.

Full function code

lambda_function.py


import json
import boto3
import base64
from datetime import datetime, timezone

def convert_b64_string_to_bynary(s):
    """Decode base64"""

def lambda_handler(event, context):
    #Image storage location setting
    s3 = boto3.resource('s3')
    bucket = s3.Bucket('xxx Name of bucket to store images xxx')

    #The binary is Base64 encoded, so decode it here
    imageBody = base64.b64decode(event['body-json'])
    TextTime = datetime.now().strftime('%Y-%m-%d-%H-%M-%S') #Set the uploaded image file name to the time at that time
    images = imageBody.split(b'\r\n',4)#Cut only the necessary part with split

    key = TextTime +".png "
    bucket.put_object(
        Body = images[4],
        Key = key
    )    
    #Return value to get the image name on the Web side
    return {
        'isBase64Encoded': False,
        'statusCode': 200,
        'headers': {'Access-Control-Allow-Origin' : '*' ,
        'Content-Type' :'application/json'},
        'body': '{"message":"'+ key + '"}'
    }

After pasting the code, enter the S3 bucket name in the part below and save the state with the save button at the top right of the screen.

lambda_function.py


bucket = s3.Bucket('xxx Name of bucket to store images xxx')

Setting environment variables

Set an environment variable to adjust the timestamp to Japanese time. If you scroll down a little from the function code screen, you will find an item for environment variables. Follow the steps below to set environment variables.

  1. Click the edit button at the top right of the environment variable item
  2. You will be taken to the edit page for environment variables, where you can click Add Environment Variable.
  3. Enter "TZ" for the key and "Asia / Tokyo" for the value
  4. Click the save button at the bottom right
スクリーンショット 2020-03-09 16.15.35.png スクリーンショット 2020-03-09 16.24.54.png

Execution role settings

If you scroll down a little from the environment variable item, you will find the execution role setting item. Follow the steps below to set the execution role.

  1. Select Use existing role in the first pull-down
  2. Make sure that the policy with the function name currently being edited is selected for the existing role (service-role / function name -...)
  3. Click the link below ** Show Function Name Role ** in the IAM console
  4. A new window will open where you can click ** Attach Policy **
  5. Enter S3 in the search field, check AmazonS3FullAccess and click ** Attach Policy **

スクリーンショット 2020-03-09 16.36.28.mo.png

スクリーンショット 2020-03-09 16.55.30.mo.png

スクリーンショット 2020-03-09 17.04.07.mo.png

This completes all the settings. Try uploading the image from the web screen, and if the image is saved in the corresponding bucket of S3, it is successful. As a bonus, you can also check that the file name is saved in the local storage at the upload time with the development tool. Please use it when you use a service that specifies the uploaded file.

スクリーンショット 2020-03-12 9.56.37.png

At the end

This time, I used to refer to S3 images with a service called Amazon Rekognition Custom Labels on AWS, so I implemented such a function. If you are interested, please see the article below for the usability. About the usability of Amazon Rekognition Custom Labels

Reference link

Why does Amazon S3 return an HTTP 307 Temporary Redirect response? How to prevent screen transition while performing POST processing with Form

Recommended Posts

POST the image selected on the website with multipart / form-data and save it to Amazon S3! !!
POST the image with json and receive it with flask
Start the webcam to take a still image and save it locally
[python] Send the image captured from the webcam to the server and save it
Upload data to s3 of aws with a command and update it, and delete the used data (on the way)
How to publish a blog on Amazon S3 with the static Blog engine'Pelican'for Pythonista
How to get the key on Amazon S3 with Boto 3, implementation example, notes
GAE --With Python, rotate the image based on the rotation information of EXIF and upload it to Cloud Storage.
Try to Normalize Cut the image with scikit-image (although it gets angry on the way)
[pyqtgraph] Add region to the graph and link it with the graph region
Return the image data with Flask of Python and draw it to the canvas element of HTML
Save images on the web to Drive with Python (Colab)
I want to cut out only the face from a person image with Python and save it ~ Face detection and trimming with face_recognition ~
Process the gzip file UNLOADed with Redshift with Python of Lambda, gzip it again and upload it to S3
Take an image with Pepper and display it on your tablet
I ran GhostScript with python, split the PDF into pages, and converted it to a JPEG image.
The story of making a tool to load an image with Python ⇒ save it as another name
Put Cabocha 0.68 on Windows and try to analyze the dependency with Python
Connect to VPN with your smartphone and turn off / on the server
Scrap the published csv with Github Action and publish it on Github Pages
Convert the spreadsheet to CSV and upload it to Cloud Storage with Cloud Functions
Get exchange rates on Heroku regularly and upload logs to Amazon S3
How to query BigQuery with Kubeflow Pipelines and save the result and notes
How to save the feature point information of an image in a file and use it for matching
Add lines and text on the image
The file edited with vim was readonly but I want to save it
Make a thermometer with Raspberry Pi and make it visible on the browser Part 3
[Python] How to save images on the Web at once with Beautiful Soup
I made an image classification model and tried to move it on mobile
I want to convert horizontal text to vertical text and post it on Twitter etc.
Scraping the holojour and displaying it with CLI
Crop the image to rounded corners with pythonista
Image characters and post to slack (python slackbot)
Generate and post dummy image data with Django
I tried to save the data with discord
Crop Numpy.ndarray and save it as an image
Introduction to Python with Atom (on the way)
Save the object to a file with pickle
Post an article with an image to WordPress with Python
Save the search results on Twitter to CSV.
Make it easy to install the ROS2 development environment with pip install on Python venv
From installing Flask on CentOS to making it a service with Nginx and uWSGI
[Note] How to write QR code and description in the same image with python
Read the csv file with jupyter notebook and write the graph on top of it
It is easy to execute SQL with Python and output the result in Excel
It is difficult to install a green screen, so I cut out only the face and superimposed it on the background image
[Python] Save the result of web scraping the Mercari product page on Google Colab to Google Sheets and display the product image as well.