I tried to introduce a serverless chatbot linked with Rakuten API to Teams

Introduction

This time I created a serverless chatbot that can search books and introduced it to Teams.

When users enter a keyword in Teams, we've created a chatbot that displays five book titles and prices related to the keyword. image.png In this article, I'll show you how to bring a serverless chatbot to Teams.

background

I will explain the background that led me to create a chatbot.

We were looking for a new initiative aimed at developing new graduates. While searching, I came up with the idea, "Why don't you try pair programming with new graduates and senior employees to create an app?"

The purpose of this proposal is the following two.

――If you use the technical skills of senior employees, you can make things in a short period of time. ――It will stimulate the motivation for learning for new graduates. ――By pair programming with senior employees, you can get various knowledge such as thinking circuit when writing code. ――For new graduates who have little coding experience, it is a valuable experience to know what senior employees are thinking about when coding.

I came up with this chatbot with the flow of trying to work on this idea with me, a new graduate, and senior employees.

By the way, I used Live Share for pair programming. Live Share is a pair programming tool available in VS Code. You can visually grasp where the people you are working with are operating, and since the movement is not heavy, you can comfortably perform pair programming.

For more information on Live Share, see Pair-Pro with Visual Studio Live Share (quoted from Developers.IO).

Configuration diagram of the created chatbot

The processing contents of the created chatbot are as follows.

  1. Send keywords in Teams.
  2. Send Webhook [^ 1] is activated by sending a keyword as a trigger.
  3. The sending webhook packs the keywords into the request Body and makes a POST request to API Gateway.
  4. The Lambda function is triggered by the request to API Gateway.
  5. Make a GET request to Rakuten API in the Lambda function.
  6. Rakuten API searches for books.
  7. Respond the search results to Teams.

[^ 1]: Outgoing webhook is a function of Teams that makes a POST request to Teams and external apps.
For more information, see Add custom bots to Microsoft Teams using outbound webhooks (https://docs.microsoft.com/ja-jp/microsoftteams/platform/webhooks-and-connectors/how-to/add-outgoing-webhook) (quoted from the Microsoft Office Developer Center).

Rakuten API uses Rakuten Books Comprehensive Search API. For details on Rakuten Books Comprehensive Search API Please refer to Rakuten Books Comprehensive Search API (version: 2017-04-04) (quoted from Rakuten Developers).

Below is the configuration diagram. image.png

Creation procedure

1. Preparing the AWS environment

This time we will mainly use two AWS services.

The Lambda function is implemented in python. The implementation contents will be introduced in 3. Script description (using python).

Set the Lambda function to be executed when a POST request is made to the URL of API Gateway. Please refer to API Creation Tutorial with API Gateway and Lambda (quoted from @vankobe) for the setting method.

2. Preparation of Rakuten API

To use the Rakuten API, you need to register as a Rakuten member and issue an app ID. Please refer to How to use Rakuten Product Search API-Find the lowest priced product (PHP)- (quoted from HP code) for how to issue the app ID.

Issuing an app ID and using Rakuten API is free.

3. Script writing (using python)

To implement the process, I created the following three functions.

  1. A function that searches for books with keywords entered by the user Function that extracts only the title and price parameters from the processing result of the function in 2.1 A function that calls the functions of 3.1 and 2 and describes a series of chatbot processes.

3-1. Function to search for books by keywords entered by the user

This function searches for books by the keyword entered by the user.

When the function is called, it receives the keyword entered by the user as an argument.

In addition, the search result of the book is received from Rakuten API with json and returned with dict type.

Below is a python script.

REQUEST_URL = "https://app.rakuten.co.jp/services/api/BooksTotal/Search/20170404"
APP_ID="Enter the issued app ID here"
def search_books(keyword):
    serch_params={
    "applicationId" : [APP_ID],
    "formatVersion": 2,
    "keyword" : keyword,
    "hits": 5,
    "availability": 0,
    "sort": "reviewAverage"
    "booksGenreId": "001"
    }
    response = requests.get(REQUEST_URL, serch_params)
    result = response.json()
    return result

Store the input parameters of Rakuten API in search_params. This time, I specified the following parameters.

For other input parameters, refer to Rakuten Books Comprehensive Search API Input Parameters version: 2017-04-04 (quoted from Rakuten Developers).

Function that extracts only the title and price parameters from the processing result of the 3-2.1 function

This function extracts only the title and price parameters from the search results processed by function 1.

When the function is called, the processing result of the function 1 is received as an argument.

In addition, the result extracted by the title and price parameters is returned as a list type.

Below is a python script.

def extract_books(result):
    item_list = [{'title': item['title'], 'price': item['price']} for item in result['Items']]
    return item_list

Loop around the book obtained by function 1 and store the result extracted by the title and price parameters in item_list as a list type. In addition, the above processing uses the inclusion notation.

A function that calls the functions 3-3.1 and 2 and describes a series of chatbot processes.

In this function, 1 and 2 functions are called to execute a series of processing required for chatbot and respond to API Gateway.

Below is a python script.

app = Chalice(app_name='Enter the API name here')
def search():
    json_text = app.current_request.json_body['text']
    keyword = json_text[json_text.find('</at>')+5:]
    keyword = html.unescape(keyword)
    result = search_books(keyword)
    text = ''
    for item in extract_books(result):
        text += f'**title**: {item["title"]}。\n\n**price**: {item["price"]}Circle\n\n------------------------------------------\n\n'
    return {'text':text}

Store the keywords entered in Teams in json_text. The content of the request from Teams to API Gateway is described with reference to CloudWatch Logs. (Chalice used to store app is explained in 4. Deploy python script to Lambda.)

When sending a keyword from Teams to API Gateway, a specific string is escaped with an html string, so The keyword is stored in keyword after setting not to escape using the unescape function.

After storing the desired information in list type in result, the process to convert list type to String type is performed.

Finally, if you return the dict type object [^ 2] whose key is set in text, the processing of the python script is completed.

[^ 2]: When responding to Teams, you need to include text in the key. If you do not include the
key, it will not be treated as text and an error will occur.

3-4. All python scripts

All the scripts required for this chatbot are listed below.

import requests
import html
from chalice import Chalice

app = Chalice(app_name='Enter the API name here')

REQUEST_URL = "https://app.rakuten.co.jp/services/api/BooksTotal/Search/20170404"
APP_ID="Enter the issued app ID here"

@app.route('/bookSearch', methods = ['POST'])
def search():
    json_text = app.current_request.json_body['text']
    keyword = json_text[json_text.find('</at>')+5:]
    keyword = html.unescape(keyword)
    result = search_books(keyword)
    text = ''
    for item in extract_books(result):
        text += f'**title**: {item["title"]}。\n\n**price**: {item["price"]}Circle\n\n------------------------------------------\n\n'
    return {'text':text}

def search_books(keyword):
    serch_params={
    "applicationId" : [APP_ID],
    "formatVersion": 2,
    "keyword" : keyword,
    "hits": 5,
    "availability": 0,
    "sort": "reviewAverage"
    "booksGenreId": "001"
    }
    response = requests.get(REQUEST_URL, serch_params)
    result = response.json()
    return result

def extract_books(result):
    item_list = [{'title': item['title'], 'price': item['price']} for item in result['Items']]
    return item_list

4. Deploy the python script to Lambda

I deployed to Lambda using AWS Chalice.

chalice is an AWS application framework that allows you to deploy code to Lambda with a single command. For more information on AWS Chalice, see What is AWS Chalice? (https://aws.amazon.com/jp/builders-flash/202003/chalice-api/) (quoted from builders.flash).

Here's how to associate your Lambda with the script you're deploying:

from chalice import Chalice
app = Chalice(app_name='Enter the API name here')
@app.route('/bookSearch', methods = ['POST'])
#Describe the processing code below

In the terminal after writing the above script

chalice deploy

You can deploy to Lambda by typing the command.

Please note that AWS Chalice requires an AWS credential file.

5. Operation check

After creating the chatbot, install it in Teams.

See below for how to deploy chatbot on Teams. AWS Lambda (Python), Amazon API Gateway and bot using Microsoft Teams Outgoing Webhooks (quoted from Yamamugi)

When you install the app on your Teams team, you will be able to launch chatbots from all channels that belong to your team.

In the actual channel, mention the chatbot app name and enter a keyword to display the book search results. image.png

Also, if you look at the logs displayed in CloudWatch Logs, you can see that you are searching with the keywords you entered. image.png

Finally

Until now, I've only used AWS services for the purpose of setting up EC2 or using ELB to distribute the load. Also, I was a beginner engineer who had only touched Java as the language.

It was a very valuable experience for me to be able to create one app using API Gateway, Lambda, Python, and external API for the first time.

By pair programming with senior employees, I was able to learn about redundant coding that I would not notice and how to code various patterns. In addition, since senior employees give advice, it is possible to avoid the phenomenon that young people, including new graduates, tend to fall into when coding does not proceed at all after worrying or investigating for a long time.

I realized that pair programming is a very meaningful educational method for the education of new graduates including myself. We encourage you to try pair programming as well.

I had a hard time suddenly using services such as python environment construction (using pipenv) and AWS Chalice that I had never even lectured on, but I learned a lot from this effort. Also, I realized that I could easily create an app by using AWS or an external API. Next, I would like to create an app by myself even if there are no senior employees.

I would like everyone to refer to this article and create various chatbots.

Recommended Posts

I tried to introduce a serverless chatbot linked with Rakuten API to Teams
I tried to make a Web API
I tried to make "Sakurai-san" a LINE BOT with API Gateway + Lambda
I tried to make a url shortening service serverless with AWS CDK
I tried to make a castle search API with Elasticsearch + Sudachi + Go + echo
I tried to make a simple image recognition API with Fast API and Tensorflow
I tried to create a table only with Django
I tried to draw a route map with Python
I tried to uncover our darkness with Chatwork API
I tried to automatically generate a password with Python3
I tried to introduce Pylint
I tried to make something like a chatbot with the Seq2Seq model of TensorFlow
I tried to implement a volume moving average with Quantx
I tried to automatically create a report with Markov chain
I tried to solve a combination optimization problem with Qiskit
I tried to get started with Hy ・ Define a class
I tried to sort a random FizzBuzz column with bubble sort.
I tried to divide with a deep learning language model
I tried to create Quip API
I tried to touch Tesla's API
[5th] I tried to make a certain authenticator-like tool with python
Rubyist tried to make a simple API with Python + bottle + MySQL
[2nd] I tried to make a certain authenticator-like tool with python
A memorandum when I tried to get it automatically with selenium
[3rd] I tried to make a certain authenticator-like tool with python
[Python] A memo that I tried to get started with asyncio
I tried to create a list of prime numbers with python
I tried to make a periodical process with Selenium and Python
I tried to make a 2channel post notification application with Python
I tried to create Bulls and Cows with a shell program
I tried to make a todo application using bottle with python
[4th] I tried to make a certain authenticator-like tool with python
[1st] I tried to make a certain authenticator-like tool with python
I tried to make a strange quote for Jojo with LSTM
How to create a serverless machine learning API with AWS Lambda
I tried to make a mechanism of exclusive control with Go
I tried to create a linebot (implementation)
I tried to implement Autoencoder with TensorFlow
I tried to create a linebot (preparation)
I tried to visualize AutoEncoder with TensorFlow
I tried to get started with Hy
I tried a functional language with Python
I tried to touch the COTOHA API
I tried to implement CVAE with PyTorch
I tried to solve TSP with QAOA
Python: I tried to make a flat / flat_map just right with a generator
I tried to delete bad tweets regularly with AWS Lambda + Twitter API
I tried to communicate with a remote server by Socket communication with Python.
I tried to implement a blockchain that actually works with about 170 lines
I tried to create a program to convert hexadecimal numbers to decimal numbers with python
I tried to create a plug-in with HULFT IoT Edge Streaming [Development] (2/3)
I tried to make a calculator with Tkinter so I will write it
I tried to draw a system configuration diagram with Diagrams on Docker
I tried to get the authentication code of Qiita API with Python.
I tried to make a traffic light-like with Raspberry Pi 4 (Python edition)
I tried to create a plug-in with HULFT IoT Edge Streaming [Execution] (3/3)
I tried to discriminate a 6-digit number with a number discrimination application made with python
I tried to send a registration completion email from Gmail with django.
[Outlook] I tried to automatically create a daily report email with Python
I tried to get the movie information of TMDb API with Python
I tried to create a plug-in with HULFT IoT Edge Streaming [Setup] (1/3)