The story of creating Botonyan that returns the contents of Google Docs in response to a specific keyword on Slack

Overview

--I made Botonyan that returns the contents of Google Docs in response to a specific keyword in Slack: https://github.com/supistar/Botnyan ――The line breaks included in the text will be reflected properly! --Born from the Ingress community. Try using it regardless of RES / ENL! : ghost:

Background of the matter

Hello spin I NYA. People on the side and people on the side often write Python, so when I started writing it, I got hooked and made Slackbot with Python + Flask.

https://github.com/supistar/Botnyan

Ingress and Slack

It all started with Ingress ^ 1 I think that many people already know Ingress, so I will omit the details, but due to the characteristics of the game, it is important for players in a specific area to form a community and interact and develop operations through that community. .. In this community, some communities are now migrating to Slack. See here's Blog [^ 2] for # migration to Slack.

Slack is the first step to joining the community

As the move to Slack progressed, there was a lot of trial and error to make the community more convenient. One of them is a guidance message when joining the community.

Slack has a special aspect, and one big barrier for users is that their applications are English-based. For engineers, English is a world before breakfast, but just as agents are not engineers, it is not always the case that everyone is accustomed to English.

Higher barriers to community participation reduce overall participation, hindering community formation and ruining the purpose of the move to Slack in the first place.

Therefore, in order to lower that barrier, we have prepared a manual with screenshots and a document with tips, and improvements have been made to guide new members. This has created tutorials and leads for first-time users, making it easier for them to participate.

Surprisingly difficult Slackbot

However, this guidance message was done manually at first

--Forget the content of the text ――In the first place, the template of the text Where was the case (it does not appear even if you search)

Due to frequent problems such as, the role has moved to Slackbot, which is provided from the beginning in Slack. This made the guidance message automated, but this bot is quite a songwriter ...

--The set keyword is an exact match (excluding delimiters such as punctuation marks and symbols) --Cannot output line breaks --It doesn't mean that all the contents will be expanded as if you could paste the URL (due to Slack specifications, if the URL is the same, it will be cached and the contents will not be expanded). --Unexpectedly, it is difficult to correct the text registered in Slackbot

Botnyan was created from the point that" I want to solve this somehow ... ".

Introducing Botnyan

Botnyan is written based on Python + Flask and runs on Slack as a bot triggered by Outgoing-Webhook.

slack.png

The operation is very simple:

--Outgoing-If the post contains the keyword specified in Webhooks, access the specified REST Endpoint (Botnyan). --Botnyan accesses Google Docs associated with the specified keyword --Botnyan <-> Google Docs connects through a service account. Get the contents of the document in text file format --Speak on the channel that responded to the keyword with the acquired content

It has become.

Since its introduction, it has been very well received by fellow agents.

--Since the content can be updated on Google Docs, anyone with authority can edit + history management --You can send messages with line breaks ** (this is important) **

After creating it, I heard that the neighboring community has a similar problem, so I decided to publish what was managed in the Private repository as a public source. At first, I made it work in Apache + WSGI environment, but I made some modifications from the original so that it can also work on Heroku.

We have prepared README-jp for the installation method, so please have a look there as well! https://github.com/supistar/Botnyan/blob/master/README-jp.md

Technical story

Here, I will briefly describe the technical contents used in Botonyan.

1. Limit the Endpoints published in Flask

Flask allows you to limit access to Endpoints by using several decorators.

Outgoing-Webhooks in Slack will access the Endpoint specified by POST + ʻapplication / x-www-form-urlencoded. Therefore, if you want to limit it to only these, do as follows. Since Cross-Origin is also allowed here, add @cross_origin ()` as well.

python


@slack.route("/webhook", methods=['POST'])
@cross_origin()
@consumes('application/x-www-form-urlencoded')
def webhook():
    ~~~

2. Outgoing-Slightly strengthened the security of Endpoint used by Webhooks

Outgoing-Webhooks must have an Endpoint that can access the Public. Basic authentication cannot be applied, so if you create an Endpoint without being aware of that,

  1. Heroku application name revealed
  2. Enter an appropriate keyword
  3. If the keywords match, the contents of the document will be leaked! Important information for the other party! !! !!

That could be the case (´ ・ ω ・ `)

However, Slack's Outgoing-Webhooks grants tokens to the request. First, let's check the tokens that Outgoing-Webhooks will give you. Take a look at Slack's Integrations at the bottom.

01.png

It's perfect with this token. Then, store this token on the application side and compare it with the token included in the actual request. If the token is not set on the application side, or if it is different from the token being requested, return an error with ʻabort (401)`.

form = request.form
request_token = Utils().parse_dic(form, 'token', 400)
token = os.environ.get('SLACK_WEBHOOK_TOKEN')
if not token or token != request_token:
    abort(401)

3. How to manage the private key of the service account

To be honest, I was most worried about this: fearful: It would be easier if I put the p12 file directly into the repository, but I rejected it immediately because I made it Public. The alternative method is to put the private key in the Config Variables.

First, take out the private key from the p12 file

cd path/to/p12directory
openssl pkcs12 -passin pass:notasecret -in privatekey.p12 -nocerts -passout pass:notasecret -out key.pem
openssl pkcs8 -nocrypt -in key.pem -passin pass:notasecret -topk8 -out google-services-private-key.pem
rm key.pem
# google-services-private-key.pem is the private key! You did it!

Set this to Config Variables.

# heroku-Using toolbelt...This way
heroku config:add GOOGLE_PRIVATE_KEY=`cat path/to/p12directory/google-services-private-key.pem`

From the Python code side, access it with ʻos.environ. The rest is the same as reading from a p12 file. If the private key is not set, it is easier to understand by calling ʻabort () to return a specific status code.

private_key = os.environ['GOOGLE_PRIVATE_KEY']
if not private_key:
    abort(401)
credentials = SignedJwtAssertionCredentials(os.environ['GOOGLE_CLIENT_EMAIL'],
                                            private_key,
                                            'https://www.googleapis.com/auth/drive',
                                            sub=os.environ['GOOGLE_OWNER_EMAIL'])
http = httplib2.Http()
credentials.authorize(http)
service = build('drive', 'v2', http=http)

4. Get the contents of the document from the GoogleDocs Document ID

It's the core of Botnyan. Get the file using the service instance and Document ID created in (3).

However, if you simply get it, an office format file will come down, so it is difficult to use. So let's get it in a file format of text / plain. If you do the following, the contents of the document will be stored as a string in content.

f = service.files().get(fileId=doc_id).execute()
if 'exportLinks' in f and 'text/plain' in f['exportLinks']:
    download = f['exportLinks']['text/plain']
    resp, content = service._http.request(download)
else:
    content = 'Failed to read'

5. Make the bot speak the acquired content

This is very easy. It just returns a JSON response similar to the following in response to an Outgoing-Webhooks request.

{"text": "It's the content of the document! ∧_∧"}

It just returns JSON using the content obtained in (4). Let's specify ʻapplication / json` for the Content-Type of the response.

dic = {"text": content}
return Response(Utils().dump_json(dic), mimetype='application/json')

Remaining challenges

That doesn't mean that everything is solved.

--Outgoing-Webhook currently can't pick up PrivateRoom remarks --It seems that you can pick it up by using hubot's slack adaptor (RTM), but if you use this, the line breaks in the document will not work ... --Why don't you publish it as a hubot plugin? ――I just wanted to write in Python! I was not reflecting! (Lol)

So

Which side of the agent am I ... without here: ghost: I think that communication problems are not only with the Ingress team but also with work, so if you like it, please use it.

I would be very happy if it helps! So! : cat2:

Reference site

Recommended Posts

The story of creating Botonyan that returns the contents of Google Docs in response to a specific keyword on Slack
The story of creating a bot that displays active members in a specific channel of slack with python
A story about creating a program that will increase the number of Instagram followers from 0 to 700 in a week
The story of creating a database using the Google Analytics API
Now in Singapore The story of creating a LineBot and wanting to do a memorable job
A story that struggled to handle the Python package of PocketSphinx
The story of creating a site that lists the release dates of books
Create a function to get the contents of the database in Go
[Python] A program that rotates the contents of the list to the left
[Google Photo & Slack Photo Bot] A story about making a bot that acquires photos in Google Photos and sends them to Slack.
How to run the practice code of the book "Creating a profitable AI with Python" on Google Colaboratory
The story of creating a store search BOT (AI LINE BOT) for Go To EAT in Chiba Prefecture (1)
How to copy and paste the contents of a sheet in Google Spreadsheet in JSON format (using Google Colab)
Create a bot that posts the number of people positive for the new coronavirus in Tokyo to Slack
How to use the Slack API using Python to delete messages that have passed a certain period of time for a specific user on a specific channel
A simple mock server that simply embeds the HTTP request header in the body of the response and returns it.
The story of creating a store search BOT (AI LINE BOT) for Go To EAT in Chiba Prefecture (2) [Overview]
How to mention a user group in slack notification, how to check the id of the user group
The story of IPv6 address that I want to keep at a minimum
How to access the contents of a Linux disk on a Mac (but read-only)
The story of Django creating a library that might be a little more useful
The story of Linux that I want to teach myself half a year ago
[Python] Change the text color and background color of a specific keyword in print output
A story of trial and error trying to create a dynamic user group in Slack
A script that transfers tweets containing specific Twitter keywords to Slack in real time
A story about trying to introduce Linter in the middle of a Python (Flask) project
Posted the number of new corona positives in Tokyo to Slack (deployed on Heroku)
A story that reduces the effort of operation / maintenance
# Function that returns the character code of a string
A story that analyzed the delivery of Nico Nama.
A server that returns the number of people in front of the camera with bottle.py and OpenCV
Create a bot that only returns the result of morphological analysis with MeCab on Discord
The story of creating a "spirit and time chat room" exclusively for engineers in the company
Get the value of a specific key up to the specified index in the dictionary list in Python
Summary of points to keep in mind when writing a program that runs on Python 2.5
[Python] Programming to find the number of a in a character string that repeats a specified number of times.
[Note] A shell script that checks the CPU usage of a specific process in a while loop.
A story that makes it easier to see Model debugging in the Django + SQLAlchemy environment
A story that contributes to new corona analysis using a free trial of Google Cloud Platform
Use Heroku in python to notify Slack when a specific word is muttered on Twitter
How to easily draw the structure of a neural network on Google Colaboratory using "convnet-drawer"
I tried to make a script that traces the tweets of a specific user on Twitter and saves the posted image at once
The story of creating a VIP channel for in-house chatwork
[Ubuntu] How to delete the entire contents of a directory
A note on the default behavior of collate_fn in PyTorch
Django returns the contents of the file as an HTTP response
Get the number of specific elements in a python list
A Python script that compares the contents of two directories
How to connect the contents of a list into a string
I will publish a shell script created to reduce the trouble of creating LiveUSB on Linux
A story that is a little addicted to the authority of the directory specified by expdp (for beginners)
Yield in a class that inherits unittest.TestCase didn't work with nose (depending on the version of nose?)
A story that didn't work when I tried to log in with the Python requests module
The story of making a tool that runs on Mac and Windows at the game development site
[Python scraping] Output the URL and title of the site containing a specific keyword to a text file
[Python] About creating a tool to create a new Outlook email based on the data of the JSON file and the part that got caught
A memo that reproduces the slide show (gadget) of Windows 7 on Windows 10.
Process the contents of the file in order with a shell script
The story that the version of python 3.7.7 was not adapted to Heroku
pandas Fetch the name of a column that contains a specific character
How to check the memory size of a variable in Python