I made a Mattermost bot with Python (+ Flask)

Caution

The article was asleep for about 9 months, kept in the draft, so the information may be out of date. .. .. Since I wrote it with much effort, I will post it as a memorial service. ..

background

It was decided to move the internal communication tool to Mattermost. I wish I could migrate my existing bot to Mattermost, so I also reimplemented Bot for Mattermost in Python [^ 1] for my own study. I've researched a lot when implementing the bot, so I'll summarize the information for myself.

environment

--CentOS 7.7 (Mattermost server) --Ubuntu 18.04 LTS (Bot server)

• Mattermost 5.14.0
• Python 3.6.8

Somehow I felt like I wanted to touch both CentOS and Ubuntu. Ubuntu 18.04 comes with Python 3 by default, so I tried using it. In short, it doesn't mean anything.

Environment

Introduction of Mattermost

Mattermost can be easily installed with Docker, so install Docker first.

CentOS

#Install Docker
$ yum install -y yum-utils device-mapper-persistent-data lvm2
$ yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
$ yum install docker-ce docker-ce-cli containerd.io

#Installation confirmation
$ docker --version
Docker version 19.03.5, build 633a0ea
$

#Start Docker
$ systemctl status docker
$ systemctl enable docker
$ systemctl start docker
$ systemctl status docker

#Operation check
$ docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
1b930d010525: Pull complete
Digest: sha256:4df8ca8a7e309c256d60d7971ea14c27672fc0d10c5f303856d7bc48f8cc17ff
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

$

Once you have Docker installed, you can install and start Mattermost below.

$ docker run --name mattermost-preview -d --publish 8065:8065 --add-host dockerhost:127.0.0.1 mattermost/mattermost-preview

Once installed, it can be started / stopped as follows.

#Start-up
$ docker start mattermost-preview

#Stop
$ docker stop mattermost-preview

#Process confirmation
$ docker ps

■ Reference ・ Qiita: Install Docker on CentOS7 -Docker docs: Get Docker Engine --Community for CentOS

Mattermost settings

Once Mattermost is up and running, you can access it from your browser.

http://xxx.xxx.xxx.xxx:8065

If you can access it, create an administrator account. Any email address is OK.

Create a team appropriately.

Translate to Japanese.

Main Menu -> Account Settings -> Display -> Language

By default, connection restrictions are applied to private networks. If the server to which the Webhook communicates is in the same segment, it is necessary to set the communication permission. (I was quite addicted here.)

Main Menu-> System Console-> Developers-> Allow Untrusted Internal Connections

By default, you cannot create a bot account, so enable it.

Main Menu-> System Console-> Bot Accounts-> Enable Bot Account Creation

Create an outgoing webhook.

Main Menu-> Integration-> Outward Web Hook

Content type: application / json (I chose JSON this time) Channel: Channel that monitors the trigger Trigger word: A word that triggers the activation of the Webhook Callback URL: URL of the message destination (Bot server) (Since Flask is used this time, the default port 5000 is specified)

Create a bot account.

Main Menu-> Integration-> Bot Account Bot post permissions are set here.

Be sure to make a note of the Bot account token as there is no way to check it later on the menu screen. However, even if you forget it, you can check it with the following command.

$ docker exec -t -e MYSQL_PWD=mostest mattermost-preview mysql -u mmuser mattermost_test -e "select * from UserAccessTokens;"
+----------------------------+----------------------------+----------------------------+-------------+----------+
| Id                         | Token                      | UserId                     | Description | IsActive |
+----------------------------+----------------------------+----------------------------+-------------+----------+
| xxxxxxxxxxxxxxxxxxxxxxxxxx | xxxxxxxxxxxxxxxxxxxxxxxxxx | xxxxxxxxxxxxxxxxxxxxxxxxxx | ?????????   |        1 |
+----------------------------+----------------------------+----------------------------+-------------+----------+
$

Check if you can post from your bot account with the curl command. First, check the channel ID of the posting destination.

Channel-> Show information

You can also check with the following command.

$ docker exec -t -e MYSQL_PWD=mostest mattermost-preview mysql -u mmuser mattermost_test -e "select * from Channels;"
+----------------------------+---------------+---------------+----------+----------------------------+------+-------------+--------------------------------------------------------+--------+---------+---------------+---------------+---------------+-----------+----------+------------------+
| Id                         | CreateAt      | UpdateAt      | DeleteAt | TeamId                     | Type | DisplayName | Name                                                   | Header | Purpose | LastPostAt    | TotalMsgCount | ExtraUpdateAt | CreatorId | SchemeId | GroupConstrained |
+----------------------------+---------------+---------------+----------+----------------------------+------+-------------+--------------------------------------------------------+--------+---------+---------------+---------------+---------------+-----------+----------+------------------+
| xxxxxxxxxxxxxxxxxxxxxxxxxx | 1574487544576 | 1574487544576 |        0 | xxxxxxxxxxxxxxxxxxxxxxxxxx | O    | Off-Topic   | off-topic                                              |        |         | 1574487544652 |             0 |             0 |           | NULL     |             NULL |
| xxxxxxxxxxxxxxxxxxxxxxxxxx | 1574489265292 | 1574489265292 |        0 |                            | D    |             | xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx |        |         | 1574489265315 |             1 |             0 |           | NULL     |             NULL |
| xxxxxxxxxxxxxxxxxxxxxxxxxx | 1574487544570 | 1574487544570 |        0 | xxxxxxxxxxxxxxxxxxxxxxxxxx | O    | Town Square | town-square                                            |        |         | 1574487544611 |             0 |             0 |           | NULL     |             NULL |
+----------------------------+---------------+---------------+----------+----------------------------+------+-------------+--------------------------------------------------------+--------+---------+---------------+---------------+---------------+-----------+----------+------------------+
$

Specify the channel ID and Bot token, and make a POST request to the API with the curl command.

$ curl -i -X POST -H 'Content-Type: application/json' -d '{"channel_id":"[Channel ID]", "message":"This is a message from a bot"}' -H 'Authorization: Bearer [Bot Token]' http://xxx.xxx.xxx.xxx:8065/api/v4/posts

I was able to confirm that I was able to post with my Bot account.

■ Reference ・ Usual Software Engineer: How to create a Bot Account with Mattermost

Response to webhook

It seemed easy to do it in Flask to do it in Python, so I implemented it in Flask. I'm just POSTing JSON to the API as I do with curl.

Ubuntu

$ sudo apt install python3-pip
$ pip3 -V
pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.6)
$
$ pip3 install flask
$ pip3 install requests

bottest.py

import json

import requests
from flask import Flask, request

BOT_TOKEN = '[Bot Token]'
CHANNEL_ID = '[Channel ID]'
MM_API_ADDRESS = 'http://xxx.xxx.xxx.xxx:8065/api/v4/posts'

app = Flask(__name__)

@app.route('/bot-test', methods=['POST'])  #Outgoing webhook callback URL
def bot_reply():
    posted_user = request.json['user_name']
    posted_msg  = request.json['text']

    reply_headers = {
        'Content-Type': 'application/json',
        'Authorization': 'Bearer ' + BOT_TOKEN,
    }

    reply_data = {
        "channel_id": CHANNEL_ID,
        "message": f"@{posted_user} Bot reply message.",
        "props": {
            "attachments": [
                    {
                "author_name": posted_user,
                "text": posted_msg,
                }
            ]
        },
    }

    reply_request = requests.post(
        MM_API_ADDRESS,
        headers = reply_headers,
        data = json.dumps(reply_data)
    )

    return reply_request

if __name__ == '__main__':
    app.debug = True
    app.run(host='0.0.0.0')

Test if you can get a response. Start Flask's test server.

Ubuntu

$ python3 bottest.py

Post a message containing a trigger word in Mattermost.

I was able to confirm that I was able to post from my Bot account using the Webhook as a trigger. All you have to do is implement the bot logic.

By the way, the JSON that comes from Mattermost's Webhook looks like this.

{
    'token': 'xxxxxxxxxxxxxxxxxxxxxxxxxx', 
    'team_id': 'xxxxxxxxxxxxxxxxxxxxxxxxxx', 
    'team_domain': 'bot-test', 
    'channel_id': 'xxxxxxxxxxxxxxxxxxxxxxxxxx', 
    'channel_name': 'town-square', 
    'timestamp': 1234567890123, 
    'user_id': 'xxxxxxxxxxxxxxxxxxxxxxxxxx', 
    'user_name': 'mmadmin', 
    'post_id': 'xxxxxxxxxxxxxxxxxxxxxxxxxx', 
    'text': '#bot test', 
    'trigger_word': '#bot', 
    'file_ids': ''
}

■ Reference ・ [NOBWAK'S LAIR: Let's make BOT of SLACK / MATTERMOST with FLASK of PYTHON](http://april.fool.jp/blogs/2016/06/02/slackmattermost%E3%81%AEbot%E3%82%92python % E3% 81% AEflask% E3% 81% A7% E4% BD% 9C% E3% 82% 8D% E3% 81% 86 /) -Qiita: Make a simple bot with Mattermost and Python + Flask

bonus

Once the bot is complete, you'll need a web server and a WSGI server to run it as a web application. After a lot of research, I chose Nginx + Gunicorn. (Because I felt it was the simplest and easiest) This area is still unclear, so I'll just rush to write it down as a memo. ..

Gunicorn is installed with pip.

Ubuntu

$ sudo pip3 install gunicorn

gunicorn [Python script name]: Launched with [Flask instance name]. (Default port is 8000) (The following launches an app instance in bottest.py)

#Run in the directory where the Python script is located
$ cd /path/bot
$ sudo gunicorn bottest:app
[YYYY-MM-DD hh:mm:ss +0900] [1699] [INFO] Starting gunicorn 20.0.2
[YYYY-MM-DD hh:mm:ss +0900] [1699] [INFO] Listening at: http://127.0.0.1:8000 (1699)
[YYYY-MM-DD hh:mm:ss +0900] [1699] [INFO] Using worker: sync
[YYYY-MM-DD hh:mm:ss +0900] [1703] [INFO] Booting worker with pid: 1703

If you want to use the same port as Flask's test server, OK (When it is troublesome to change the Webhook settings on the Mattermost side)

$ sudo gunicorn bottest:app -b 0.0.0.0:5000

You can use the chdir option without having to go to the location of the Python script.

$ sudo gunicorn --chdir /path/bot bottest:app -b 0.0.0.0:5000

When stopped, press Ctrl + C to kill the process.

If you want to run Gunicorn alone, this is fine, but you can't work with Nginx as it is. First, install Nginx.

Ubuntu

$ sudo apt install nginx
$ nginx -v
nginx version: nginx/1.14.0 (Ubuntu)
$

Set Nginx. Socket is required for communication with Gunicorn. (Created in / tmp below)

/etc/nginx/sites-available/default

upstream bot-test {
    server unix:/tmp/bot-test.sock;
}

server {
    listen 5000;

    root path/bot;
    server_name localhost;

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    location /bot-test {
        try_files $uri @flask;
    }

    location @flask {
        proxy_pass http://bot-test;
    }
}

Operation check of Nginx.

$ sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
$
$ sudo systemctl status nginx
$ sudo systemctl restart nginx
$ sudo systemctl status nginx

If there is no problem with Nginx settings, create a Gunicorn configuration file. The location and name of the configuration file can be anything Describe the Socket described in the Nginx configuration file earlier.

$ vim gunicorn_conf.py

gunicorn_conf.py

bind = 'unix:/tmp/bot-test.sock'
daemon = True
reload = True

Specify the configuration file with the -c option and OK if it can be started

$ sudo gunicorn bottest:app -c gunicorn_conf.py

When stopped, kill the process.

$ sudo pkill gunicorn

To be honest, I'm doing the extra part with the spirit of moving for the time being, so There must be a better way. ..

■ Reference ・ Qiita: gunicorn + Flask + nginx + Systemd ・ [Diary of "Technical college student who entered the wrong department": Procedure to manually deploy Ubuntu + Nginx + Gunicorn](https://nnsnodnb.hatenablog .jp / entry / ubuntu-nginx-gunicorn-deploy) -Qiita: A memo to daemonize python

[^ 1]: Reinventing the wheel https://www.slideshare.net/RansuiIso/python-115121978 https://www.youtube.com/watch?v=kO4FNg648qE