I made a system that allows you to tweet just by making a phone call

I want to tweet, but I don't have a smartphone or computer at hand. But there are landlines. What's the matter?

Even in such a situation, we have created a system that allows you to tweet with a single phone call.

How it works

Technology to use

• Twilio
• Bing Speech API
• Twitter API
• PHP
• Python

Preparation

Do it in a Linux environment. This time I used Debian with VPS.

There is no need to install a special library, just build a LAMP environment. Please change the directory in the source code as appropriate.

It seems that Twilio can also recognize voice, but since it does not support Japanese, this time we adopted the Bing Speech API for voice recognition.

Preparing Twilio

Just get an account. There are paid accounts and free accounts (trial is limited), but in this case we were able to implement it with a free account.

This area will be helpful for account acquisition and mechanism. Twilio First Steps

Preparing the Bing Speech API

You can get it from the here page.

You can get the key immediately by agreeing to the terms of use from "Getting API Key" and logging in. You can also log in with your Github account.

--Key endpoint (https://api.cognitive.microsoft.com/sts/v1.0) --Key 1 (32-digit alphanumeric characters) --Key 2 (32-digit alphanumeric characters)

Two types of keys will be issued, but it seems that either one can be used.

program

Twilio's processing is written in TwiML and PHP, and other processing is written in Python.

Write TwiML

Place the following xml in a location that can be accessed from the Internet, such as a web server. Then set the URL from your twilio account page.

record.xml

<?xml version="1.0" encoding="UTF-8"?>
<Response>
<Say language="ja-JP">Record the content of the tweet. After recording, press Sharp.</Say>
<Record method="GET" timeout="60" maxLength="30" finishOnKey="#" action="./get.php" />
</Response>

These few lines will automatically speak and even record. TwiML It's good because it's easy to understand and easy. See Official Reference for more information.

Write PHP

This is the part of ʻaction ="./get.php "written inof TwiML. The process is passed toget.php` after the call is closed.

get.php

<?php

//Get the recording file URL
$recordingUrl = $_GET['RecordingUrl'];
$recordingUrl .= ".wav";

//Download recording file
exec("wget -N -O ./record.wav $recordingUrl");

//Run python
exec("python ./tweet.py");

?>

As you can see in the reference, RecordingUrl is sent as a parameter from the ʻactionattribute. You can receive it by GET or POST. It's the same as HTML and other forms. This time, I specified GET withmethod` of TwiML, so it is GET.

Note that parameters may not be obtained or audio may not be saved depending on the permission settings of the file or save destination directory. I was worried for a few days because I couldn't solve the super-super rudimentary permission-related problems.

After downloading the recorded voice file, execute the python program to perform voice recognition → tweet processing.

Write python

This article helped me create a Python program. How to convert Twilio's voice into text using Microsoft's Bing Speech API

First, create settings.py to refer to the API key for tweets.

settings.py

CONSUMER_KEY = "xxxxxxxxxxxxxxxxxxxxxxxxx"
CONSUMER_SECRET = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
ACCESS_TOKEN = "xxxxxxxxxxxxxxxxxx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
ACCESS_TOKEN_SECRET = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

Next is a program that performs everything from voice recognition to tweets.

tweet.py

#coding:utf-8
from requests_oauthlib import OAuth1Session
import json
import settings
import requests
import urllib

#Get token from API
def authorize():

    url = "https://api.cognitive.microsoft.com/sts/v1.0/issueToken"

    headers = {
        "Content-type": "application/x-www-form-urlencoded",
        "Ocp-Apim-Subscription-Key": "Obtained key"
    }

    response = requests.post(url, headers=headers)

    if response.ok:
        _body = response.text
        return _body
    else:
        response.raise_for_status()

#Store the acquired token
token = authorize()

#File selection
infile = open("record.wav","r")
data = infile.read()

#Throw to API and get results
def speech_to_text( raw_data, token, lang="ja-JP", samplerate=8000, scenarios="ulm"):
    data = raw_data
    params = {
        "version": "3.0",
        "requestid": "b2c95ede-97eb-4c88-81e4-80f32d6aee54",
        "appid": "D4D52672-91D7-4C74-8AD8-42B1D98141A5",
        "format": "json",
        "locale": lang,
        "device.os": "Windows",
        "scenarios": scenarios,
        "instanceid": "565D69FF-E928-4B7E-87DA-9A750B96D9E3"
    }

    url = "https://speech.platform.bing.com/recognize?" + urllib.urlencode(params)
    headers = {"Content-type": "audio/wav; samplerate={0}".format(samplerate),
               "Authorization": "Bearer " + token }

    response = requests.post(url, data=data, headers=headers)

    if response.ok:
        result = response.json()["results"][0]
        return result["lexical"]
    else:
        raise response.raise_for_status()

#Store the data acquired by API
message = speech_to_text(data,token,lang="ja-JP", samplerate=8000, scenarios="ulm")

#settings.Get Token from py
twitter = OAuth1Session(settings.CONSUMER_KEY, settings.CONSUMER_SECRET, settings.ACCESS_TOKEN, settings.ACCESS_TOKEN_SECRET)

#Tweet
params = {"status":"%s"%message}
req = twitter.post("https://api.twitter.com/1.1/statuses/update.json",params = params

Try to call

When you call your Twilio number, you'll be told "Record your tweet. Press Sharp after recording." Tell them what you want to tweet. It will be tweeted after a while.

It doesn't seem to work if it's too long or too short.

Challenges point

It worked for the time being, but there are some issues.

1. Somehow an error occurs in Twilio (Added on October 26, 2017)

At the end of the call, it says "An application error has occurred". Twilio's debugger also shows an error, but it's not resolved.

→ I solved it, so I wrote an article. About system errors that allow you to tweet just by calling

2. Setting permissions

This time I was quite addicted to the rudimentary permission settings. There are some parts that I haven't grasped yet, so study is required.

3. Security story

I feel that it is not security-friendly, such as forcibly executing it with ʻexec ()` during the program. (Command injection) After that, it is necessary to devise PHP data transfer as a countermeasure for CSRF.

I want to restrict access to TwiML, but I wonder if it can be done.

4. Recorded files remain on Twilio (Added on October 27, 2017)

Recording files will remain on Twilio and will accumulate each time you tweet. It seems that it can be deleted with the API, so it seems that this problem can be solved.

→ I wrote a program for deletion, so I made it an article. Delete all recorded data with Twilio