The story of making a box that interconnects Pepper's AL Memory and MQTT

It seems that you read the other day Aggregating the status confirmation and operation of IRKit and Philips Hue with MQTT, and after all IoT is hot. I have a feeling.

So this time, let's talk about MQTT with Pepper. This time, I felt that ** "If you interconnect the memory (AL Memory) mechanism and MQTT, you can easily write cloud cooperation or external cooperation application", so I made a box like that **.

As for handling MQTT with Pepper, Suna has already tried it, so I used it as a reference. It is very carefully organized, so please refer to here for the technical background and elements.

-Make MQTT Publisher Box -Make an MQTT Publisher Box Part 2 -Make MQTT Subscriber Box

By the way, there are some personal tips such as how to easily distribute the one that uses the module in the Python box with the box library. I'm doing it as an individual development to the last, but since I've done it all, I feel like taking notes on how to use it and how to think about it.

With ALMemory, MQTT and Pub / Sub models

NAOqi is the software foundation of Pepper, and at the center of this framework is a module called ** ALMemory **. It supports firing events, subscribing to them, setting and retrieving values, etc., and monitoring sensors and retrieving values from applications is basically via this module. It is designed to do so. (For details, see [Events and value acquisition via ALMemory](http://qiita.com/Atelier-Akihabara/items/4162192129f366da1240#almemory%E3%82%92%E4%BB%8B%E3%81%97] % E3% 81% 9F% E3% 82% A4% E3% 83% 99% E3% 83% B3% E3% 83% 88% E5% 80% A4% E3% 81% AE% E5% 8F% 96% E5 See around% BE% 97))

On the other hand, ** MQTT **, as you can see from Mr. Shiguredo's Learn more about MQTT, various devices issue messages and It is a protocol that allows you to subscribe to messages issued by another device. Various devices can publish and subscribe to messages through what is called an MQTT Broker.

These models, such as issuing messages and subscribing to them, are called Pub / Sub models, but ALMemory and MQTT are ** the same model, and I'm sure they have a high affinity **. That's why.

Pepper and cloud

I helped develop with Pepper at events, etc., and the feeling that "By linking Pepper with various external services, not only Pepper itself but also the value of the service will be further enhanced" becomes stronger and stronger. I will.

Each service provides a very convenient client library, but it is quite difficult to use these libraries directly from the box library provided by Pepper's development tool Choregraphe, and in many cases it is a wall of coding. I can't avoid it. Also, although NAOqi is a Python-based execution environment, it is an environment where coding is easy because you cannot freely use package management tools such as pip, and there are things like "NAOqi-specific manners". The current situation is that this is not the case.

Moreover, although Pepper's CPU and memory have been strengthened for general sale, it is necessary to use computing power for autonomous control such as safety and parts such as emotion maps, and it is also possible to circulate calculations with the application. There seems to be a problem. It seems that heavy calculations such as recognition processing will need to be considered in the cloud or on a server somewhere outside Pepper.

MQTT Adapter Box: ALMemory-MQTT Interconnect Pepper-Cloud Integration

That's why I thought that by interconnecting ALMemory and MQTT, it would be possible to easily link with an external system such as the cloud with just the box that handles ALMemory.

image

In the MQTT Adapter box, specify ** the association between the key in ALMemory and the MQTT topic ** in advance. By giving this association and connection information to MQTT Broker, all you have to do is start the MQTT Adapter box.

--If your app fires an event that corresponds to an MQTT topic in the Raise Event box, you can send a message to an external application that subscribes to that topic via MQTT. --If the app subscribes to an event that corresponds to an MQTT topic, such as in the Subscribe to Event box, you can receive messages issued by the external application for that topic via MQTT.

I think it would be interesting to be able to move like this.

MQTT Adapter Box: How to use

That's why I made the MQTT Adapter box.

I'm trying to add it to the same box library as the previous Subscribe to Event box rewritten with qi Framework. GitHub is below

The ** web-boxes ** directory is the box library. I will explain how to load and use the box library.

Loading the box library

To place an MQTT Adapter box in your app, you must first open the box library in Choregraphe. Do the following:

  1. Select Open Box Library from the Box Library panel

    BoxLib_Open.png

  2. Select the ** web-boxes directory ** from the directories ZIP downloaded or cloned from GitHub.

    BoxLib_OpenDir.png

Note that you must select Box Library Directories as the file type.

  1. ** web-boxes Box Library ** will now appear in the Box Library panel

    BoxLib_Loaded.png

You can now use the web-boxes box library, which includes the MQTT Adapter box.

MQTT Adapter box placement

The Network> MQTT Adapter box in the web-boxes box library is used as follows.

  1. Drag and drop the Network> MQTT Adapter box from the web-boxes box library onto the Flow Diagram panel.

    MQTTAdapter_DnD.png

This MQTT Adapter box uses a module called paho, which is automatically under project when you drag and drop the MQTT Adapter box. It is designed to be imported into.

  1. Open the MQTT Adapter box settings dialog and set the following parameters The following values can be set. --Host ... MQTT Broker host name --Port ... MQTT Broker port number --Username ... MQTT Broker connection username --Password ... MQTT Broker connection password --Key-Topic Bindings ... Correspondence between ALMemory keys and MQTT topics. Details will be described later

    MQTTAdapter_Settings.png

  2. Make sure to start the MQTT Adapter box at the right time --Start interconnection between MQTT and ALMemory with onStart input ... After executing connection processing, onConnected output is fired. After that, event relay between ALMemory and MQTT will be executed until onStop input is input. --The onStop input ends the interconnection ... When the connection ends, the onStopped output is fired. --When an error occurs ... onError Output is fired with a message

The details of the definition method and the execution example will be explained below.

Key-Defining topic correspondence

The Key-Topic Bindings setting in the MQTT Adapter box allows you to specify which key value in ALMemory to be associated with which topic in MQTT.

Correspondences are specified by separating them with the key of ʻALMemory: topicof MQTT, etc., separated by a colon. You can define multiple correspondences by separating them with;`.

By adding / # to the end of both the key name and topic name, you can specify the following key topics at once. For example

--If you say TestMQTT / Hoge / #: test-yacchin1205 / Hoge / # etc., if you issue an event with a name starting with TestMQTT / Hoge / in ALMemory, for example TestMQTT / Hoge / A, MQTT will be notified. A message will be issued to the topic test-yacchin1205 / Hoge / A --You can subscribe to the topic test-yacchin1205 / Hoge / B in MQTT by subscribing to the event TestMQTT / Hoge / B in ALMemory.

You can also define value conversions by setting ʻALMemory key: MQTT topic: type`, etc.

--If you do not specify the type, the value will not be converted during the event conversion of ALMemory-MQTT, and the value will be sent with the same type. Therefore, if the event has a value of a type that does not correspond to the MQTT payload, it may fail to publish. --If the type is str, it will be converted to the character string type at the time of ALMemory → MQTT conversion. The same applies to MQTT → AL Memory --If the type is json, dump to json at the time of ALMemory → MQTT conversion. Json analysis is performed when converting from MQTT to ALMemory.

Below is an example of how to use it ...

Passing a value from MQTT to Pepper Example: Say instructions to Pepper from another device

You can throw a value from another device into Pepper's ALMemory via MQTT Adapter and use it to do something like Say Text. See ** samples / mqtt-say-text ** on GitHub for samples.

  1. Place and set the MQTT Adapter box. This time as an example --Use test.mosquitto.org for MQTT Broker --Set the MQTT topic test-yacchin1205 / sample1 / Message to be interconnected with ALMemory MQTT / Test / Sample1 / Message

    mqtt-say-text_settings.png

  2. Create an input to subscribe to the MQTT / Test / Sample1 / Message event in the flow diagram and connect it to the Say Text box

    mqtt-say-text_flow.png

As a test, with this app running, try throwing a message to MQTT with the mosquitto_pub command installed on your Raspberry Pi. For example, if you throw a test-yacchin1205 / sample1 / Message topic (ALMemory in MQTT / Test / Sample1 / Message) to the message Hello from `` MQTT is as follows.

pi@raspberrypi ~ $ mosquitto_pub -h test.mosquitto.org -t test-yacchin1205/sample1/Message -Hello from m MQTT

The Say Text box will then respond via ALMemory.

mqtt-say-text_robotview.png

By going through MQTT in this way, you can easily throw instructions to Pepper from outside Pepper.

Passing a value from Pepper to MQTT Example: Control Hue from Pepper

As introduced in Aggregating the status confirmation and operation of IRKit and Philips Hue with MQTT, Hue at my home can be operated via MQTT. ..

This time, I will write an example that controls the lighting of the Hue lamp by stroking Pepper's head.

Correspondence between MQTT and Hue

At home, ʻusername / hue / xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx (bridge UDN) / light / x (light ID) / status` By throwing JSON data to the topic, the lighting status of the light can be changed as follows. I am trying to control it.

username/hue/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx/light/x/status {"on": true, "saturation": 254, "brightness": 204, "hue": 34297}

Here, if you give a value related to the lighting state of Hue to ALMemory, you want to change the state of Hue, but if you give such JSON data (dict), in ALMemory [[" on ", True] , ["saturation", 254], ["brightness", 204], ["hue", 34297]] is automatically converted.

Creating an app

Create the app as follows. Information such as ID and password will be included, so only an overview ...

  1. Place and set the MQTT Adapter box --Use Sango for MQTT Broker. Use the information specified by Sango for the host, port, username and password --Since we are trying to publish Hue events under the username / hue / topic, set all these topics to interconnect with keys starting with MQTT / Hue / in ALMemory. The format is json

    pepper-hue_settings.png

  2. Place Sensing> Touch> Tactile Head box and new Python box (Hue On, Hue Off), Programming> Memory> Raise Event box

    pepper-hue_flow.png

  3. The Hue On box should output a value that turns Hue on.

    def onInput_onStart(self):
        self.onStopped([["on", True], ["saturation", 254], ["brightness", 204], ["hue", 34297]])
    
  4. The Hue Off box should output a value that turns Hue off.

    def onInput_onStart(self):
        self.onStopped([["on", False], ["saturation", 254], ["brightness", 204], ["hue", 34297]])
    
  5. Connect the Raise Event box to the Hue On box and Hue Off box, and specify the key corresponding to the Hue bridge and light.

    pepper-hue_eventname.png

With this alone, if you touch the front of your head, the data indicating lighting will be transmitted to the control program of Hue Bridge via MQTT, and you can turn the lamp on and touch the back to turn it off.

Small story: Include Python code in box library

In Creating a Tweet box for Pepper, I wrote a method to include a Python module (.py) in a project and import it from a Python box. This procedure requires procedures such as file copying when creating a project, and it is a little troublesome to use for implementation. ** It would be nice to have a mechanism to keep the files of these Python modules in the box library so that when you copy the box to your project, you can also copy the Python code along with it **.

This time, we've achieved this by using the ** attachment ** mechanism. If you look at the contents of the MQTT Adapter box, you will find that the MQTT Adapter Impl box of the Python box is actually inside.

python-module-attach-1.png

In this box, there are variables of the [Attachment] type called LibFile1, ..., LibFile4. Here, I've specified the files I want to include when copying from the box library to the behavior.

python-module-attach-2.png

By doing this, the files under / paho / ... will be copied when they are copied from the box library to the behavior.

I purposely made the MQTT Adapter Impl box as a child of the MQTT Adapter box because I didn't want to show the items related to these attachments to the box user (to prevent confusion etc.).

At the moment, it seems to work fine, but when creating the box library, when copying from the behavior to the box library, the path of the attachment looks like the relative path in the project (not the relative path in the behavior). There may be some pitfalls and there may be other pitfalls, so this time it's just a trial.

Summary

Through MQTT support of Hue and IRKit, I felt that it would be easier to link apps by using MQTT, so I made a box that can handle MQTT even with Pepper and made it a form that can be distributed as a box library.

I've done it without thinking about encryption for the time being, but I still want to support TLS. I will update it on GitHub, so please have a look. Also, pull requests are welcome.

However, PubSub models like MQTT are nice to be able to freely code communication between nodes, but if I was flirting and writing various codes, I accidentally made code that ping-pong messages, and in a blink of an eye 10000 There was also a tragedy like sending a message or running out of free slots. I feel that it would be better to set up an MQTT Broker by yourself using Mosquitto for testing.

Recommended Posts

The story of making a box that interconnects Pepper's AL Memory and MQTT
The story of making a question box bot with discord.py
The story of making a module that skips mail with python
The story of making a tool that runs on Mac and Windows at the game development site
The story of making a sound camera with Touch Designer and ReSpeaker
The story of making a package that speeds up the operation of Juman (Juman ++) & KNP
The story of making a lie news generator
The story of making a mel icon generator
The story of making a web application that records extensive reading with Django
The story of making a Line Bot that tells us the schedule of competitive programming
A story that reduces the effort of operation / maintenance
The story of making a music generation neural network
A story that analyzed the delivery of Nico Nama.
The story of Python and the story of NaN
The story of writing a program
A story that struggled to handle the Python package of PocketSphinx
The story of making a standard driver for db with python.
The story of creating a site that lists the release dates of books
A story that visualizes the present of Qiita with Qiita API + Elasticsearch + Kibana
The story of making a university 100 yen breakfast LINE bot with Python
This and that of the inclusion notation.
The story of making an immutable mold
The story of developing a web application that automatically generates catchphrases [MeCab]
The story of making a slackbot that outputs as gif or png when you send the processing code
The story of trying deep3d and losing
The story of blackjack A processing (python)
The story of creating (probably) the smallest skill that implements personalization and in-skill billing
The story of IPv6 address that I want to keep at a minimum
The story of Django creating a library that might be a little more useful
[Python] A program that calculates the number of updates of the highest and lowest records
The story of making the Mel Icon Generator version2
A discussion of the strengths and weaknesses of Python
The story of Airflow's webserver and DAG, which takes a long time to load
The result of making a map album of Italy honeymoon in Python and sharing it
The story of Linux that I want to teach myself half a year ago
A story that Seaborn was easy, convenient and impressed
The story of launching a Minecraft server from Discord
[Python] A program that counts the number of valleys
The story of Python without increment and decrement operators.
I made a threshold change box of Pepper's Dialog
Make a BOT that shortens the URL of Discord
A story of trying out pyenv, virtualenv and virtualenvwrapper
# Function that returns the character code of a string
A story that struggled with the common set HTTP_PROXY = ~
Generate that shape of the bottom of a PET bottle
A story about changing the master name of BlueZ
The story that the return value of tape.gradient () was None
Zip 4 Gbyte problem is a story of the past
[Python] A program that compares the positions of kangaroos.
Now in Singapore The story of creating a LineBot and wanting to do a memorable job
A server that returns the number of people in front of the camera with bottle.py and OpenCV
The story of creating a "spirit and time chat room" exclusively for engineers in the company
A library that monitors the life and death of other machines by pinging from Python
A story that makes it easy to estimate the living area using Elasticsearch and Python
A story about porting the code of "Try and understand how Linux works" to Rust
[Python] A program to find the number of apples and oranges that can be harvested
The story of sys.path.append ()
A tool that automatically turns the gacha of a social game
The story of creating a VIP channel for in-house chatwork
Get the id of a GPU with low memory usage
The story of a Django model field disappearing from a class