Pepper Tutorial (5): Using a Tablet

Contents of this tutorial

In this tutorial, we will explain how to use the boxes provided by the tablet box library as a way to control the chest tablet, which is a feature of Pepper.

1. Tablet specifications
2. Project content panel
3. How to use the box: Display the contents
4. How to use the box: Get the touch event

Note that ** the tablet service is not supported by the virtual robot and an error will occur **, but in this tutorial, the contents of the box written in Python are used to analyze what the tablet-related box is doing. It also explains how to see, so if you are interested, please do.

Tablet specifications

The main specifications of Pepper tablets are as follows.

In addition to output such as displaying images and videos, it is possible to acquire touch events and reflect them in behavior.

Project content panel

In order to display the content on the tablet, the content you want to display must be imported into the project in advance. At this time, it is necessary to understand the ** project contents ** panel, so I will explain it in detail here.

1. Project file ... A file included in the project. Can be hierarchized by directory and files can be moved by dragging
2. Add File / Directory ... Add an element to your project file. The following menu will appear

• Creating a directory ...
• Creating an action ...
• Creating a dialog topic ...
• Creating a plane motion ...
• Importing behavior ...
• File import ...
• Import folder ...

3. Delete selected file / directory

For content output on a tablet, ** prepare an html directory in the project file ** for both images and videos, and place the files you want to display there. Also, on the box side that refers to these files, the files will be specified by the path ** starting from the ** html directory. The details of the operation method will be explained in the tutorial.

How to use the box: Display the content

Image output

First, let's easily display the image on a Pepper tablet. Please prepare an appropriate image file in PNG, JPEG, or GIF format and perform the work.

Try to make

1. Preparing the box to use ... Use the box on the tablet tab
   • tablet > Show Image
2. Connect the boxes

• Just show the image on your tablet in the Show Image box with the application onStart

3. Create an html directory for your tablet Click the Add button in the project content panel, then click ** [Create Directory ...] ** If you create the name as html, an html directory will be created as a project file

4. Import the image Click ** Import File ... ** to open a file selection dialog. When you select the image file you want to display, the image file is registered as a project file.

5. Drag the image file and move it into the html directory The image file is now ready for viewing on your tablet.

6. Click the parameter button in the Show Image box and enter the image file name in ImageUrl. This value is a path representation starting from the html directory.

7. ** [Magic] ** Move behavior out of directory As a magic trick to make Show Image work, change the position of the behavior (saved in a file called behavior_1 / behavior.xar after creating a new project). Drag behavior.xar out of the behavior_1 directory so that it is the top-level file of your project file, as shown below. We'll explain why it's magical through exploring how Show Image is achieved.

You now have an application that displays images on your tablet.

Operation check

To check the operation, connect to Pepper and play it.

In the case of a virtual robot, the Show Image box turns red to indicate an error, and the following error message is displayed in the log view.

[ERROR] behavior.box :_getTabletService:20 _Behavior__lastUploadedChoregrapheBehavior1798150472__root__ShowImage_13: Cannot find service 'ALTabletService' in index
[WARN ] behavior.box :onInput_onStart:58 _Behavior__lastUploadedChoregrapheBehavior1798150472__root__ShowImage_13: No ALTabletService, can't display the image.

(Supplement) Let's see the implementation of Show Image box

A quick way to find out the magical reason for behavior.xar is to look at the Show Image box. The Show Image box is a ** Python box **, so unlike the flow diagram box and timeline box, double-clicking the box opens the script editor.

Let's take a look at the contents of the code. Of this code, the following code is invoked by onStart.

    def onInput_onStart(self):
        # We create TabletService here in order to avoid
        # problems with connections and disconnections of the tablet during the life of the application
        tabletService = self._getTabletService()
        if tabletService:
            try:
                url = self.getParameter("ImageUrl")
                if url == '':
                    self.logger.error("URL of the image is empty")
                if not url.startswith('http'):
                    url = self._getAbsoluteUrl(url)
                tabletService.showImage(url)
            except Exception as err:
                self.logger.error("Error during ShowImage : %s " % err)
                self.onStopped()
        else:
            self.logger.warning("No ALTabletService, can't display the image.")
            self.onStopped()

In the virtual robot, the ALTabletService to be acquired in this self._getTabletService () cannot be found and an error occurs.

By the way, what is related to the magic is this part self._getAbsoluteUrl (url) that converts the path to the absolute URL. Looking further,

    def _getAppName(self):
        import os
        if self.frameManager:
            behaviorPath = os.path.normpath(self.frameManager.getBehaviorPath(self.behaviorId))
            appsFolderFragment = os.path.join("PackageManager", "apps")
            if not (appsFolderFragment in behaviorPath):
                self.logger.error("appsFolderFragment is not in behaviorPath")
            fragment = behaviorPath.split(appsFolderFragment, 1)[1]
            return fragment.lstrip("\\/")
        else:
            self.logger.warning("No ALFrameManager")

    def _getAbsoluteUrl(self, partial_url):
        import os
        subPath = os.path.join(self._getAppName(), os.path.normpath(partial_url).lstrip("\\/"))
        # We create TabletService here in order to avoid
        # problems with connections and disconnections of the tablet during the life of the application
        return "http://%s/apps/%s" %(self._getTabletService().robotIp(), subPath.replace(os.path.sep, "/"))

In the process of generating the URL to instruct the tablet, the path of the behavior to which the Show Image box belongs is used, which is obtained by self.frameManager.getBehaviorPath (self.behaviorId). If the behavior is not at the top of the project file, the URL assumed here cannot be obtained and the image display will fail.

In addition to repositioning the behavior, you can also insert the following code into the Python code in the Show Image box to work around this issue.

    def _getAppName(self):
        import os
        if self.frameManager:
            behaviorPath = os.path.normpath(self.frameManager.getBehaviorPath(self.behaviorId))
            appsFolderFragment = os.path.join("PackageManager", "apps")
            if not (appsFolderFragment in behaviorPath):
                self.logger.error("appsFolderFragment is not in behaviorPath")
            fragment = behaviorPath.split(appsFolderFragment, 1)[1]
            #Add the following line
            fragment = fragment.split('/')[1]
            return fragment.lstrip("\\/")
        else:
            self.logger.warning("No ALFrameManager")

In this way, you can check the contents of the Python box to deepen your understanding of the box, add log output code, and make other modifications.

Video output

You can also play video files in the same way as image files. It supports mp4 and mov.

Try to make

1. Preparing the box to use ... Use the box on the tablet tab
   • tablet > Play Video
2. Connect the boxes

• The Play Video box will send an onStopped signal at the end of video playback, so be sure to end the behavior accordingly.

3. Create an html directory and import the video file

• Same procedure as image file

4. Set the path to the video file in the parameter of the Play Video box

5. ** [Magic] ** Move behavior out of directory As with the image file, we will also carry out magic

Now you can have an application that plays the video and ends the behavior when it finishes playing.

Operation check

Connect to Pepper and try playing. The video should play on your tablet.

The Play Video box also has inputs such as onPauseVideo (pause) and onResumeVideo (resume), so for example, it keeps playing the video only when it is judged that there is a person in front of you, and stops the video when there are no more people. It may be interesting to do that.

How to use the box: Get the touch event

It is also possible to process with a touch event for the tablet. Here, let's say "yes" when the tablet is touched.

Try to make

1. Preparation of the box to use

• tablet tab
• tablet> Touch Detection ... Touch detection ...
  • standard
• Audio> Voice> Set Language ... Change language settings
• Audio> Voice> Say ... say hello

2. Connect the boxes

3. Set the parameters

• For the parameter of the Touch Detection box, set the variable Action to [On touch down].
• Set Language box sets Japanese

4. Customize the Say box

• Double-click the Say box and set Localized Text in Japanese to Yes.

Now you can run the Say box when touched.

Operation check

Try connecting to Pepper, playing, and touching your tablet. Say "yes" and you're successful.

The onTouched output is passed information about which coordinates were touched. It might be interesting to be able to change the behavior of Pepper with this information.

Tablets are a very powerful element in presenting information from Pepper to humans. In addition to the images and videos listed here, you can do various things such as displaying HTML files.