[JAVA] Register a DLNA compatible TV in SORACOM Inventory and operate it from the Web console
Introduction
This is Yaskawa from Soracom. SORACOM Inventory released as Public Beta at SORACOM Technology Camp held in April / 04/26 / inventory-public-release /) Did you guys try it?
As you can see in the linked blog, when making Public Beta, we released it in the form of adding new functions based on the feedback from customers who tried it with Private Beta and Limited Preview, but one of the new functions One is that it supports custom models. This time, I would like to take advantage of that function to do Deep Dive.
In SORACOM Inventory, the agent running on the device and the server on the SORACOM side follow the standardized device management protocol called OMA DM LwM2M. Communicate. At the beginning of the release, only Standard object definition was supported, so general-purpose devices such as temperature sensors and acceleration sensors Although it was possible to implement according to the existing definition, there was a problem that it could not be expressed well in Inventory when trying to connect devices that do not meet the standard.
As the name suggests, the custom model support released this time is used to register arbitrary object model definitions and use them for message interpretation when the server and device interact on SORACOM Inventory, or for displaying on the Web console. It is a function that can be done. In other words, if you use this function, you can register your own definitions and manage them in Inventory even if the device types and specifications are not defined in the standard.
It's been a long introduction, but I've always wanted to do it since I was developing this feature, and I've always wanted to write an article that actually tried it. As the title suggests, it's about operating the TV in front of you with SORACOM Inventory!
(By the way, this blog is intended to be an article I tried on Deep Dive, so I'm close to the deadline, but I would like to apply for the Try SORACOM Challenge Campaign. I think. Target: Anyone, regardless of company or individual, can participate. Because it was written, I hope it will be a cool novelty.)
Operate a DLNA-compatible TV
Nowadays, most home TVs are DLNA compatible and are connected to your home network via Ethernet or WiFi. I'm sure that your TV at home is also communicating with other devices via DLNA on a regular basis. If you have such an environment in front of you, you can actually try what is written in this blog.
Specifically, it is a story of discovering a DLNA compatible TV with UPnP, connecting to the OMA DM LwM2M protocol SORACOM Inventory, and operating it from the Inventory Web console. (If you are allergic to the abbreviation of the alphabet, I'm sorry, I think you're getting rejection, but for the time being, it's okay if you think that you're messing up and operating the TV.)
I wrote a blog called Discover Thing and Beam at the time of the 2015 Advent calendar, but at that time I discovered DLNA compatible TV with UPnP Then, it was a story of registering and operating AWS IoT with Beam MQTT. It's easy to understand if you think that MQTT, AWS IoT has been replaced by LwM2M, SORACOM Inventory (easy to understand, isn't it?).
By the way, at that time I made something like UPnP <-> MQTT agent with node.js, but in the case of SORACOM Inventory, agent construction from custom model definition is very smooth if you use Java Client library, so this time with Java I would like to go.
As with the previous blog writing, I would like to implement this time focusing on use cases for playing videos, raising and lowering Volume, and recognizing TV power on / off.
I took a video of how it actually works, so I hope you'll take a look here to get an idea of what it's doing, and understand that the steps to implement it are described below.
I will actually make it
As the SDK of SORACOM Inventory, the Java Client library made by @ c9katayama is open to the public, and the Detailed Agent Implementation Procedure is included in the Readme. /blob/master/README_ja.md) is written, so it will be smooth if you proceed while looking at this.
Specifically, we will define a custom model for DLNA TV, generate Java code from the definition file, and fill it in.
1. Create a custom model definition
The model definition supports both XML and JSON. This time, I copied and edited the example in the Readme of the Java Client library, so it is XML. To implement this use case, UPnP's Rendering Control Service and [AV Transport Service](http: / /www.upnp.org/specs/av/UPnP-av-AVTransport-v3-Service-20101231.pdf) is required, so we will define Action parameters and actions while referring to each specification.
I was worried about the value of the ID attached to each resource, but for the time being, I decided to go with a very crude option of using the section number on the specification of each parameter and action.
I will write the definition of parameters and Action like this.
resource-example.xml
<Item ID="2216">
<Name>Volume</Name>
<Operations>R</Operations>
<MultipleInstances>Single</MultipleInstances>
<Mandatory>Mandatory</Mandatory>
<Type>Integer</Type>
<RangeEnumeration />
<Units></Units>
<Description>Current audio volume of the renderer</Description>
</Item>
(I'm tired of trying to do everything, so I focused on the use cases I wanted to implement.)
Here is the completed object definition file:
If you register this as a custom model, the preparation on the SORACOM Inventory side is completed, and when the device comes online after that, the corresponding item will automatically appear on the console.
For details on custom object model definition, refer to here.
2. Generate Java code from the object definition file and implement the method
The Inventory client library is great because it also supports class file generation from object definitions, so let's use that feature.
Here is the Source Generator code that has been slightly modified to generate a class file from the file created in the previous section, referring to the sample code on Github.
JavaSourceGenerator.java
package models;
import io.soracom.inventory.agent.core.util.TypedAnnotatedObjectTemplateClassGenerator;
import java.io.File;
public class JavaSourceGenerator {
public static void main(String[] args) {
String javaPackage = "models";
File sourceFileDir = new File("src/main/java");
TypedAnnotatedObjectTemplateClassGenerator generator = new TypedAnnotatedObjectTemplateClassGenerator(
javaPackage, sourceFileDir);
File[] modelFiles = new File("src/main/resources").listFiles();
for (File modelFile : modelFiles) {
generator.generateTemplateClassFromObjectModel(modelFile);
}
System.out.println("Finished generating Java source.");
System.exit(0);
}
}
Doing this will generate a class corresponding to the models package.
Looking at the contents, it looks like this.
UPnPRenderingControlServiceObject.java
package models;
import io.soracom.inventory.agent.core.lwm2m.*;
import java.util.Date;
import org.eclipse.leshan.core.node.ObjectLink;
/**
* LwM2M device object model for UPnP Rendering Control service.
**/
@LWM2MObject(objectId = 30001, name = "UPnP Rendering Control Service", multiple = true)
public class UPnPRenderingControlServiceObject extends AnnotatedLwM2mInstanceEnabler {
/**
* Mute status of the renderer
**/
@Resource(resourceId = 2215, operation = Operation.Read)
public Boolean readMute() {
throw LwM2mInstanceResponseException.notFound();
}
/**
* Current audio volume of the renderer
**/
@Resource(resourceId = 2216, operation = Operation.Read)
public Integer readVolume() {
throw LwM2mInstanceResponseException.notFound();
}
/**
* Sets the renderer audio volume.
**/
@Resource(resourceId = 2430, operation = Operation.Execute)
public void executeSetVolume(String executeParameter) {
throw LwM2mInstanceResponseException.notFound();
}
}
After that, you can implement it so that it calls the UPnP action corresponding to each method. It feels like it's going along the flow, so it's nice to know what to do!
3. Implement each method corresponding to the LwM2M resource
This time we are targeting UPnP devices, so next we will implement it using the UPnP library. I decided to use cling library because it seemed to be easy to use.
Add the following to build.gradle and you're ready to go.
build.gradle
apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'idea'
apply plugin: 'application'
repositories {
mavenCentral()
maven { url 'https://soracom.github.io/maven-repository/' }
maven { url 'http://4thline.org/m2/' } // <-add to
}
def INVENTORY_AGENT_VERSION="0.0.5"
dependencies {
compile "io.soracom:soracom-inventory-agent-for-java-core:$INVENTORY_AGENT_VERSION"
compile "org.fourthline.cling:cling-core:2.1.1" // <-add to
testCompile 'junit:junit:4.12'
}
jar {
manifest {
attributes "Main-Class": "App"
}
}
mainClassName = 'App'
Looking at some of the methods of ʻUpnpRenderingControlServiceObject` that I implemented, it looks like this.
UpnpRenderingControlServiceObject.java
/**
* Current audio volume of the renderer
**/
@Resource(resourceId = 2216, operation = Operation.Read)
public Long readVolume() {
ActionInvocation actionInvocation =
new ActionInvocation(renderingControlService.getAction(ACTION_GET_VOLUME));
actionInvocation.setInput(ARG_INSTANCE_ID, new UnsignedIntegerFourBytes(0));
actionInvocation.setInput(ARG_CHANNEL, VALUE_CHANNEL_MASTER);
Map<String, ActionArgumentValue> output = UpnpController.getInstance().executeUpnpAction(actionInvocation);
return ((UnsignedIntegerTwoBytes)output.get(ARG_CURRENT_VOLUME).getValue()).getValue();
}
/**
* Sets the renderer audio volume.
**/
@Resource(resourceId = 2430, operation = Operation.Execute)
public void executeSetVolume(String executeParameter) {
ActionInvocation setVolumeInvocation =
new ActionInvocation(renderingControlService.getAction(ACTION_SET_VOLUME));
setVolumeInvocation.setInput(ARG_INSTANCE_ID, new UnsignedIntegerFourBytes(0));
setVolumeInvocation.setInput(ARG_CHANNEL, VALUE_CHANNEL_MASTER);
setVolumeInvocation.setInput(ARG_DESIRED_VOLUME, new UnsignedIntegerTwoBytes(executeParameter));
UpnpController.getInstance().executeUpnpAction(setVolumeInvocation);
}
In short, it feels like converting the request received by LwM2M into the corresponding Action of UPnP. Both protocols have similar concepts of how to express the device, which is the basis, so conversion was relatively easy.
Here are the two files that have been implemented:
If you create an Agent that implements these two objects (and the basic Device object), you can register it in Inventory and operate it with the API and User Console.
This time I made it with the name MediaRendererAgent.
Although not listed here, if you find an UPnP device, create a separate class called ʻUpnpDeviceAgent` that corresponds to the Device object of LwM2M as a base class, and MediaRendererAgent inherits it and creates the two objects defined this time. It is designed to be added.
MediaRendererAgent.java
package org.yasukawa.inventory.upnp.media_renderer;
import org.fourthline.cling.model.meta.RemoteDevice;
import org.fourthline.cling.model.meta.Service;
import org.fourthline.cling.model.types.ServiceType;
import org.yasukawa.inventory.upnp.UpnpDeviceAgent;
public class MediaRendererAgent extends UpnpDeviceAgent {
public static final String MEDIA_RENDERER_DEVICE_TYPE = "urn:schemas-upnp-org:device:MediaRenderer:1";
public static final String RENDERING_CONTROL_SERVICE_TYPE = "urn:schemas-upnp-org:service:RenderingControl:1";
public static final String AV_TRANSPORT_SERVICE_TYPE = "urn:schemas-upnp-org:service:AVTransport:1";
public static final String ATTR_LAST_CHANGE = "LastChange";
public static final String ATTR_INSTANCE_ID = "InstanceID";
public MediaRendererAgent(RemoteDevice device){
super(device);
for ( Service service : device.getServices()){
System.out.println(service.getServiceType());
System.out.println(service.getServiceId());
}
super.initializer.addInstancesForObject(new UpnpRenderingControlServiceObject(
device.findService(ServiceType.valueOf(RENDERING_CONTROL_SERVICE_TYPE)))
);
super.initializer.addInstancesForObject(new UpnpAvTransportServiceObject(
device.findService(ServiceType.valueOf(AV_TRANSPORT_SERVICE_TYPE)))
);
}
}
4. After discovering the UPnP device, create an Inventory Agent and register it.
After that, discover the UPnP device, and if you find a Media Renderer such as a TV, create an instance of the Media Renderer Agent and register it, and you will be able to operate the TV with the API of Inventory! hot!
With cling used this time, if you start Discovery after registering an instance of the class that implements the interface called ʻUpnpRegistryListener` in UPnP control point, It was designed so that you can call back every time you find a device. So, when I get that callback, I feel like writing the logic to create an instance of Media Renderer Agent and start it if the device found is Media Renderer. Specifically, it looks like the following.
InventoryUpnpRegistryListener.java
@Override
public void remoteDeviceAdded(Registry registry, RemoteDevice device) {
logger.info("Device {} with type {} is added", device.getIdentity().toString(), device.getType().toString());
if (MediaRendererAgent.MEDIA_RENDERER_DEVICE_TYPE.equals(device.getType().toString())){
logger.info("Media renderer found: " + device.getDetails().getFriendlyName());
MediaRendererAgent agent = new MediaRendererAgent(device);
logger.info("Starting agent for {} and registering to SORACOM Inventory",
device.getIdentity().toString());
agent.start();
agentMap.put(device.getIdentity().toString(), agent);
}
}
@Override
public void remoteDeviceRemoved(Registry registry, RemoteDevice device) {
logger.info("Device {} with type {} is removed", device.getIdentity().toString(), device.getType().toString());
if (agentMap.containsKey(device.getIdentity().toString())){
logger.info("Stopping agent for {} and deregistering from SORACOM Inventory",
device.getIdentity().toString());
agentMap.remove(device.getIdentity().toString()).stop();
}
}
By the way, when the event that the device is deleted comes, I will stop the Agent. By doing this, if you turn off the power of the TV, even the Inventory can move to Offline.
After that, register this ʻUpnpRegistryListener` in UPnP Control Point in the main App.java and start Discovery, and it's done!
App.java
import org.fourthline.cling.UpnpService;
import org.fourthline.cling.UpnpServiceImpl;
import org.fourthline.cling.model.message.header.STAllHeader;
import org.yasukawa.inventory.upnp.InventoryUpnpRegistryListener;
import org.yasukawa.inventory.upnp.UpnpController;
public class App {
public static void main(String[] args) throws Exception {
UpnpService upnpService = new UpnpServiceImpl();
UpnpController.initialize(upnpService);
upnpService.getRegistry().addListener(new InventoryUpnpRegistryListener(upnpService));
upnpService.getControlPoint().search(new STAllHeader());
}
}
I will actually move it
The code created this time is in this repository, and you can create a zip archive for distribution by following the steps below.
$ git clone https://github.com/kntyskw/inventory-upnp
$ cd inventory-upnp
$ gradle distZip
If you copy the created ./build/distributions/inventory-upnp.zip to a Raspberry Pi etc. connected to both your home LAN and SORACOM Air and unzip it, you are ready to execute (assuming JRE is there). ..
After that, deploy this archive to a device that runs as a UPnP gateway such as Raspberry Pi and execute it.
As an aside, recently I've been using resin.io, which allows me to deploy Docker containers to IoT devices, as an easy way to deploy software to Raspberry Pi. For details, please see the website of the head family, but if you have already set up the environment of resin.io, you can deploy the updated container to Raspberry Pi by the following procedure.
$ git add inventory-upnp.zip
$ git commit -m "add/update inventory-upnp.zip"
$ git push resin master
In addition, you can SSH to your Raspberry Pi from the resin.io web console wherever you are, and reboot and update your containers at will. Please try it if you like.
Regardless of the procedure, after deploying and extracting the archive, execute the following command in the destination directory.
$ ./bin/inventory-upnp
Then Raspberry Pi will act as an UPnP gateway, and if you find a DLNA-compatible TV, it will be registered in SORACOM Inventory and made operational! From that point onward, you can operate as shown in the video at the beginning.
in conclusion
What did you think? The concept of handling UPnP and LwM2M (more specifically, OMA DM) devices is fundamentally similar, so I felt that I was able to smoothly implement the process of registering UPnP devices in SORACOM Inventory.
This time, we covered UPnP, but if the protocol has a mechanism such as Discovery, command execution, and Subscribe to Event, you can implement Inventory Agent and make it available from SORACOM Inventory. For example, if you use Bonjour for Apple TV, you can do the same.
This means that if you run SORACOM Air SIM and Inventory Agent on a device that will be a protocol gateway, you can manage all the surrounding devices from Inventory. (And with the combination of SORACOM Air + Inventory, authentication information sharing to the protocol gateway is automatic!) Why don't you register your home devices in SORACOM Inventory so that you can monitor and control them from anywhere?
By the way, as I remembered when I subscribed to Discovery and Event, have you already subscribed to the event called SORACOM Discovery on July 4th? If you haven't done so, the number of seats left is decreasing, but I think it will still be in time, so please apply from this link! By the way, this link will be flagged as my introduction, so please this link ky) Please. (Persistent w)
I'm just waiting for a flight from Silicon Valley to Tokyo. We look forward to discussing maniac topics with you again this year at the SORACOM Discovery venue, including things like UPnP and SORACOM Inventory!
Recommended Posts