Basic Hue Lighting Control: Part 1

screenshot
Screenshot of Chrome application for controlling Hue lighting.

Continuing from the post I made on SSDP discovery with Chrome, I’m making an application that will do more than just discovery. For this post I’m going to show the starting point of a Chrome application for controlling your home Hue lighting. I’ve divided this into two parts. In this first part I’m showing the process of pairing with the bridge. In the second part I’ll control the lights.

The features that this application will implement will include bridge discovery and pairing; the power state of the light; and the brightness level of the light. There’s many other features that could still be implemented.  Given the full range of capabilities that the Hue kits support (changing color, timers, response to motion sensors, etc.) this will not be an application that utilizes the full capability of the Hue lighting sets.

Chrome Only

This application is designed to only run in Chrome. If you want to adapt it to run outside of Chrome, you can do so by first disabling SSDP discovery. (Other HTML application platforms might not support UDP for discovery.)

The other discovery methods (querying Hue’s discovery web service or asking the user to enter the IP address) can still work. A non-chrome target will also need to allow CORS to be ignored and allow communication without SSL.

What is Hue Lighting?

Hue Lighting is an automated lighting solution made by Philips. Generally the lighting kits are sold in a package that contains three LED based light bulbs and a bridge. The bridge is a device that connects to your home network with an Ethernet jack and communicates with the light bulbs.

Philips also makes free applications for iOS and Android for controlling the lights. For any Hue light the light’s brightness and whether or not it is turned on can be controlled through the applications. Some lights also allow the color temperature to be changed (adjusting the tint between red, yellow and blue). Some lights support RGB (Red, Green, Blue) parameters so that their colors can be changed.  These settings can be individually adjusted or settings for a collection of the lights can be defined together as a “scene.” When a scene is activated the state of all of the lights that make up the scene are updated. Scenes can be activated through special light switches, through an app, through a schedule, or in response to a Hue motion sensor detecting motion.

Discovery: Review and New Methods

The central piece of hardware for the Hue lighting is the Hue Bridge. At the time of this writing there are two versions of the bridge. For the functionality that this application will utilize, the differences between the two bridges will not matter. The messaging and interaction to both versions of the bridge will be the same. My UI will properly represent the bridge that the system discovers. The first version of the Hue Bridge is round. The second version of the Hue Bridge is square. In either case we must first find the bridge’s IP address before we can begin interaction.

phillipsbridge
Phillips Hue Bridge Version 1 (left) and Version 2 (right)

The Hue bridge can be discovered in multiple ways. It can be discovered using SSDP. The basics of SSDP discovery were previously discussed here. Please refer back to it if you need more detail than what is found in this brief overview.  Devices that support SSDP discovery join a multicast group on the network that they are connected to. These devices generally wait for a request for discovery to be received. An SSDP request is sent as an HTTP over UDP message and every SSDP device that receives it responds with some basic information about itself and a URL to where more information on the device can be found. Examples of some devices that support SSDP are network attached storage; set top boxes like Android TVs and Rokus; printers; and home automation kits.

Two other methods of discovering a bridge include asking the user to enter an IP address and asking for a list of IP addresses of bridges on your network through the Hue discovery service.  If you have a Hue bridge connected to your network right now you can see it’s IP address by visiting https://discovery.meethue.com/ . If you are on a shared network then you may also see IP addresses of other bridges on your network. It is also possible that not all bridges on your network are reachable.  This method is much easier to implement than SSDP based discovery. But on a network for which there is no Internet connection (whether by design or from an outage) this method will not work. The SSDP method is only dependent on the local network.

function discoverBridge() { 
    discovredHueBridgeList = [];
    fetch(' https://discovery.meethue.com')
        .then(response => response.json())
    .then(function (hueBridgeList) {
        console.info(hueBridgeList);
        hueBridgeList.forEach((item)=> {
         // each item processed here has a bridge IP address
         // and serial number exposed through item.id and 
         // item.internalipaddress
       }
     );
}

Once I have a bridge IP address I attempt to query it for more information. If communication succeeds, then I show a representation of the bridge with an icon that matches the version of the bridge that the user has. The UI layout has two images ( one named hueBridgev1 and the other hueBridgev2) I show the appropriate image and hide the other.

Pairing

Now that the bridges have been discovered, it is up to the user to select one with which to pair. After the user selects a bridge, she is instructed to press the pairing button on the bridge. While this instruction is displayed the application is repeatedly attempting to request a new user ID name from the bridge. This should be viewed more as an access token. The Hue documentation uses the term “user name” but the actual value is what appears to be a random sequence of characters. To request a user name a JSON payload with one member named devicetype is posted to the bridge. The value assigned to devicetype matters little. It is recommended that it be a string that is unique to your application. The payload is posted to http://%5Byour bridge IP address]/api. A failure response will result. This is expected. The application must repeatedly make this request and prompt the user to press the link button on the bridge.  The request will fail until the pairing button on the bridge is passed.

function pairBridge(ipAddress) {
   console.info('attempting pairing with address ', ipAddress);
   var req = { devicetype: "hue.j2i.net#browser" };
   var reqStr = JSON.stringify(req);
   var tryCount = 0;
   return new Promise(function(resolve, reject)  {
      var tryInterval = setInterval(function () {
      console.log('attempt ', tryCount);
      ++tryCount;
      if (tryCount > 60) {
        clearInterval(tryInterval);
         reject();
         return;
      }
      fetch(`http://${ipAddress}/api`, {
         method: "POST",
         headers: {
            "Content-Type": "application/json"
         },
         body: reqStr
      })
      .then(function(response)  {
         console.log('text:',response);
         return response.json();
      })
      .then(function(data)  {
          console.log(data);
          if (data.length > 0) {
             var success = data[0].success;
             var error = data[0].error;
             if (success) {
                console.log('username:', success.username);
                var bridge = {
                   ipAddress: ipAddress,
                   username: success.username
                };
                clearInterval(tryInterval);
                 resolve(bridge);
                  return;
               }
               else if (error) {
                  if (error.type === 101) {
                     console.log('the user has not pressed the link button');
                  }
               }
            }
         });
      }, 2000);
   });
}

Once the button is pressed the bridge will respond to the first pairing request it receives with a user name that the application can use. This user name must be saved and used for calls to most of the functionality that is present in the bridge. I save the bridge’s serial number, IP address, and the name that must be used for the various API calls to an indexedDB object store. The access information for multiple paired bridges could be stored in the object store at once. But the application will only be able to communicate with one bridge at a time.

Continued in Part II

Advertisements

My Samsung HoloLab Model Arrived

Last week I received an e-mail from Samsung regarding the HoloLab scans that I did at the 2018 Samsung Developer’s Conference.  Shortly after the conference, I wrote about the rig that was used to do the scan.

When the photographs were taken for the scan, three poses were requested.  The first pose was standing with your arms crossed.  The second pose was standing with both of your arms out to the side.  The third pose was allowed to be a freestyle that could be whatever you wanted (just for the fun of it).

Of these three poses, I was most interested in the second pose, because arms out to the side is the most appropriate pose to use when importing a model into software for animating.  Sadly, what arrived in my e-mail was only one of the three poses, the first one.

I’m still happy to have received the one pose that I did. The model definitely resembles me.  This is speculation on my part, but I imagine that the processing of the 52 images that make up a single scan is time consuming. Considering the large number of participants at the conference who had the scans done, receiving all three model poses may be wishful thinking.

 

HololabScan

SSDP Discovery in HTML

While implementing a few projects I decided to implement them in HTML since it would work on the broadest range of my devices of interest. My projects of interest needed to discover additional devices that are connected to my home network. I used
SSDP for discovery.

SSDPDiscovery

SSDP (Simple Service Discover Protocol ) is a UDP based protocol that is a part of UPnP for finding other devices and services on a network. It’s implemented by a number of devices including network attached storage devices, Smart TVs, and home automation systems. There are a lot of these devices that expose functionality through JSON calls. You can easily make interfaces to control these devices. However, since the standards for HTML and JavaScript don’t include a UDP interface, how to perform discovery isn’t immediately obvious. Alternatives to SSDP include having the user manually enter the IP address of the device of interest or scanning the network. The latter of those options can raise some security flags when performed on some corporate networks.

For the most part, the solution to this is platform dependent. There are various HTML based solutions that do allow you to communicate over UDP. For example, the BrightSign HTML5 Players support UDP through the use of roDatagramSocket. Chrome makes UDP communication available through chrome.udp.sockets. Web pages don’t have access to this interface (for good reason, as there is otherwise potential for this to be abused). Although web apps don’t have access, Chrome extensions do. Chrome Extensions won’t work in other browsers. But at the time of this writing Chrome accounts for 67% of the browser market share and Microsoft has announced that they will use Chromium as the foundation for their Edge browser. While this UDP socket implementation isn’t available in a wide range of browsers, it is largely available to a wide range of users since this is the browser of choice for most desktop users.

To run HTML code as an extension there are two additional elements that are needed: a manifest and a background script. The background script will create a window and load the starting HTML into it.

chrome.app.runtime.onLaunched.addListener(function() {
    chrome.app.window.create('index.html', {
        'outerBounds': {
        'width': 600,
        'height': 800
        }
    });
});

I won’t go into a lot of detail about what is in the manifest, but I will highlight its most important elements. The manifest is in JSON format. The initial scripts to be run are defined app.background.scripts. Other important elements are the permission element, without which the attempts to communicate over UDP or join a multicast group will fail and the manifest_version element. The other elements are intuitive.

        {
            "name": "SSDP Browser",
            "version": "0.1",
            "manifest_version": 2,
            "minimum_chrome_version": "27",
            "description": "Discovers SSDP devices on the network",
            "app": {
              "background": {
                "scripts": [
                  "./scripts/background.js"
                ]
              }
            },
          
            "icons": {
                "128": "./images/j2i-128.jpeg",
                "64": "./images/j2i-64.jpeg",
                "32": "./images/j2i-32.jpeg"
            },
          
            "permissions": [
              "http://*/",
              "storage",
              {
                "socket": ["udp-send-to", "udp-bind", "udp-multicast-membership"]
              }
            ]
          }    

Google already has a wrapper available as a code example chrome.udp.sockets that was published for using Multicast on a network. In it’s unaltered form the Google code sample assumes that text is encoded in the 16-bit character encoding of Unicode. SSDP uses 8-bit ASCII encoding. I’ve taken Google’s class and have made a small change to it to use ASCII instead of Unicode.

To perform the SSDP search the following steps are performed.

  1. Create a UDP port and connect it to the multicast group 239.255.255.250
  2. Send out an M-SEARCH query on port 1900
  3. wait for incoming responses originating from port 1900 on other devices
  4. Parse the response
  5. Stop listening after some time

The first item is mostly handled by the Google Multicast class. We only need to pass the port and address to it. The M-SEARCH query is a string. As for the last item, it isn’t definitive when responses will stop coming in. Some devices appear to occasionally advertise themselves to the network even if not requested. In theory you could keep getting responses. At some time I’d suggest just no longer listening. Five to ten seconds is usually more than enough time. There are variations in the M-SEARCH parameters but the following can be used to ask for all devices. There are other queries that can be used to filter for devices with specific functionality. The following is the string that I used; what is not immediately visible, is that after the last line of text there are two blank lines.

M-SEARCH * HTTP/1.1
HOST: 239.255.255.250:1900
MAN: "ssdp:discover"
MX: 3
ST: ssdp:all
USER-AGENT: Joel's SSDP Implementation
    

When a response comes in, the function that we assign to MulticastScoket.onDiagram will be called with a byte array containing the response, the IP address from which the response came, and the port number from which the response was sent (which will be 1900 for our current application). In the following code sample, I initiate a search and print the responses to the JavaScript console.

const SSDP_ADDRESS = '239.255.255.250';
const SSDP_PORT = 1900;
const SSDP_REQUEST_PAYLOAD =    "M-SEARCH * HTTP/1.1\r\n"+
                                "HOST: 239.255.255.250:1900\r\n"+
                                "MAN: \"ssdp:discover\"\r\n"+
                                "MX: 3\r\n"+
                                "ST: ssdp:all\r\n"+
                                "USER-AGENT: Joel's SSDP Implementation\r\n\r\n";

var searchSocket = null;

function beginSSDPDiscovery() { 
    if (searchSocket)
        return;
    $('.responseList').empty();
    searchSocket = new MulticastSocket({address:SSDP_ADDRESS, port:SSDP_PORT});
    searchSocket.onDiagram = function(arrayBuffer, remote_address, remote_port) {
        console.log('response from ', remote_address, " ", remote_port);
        var msg = searchSocket.arrayBufferToString8(arrayBuffer);
        console.log(msg);        
    }
    searchSocket.connect({call:function(c) {
        console.log('connect result',c);
        searchSocket.sendDiagram(SSDP_REQUEST_PAYLOAD,{call:()=>{console.log('success')}});
        setTimeout(endSSDPDiscovery, 5000);
    }});    
}

Not that parsing the response strings is difficult, by any means it would be more convenient if the response were a JSON object. I’ve made a function that will do a quick transform on the response so I can work with it like any other JSON object.

function discoveryStringToDiscoveryDictionary(str) {
    var lines = str.split('\r');
    var retVal = {}
    lines.forEach((l) => {
        var del = l.indexOf(':');
        if(del>1) {
            var key = l.substring(0,del).trim().toLowerCase();
            var value = l.substring(del+1).trim();
            retVal[key]=value;
        }
    });
    return retVal;
}    

After going through this transformation a Roku Streaming Media Player on my network returned the following response. (I’ve altered the serial number)

{
    cache-control: "max-age=3600",
    device-group.roku.com: "D1E000C778BFF26AD000",
    ext: "",
    location: "http://192.168.1.163:8060/",
    server: "Roku UPnP/1.0 Roku/9.0.0",
    st: "roku:ecp",
    usn: "uuid:roku:ecp:1XX000000000",
    wakeup: "MAC=08:05:81:17:9d:6d;Timeout=10"    ,
}

Enough code has been shared for the sample to be used, but rather than rely on the development JavaScript console,  I’ll change the sample to show the responses in the UI. To keep it simple I’ve defined the HTML structure that I will use for each result as a child element of a div element of the class palette. This element is hidden, but for each response I’ll clone the div element of the class ssdpDevice; will change some of the child members; and append it to a visible section of the page.

        
 <html>
    <head>
        <link rel="stylesheet" href="styles/style.css" />
        http://./scripts/jquery-3.3.1.min.js
        http://./scripts/MulticastSocket.js
        http://./scripts/app.js
    </head>
    <body>
Scan Network

 

</div>

address:
location:
server:
search target:

</div> </div>

</body> </html>

 

The altered function for that will now display the SSDP responses in the HTML is the following.

        function beginSSDPDiscovery() { 
            if (searchSocket)
                return;
            $('.responseList').empty();
            searchSocket = new MulticastSocket({address:SSDP_ADDRESS, port:SSDP_PORT});
            searchSocket.onDiagram = function(arrayBuffer, remote_address, remote_port) {
                console.log('response from ', remote_address, " ", remote_port);
                var msg = searchSocket.arrayBufferToString8(arrayBuffer);
                console.log(msg);
                discoveryData = discoveryStringToDiscoveryDictionary(msg);
                console.log(discoveryData);
        
                var template = $('.palette').find('.ssdpDevice').clone();
                $(template).find('.ipAddress').text(remote_address);
                $(template).find('.location').text(discoveryData.location);
                $(template).find('.server').text(discoveryData.server);
                $(template).find('.searchTarget').text(discoveryData.st)
                $('.responseList').append(template);
            }
            searchSocket.connect({call:function(c) {
                console.log('connect result',c);
                searchSocket.sendDiagram(SSDP_REQUEST_PAYLOAD,{call:()=>{console.log('success')}});
                setTimeout(endSSDPDiscovery, 5000);
            }});    
        }    

Working with non-SSL Web Services within an SSL page

I was making a Progressive Web App (PWA) and encountered a problem pretty quickly.  PWAs need to be served over SSL/HTTPS.  The services that they access must also be served over SSL (a page served over SSL cannot access non-SSL resources).  Additionally, since my app is being served from a different domain, there must be a Cross Origin Resource Sharing header permitting the application to use the data.  My problem is that I ran into a situation where I needed to access a resource that met neither of these requirements.

Failed to load http://myUrl.com: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://SomeOtherURL.com' is therefore not allowed access.

The solution to this seemed obvious: a proxy service that would consume the non-SSL feed and make the results available over HTTPS.  There exists some third party services that can do this for you (My SSL Proxy, for example).  But the services that I found were not meant for applications and generally don’t add the required CORS headers.  Implementing something like this isn’t hard, but for a lightweight application for which I wasn’t planning on making any immediate revenue, I wanted to minimize my hosting costs.  This is where two services that Google provides come into play.

The first Google service is Firebase.

Firebase (available at https://Firebase.Google.com) allows you to host static assets in the Google cloud.  These assets are servers over SSL.  This was a perfect place for hosting most of the source code that was going to run on the mobile device.

As for the service proxy, I made a proxy service that ran on the second Google service: App Engine.  Google’s cloud service App Engine (available at https://cloud.google.com/appengine/) allowed me to write my proxy service using NodeJS (available at https://nodejs.org/).  I had it query the data I needed from the non-SSL service and cache the data for 30 seconds at a time.  All of Google’s services use SSL by default, so I didn’t have to do anything special.  When returning the response I added a few headers to handle CORS requirements.  Here’s the code for the node server.  If you use it, you will need to modify it so that any parameters that you need to pass to the non-SSL service are passed through.

const http = require('http')
const port = 80;
const MAX_SCHEDULE_AGE = 30;
const SERVICE_URL=`YOUR_SERVICE_URL`

var schedule = '[]';
var lastUpdate = new Date(1,1,1);


function timeDifference(a,b) { 
    var c = (b.getTime() - a.getTime())/1000;
    return c;
}

function sendSchedule(resp) {
    resp.setHeader('Access-Control-Allow-Origin', '*');
	resp.setHeader('Access-Control-Request-Method', '*');
	resp.setHeader('Access-Control-Allow-Methods', 'OPTIONS, GET');
	resp.setHeader('Access-Control-Allow-Headers', '*');
    resp.end(schedule);
}
const requestHandler = (request,response) => {
    var now = new Date();
    var diff = timeDifference(lastUpdate, now);
    if(diff>MAX_SCHEDULE_AGE) {
        console.log('schedule is stale. updating');
        sendSchedule(response);
        return;
    }
    updateSchedule((d)=> {
        console.log('schedule updated')
        sendSchedule(response);
    });
    console.log(request.url);

}

const server = http.createServer(requestHandler);

const https = require('https');


function updateSchedule(onUpdate) { 
    https.get(SERVICE_URL, (resp) => {
        let data = '';
        resp.on('data', (chunk) => {
            data += chunk;
        });
        resp.on('end', ()=> {
            schedule = data;
            lastUpdate = new Date();
            if(onUpdate) {
                onUpdate(schedule);
            }
        })
    });
}

server.listen(port, (err) => {

    if(err) {
        return console.log('something bad happened');
    }
    console.log(`server is listening on port ${port}`);
    updateSchedule();
}) 

One of the other advantages of having this proxy service is that there is a now a layer for hiding any additional information that is necessary for accessing the service of interest.  For example, if you are communicating with a service that requires some key or app id for access, that information would never flow through to the client.

Some configuration was necessary for deployment, but not much.  I had to add a simple app.yaml file to the project.  These are the contents.

# [START runtime]
runtime: nodejs10
# [END runtime]

Deployment of the application was unexpectedly easy.  I already had the source code stored in a git repository.  App Engine exposes a Linux terminal through the browser.  I cloned my repository and typed a few commands.

$  export PORT=8080 && npm install
$  gcloud app create
$  gcloud app deploy

After answering YES to a configuration prompt, the application was deployed and running.

One might wonder why I have the code for my application hosted in two different services.  I could have placed the entire thing in App Engine.  My motivation for separating them is that I plan to have some other applications interface with the same service.  So I wanted to keep the code (for specific clients of the code) separate from the service interface.

Augmented Reality with Samsung XR SDK

Samsung showed the XR SDK at the 2018 Developers Conference. While Microsoft has generally presented their reality technologies as being along a spectrum (ranging from completely enveloping the user to only placing overlays on the real world) it has always been something that has involved a head mounted device. Samsung presents AR as something that is either viewed through a head mounted device or something that a person views through a portable hand held window into another world.  The language used by various companies varies a bit. Microsoft calls the their range of technologies “mixed reality.” Samsung calls theirs SXR which stands for Samsung Extended Reality.

It was several years ago that Samsung first showed it’s take on VR with the release of the Note 4 and the developer’s edition GearVR. The GearVR is now available as a consumer product, but Samsung took an economical approach to initial hardware for head mounted augmented reality. Instead of creating custom hardware they took some off the shelf products and mixed them together to make an economical headset.

Samsung AR Headset
Experimental AR Headset using off the shelf parts
Part Description Cost Source
AR Headset 90° FOV “Drop-In” phones 4.5 inches to 5.5 inches, 180g 65.99 USD
External Camera ELP VGA USB camera module with 100° FOV lens 24.69 USD
OTG connector Wavlink USB 3.1 Type C Male to USB 3.0 Type A Female OTG Data Connector Cable Adapter 5.99 USD Amazon
Total Cost 95.USD

The Samsung XR SDK is almost a super set of the the GearVR SDK. I say “almost” because with a proper super set you would find all the same class names that you would expect from the GearVR SDK. In the Samsung XR SDK the classes exists within a new namespace and have been renamed. GearVR programs could be ported over with some changes to the class names being invoked.

In development is an API standard for AR/VR experiences named OpenXR. Once the standard is defined and released Samsung plans for their XR SDK to be an implementation of this standard.

While the GearVR SDK was specifically for Samsung devices and the Samsung headset the Samsung XR SDK will run on non-Samsung devices for through-the-window AR but will run on the Oculus GO and Samsung devices for stereoscopic experiences.

 

Linux On Dex: Works on WiFi Tab S4 Models Only

Update 2018-Dec-11: I’ve spoken to a LoD team member and to jump straight to the point of you have a LTE Tab S4 then simply put the required update isn’t available at this time and there is no information on when it will be available.

Some people trying to install Linux on Dex are running into an obstacle. After installing he app and trying to run it they get the following error message.

Linux on Dex requires your device to have the latest software o support some features.

After this message is acknowledge the application closes. If someone with this error checks for updates in the app store or for updates to the operating system they get notification that everything is up to date. What’s going on? I contacted LoD support about this and got back the following response.

Currently, the Linux on DeX(beta) requires latest SW for Galaxy Note9 and Galaxy Tab S4. SW update schedule may vary depends on the region and carrier.

Currently, the Linux on DeX(beta) requires latest SW for Galaxy Note9 and Galaxy Tab S4.
SW update schedule may vary depends on the region and carrier.

What does this mean? It means that your device doesn’t have a update that is required for DeX and that your carrier might not have released it.  Devices sold through a carrier can be a bit slower in receiving their updates. Samsung hasn’t been specific on the updated needed.  I’ve communicated with someone on the Linux on Dex team and was told that LTE tablets in general do not have the update that is required for Linux on Dex. Additionally the person told me that there is no information available on when particular updates will work their way through certain carriers.

BTW: Unlocking your device and installing a SIM from another carrier will not change this; this behaviour is dependent on the carrier for which the device was made, not on the SIM that happens to be in the device at the time.