Bixby Developer Studio

Samsung says they would like to have AI implemented in all of their products by 2020. From the visual display shown during the SDC 2018 conference it appears their usage of “all” is intended to be widely encompassing. Phones, car audio systems, refrigerators, air conditioners…

Samsung is inviting developers to start engaging in development for their conversational AI. Now they have made the same tools that they use for Bixby development internally available publically. The development portal and the development tools for Windows and OS X are available now at: https://bixbydevelopers.com

Galaxy Home, a Bixby enabled smart speaker, was showcased as a target implementation for the SDK.

The “Media Control API” will be available to content partners this December for adding deeper control into applications. Samsung says Netflix and Hulu are on board and will begin development with it next year.

The Samsung Frame TVs are also being opened to developers by way of the Ambient Mode SDK. This will allow developer content to show when the TV is in it’s standby mode.

Advertisements

Samsung HOLOLAB

1T3A0114.JPG

One of the technology implementations displayed at SDC 2018 was their HOLOLAB. Curious as to what it is I walked over. It was a rig for performing 3D scanning of subjects. A few of the tangible outcomes included some pretty elastic 3D printed statues of people, a volume metric display showing a subject, and a display showing one of the scanned subjects dancing.

I was more curious about the implementation. I got a picture peering into the space to see how the physical structure was built. Inside the space the ambient lighting was ramped up by way of LEDs distributed around the space and 52 Point Grey Research cameras, all mounted to various places on an 8020 rig.

The software is custom built. I asked of its commercial availability and the Samsung rep I spoke to told me, “it was a work in progress.” The rep indicated that they are planning to have some popup locations available in the future for more people to try it and receive a copy of their models.

It is still early in the conference, but I might take a moment to try it out myself.

1T3A0122.JPG
Inside the HoloLab

1T3A0115.JPG

 

Tizen 5.0 Released

Tizen-Pinwheel-On-Light-RGB

Tizen 5.0 was released a few days ago. This is a week in advance of the Samsung Developer’s Conference 2018. For those keeping count until now the most recent version of Tizen prior to now (4.0) could be found on the Galaxy Watch and Samsung TVs. There are Mobile devices that run Tizen. But I am in the USA and those devices are not sold here (so I won’t speak on them much).

What’s new in Tizen 5.0?  There is improved IoT support, support for Bixby, and support for glTF. glTF is a format that aims to provide efficient loading of 3D scenes, but it is being added to Tizen with intent for it to be applied to watch faces.  With the upgrade we are also getting improved debugging support and a new version of Tizen.Net.  It also looks that they are deprecating the UI Builder tool in Tizen.

I get the feeling that I’ll hear more about this update when I go to the conference next week. I’ll post more from the conference as I find out 🙂

  • C# API for Display Control
  • Native API for Multi-LED control
  • Compressed File System
  • Low Memory Management
  • Upgrade to source libraries
    • icu (60.2)
    • sqlite (3.24.0)
    • json-glib (1.4.2)
    • wayland (1.15.0)
    • efl (1.21)
  • Updated Watchface Complication Framework
  • App Control API
  • Minicontrol API
  • WebView Control added
  • Network Firewall
  • Bluetooth 5.0
  • Neural Network Runtime

Samsung Developer Conference 2018 Registration Open

I previously mentioned that the dates for the Samsung Developer’s conference were announced. Registration is now open.  If you register before September 12 you can register for the lowest available price. The registration form is available over at https://www.samsungdeveloperconference.com/ . Registration is also possible on site. Currently Registration is 299 USD (+tax). After the 12th it will go up to 399 USD. On site registration is 499 USD.

Registration for the Samsung Developer's Conference, 2018

Obtaining the Connection String for a Provisioned Windows IOT Device

Playing with the code that I was using to get data from my car and stream it to the cloud I did something that I knew was a no-no; I hard coded the connection string in the code. There’s a number of reasons to not do this*; it’s less secure as someone can potentially extract the connection string and use it for unauthorized access and if the connection string ever needs to change then code needs to be recompiled and redeployed.

When a Windows IOT device is provisioned there is a connection string that is managed by the device; your application can take advantage of this and need not worry about the details of how it is stored. To make use of this there are a few libraries that you need to add to your UWP project. These include the followings.

  • Microsoft.Azure.Devices
  • Microsoft.Azure.Devices.Client
  • Microsoft.Devices.Tpm

With the classes in these libraries you can obtain the ID of the device and then use that ID to request an Azure DeviceClient class that is initialized with the connection string that the device is managing.

Here’s the code to do this.

DeviceClient _deviceClient;

void InitClient()
{            
    TpmDevice tpm = new TpmDevice(0);
    string hostName = tpm.GetHostName();
    string deviceId = tpm.GetDeviceId();
    string sasToken = tpm.GetSASToken();            
    var client = DeviceClient.Create(
        hostName, 
        AuthenticationMethodFactory.CreateAuthenticationWithToken(deviceId, sasToken),
        TransportType.Mqtt
    );
}

Connecting to Bluetooth RFCOMM with UWP

Soon I will be finished with a project for streaming engine data from a car and into an Azure Data Lake for analysis. I plan to post the full code for this project. But before I do I wanted to talk just about making the Bluetooth connection. The last time I wrote on this procedure it was for a Windows phone 8 project. While part of the UWP ancestry the method of connection in that project would not work in a UWP application today. Things have changed since then and the code that I used there for establishing a connection would not work today.  Treat the code used in this project as psuedo code for now. While it was copied from a working project I’ve done some modifications to simplify it and focus on a few things here.

Getting information from the engine is easy. Every car sold in the USA (and many  parts of the world) have a diagnostic connector that can be used to query various data from the car. Getting this data is just a matter of having an interface that uses what ever protocol (there’s more than one) that your car happens to use. Don’t worry, it’s not necessary to figure this out yourself. There are products that will abstract this away and provide a common interface and protocol that can be used to communicate with all of these different protocols. One of the most popular solutions is based on a chipset known as the ELM327. Solutions based on the ELM327 (or an ELM327 clone) will give you a serial interface through which you can send AT commands and other queries to get this information. The products that I am using use a Bluetooth interface for communicating with a computing device. You can also find implementations that use WiFi and RS232.

For development and testing there’s going to be a lot of time spent with a computer in or near a car (unless you also go out and purchase an OBD emulator). To minimize the amount of time that you have to spend around a car you may want to power one of the adapters outside of a car. I did this using a battery (sometimes a power supply) and and an OBD wire harness. Pin number 4 on the harness needs to be connected to ground and pin 16 to positive. A voltage range of 9 to 16 volts should work fine. With the device connected to power you can query information about the device itself but nothing about the car.  While this doesn’t sound very useful at first it’s enough to get the basic Bluetooth connectivity code working.

OBDSetup

For development hardware I am using a Dragonboard 410x running Windows IOT. The UWP based solution, while targeting Windows IOT, runs just fine from a PC. I believe it will run fine from a Raspberry Pi 3 also. Just for fun I tried an old Windows 10 device that I still have in the house and the code ran without any modifications needed from there. So grab what ever Windows 10 device that you have and try it out!

For the application to be able to connect to the device over Bluetooth there are some capabilities that must be declared in the UWP application. I don’t suggest using the Visual Studio for doing this. It doesn’t expose all of the necessary capability settings that are needed. It’s easier to do this in the text editor.

Right-clicking on the project’s manifest gives the option to open it in the code editor. Scrolling down to the Capabilities section I added the following. Some of these properties are needed to access the Bluetooth serial port. Some others are there because they are parts of the project that I will talk about in future post.

  
  <Capabilities>
    <apability Name="internetClient" />
    <DeviceCapability Name="bluetooth" />
    <DeviceCapability Name="proximity" />
    <DeviceCapability Name="location" />
    <DeviceCapability Name="bluetooth.rfcomm">
      <Device Id="any">
        <Function Type="name:serialPort" />
      </Device>
    </DeviceCapability>
  </Capabilities>

I had already paired the Bluetooth device with my Windows IOT device. While the devices are paired with each other the application doesn’t know about the pairing. It still needs to discover the device. To do this I use a device watcher to scan for devices. I know the one that I want to find is named “OBD”. I’ve hard coded this into my solution. The end solution that I’m making doesn’t have a UI, If it did then I would have also made use of the UI that allows a user to select a device being paired.

DataReader _receiver;
DataWriter _transmitter;
StreamSocket _stream;
DeviceWatcher _deviceWatcher;

string[] requestedProperties = new string[] { "System.Devices.Aep.DeviceAddress", "System.Devices.Aep.IsConnected" };
_deviceWatcher = DeviceInformation.CreateWatcher("(System.Devices.Aep.ProtocolId:=\"{e0cbf06c-cd8b-4647-bb8a-263b43f0f974}\")",
                                               requestedProperties,
                                               DeviceInformationKind.AssociationEndpoint);
_deviceWatcher.Stopped += (sender,x)=> {
    _isScanning = false;
    Log("Device Scan Halted");
};
EngineDataList.Add("started");
_deviceWatcher.Added += async (sender, devInfo) =>
{
    Log($"Found device {devInfo.Name}");
    System.Diagnostics.Debug.WriteLine(devInfo.Name + "|" + devInfo.Kind + "|" + devInfo.Pairing);
    if (devInfo.Name.Equals("OBDII"))
    {
     // More Code Goes Here
     //Lets talk about that in a sec
    }
};

_deviceWatcher.Start()

The above code will give result in the DeviceInfo object for the Bluetooth adapter being returned. Now we can connect to it and open up streams for reading and writing. The code for creating these streams would be where I have the comment placed in the above. The following is the code that would go in that place.

try
{
    DeviceAccessStatus accessStatus = DeviceAccessInformation.CreateFromId(devInfo.Id).CurrentStatus;
    if (accessStatus == DeviceAccessStatus.DeniedByUser)
    {
        Debug.WriteLine("This app does not have access to connect to the remote device (please grant access in Settings > Privacy > Other Devices");
        return;
    }
    var device = await BluetoothDevice.FromIdAsync(devInfo.Id);
    Debug.WriteLine(device.ClassOfDevice);

    var services = await device.GetRfcommServicesAsync();
    if (services.Services.Count > 0)
    {
        Log("Connecting to device stream");
        var service = services.Services[0];
        _stream = new StreamSocket();
        await _stream.ConnectAsync(service.ConnectionHostName,
        service.ConnectionServiceName);
        _receiver = new DataReader(_stream.InputStream);
        _transmitter = new DataWriter(_stream.OutputStream);
        
        ReceiveLoop();
        QueryLoop();
        _deviceWatcher.Stop();
    }
}catch(Exception exc)
{
    Log(exc.Message);
}
catch
{
    Log("Native Error");
}

After this runs the devices are connected to each other. The devices would interact by the Win IOT device sending a query and getting a response. For now the only query that will be sent is the string “AT\n”. On devices that accept AT command sets the command AT should only result in the response OK. This is useful for testing the connection without actually having the device perform an operation that would change the state of the system being used.

void QueryLoop()
{
    Task t = Task.Run(async () =>
    {
        
        while (true)
        {
            _transmitter.WriteString(msg);
            _transmitter.WriteString("\r");
            await _transmitter.StoreAsync();
            await Task.Delay(50);
        }
    }
    );
}

void ReceiveLoop()
{
    Task t = Task.Run(async () => {
        while (true)
        {
            uint buf;
            buf = await _receiver.LoadAsync(1);
            if (_receiver.UnconsumedBufferLength > 0)
            {
                string s = _receiver.ReadString(1);
                receiveBuffer.Append(s);
                if (s.Equals("\n")||s.Equals("\r"))
                {
                    try
                    {
                      Debug.WriteLine("Message Received:"+receiveBuffer.ToString());
                        receiveBuffer.Clear();
                    }
                    catch(Exception exc)
                    {
                        Log(exc.Message);
                     
                    }
                }
            }else
            {
                await Task.Delay(TimeSpan.FromSeconds(0));
            }
        }
    });
}

When the above runs the debug output stream will print a series of OK responses. To do something more useful with it we need to know what commands to send to it and how to parse the responses. If you send the command 01 0C while the adapter is connected to a car you will get back a hex response that starts with 41 0C followed by some additional hex numbers. The additional hex numbers are the RPMs of the car. That’s one of the metrics I started off querying because it’s something that I can change without driving the car; for this I can pull the car out of the garage, put it in the driveway, and rev the engine without worrying about carbon monoxide poisoning. I’ll talk about how to find out what metrics that you can query in your car in my next post on this project.