Augmented Reality with Samsung XR SDK

Samsung showed the XR SDK at the 2018 Developers Conference. While Microsoft has generally presented their reality technologies as being along a spectrum (ranging from completely enveloping the user to only placing overlays on the real world) it has always been something that has involved a head mounted device. Samsung presents AR as something that is either viewed through a head mounted device or something that a person views through a portable hand held window into another world.  The language used by various companies varies a bit. Microsoft calls the their range of technologies “mixed reality.” Samsung calls theirs SXR which stands for Samsung Extended Reality.

It was several years ago that Samsung first showed it’s take on VR with the release of the Note 4 and the developer’s edition GearVR. The GearVR is now available as a consumer product, but Samsung took an economical approach to initial hardware for head mounted augmented reality. Instead of creating custom hardware they took some off the shelf products and mixed them together to make an economical headset.

Samsung AR Headset
Experimental AR Headset using off the shelf parts
Part Description Cost Source
AR Headset 90° FOV “Drop-In” phones 4.5 inches to 5.5 inches, 180g 65.99 USD
External Camera ELP VGA USB camera module with 100° FOV lens 24.69 USD
OTG connector Wavlink USB 3.1 Type C Male to USB 3.0 Type A Female OTG Data Connector Cable Adapter 5.99 USD Amazon
Total Cost 95.USD

The Samsung XR SDK is almost a super set of the the GearVR SDK. I say “almost” because with a proper super set you would find all the same class names that you would expect from the GearVR SDK. In the Samsung XR SDK the classes exists within a new namespace and have been renamed. GearVR programs could be ported over with some changes to the class names being invoked.

In development is an API standard for AR/VR experiences named OpenXR. Once the standard is defined and released Samsung plans for their XR SDK to be an implementation of this standard.

While the GearVR SDK was specifically for Samsung devices and the Samsung headset the Samsung XR SDK will run on non-Samsung devices for through-the-window AR but will run on the Oculus GO and Samsung devices for stereoscopic experiences.

 

Linux On Dex: Update Your Software

Update 2018-Dec-11: I’ve spoken to a LoD team member and to jump straight to the point of you have a LTE Tab S4 then simply put the required update isn’t available at this time and there is no information on when it will be available.

Some people trying to install Linux on Dex are running into an obstacle. After installing he app and trying to run it they get the following error message.

Linux on Dex requires your device to have the latest software o support some features.

After this message is acknowledge the application closes. If someone with this error checks for updates in the app store or for updates to the operating system they get notification that everything is up to date. What’s going on? I contacted LoD support about this and got back the following response.

Currently, the Linux on DeX(beta) requires latest SW for Galaxy Note9 and Galaxy Tab S4. SW update schedule may vary depends on the region and carrier.

Currently, the Linux on DeX(beta) requires latest SW for Galaxy Note9 and Galaxy Tab S4.
SW update schedule may vary depends on the region and carrier.

What does this mean? It means that your device doesn’t have a update that is required for DeX and that your carrier might not have released it. Devices sold through a carrier can be a bit slower in receiving their updates. Samsung hasn’t been specific on the updated needed.  I’ve communicated with someone on the Linux on Dex team and was told that LTE tablets in general do not have the update that is required for Linux on Dex. Additionally the person told me that there is no information available on when particular updates will work their way through certain carriers.

BTW: Unlocking your device and installing a SIM from another carrier will not change this; this behaviour is dependent on the carrier for which the device was made, not on the SIM that happens to be in the device at the time.

Samsung Announced Exynos 9 with NPU

Exynos-9-9820_main

Consistent with what they said at the developer’s conference about wanting to extend the reach of their A.I. Samsung has announced a new System on Chip (SoC) with some A.I. related features. The Exynos 9 Series 9820 processor. The processor contains an NPU, a unit for processing neural networks at speeds faster than what could be done with a general purpose processor alone. The presence of this unit on the device hardware makes possible device side experiences that would have previously required that data be sent to a server for processing. This may also translate into improvements in AR and VR experiences.

The NPU isn’t the only upgrade that comes with the processor. Samsung says the 9820’s new fourth generation custom core delivers a 20% improvement in single core performance or 40% in power efficiency compared to is predecessor. Multicore performance is said to be increased around 15%. The Exynos 9820 also has a video encoder capable of decoding 4K video at up to 150 frames per second in 10-bit color. The processor goes into mass production at the end of this year.

Source: Samsung

Bixby Developer Studio

Samsung says they would like to have AI implemented in all of their products by 2020. From the visual display shown during the SDC 2018 conference it appears their usage of “all” is intended to be widely encompassing. Phones, car audio systems, refrigerators, air conditioners…

Samsung is inviting developers to start engaging in development for their conversational AI. Now they have made the same tools that they use for Bixby development internally available publically. The development portal and the development tools for Windows and OS X are available now at: https://bixbydevelopers.com

Galaxy Home, a Bixby enabled smart speaker, was showcased as a target implementation for the SDK.

The “Media Control API” will be available to content partners this December for adding deeper control into applications. Samsung says Netflix and Hulu are on board and will begin development with it next year.

The Samsung Frame TVs are also being opened to developers by way of the Ambient Mode SDK. This will allow developer content to show when the TV is in it’s standby mode.

Samsung HOLOLAB

1T3A0114.JPG

One of the technology implementations displayed at SDC 2018 was their HOLOLAB. Curious as to what it is I walked over. It was a rig for performing 3D scanning of subjects. A few of the tangible outcomes included some pretty elastic 3D printed statues of people, a volume metric display showing a subject, and a display showing one of the scanned subjects dancing.

I was more curious about the implementation. I got a picture peering into the space to see how the physical structure was built. Inside the space the ambient lighting was ramped up by way of LEDs distributed around the space and 52 Point Grey Research cameras, all mounted to various places on an 8020 rig.

The software is custom built. I asked of its commercial availability and the Samsung rep I spoke to told me, “it was a work in progress.” The rep indicated that they are planning to have some popup locations available in the future for more people to try it and receive a copy of their models.

It is still early in the conference, but I might take a moment to try it out myself.

1T3A0122.JPG
Inside the HoloLab

1T3A0115.JPG

 

Tizen 5.0 Released

Tizen-Pinwheel-On-Light-RGB

Tizen 5.0 was released a few days ago. This is a week in advance of the Samsung Developer’s Conference 2018. For those keeping count until now the most recent version of Tizen prior to now (4.0) could be found on the Galaxy Watch and Samsung TVs. There are Mobile devices that run Tizen. But I am in the USA and those devices are not sold here (so I won’t speak on them much).

What’s new in Tizen 5.0?  There is improved IoT support, support for Bixby, and support for glTF. glTF is a format that aims to provide efficient loading of 3D scenes, but it is being added to Tizen with intent for it to be applied to watch faces.  With the upgrade we are also getting improved debugging support and a new version of Tizen.Net.  It also looks that they are deprecating the UI Builder tool in Tizen.

I get the feeling that I’ll hear more about this update when I go to the conference next week. I’ll post more from the conference as I find out 🙂

  • C# API for Display Control
  • Native API for Multi-LED control
  • Compressed File System
  • Low Memory Management
  • Upgrade to source libraries
    • icu (60.2)
    • sqlite (3.24.0)
    • json-glib (1.4.2)
    • wayland (1.15.0)
    • efl (1.21)
  • Updated Watchface Complication Framework
  • App Control API
  • Minicontrol API
  • WebView Control added
  • Network Firewall
  • Bluetooth 5.0
  • Neural Network Runtime