Augmented Reality with Samsung XR SDK

Samsung showed the XR SDK at the 2018 Developers Conference. While Microsoft has generally presented their reality technologies as being along a spectrum (ranging from completely enveloping the user to only placing overlays on the real world) it has always been something that has involved a head mounted device. Samsung presents AR as something that is either viewed through a head mounted device or something that a person views through a portable hand held window into another world.  The language used by various companies varies a bit. Microsoft calls the their range of technologies “mixed reality.” Samsung calls theirs SXR which stands for Samsung Extended Reality.

It was several years ago that Samsung first showed it’s take on VR with the release of the Note 4 and the developer’s edition GearVR. The GearVR is now available as a consumer product, but Samsung took an economical approach to initial hardware for head mounted augmented reality. Instead of creating custom hardware they took some off the shelf products and mixed them together to make an economical headset.

Samsung AR Headset
Experimental AR Headset using off the shelf parts
Part Description Cost Source
AR Headset 90° FOV “Drop-In” phones 4.5 inches to 5.5 inches, 180g 65.99 USD
External Camera ELP VGA USB camera module with 100° FOV lens 24.69 USD
OTG connector Wavlink USB 3.1 Type C Male to USB 3.0 Type A Female OTG Data Connector Cable Adapter 5.99 USD Amazon
Total Cost 95.USD

The Samsung XR SDK is almost a super set of the the GearVR SDK. I say “almost” because with a proper super set you would find all the same class names that you would expect from the GearVR SDK. In the Samsung XR SDK the classes exists within a new namespace and have been renamed. GearVR programs could be ported over with some changes to the class names being invoked.

In development is an API standard for AR/VR experiences named OpenXR. Once the standard is defined and released Samsung plans for their XR SDK to be an implementation of this standard.

While the GearVR SDK was specifically for Samsung devices and the Samsung headset the Samsung XR SDK will run on non-Samsung devices for through-the-window AR but will run on the Oculus GO and Samsung devices for stereoscopic experiences.



Mango Beta 2 Available for Phones Today!

The Beta 2 Mango Windows Phone Tools are available to developers today! Included with the beta is the ability for developers registered with the AppHub to flash their retail devices.

I know there are some non-developers out there that want to also flash their phones and they may wonder how they get get their phones reflashed with the Mango beta. For the time being they cannot. There is an inherent risk in reflashing the phone; you could end up with a bricked phone if something goes bad. If this happens Microsoft has budgeted to take care of repairing up to one phone per developer. But Microsoft doesn’t see this risk as being appropriate for user audiences. [Some] developers on the other hand are willing to risk their device’s life and limb to have early access to something new. If you brick your device today Microsoft won’t be prepared to act on it for another couple of weeks. That’s not the best case scenario. But the alternative was to wait another couple of weeks before releasing the Mango tools. If you don’t feel safe walking the tight rope without a safety net then don’t re-flash your device yet.

According to the Windows Phone Developer site if you are a registered developer you will receive an e-mail inviting you to participate in early access to Mango.

Changing the Pitch of a Sound

I got a tweet earlier today from some one asking me how to change the pitch of a wave file. The person asking was aware that SoundEffectInstance has a setting to alter pitch but it wasn’t sufficient for his needs. He needed to be able to save the modified WAV to a file. It’s something that is easy to do. So I made a quick example

Video Example

I used a technique that comes close to matching linear interpolation. It get’s the job done but isn’t the best technique because of the opportunity for certain types of distortion to introduced. Methods with less distortion are available at the cost of potentially more CPU cycles. For the example I made no matter what the original sample rate was I am playing back at 44KHz and adjusting my interpolation accordingly so that no unintentional changes in pitch are introduced.

To do the work I’ve created a class named AdjustedSoundEffect. It has a Play() method that takes as it’s argument the factor by which the pitch should be adjusted where 1 plays the sound at the original pitch, 2 plays it at twice its pitch, and 0.5 plays it at half its pitch.

If you are interested the code I used is below.

using System;
using System.IO;
using System.Net;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Documents;
using System.Windows.Ink;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Animation;
using System.Windows.Shapes;
using Microsoft.Xna.Framework.Audio;

namespace J2i.Net.VoiceRecorder.Utility
    public class AdjustedSoundEffect
        //I will always playback at 44KHz regardless of the original sample rate. 
        //I'm making appropriate adjustments to prevent this from resulting in the
        //pitch being shifted. 
        private const int PlaybackSampleRate = 16000;
        private const int BufferSize = PlaybackSampleRate*2;

        private int _channelCount = 1;
        private int _sampleRate;
        private int _bytesPerSample = 16;
        private int _byteCount = 0;
        private float _baseStepRate = 1;
        private float _adjustedStepRate;
        private float _index = 0;
        private int playbackBufferIndex = 0;
        private int _sampleStep = 2;

        private bool _timeToStop = false;

        private byte[][] _playbackBuffers;

        public bool IsPlaying { get; set;  }

        public object SyncRoot = new object();

        private DynamicSoundEffectInstance _dse;

        public static AdjustedSoundEffect FromStream(Stream source)
            var retVal = new AdjustedSoundEffect(source);
            return retVal;

        public AdjustedSoundEffect()
            _playbackBuffers = new byte[3][];
            for (var i = 0; i < _playbackBuffers.Length;++i )
                _playbackBuffers[i] = new byte[BufferSize];
                _dse = new DynamicSoundEffectInstance(PlaybackSampleRate, AudioChannels.Stereo);
            _dse.BufferNeeded += new EventHandler<EventArgs>(_dse_BufferNeeded);

        void SubmitNextBuffer()
            lock (SyncRoot)
                byte[] nextBuffer = _playbackBuffers[playbackBufferIndex];
                playbackBufferIndex = (playbackBufferIndex + 1)%_playbackBuffers.Length;
                int i_step = 0;
                int i = 0;

                int endOfBufferMargin = 2*_channelCount;
                for (;
                    i < (nextBuffer.Length / 4) && (_index < (_sourceBuffer.Length - endOfBufferMargin));
                    ++i, i_step += 4)

                    int k = _sampleStep*(int) _index;
                    if (k > _sourceBuffer.Length - endOfBufferMargin)
                        k = _sourceBuffer.Length -endOfBufferMargin ;
                    nextBuffer[i_step + 0] = _sourceBuffer[k + 0];
                    nextBuffer[i_step + 1] = _sourceBuffer[k + 1];
                    if (_channelCount == 2)
                        nextBuffer[i_step + 2] = _sourceBuffer[k + 2];
                        nextBuffer[i_step + 3] = _sourceBuffer[k + 3];
                        nextBuffer[i_step + 2] = _sourceBuffer[k + 0];
                        nextBuffer[i_step + 3] = _sourceBuffer[k + 1];

                    _index += _adjustedStepRate;

                if ((_index >= _sourceBuffer.Length - endOfBufferMargin))
                    _timeToStop = true;
                for (; i < (nextBuffer.Length/4); ++i, i_step += 4)
                    nextBuffer[i_step + 0] = 0;
                    nextBuffer[i_step + 1] = 0;
                    if (_channelCount == 2)
                        nextBuffer[i_step + 2] = 0;
                        nextBuffer[i_step + 3] = 0;

        void _dse_BufferNeeded(object sender, EventArgs e)

        private byte[] _sourceBuffer;

        public AdjustedSoundEffect(Stream source): this()
            byte[] header = new byte[44];
            source.Read(header, 0, 44);

            // I'm assuming you passed a proper wave file so I won't bother 
            // verifying  that  the  header  is properly formatted and will 
            // accept it on faith :-)

            _channelCount = header[22] + (header[23] << 8);
            _sampleRate = header[24] | (header[25] << 8) | (header[26] << 16) | (header[27] << 24);
            _bytesPerSample = header[34]/8;
            _byteCount = header[40] | (header[41] << 8) | (header[42] << 16) | (header[43] << 24);
            _sampleStep = _bytesPerSample*_channelCount;
            _sourceBuffer = new byte[_byteCount];
            source.Read(_sourceBuffer, 0, _sourceBuffer.Length);

            _baseStepRate = ((float)_sampleRate) / PlaybackSampleRate;

        /// <summary>
        /// </summary>
        /// <param name="pitchFactor">Factor by which pitch will be adjusted. 2 doubles the frequency,
        /// // 1 is normal speed, 0.5 halfs the frequency</param>
        public void Play(float pitchFactor)
            _timeToStop = false;

            _index = 0;
            lock (SyncRoot)
                _adjustedStepRate = _baseStepRate * pitchFactor;
                _index = 0;
                playbackBufferIndex = 0;
                IsPlaying = true;

        public void Stop()

Adding an E-Mail Account to the WP Emulator

For one reason or another you may find that you want to add a real e-mail account to the Windows Phone emulator. Unfortunately the emulator doesn’t directly expose a way for you to do this; the settings area on the phone doesn’t display the tile to access the e-mail settings. You can get to the settings application indirectly though. This path is convoluted, but it works.

You’ll need to make a simple application that does nothing more than show a phone call task. Once the task is displayed accept the phone call then select the option to add another caller. This takes you to the People Hub. Swipe through the People Hub to the &quotWhat’s New" and you will be prompted to add a Facebook or Twitter account. Select the option to do this (even though you are not really adding an account of that type) and when you asked what type of account you want to add you can select one of the e-mail account types.

Setting Custom Ringtones from Code [Mango:Beta 1]

Written against pre-release information

One of the new features coming with the next update to Windows Phone 7 is the ability to set custom ring tones. From within code you can make a ring tone available to a user (it’s up to the user to accept the ring tone, so user settings won’t ever be changed without user permission). I was looking at the new API for doing this, the SaveRingtonTask()

To use the API you first need to get the ringtone of interest into isolated storage. It can be either an MP3 file or a WMA file up to 30 seconds in length. If the file is a part of your application. Just set it’s build type to "Resource".

file settings

Getting the file from being packed in the application to isolated storage is a matter of reading from a resource stream and writing to isolated storage.

s =
    Application.GetResourceStream(new Uri("/MyApplicationName;component/1up.mp3",
    using (var f = IsolatedStorageFile.GetUserStoreForApplication().CreateFile("1up.mp3"))

        var buffer = new byte[2048];
        int bytesRead = 0;

            bytesRead = s.Stream.Read(buffer, 0, 1024);
            f.Write(buffer, 0, bytesRead);
        } while (bytesRead > 0);


Once the file is in isolated storage you must pass the URL to the SaveRingtoneTask(). URIs to isolated storage are preceded with "isostore:" (there is also an "appdata:" prefix, but we won’t be using it here). Give the ringtone a display name and call the show method to present the user with the option to save it. If you don’t set the

SaveRingtoneTask srt = new SaveRingtoneTask();
srt.DisplayName = "1up";
srt.IsoStore= new Uri("isostore:/1up.mp3", UriKind.Absolute);
srt.IsShareable = true;

Peer Communication on Windows Phone 7

Written against pre-release information

One of the new things that we get with Windows Phone 7 is socket support. While I expected to be able to open sockets to other machines with servers running on them one thing caught me by surprised; that you can also send communication from phone to phone using UDP. I’ve got to give credit to Ricky_T for pointint out the presence of this feature and posting a code sample. I wanted to try this out myself. So I made a version of the code sample that would run on both Windows Phone and on the desktop (Silverlight 4 in Out of Browser mode). I was pleasantly surprised to that I was able to open up peer communication between the desktop and phone without a problem. This capability provides a number of solutions for other problems that I’ve been considering, such as automatic discovery and configuration for communicating with services hosted on a user’s local network. 

Most of the code used in the desktop and phone version of this example are identical; I’ve shared some of the same files between projects. From the files that are not shared the counterparts in the phone and desktop version are still similar.  The core of the code is in a class called Peer. Let’s take a look at part of the body of that class. 


//Define the port and multicast address to be used for communication
private string _channelAddress = "";
private int _channelPort = 3007;

//The event to be raised when a message comes in
public event EventHandler<MessageReceivedEventArgs> MessageReceived; 

//the UDP channel over which communication will occur.
private UdpAnySourceMulticastClient _channel;

//Create tje cjamme;
public void Initialize()
    _channel = new UdpAnySourceMulticastClient(IPAddress.Parse(_channelAddress), _channelPort);

//Open the channel and start listening
public void Open()
    if (_channel == null)
    ClientState = ClientStatesEnum.Opening;

    _openResult = _channel.BeginJoinGroup((result) =>
                                                    ClientState = ClientStatesEnum.Opened;
                                                }, null);   


//The receive method is recursive. At the end of a call to receive it calls itself 
//so that the class can continue listening for incoming requests.
void Receive()
    byte[] _receiveBuffer = new byte[1024];

    _channel.BeginReceiveFromGroup(_receiveBuffer, 0, _receiveBuffer.Length, (r) =>
            IPEndPoint source;
            int size= _channel.EndReceiveFromGroup(r, out source);
            OnMessageReceived(_receiveBuffer, size,  source);                                                                                   
            catch (Exception )
    }, null);
public void Send(byte[] data)
        _channel.BeginSendToGroup(data, 0, data.Length, (r) => _channel.EndSendToGroup(r),null);

This class only sends and receives byte arrays. My only goal here was to see the code work so there are other considerations that I have decided to overlook for now. I made a client to use this code too. The client sends and receives plain text. Before sending a block of text it is necessary to convert the text to a byte array. The encoding classes in .Net will take care of this for me. When a message comes in I can also use an encoder to convert the byte array back to a string.

For this program I am adding the incoming message to a list along with the IP address from which it came

void _peer_MessageReceived(object sender, MessageReceivedEventArgs e)
    Action a = () =>
                        string message = System.Text.UTF8Encoding.Unicode.GetString(e.Data, 0, e.Size);
                        MessageList.Add(String.Format("{0}:{1}", e.Endpoint.Address.ToString(), message));
                        OnIncomingMessageReceived(message, e.Endpoint.Address.ToString());
    if (UIDispatcher == null)

public void SendMessage(string message)
    byte[] encodedMessage= UTF8Encoding.Unicode.GetBytes(message);

When the code is run on any combination of multiply phones or computers a message types on any one of the devices appears on all of them. Nice! Now to start making use of it.