HoloLens app research

Platform

The choice of platform for the 2nd iteration of Peach Reality project was heavily dictated by the device we were meant to use - Microsoft HoloLens. HoloLens runs on Windows Holographic platform, which is a brand-new version of the famous operating system meant to support all augmented and mixed reality developments Microsoft is currently working on or will work in the future.[1] That said, considering the scope of our project we could potentially target not only Mixed Reality (MR) headsets such as HoloLens but also various Virtual Reality (VR) headsets, for example Oculus Rift or HTC Vive. Unfortunately at the moment each device requires its own proprietary libraries to be used in development, so developing a cross-platform application targeting MR and VR is quite a challenging task. Not so long ago, on 7th of December 2016, the biggest players in the VR industry have come together to establish the Global Virtual Reality Association which will take responsibility for developing VR standards.[2] While this event is likely to affect our project in the future, we cannot afford to wait until said standards will be finalised so we're planning to develop our application specifically for HoloLens, while trying to abstract as many features as possible behind Adapter interfaces[3] to make it easier to migrate our project to other headsets in the future.

Tools

Applications for HoloLens can be written using Universal Windows Platform (UWP), which is a new application architecture Microsoft has been promoting since Windows 10 was released.[4] There are several core ideas behind UWP:

UWP applications target families of devices instead of focusing on a single operating system.
UWP exposes a common API among all devices. Specific features of each device can be then accessed by enabling and Extension SDK.
All applications are available through a single, united app store.
Interfaces and controls adapt to interfaces of specific devices.

Since our application involves working with 3D objects, the only feasible options we have is to develop our application using Unity 3D[5] or by developing a UWP application that references HoloLens' DirectX 11 APIs[6] directly. While both options have their advantages, the learning curve for the latter is way steeper than that for the former so considering the limited amount of time we have to work on this project, we decided to stick with Unity.

It must be noted that Unity just introduces a level of abstraction over UWP by providing its own libraries, a 3D engine ready for use out-of-the-box and various other useful tools such as shaders, model importers, file storage managers and so on. That means that in the end we still have to compile our Unity project into a UWP project to upload it to HoloLens, but with Unity this process is way more trivial and straightforward than it is with out it.

Additionally, we decided to use Visual Studio (VS) as our primary IDE as it works very well with Unity and supports the HoloLens emulator, which allows user to test their HoloLens application without having the actual device. Using VS will not only significantly speed up the development process, but will also allow us to publish our application in the Windows Store, from where our client will be able to download it and update it when necessary.[7, 8]

Microsoft is developing a collection of tools and scripts called HoloToolkit[9], which is a collection of tools and helper scripts to address common issues in HoloLens development, such as scripts to handle spatial mapping, voice command recognition and hologram sharing. The version of HoloToolkit aimed at Unity[10] contains a lot of useful scripts that can all be used with Unity, including those that were not exclusively designed for HoloLens. For example the toolkit contains scripts to create tagalong object in Unity - they will the main camera in the scene around making sure they always stay in user's field of view, and scripts to create billboard object which will rotate to ensure that their front face is always facing the user.

Languages

Unity supports scripting in a number of languages, including C#, UnityScript (Unity's custom flavour of JavaScript) and Boo, an object-oriented Python-like language. Although every member of our team has some experience with JavaScript, we decided to stick with C# for a number of reasons.

First of all, the majority of Unity development (around 80%) is done in C#, which means that the official and user-contributed knowledge base for this language is likely to be the biggest out of the 3. This is a major factor for our team as none of us had any prior experience with Unity nor C# and considering the tight development schedule, we wanted to minimise the amount of work spent learning the language as opposed to using it.

Secondly, C# as a language is being maintained by Microsoft and considering that our app is designed specifically for Microsoft HoloLens which has most of its libraries written in C# (including Unity-HoloToolkit) for the sake of code consistency and to avoid mixing up different languages we decided to use C# for all HoloLens development.