Experiments

Below you can find the summary of all experiments we've conducted with the outcomes and successes of each one summarised. Additionally, in the sidebar menu you can find the links to pages where we provide additional commentary or videos for the most interesting experiments.

Experiment #	Title	Details	Results	Conclusion	Done By	Date
1	HoloLens Model Rotation	Allows user to inspect the model from different angles by rotating the model	Feature performed well when having the model yaw, although pitch and roll felt unintuitive and easy to get wrong	It was decided to keep this feature in the final prototype, but only the yaw motion as the others were not nice to use and overcomplicated it	Fraser Savage	11/11 - 13/11
2	HoloLens Model Viewer #1	Details can be found below.			Timur Kuzhagaliyev	14/11 - 24/11
2.1	- Model exploding	Allowed users to expand the model to study every component separately	Feature performed well and client liked the idea	It's been decided to keep this feature in the final prototype	Timur Kuzhagaliyev	14/11 - 16/11
2.2	- Colour adjustment	Allowed users to adjust hue and transparency of different parts of the model	Client liked the feature but adjusting hue was deemed unncessary	Removed the hue adjustment from application but left transparency adjustment in	Timur Kuzhagaliyev	17/11 - 19/11
2.3	- Voice commands	Allowed users to expand the model using "Switch View" voice command	Although sometimes unreliable feature performed well enough	It's been decided to keep voice commands in the application	Timur Kuzhagaliyev	20/11 - 22/11
2.4	- Bounding boxes	Generates grey bounding boxes around models that become visible on hover	It was agreed that bounding boxes are a feasible way to indicate which objects are active	It's been decided to keep bounding boxes in the final prototype with some cosmetic improvements	Timur Kuzhagaliyev	23/11 - 24/11
3	API First Prototype	Details can be found below.			Fraser Savage
3.1	- Basic model conversion	Converts an STL model to an OBJ model	Although achieved through calling C++ code the desired functionality works	It has been decided to keep this functionality in the final prototype and attempt to extend it to accomodate additional file types.	Fraser Savage	16/11 - 17/11
3.2	- Archive serving of model	Allows users to download a zip archive containing all the meshes for a model	The feature performed as required	It has not yet been decided whether to ship model metadata in the archive or separately	Fraser Savage	18/11 - 19/11
3.3	- Archive serving of DICOM	Allows users to download a zip archive containing all of the slices of a DICOM	The feature performed as required, although the archives are bit large	As with the archive serving of the model it has not yet been decided whether to ship DICOM metadata in the archive or separately. Future iterations of this may make adjustments to how all of the data is served because of the size of the archive.	Fraser Savage	18/11 -19/11
3.3	- Require authorisation to access data	Requires an authorisation token to be provided when attempting to access sensitive data through the API	The feature works as intended	It has been decided to keep this functionality in some form for the final prototype.	Fraser Savage	22/11 - 24/11
3.4	- User authentication	Assigns an authorisation token to a user if their provided credentials are correct	This feature works as intended, although handling of the credentials needs to be better secured on the server side	This functionality will be kept in the final prototype although the implementation security should be improved upon	Fraser Savage	26/11 - 27/11
4	HoloLens consumption of the API First Prototype	Allows user to connect to and authenticate with the API, then download a model archive and extract it so the model can be viewed	The feature works mostly as intended. The model once downloaded is not successfully opened by the model viewer, although the models downloaded appear to check out in 3D applications.	This feature needs improvement and fixing before it goes into the final prototype. The current interaction used to load the model off the API needs to be improved.	Fraser Savage	02/12 - 04/12
5	HoloLens DICOM image viewing	Attempted loading DICOM images dynamically during runtime, from both original .dcm files and from .jpg files generated beforehand	Loading DICOM images from .dcm files proved to be very challenging due to limited choice of existing C# libraries to achieve that. Loading .jpg files and scrolling through them worked well.	We decided to abandon the idea of loading .dcm files on HoloLens but rather use .jpg files instead. Conversion of .dcm to .jpg will be handled by the API.	Timur Kuzhagaliyev	24/11
6	HoloLens tagalong main menu	Tried using Unity UI components with some custom scripts and scripts from Unity HoloToolkit to create tagalong menu	Feature worked well, and Billboard and Tagalong scripts from Unity HoloToolkit covered most of the calculations. Additionally Unity UI worked well with HoloLens after several adjustments.	Considering how well Unity UI worked and how customiseable it is, we'll definitely continue using it for other elements of our application.	Timur Kuzhagaliyev	24/11
7	HoloLens runtime .obj loading	Attempted importing models from .obj stored in Unity's assets folder during runtime	Although there were some issues with importing the models with correct mesh data, in the end I managed to load all componenets of the model so that they had correct coordinates relative to each other.	Although the experiment was successful loading time was a bit high and we considered decimating meshes before import to improve performance.	Timur Kuzhagaliyev	24/11 - 25/11
8	HoloLens First Prototype	Details can be found below.			Timur Kuzhagaliyev	26/11 - 04/12
8.1	- Context menus	Implemented a framework for generation of collections of buttons that appear above models, DICOM images and potentially any other objects. Framework also supports callbacks and dynamic button creation/deletion.	Framework performed very well and proved a feasible concept for our UI and UX on HoloLens.	We're definitely going to use the context menu framework in our prototype.	Timur Kuzhagaliyev	26/11 - 28/11
8.2	- Improved bounding boxes	A couple of cosmetic improvements to bounding boxes and the way they're generated. Based on the code from experiment 2.4 but with more flexibility and a greater amount of parameters.	The improved bounding boxes were easier to reference in other code and helped avoid a lot of code duplication.	This experiment improved the general quality of the code a lot and also helped me better understand the structure of Unity C#.	Timur Kuzhagaliyev	29/11
8.3	- Manipulation mode framework	Using the new context menu framework, developed a framework that would handle different manipulations in a consistent way. At this stage the supported manipulations were scrolling (for DICOM images), rotating, scaling, transparency adjustment (for models) and exploding (for models).	Client liked the new concept and it fit our usability requirements pretty well by making UX consistent. Additionally, it removed a lot of code duplication.	It was decided to implement this feature in the final prototype after making several adjustments and improving the abstraction of a generic manipulation controller.	Timur Kuzhagaliyev	30/11 - 01/12
8.4	- Multiple object support	Refactored code from experiments 5 and 7 to extract bits responsible for object loading. Create a new ObjectManager class that handles loading objects and can load objects continiously as long as the memory can support it.	We were able to load multiple models and DICOM images into the same scene without any issues, fulfilling one of the requirements set for our application.	The ObjectManager script in this experiments will be used in the final product after some changes to improve its performance.	Timur Kuzhagaliyev	02/12
8.5	- Data markers	Implemented data markers that could be placed on top of meshes and would hold text notes and voice recordings.	We had a chance to see how well integration with HoloLens' virtual keyboard works, which performed pretty well. Additionally, we got some feedback saying that voice recordings might not be feasible and we might want to add an ability to move markers or create markers of specific type.	The concept of data markers will definitely be added to the final product, but we'll have to add more features to them like holding SNOMED CT codes and various multimedia files.	Timur Kuzhagaliyev	03/12 - 04/12
9	Creating Webapp User Interface	Setting up a webapp directory using the electrode.io framework, and creating a homepage that displays the UI title	Used the getting started tutorial so easy to create initial directory however limited documentation beyond that point so developing existing pages proved challenging	Successful but decision made to abandon electrode.io for now due to dificulty of implementation	Laura Foody	19/11 - 21/11
10	User Authentication on Webapp Frontend	Creating a web app directory with login, registration and my account pages along with user authentication functionality using stormpath as a test server	Stormpath has a built in library which makes implementing user authentication very easy	Successfully created a login and registration page. The my account page exists but there is no functionality. CSS needs to be added	Laura Foody	25/11 - 29/11
11	Establishing Communication with the API in the Webapp Frontend	Linking the front end registration/login functionality to the back end using the login API	Not able to implement	Not successful so far because its not clear how the login information is stored when using the stormpath library so unable to integrate with the API	Laura Foody	29/11 - 06/12