Experiments

Experimentation

Given the very complex nature of our project, we switched from our approach at the beginning of the project, to get stuck in and start doing whatever we could, to following a very methodical, step by step approach. This has naturally led to us refining our specifications at each step to make our project better.

Most of the most significant progress that we have made has been with creating the 3D scans. After we received our 3D scanner a few weeks into the second half of the term, our focus has been on perfecting the scans, especially given the recently limited access to the Hololens, given the fact that there are a number of teams using it. Below you can see the output of our scans alongside a short description of what we were trying to do/scan:


1) This scan is from the first time the scanner was ever used by us. It is a scan of a grey Palace hat. Since Haran is the closest person to the university, he took possession of the scanner and tried this out at his flat. This showed us a few things: firstly how realistic the 3D models would be and secondly that ideally you need something that you can physically walk 360° around.



2) The scan on the left as you can see, is of Yll’s head and this was our first experimental scan as a group. Although the scan was incomplete, it built our confidence, as it looked very photorealistic, mapping the colour and the texture very well. The incompleteness of the scan mostly our fault as well as we didn’t quite hold the camera steady as we went around and we went around Yll at a varying radius.



3) This scan is from the first time we all used the scanner together. It is a scan of the cardboard box that the scanner came in and it is an important one as it was our first scan that was somewhat successful. It was crucial, if anything, for our confidence in the product, because we still had some doubts about the quality of the scanner before this point.


4) After instilling confidence with our previous scan, we decided to make the step up to scanning a garment of clothing. In order to do this, we got the help of Dr Treleaven and his colleague, as the last time we spoke to them, they said we would be able to use their mannequins. The first scan we did on the mannequin, using Vania’s coat was dreadful, as can be seen on the left. This was because, since the mannequin was on an axle, we tried to keep the camera still and spin the mannequin around. This caused us to find out that you have to physically move the camera 360 degrees around the object.



5) In light of the information about the motion requirements of the camera, we took the mannequin off the axle and moved the camera around the mannequin. This however exposed another problem: the space requirements: as we were trying to do the scan in Dr Treleaven office, the space was limited and even though the scan was fairly successful, we realised that we had to go outside into the hall to get more space and hence a better scan.

6) Aside from going out into the hall, another thing that we noticed in the previous scan was that the scanner did not do well with fur. Perhaps this was because of the background in the office, or perhaps because of the light, so we detached the fur hood lining before doing the scan. The effect of this was phenomenal as we got a near perfect scan. The only slight blind spots were the bottom lining of the jacket and part of the hood, which again was lined with fur, but black fur, which we didn’t think would be an issue. However we believed the issue was to do with the way the light was reflecting on it. Another issue we had was we had still not left enough space from the wall. The final issue was that since Haran had been moving around the mannequin to keep it in place whilst it was being scanned, was that the scan was distorted by him.



7) This scan was pretty much the same as the previous one except for the fact that we re-added the fur hood lining given the fact that our environment and lighting was different and we also left more space from the wall. Haran also moved around at the correct angle so as to not be picked up by the camera. However the scanner still had issues with the fur hood lining, so we removed it in the next scan.




8) This is our most recent scan, where we got pretty much everything right, leaving the perfect amount of space, moving around the camera at the correct height, and Haran moved around at the correct angle, allowing for our best possible scan in that environment.
Aside from working on the 3D scans, we also went through some of the tutorials on the Microsoft Holographic Academy, completing the “Holo” world, Gaze, Spatial Sound, Spatial Mapping, Gestures and Voice tutorials. Below are some screenshots of some the things that we were able to create from the tutorials.

From our limited use with the HoloLens, we managed to start creating a PoC of the UI for our personal shopping experience. We did this based on the UI concepts we made in Sketch.
  1. We started learning how to use Unity and the basic concepts in how the camera and lighting affects the way models are displayed.
  2. We used Unity to create a standard UI in our intended style of a floating panel that disappears when the main stage of display is needed.
    1. The user interface was originally using a ‘Panel’ component but that didn’t allow the customisation we wanted it to.
    2. We then switched to using the ‘Image’ which, contradicting its name, could be used as a simple page background for the UI.
    3. The other components were more intuitive in that they were called ‘Button’ and ‘Raw Image’.
  3. Using the things we learnt from the tutorials and resources like the HoloToolkit, we adapted our Unity project for use with the HoloLens which involved constructing scenes with a certain hierarchy and changing subtle settings within the different components.
  4. Then running it using Visual Studio and the HoloLens emulator, we could see our project close to how it would be seen using the HoloLens.

Experiment Log

Further Development