MotionInput.

Final MoSCoW List

Nearing the end of our development we reflected on all the goals we set out to do at the beginning of the project and reflected on the progress that had been made throughout the time working on it.

ID	Requirement	Priority	State	Contributors
1	Use the Hand Modules to create a sub-system that effectively handles both movement and aiming in game	MUST	✅	Abid Taha
2	Distinct customized feature profiles for each game	MUST	✅	Abid Taha
3	Two profiles for each game for both standing and sitting	MUST	✅	Abid Taha
4	A full body gesture recorder that records movement over time.	MUST	✅	Weiyi Abid
5	One MFC app launched to the Microsoft Store to launch MotionInput and configure settings	MUST	✅	Weiyi
6	A uniform format for gesture data that would be used by future developers of MotionInput	MUST	✅	Abid
7	Overlay on top of games	MUST	✅	Full Team 26
8	Utilize multiple modules concurrently to make up modes.	MUST	✅	Full Team 26
9	Have clear documentation on how to continue using the modules we build	MUST	✅	Abid David
10	Ability to extend for other modules	MUST	✅	Abid
11	Adaptability for most PC games	MUST	✅	Full Team 26
12	Ability to record gestures from pre-recoded videos	MUST	✅	Abid
13	Ability to record gestures live	MUST	✅	Weiyi Abid
14	Allow users to use multiple gestures at once in game modes	MUST	✅	Abid Taha
15	Allow the user to choose the mode of movement and aiming from a selection	SHOULD	✅	Abid Taha
16	Clear UI on the user side	SHOULD	✅	Full Team 26
17	Clear instructions on how to use the system for the user	SHOULD	✅	Weiyi Abid
18	Ease of experience for users to connfigure the system	SHOULD	✅	Weiyi Abid
19	Ease of experience for users to customize game profiles and controls	SHOULD	✅	Weiyi Abid
20	Require nothing more than a low-cost webcam and PC.	SHOULD	✅	Full Team 26
21	Customizability for default game modes.	SHOULD	✅	Abid
22	A non full body gesture recorder	SHOULD	✅	Abid
23	A static pose recorder	SHOULD	✅	Abid
24	Ability to play in multiplayer	COULD	✅	Abid
25	Generic gamepad moves for retro controllers	COULD	✅	Taha
26	Automatically load recorded gestures into game profiles	COULD	❌	N/A
27	Optimized detection times for gestures	COULD	❌	N/A
28	Visual feedback on which controls are being activated	COULD	❌	N/A

Critical Evaluation

Overall

We have managed to complete all the main requirements set out for us and exceeded those expectations. We have allowed users to use custom gestures and poses in games with the ability to record and load new ones if they like. Our other primary achievement is that we’ve found ways to create a standardised formats for modes that allow users to play games with a default control set or if users like change those modes or make new modes. All of our other requirements were built around this and worked together to build a more robust and cohesive solution. There are however minor issues that need to be addressed and will influence our future work in the next section.

As a general point to make, we found that out modes allowed us to play complex games like Spider Man with nearly the entire control set – this is definitely a strong point but can also be a caveat as well. Multiple sources of feedback have said that if children (who are likely to be users) were to play, they would find this control set far too complex to understand and as a result, not have a very good experience with it. Accommodating for less-able users is something we did focus on by incorporating generic gamepad modes, but we need to do more to include simpler modes for specific games.

Gestures/Poses

Gestures and Poses work excellently on standing and sitting modes respectively; many gestures can be used within the same mode to the point where entire control sets can be done using only gestures. The poses are extremely responsive which allows for people playing in sitting modes to control their character in game almost as easily as if they were playing normally. However, the gestures could be slightly more responsive as it already takes some time to detect the gesture. It could be possible to sidestep the problem and allow for chaining of multiple gestures with a few actions but either way, with the intense nature of some games, the gestures used will need to be faster to improve effectiveness.

Movement and Aim Boxes

The new joystick hand box modules integrated for our modes provide an excellent alternative to the original default aim boxes and hit triggers. These are most effective in sitting modes where they can be used in conjunction with poses to use multiple controls at the same time. Furthermore, both movement and aim boxes can be used with the hand in any orientation allowing for a high level of flexibility.

However, one of the main issues which is mainly prevalent in standing modes is that using gestures with hand boxes directly conflict with one another. By doing a body gesture, the movement of a user’s hands interact with the hand boxes hence both moving and changing the aim unintentionally. This issue has been mitigated with the use of walking events and gestures that do not use the aim hand. This has provided a decent solution to this problem but is not a desired solution. We need to look more into how we can integrate seamless aiming and gestures in standing modes.

MFC

The MFCs are effective at launching and configuring our applications. The instructions are clear for the user to see and hence the overall user experience is easy. Performance is good and stable for both. They provide a wide range of options to customize modes and gestures, however there are limitations. Specifically, you cannot add an unlimited number of gestures to your modes. We want to have a higher level of customization so that the user can really feel in control of how they play.

Data Management (JSON)

Our work expanded on the data management methods that MotionInput used as a whole by using JSON files throughout to store the configuration data of the application. However, our solutions introduced using individual JSON files to represent the individual modes, gestures and poses. For the modes we could unify different module usage into one file so that it could be made explicitly clear which gestures, events and speech commands were used. This makes modes highly extendable; each file can contain whatever controls it needs to be as simple or complex as possible. Additionally poses and gestures store values as ratios of landmarks allowing uniformity of use across any range of users. Overall, the JSON file management goes above and beyond to make data handling far simpler and yet more functional than base MotionInput.