Project Background
Our Project Title is: AX-CS FPS, Action and Adventure games with UI Layers. AX-CS stands for Accessibility and Computer Science - hence the aim of our project is to use make the world of computers accessible to all, with a focus on FPS, Action and Adventure games such as driving, 2D Side Scroller and 3D Action games. Traditionally, video games are played with the mouse, keyboard or game controllers which limits them from children with disabilities. Therefore, our task is to integrate MotionInput - a software package for touchless computer interactions - with such games.
Motion Input was first proposed in 2020 with v1 released in January 2021. It has since been in continual development and progression. We will be utilising the latest version of Motion Input which is v3.4. This version encompasses various functionalities which we will implement, such as using different body parts (i.e. nose, head, elbows) for interaction, as well as using gestures and speech commands. We will make games compatible with Motion Input by binding in-game controls to real-life actions, for example pressing virtual buttons or using a speech command. To accommodate the different needs of our intended users, who may not be technical, we will create user interfaces through which they may customise settings for Motion Input easily and launch this accessibility software for each game.
We are leading the integration of MotionInput V3.4 with FPS, Action and Adventure Games with our primary stakeholders being: Prof Dean Mohamedally, and additional clients: Lee Stott (Microsoft), John McNamara (IBM) and Greg Chown (IBM).
Initial Project Objectives
Following our initial project brief, we produced a list of tentative project objectives before conducting MotionInput research.
Examine the current codebase of MotionInput v3.4 to gather information about what features we can implement.
Examine last year's MFC codebase for video games to build upon their work and make improvements.
Use Visual C++ to create MFC User Interface Applications for Rocket League and Forza Horizon.
By testing each game, evaluate each in-game control and choose crucial ones to be implemented.
Through user interviews and evaluation, decide what settings should be customisable in our UIs.
Work with final-year students who are doing their industry project with MotionInput, to integrate new features.
Project Analysis
As we were tasked with creating UIs for the integration of Motion Input with specifc games, our requirements dictated that we analyse the current v3.4 codebase in order to discover possible features we may implement. We must also analyse last year's MFC codebase in order to improve upon it. In our analysis, we documented a number of issues with the MFC codebase in order to ensure that our UIs do not repeat these issues across all levels.
Features to be implemented from Motion Input v3.4Possible methods of control include a "steering mode" using "drive.py" and "joystick mode" using "move_joystick.py".
Wrists, elbows, shoulders, knees and feet can be used to trigger controls.
Bodypoint tracking allows any bodypoint, including nose, to be used for joystick.
Speech commands can be used.
Gestures available include "raise eyebrows", "open mouth", "fish face" etc.
Triggers of games supported were hardcoded, users could not adjust their positions. Virtual buttons did not have skins, making them visually unappealing and their functions unclear.
Key bindings for in-game controls must be manually added through the MFC, which could be difficult for non-technical users.
Previously supported games like Spider-man had too many triggers, cluttering the screen.
Documentation and comments were provided for multi touch only but not for any games.
Final Requirements
After analysis of previous codebases and discussions with our clients, we made a set of final requirements at the start of our project development phase:
Design and build MFCs, that are created for non-technical users with different needs, hence they must be intuitive and they must allow users to easily customise settings.
Ensure our work is extendable by developers. That is, they can easily add other in-game controls, speech or gesture commands for each game.
Ensure that our architecture implementation uses good programming practices, including consistent, non-repeating code and design patterns to ease maintainability and future development.
Make Action and Adventure Games (Driving Games, 2D Side Scroller Games, 3D Games) accessible by creating easy-to-use MFC UIs that non-technical users can use.
Make driving games (Rocket League, Forza Horizon) accessible by creating easy-to-use MFC C++ UIs that non-technical users can use.
Conduct user testing to gather feedback and iteratively enhance the accessibility features of the game.
Create detailed user manuals and developer-oriented documentation for each MFC.
MOSCOW Project Goals
After finalising our requirements, we created a MoSCoW requirement list to inform our goals for the project. This allowed us to easily check our progress by comparing the number of points on the list that we have successfully implemented or achieved.
An intuitive User Interface for specific driving games, namely Rocket League and Forza Horizon.
Options to allow users to select mode of driving and bodypoints to play according to their needs.
Complete documentation and user manuals
Handle potential errors in our MFCs
A general driving UI that supports a range of driving games.
UIs for specific 2D side scroller and 3D action games, such as Sonic and Lego Avengers and Batman Arkham Asylum GOTY.
Motion Input UI that does not obscure the screen during gameplay.
Descriptive and appealing virtual buttons by integrating skins.
A general 2D Side Scroller UI that supports a range of such games.
Integrate speech and gestures to allow more options of triggering in-game controls.
Implement bodypoint tracking, such as nose tracking to control the movement of the car or game character.
Implement lower body control options for users.
Create a general 3D Action Games UI.
Due to time constraints, we won't have customisation of the position of virtual buttons. Instead, they will have predetermined positions based on user feedback.
As each type of Action and Adventure games differ greatly, we won't have a general UI that supports all such games.