Summary of Achievements
-
MOSCOW List
Achievement Table of MOSCOW functionalities
Percentage of key functionalities (MUST have and SHOULD have) completed: 100%
Percentage of optional functionalities (COULD have) completed: 100%
ID Description Priority State Contributors 1 Provides access to a high quality reading experience MUST Complete Nandini 2 Allows user to interact with the device whilst facing the webcam and facing away from the webcam MUST Complete All 3 User can navigate across the full screen MUST Complete All 4 Calibrate MotionInput to the angle the user is sat at from the camera. (Modules: Hand, Face) MUST Complete Peter, Donghyun 5 Using calculations or models to calibrate the camera's angle e.g. a model is created using sklearn's linear regression. SHOULD Complete Donghyun, Nandini 6 The model is stored in a file called 'angle_model.joblib' in the MotionInput's data folder using Python's joblib library. COULD Complete Nandini 7 Successfully use the calculations or model to find the position of the cursor given the (x, y, z) coordinates of the bottom of the middle finger. SHOULD Complete Donghyun 8 Successfully use the calculations or model to find the position given the (x, y, z) coordinates of the nose tip SHOULD Complete Donghyun 9 Use a gaussian filter and laplacian smoothing to reduce cursor jittering and noise. SHOULD Complete Team 32, Nandini, Peter 10 Calculate the yaw, pitch, roll values of the user's head pose at a given frame COULD Complete Peter 11 Map the user's head pose to the direction the user is facing e.g. right, left, up, down using calibration. COULD Complete Peter 12 Each head pose gesture (right, left, up, down) will trigger an event. COULD Complete Nandini 13 New games and web pages on FISECARE targeted towards children SHOULD Complete Nandini 14 Optimise hand and face models using OpenVINO's model optimiser. WON'T Incomplete Nandini 15 Improve gesture detection such as index pinching, or open mouth gesture from different angles to the camera. COULD Complete Team 36, Nandini 16 The user can add their own apps to use on FISECARE by editing a JSON file COULD Complete Nandini 17 Programmed in Python MUST Complete All 18 Python program can be run on an Intel fan-less computer MUST Complete Nandini, Peter 19 Integrated with MotionInput's face module. (Libraries MediaPipe, openCV used for face landmark detection) MUST Complete Nandini 20 Integrated with MotionInput's hand module. (Libraries MediaPipe, openCV used for hand landmark detection) MUST Complete Nandini 21 Be compilable to an executable without additional dependencies MUST Complete Nandini 22 Executable can be installed on an Intel fan-less computer such as MeLe mini PC. MUST Complete Nandini 23 Instructions for calibration are easy and natural to follow MUST Complete All 24 MotionInput should be fast and responsive with the addition of new Machine Learning models running in the background SHOULD Complete All 25 New games and web pages added to FISECARE should be easy to use with MotionInput. COULD Complete Nandini 26 The MFC provides options to select a mode that suits the user's abilities COULD Complete Nandini 27 The MFC should provide information for each option so the user understands what modes are available to them COULD Complete Nandini 28 An NDI version of the camera added to the MotionInput API will allow the user to connect to MotionInput with their team's camera via their phone. COULD Complete Arvind (team 31) and Abriel (team 29) 29 MFC provides a setting to change the cursor speed for comfortability COULD Complete Nandini 30 MFC provides a setting to change tracking method for comfortability COULD Complete Nandini 31 MFC provides a setting to either use speech commands or gestures to interact with the system so user can select what is comfortable for them COULD Complete Nandini 32 FISECARE provides services that are entertaining to children COULD Complete Nandini -
List of known bugs
ID Bug description Priority 1 MotionInput crashes in rare cases. We only face this issue working the Intel MeLe. Due to the MeLe overheating, UCL FISECARE or MotionInput may crash. Low 2 The ‘Launch’ button on the MFC is clickable before the user has selected the necessary options. Medium -
Individual Contribution Distribution Table
Task Nandini Peter Donghyun Vincent Client liaison 50% 30% 20% 0% Liaison with other teams 75% 20% 5% 0% HCI 40% 30% 30% 0% Requirement Analysis 33% 33% 33% 0% Pitch presentations 40% 35% 25% 0% Front-End Coding 80% 10% 10% 0% Back-End Coding 20% 50% 30% 0% Blog 100% 0% 0% 0% Testing 30% 60% 10% 0% Report website 20% 70% 10% 0% Monthly Videos 10% 85% 5% 0% Non-technical Video 5% 5% 90% 0% Video Editing 0% 0% 100% 0% Final Technical Video 15% 15% 70% 0% Overall 36% 32% 32% 0% Main Roles Team Lead, Front-End Developer, Client liaison Back-End Developer, Report Editor, Tester Researcher, Video Editor
Critical Evaluation
-
User Interface / User Experience
The FISECARE portion of the project builds upon the work of 21/22 Team 35. Thanks to their design, the user interface features large buttons and easy-to-understand text descriptions, which helps to create a pleasant user experience. Following on from their use of symbols and icons for the buttons making it easier for users to interpret the purpose of each button, we have stayed consistent with the design.
The user interface of MotionInput primarily consists of calibration windows, which display text instructions as well as the image captured by webcam. During calibration process, user is only required to perform a simple and intuitive task of tracking a flickering dot on the window, with either their head or hand. Our test cases reveals that users were able to interpret the instructions and perform the calibration correctly in most cases.
Lastly, the MFC settings app provides links to plenty of resources and information to guide them through using the Bedside Navigator. Easy-to-access information buttons are placed regularly at every option for the user. Although, using more graphics or pictures would be more pleasing to the eye.
-
Functionality
FISECARE offers a range of entertainment services for elderly people, such as online shopping and caregiver calling. We have also expanded its capabilities by adding a new section for children, featuring educational websites, games, and work from other teams. This has successfully encouraged our users to stay active and connected through the internet. As well as increase the use of technology amongst age groups or a population of people who otherwise avoid technology.
The Bedside Navigator also provides the functionality to adapt their app to their needs with the personalisation of apps on FISECARE. The MFC Settings Apps includes a quick and easy way to edit their apps on FISECARE. This makes the MFC the main means of personalising their experience of the Bedside Navigator which is easier to manage rather than changing the settings on multiple files.
Users can navigate the application using MotionInput side-of-head or side-of-hand tracking modes, with support from gesture and speech control.
-
Stability
The FISECARE application is stable and responsive thanks to its simple design using the WPF framework. MotionInput navigation modes are stable in most use cases as well. However, during our testing on MeLe, the MotionInput build crashed a few times, which we suspect is due to compatibility issues with fan-less computers.
Despite the intense load of MotionInput, the Bedside Navigator functions well for long periods of time on most Intel Mini Computers. As well as, on numerous other devices such as laptops. There is minimal latency experienced, although there can be some latency in lower-end devices.
-
Efficiency
The Bedside Navigator program was tested on multiple fan-less computers, with the Intel MeLe Quieter 3Q serving as our primary test device.
Currently, the program consumes a maximum of 700 MB of RAM and 33% of CPU usage, and takes an average of 5 seconds to start up both FISECARE and MotionInput.
As a prototype application, Bedside Navigator is subject to optimization.
-
Compatibility
The FISECARE application runs smoothly on all tested Windows machines, including Intel MeLe, Intel NUC NUC11TNKi5, and NUC11TNKi7.
MotionInput navigation modes are generally reliable on these platforms, with only occasional crashes reported.
-
Maintainability
The FISECARE application is developed using the WPF framework, which offers a wide range of development tools that can help maintain and extend the application.
The MotionInput navigation modes are developed in Python, and come with deployment and user manuals. Additionally, the source code is well-documented at key parts of the project, helping to identify the architecture and maintain the application.
-
Project Management
The project workload is distributed evenly among all active team members, with an effort made to ensure that tasks align with each member’s strengths and interests. The team has chosen WhatsApp as the primary communication channel, Notion and OneDrive as shared workspaces to collaborate on all research findings, and GitHub to store the codebase. These tools have enabled the team to be flexible and responsive in meeting customer needs quickly.
Future Work
The FISECARE v2 project currently provides a proof-of-concept for an entertainment system with a navigation suite intended to be installed at the user's bedside. While our developers have implemented most of the proposed functionalities, there are still many possible ways to improve its current state. Here are a list of possible directions in which FISECARE v2 could benefit:
- Refactor the entire workflow of calibration into a MVC paradigm, to improve user experience.
- Gamify the calibration process, such as shooting targets for hand calibration, or drawing a certain pattern for head calibration.
- Ultimately, pre-train the predictors to remove the calibration process!
For calibration windows:
- Instead of improving on head modes that move the cursor more accurately, it would make more sense from an HCI perspective, to use head motion to control the selection of clickable components on web pages. For this, it is necessary to redesign the browser built inside FISECARE, with shortcut keys to navigate directly among adjacent nodes of clickable component.
For the MotionInput navigation modes:



