Evaluation
Achievements
Functional Requirements
As shown in the table below, our team managed to successfully complete all of our "must have" requirements, and we had a good attempt at completing our "should have" and "could have" requirements.
| ID | MoSCoW | Requirement | Status | Contributors |
|---|---|---|---|---|
| 1 | Must | BLE proximity connection to Raspberry Pi | ✅ | Marwan |
| 2 | Must | Facial recognition via Raspberry Pi (using camera module) | ✅ | Raghav, Marwan |
| 3 | Must | Accessibility settings fetching | ✅ | Abdul Muhaymin, Raghav, Abdulhamid |
| 4 | Must | Server to host user profiles | ✅ | Marwan, Abdul Muhaymin |
| 5 | Must | Post-quantum cryptography encryption | ✅ | Abdul Muhaymin, Marwan |
| 6 | Must | AI chatbot assistant | ✅ | Raghav, Marwan |
| 7 | Should | Website to register device and setup initial settings | ⚠️ | Marwan |
| 8 | Should | Directional proximity management | ✅ | Abdulhamid, Marwan |
| 9 | Could | API Layer to make system reproducible | ✅ | Raghav, Abdulhamid |
- 100% of our "must have" requirements were fully completed.
- 100% of our "should have" requirements were partially completed.
- 100% of our "could have" requirements were fully completed.
Known Bugs
During the creation of each feature in our system, we performed extensive testing. Any known bugs preventing the system from functioning as intended was dealt with immediately. However, due to time constraints some bugs remained, luckily all of these have easy remedies.
| ID | Description | Severity |
|---|---|---|
| 1 | UART connection issues Raspberry Pi → Pico, due to lack of soldering | 🟡 Moderate |
| 2 | Small models hallucinating commands | 🟢 Low |
| 3 | Preferences feature on IBM Proximity Agents | 🟡 Moderate |
| 4 | BLE connections failing whilst connected to eduroam | 🟡 Moderate |
Individual Contribution
| Work Package | Marwan | Raghav | Abdulhamid | Abdul Muhaymin |
|---|---|---|---|---|
| Project Partner Liaison | 90% | 10% | 0% | 0% |
| Presentations | 30% | 30% | 20% | 20% |
| Research | 20% | 40% | 10% | 30% |
| Server | 35% | 25% | 5% | 35% |
| Registration Site | 90% | 10% | 0% | 0% |
| ESP32 | 90% | 10% | 0% | 0% |
| Raspberry Pi | 60% | 20% | 5% | 15% |
| Desktop App | 30% | 50% | 5% | 15% |
| Testing | 10% | 10% | 10% | 70% |
| Work Package | Marwan | Raghav | Abdulhamid | Abdul Muhaymin |
|---|---|---|---|---|
| Website Template and Setup | 10% | 5% | 5% | 80% |
| Videos | 34% | 33% | 33% | 0% |
| Home | 20% | 20% | 40% | 20% |
| Requirements | 5% | 10% | 50% | 35% |
| Research | 20% | 0% | 40% | 40% |
| Algorithms | 20% | 5% | 70% | 5% |
| UI Design | 0% | 0% | 20% | 80% |
| System Design | 85% | 0% | 0% | 15% |
| Implementation | 0% | 100% | 0% | 0% |
| Testing | 70% | 0% | 0% | 30% |
| Evaluation | 20% | 0% | 0% | 80% |
| Appendices (Manuals, Legal, Blog, etc.) | 40% | 50% | 10% | 0% |
Combined Overall
It should be noted that some sections are not weighted equally, therefore the values below many not exactly align with the raw data given in the tables above.
| Marwan | Raghav | Abdulhamid | Abdul Muhaymin |
|---|---|---|---|
| 27% | 26% | 23% | 24% |
Critical Evaluation
Great effort has been put into this project by all members, however it is still not without fault. In the section below, we will evaluate which areas were completed to a high standard, and which areas could be improved upon given more time.
During the project we ensured that the digital user interfaces were accessible for most users as that was a crucial point of our project. We frequently iterated on the design of our AI Agents Desktop App, and performed responsive design testing and user tests to ensure that it met the requirements of our users. From the initial sketches, to the prototypes, to the final production design, we have considered our clients every step of the way. The result is beautifully designed user interfaces, that provide an excellent user experience.
This was done very well!
Our system delivers on all of our requirements to at least some degree, with the overwhelming majority being completed to an even higher standard than what was originally specified. Our quantum proximity gateway provides a seamless method of authentication, whilst being quantum secure. The AI Agents Desktop App also provides an easy-to-use interface to load and update user preferences. Together, these create a highly functional system that would benefit most, if not all, users that decide to use it.
This was done very well!
Throughout the project duration, we created and carried out a very comprehensive set of tests which included both manual and automated testing. These tests ensured that our system was stable, and revealed any bugs that would need to be fixed. Our manual testing consisted of integration testing, responsive design and compatibility testing, and user acceptance testing. These tests ensured that our system components worked well together, and that our interfaces ran in different envrionemnts. Furthermore, our automated testing, wich consisted of unit testing and continuous integration, ensured that our server backend and desktop app were functioning correctly without any unexpected issues.
This was done very well!
Efficiency is a crucial part of our system that we have made one of our priorities. At each stage of development, we have always considered the most optimal way of doing things, from our choice of programming languages and frameworks, to building our API endpoints, and developing AI Agents Desktop App. We knew that all components of our system must be efficient, especially as our system scales to accomodate orders of magnitude more users. Any parts that are less efficient would eventually cause a bottleneck that would severely hurt the usability of our system, and waste the time of our users.
This was done well!
During development, we carried out compatibility testing on the AI Agents Desktop App, and Registration Site to ensure that they run correctly in different environments. The desktop app was tested on MacOS, Linux, and Windows, and was observed to be mostly compatible across all platforms. Moreover, the registration site was compatible in the major chromium browsers, however it was not completely compatible with browsers such as Firefox out of the box due to lack of support for WebSerial. Although there are some issues, overall we have tried our best in ensuring compatibility with different environments.
This was done okay!
For each distinct part of our system, we have maintained the code in a git repository. We have dedicated ourselves to producing clean code, and adding comments wherever necessary to aid with maintainability. Our directory structures are well organised, and there is also plenty of documentation detailing how to setup different parts of the system, as well how to run tests. Practices like object-oriented programming have also been used where required to ehance modularity. This effort to ensure that our codebase is easy to understand and navigate also enables ease of development for future developers.
This was done well!
Throughout the entire project, we have consistently met up as a group during weekly lab sessions. We have also maintained constant communication in a WhatsApp group chat, and have used GitHub's issue tracking, branching, and pull requests to communicate about our progress and code changes. Tasks were split amongst ourselves based on interests and capabilities to ensure that each part of our system and report was completed on time, and to a high standard. This teamwork has allowed us to develop a robust, performant system.
This was done very well!
Future
As a team, we are happy to say that we have completed practically all of the functional requirements of our project that we laid out initially, as well as some additional features. With that said, as with any system, there is always room for improvement, and additional extensions that could be added:
- Improved Hardware Design — The current housing for the Raspberry Pi and camera module is underdeveloped. Ideally the camera module should be situated in a convenient location dependent on the device, such as above the monitor of the computer being logged into.
- Battery-Powered Keys — Currently our ESP32s are powered via a USB-C connection, however in practice it should be wireless and battery-powered.
- More Reliable Connections — There were some UART connection issues between the Raspberry Pi and Pico that still need to be resolved. Furthermore, there is sometimes inconsistencies with the BLE connections in certain environments that needs to be further examined.
- Additional Device Support — Now that we have successfully implemented a working example of our system for desktop PCs, it would be a good idea to look towards other devices such as tablets, printers, TVs, etc.
- Advanced Gestures — An advanced touchless gestures solution such as MotionInput could be integrated with our camera module to allow for gestures that can peform certain tasks, such as logging on/off, opening new windows, changing settings, etc.
- More Wireless Protocols — Even though BLE was ultimately chosen as our wireless proximity technology of choice, it may be a good idea to support other protocols depending on the context.
- Extra UI Settings — To improve our user interfaces even further, we could include multi-language support, as well as different themes to choose from depending on the user's likings.