Evaluation
Summary of achievements
Achievement table
The achievements of NeuroResponse Mobile lie in the success of deploying key functionalities that met the client’s expectations. Even though we could not manage to link our app with preexisting medical records (since the NHS does not have an accessible API), we were able to produce the rest of them with high standards.
Furthermore, despite the fact that the mobile app was never officially deployed we fulfilled the requirement of our client to test it with 10 users. Therefore, it can still be classed as a success overall because we managed to deliver a product that is fully-tested, has the main functionalities in place and takes into consideration further scalability (we can expand our API) and integration.
Functional
| No. | Requirement | Type | Achieved | Contributors |
|---|---|---|---|---|
| 1 | The app should allow users to track their symptoms, such as related to vision, cognition, physical impairments and mental health (symptom tracker). | Must | ✔ | Sanzhar, Miquel |
| 2 | The app should allow users to respond different sets of questionaries to the treatment they are on (side-effect check). | Must | ✔ | Sanzhar, Miquel |
| 3 | The app should allow users to set up custom notifications regarding future medical appointments and notify them accordingly. | Must | ✔ | Sanzhar, Miquel |
| 4 | The app should allow users to send both symptoms records and side-effects checks to their clinicians. | Must | ✔ | Elvinia, Miquel |
| 5 | The app should allow users to comment on each symptom tracker and side-effect answers with a designated scale. | Should | ✔ | Sanzhar, Miquel |
| 6 | The app should allow users to authenticate themselves via the NHS number and an specific code that will be given to them from their clinicians. | Must | ✔ | Elvinia, Sanzhar, Miquel |
| 7 | The app should allow users to grab data from their clinical records (the clinical will specify the kind of data and its accessibility). | Could | ✘ | - |
| 8 | The app should allow users to visualise the whole spectrum of collected data (symptoms). | Could | ✔ | Sanzhar, Miquel |
| 9 | The iOS app should offer Active Tasks from the Research Kit framework. | Should | ✔ | Miquel |
Non-functional
| No. | Requirement | Type | Achieved | Contributors |
|---|---|---|---|---|
| 1 | The app should be accessible by users with colourblindness and impaired performance of motor skills. | Must | ✔ | Sanzhar, Miquel |
| 2 | The app should follow an easy-to-read and intuitive layout with specific colour tones. | Must | ✔ | Sanzhar, Miquel |
| 3 | The app should automatically handle when users don’t have internet connection, to send the results when connection is reestablished. | Could | ✔ | Sanzhar (Android) |
| 4 | The initial system should be deployed to be tested with 10 users. | Must | ✔ | Miquel |
| 5 | The iOS app should be integrated with Research Kit. | Should | ✔ | Miquel |
| 6 | The system should offer and end-to-end encrypted API. | Must | ✔ | Miquel |
| 7 | The system should encrypt all patient data stored in the database. | Must | ✔ | Miquel |
| 8 | The app must be a metric device and record its interaction with the user (accelerometer data, multi touch and finger position). | Could | ✔ | Sanzhar (Android) |
| 9 | The app should be available for iOS and Android. | Must | ✔ | Sanzhar, Miquel |
Summary
- Must have and should have: 100% (14/14)
- Could have: 75% (3/4)
List of known bugs
| ID | Platform | Description | Priority |
|---|---|---|---|
| 1 | iOS | Transition between general symptoms and symptom list produces black background. | Low |
| 2 | iOS | Graph in Tracking View appears a bit later compared to the loaded screen. | Low |
Individual contribution
| Work Packages | Sanzhar | Elvinia | Miquel |
|---|---|---|---|
| Client liaison | 33% | 33% | 33% |
| Requirement analysis | 33% | 33% | 33% |
| Research | 33% | 33% | 33% |
| UI Design | 40% | 20% | 40% |
| Programming | 35% | 25% | 45% |
| Testing | 40% | 0% | 60% |
| Bi-weekly reports | 20% | 80% | 0% |
| Website editing | 20% | 20% | 60% |
| Video editing | 0% | 0% | 100% |
| Poster editing | 0% | 0% | 100% |
| Overall contribution | 30% | 28% | 40% |
| Roles | Android UI Designer and Developer, Tester | Research, Complementary Clinician Portal | iOS UI Designer and Developer, Backend Developer, Tester, Report Editor |
Critical evaluation
UI/UX
Designing the UI for both mobile apps has not been an easy task, since we had to consider a lot of factors as mentioned in the HCI, such us patient reduced vision, impaired motor skills... However we have tried to do our best in making a simple, intuitive and easy to use UI.
Another challenge faced designing the UI was trying to make both apps, iOS and Android look as similar as possible. However, this is pretty complicated since we were trying to follow the native human interface guidelines for both platforms. For example, in iOS is common to have a tab bar on the other hand, on Android, is more common to have tabs. Despite this fact, we have tried making both apps look analogous using a similar layout for the main actions and using the same color scheme.
It is important to be consistent with the UI because it can affect how the patients use our app. Therefore, if we are recording information and tracking patients, both apps should be the most similar possible. This is because a small change in UI can alter the workflow and subsequently, the meaning of the data collected. If this happens, our data would have to be split according to the platform, iOS and Android.
Functionality
We have fulfilled all of the main requirements for our client. However there are a few things that we should consider. Again, they are related to make the app as homgeneous as possible between the two platforms.
- Implement user interaction recording on the iOS app, such as where the patient is pressing, for how long, etc...
- Implement a similar alternative to Apple ReserachKit on Android. Apple ResearchKit seems a really promising framework with a lot of potential to monitor patients, therefore, if we could extract the same data on Android devices it would be really helpful.
In this way, the source of data collection (the mobile apps) would be more homogenoeus and thefore the data that comes from both plaftoms, iOS and Android, could be treated as one, increasing the quality and quantity of it.
Stability
Our mobile apps as mentioned before, contain minor bugs, most of the are related with UI. The apps are stable and they have not crash, even under stress conditions.
Regarding our testing deployed server, the requirement of our client was to test the app with 10 users. Our final server has 100% response success up to 1500 users and then the success rate decreases down to 10% when 2500 users are concurrently connected. This server can handle up to 1500 users concurrently without crashing, therefore that is more than enough to fulfil the requirement.
Compatibility
In terms of our server RESTFul API, is 100% compatible with all the clients that can make http requests.
Regarding our mobile apps, as mentioned in the testing, for iOS we are compatible with 93% of the current devices and on Android 71% of all devices. To get a higher compatibility we would have to sacrifice features such as ResearchKit in order to compatible with lower versions. In the end, we think we have done the right thing, because as time passes, more devices will have newer versions of the different OS's, and therefore, our app is going to be compatible with them.
As a real use case, there are companies such as Facebook that do not offer WhatsApp to lower versions of OS. Why? Because they know that is a matter of time that the outdated devices will be replaced by newer ones and also, since the marketshare is so low, that is not worth investing the time.
Maintainability
Server: In terms of efficiency, we are running it in one of the low-end performance VM and it can serve up to 1500 users. In terms of scalability, using a more powerful machine, it would enable more users. Regarding code maintainability and scalability, since we have used Django every component is modularised, so if for example, we wanted to add another field to users, we would just have to edit the patient model and make a migration to update the database. Another example, if we wanted to use another authentication system, we would just have to change a few parameters and a line of code in settings. The same applies to adding another endpoint, we would just need to add another view and that would be enough. We can see that our API can be easily extended.
Mobile apps: for both platforms adding new symptoms and side effects will not be a problem, because of the way we have implemented them. However, on the iOS app we have suffered the consequences of using the Model View Controller pattern and one of its downsides, turning it into a Massive View Controller. We have accumulated things like networking and parsing in the controller making refactoring under some circumstances complicated.
Project management
We managed the project well overall. In terms of balancing the workload, we did this efficiently as we allocated tasks based on the strengths of each team member. We made use of a Gantt chart to effectively timetable ourselves and this was agreed upon with the client. We communicated well with the client and kept them updated throughout the project, which was very important as it meant that the client was part of the iterative design process throughout. In addition, our internal team communication was excellent thanks to communications tools such as Slack and Google drive for our external documents to share ideas, documents and plan our next steps
All the source code has been hosted in GitHub, allowing us to have different branches and the ability to work independently with our specific components.
Our developments tools have mainly consisted of Android Studio for the Android app, Xcode for the iOS and Visual Studio code for the server. The first two choices are the standard ones if we want to make use of native SDK, such as Java and Swift, allowing us to use a simulator or test it with the real device without too many complications.
Future work
UI/UX try to make both apps look exactly the same, even if we have to ignore native human interface guidelines. Since the device can become a metrical tool, we want the source of data (mobile phone apps) to be as homogeneous as possible.
Symptoms and side/effects extensibility: right now our mobile apps have the symptoms and side effects hard-coded in them. One improvement could be to parse the symptoms and side effects from the API. In this way the clinician would have the ability to update symptoms and side effect questions on the go, without having to release an update of the mobile apps.
iOS refactoring: get rid of the Massive View Controller and modularise more the parsing and networking components to make the app more extensible.
Android app: implement a similar alternative to Apple's framework ReserachKit, since it can have a lot of potential.
Deployment: deploy the app throughout the NHS. We have gone through the testing phase with 10 users using our deployed testing server. Now that we know the bugs and the bottlenecks of our system, we can start considering to deploy it for the real world. The server will have to be an NHS N3 server.
Implement communication with 3d party apps: in this way we would be able to acccess to the clinician records or access GP information. At the same time, there could be a validation of the care pathways thanks to these 3rd party integrations.
Consent forms: before deployment, since our app collects the data recorded by patients, we will have to implement a consent form inside of it where each patient will be informed about the nature of the app. NeuroResponse will have to create the consent form and contact with a suitable institutional review board (IRB), or with an ethics committee (EC), in order to proceed with the data collection.
Data analytics: once deployed we would be able to gather really valuable data from patients that can be used to perform data analyics and ML to improve drastically the care they recive. For example, we could use a predictive analysis model to improve the care pathway.