Summary of Achievements

Functional Moscow List Achievement

ID	Description	Priority	State	Contributors
1	Allow multiple users to create/edit/cancel appointments at the same time and automatically update after editing by refreshing the screen.	Must	✔	Arslan Kamil
2	Record and view the changes in an appointment.	Must	✔	*self fulfilled
3	Show appointments for today by default and be able to show appointments for selected date and selected week.	Must	✔	Arslan Kamil Yue
4	Ability to view specific patients' detail by click patient's name.	Must	✔	Arslan Kamil Yue
5	Ability to view and search appointments.	Must	✔	Arslan Kamil
6	Ability view and search patients.	Must	✔	Arslan Kamil Yue
7	Ability to see and hide team rota.	Must	✔	Arslan Kamil
8	Cancelled appointments should be automatically removed from rota to reduce repetitive work.	Must	✔	Arslan Kamil
9	Ability show all free slots.	Must	✔	Arslan Kamil
10	Schedule appointment to multiple slots if it duration is longer than 1 regular slot.	Must	✔	Arslan Kamil
11	User authentication for the ocularists and admin staff.	Must	✔	Arslan Kamil
12	Use distinct and intuitive colours to indicate different appointment type.	Should	✖
13	Options to filter information about a patient at waitlist page.	Should	✔	Arslan Kamil
14	Hide patient already seen from timetable.	Should	✖	**self fulfilled
15	Automatically fill in patient details when booking new appointments.	Should	✔	Arslan Kamil
16	Send notifications if appointments due within 2 days are cancelled.	Should	✔	Arslan
17	Suggest patients from waitlists according to their waiting time and urgency.	Could	✖
18	Automatic time slot suggestion when scheduling a patient.	Could	✖
19	Manufacturing Tracking.	Won't
20	Statistics Tracking.	Won't
Key functionality (Must have & Should have)			87.5%
Optional functionality			0%

*PAS provides this functionality

**Previous day's appointments no longer appear on the default view.

We are happy to present a proof-of-conecpt that fulfils 87.5% of the key functionality outlined in the functional MosCoW list requirements. We recognise the hard work and dedication our development team has made throughout the project.

Non Functional Moscow List Achievement

ID	Description	Priority	State
1	Be delivered with documentation in the form of a website.	Must	✔
2	Applied the the design principles of visibility, consistency and feedback wherever it is deemed nessecary.	Must	✔
3	Deliver an extensible system.	Must	✔
4	Prevent unauthorised access/only allow access to legitimate users.	Must	✔
5	Deliver a codebase that follow best practices.	Must	✔
6	Be able to run in multiple platforms.	Should	✔
7	High learnability.	Should	✔
8	Be scalable for larger numbers of patients database.	Should	✔
9	Automatic deployment/CI Pipeline.	Could	✔
10	System integration with PAS and OpenEyes.	Won't
Key functionality (Must have & Should have)			100%
Optional functionality			100%

Individual Contribution

Part of Project	Arslan	Kamil	Yue
Project Partners Liaison	60%	40%	0%
Requirement Analysis	37.5%	37.5%	25%
Research and Experiments	47.5%	47.5%	5%
UI Design	35%	35%	30%
Coding	45%	45%	10%
Testing	0%	100%	0%
Deployment	100%	0%	0%
Report Website	14.33%	57.33%	28.33%
Blogs	87.5%	0%	12.5%
Presentations	33%	33%	33%
Monthly Videos	0%	50%	50%
Overall Contribution	45%	45%	10%
Main Roles	Programmer, Blog Editor, Deployer	Programmer, Website Editor, Tester	Video Maker, UI Designer, Website Editor

Known Bugs

This is a list of known bugs, their severity is ranked from Low, Medium and High.

Bug Description	Severity	Solution
Users can set an already cancelled appointment back to booked status. This should not be allowed. Users should instead create a completely new appointment to replace the cancelled one.	Medium	Add tighter validation to UpdateAppointmentView.py to remove the option to adjust the appointment status The user may only change an appointment from booked to cancelled, and never the other way around. This functionality is available from the appointment list already.
If searching for a cancelled appointment, the result will be displayed under 'Upcoming Appointments' and a cancel button will be displayed. A user should not be able to cancel an already cancelled appointment.	Medium	The methods used to fetch appointments in AppointmentView.py is mismatched to the one that is called when searching for an appointment. i.e. the get method for this class correctly applies and returns the tags for appointment status, however post has not been updated to do the same!
On the appointment form, the dropdown list for appointment code section displays "appointment code object N07" instead of "N07".	Low	The Appointment model currently stores AppointmentCode as appointment_code = models.ForeignKey( AppointmentCode, ...). I.e. it saves the entire AppointmentCode object as the status, we need to replace it with appointment_code = models.ForeignKey( AppointmentCode.appointment_code, ...)

Critical Evaluation

In order to perform a critical evaluation on ourselves we will evaluate our performance in different criteria and give ourselves a ranking on the following scale:

Bad, Not Good, Good, Very Good, Excellent

User Interface and Experience

We followed HCI design principles of visibility, consistency and feedback while implementing our user interfaces.
There are various examples of where we applied these principles across our system, these examples can be found by clicking the 'UI Design' tab on the navigation bar.

Also, we always made attempts to evaluate the project from a user's perspective, this was done in 2 ways.

☑ We demonstrated our progress with the project in our weekly meetings with Moorfields. As our client was going to be the main user of the system, their feedback was from a user's perspective. We considered their feedback to help build a system that provided the best user experience. We balanced their feedback with the time constraints to avoid scope creep.

☑ We conducted a user acceptance test's on users who have not used the system before. We used users who have different levels of software proficiency to perform tasks on our system. The result shows that all testers found our application easy to use. A senior ocularist at Moorfields said they were particularly impressed by the design of the rota page. Testers pointed out that found difficulty in selecting and viewing multiple ocularists' rotas on the team rota page. To remedy this, we created a user manual that gives new users instructions on how to use our system. Further feedback from our user acceptance tests can be found on the 'Testing' tab on the navigation bar.

Overall, from the feedback from our client (who is also the user), and our user acceptance testers, our application provides a simple, easy to use and usable user interface. We believe that this was achieved by stringently following the HCI design principles.

We rate our achievement in 'User Interface and Experience' as Excellent.

Functionality

We successfully delivered workflow management proof-of-concept. The application resolves all of the major frictions in Moorfields' current system which we identified at the start of the project.

The major frictions and their implemented solutions are as follows:

Friction	Implemented Solution
Too much information presented at once.	Organised data representation using filters, expanding windows and search functionality.
Tedious to triage patients	Avoid repetitive work by reflecting changes across the system.
Multiple users can't use the system at once without overwriting changes.	Synchronisation between users to prevent overwriting.

We are very proud of accomplishing 87.5% of the key functionality in the functional MOSCOW list. It is also important to note that our client is very satisfied with the proof-of-concept that we handed over to them.

For the requirements that we didn't fulfil, specifically requirements 17 and 18 these requirements involve automatically arranging appointments and suggesting patients. We believe that the best way to implement this functionality involves machine learning and is quite complex. The complexity of this is important because, due to the time constraints, we decided that implementing this complex functionality was not feasible. Therefore, we mutually agreed with Moorfields that this functionality would not be included in the proof-of-concept that we would hand over to them. Moorfields confirmed that ocularists and admins prefer to manually book in appointments so we decided to narrow down the scope and focus on fulfilling other major requirements.

Overall we produced a proof-of-concept that Moorfields is very satisfied with and solves Moorfields' major frictions with the current system and achieves 87.5% of the functionality outlined in our functional MoSCoW list.

We rate our achievement in 'Functionality' as Excellent.

Stability

We conducted two different types of automated testing to ensure our application is stable, we used unit tests and integration tests. We managed to reach a code coverage of 97%. Passing all unit tests informs us that individual functions work as we intended. Passing all integration tests informs us that views integrate with templates to render or redirect to the correct template.

One way that we could improve our testing procedure to more rigorously measure the stability is to check what values are passed to each template in the response. Currently, this check is not performed on every integration test. Implementing this check would only require adding one extra assert statement to the test so there is not much extra work for future developers.

Overall we have a high test coverage with all tests passing. Some tests could be extended to be more rigorous.

We rate our achievement in 'Stability' as Very Good.

Efficiency

Since our codebase is very simple, it’s very efficient to run. Currently, our application responds to a request within a few seconds. It processes searches and logins within a few seconds too.

This proof-of-concept will not be used with real patients data, it will be used to aid development for Moorfields. Because of this, it was a low priority for us to measure efficiency, particularly with a large database of patients as this was unlikely to happen under Moorfields normal usage of the system.

Currently, using a small database of patients we rate our achievement in 'Efficiency' as Excellent.

*To further measure efficiency, beyond what we measured, future developers should test on larger datasets.

Compatibility

We deployed our application by running it as a docker image and hosting it on Azure Cloud. This allows our application to be accessed by multiple browsers, across multiple devices from anywhere with an internet connection.

In addition to this, we programmed the front-end in Bootstrap, this improves the compatibility because Bootstrap supports automatic formatting for different devices. To elaborate, the page and the navbar will automatically change based on screen size in order to display all of the contents in a clear and logical fashion.

If multiple systems were to be integrated such as synchronising our workflow management app with PAS, a virtual machine could be a better option than docker as docker is better suited to supporting one application. If this is the case, Moorfields can implement this as the source code for the project has been provided via a git repository.

Overall our system has compatibility across multiple devices. It has good potential for compatibility with other systems that Moorfields use. If Moorfields would like to implement this compatibility across systems we have suggested how they could do so.

We rate our achievement in 'Compatability' as Excellent.

Maintainability

Our code comes with documentation in for form of a website. Other developers should not have any trouble understanding what the code does and how it does it. Meaning it would be easy for them to modify the code to implement new features

Our code also comes with an easy to use, CI pipeline. Developers only need to push to main for changes to be reflected on the live server, more information about the CI pipeline can be seen on the 'Deployment' tab on the navigation bar.

For the most part, our code complies with PEP8 standards. We performed checks on our code with flake8 and fixed the syntax standard violations. We did this to make our code as readable as possible for future developers, we believe that the more readable the code is, the more maintainable it can be.

We also have a small codebase with sensible variable names which further makes the code readable and easy to follow.

We implemented separation of concerns this can further help with maintainability, we did this when we separated the views into different files and and when we followed the MVT design pattern.

Overall we took multiple actions to make our codebase as maintainable as possible in order to aid development during the project and future development by Moorfields.

We rate our achievement in 'Maintainability' as Excellent.

Project Management

Our team used a Gantt chart timeline to make sure our project is on track and we used Trello board to organise our work and to set tasks for teammates during the design phase. We shared these with our TA, our IXN mentor and Moorfields.

We met our client weekly on Thursday afternoons to discuss progress. During each of these meetings, we took minutes so we could refer back to them during the week. We also had weekly meetings with our TA and our IXN mentor.

During the implementation phase, it was easier for us to allocate tasks on a Google doc than on Trello. The Google doc had a list with checkboxes of tasks that needed to be completed. The tasks were colour coded based on priority and who was assigned to it. A majority of the team completed tasks by the agreed deadlines.

We used telegram to communicate with teammates and email to communicate with Moorfields.

Overall, we rate our achievement in 'Project Management' as Very Good.

Further Work

If more time was given, we would make implement the following features to further improve our system:

1. We would integrate our system with PAS. Doing so would save time on things like adding patients to the workflow application. We could also add functionality to automatically create and send letters to patients once appointments have been made, this would save time for the admin staff.

2. Automatic appointment allocation. This was one of the suggestions made when exploring the requirements at the start of the project. We thought it was too complex to fully implement in our timeframe. Future developers could use information about previous appointments to predict when the best time is to book a new appointment and automatically book this appointment.

3. Further support for mobile device users. Moorfields primarily use computers to manage patients so we spent all of our time designing our UI for a computer user, however, using Bootstrap meant that all of the content is still visible on mobile devices. In the future, we believe we could provide a more usable UI for mobile devices if we spent more designing specifically for these devices.

4. Improve the login experience. Currently, our login page is very bare-bones. If we had extra time, we would consult users on how to improve the experience. We believe that implementing 'Forgot Password' functionality may be one way to improve the user experience when logging in.

5. Prompts that require users to confirm their actions. Currently, a user can delete an appointment at the click of a button. Although the purpose of the button is very clear, it may be a good idea to ask users to confirm their actions to avoid mistakes from occurring when using the system. If it takes a significant amount of extra admin work to rebook the appointment, this functionality could be incredibly useful.

6. Functionality for manufacturing tracking. We explored this functionality at the start of the project however we decided to narrow the scope and not implement this in our system. Once our system is polished it could be useful for Moorfields to track which materials were used for which prosthetic.

7. Given more time, we would love to further refactor our codebase to make things even more modular. A lot of our functionality refreshes the webpage the user was currently on, which means that we duplicate sections of code in a lot of places. For example, at many points we have to fetch the list of appointments, and we in fact have a known bug where sometimes the list of appointments is not tagged properly! This can be all solved by introducing a base method that returns the information needed to populate the template, and then overriding some of the other functionality, to differentiate the operations.

8. Currently, we have tested the system for simultaneous use; and it works! The backend model on the server will track changes in real time and prevent overwriting of data. However the visuals on screen are not always seen in real time, you have to infact refresh to see the changes to the rota or appointment list. Given the time and resources, we would like to implement a solution that uses AJAX in order to automatically refresh the template whenever the backend Model changes.

9. We have set user groups and permissions seperately for admin staff and ocularist staff, and given then time it would be natural to retrict their permissions when using the app. This way the PAS checklist can only be confirmed by a member of the admin users group. Features like This would make the system more reflective of who has what work to do, and decreases the chance for someone to make a mistake.

10. Everytime that we Dockerise and deploy the application to Azure, the local database is used to create the Docker image. As such, all remote data, i.e. on Azure is lost and overwritten by the local copy of the database. Though this is fine for a proof of concept, we should have the backend database mounted as a persistent file storage to our container registry or container instance. There already exist solutions to this, and it would be natural during real world use to want to maintain the information located in the database. Here is what a quick search reveals about this!