Requirements
Project Background and Client Introduction
We started the project with the goal of being able to socialize the idea of AI model testing with a focus on user centric development. Current solutions host datasets, but what is provided by data scientists are either research papers, or at best source code. Training models or requesting code takes time and recourses, and you can run into problems with dependencies, versioning, and lack of expertise.Our solution will provide trained, dockerised, machine learning models ready to be run on local machines. Our service will also hosts datasets for testing these models, and users can upload results back onto the platform to provide feedback.
Our client is Dr Joseph Connor of the NHS, and he is representative of the clinicians and data scientists that would use our product in the future.
Project Goals
Machine Learning Models such as Image Classifiers are very useful tools to help aid doctors in the diagnosis of various diseases. While developing these tools, it is imperative to have a collaborative platform to host models and datasets to help with versioning, collecting feedback and sharing models (as well as training them on various datasets uploaded by clinicians.
This is why we set out to build this brand new platform, to socialise the goal of AI model testing within the NHS. Current solutions host datasets, but what is provided by data scientists are either research papers, or at best source code. Our goal was to provide trained docker models, ready to be run on local machines. Datasets will be displayed with their compatibilities and users can upload results back into the platform to provide feedback. This allows to effective and secure collaboration. More specific functionality are stated below:
- A closed system, where administrators can add new users with either read permissions or read and write permissions.
- Upload and download of datasets and trained machine learning models which are dockerised server side.
- These uploaded docker images can be downloaded and users can test these models with provided feedback.
- Users themselves can them submit feedback on these models to support collaboration
- These requirements have turned into finished functionality and hosted on Linode
Requirement Gathering and Personas
How did we collect requirements?
We conducted interviews with pseudo–NHS Clinicians and Data Scientists to understand our two main user types, their underlying motivations and what features they want the app to have. Our findings have been consolidated into notes and aided in devising our personas and scenarios. We used the features our pseudo users expressed to formulate a list of requirements which we then discussed with our client. After negotiation on what is needed and what we could deliver efficiently and quickly, we came up with our functional and non- functional MoSCoW chart.
How we designed the questions + analysis of responses (Personas)
We conducted all requirements gathering in the form of video interviews with pseudo users. Based off the problem and project background we were presented with from our client, we designed a survey. We went through this survey with our pseudo users and user their insights to devise our feature list. We ensured the list of questions were open ended, to prevent an interviewer bias.Personas
Use Cases
Use Case Diagrams
Master Use Case Table
| ID | Use Case |
|---|---|
| UR01 | Viewing the datasets and models |
| UR02 | Download dataset |
| UR03 | Download and run model |
| UW01 | Upload dataset |
| UW02 | Upload model |
| UW03 | Delete dataset |
| UW04 | Delete model |
| UU01 | Add new users |
Use Case Table Breakdowns
| ID | Actor | Description | Main Flow | Result |
|---|---|---|---|---|
| UR01 | User with Read Permission | Viewing Datasets and Models | Users logs in. Browses through the pages using the given navigation buttons at the bottom of the page. They can then choose to view the datasets or the models by clicking corresponding buttons. | Viewing Datasets and Models |
| UR02 | User with Read Permission | Download Dataset | The user searches for a dataset and views the dataset. They can also download the dataset and open the downloaded dataset. | Local copy of wanted dataset on user’s machine |
| UR03 | User with Read Permission | Download and run model | The user searches for a model and views the model. Then, they download the model, run it with the instructions from the readme and test it. Optionally, the user can give feedback on the platform. | Local copy of wanted model on user’s machine and they can run on datasets to view results and provide feedback. |
| UW01 | User with Read and Write Permissions | Upload dataset | The user uploads a dataset using the provided information, after uploading, the user can search for the uploaded dataset. Finally, the user can navigate to the dashboard and verify dataset has been uploaded. | Dataset hosted on the platform |
| UW02 | User with Read and Write Permissions | Upload a model | The user uploads a model using the provided information and set up their own requirements and config files. After the user uploads the relevant files, they can navigate to the dashboard and verify the model has been uploaded. | Model hosted on the platform |
| UW03 | User with Read and Write Permissions | Delete dataset | The user navigates to the dashboard and views the datasets they have uploaded, optionally, they can delete their datasets | Dataset deleted off the platform |
| UW04 | User with Read and Write Permissions | Delete model | The user can navigate to the dashboard, view the models they have uploaded, optionally, they can delete their models | Model deleted off the platform |
| UU01 | User with Read, Write and Create New User Permissions | Add new users | The user navigates to the dashboard section, and clicks the ‘add new user’, they then enter the new user's credentials and permissions before adding them to the system. | New user added to the team |
MoSCoW requirements
Functional
| ID | Requirements | Priority |
|---|---|---|
| 1 | A closed authentication / login system where clinicians have download access and data scientists have download and upload access. | Must |
| 2 | Upload of trained machine learning models which are dockerised | Must |
| 3 | Docker images can be downloaded and run locally with a provided frontend, allowing users to make predictions. | Must |
| 4 | Upload of datasets as zips | Must |
| 5 | Download of datasets as zips | Must |
| 6 | Functioning deployment onto Linode | Must |
| 7 | Administrative user type which can create new users | Should |
| 8 | Delete uploaded models | Should |
| 9 | Delete uploaded datasets | Should |
| 10 | Search for datasets / models through a search bar | Should |
| 11 | Private dashboard to view the users own uploaded datasets and models | Should |
| 12 | Upload results / feedback onto the platform | Could |
| 13 | Records of model downloads stored to prompt users to give feedback | Could |
Non Functional
| ID | Requirements | Priority |
|---|---|---|
| 1 | Security | Must |
| 2 | Performance | Must |
| 3 | Reliability | Must |
| 4 | Usability | Must |
| 5 | Software Portability | Should |
| 6 | Maintainability | Should |
| 7 | Testability | Should |
| 8 | Interoperability | Could |
| 9 | Extensibility | Could |
| 10 | Open Source | Won't |