Testing Strategy
Testing the platform was essential for us to ensure the platform is fully functional and to spot any hidden bugs.
We have 3 automated test suites, which run on the GitHub repository automatically on every push using GitHub Actions:
Frontend component unit tests
We did this to ensure, that at the lowest level possible, the standalone parts of our platform were fully functional.
Having a 100% passing test suite gives us confidence that each button, text box, dropdown, etc. has no bugs, so clearing the platform of any simple and obvious problems.
Backend REST API integration tests
Mirroring the logic of the frontend component unit tests, we performed these tests to ensure that at the lowest level testable the backend was fully functional.
Having a 100% passing test suite, with 100% test coverage, again gives us confidence that each HTTP method for every endpoint performs correctly.
Entire platform end-to-end tests (inc. compatibility tests)
Having confidence in both the frontend and backend parts of our platform working standalone, we performed these tests to ensure the platform worked together as a whole, and that there are no problems with how the backend and frontend interact.
Although all three are needed to fully test the platform, the end-to-end tests are perhaps the most important as they also double check the functionality of both the front end and back end.
We also performed Accessibility Testing and User Acceptance Testing.
The rest of this page describes these test suites and testing efforts in more detail.
Frontend component unit tests
After conducting research on how to test React/Next.js projects we learnt a common method is using Enzyme and Jest.
Jest is an open source JavaScript testing framework and assertion library which we found was well documented and had a wide range of APIs for flexibility in performing different types of tests. Another advantage of Jest is that tests are isolated and can be parallelized which allowed us to create a huge test suite that can be run in seconds.
Enzyme allows us to manipulate/use components in our test suite using methods such as mount (to mount a component for testing purposes) and simulate (to simulate interactions with components like clicks).
We now have over 70 unit tests for all our components, and no test failures.
Backend REST API integration tests
Testing our backend was essential too. This required research into best practices and what type of testing is preferred for REST (CRUD) APIs.
We decided against unit-testing the API, as it would have been complex to isolate the logic of the APIs from the network handling and database interactions. Moreover, we wanted to test the database interaction aspect of our API, so isolating it made less sense for this platform.
As a result, we wrote integration tests on our REST API to test its logic and database interactions.
We used Jest's mocking capabilities to mock the session object to test different user roles and the openapi-response-validator open-source library to assert that the received responses matched our OpenAPI specification we created for our API. This had many benefits, such as not needing to write code again to test the response structure when we already have it in our OpenAPI specification.
We also used Jest Test Environments to perform test setups and teardowns to create/clear the database for each test suite.
We now have over 200 tests for our backend REST API, and no test failures.
We achieved 100% test coverage for the API, excluding the Keyloak interactions which were mocked (as these are tested in the end-to-end tests detailed later).
Entire platform end-to-end tests (inc. compatibility tests)
We also created an end-to-end (E2E) testing suite which tests the system as a whole from a user’s perspective, by programmatically navigating to the site, interacting with it, and asserting that the UI displays/functions correctly. We use Puppeteer, a popular Node.js library which provides a high-level API to control headless Chrome over the DevTools Protocol.
We used Jest Test Environments to spawn a Puppeteer browser instance, reset the database, and start a local Next.js web server for each test suite. We went through each use case of our platform and what actions users can take on the platform to test every single feature.
The tests also cover security and access management by ensuring users with limited permissions cannot access unauthorised parts of the platform.
We now have over 50 end-to-end tests, and no test failures.
Additionally, we perform compatibility testing to ensure the platform works on multiple browsers, not just Puppeteer's default of Google Chrome. We utilized Puppeteer's experimental support for controlling Mozilla Firefox and slightly tweaked the tests to use Puppeteer APIs that work in both browsers. Consequently we now have an end-to-end testing suite that passes 100% of the tests on both Google Chrome and Mozilla Firefox.
Accessibility tests
We also performed accessibility testing by following the WAI-ARIA and WCAG. This was very important to make our platform accessible to all potential users so that it can help anyone working in healthcare.
The Chrome Developer Tool Lighthouse provided us with a simple report which gave us pointers on where we could improve in different areas.
We focused on the contrast of the colours used on the platform to ensure all text is readable, using tools such as http://colorsafe.co and https://webaim.org/resources/contrastchecker/. We systematically checked all the colours of our platform in light and dark mode, changing colours to adhere to the WCAG AAA where needed.
We also used different ARIA functionalities like aria-label and role tags which are used by assistive technology to make websites more accessible to people with disabilities. We met our internal goal of reaching 85% accessibility, with most pages exceeding 90%. The Lighthouse report for the various pages is shown below:
User Acceptance Testing (UAT)
Testers
To continue our user-centered development, we created a User Acceptance Testing (UAT) form using Microsoft Forms to receive feedback from our end-users (clinicians).
The form contained an introduction at the top for the platform and gave instructions on how to log in/register. It was split into different sections corresponding to our platform: registration process, self reporting page, statistics page, manage page and an overall thoughts section.
Users were able to use these sections as a guide on navigating around the site themselves, enabling them to provide valuable feedback to us.
An excerpt of the UAT form is below:
Test Cases
We received a number of quality responses from our users via the UAT form on which we were able to take relevant action. For example, we made it clear that the final two single-word questions in the self report optional, instead of mandatory:
Feedback
The overall feedback from our client and users was extremely positive. They thought the platform was very well designed/implemented and solved their problem. Because of their day to day activity and expertise in the field, they were able to guide us in adding a list of features to the platform that we had not thought about ourselves such as the creation of 2 word clouds: 'enablers' and 'barriers', to clearly separate positive thoughts from negative ones, and a bug in the user registration process.
Users also suggested a few features that we did not implement but that we highly recommend the next team of developers to work on, once we handover the project. See the evaluation page for more details.