Testing

Details about testing.

Test Strategy

To test the project, we used a combination of testing strategies. Since our solution involves recording audio with Brilliant Lab glasses, generating transcripts, and creating clinical reports, we focused on ensuring the system was intuitive and met the needs of clinical doctors. Continuous feedback was gathered from initial prototypes through to the final product. Additionally, each feature was rigorously tested to verify accurate audio recording, precise transcription, and correct report generation.

Unit Testing

In order to modularize the test structure, we test each feature group separately. We use mocking techniques (using the unittest.mock library, MagicMock, and the patch decorator) to isolate dependencies, allowing tests to focus on the current module's function by replacing external dependencies. This approach also allows us to simulate various edge cases which are difficult to trigger during normal execution.

For example, when testing the OpenVINO acceleration module, we replaced the actual OpenVINO library with mocks. This allowed the tests to run without specific hardware and cover different scenarios (such as model presence or absence, and processing success or failure) to ensure independence of tests.

In the case of the FrameGlassRecorder class, which handles audio recording and hardware interactions with Frame glasses, asynchronous testing was leveraged with pytest and pytest-asyncio. By mocking hardware dependencies like the microphone and display, we were able to test functionalities such as starting and stopping recordings, updating the UI, and handling tap events. This approach helped simulate edge cases, such as hardware failures, ensuring the class behaves reliably under various conditions.

For testing the database model, we use an SQLite in-memory database to validate models (such as Doctor, Patient, DoctorPatient relational tables, and Meeting). The in-memory database offers fast operation and complete isolation. Each test runs against a fresh database environment using setUp (to create a new database and session) and tearDown (to clean up resources), ensuring that a change in one test’s data won’t affect others.

User-Acceptance Testing

On March 11th, we demonstrated our project to our client and interested parties, collecting valuable comments and feedback.

Project Feedback

Positives:

• The transcription and report generation worked very well in a quiet room.
• The report was presented in a standardized format.
• Everything ran offline.
• Users were able to view and edit the report.

Some Issues:

• Glasses Reliability: The glasses are unreliable when starting and stopping audio due to an overly sensitive tap feature, resulting in errors. Additionally, they require a quieter environment.
• UI Design: Some UI design choices were over-complicated. Users were forced to copy and paste the transcript into different sections for report generation and fact checking. The process should automatically use the transcript for those sections to improve user experience.

How We Improved

• Since the glasses proved unreliable, we added the option for the user to record audio using the laptop microphone.
• The system now automatically uses the transcript for report generation and fact checking, eliminating the need for manual copy and paste.

Additional Feedback

• Some users expressed interest in multilingual translation—an area for future projects.
• There is interest in applying such programs to other fields, such as parent-teacher meetings at schools.
• The concept of smart glasses technology was well received because they are easy to wear and carry.