Testing

Contents [-]

Testing Strategy

Testing Jamboxx-Infinite was a crucial step to ensure our application meets the standards of our goals. Our main testing strategy was through user testing as we focus on delivering an application that offers user experience that is desirable. We implemented pseudo-user tests and user tests to inquire about the functionalities, usability, and receive feedbacks about further improvements and features throughout the development of our project.

To ensure the features of each section of our application works as intended, we identifed the sections of Jamboxx-Infinite for testing.

Virtual Instruments Tests

  • Inputs from mouse, Facial Navigation, keyboard
  • Rearranging notes of virtual instruments
  • Assigned keys to notes display correctly
  • Recording playing virtual instruments

Minigames Tests

  • Playability of note recognition by sound
  • Playability of note recognition by note
  • Playability of rhythm and pattern challenge

AI Music Teacher Tests

  • Lyrics generation from user descriptions
  • Chatbot feature for asking music-related questions

AI Voice Conversion Tests

  • Each step of voice conversion works as intended
  • Conversion of voice from input file to selected model

Settings Tests

  • Ensure play mode feature works as selected
  • Ensure mapping of keys to notes work for all virtual instruments

User Acceptance Testing

Our team had the opportunity of conducting a series of user tests at Helen Allison School, under the National Autistic Society. Two of our team members travelled to the school to collect feedbacks and test our application with users who will be using our app. During the visit, we conducted tests with both students and teachers at the school.

Furthermore, our team took advantage of school visits during lab sessions to test Jamboxx-Infinite with secondary students who are studying Computer Science. We also showcased our project to our clients and other industry partners of this project during the AI for Good Projects Showcase.

Testers

We grouped our testers into several test groups from conducting tests with approximately 35-40 users. This was done to maintain confidentiality of testers and to summarise feedbacks from users.

Test Group Details
Group 1 Students on the autism spectrum around the age of 13-18
Group 2 Teachers at Helen Allison School
Group 3 Students with music background
Group 4 Students around age 15-19 learning Computer Science for A-Levels
Group 5 Students at UCL
Group 6 Project partners and clients

Test Cases

After identifying the sections of Jamboxx-Infinite for testing, we planned out the test cases for each section.

Note: some test cases are not applicable for certain test groups due to features being added after testing with that test group or feature unable to be tested

List of test cases for each section of our application:

1. Virtual Instruments

  1. Playing virtual instruments with mouse
  2. Playing virtual instruments with Facial Navigation motion input
  3. Rearranging notes of virtual instrument with move mode
  4. Record playing virtual instruments

Results and Feedbacks:

Test Case Virtual Instruments

2. Minigames

  1. Playing note recognition by sound
  2. Playing note recognition by note
  3. Playing rhythm and pattern challenge

Results and Feedbacks:

Test Case Minigames

3. AI Voice Conversion

  1. Generate music using Suno AI
  2. Convert voice from music file using DDSP-SVC

Results and Feedbacks:

Test Case AI Voice Conversion

4. Multiplayer

  1. Test hosting and joining server
  2. Test syncing of audio across devices
  3. Test recording multiplayer jam session

Results and Feedbacks

Test Case Multiplayer

5. AI Music Teacher

  1. Test lyrics generation from user prompts
  2. Test AI chatbot

Results and Feedbacks

Test Case AI Music Teacher

6. Settings

  1. Test assigning play modes
  2. Test key assignment for notes of each instrument

Results and Feedbacks

Test Case Settings

Analysis and Conclusion

Our team discussed the feedbacks we received and decided to make a few adjustments:

  • Addition of hover to play mode to make Facial Navigation easier to play virtual instruments as most difficulties while playing was due to users trying to open mouth to click
    • Added in settings page
  • Adjusted keys for musical notation by sound minigame to be in order
  • From a discussion with Professor Dean, we decided to not use Suno as the model is too complex and large. However, as the users enjoyed creating music, we decided to keep AI lyrics generation using a smaller LLM, Llama.ccp
    • AI lyrics generation added in AI Music Teacher page
  • Addition of settings page to assign keys to specific notes for users customisability
  • Modified saving recording feature for virtual instruments so that users can select the path where to save it (similar to multiplayer)
  • Changes to overall UI to follow suggestions from clients