To wrap off the project, this week was filled up by finishing our reports, fixing a few minor bugs and explaining our project ready for the hand over to the next year of. It was a great experience for all of us and we wish the best of luck to the next group carrying on with our project.

As our final round of public showcase, we presented in UCL’s AI for Good Showcase, allowing us to demonstrate our work to a wider audience.

These last few weeks were focussed on making some minor adjustments to our product as well as writing up our website to be published. Talking with our associate professor, we figured out that the youtube downloader may not be suitable even for use in charities such as the NAS. So we built a separate mode which allows the user to record their system audio to generate songs from any website as well as allowing the users to upload their custom audio files.

We conducted our second round of public user testing at the Hellen Allison school, one of NAS’ specialist schools. It was a great experience testing our software with the children who very much enjoyed our product. Although a lot of the testers were non-verbal, their actions and facial expressions.

From the feedback, we improved the particle interaction system to generate more particles and polished the UI layout at the same time.

We also started to test out the packaged version of our product, which will be deployed to our customers at the end of the project. Although we were troubled with the change in relative paths after packaging, a large part of this was fixed in the code.

This week, we had our first round of public user testing with the students from Haggerston school and Enfield Grammar school. Although our product was still early in development, we managed to show the AI functions and a basic shader / particle system working to them.

Stable Diffusion did not work for some reason on OpenVINO so we switched back to using python just for the image generation model. By using a separate library, the library compiled without any problems, allowing us to focus on finding the correct model for our library. Although we managed to find a suitable library which generated images very quickly, the settings for the image generation needed to be changed to create an acceptable image.

This week, we finished the implementation of the youtube downloader and the prototype of the database system. The database uses a file based system to store the data as a json file. The user can now download any youtube video by entering the url, and we store only the audio file, thumbnail and the link to the uploader of the video.

As the LLM worked correctly with OpenVINO GenAI, we managed to create a seamless connection with the speech 2 text model (whisper) to convert the lyrics.

We also started to make large progress with Phillips Hue, after building a custom implementation of the PHEA module, we got streaming to the bridge at a high refresh rate working.

This week, we discovered that Spotify had changed their terms and conditions, which meant that the spotify API which we had relied upon is now unusable. This meant that the beat analysis, sentiment analysis, and other data we relied upon with our software had now become unusable.

Therefore, we switched to using youtube as the audio file and analyse the mp3 directly from the data.

Tested various LLM models but Gemma 2 9b seemed to be the best out of the offline models we tested with. The python compilation still did not work so switched to using C++ implementation with OpenVINO C++ instead. This lacked a lot of documentation as OpenVINO mainly focussed on python examples and the cmake configuration did not work correctly. However, a new release of the OpenVINO runtime for 2025 came with new sample and documentation which allowed us to fix the cmake files, correctly configuring the OpenVINO GenAI library with the Gemma model.

This week we also managed to control our Hue lights through our application over the v2 HTTP web api. This was a huge milestone however we noticed that when trying to sync this with the beat, we were being rate limited, so we had to come up with a different solution.

Research started on the Phillips Hue and it’s integration into our project. From a brief check of the API, it seemed to be comprehensive with uses from it’s own app, with a few other APIs which allows us to call it from our own app.

A basic prototype of streaming Spotify was finished, allowing the user to log onto their personal spotify account from the browser or the spotify app. By connecting to the spotify API, we could stream the analysis data provided by spotify such as their beat analysis, sentiment analysis which can be used for both our Phillips Hue analysis and LLM song analysis.

The majority of the particle physics was finished this week, as we adopted to use a custom made physics system with a preset list of particles and associated physics. Instead of the AI generating the particles from scratch every single time, it will select from a list of particle systems. This will offload the uncertainty of the particle quality to a semi-automatic format.

Testing out with OpenVINO Python worked very well, but compilation with pyinstaller and nuitka seems to be a challenge as it could not find the dll files after packaging into an executable, giving many errors.

We also had the opertunity to present our project at BETT, an education conference that has over 30,000 attendees. We attended a panel on e-sports and gaming in education.

Our main focus for this week was the elevator pitch and creation of a demo relating to that. As we finished a mock up sample of the UI using react, we started focussing on the more technical aspects such as how to add the Stable Diffusion to generate background images and create a database where we could store the data for our songs. We tried to weigh the pros and cons of offline v.s. online server based databases and reached the conclusion that offline databases stored on the machines are better as it allows the clients to run the program without internet connnection.

Further research was conducted into the offline AI libraries such as WebLLM and transformer js but they did not seem to generate a nice stable diffusion image under reasonable times.

After trying to implement particles and animations in Dear Imgui 5, we noticed that it isn’t ideal as the compatibility with UE5 was placing Dear Imgui widgets into the UE5 for debugging purposes, and the main heavy lifting 3D engines were actually third party such as OpenCV. Upon discussion with our clients, we settled on switching to a 2D animation instead with an electron application, using the electron-react-boilerplate code. The UI implementation proved to be a lot faster and easier now as we could use react, which all of our members were familiar with.

The shader could now be done using three js and the particle animation was still in investigation by one of our team members.

Various offline AI libraries on JavaScript was searched but none seemed to be ideal for the project.

Research was continued on Spotify. Looking further at their terms and conditions, we managed to find a statement saying it will be permitted on charities, which is what our target is.

For animation libaries in C++, we had a look into Qt but this required our product to be open source, which we cannot do from our T&C. Instead, we realised that Dear Imgui has compatibility with Unreal Engine 5 which has a 3D engine and supports very nice particle physics. For using offline AI, OpenVINO C++ was checked but it lacked a lot of documentation and wasn’t very well supported.

A mockup of the UI using Figma was also generated at this time, getting ready to convert it into a coded format.