Super Happy Space README
Project Overview
This project provides an affordable, AI-powered sensory room designed to support autistic children by creating immersive, adaptive environments tailored to their emotional needs. Using accessible hardware like Intel-based computers, Philips Hue lights, and motion-tracking technology, the room dynamically responds to speech and gestures, adjusting visuals, sounds, and lighting. By enabling intuitive interactions and emotional regulation through sensory stimuli, this solution offers an effective and scalable approach to improving communication skills, emotional resilience, and overall comfort for children with autism.
Particle Visualiser
The Particle Visualiser creates dynamic, responsive and interactive particle systems tailored to each song's theme and mood. Using a physics-based engine, particles exhibit realistic behaviors including gravity, bounce effects, air resistance, and custom lifespans.
Key features include:
-
Customisable Particles: Each particle type has adjustable physical properties (weight, gravity, bounce, air resistance, lifespan) that can be modified through an intuitive settings interface.
-
Custom Imagery: Upload and manage multiple images for each particle type, allowing for rich visual diversity.
-
Interactive Experience: Particles respond to user interaction - they can be generated with clicks and move according to the music's rhythm and intensity.
-
Song-Specific Selection: Each song can have multiple particle types assigned to it, with a simple interface to add or remove particles based on the desired visual effect and mood.
-
Create Custom Particles: Users can create their own particle types with custom names and configure all their properties to achieve unique visual effects.
-
Glow Effects: Toggle glow effects for particles to enhance visual appeal during moments of musical intensity.
The particle system is designed to be highly performant while providing soothing yet engaging visual representations of sound that can be fully customised to support different sensory preferences and needs.
Shader Visualiser
The shader visual allows you to interact with the particles simply by moving your mouse (or hand gestures with motion input, COMING SOON!). The particles read from a specific texture and will make move ments according to the music at the same time.
You can also set colour, background and particle shape yourself from the settings in each song.
Hue Integration
Our integration with the Philips Hue Entertainment API provides an immersive experience, synchronising room lighting in real-time with on-screen visuals and audio. Lights dynamically respond to music beats, vocals, and directional cues—for example, on-screen activity occurring on the right side will trigger more intense lighting effects on the corresponding side of the room, creating an engaging and interactive environment.
Installation
Recommended Installation
Download our zip file from our release on github. The zip file contains SuperHappySpace.exe, which is the main executable for our product.
Github installation
- Clone the repository TODO: Change to closed repo
git clone git@github.com:hugomarfani/AudioVisuliserV3.git
- Install dependencies
Node.js dependencies
npm install
AI dependencies
Go to the resources folder from the root directory. This will automatically contain the executables required.
cd resources
For default settings (in accordance with our product created for Super Happy Space), create a directory named AiResources and download the models in there either manually or using git lfs
mkdir AiResources
cd AiResources
# download and move all 3 models from huggingface here
If using git to install the models:
- download Git LFS following the instructions here
- Run the following commands
git lfs install
git clone https://huggingface.co/OpenVINO/distil-whisper-large-v3-int8-ov
git clone https://huggingface.co/OpenVINO/gemma-2-9b-it-int4-ov
git clone https://huggingface.co/IDKiro/sdxs-512-dreamshaper
Otherwise, download the models from the following links:
And place them in the AiResources directory.
Finally, download the openvino package from the zip file here.
Unzip the file and rename it to openvino_2025 and place it in the AiResources directory.
Compiling from Source
OpenVINO C++
The C++ implementation using OpenVINO and OpenVINO GenAI library has 2 ways to compile, using a dynamic library format or a static library format. The dynamic library format is recommended.
Dynamic Build
Download the openvino package from the zip file here.
Unzip the file and rename it to openvino_2025 and place it in the AiResources directory.
Run the setup script from the root directory
./resources/AiResources/openvino_2025/setupvars.bat
# or setupvars.ps1 for powershell systems
From the root directory, move to the external directory and create a build directory
cd external
mkdir build
Run the following commands to build the project
cmake -S ./project -B ./build
cmake --build ./build --config Release
The executable will be in under ./external/build/Release/cppVer.exe.
Move this file to ./resources to be used by the main application.
Static Build
It is possible to create static build of openvino from the source files. However, this is not a recommended method as it is more complex and the nuumber of errors the user encounters heavily depends on their setup.
First clone the necessary repositories and submodules
git clone https://github.com/openvinotoolkit/openvino.git
git clone https://github.com/openvinotoolkit/openvino.genai.git
git submodule update --init --recursive
mkdir build & cd build
Run the following commands to build the project, assuming you have Visual Studio 2019 installed, choose the correct generator for your system.
cmake -G "Visual Studio 16 2019" -DBUILD_SHARED_LIBS=OFF -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE={path to openvino}/cmake/toolchains/mt.runtime.win32.toolchain.cmake -DOPENVINO_EXTRA_MODULES={path to openvion.genai} ../openvino
Compile the cmake project
cmake --build . --target openvino --config Release
Install the project onto your machine
cmake -DCMAKE_INSTALL_PREFIX=C:\Users\billy\Documents\coding\temp\install -P .\build\cmake_install.cmake
After these steps, the openvino library will be installed on your machine and it should be possible to link it to the C++ project using cmake.
Diffusers Python Pyinstaller
Go to the external directory and run the following command:
cd external
pip install requirements.txt
pyinstaller –F SD.py
The build was done with python version 3.10.0. The executable will be in the dist folder. Move the executable to the resources folder to be used by the main application.
Starting Development
Start the app in the dev environment:
npm start
Packaging for Production
To package apps for the local platform:
npm run package
The folder with the executable will be under ./release/build under the name win-unpacked.
Coders
Dependencies
This repository was forked off electron-react-boilerplate.