System Design

System Architecture Diagram

Our architecture diagram presents an abstracted view of how the key components of our system fit together.

Key Components

System Launcher MFC

• The system launcher MFC is an application which the user launches as the first step to run the UCL Open-Illumiroom V2 system.
• Upon opening the launcher, the user can select their display, calibrate the system, select their projection mode and mode settings, and open and close the projection window (the frames the projector projects onto the user’s TV background.

Settings JSON Files

The JSON files store the settings for UCL Open-Illumiroom V2. There are 2 settings JSON files:

• general_settings.json – for general settings such as details about the connected displays, the selected mode, information about the user's webcam and more
• mode_settings.json – for storing data used for initialising and running each projection mode

Main System

The main system is the application for UCL Open-Illumiroom V2 which has been compiled from Python to C. It defines and executes the system features. The main system contains:

• Projection mode classes - each class contains the methods to trigger the mode’s operation e.g., the Rain mode is concerned with creating a falling rain animation and maps this onto the TV background.
• System util classes - these are classes which allow the projection modes and the rest of the system (e.g., the system setup) to run. These include methods to access the settings, analyse audio, capture display data from the TV, and display the output using the projector. Additionally, the calibration and display selection options in the MFC call the utils here.
• Projection display - the main system runs the DisplayOutput util class, which receives frames from the projection modes and displays them onto the projector display.

Object-Oriented Design

UCL Open-Illumiroom V2 was designed with an object-oriented design in mind. These were well established to ensure that the system is robust, efficient, and extendable.

Design Principles & Patterns

By following object-oriented programming principles, we have used the factory method to create an instance of the Mode class, separating object creation from implementation. The Mode class is a superclass for all other projection modes, so they inherit its abstract ‘trigger’ method to define their own implementation, generating frames for the mode’s effect. By using inheritance in this way, we take advantage of polymorphism of the ‘trigger’ method whilst encapsulating the implementation of each projection mode class. Here is a simplified version of how this works:

            
            mode_object = mode_factory.get_mode()
            # Trigger the mode object to get the frames to display
            for frame in frames:
               # Display the frame on the projector
               display_output.display_frame(frame)
            
          

The PySide2 library is used to display full screen windows onto the projector (as defined in our DisplayOutput class). For the projection modes, the window is updated with the frames received from triggering the mode.

Dependency Injection

Dependency injection is used to create single instances of util classes to pass to projection modes and other objects that make use of them, allowing functionality to be shared between objects without needing to redefine and repeat code. An example is shown here in the main function: the settings_access, room_image_obj, display_capture and fps util objects are instantiated, then passed to the AudioCapture and ModeFactory objects, as well as the main_loop (where each projection mode object is initialised, triggered and the frames are displayed).

Extendibility and Modularity

Abstraction is also used via the utils classes (e.g., SettingsAccess, DisplayCapture, DisplayOutput) to ensure that objects that use the utils only have access to the required methods, ensuring object security.

A key aim when developing UCL Open-Illumiroom V2 was designing the system that can be extended by future developers, allowing them to create new projection modes and methods of information capture. By making use of object-oriented design, new modes can easily be defined and implemented. For the same reason, we also prioritised modularity within the code. All utilities can be reused (avoiding code duplication) and the output of the modes, the generated frames, can be displayed on the projection display.

System Configuration

All general and mode settings can be changed in the respective JSON files. By not hard-coding values (i.e., using magic numbers), users and developers have greater control and customisability over the projection modes. Since we have linked these settings to the MFC, they also have an interface through which they can administer these changes.

Packages Used

The UCL Open-Illumiroom V2 system is primarily built in Python, as such the packages are Python based. The calibration system was implemented in C++ and compiled to an executable.

CV2 (OpenCV)

CV2 was used for many different functions throughout UCL Open-Illumiroom V2. These include but are not limited to:

• Blurring frames in the blur mode
• Resizing frames
• Remapping calibrated frames
• Generating effects such as rain, snow, and weather

NumPy

NumPy was generally used for array related functions, such as:

• Generating arrays of 0s
• Reshaping arrays
• Storing pixels of the TV background image to apply sinusoidal transformations for the Wobble mode

MSS

The MSS library is used to get information about the displays connected to the user’s computer, as well as take screen grabs in the DisplaySelection class. Screen captures are also used by the DisplayCapture class to capture the frames displayed on the user’s TV.

PySide2

PySide2 (the Python binding of the GUI toolkit Qt) is used to create a window for the projector to display each projection mode. The window is updated with the processed frames before being displayed.

Nuitka

Nuitka is used to compile the Python source code to C. This allows a distributable executable to be deployed to various sources such as the Microsoft Store.

Librosa

Librosa is a library that can be used to perform audio analysis in python. This was used to analyse the audio patterns generated by gunshots and explosions for the Wobble mode. Unfortunately, due to compilation problems, the version of Wobble including this library could not be deployed. This is because Nuitka does not support compilation of Librosa. However, our open-source release of the project includes this.

Pytesseract

Pytesseract’s OCR capabilities are used in the Speed Blur mode to read the speed of a car in a racing game from the speedometer. The read in value is then converted to an integer to allow the blurring to be scaled based on the player’s car’s speed.

Keras

Keras is used to load in the machine learning models for automated weather detection and audio classification for the Weather and Wobble modes. These models are pre-trained and loaded as assets in the ml_models folder.