System Design

The system is designed to integrate hardware, application, and AI services to process audio data and generate meaningful outputs. It consists of the following tiers:

• Client Tier: An Electron-based application with an HTML/CSS/JS UI that communicates with the server using HTTP and Server-Sent Events (SSE).
• Application Tier: A FastAPI server that manages API endpoints, session management, and streaming. It includes an audio pipeline for processing audio files and AI services for tasks like transcription, fact-checking, and diarization.
• Hardware Tier: Includes frame glasses and a laptop microphone for capturing audio input.
• Data Tier: A SQLite database managed with SQLAlchemy ORM for storing processed data and metadata.
• Resource Tier: Stores raw and processed files, as well as AI models and embeddings used for tasks like transcription and fact-checking.

The system architecture ensures seamless interaction between these tiers, enabling efficient audio processing, transcription, and report generation.

System Architecture Diagram

Below is the overall system architecture diagram:

Sequence Diagram for (server) Observer and Glasses/Recorder Initialisation

This diagram illustrates how the observer on the server and the glasses/recorder are initialised. It works in the background, so it starts when the application starts.

Sequence Diagram for Audio Recorder

This diagram demonstrates the steps taken by the audio recorder component.

Sequence Diagram for Transcript

This diagram shows the process of converting audio to a transcript.

Sequence Diagram for Report Generation

This diagram illustrates how a report is generated from the transcript.

Sequence Diagram for Fact-Check

This diagram details the fact-check process within the system.

ER Diagram

This Entity-Relationship diagram represents the data structure and relationships used in the system.