Phillips Hue Integration

The Philips Hue Entertainment integration is handled primarily by the HueService class. This class manages communication between the Electron application and the Philips Hue Bridge, enabling real-time synchronisation of lighting effects with audio and user interactions. Unlike standard Hue APIs that operate over simple HTTP requests, Hue Entertainment employs DTLS (Datagram Transport Layer Security) for secure, low-latency streaming.

HueService initialises the necessary network components, including setting up HTTPS and DTLS communication via axios and the specialised node-dtls-client library. The service class maintains internal state information such as active light IDs, their current and target colours, and positional data for spatially accurate lighting effects. Below we explain how the service is implemented and what it is capable of.

Hue Bridge Discovery and Registration

The discovery of available Philips Hue Bridges on the local network is performed using the official Hue discovery endpoint (https://discovery.meethue.com/). The method handleDiscover() sends an HTTP GET request via Axios, obtaining details such as the bridge ID and IP address.

private handleDiscover = async (): Promise<any[]> => {
    try {
        const response = await axios.get('https://discovery.meethue.com/');
        return response.data.map((bridge: any) => ({
            id: bridge.id,
            ip: bridge.internalipaddress,
            name: 'Hue Bridge'
        }));
    } catch (error) {
        console.error('Bridge discovery failed:', error);
        return [];
    }
};

After discovery, registration with a Hue Bridge is handled by the handleRegister() method. Users are required to press the physical link button on the bridge, after which the app sends a POST request to generate secure credentials (username and clientkey).

private handleRegister = async (_: any, ip: string): Promise<any> => {
    try {
        const response = await axios.post(`http://${ip}/api`, {
            devicetype: "audio_visualizer#app",
            generateclientkey: true
        });

        if (response.data[0].success) {
            return {
                username: response.data[0].success.username,
                clientkey: response.data[0].success.clientkey
            };
        } else {
            throw new Error(response.data[0].error.description);
        }
    } catch (error) {
        console.error('Registration failed:', error);
        throw error;
    }
};

Hue Entertainment Group Management

Entertainment Groups are collections of lights organised spatially for synchronised lighting effects. The handleFetchGroups() method retrieves available Entertainment Groups using the Hue CLIP v2 API, with fallback support to the older v1 API for maximum compatibility.

Each group’s details, such as UUID, numeric ID, and associated lights, are stored for later reference.

Persistent Configuration

To enhance user convenience, Hue Bridge settings and credentials are stored persistently in a JSON file located in the application’s user data directory. Methods handleSaveSettings() and handleGetSettings() are responsible for writing and reading these settings, enabling seamless session continuity.

private handleSaveSettings = async (_: any, settings: HueSettings): Promise<boolean> => {
    try {
        fs.writeFileSync(this.settingsFilePath, JSON.stringify(settings, null, 2));
        return true;
    } catch (error) {
        console.error('Failed to save Hue settings:', error);
        return false;
    }
};

Streaming

Streaming via DTLS Connection

Once authenticated, the app establishes a DTLS connection to the Hue Bridge using the method createDtlsSocket(). This method configures the DTLS client using the previously obtained credentials (username and clientkey). It leverages the node-dtls-client library to establish a secure UDP-based connection suitable for high-frequency streaming.

private createDtlsSocket(ip: string, username: string, clientkey: string): Promise<any> {
    return new Promise((resolve, reject) => {
        const pskBuffer = Buffer.from(clientkey, 'hex');
        const options: DtlsOptions = {
            type: "udp4",
            address: ip,
            port: 2100,
            psk: { [username]: pskBuffer },
            timeout: 10000,
            ciphers: ["TLS_PSK_WITH_AES_128_GCM_SHA256"]
        };

        const client = dtls.createSocket(options);

        client.on("connected", () => resolve(client));
        client.on("error", (error: any) => reject(error));
        client.on("close", () => this.isStreaming = false);
    });
}

Starting and Stopping the Streaming Session

The methods handleStartStreaming() and handleStopStreaming() manage initiating and terminating the streaming session. Starting streaming involves enabling the Entertainment Group, establishing the DTLS connection, and initiating continuous streaming of lighting commands. Conversely, stopping streaming gracefully closes the DTLS socket and disables streaming mode on the bridge.

private handleStartStreaming = async (_: any, {
    ip,
    username,
    psk,
    groupId
}): Promise<boolean> => {
    try {
        await this.startEntertainmentGroup(ip, username, groupId);
        this.socket = await this.createDtlsSocket(ip, username, psk);
        this.isStreaming = true;
        this.startStreamingManager(); // continuous updates
        return true;
    } catch (error) {
        console.error('Failed to start streaming:', error);
        return false;
    }
};

Light State Management and Streaming

Real-time updates of the Hue lights’ colour and brightness are managed by continuously streaming commands over the DTLS connection. The method createCommandBuffer() prepares the commands as specified by the Hue Entertainment protocol, converting RGB values to 16-bit format and packaging them into protocol-compliant buffers.

private createCommandBuffer(entertainmentId: string, lightCommands: { id: number, rgb: number[] }[]): Buffer {
    const protocolName = Buffer.from("HueStream", "ascii");
    const header = Buffer.from([
        0x02, 0x00, this.sequenceNumber & 0xFF, 0x00, 0x00, 0x00, 0x00
    ]);

    const entertainmentConfigId = Buffer.from(entertainmentId, "ascii");
    const lightBuffers = lightCommands.map(light => {
        const [r, g, b] = light.rgb.map(val => Math.round((val / 255) * 65535));
        return Buffer.from([
            light.id,
            (r >> 8) & 0xFF, r & 0xFF,
            (g >> 8) & 0xFF, g & 0xFF,
            (b >> 8) & 0xFF, b & 0xFF,
        ]);
    });

    this.sequenceNumber = (this.sequenceNumber + 1) % 256;
    return Buffer.concat([protocolName, header, entertainmentConfigId, ...lightBuffers]);
}

Connection Sequence Diagram

Cursor Interaction

Cursor interactions are implemented to provide real-time visual feedback by directly linking the user’s mouse movements with dynamic Phillips Hue lighting. This interactive feature is managed by capturing and normalising cursor positions relative to the dimensions of the visualisation area, enabling accurate spatial mapping onto physical Phillips Hue lights.

Capturing and Normalising Cursor Positions

The cursor interaction begins with mouse events (onMouseMove) attached to the visualisation component. These events capture precise cursor coordinates (clientX, clientY) within the application window. To normalise these positions, the component calculates their position relative to the visualisation container’s dimensions:

const handleMouseMove = (e: React.MouseEvent<HTMLDivElement>) => {
  const rect = e.currentTarget.getBoundingClientRect();
  const cursorX = (e.clientX - rect.left) / rect.width;  // Normalised X (0–1)
  const cursorY = (e.clientY - rect.top) / rect.height;  // Normalised Y (0–1)

  updateLightPositions(cursorX, cursorY);
};

Here, cursorX and cursorY values range between 0 and 1, providing a consistent and resolution-independent representation of cursor position.

Mapping Positions to Phillips Hue Lights

Normalised cursor coordinates (cursorX, cursorY) are subsequently translated into physical coordinates corresponding to the Phillips Hue lights’ spatial arrangement. Each Hue light within an Entertainment Group has predefined positional coordinates. The cursor’s normalised position serves as an interactive input, influencing how lights respond in real-time:

const updateLightPositions = (cursorX: number, cursorY: number) => {
  const updatedLightCommands = hueLights.map((light) => {
    const distance = calculateDistance(cursorX, cursorY, light.x, light.y);
    const brightness = Math.max(0.1, 1 - distance); // Lights closest to cursor are brighter

    return {
      id: light.id,
      rgb: calculateRGBBasedOnDistance(brightness),
    };
  });

  processHueLightingUpdate(updatedLightCommands);
};

In this snippet:

• calculateDistance() determines each light’s distance from the cursor, adjusting brightness inversely.
• calculateRGBBasedOnDistance() dynamically computes RGB values highlighting proximity between the cursor and each Hue light.

Real-time Communication with Phillips Hue Bridge

The generated lighting commands are streamed to the Phillips Hue Bridge using the previously established DTLS connection, facilitated by the HueService class.

const processHueLightingUpdate = (lightCommands: Array<{ id: number; rgb: number[] }>) => {
  window.electron.hue.sendLightingCommands({
    isInteractive: true,
    lights: lightCommands,
  });
};

Beat Detection

Beat detection is integrated with the lights, allowing the system to react dynamically to music playback through brightness and colour transitions. We explain how the beat detection algorithm is implemented in the Audio Processing section.