Hand out of the frame

Camera sampling when looking for the hand

When developing a system that requires the use of hand gestures to switch between multiple cameras, several factors must be considered to ensure that the system operates efficiently and effectively. One of the key challenges that may arise is the placement of the cameras, which can greatly affect the system’s ability to recognise and interpret hand gestures accurately. For instance, if the cameras are located far apart, there may be no position where the user’s hand would be visible in the frame for both the current camera and the one they wish to switch to. As a result, there may be better options than relying on a single-hand motion to switch between cameras.

To address this significant issue, we utilised camera sampling, which involves the algorithm sampling the cameras until it identifies the camera that provides the best view of the user’s hand (i.e., the camera with the lowest angle to the hand) when it is not visible in the current camera.

The sampling algorithm must run continuously until the hand returns to the current camera or is detected by another camera. In the event that none of the cameras detects the user’s hand, the algorithm continues to sample one additional camera every frame, resulting in a halving of performance. Nonetheless, this is a good performance trade-off, as the user cannot utilise the MotionInput feature without a visible hand and hence would not perceive any drop in frame rate. Therefore, the impact on performance is marginal.

The innovative solution that we have developed allows users to experience a seamless transition when moving from one camera or display to another, which makes it possible to work more efficiently and effectively. Specifically, users can now grab a window on one screen, move their hand out of sight of the current camera, and walk away to the next camera/display, all while the algorithm detects the hand and automatically switches to the new display. This breakthrough not only saves users time and effort but also increases their productivity and comfort as they can work more flexibly and without interruptions. Additionally, the solution gives users greater control over their work environment, allowing them to personalise their setup to suit their specific needs and preferences.