Working with 2 displays

Working with 2 displays

Once we added the code to track the angle of the hand to the camera, it was easy to make it work with two displays, as we used the existing code to switch displays and cameras. This was a proof of concept for our camera-switching algorithm. At this point, the user can only switch to the next display, hence only being able to use two displays. There was pre-existing code to switch to a different camera using its index; therefore, we were alternating between the first two cameras. The user switched displays when the angle of the triangle normal was greater than 70º.

It was simple to make it operate with two displays once we added the code to track the hand’s orientation with respect to the camera because we could utilise the existing code to swap between displays and cameras. This served as a demonstration of our camera-switching algorithm. Users can only use two screens now because they can only switch to the next display. We were switching between the first two cameras because there was already code in place to move to a different camera while using its index. The user switched displays when the triangle’s normal angle was greater than 70 degrees.

The user must also ensure their hand is pointed towards the camera first to switch to the next one; otherwise, the system gets caught in a loop of switching between cameras. For example, with a setup of two cameras, if the user starts the system with their hand at an angle greater than the threshold value for both cameras, the system will continuously attempt to switch between them. This behaviour can significantly decrease the performance of the system. Therefore, the algorithm must only switch cameras after the user’s hand has been below 30 degrees to the camera in use, at which point it is assumed to be flat.

At this stage, we noticed that switching to the next camera had a very noticeable delay. This was due to only one camera running at a time, and when a user tried to switch to a new camera, the other camera had to be started, which took a couple of seconds. This made the system slow and not user-friendly. We made all the cameras start when the program launched to fix this.

Here we demonstrate the implementation of changing back and forth between two displays touchlessly. When the user’s hand is detected to have turned to the 70º threshold value, the system loads the camera not currently in use and switches to it. This facilitates touchless control over the mouse for the user across both displays.