• Popularity and Usage

  The reason usage levels and popularity was important to us is because, as more developers use Javascript, there will be a plethora of learning resources available online. These resources are pivotal to the execution of our front-end.
  
  Furthermore, there will be a bigger community for Javascript, where community answers to development related questions will be available. This means, by choosing Javascript, there is a higher likelihood to encounter similar issues that have been previously be posted online. This is very helpful and saves times when debugging. [1]
  
  

  

• Performance

  In Teams toolkit, frontend libraries are prominent. With JavaScript development being formatted in React and the Typescript in Angular. We have to compared the performance of each of these frameworks to established which one will provide the best experience within Microsoft Teams. [2]
  
  

  Memory

  

  The lower memory usage in React framework is attractive. One reason is because part of our target demographic are elderly patients who may have join Teams meeting on an old machine or mobile phone. Less memory usage will help these patients enjoy using our application.
  
  

  Speed

  

  With our NDI-IN solution being now blocked for a while, live collaboration had not been possible for the last few weeks. However, we can now use LiveShareSDK for Teams, which uses Microsoft's fluid service, connected to a network, to perform state sharing between participants in a Teams meeting.
  
  When sharing state, usually through JSON, the duration of standard data manipulation must be quick, to ensure latency and speed does not hinder the collaborative game experience.
  
  

• Analysis

  From data evidence gathered, it is meticulous that using Javascript will give us the best support in development, in getting started and in debugging stages. The availability of previous projects will give us examples, and will make it easier to find open-source games to integrate into our app and render over Microsoft Teams. The initial memory usage in React is extremely useful for app users.
  
  Despite Angular framework being slightly quicker in data formatting and manipulation, React library in Javascript has more important advantages as previously explained, as well as the fact that the ONLY LiveShare SDK example code uses Javascript. LiveShare SDK (Software Development Kit) for Teams, will be a great additional feature. This is a new feature released by Microsoft and operates by managing Fluid state. Similar SDK’s in Angular do not exist.
  
  

• PSI and Microsoft Graph

  PSI (Platform for Situated Intelligence), is an open and flexible framework developed by Microsoft to support the development of solutions that process and stream data such as audio and video. On the other hand, Microsoft Graph is a developer platform that integrates multiple Microsoft services and devices, empowering developers to create apps. [3]
  
  As a potential solution, we explored the idea of using Microsoft Graph to connect to a Teams tenant and the calls made through that tenant. By doing so, we could access live video and audio feeds that could be managed using the PSI framework. Our team looked into a solution where we combine these two with an Azure bot that can be summoned back into a Teams call. This would allow us to create live video feeds by merging member video streams and games, which can then be called back into a Teams call with Azure to display them in the call itself. [4]

• NDI

  NDI, or Network Device Interface, is a protocol developed by NewTek that allows for the transmission of multiple high-quality video and audio feeds over a network. By utilizing this technology, we can capture and stream the audio and video feeds in real-time from a Teams call. To explore its potential, we considered two possible paths. The first option was to integrate NDI with OBS (Open Broadcaster Software) to create a customized build. This would involve taking video and audio feeds from a Teams call and using them to create a graphics overlay that includes games such as ping pong, which can be overlaid over the feeds. We would then use NDI-In technology to relay this overlay back into the Teams call. [5]
  
  Alternatively, we also explored building an NDI solution directly into Python. By integrating NDI receivers and finders into the MotionInput codebase, we could replace the standard desktop webcam as a video source for the motion software and instead use a selected video stream broadcasted over NDI technology via the Teams call. [6]

• Analysis

  In conclusion, we explored two potential solutions for our remote solution – the PSI Bot and Microsoft Graph solution and the NDI solution. While the PSI Bot and Microsoft Graph solution seemed promising, it would have involved a high level of complexity and numerous dependencies on Microsoft Graph and Azure, making it difficult for patients and physiotherapists to use, as well as institutions or hospitals trying to set this up at a large scale. This would have gone against our goal of providing a user-friendly and enjoyable experience for all.
  
  On the other hand, the NDI solution showed a lot of potential, particularly in integrating it with custom OBS overlays to create visually appealing graphics in the Teams call. However, this solution was halted due to the current blocking of NDI-In software by Microsoft.
  
  As a result, we went ahead with the NDI solution built directly into MotionInput, which still allowed remote MotionInput control from any device, anywhere, providing a practical solution for our needs. While the blocking of the NDI-In software was a setback, we remain optimistic about the current technologies we have in-place.

Data Collection

In developing our remote physiotherapy solution, we recognized the importance of accurately tracking the movements and positions of patients in real-time, and providing personalized feedback on their progress.

After researching various technologies and tools, we identified Mediapipe as a promising option for pose estimation, which could help us accurately track patients' body positions and movements during remote sessions. To visualize and analyze the data collected through Mediapipe, we decided to use scatter plots.