As described in our 'Research' page, our main technologies are
Xamarin, .NET MAUI and C#
Dependencies & Tools
We also imported some plugins & dependencies in our project to help
us test & build our project
Plugin.BLE
Given that the .NET MAUI framework does not offer in-built
bluetooth libraries, we had to rely and a community driven
plugin called Plugin.BLE which
provided us with a reilable way to communicate with
Bluetooth Low Energy (BLE) devices - in our case, sensors.
It gave us the ability to scan, discover and connect to BLE
devices whilst also being able to read their
characteristics. In our case, we used it to discover heart
and cadence sensors and read their information during a
workout.
CommunityToolkit.Maui
A plugin that offers a range of benefits to developing
cross-platform applications using .NET MAUI. It enables
greater control over handling data input, validation,
dialogs and animation. Furthermore, it also provides
platform-specific implementations and code which is helpful
to our project given the current scope of the app is for
Android devices only.
CommunityToolkit.Mvvm
Offers an implementation of the Model-View-ViewModel(MVVM)
architecture which helps separate concerns thus promoting
industry software engineering standards. It helps create
maintanable testable and scalable code whilst providing a
range of utilities for creating and binding to Viewmodel
classes which was primarily used in the sensor integration
development given several models were required to build a
functioning bluetooth system. With the plugin, it provides
more structure to an otherwise unstructured codebase.
Syncfusion
Enables the use of data visualization. The plugin offers a
range of chart types, in our instance we primarily use line
graphs to show how sensor data alternates throughout a
workout. The charts are customizable allowing us to adjust
the fonts, colors and markers.
Overview of the Process
We will explore how our accessibility-first fitness tracking app was
designed and developed to its completion. The process will be broken
down and explained in multiple segments for better understanding and
overall comprehension of the numerous aspects involved in the app’s
final design and implementation. During the development, to
guarantee smooth progression, we established a set of technical
objectives to complete, which implementation we will cover more
in-depth in the following sections.
System Architecture Diagram
Overview of the main process
Tools and Dependencies
Workouts
Workouts are the most important aspect of our app, they are core
structure which allows users to name their future exercises, record
their activity type, intent, date and much more. They are versatile,
yet stable and easy to use. They are represented as objects,
adhering to the structure of the Workout class, which classifies as
a Model in our system architecture diagram.
Workout class classification in the system architecture diagram
namespace Project_VeloCity.Models;
internal class Workout
{
public static List<string> ActivityTypes =
new()
{ "Basketball", "Table Tennis", "Swimming", "Cycling"};
public static List<string> TrainingIntents =
new()
{ "Indoor", "Outdoor"};
public string Filename { get; set; }
public DateTime Date { get; set; }
public string Name { get; set; }
public string ActivityType { get; set; }
public string TrainingIntent { get; set; }
The Workout class consists of two public static properties which are
lists of activity types and training intents users can choose from.
But the main reason for this implementation was to allow for
accessibility-first multi-option buttons users interact with when
creating or editing a workout. Next sections will describe specific
implementation of core actions with different types of workouts
within the app.
New workouts
Creating New Workouts For Fast (Re)use
Seamless creation of a new workout. The user just needs to set the
workout name. select activity type and training intent using two
universally-accessible multi-option buttons with Text-To-Speech
integration. In this section we will talk about how this is actually
implemented from the developer point of view. We had to define a
HomePage XAML file and its corresponding code behind, which upon
generating UI components allow the user to navigate to screen for
creating new workout.
namespace Project_VeloCity.Views;
public partial class HomePage : ContentPage
{
public HomePage()
{
InitializeComponent();
}
// REST OF THE CODE
private async void NewWorkout_Clicked(object sender, EventArgs e)
{
await Shell.Current.GoToAsync(nameof(EditWorkoutPage));
}
// REST OF THE CODE
}
Upon pressing the New Workout button, XAML will trigger the
corresponding method in the code behind, which will navigate to the
registered EditWorkoutPage. The aforementioned page has the form for
creating a new workout. Having a single page for both editing and
creating workouts leads to much less redundant code and greater
readability of the repository. The sequence diagram and its
corresponding wireframe describe the process background but also
foreground, respectively.
Sequence diagram for Creating New Workouts For Fast (Re)use
User-flow For Creating New Workouts
The ease-of-use lies in having only two pathways forward from this
point in navigation, either saving the workout for later use, or
following the stream-lined navigation pattern and starting the
workout, which will be automatically saved in the process.
Therefore, we have chosen to separately describe the two processes
starting with explaining the implementation of the first one, saving
new workout.
Accessibility-First Multi-Option Buttons
These are the aforementioned buttons users interact with when
creating or editing a workout. They allow for accessible choosing of
activity type and workout intent, allowing both individuals
blindness and visual impairments to use them with ease. Instead of
searching the list of options in the pop-up windows, we encapsulated
the process in a simple click of a button. Upon each click the next
activity type or training intent is chosen and presented to the
user. Memorising the number of clicks rather than the specific
position on the screen when creating workouts is something is
undoubtedly much easier and comprehensive for not only disabled but
also for non-disabled users. It has been implemented through
iteration of the static lists of workout types and training intents
from the Workout class, which we have shown in its corresponding
section.
Part of the Sequence diagram for Creating New Workouts For Fast
(Re)use corresponding to Activity type Multi-Option button
using Microsoft.Maui;
namespace Project_VeloCity.Views;
[QueryProperty(nameof(ItemId), nameof(ItemId))]
public partial class EditWorkoutPage : ContentPage
{
// REST OF THE CODE
private async void ActivityTypeButton_Clicked(object sender, EventArgs e)
{
if (BindingContext is Models.Workout workout)
{
if (workout.ActivityType == "Activity")
{
workout.ActivityType = Models.Workout.ActivityTypes[0];
}
else
{
workout.ActivityType =
Models.Workout.ActivityTypes[
(Models.Workout.ActivityTypes.IndexOf(
workout.ActivityType
) + 1) % Models.Workout.ActivityTypes.Count
];
}
ActivityButton.Text = workout.ActivityType;
await TextToSpeech.SpeakAsync(workout.ActivityType);
}
}
// REST OF THE CODE
}
Saving New Workout
In the case the user wants to setup the workouts prior to the
exercise, it can seamlessly saved them by a click of a button. What
happens in the background is once again triggering of a button, but
this time saving the file in the app data directory with extension
“.workout.txt”, giving us the ability to fetch all the workout files
and use them repeatedly when needed.
<?xml version="1.0" encoding="utf-8" ?>
<ContentPage
x:Class="Project_VeloCity.Views.EditWorkoutPage"
xmlns="http://schemas.microsoft.com/dotnet/2021/maui"
xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
Title="Setup Workout">
<!-- REST OF THE CODE -->
<Button
Grid.Column="0"
BackgroundColor="{StaticResource Primary}"
BorderWidth="2"
Clicked="SaveButton_Clicked"
CornerRadius="50"
FontSize="32"
Text="Save"
TextColor="White" />
<!-- REST OF THE CODE -->
</ContentPage>
using Microsoft.Maui;
namespace Project_VeloCity.Views;
[QueryProperty(nameof(ItemId), nameof(ItemId))]
public partial class EditWorkoutPage : ContentPage
{
// REST OF THE CODE
private async void SaveButton_Clicked(object sender, EventArgs e)
{
if (BindingContext is Models.Workout workout)
{
File.WriteAllText(
workout.Filename, $"{workout.Name},{workout.ActivityType},{workout.TrainingIntent}"
);
}
await Shell.Current.GoToAsync("..");
}
// REST OF THE CODE
}
Upon pressing the save button, the XAML triggers the associated
method for file system management. It checks whether the current
binding context is of the workout type and writes it into a file in
a CSV format to allow for future use and straightforward
deserialization. In order to create a new workout and save it in
such a simple way, we had to abstract the complexity to other
methods in the class. Therefore upon getting to the EditWorkoutPage
itself we had to first preload the empty workout and set the binding
context in the following way:
using Microsoft.Maui;
namespace Project_VeloCity.Views;
[QueryProperty(nameof(ItemId), nameof(ItemId))]
public partial class EditWorkoutPage : ContentPage
{
private string workoutFilename;
public EditWorkoutPage()
{
InitializeComponent();
string appDataPath = FileSystem.AppDataDirectory;
string randomFileName = $"{Path.GetRandomFileName()}.workout.txt";
LoadWorkout(Path.Combine(appDataPath, randomFileName));
}
// REST OF THE CODE
private void LoadWorkout(string fileName)
{
Models.Workout workout = new Models.Workout();
workout.Filename = fileName;
workoutFilename = fileName;
if (File.Exists(fileName))
{
string[] elements = File.ReadAllText(fileName).Split(',');
workout.Name = elements[0];
workout.ActivityType = elements[1];
workout.TrainingIntent = elements[2];
workout.Date = File.GetCreationTime(fileName);
}
if (workout.ActivityType == "" || workout.ActivityType == null)
{
workout.ActivityType = "Activity";
}
if (workout.TrainingIntent == "" || workout.TrainingIntent == null)
{
workout.TrainingIntent = "Intent";
}
BindingContext = workout;
}
// REST OF THE CODE
}
In order to better understand its overall architecture, we have
created the sequence diagram which describes the process with
respect to time. It is followed by the user’s point of view, where
the only form of interaction needed is a simple press of a button.
Sequence Diagram For Saving New Workouts
User-flow for Saving New Workout
Starting New Workout
Streamlined navigation design allowed for a single-path approach to
initiating workouts. User only needs to follow the path of bright
buttons and it will get the app into its desired state. The sequence
consists of confirming selected workout name, activity type and
training intent, before checking connected sensors and commencing to
the active workout screen itself, where the user begins the exercise
related to the chosen workout.
Sequence Diagram For Starting New Workout
What happens in the background is actually stack based approach,
where new pages are added on top of each other, making it easier to
revert back but also making the navigation stream-lined and
connected to the same root. We start from EditWorkoutPage XAML file
and its code behind, which upon the button trigger adds the
SensorConnectionPage on top of the stack. XAML file of newly added
page compiles and produces the UI elements visible to the user in
the form of state of the connections with the Bluetooth sensors.
Once that’s confirmed by the user, the next button gets initiated
and the ActiveWorkoutPage sets up the UI environment for finally
initiating the workout exercise. We will show how this is
implemented in order from the beginning of the sequence to the end.
Notable feature for our users is that by pressing next, the workout
gets automatically saved, eliminating the need for repetitive
user-flow patterns.
Commencing from EditWorkoutPage:
<?xml version="1.0" encoding="utf-8" ?>
<ContentPage
x:Class="Project_VeloCity.Views.EditWorkoutPage"
xmlns="http://schemas.microsoft.com/dotnet/2021/maui"
xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
Title="Setup Workout">
<!-- REST OF THE CODE -->
<Button
Grid.Column="1"
BackgroundColor="{StaticResource Secondary}"
BorderWidth="2"
Clicked="NextEditWorkout_Clicked"
CornerRadius="50"
FontAttributes="Bold"
FontSize="32"
Text="Next"
TextColor="{StaticResource Primary}" />
<!-- REST OF THE CODE -->
</ContentPage>
}
using Microsoft.Maui;
namespace Project_VeloCity.Views;
[QueryProperty(nameof(ItemId), nameof(ItemId))]
public partial class EditWorkoutPage : ContentPage
{
// REST OF THE CODE
private async void NextEditWorkout_Clicked(object sender, EventArgs e)
{
if (BindingContext is Models.Workout workout)
{
File.WriteAllText(
workout.Filename, $"{workout.Name},{workout.ActivityType},{workout.TrainingIntent}"
);
}
await Shell.Current.GoToAsync($"{nameof(SensorConnectionPage)}?{nameof(SensorConnectionPage.ItemId)}={workoutFilename}");
}
// REST OF THE CODE
}
Then, we need to define SensorConnectionPage, keeping in mind that
we are propagating the filename of the workout so that we can save
it for the insight creation, which happens upon finishing the actual
exercise. We will be doing it in the same way we did so far through
QueryProperty, getting and setting the ItemId to the workout
filename itself upon each screen transition.
<?xml version="1.0" encoding="utf-8" ?>
<ContentPage
x:Class="Project_VeloCity.Views.SensorConnectionPage"
xmlns="http://schemas.microsoft.com/dotnet/2021/maui"
xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
Title="Connect Sensors">
<!-- REST OF THE CODE -->
<Button
Grid.Row="4"
BackgroundColor="{StaticResource Secondary}"
BorderWidth="2"
Clicked="NextSensorButton_Clicked"
CornerRadius="50"
FontSize="45"
Text="Next"
TextColor="{StaticResource Primary}" />
<!-- REST OF THE CODE -->
</ContentPage>
namespace Project_VeloCity.Views;
[QueryProperty(nameof(ItemId), nameof(ItemId))]
public partial class SensorConnectionPage : ContentPage
{
private string workoutFilename;
public SensorConnectionPage()
{
InitializeComponent();
}
public string ItemId
{
set
{
// Loads value from the query
// which is filename of the workout
workoutFilename = value;
}
}
private async void NextSensorButton_Clicked(object sender, EventArgs e)
{
await Shell.Current.GoToAsync($"{nameof(ActiveWorkoutPage)}?{nameof(ActiveWorkoutPage.ItemId)}={workoutFilename}");
}
}
Now we will find ourselves at the ActiveWorkoutPage, which will be
explained in-depth in its own respective section, while we will now
shift our focus to checking and selecting saved workouts.
Saved Workouts
Viewing List Of Saved Workouts
The important aspect of the app was to allow users to see the list
of saved workouts, thereby eliminating the need to recreate workouts
from scratch each time they want to exercise. We implemented this
through a model-based approach. Having ObservableCollection of
workouts and LoadSavedWorkouts method in the SavedWorkouts class
allowed for seamless fetching of existing workouts and their
continuous updates in case of any deletions.
Saved wiorkouts User-flow
using System.Collections.ObjectModel;
namespace Project_VeloCity.Models;
internal class SavedWorkouts
{
public ObservableCollection<Workout> Workouts { get; set; } = new ObservableCollection<Workout>();
public SavedWorkouts()
{
LoadSavedWorkouts();
}
public void LoadSavedWorkouts()
{
Workouts.Clear();
string appDataPath = FileSystem.AppDataDirectory;
IEnumerable<Workout> workouts = Directory
.EnumerateFiles(appDataPath, "*.workout.txt")
.Select(filename => {
string[] elements = File.ReadAllText(filename).Split(',');
return new Workout()
{
Name = elements[0],
ActivityType = elements[1],
TrainingIntent = elements[2],
Date = File.GetCreationTime(filename),
Filename = filename,
};
}
)
.OrderBy(workout => workout.Date)
.Reverse().ToList();
foreach (var workout in workouts)
{
Workouts.Add(workout);
}
}
}
To effectively display the workouts on the screen we had to define
user-interface using SavedWorkoutsPage XAML file and its respective
code behind. We use SavedWorkouts as our binding context, allowing
for observable collection integration and constant updates. We had
to develop the design of the UI elements for both empty view and
item template, as well as smooth animation with brightness change,
especially useful for users with colour blindness.
In some cases, the user might have stopped exercising according to a
certain workout, therefore to mitigate the cluttering of the saved
workouts page with old workouts, we gave the ability to the user to
delete those unused with a simple click of a button. The
EditWorkoutPage XAML code triggers the method in its ode behind
which finds the file in the app data directory and deletes it,
removing it from the list of saved workouts as a result. The delete
button is in the form of a toolbar item in order to save screen
space, but also since it’s a less frequently encountered in a
typical use case scenario.
using Microsoft.Maui;
namespace Project_VeloCity.Views;
[QueryProperty(nameof(ItemId), nameof(ItemId))]
public partial class EditWorkoutPage : ContentPage
{
// REST OF THE CODE
private async void Delete_Workout(object sender, EventArgs e)
{
if (BindingContext is Models.Workout workout)
{
if (File.Exists(workout.Filename))
{
File.Delete(workout.Filename);
}
}
await Shell.Current.GoToAsync("..");
}
// REST OF THE CODE
}
Saved wiorkouts User-flow
Saved wiorkouts User-flow
Active workout
Stopwatch of workout time
Calculation of the workout time is implemented with TimeSpan struct
in the form of “hour : minute : second”. We update the workout data
and enable speaker within certain interval, providing immediate
status for users to adjust their workout intensity.
Display
The current workout start and update Heart Rate, Cadence values per
second. The speaker announce instant data values every 5 seconds by
using Listen().
The background of Heart Rate on UI will gradually change from low to
high opacity along with the increment of heart rate above 120bpm to
inform the users of Heart Rate intensity.
Border is a .NET MAUI container which can draw border, background
color for another control, e.g. Label control. Border control is
used with Heart Rate, Cadence Label control to clearly separate two
data and better alteration for Heart Rate background color.
Heart Rate & Cadence data
Both data are displayed on UI and store in insights class per second
by using AddHRInsight(),
AddCadenceInsight(). The Insight class store heart rate
and cadence in type List<int> which are referred…
Each workout information is stored as a string within a
corresponding file using File.WriteAllText(filename, infoString). To
view the current workout data, the “End” button is clicked and
navigate to InsightPage. The ViewModel class is used after to show
individual insight chart.
insight.Filename = Path.Combine(appDataPath, randomFileName);
string cadenceDataStr = string.Join(",", insight.CadenceData);
string heartRateDataStr = string.Join(",", insight.HeartRateData);
int hour = currTime.Hour;
int minute = currTime.Minute;
int second = currTime.Second;
File.WriteAllText(
insight.Filename,
$"{insight.WorkoutName}|{insight.WorkoutActivityType}|{insight.WorkoutTrainingIntent}|{hour},{minute},{second}|{heartRateDataStr}|{cadenceDataStr}");
ViewModel.insight = insight;
await Shell.Current.GoToAsync($"{nameof(InsightPage)}?{nameof(InsightPage.ItemId)}={insight.Filename}");
View Insight
We choose to display both data in line charts as users can view the
dynamic of them. The charts are drawn by using MAUI package
Syncfusion. After passing the current insight to ViewModel, the
ViewModel will construct two line charts for Heart Rate and Cadence
data. SensorData objects are created to bind x-axis(second) and
y-axis(sensor value).
public class ViewModel
{
public void SetHeartChart()
{
int s = 0;
foreach (int hr in insight.HeartRateData)
{
HeartRate.Add(new SensorData { SensorAtTime = hr, Time = s.ToString() });
s += 1;
}
}
}
public class SensorData
{
public int SensorAtTime { get; set; }
public string Time { get; set; }
}
We only calculate data insights (average, minimum, maximum) on
InsightPage.xmal.cs after initialize insight page. Also, a part,
Heart Rate overview, provides the intensity level of current workout
by using the definition of heart rate zo
The InsightPage is implemented using ScrollView in .Net Maui.
Average, minimum & maximum, charts are showed in separate rows with
two Grid view for Heart Rate and Cadence. RowDefinition Height
differ in both Grid view as different distance is needed to display
data properly. “Done” button on the end of the ScrollView and “DEL”
button on the top right will finish viewing or delete current
insight, and navigate back to Home page.
Insights are the very important aspect of our app, they are core
structure which allows users to save the data connected to their
finished exercises and get the useful feedback around it. They
include all of the information connected to the performance of the
user, allowing feedback driven approach towards accomplishing the
desired fitness goals. They are represented as objects, adhering to
the structure of the Insight class, which classifies as a Model in
our system architecture diagram.
Insight class classification in the system architecture diagram
namespace Project_VeloCity.Models;
public class Insight
{
public List<int> HeartRateData;
public List<int> CadenceData;
public Insight()
{
HeartRateData = new();
CadenceData = new();
}
public Insight(string fileName)
{
HeartRateData = new();
CadenceData = new();
Filename = fileName;
if (File.Exists(Filename))
{
string[] elements = File.ReadAllText(Filename).Split('|');
WorkoutName = elements[0];
WorkoutActivityType = elements[1];
WorkoutTrainingIntent = elements[2];
List<int> timeLst = elements[3].Split(',').Select(int.Parse).ToList();
Duration = new(timeLst[0], timeLst[1], timeLst[2]);
HeartRateData = elements[4].Split(',').Select(int.Parse).ToList();
CadenceData = elements[5].Split(',').Select(int.Parse).ToList();
Date = File.GetCreationTime(Filename);
}
}
public string Filename { get; set; }
public DateTime Date { get; set; }
public string WorkoutName { get; set; }
public string WorkoutActivityType { get; set; }
public string WorkoutTrainingIntent { get; set; }
public TimeOnly Duration { get; set; }
}
Reviewing Workout Performance History
Upon choosing the insights tab, user can review the average data
obtained from all previous workout sessions. In addition to
scrolling through all of them, the user can check their specifics
accordingly. For this, we created the SavedInsightsPage XAML file
which contains the collection of all the exercises and their total
average. This is done through SavedInsightsPage code behind which
calls the respective methods from the SavedInsights class and
changes the data for the XAML file as needed.
Insights User-Flow
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;
namespace Project_VeloCity.Models;
internal class SavedInsights
{
public ObservableCollection<Insight> Insights { get; set; } = new ObservableCollection<Insight>();
public SavedInsights()
{
LoadSavedInsights();
}
public List<string> LoadSavedInsights()
{
Insights.Clear();
string appDataPath = FileSystem.AppDataDirectory;
IEnumerable<Insight> insights = Directory
.EnumerateFiles(appDataPath, "*.insight.txt")
.Select(filename => {
return new Insight(filename);
}
)
.OrderBy(insight => insight.Date)
.Reverse().ToList();
foreach (var insight in insights)
{
Insights.Add(insight);
}
return LoadAverages();
}
public List<string> LoadAverages()
{
List<string> lst = new List<string>();
lst.Add("0");
lst.Add("0");
lst.Add("0");
if (Insights.Count > 0)
{
Insight insight = Insights[0];
double averageHeartRate = getAverageSensor(insight.HeartRateData);
double averageCadence = getAverageSensor(insight.CadenceData);
long averageDuration = insight.Duration.Ticks;
for (int i = 1; i < Insights.Count; i++)
{
insight = Insights[i];
averageHeartRate = (averageHeartRate + getAverageSensor(insight.HeartRateData)) / 2;
averageCadence = (averageCadence + getAverageSensor(insight.CadenceData)) / 2;
averageDuration = (averageDuration + insight.Duration.Ticks) / 2;
}
lst[0] = ($"{(long)averageHeartRate}");
lst[1] = ($"{(long)averageCadence}");
lst[2] = ($"{(long)averageDuration}");
}
return lst;
}
private double getAverageSensor(List<int> data)
{
return data.Count > 0 ? data.Average() : 0.0;
}
}
using Project_VeloCity.Models;
namespace Project_VeloCity.Views;
public partial class SavedInsightsPage : ContentPage
{
public SavedInsightsPage()
{
InitializeComponent();
BindingContext = new SavedInsights();
}
protected override void OnAppearing()
{
List<string> sensorsAndDuration = ((SavedInsights)BindingContext).LoadSavedInsights();
HeartRateLabel.Text = sensorsAndDuration[0];
CadenceLabel.Text = sensorsAndDuration[1];
TimeOnly currTime = new TimeOnly(long.Parse(sensorsAndDuration[2]));
DurationLabel.Text = $"{currTime.Hour:00}:{currTime.Minute:00}:{currTime.Second:00}";
}
private async void SavedInsightsCollection_SelectionChanged(object sender, SelectionChangedEventArgs e)
{
if (e.CurrentSelection.Count != 0)
{
var insight = (Insight)e.CurrentSelection[0];
ViewModel.insight = insight;
await Shell.Current.GoToAsync($"{nameof(InsightPage)}?{nameof(InsightPage.ItemId)}={insight.Filename}");
savedInsightsCollection.SelectedItem = null;
}
}
}
Text-To-Speech
Ramon will ad
Text-To-Speech classification in the system architecture diagram
Speech Recognition
Speech Recognition classification in the system architecture
diagram
This part was especially useful for implementing the virtual coach.
Thanks to the team at .NET MAUI, especially the Senior Software
Engineer for its development Gerald Versluis, we were able to adapt
the potential future plugin for the use case of our app. It uses
platform specific integration, the android speech recognition
service. The implementation consists of a Listener which implements
the IRecognitionListener and ISpeechToText interface which
encapsulates the main functionalities and provides a future-proof
design for implementations in other platforms.
The android specific implementation, which we used in our case
implements these methods and using the SpeechRecognitionListener
methods gets the recognised text
public class SpeechToTextImplementation : ISpeechToText
{
private SpeechRecognitionListener listener;
private SpeechRecognizer speechRecognizer;
public async Task<string> Listen(CultureInfo culture, IProgress<string> recognitionResult, CancellationToken cancellationToken)
{
var taskResult = new TaskCompletionSource<string>();
listener = new SpeechRecognitionListener
{
Error = ex => taskResult.TrySetException(new Exception("Failure in speech engine - " + ex)),
PartialResults = sentence =>
{
recognitionResult?.Report(sentence);
},
Results = sentence => taskResult.TrySetResult(sentence)
};
// ... REST OF THE CODE
}
Virtual Coach (STT-TTS Integration)
Makes use of both Speech Recognition (Speech-To-Text) and
Text-To-Speech to achieve seamless two-way communication between the
user and the virtual coach. Imporant feature of it, is that it’s
adaptable and future-proof. Simple interface for adding more
instructions and functionalities will allow the app to not degrade,
but improve over time using the feedback and wishes from the users.
It’s implemented by combining the two APIs during active workouts.
Virtual Coach classification in the system architecture diagram
It has been implemented by repeatedly initiating the listening
operation every 5 seconds, therefore producing the effect of
continuous listening without imposing the need for additional
memory. Being able to periodically check for the input from the
user, without the unnecessary consumption of large amount of data is
something that really makes this implementation powerful for the
specific requirements of the fitness tracking application like ours.
Sequence Diagram of Virtual Coach Feature in Active Workout
using Project_VeloCity.Models;
using Project_VeloCity.ViewModels;
using System.Globalization;
namespace Project_VeloCity.Views;
[QueryProperty(nameof(ItemId), nameof(ItemId))]
public partial class ActiveWorkoutPage : ContentPage
{
// REST PF THE CODE
private async void Listen()
{
var isAuthorized = await speechToText.RequestPermissions();
if (isAuthorized)
{
try
{
RecognitionText = await speechToText.Listen(CultureInfo.GetCultureInfo("en-us"),
new Progress<string>(partialText =>
{
RecognitionText = partialText;
OnPropertyChanged(nameof(RecognitionText));
}), tokenSource.Token);
// checking recognized text
CheckForKeyword(RecognitionText);
}
// ... REST OF THE CODE
}
// REST OF THE CODE
private async void CheckForKeyword(string sentenceToCheck)
{
if (sentenceToCheck.Contains("heart rate"))
{
await TextToSpeech.SpeakAsync($"It is {HeartRateLabel.Text} beats per minute.");
}
else if (sentenceToCheck.Contains("cadence"))
{
await TextToSpeech.SpeakAsync($"It is {CadenceLabel.Text} rpm.");
}
else if (sentenceToCheck.Contains("information"))
{
await TextToSpeech.SpeakAsync($"{HeartRateLabel.Text} beats per minute with {CadenceLabel.Text} rpm.");
}
else if (sentenceToCheck.Contains("motivation"))
{
await TextToSpeech.SpeakAsync($"You are doing great, keep it up.");
}
}
// REST OF THE CODE
Bluetooth Implementation
Tools and Dependencies
Algorithm walkthrough
Checking and Requesting Permissions
In order to scan, discover and connect to BLE devices, one must ask
for bluetooth & location permissions from the target device.
PermissionsStatus is a Permissions class that allows the app to
check and request permissions at run time, it is required for the
functioning of this app.
public async Task<PermissionStatus> RequestBluetoothPermissions()
{
PermissionStatus status = PermissionStatus.Unknown;
try
{
status = await Permissions.RequestAsync<BluetoothPermissions>();
}
#if permission is denied an exception is caught
catch (Exception ex)
{
Console.WriteLine($"Unable to request Bluetooth LE permissions: {ex.Message}.");
await Shell.Current.DisplayAlert($"Unable to request Bluetooth LE permissions", $"{ex.Message}.", "OK");
}
return status
Check whether bluetooth permissions have been granted
PermissionStatus permissionStatus = await BluetoothLEService.CheckBluetoothPermissions();
if(permissionStatus != PermissionStatus.Granted)
{
permissionStatus = await BluetoothLEService.RequestBluetoothPermissions();
if(permissionStatus != PermissionStatus.Granted)
{
await Shell.Current.DisplayAlert($"Bluetooth LE permissions", $"Bluetooth LE permissions are not granted.", "OK");
return;
}
}
Check whether bluetooth is one & available on the device
if (!BluetoothLEService.BluetoothLE.IsAvailable || BluetoothLEService.BluetoothLE.State != BluetoothState.On)
{
Console.WriteLine($"Bluetooth is missing.");
await Shell.Current.DisplayAlert($"Bluetooth", $"Bluetooth is missing.", "OK");
return;
}
try
{
if (!BluetoothLEService.BluetoothLE.IsOn)
{
await Shell.Current.DisplayAlert($"Bluetooth is not on", $"Please turn Bluetooth on and try again.", "OK");
return;
}
Starting scan
Once the permissions have been accepted, the app can start
scanning deviceCandidates represents the list of devices that have
been discovered by the
BluetoothLEService.ScanForDevicesAsync()
If sensors/BLE devices are found the deviceCandidate list will be
represented here as a grid of devices with each row being each
devices name and Guid (global unique identifier)
Once a sensor has been selected it will navigate to its
correspondoning page (Heart/Sensor) and you will be given the
ability to connect/disconnect (see above)
Connectig to a device
The adapter will connect to a device given a known global user
identifier (Guid) which is extracted from
BluetoothLSEService.NewDeviceCandidateFromHomePage.Id
Once a connection has been made, the system can extract specific
characterisitcs from the BLE device given general identifiers. Below
are our full list of Guids stored in class HeartRateIdentifiers
class HeartRateIdentifiers
{
// primary service for heart rate 8CE5CC02-0A4D-11E9-AB14-D663BD873D93"
//public static Guid HeartRateService = Guid.ParseExact("0000180d-0000-1000-8000-00805f9b34fb", "d");
public static Guid[] HeartRateServiceUuids { get; private set; } = new Guid[] { new Guid("0000180d-0000-1000-8000-00805f9b34fb") }; //"Heart Rate Service"
public static Guid HeartRateServiceUuid { get; private set; } = new Guid("0000180d-0000-1000-8000-00805f9b34fb"); //"Heart Rate Service"
public static Guid HeartRateMeasurementCharacteristicUuid { get; private set; } = new Guid("00002a37-0000-1000-8000-00805f9b34fb"); //"Heart Rate Measurement"
public static Guid BatteryLevelServiceUuid { get; private set; } = new Guid("0000180f-0000-1000-8000-00805f9b34fb"); //"Battery Service"
public static Guid BatteryLevelCharacteristicUuid { get; private set; } = new Guid("00002a19-0000-1000-8000-00805f9b34fb"); //"Battery Level"
}
Heart rate information being extracted from heart sensor with
provided characteristic Guids
Given the heart rate updates throughout the workout, system must
continously extract heart rate information from the sensor as long
as there is a stable connection
if (HeartRateMeasurementCharacteristic.CanUpdate)
{
Title = $"{BluetoothLSEService.Device.Name}";
#region save device id to storage
await SecureStorage.Default.SetAsync("device_name", $"{BluetoothLSEService.Device.Name}");
await SecureStorage.Default.SetAsync("device_id", $"{BluetoothLSEService.Device.Id}");
#endregion save device id to storage
HeartRateMeasurementCharacteristic.ValueUpdated += HeartRateMeasurementCharacteristic_ValueUpdated;
await HeartRateMeasurementCharacteristic.StartUpdatesAsync();
}
.png)
.png)
.png)
.png)
.png)
.png)
.jpg)
.png)
