System
Research

Outlining the necessary research processes taken to gain a better understanding of how the system is to be implemented as well as beginning to gather basic information on how the overall system will work together.

Research Overview

Our project was to develop a virtual assistant service for elderly users as part of IBM FISE v2 Concierge. This service will be available in the form of an app which utilises speech as its primary mode of interaction and should help users in their daily lives. The app should support a range of features, implemented using RESTFUL APIs.

Alongside the app, we would also need an Admin website which will be used by the elderly user's loved ones. The website is used to set up an elderly user’s app, enable access to specific features of the app and view logs containing details about the elderly user’s interactions with the app.

Finally, we would also need a backend to support both the app and the website. This backend would be responsible for data storage.

To summarise, we spent time researching the following areas:

Existing Solutions

Part of our research involved finding products similar to ours to see if we could learn anything from them that may aid us in development. We found two similar products, Dalia and Alfred.

Dalia

Dalia is a virtual assistant specifically for the elderly serving a remarkably similar purpose as our intended solution. However, it seems development has ceased (no updates to their app or website since 2016) and it appears to focus on the (tracking of) physical health of its users. We aim to provide a more general  solution.

Alfred

Like Dalia, Alfred is another virtual assistant for the elderly. However, it again seems to be focused on the physical health of the user, mainly trying to get them to be more active through games and exercises. Additionally, it also seems that development has  stopped, and the project is inactive at the time of writing.

Unfortunately, we were not able to learn much from these products due to both of them currently being inactive and incomplete.

App Frontend

Before we could begin production on the app, we would have to evaluate what platforms we would like to develop the app for and what framework and language we would utilise. There are two leading mobile operating systems we had to consider, iOS (Apple) and Android (Google). As such, we researched and considered three possible frameworks.

NativeScript

NativeScript is a relatively new framework, released in 2014, for the development of apps for iOS and Android. NativeScript apps are developed using JavaScript or any other programming language that translates to JavaScript (e.g. Vue and Angular).

NativeScript would provide the following advantages if we were to develop with it:

  • Can create apps for both mobile operating systems from the same code base - reduced development time
  • Built-in speech recognition and speech synthesis (which can support multiple languages)
  • Familiarity as our team already has experience with Vue

However, there were also fundamental disadvantages with NativeScript:

  • Development environment dependent on Xcode which is only available on MacBook (if you want to develop for iOS)
  • Overshadowed by React Native so less available resources should we run into difficulties

Java

Java is an object-oriented programming language. It can be used to develop apps for Googles Android OS. Android Studio is the official IDE created by Google for developing apps for Android and released in 2014 as a replacement to the previous Android Development tool. It is designed specifically for the creation of Android apps and provides many useful tools to aid in development.

Android Studio would provide the following advantages if we were to develop with it:

  • Built-in speech recognition and synthesis (which can support multiple languages)
  • Specifically designed for Android so greater control over the device and its built-in features
  • Familiarity as our team has extensive experience in Java
  • Many built-in developments tools
  • A large amount of extensive and detailed documentation

However, there were also disadvantages with Java & Android Studio:

  • We would develop only for Android as there would not be sufficient time to develop for iOS as well

Swift

Swift is a general-purpose compiled programming language developed by Apple and released in 2014. It was designed to replace Objective C, previously been used by Apple for all its products. It is included in Xcode - Apple's official IDE for developing apps for all their platforms but in this case, we are interested in its uses for iOS development.

Swift would provide the following advantages if we were to develop with it:

  • Designed for development on Apple's operating system so greater control over the device and its built-in features
  • Built-in speech recognition and speech synthesis (which can support multiple languages)
  • A large amount of extensive and detailed documentation
  • Supports most Java libraries

However, there were also fundamental disadvantages with Swift:

  • iOS applications can only be developed on macOS
  • We would develop only for iOS as there would be insufficient time to develop for Android as well

For this project, we selected to utilise Java as our development language and therefore develop only for Android. While we initially considered NativeScript, its dependency on Xcode for its development environment eliminated it as not all of us utilise MacBooks for development those without would be unable to run their code. We also asked the client on their opinions about this decision who did not object to it.

Admin Website Frontend

As mentioned above the Admin website will be used by elderly users loved ones to register users, provide selective access to the apps features and view activity logs.

To create this website, we considered two possible frameworks: Vue.js and React. The reason we focused on these two frameworks was that they both utilise the concept of the Virtual DOM (Document Object Model). The Virtual DOM is an abstraction of the HTML DOM which is a tree structure of nodes which are HTML elements. When creating interactive websites with the standard HTML DOM, this would result in having to re-render the entire page when an HTML element changes. The advantage of utilising a Virtual DOM is that only the objects that change would need to be re-rendered; this saves time and (processing) resources and is especially useful on more interactive websites (such as ours) which need to update frequently.

Vue.js

Vue.js is a JavaScript framework designed for building user interfaces. It was created in 2014 and aimed to extract parts of AngularJS to make a more lightweight framework. It is available to use and develop with under an open-source license.

Were we to develop the admin website using Vue.js it would provide the following benefits:

  • Familiarity as some of us already have prior experience with Vue
  • Easy to develop with due to its clear structure
  • Good and concise documentation

However, there were also disadvantages with Vue:

  • Fewer plugins and available packages than React
  • The Vue.js community is much smaller than that of React meaning there are fewer resources and help available

React

React is a JavaScript framework also designed for building user interfaces. Created by Facebook to serve their specific needs, it is also available to everyone under an open-source license.

Were we to develop the admin website using React it would provide the following benefits:

  • React is backed by Facebook and hence, will have long-term support
  • High popularity meaning there is an abundance of resources for us to use
  • Lots of available packages to use with it

However, there were also disadvantages with React:

  • Harder to learn than Vue - increased overall development time
  • Subpar documentation meaning it is more difficult to find help when needed

For this component of the project, we elected to utilise Vue.js. The main reasoning behind this was that it was easy to learn and develop with while still being able to produce high-quality results. We valued this highly as our project has many components that need to be built and given our strict time constraints, a faster development period would ensure our ability to implement all system components.

System Backend

We also require a backend to support the app and admin website. This backend will be responsible for storing and manipulating data generated by the users and relaying back where appropriate.

The backend will also be responsible for placing appropriate API calls from the application and then relaying the relevant information. For our backend, we considered two potential languages to code with Python and Java.

These are the languages we considered as they are the most popular backend development languages, and, we all had prior experience with them.

Python

Python is an interpreted general-purpose programming language. Released in 1991 and continuously developed since Python is supported by a wide range of devices. Python has a very straight forward and easy to understand syntax that can be very flexible and used for a variety of purposes.

If we were to use Python to develop our backend, it would offer the following advantages:

  • Simple syntax so it would be straightforward and quick to use
  • High popularity meaning there are plenty of resources online for us to utilise
  • Has access to a large collection of libraries
  • Familiarity as our team already has experience with Python

However, there were also fundamental disadvantages with Python:

  • Python is a lot slower when compared to a lot of languages such as C, even Java
  • Python can have high memory consumption depending on the task

Java

Java is a compiled object-oriented programming language. Released in 1995 and, like Python, has been continuously developed since. Java can run on most machines due to its use of the JVM, giving it superior platform independence. Java syntax is not as simple as Python's - this is a non-issue as we all have prior experience with Java.

If we were to use Java to develop our backend, it would provide the following advantages:

  • A large amount of code reusability due to OOP approach which would speed up development
  • Faster than Python due to its compiled nature
  • Extensive collection of libraries
  • Familiarity as our team already has experience with Java

However, there were also disadvantages with Java :

  • More syntactically verbose than Python - increased development time
  • Slow compared to other languages such as C

For the backend of our project, we decided to utilise Java. This is because of the two options we believe that the advantages Java offered outweighed those offered by Python. For starters, speed is important as long load times can have a large negative affect on the usability of both our app and our admin website. Secondly, we are all familiar with Java and as such our rate production of code is unlikely to be greatly affected even though Python is the syntactically easier language to use. Finally, in this context we value the decreased memory consumption of Java as the backend is intended to be run locally and as such it should be resource light so that it can be more accessible.

Voice Assistant

One of the highlights of our system will be the voice assistant within the Concierge App. This voice assistant will utilise natural language processing to decode a user’s command such that users will have an increased amount of freedom of expression.

When it comes to natural language processing, Python is the go-to language due to its expensive array of libraries, especially with regards to machine learning. For our NLP library of choice, we considered two options: Spacy and RASA.

Spacy

Spacy is an advanced natural language processing library focused on creating NLP algorithms which are for production usage. Spacy utilises convolutional neural network models that allow it to perform speech tagging, tokenizing and named entity recognition

If we were to use Spacy to develop our voice assistant, it would offer the following advantages:

  • Speed. Spacy is faster than Rasa
  • Less rigid than Rasa, making it easier to mold to a specific use case
  • Extensive amounts of documentation

However, there were also certain disadvantages with using Spacy:

  • It can be difficult and time consuming to write training data for Spacy
  • Is more complex to work with than Rasa

Rasa

Rasa is a natural language processing framework designed specifically for building contextual voice assistants. Rasa builds upon Spacy and abstracts some of the complexities of machine learning. It provides many of the same features as Spacy such as name entity recognition or tokenizing.

If we were to use Rasa to develop our backend, it would provide the following advantages:

  • Easier to use compared to Spacy
  • Faster and easier to write training data for Rasa
  • Extensive documentation
  • Has built in features that allow it to be exposed as an API

However, there are also disadvantages with Rasa:

  • Slower than Spacy
  • Can be less flexible than Spacy

For our app's voice assistant, we elected to utilise Spacy. The main factor behind this decision was speed. As mentioned above speed plays an important part in the system as it can drastically impact usability and lower the standard of a users experience. While speed was the main factor considered in this decision, Spacy's increased flexibility was also considered as it would give us greater customisation over the voice assistant and its capabilities.

Services

Below is the full index of services we plan to include in the app, supported with natural language processing.

References

AAL Programme. 2021. DALIA - AAL Programme. [online] Available at: [Accessed 29 March 2021].

Alfred.eu. 2021. ALFRED | Interactive Assistant for Independent Living and Active Ageing. [online] Available at: [Accessed 29 March 2021].

NativeScript.org. 2021. Native mobile apps with Angular, Vue.js, TypeScript, JavaScript - NativeScript. [online] Available at: [Accessed 29 March 2021].

Android Studio Docs. 2021. [online] Available at: [Accessed 29 March 2021].

Developer.apple.com. 2021. Swift - Apple Developer. [online] Available at: [Accessed 29 March 2021].

Vuejs.org. 2021. Vue.js. [online] Available at: [Accessed 29 March 2021].

Reactjs.org. 2021. React – A JavaScript library for building user interfaces. [online] Available at: [Accessed 29 March 2021].

Spacy. 2021. [online] Available at: [Accessed 29 March 2021].

Rasa. 2021. Open source conversational AI. [online] Available at: [Accessed 29 March 2021].