John McNamara

IBM Master Inventor, Honorary Professor and IBM UK University Programs Lead

Dr Dean Mohammedally

Associate Professor in Software Engineering and Industry Projects at UCL

Project Background

The IBM FISE portfolio of projects strives to reduce loneliness in the elderly. Due to current circumstances, and the application of self-isolation and social distancing guidelines, this issue has been worsened. For many, technology offered a way to overcome the challenges imposed by these guidelines. Services such as online shopping or video calls were able to replace in-person activities. However, for some, these services were confusing to access and to utilise. In particular, the elderly were found to struggle the most.

The IBM FISE v2 Concierge aims to solve this by providing an easy to use, speech-controlled virtual assistant. This virtual assistant will be made available through an app that users will be able to download. The virtual assistant will offer a range of functionality such as the ability to place phone calls to loved ones or more general services such as checking the weather.

Gathering Requirements

To better understand our clients' expectations, we arranged a meeting with them at the beginning of our project. During this meeting, we asked them a series of questions so that we could gain an understanding of what their requirements for the project were. Their responses would help to generate our MoSCoW list. Below are some of the questions (and answers) asked to our two clients.

What is the overall aim of the system?

To create a concierge service that makes it easier to access services from home.

How would you describe the system?

It is an easy to use system that would be similar to having a butler.

Who is the target audience for the system?

It should be targeted towards the elderly and less computer literate individuals.

What is the motivation behind the system?

To help the elderly and less computer literate individuals deal with the problems of social isolation.

Compared to existing systems, how will this system be different?

Our system will be simpler and have more focused functionality.

What platforms will the system be available on?

Mobile Devices

Are there any other systems our system will need to interact with?

The organisation will be able to implement their services into our system via APIs.

Are there any data security requirements for the system?

Yes. We would like it if speech processing was done locally.

User Interviews

Once we had a better idea of the needs of our clients, we conducted interviews with potential end-users to see what they would expect from our system. We elected to give our potential elderly users a questionnaire as this would allow them to answer questions at their own pace. For the admin users, we elected to use semi-structured interviews to gather requirements as each user would have different needs, and we would need to be able to probe appropriately. Below are samples from a questionnaire with an elderly user (left) and an interview with potential admin users (right).

What is your date of birth?

22/10/1960.

How do you usually access the internet?

Using a computer.

On average, how much time do you spend on the internet every day?

Less than 1 hour a day.

Which of these online services do you frequently use: online shopping, online banking, checking the weather, other?

Online Banking, Online Shopping.

Is there anything about these services that makes them difficult to use?

The interfaces are too complicated. They don't guide me through the website.

Do you communicate with others online?

Yes.

Have you encountered any problems or difficulties when trying to communicate with others online?

I can't fully work FaceTime so, if people want to talk to me, they have to call me first.

Have you ever used a digital assistant before?

No.

Are there any people over the age of 60 in your family?​

Yes, my father and mother are both over 60.

Do they use any online services? If so, which ones?

Yes - they both regularly shop online using Amazon and, they also utilise online banking.

Have they ever had difficulties using these?

Yes, I occasionally get a call from them asking for help.

Do you think they would find a virtual assistant, such as Alexa, useful?

I think they would find it useful, but some of Alexa's features can be a bit complicated which they could struggle using.

What are your opinions on a virtual assistant service designed for your parents which you could manage remotely?

I think it sounds like a good idea. It would make both our lives easier. They could enjoy the benefits of a virtual assistant and, if they run into issues, it would be much easier for me to help them.

What features would you like when it comes to managing your parents' virtual assistants?

I would like to be able to see how they interact with the virtual assistant, so some sort of log would be useful. I would also like to be able to check in on them directly through the system in the case of an emergency.

Personas & Scenarios

Personas and scenarios allow us to think more carefully about how and why users will use our system.

Elderly User

Click to enlarge Elderly user persona and scenario diagram Elderly user persona and scenario diagram

Bob is a retired pensioner who has little technical knowledge or experience. He prefers to avoid using technology where possible but has recently had to use it more due to the effects of the pandemic. He is aware of speech recognition but has little to no experience using it. However, knowing it'll be easier to conduct tasks using his voice, Bob is open to trying it out. He, therefore, opens the app to do something he does a lot: check his bank balance; he opens the app and clicks on the button to activate speech recognition. He then says his command, "what's my bank balance?" which is transcribed to the screen. The app (locally) processes the request and shows him his bank balance. Bob was therefore easily able to do something technical with minimal effort making it easier to conduct digital tasks.​ ​

Admin User

Click to enlarge Admin user persona and scenario diagram Admin user persona and scenario diagram

Tim is a teacher who has an average amount of technical knowledge and can easily navigate and use online services. However, his parents struggle using technology, often needing his help. While Tim is happy to help, his time is usually limited and, he finds troubleshooting over the phone difficult. Tim suggests to his parents to get the Concierge app. He can quickly set up accounts for his parents with himself as their admin; he finds it easy to guide them through the process of setting up the app; he enables some services for his parents to use. After a few days, his parents ask Tim if they can have access to the food delivery service. He quickly logs on and enables it with a few clicks. Tim was, therefore, able to help his parents access services they would have previously been unable to and, he was able to troubleshoot their issue within a few minutes.

Use Case Diagrams

We created two Use Case diagrams to illustrate how users would interact with our system.

We split the system based on the two main user components, the Concierge App and the admin website. We then created Use Case diagrams for each

Click to enlarge
Click to enlarge

MoSCoW Requirements

Using all the data we had gathered from both our clients and our users, we were able to create our Moscow list of requirements.

Functional Requirements

Must Have

  • Mobile App

    The IBM FISE Concierge Virtual Assistant must be available as an easy to use and intuitive app.

  • Simple UI

    The app must have a simple interface, with minimal menus, making it easy to use for the elderly.

  • Call loved ones

    The app should allow users to call people on their contact list.

  • Message loved ones

    The app should allow users to SMS message people on their contact list.

  • Open/Search Websites

    The app should allow users to open and search websites in their default browser.

  • Food-related Service APIs

    The app should provide services centred around food (e.g., recipes, dietary information etc.).

  • Weather Information Service APIs

    The app should provide services centred around weather information (e.g., current weather, forecasts, air quality etc.).

  • Transport Information Service APIs

    The app should provide services centred around transport information (e.g., nearest bus/train station etc.).

  • FISE Logs/Data Storage

    The system must be able to store data about user's interactions with the virtual assistant. It must also store relevant data for each user.

  • Admin Panel

    The system must provide an interface for loved ones to monitor and enable elderly users interactions with the Concierge app.

  • Add/Remove Services from Elderly Users

    The admin panel should allow admins to add and remove services from elderly user's accounts.

  • Manage Elderly user data

    The admin panel should provide admins with the ability to modify the data of the users that they control (e.g., phone number, name etc.).

Should Have

  • Alarms

    The app should allow users to create, read, update and delete alarms.

  • Reminders

    The app could provide the ability for users to create, read, update and delete reminders.

  • Timers

    The app could provide the ability for users to create timers.

  • Search Device for Contacts

    The app should have the ability to search the device's contacts based on the user's speech input.

  • Entertainment Service APIs

    The app should provide services centred around entertainment (e.g., books, news etc.).

  • Finance Service APIs

    The app should provide services centred around finance (e.g., stocks).

Could Have (Optional Requirements)

  • Query backend for user info (e.g., GP Phone number)

    The system could store relevant user information (e.g., GP phone number) in the backend, allowing the admin to modify this information and to be used by the app.

  • Device Navigation

    The app could provide the ability for users to open other apps through speech.

Won't Have

  • Multi-Lingual Support

    It was suggested to us, by our clients, that the app could support multiple different languages. However, is simply not enough time to work on the translations needed to support multiple different languages. Additionally, we plan to use natural language processing, meaning it would be extremely time-consuming trying to perform NLP in different languages.

Non-functional requirements

Must Have

  • Localised Speech Recognition

    The app must be able to process a user speech into a given command locally.

  • Interaction with other RESTful APIs

    The app must use other external RESTful APIs to provide services to users.

  • Speech Synthesis

    The app must provide speech synthesis to give verbal responses from its services.

  • Local Data Storage

    Where possible, data storage must be primarily local.

  • Service API interaction as a RESTful API

    The interaction with external RESTful APIs must itself be a standalone component, implemented as a RESTful API.

  • Integration into wider FISE ecosystem

    IBM FISE Concierge must integrate with the other two teams (Teams 38 and 39) working on the FISE portfolio of social isolation projects.

    Note - we were only notified of this in February.

Should Have

    Could Have (Optional Requirements)

    • Ability to utilise user's location

      The app must be able to process a user speech into a given command locally.

    • Natural Language Processing for User Requests

      Natural language processing could be used to provide a more natural form of verbal communication with Concierge, as well as providing more flexibility.

    Won't Have