AskBob

AskBob – a customisable, open-source framework for developing federated, privacy-safe voice assistants in partnership with IBM and the NHS.

Speech is a much more natural and increasingly ubiquitous way to interact with the technology around us, but many existing voice assistants are not easy for non-experts to modify and extend, and often require cloud access to function, raising privacy concerns. This limits the scope for their use in health and social care, and other settings where they have the potential to be leveraged more widely.

For our clients at IBM and the NHS, we have therefore been building Ask Bob, a customisable framework for developing federated, privacy-safe voice assistants operable both: within IBM’s FISE ecosystem aiming to combat social isolation; and interactively standalone on compatible low-power Windows and Linux desktop devices, such as the Intel NUC, on which all speech and data processing is performed locally to help safeguard user privacy.

It is also accompanied with a progressive React-based configuration generator web app to help non-experts design new AskBob plugins, as well as a progressive ‘skills viewer’ web app to visually inspect the skills available on Ask Bob servers by installed plugin name and category.

AskBob has already been integrated as the central component of a wider ecosystem into the FISE concierge and video conferencing lounge projects of teams 25 and 38, respectively, to provide voice assistant capabilities that are privacy-safe over a private local area network. Moreover, it has a modular skills architecture so that its skill set can be extended via additional componentised plugins installed at build-time that, for example, interface with external services. This will allow third-party developers and researchers to make use of AskBob in their own open-source projects in the future.

As an example, our proof-of-concept prototype device based on the Intel NUC, following further development to develop tailor-made plugins specific to use cases in health and social care, could then be ready to be used in initial studies with our NHS partners.