Implementation

Chatbot Framework Implementation

Tools and Dependencies


watson_assistant_logo

IBM Watson Assistant is used for the chatbot framework. It's easy to implement and modify, even for non-IT people.


bing_logo

Bing API is a web search API for real-time conversations.


cognitive_logo

Azure Cognitive Service for Language is used to evaluate the results from Bing API and provide more reasonable answers.


azure_functions_logo

Azure Functions helps transfer information seamlessly between the chatbot generation service, Watson Assistant, and other Azure services.

Implementation Overview

Real-time Information Retrieval Webhook
To allow our chatbot answer a wider variety of questions that is not predefined during the chatbot generation phase, we create this webhook in our customised Watson Assistant for retrieving information during a conversation. This webhook in integrated in the template JSON in the template_json.py file.

"webhooks": [
    {
        "url": "",
        "name": "main_webhook",
        "headers": [
            {
                "name": "Ocp-Apim-Subscription-Key",
                "value": ""
            },
            {
                "name": "Content-Type",
                "value": "application/json"
            }
        ]
    }
],

Because Watson Assistant webhook can only send POST requests while Bing API only allows GET requests, we build the Azure Function bingAccessWebHook to convert the request type.

const sendGetReqToBing = async (query, key) => {
    …
    const res = await fetch('https://api.bing.microsoft.com/v7.0/search' + queryStr,{
        method: 'GET',
        …
    });
    …
}

module.exports = async function (context, req) {
    const body = req.body
    const params = {…}
    const key = req.headers["ocp-apim-subscription-key"]
    let data = await sendGetReqToBing(params, key)
    …
}

We later upgraded this Bing API webhook to Bing API and Azure Language Service webhook, which is the Azure Function useQnA2ImproveAnswer. This function calls the Bing API first and then use Azure Language Service to evaluate the search results and conclude a reasonable answer according to the confidence score. Although eventually not used, the Azure Function bingAccessWebHook is not deleted from the GitHub repository because it can offer a cheaper solution for trusts that has extremely limited budget.

…
module.exports = async function (context, req) {
    …
    let data = await sendGetReqToBing(params, key)
    context.log(`received ${data["webPages"]["value"].length} data from bing`)
    const bingResults = data["webPages"]["value"].map(d => {
        return {
            "name": d.name,
            "url": d.url,
            "snippet": d.snippet
        }
    })
    data = data["webPages"]["value"].map(d => d["snippet"])
    …
    data = await sendBingResToQnA(data, body.q)
    …
    const topAnswer = data.answers[0]
    …
    if(topAnswer.confidenceScore > miniumConfidenceScore){
        result = topAnswer.answer
        if(topAnswer.answerSpan.confidenceScore > miniumCS4ConciseAns){
            resultConcise = topAnswer.answerSpan.text
        }
    }
    context.res = …
}

Frontend Implementation

Tools and Dependencies


react_logo

React.js is used as the frontend framework and for creating the generation service website.


axios_logo

Axios is used in our React.js frontend to communicate with our backend.


nodejs_logo

Node.js is the JavaScript runtime environment that is used.


watson_logo

IBM Watson Assistant API is used to build the preview chatbot in the chatbot generation service frontend.

Implementation Overview

The frontend of NHS Chatbot Generation Service is used for NHS IT admins to generate chatbots for their trust's websites. This React.js frontend mainly consists of 3 parts:
1. Chatbot Generation
2. Chatbot Preview
3. Chatbot Generation History


Backend Implementation

Tools and Dependencies


django_logo

Django is used as the backend framework.


scrapy_logo

Scrapy is an open-source web scraping framework written in Python. It's used to write "spiders" that automate the process of extracting data and crawling URLs under the NHS trust's domain.


ibm_nlu_logo

IBM Watson NLU API is a cloud-based tool that can analyse and understand the text and extract valuable insights and information from it. It's used to preprocess our raw text scraped from the website.


mongodb_logo

MongoDB is used as the database for storing the chatbot JSON files.

Implementation Overview

The backend of NHS Chatbot Generation Service consists of 4 big parts:
1. HTTP Request Handling
2. Chatbot File Generation
3. Database Structure
4. Web Scraping and Information Filtering