It is crucial to ensure that users enjoy using our application, whether it is interface-related or functionality-related.
The bot can objectively be used for answering questions over tabular data. It is capable of processing the tabular data and then retrieving some-what accurate answers. It is suitable for easy questions such as returning variables in a column, answering questions which require aggregate operations, such as adding, subtracting, averaging, comparing and others which cover most of the questions.
Most of the time, the chatbot is able to return accurate answers. However, there are occasions where replies are not as accurate. This is because the TaPas model at the backend has an accuracy limit of approximately 50%, even though it is trained on large datasets. This might be due to inference, which is highly dependent on how the question is asked.
Due to limited time, we did not carry out sufficient experiments on model fine-tuning. If it is carried out more rigorously, the accuracy might be raised further, therefore making the chatbot more stable.
Each time a user asks a question the file is sent to the Tapas model for processing. The model has to compute tokenization as well as a lot of comparisons, operations and therefore it takes a few seconds to retrieve the most probable answer for the asked question. Additionally, sometimes the bot hangs on a question where it cannot find a suitable answer. Overall efficiency is something that should be improved in further development.
Our chatbot is compatible with popular browsers such as Google Chrome, Safari and Firefox. This is to accommodate users’ preferences. As for system compatibility, it is currently not supported on M1 Mac as the Rasa module is currently not supported there. However, it will be supported once a compatible version of Rasa is released.
We made use of the single-responsibility principle making certain all our classes were single purpose, this is seen evident given that we have separated each application and feature into their own separate classes. Modularizing the project makes it easier to find bugs and fix them without having to dig through other parts of the code. Furthermore, especially when working in a group, separating the code helps the other group members understand and pick up changes quicker.
Our project management consisted of two key pillars, first, team communication and second organisation.
Team communication is important in determining the success of any project. It was crucial that each member of the team was kept up to date on progress and had the platform to address issues and talk over matters quickly and efficiently. This made certain that blockers in the development process would halt the progress being made. Our main communication platforms are Whatsapp for text messages and Microsoft Teams for video meetings. They played an important part for tasks discussion and distribution. Moreover, to keep track of these meetings, we used a Google Calendar plugin called Timely. This helped us synchronise our calendars and schedule a time when we were all free.
Organisation was the second key pillar and another vital component that helped us manage our project. We used an online tool called Trello, a website that broke up the project into boards and split our pipeline into sprints. Each sprint contained tasks that were distributed between us. As for the project code repository, we used GitHub as part of our version control system. This is where we kept a centralised version where we could branch off, refactor and develop as a group in a safe and sustainable way.
Overall, our project is well managed and tasks are completed on time.
Our project has a number of future works that would improve some aspects of the project. A lot of our time has been researching and implementing features - some that have improved the model and some that haven’t. With the time we had, we couldn’t explore all possible solutions to the problems we encountered and therefore future work would be to continue trying to find the solutions to improve the bot's ability to answer questions in the medical field.
Adapting the clinical BERT training approach, possible future experiments include preparing a huge amount of reliable clinical tabular data of all types and training the clinical TAPAS from the TAPAS-Base model. Following the pre-training process, fine-tuning tasks could be done to allow support and increase accuracy for SQA, WikiSQL, WTQ and TabFact.
Other than that, another experiment is concatenating tables into appropriate sentences such as “Header” + “Grid”, then passing the resulting sentence into the clinical BERT model.
Both of the approaches stated above have yet to experiment and therefore, the final outcome remains uncertain.
As the bot interacts with more reliable data, more references would be explored to expand the database that we use for post processing so that it would become more robust and accurate, thus, improving the bots ability to predict if a TaPas response is precise enough to be sent back to the user.
Currently, the chatbot can only input and analyse .csv files. In the future, file extensions feature will be extended and clients will be able to use files in the .txt, .pdf or other formats they wish to facilitate their interactions with the bot.