Evaluation

MoSCoW List Achievement

ID Description Priority State Contributors
1 An interface (webpage) that allows the user to input queries and output an answer in bullet points. MUST COMPLETED All
2 Users shouldn’t be able to query regarding irrelevant information, queries are limited to socio-behavioural information MUST COMPLETED All
3 Information provided must be relevant and up-to-date MUST COMPLETED Hao
4 Answer must have corresponding sources MUST COMPLETED Berken
5 Queries should exclusively rely on the information provided in the data sources MUST COMPLETED All
6 The chatbot should respond rapidly (within 10 minutes) MUST COMPLETED All
7 User manual should be written for how to use the system MUST COMPLETED Hao
8 An loading animation showing that the response is being generated and prompts the user to wait MUST COMPLETED Hao
9 A button to halt response generation SHOULD COMPLETED Hao
10 History tab with past conversations SHOULD COMPLETED Hao
11 A button to delete past conversations SHOULD COMPLETED Hao
12 An interface for users to input valid sources SHOULD COMPLETED Lucy
13 Users should be able to edit and delete inputted sources SHOULD COMPLETED Lucy
14 Users can ask the chatbot to expand on points raised in a previous answer COULD COMPLETED Berken
15 Multilingual support for multiple language (Popular languages like French, Spanish) COULD COMPLETED Berken
16 Fully integrated platform with user authentification (login/logout) COULD COMPLETED Hao
17 Multiple login options including Google, Microsoft COULD COMPLETED Hao
18 Webpage deployed online COULD COMPLETED Hao

Key Functionalities: 100%
Optional Functionalities: 100%

Bug List

ID Description Priority
1 Depending on browser, microsoft login doesn't work sometimes HIGH
2 White bar at the bottom of the screen LOW

Individual Contributions List

Work Packages Berken Hao Lucy
Project Partners Liaison 30% 40% 30%
Requirements Analysis 33% 33% 34%
Research and Experiments 30% 50% 20%
UI Design 70% 15% 15%
Coding 25% 50% 25%
Testing 0% 100% 0%
Report Website 40% 25% 35%
Video Editing 0% 0% 100%
Overall Contribution 30% 40% 30%
Main Roles UI Design
Report Website
Requirements Analysis
Tester
Project Partners Liasion
Research & Experiments
Video Editing
Report Website
Requirement Analysis

Critical Evaluation

User Interface and Experience

We evaluate our interface from the perspective of a user.

For the chat page, we've implemented a messaging interface designed to guide users seamlessly through their interactions with the chatbot. To enhance user experience during wait times, a loading overlay appears, signaling users to wait while their response is being prepared. In instances where the chatbot lacks information on a specific query, it transparently communicates its lack of knowledge, maintaining clear and honest communication. Additionally, to aid users in navigating through their chat history, the interface displays each chat with a small excerpt, allowing users to easily distinguish between different conversations and topics.

On the data sources page, users can easily identify the status of each data source, with labels clearly marked as 'Accepted' or 'Pending'. Users can add new data sources via buttons, which are inscribed with messages indicating the option to either add a website or a PDF file. For managing existing data sources, the interface includes straightforward buttons featuring universally recognizable icons: a pen on paper for editing and a bin for deleting. These icons ensure users can easily navigate and modify their data without confusion.

Functionality

In the development of our platform, we diligently followed all specified requirements to ensure maximum utility for our users. We successfully integrated essential functionalities, including the ability to read text from websites and PDFs, which broadens the platform's accessibility by accommodating various data formats. Additionally, our platform is designed to formulate responses by distilling only the necessary information, thereby enhancing efficiency and preventing information overload. Furthermore, we crafted a user-friendly and intuitive interface, making the chatbot accessible to users with varying levels of technical proficiency and ensuring a seamless interaction.

Stability

The choice of Netlify and Azure for deployment services, was their guarantee of uptime and availability. Accordingly, but using Azure App Services, it employs autoscaling capabilities which can adjust system's resources depending on current demand.

Moreover, both deployment services have analytics and monitoring tools which can be used to track the system's performance and identify potential issues. This enables the admins to identify and address performance issues proactively.

A stability concern for our system is its dependency on the OpenAI API, which necessitates a continuous financial commitment to maintain access to the API's capabilities. Consequently, the sustainability of our system is tied to the availability of our budget. In the event that our financial resources are exhausted, our system will encounter operational challenges, disrupting our service offerings.

Efficiency

Our project's performance may experience a decline during extended interactions with the chatbot, particularly when users engage in lengthy conversations. This performance degradation is attributed to our method of preserving context within the chat, wherein we feed all preceding queries from the conversation to the Large Language Model (LLM) to maintain an understanding of the dialogue's context. As the conversation progresses and accumulates more queries, the LLM is required to process an increasingly larger volume of information. This heightened demand on the LLM's processing capabilities can lead to longer response times and a decrease in overall performance as the conversation lengthens.

Compatability

We deployed our backend using Azure, which is well-suited for a diverse range of applications, ensuring a flexible and scalable foundation for our system. On the front end, we utilised Netlify for deployment, which optimises performance and simplifies the development process, thereby broadening our system's reach to accommodate users from various technological backgrounds. This strategic combination ensures that our platform is accessible and user-friendly, catering to a wide audience with different needs and tech preferences.

Maintainability

Our project has been coded following object-oriented programming (OOP) and software engineering principles, ensuring a robust and scalable structure. By adhering to these principles, we have organized the code into logically distinct files, facilitating modularity and maintainability. This approach not only promotes code reusability but also enhances the readability and navigability of the codebase, allowing for efficient debugging and seamless expansion. Consequently, any new features can be integrated with minimal disruption, and potential bugs can be swiftly identified and rectified, ensuring the project's long-term sustainability and adaptability.

Project Management

In our group, we divided the workload, ensuring that each member focused on a specific task to maximize efficiency and cohesion. We employed a Gantt chart to track our progress and maintain alignment with our project timeline. Despite these efforts, we encountered challenges with the frontend development due to the unexpectedly high volume of code required, which made us fall behind slightly, causing a deviation from our planned schedule.

In light of this, we have strategised for future projects to allocate one team member to commence frontend development from the onset. This approach will allow for parallel progress, with the remaining team members focusing on backend tasks, which typically demand less time. Once the backend is complete, these members can then assist in frontend development, facilitating a more balanced workload and efficient use of time.

Throughout the project, we maintained open lines of communication with our clients through weekly meetings, ensuring that their feedback and requirements were continuously integrated into our development process. We held additional meetings as needed to address urgent issues and utilised Google Drive to effectively share and manage files amongst the team. This structured approach to project management and client interaction allowed us to navigate challenges and adapt our strategies accordingly.

Future Works

1. We can enhance our system to read data from tables, allowing for the extraction and utilisation of structured information. By incorporating advanced parsing techniques and AI-driven algorithms, our system can interpret complex table structures, translating raw data into actionable intelligence. This addition would significantly broaden the system's functionality, enabling users to seamlessly interact with and analyse data presented in tabular formats..

2. We could enhance our system's capabilities by processing various types of files, not limited to PDFs alone. Adding compatibility with Word documents, Excel spreadsheets, and other widespread file formats would significantly boost our platform's flexibility and cater to the diverse requirements of our users. To accomplish this, we can incorporate several Python libraries designed specifically for these data sources. For instance, python-docx facilitates the reading and writing of .docx files, allowing us to extract both text and metadata from Word documents. For Excel spreadsheets, libraries such as openpyxl or pandas can be employed for effective data manipulation and extraction, addressing various levels of complexity and user needs. This enhancement would permit users to retrieve and utilize information from an expanded range of sources, thereby elevating the system's overall utility.

3. Given the global reach of the IFRC, it would be beneficial to add more language options to our system. Currently, we support French and Spanish, but expanding our language offerings to include German, Italian, and other widely spoken languages could significantly enhance the accessibility and utility of our platform for users worldwide. This expansion would align with the IFRC's international presence and further our commitment to serving diverse communities.

4. Implementing a sophisticated algorithm to automate the verification of data sources would be a significant advancement for our system. By evaluating the reliability and accuracy of website content, this approach would streamline data integration, uphold the consistent use of credible data, and bolster user trust and efficiency, all achieved without the need for manual intervention.

Site Info

This is the website for UCL Year2 2023-2024 COMP0016 Team 35: Social Behavioural Synthesis, which introduces the team project and contains all the portfolios.



Project Partners