Evaluation
MOSCOW Requirement Evaluation
Requirement | Priority | Completed? | Contributors |
---|---|---|---|
Users must be able to upload JSON files | MUST | YES | Mujtaba, Anny |
Free form text input for entering keywords and regex | MUST | YES | Mujtaba, Samuel, Varun |
Users can fully customise the advanced filters | MUST | YES | Mujtaba, Samuel |
The filters set by the user must not be lost when the page is refreshed | MUST | YES | Mujtaba, Samuel, Varun |
A delete button with an X or rubbish bin icon that removes this line of filter for both the content and all the UI elements. | MUST | YES | Samuel |
Match case Tick Box option | MUST | YES | Mujtaba, Samuel |
Regex match Tick Box option | MUST | YES | Mujtaba, Varun |
Filter application could be done efficiently | MUST | YES | Mujtaba, Varun |
Search results should be responsive | MUST | YES | Mujtaba, Varun |
Creating filters must be intuitive for users | MUST | YES | Mujtaba, Samuel |
Results of filter application on the dataset should be done efficiently, and the response must be fast. | MUST | YES | Mujtaba, Varun |
Security of the data must be ensured as the web-app will be handling internal Cisco log file data. | MUST | YES | Mujtaba, Samuel, Varun |
Date-Time interval filtering | SHOULD | YES | Varun |
AI agent should be able to summarize and give quick statistics about logs | SHOULD | YES | Mujtaba |
AI agent should be able to generate filter groups for the user based on the query | SHOULD | YES | Varun, Mujtaba |
AI Agent is accurate | SHOULD | YES | Mujtaba |
AI Agent should be responsive | SHOULD | YES | Mujtaba |
Expandability of codebase | SHOULD | YES | Mujtaba, Varun, Samuel |
Program should not introduce many new dependencies | SHOULD | YES | Mujtaba, Samuel, Varun |
Backend design for loading multiple log files | SHOULD | YES | Mujtaba, Varun |
User should be able to define the context for the AI agent | COULD | YES | Mujtaba |
AI agent should provide recommendations for resolving identified issues | COULD | YES | Mujtaba |
Offline AI compatible across all devices | COULD | NO | |
Implement adaptability for different languages/regions | WON'T | NO | |
Predictive analysis on data | WON’T | NO |
Individual Contribution to System Artefact
Work Package | Samuel (%) | Varun (%) | Mujtaba (%) | Anny (%) |
---|---|---|---|---|
Research & Experiments | 15 | 15 | 55 | 15 |
UI Design | 40 | 10 | 35 | 15 |
Coding | 25 | 22.5 | 45 | 7.5 |
Testing | 15 | 40 | 40 | 5 |
Overall Contribution | 23.75 | 21.875 | 43.75 | 10.625 |
Individual Contribution to Website Report
Work Type | Samuel | Varun | Mujtaba | Anny |
---|---|---|---|---|
Client Liaison | 20 | 30 | 40 | 10 |
Research | 5 | 30 | 60 | 5 |
Requirement Gathering | 20 | 30 | 20 | 30 |
Sketches & Prototyping | 25 | 25 | 25 | 25 |
HCI Presentation | 25 | 35 | 10 | 30 |
Coding | 20 | 15 | 60 | 5 |
Algorithm | 10 | 70 | 20 | 0 |
UI Design | 40 | 40 | 5 | 15 |
System Design | 15 | 20 | 65 | 0 |
Implementation | 50 | 10 | 40 | 0 |
Testing | 10 | 40 | 50 | 0 |
Evaluation | 0 | 90 | 10 | 0 |
User & Deployment Manual | 20 | 0 | 70 | 10 |
Blog writing | 34 | 33 | 33 | 0 |
Video scripting & shooting | 45 | 35 | 20 | 0 |
Overall Contribution | 22.6 | 33.53333333 | 35.2 | 8.666666667 |
Critical Evaluation
This section is a critical evaluation of the core functionalities of the WebEx log viewer app. We run a thorough and objective assessment of the strengths, weaknesses, and effectiveness of the web-based solution that we have built. We also identify areas for improvement which will be discussed further in the Future Work section.
Ported and Prototype Features
Below is a clear overview of features categorized by their implementation status—whether successfully ported (🟩) or currently part of our prototype (🟨):
Feature | Status | Comments |
---|---|---|
Customizable Filter Groups | 🟩 Ported | Successfully integrated, allows tailored filter configurations enhancing log analysis capabilities. |
Advanced Search (Regex) | 🟩 Ported | Implemented robust regex search functionalities enabling powerful query flexibility. |
Keyword/Regex Highlighting | 🟩 Ported | Seamlessly highlights matched patterns using dynamic color generation for increased visibility and readability. |
Split-Pane View With Record Jumping | 🟩 Ported | Two synchronized views for optimal simultaneous monitoring of filtered and unfiltered logs. |
Asynchronous and Incremental Highlighting | 🟨 Prototype | Complex synchronization logic would need extensive refactoring of legacy code. We did not have enough time to port this feature. |
LLM ChatBot Assistance | 🟨 Prototype | Highly ambitious AI-driven feature, providing real-time intelligent insights. Requires extensive user feedback and deeper integration feasibility studies. |
Notably, despite the challenges of working with legacy code that did not align with modern JavaScript best practices, substantial effort was invested in refactoring, modernizing, and thoroughly documenting the codebase wherever necessary. The features successfully ported—customizable filter groups, advanced regex search, keyword highlighting, and the split-pane view—represent all the requirements explicitly requested by Cisco, making this project a complete success. In addition, we ambitiously developed innovative prototype features such as asynchronous incremental highlighting and AI chatbot assistance, representing enhancements beyond Cisco’s expectations. These prototype features are currently awaiting further development, comprehensive user validation, and deeper technical exploration before potential integration into Cisco's extensive and established log viewer platform.
User Interface / User Experience
After our final presentation of our project to our clients, we can confidently say that our user interface is a strong point of the product. Their core UI requirements for the split pane view, match case and match regex toggles were all implemented well, and we based our UI off their existing solution to ensure that the team can easily familiarise themselves with the new tool. We made use of colour theory to ensure that the randomly generated colours for highlighting were selected such that they were easy on the eyes, and this was a feature appreciated by the clients.
Functionality
All of the core functional requirements were completed, and our clients were very pleased with the result. The customisable filter groups feature, in particular, was built to their expectations, and was merged within their production codebase the earliest, so we got to have feedback of several Cisco engineers using it, which was overall very positive. We have extended on the core requirement by making all of their static filters editable too, which further enhances the customizability and flexibility of our tool. This was also met with positive user feedback, empowering them to adjust filters in real-time and tailor the experience to their specific needs.
However, the functionalities are not without room for improvement. As part of our experimental features in our prototype, we added asynchronous and incremental filtering which we believed was the ultimate feature that improves snappiness and responsiveness of filtering. But unfortunately, as has been made clear by our clients, adding this to the existing codebase would have proved extremely challenging requiring major code refactors. Perhaps if we had access to the repository a bit earlier than only 1 month prior to the deadline, we might have made progress towards this feature as well.
Efficiency
Efficiency has been a key requirement that has been at the forefront of all the features that we have implemented in the log viewer. We have ensured responsive and efficient loading of log files through asynchronous programming. The technologies that we decided to use for the backend, namely elasticsearch, was due to its high speed processing of log files. We have also created caching techniques to store previously computed RegEx patterns, so that they are not recompiled. We have also created efficient custom data structures for faster retrieval of previously searched substring/regex patterns.
Compatibility
We have built a highly compatible product. Cross compatibility with multiple devices is one of the core requirements established from the beginning. The issue with Cisco’s current call-tracker app is that it is a downloadable app, rather than a web-app which has a much easier accessibility. Therefore, we went down the web-app route, to make this log viewer compatible across a range of devices. We even chose an online LLM Agent as offline ones can be OS native, whereas online models such as ChatGPT are machine agnostic.
We might have been able to achieve full compatibility with offline models as well if we had more time. Unfortunately, llama.cpp, the inference code we used for our offline mode, has compatibility issues running with accelerated hardware in Windows & Linux, and that’s why we have disabled offline AI by default. If we had experimented with other inference tools such as ollama, we might have achieved compatibility with this part of our project as well.
Maintainability
We have strived to ensure a high level of maintainability through good coding practices. Since this project will be merged into a production environment, and will likely be built on further by Cisco’s engineers, we had to ensure that our code adheres to their standards. We made sure to create detailed documentation too, so that the engineers at Cisco will be able to easily understand the code.
Project Management
Throughout the project, the team made good use of agile techniques and demonstrated strong collaboration.
Communication was open and frequent, both during scheduled meetings and through daily interactions on messaging platforms. This helped maintain momentum and allowed quick adjustments when project requirements shifted. The use of agile principles like iterative development and continuous feedback helped ensure that deliverables stayed aligned with stakeholder expectations. Each week, we presented our latest developements, and made a detailed todo list for the features that have to be developed/modified for the upcoming weeks.
We made effective use of GitHub issues and PRs to track the tasks that we had to do and task assignment. Our communication with the clients was done on WebEx chatrooms, which was very useful as the material & documentation was all in one location. We discussed with our clients the tools that we can use to track tasks, comparing products like Jira, Kanban etc. We wanted our workflow to align with Cisco's as much as possible, therefore, we used GitHub and WebEx as the main forms of communication.
Finally, we also studied and aimed to enforce good Git principles throughout the project. This included maintaining a clean commit history, writing clear and descriptive commit messages, and following a consistent branching strategy to avoid conflicts and maintain code quality. Feature branches were used to isolate development work, and pull requests were reviewed by team members to ensure code quality and encourage collaboration. We also made use of `.gitignore` to manage unnecessary files and avoided committing sensitive or temporary data.
However, there was room for improvements too. At times, sprint goals could have been more clearly defined, which occasionally led to scope creep or misaligned priorities, such as putting too much emphasis on the frontend instead of ensuring certain functionalities. Another issue was in the delay that we had in gaining access to Cisco's repository. Some team members also got access to the repo faster than others, resulting those team members being further along the developement process.
Future Work
Although all of the main requirements were completed, there were many areas where the project could be expanded.
AI & Automation Enhancements
- If we had more time, we could have done more detailed research into the AI agent. Adding more bespoke features, and training it on custom data would have been an excellent way to automate error finding in the log files. We could have even created a system that allows each team to train this on their own data, thus creating a fully bespoke model
- If we had more insights into how different errors are resolved, we could have created a page which guides the users in fixing the common issues that they see in the data. By analyzing how previous issues were resolved, we could create a knowledge base that not only shows the root cause but also explains why the error happened and how it was fixed last time. This adds educational value and could reduce reliance on senior developers for troubleshooting. Therefore, this tool becomes an all-inclusive website to both locate errors, and learn how to fix them.
User Interaction & Collaboration
- We would have liked to create an annotation tool, where users can flag different records and make notes about them. This could also include the ability to categorize issues by type or severity and mark them as "under investigation" or "resolved".
- We could have created an easy way to share the version of your file with other members of teams, including the annotations and flagged records. This would improve the collaboration between members, and further enhance productivity.
- Introducing a plug-in architecture, where annotated log-entries can be exported to Excel or Jira would further enhance collaboration within the team, and reduces the overhead needed to resolve issues found in the log files, as the webapp becomes a single resource to both find and fix errors.
Feature Expansion
- Adding support for multiple different file formats like CSV and XML would expand the versatility of the tool, and make it applicable to clients outside of this one specific team.
- Integrating authentication and user management features would allow for personalized settings and secure multi-user access.
- We could offer multiple ways to sort the data, such as sort by time, or sort by priority of issue.
Data Visualization & Insights
- Incorporating visual analytics, like charts and timelines, could help users identify patterns or anomalies in log data over time. This can be expanded by displaying trend lines and predictive analytics for numerical data.