Evaluation

We built the platform to feel modern and easy to use. We added clear feedback when users interact with the interface, and simple elements that make moving between screens feel smooth and not confusing.

Search, filters, and clear navigation help users find what they need without extra steps. We wanted using the app to stay as simple as possible.

Dark mode works well across the whole product, which was key for our industry partner.

We regularly reviewed the UI with our industry partner, Avanade, to stay aligned with their expectations. We also looked at similar platforms, such as Microsoft AI Foundry, to compare layout and how information is presented. We showed the platform to people outside the project and used their feedback to make small improvements.

The design hides heavy technical detail by default. Users who want a simple overview can use it easily, while those who want more depth can open the technical sections when needed. This helps avoid overwhelming less technical users.

Rating: great. We are very happy with how it feels, and the positive feedback we received from our partner on it.

We delivered all must-haves and could-haves in the MoSCoW traceability tables. The only item not implemented as written is one should-have: the extra sandbox field on the vulnerability page, which we deliberately skipped because the Testing area already covers with/without mitigation runs (see that row for the rationale). All other core and optional features, including dual-framework automated testing, are reflected in the product: Garak is the main scan path in the UI, and PyRIT is present in the backend runner layer alongside it for the same pipeline family.

In addition, we extended the platform beyond the original scope. We introduced live demonstrations for vulnerabilities, knowledge quizzes to support learning, expanded testing capabilities, and improvements to navigation. These additions helped make the platform more complete and easier to use, both for learning and for hands-on interaction.

The only limitations in functionality come from factors outside the platform itself. The system relies on external language models and APIs, whose behaviour can change over time as providers update their systems. This can affect outputs or responses in ways that are not fully predictable. However, this dependency was known from the start and is a normal part of working with modern AI systems, rather than a flaw in the implementation.

Rating: great. We delivered all planned features and added meaningful improvements, with the only variability coming from external systems beyond our control.

The backend is supported by a strong automated test suite, with over 300 tests and more than 96% test coverage. All tests pass in a standard setup, giving us high confidence that the system behaves as expected.

On the frontend, we tested all main user flows carefully. We used the platform ourselves, shared it with our industry partner, Avanade, and also gave it to external users to try. Across all of these, we did not encounter any issues or crashes in normal use.

The main improvement we would add is clearer in-app feedback when something fails due to external API errors. At the moment, these cases are handled and messages are shown, but they could be more explicit and helpful.

Rating: very good. The platform is stable in normal use, with only minor improvements needed in user-facing error messages.

We designed the platform to keep model usage as low as possible while still delivering good results. We simplified where model calls are needed and avoided unnecessary requests, so the system stays fast and responsive.

For features such as the prompt enhancer and the chatbot, we use lower-cost models by default that are still capable of handling the task well. This allows users to interact with these features without high usage costs. At the same time, users can choose to switch to more advanced models if they want more complex outputs.

This approach keeps both latency and cost under control in normal use.

One improvement for the future would be to support parallel execution for more complex testing scenarios. Running multiple tests at the same time would reduce the total time needed to complete a full testing audit and return results to the user more quickly.

Rating: very good. The platform is efficient for its intended use, with clear opportunities to improve performance further for advanced scenarios.

The web app runs in modern browsers and works well on desktop for daily use.

Users can connect different models, either from common cloud providers or from their own endpoints, such as self-hosted models or small services. The platform is not tied to a single provider: you choose and configure the models that fit your setup, so one application can work with the providers you already use or with models you run yourself.

The design mainly focuses on desktop use. While it can run on mobile, the layout has not been fully optimised for smaller screens.

Rating: good. Desktop experience is strong and model setup is flexible, but mobile use could be improved.

We reuse code where possible, so shared logic is kept in one place and does not need to be fixed in multiple files.

Adding new vulnerabilities or content currently requires changes in the codebase. This works well for technical users. Less technical contributors can still help, but they would need access to the repository and some guidance.

There is no CMS, as all structured content is stored in the repository. For a technical team, this works well because it provides clear version history and a single source of truth. Avanade was comfortable with this approach. If the product later needs frequent updates from people who do not edit code, moving to a CMS would be a natural next step; we already flag that in our future work in case that becomes a priority in the future.

We also keep integrations with external AI providers in a single place, so updates are easier to manage if APIs change.

Rating: good. The project is clear and easy for developers to extend, though it is less accessible for non-technical contributors (not expected to make changes to the platform at this stage).

We managed the project in a structured and consistent way throughout. We stayed on track with all deadlines and shared regular progress updates with our UCL supervisors and our industry partner, Avanade. We also scheduled weekly meetings to share progress and get feedback, while keeping them short and focused to respect our partner’s time.

Internally, we used planning documents to define responsibilities and track tasks. We also shared frequent updates through our group chat, which helped keep everyone aligned and aware of progress.

We successfully delivered everything we set out to build, along with additional features, within the timeline.

Work was divided so that each person focused on specific parts of the platform. While this worked well for efficiency, it meant we had less direct involvement across other areas of the project. That said, we maintained strong collaboration through code reviews, requiring at least two approvals before merging any changes.

Rating: very good. We stayed organised, communicated well, and delivered beyond the original scope, with only minor room to improve cross-team involvement.