Testing for the IFRC VSR platform focuses on the project’s highest-risk areas: invalid request payloads, broken authentication, unstable API contracts, malformed LLM output, and regressions between the analyst, operations, and admin workflows.
Jump directly to the main testing views.
The strategy is layered so failures are caught as close as possible to their root cause.
The remaining automated coverage consists of 35 helper tests and 23 serializer tests, giving the backend strong validation around request parsing, normalisation, and response shaping before the frontend consumes the data.
APITestCase exercises public, authenticated, and admin-only endpoints.The backend test suite is strongest around API contracts, auth boundaries, and report quality invariants, with CI/CD build checks adding release-level assurance.
The lower layers of the backend are covered by serializer and helper tests. These checks verify that invalid dates, missing fields, key-format mismatches, and malformed section payloads are caught before they reach more expensive API or orchestration paths.
The Django suite exercises session login/logout, CSRF handling, public endpoints, and admin-only endpoints
through APITestCase. This gives confidence that the API contracts used by the React frontend are stable and
that privileged actions remain protected.
The orchestrator tests focus on the risks unique to LLM-based generation: broken section ordering, missing citations, unstable output schemas, and failed retry logic. Fixture-based end-to-end tests are especially important here because execution alone does not prove that the report is still usable.
In addition to the repository test suites, GitHub Actions is used in the delivery pipeline to run automated build checks before deployment. On the main branch, the frontend workflow builds the React application for Azure Static Web Apps on pushes and pull requests, while the backend workflow sets up Python 3.12, installs dependencies, packages the Django service, and deploys the artefact to Azure Web App. This means deployment-breaking integration issues are checked automatically rather than being left to manual testing alone.
Taken together, the automated tests give strongest assurance over backend correctness, report quality, and security boundaries. The main remaining weakness is that the repository does not yet include browser-level automation for the frontend, so some user-facing confidence still depends on manual walkthroughs.
Each category below states why it exists, how it is run, and what conclusion it supports.
Helper and serializer tests exist to fail fast. They are run with valid, invalid, minimal, and typo-prone inputs so the team can detect request-shape problems before a long report-generation workflow is started.
Authentication testing focuses on the admin boundary because that is where the highest operational risk lies. The test suite checks session persistence, CSRF handling, and the difference between public, authenticated, and admin-only behaviour rather than assuming the framework will enforce this correctly by default.
API integration tests call representative endpoints with realistic parameters, while the orchestrator suite checks section ordering, output cleanliness, citations, retry behaviour, and dual-view stability. GitHub Actions also provides automated build verification in the CI/CD path for both the React frontend and Django backend, so integration confidence does not rely solely on manual walkthroughs. This is especially important because an LLM workflow can produce poor output even when no exception is raised.
Manual walkthroughs remain part of the methodology because there is no committed browser automation framework. Analyst, operations, admin, and map routes are each exercised directly so that front-end usability can be checked alongside raw backend correctness.
Results from user testing sessions and feedback.
User acceptance testing complemented the automated suites by looking at whether the platform was usable in realistic environments, not just whether the backend returned technically valid responses. The acceptance sessions therefore focused on complete journeys across the analyst, operations, admin, and map views, where navigation, clarity, and confidence in the generated content mattered as much as correctness.
The testing covered representative tasks such as opening a crisis, reviewing report sections, checking attached sources, comparing outputs across views, and using protected admin actions. This helped confirm that the implemented routes and API contracts supported end-to-end task completion rather than isolated component behaviour.
The most valuable feedback was mainly about usability rather than low-level defects. In particular, the sessions reinforced the need for clear navigation and a visible distinction between analyst and operations views. Another important requirement that arose was readable section headings, and obvious boundaries around admin-only functionality.
The overall outcome suggested that the core user journeys were understandable once users reached the right page, but that presentation and discoverability had a strong effect on confidence. This supported the project’s emphasis on separation of views, clearly labelled report sections, and transparent sources.
Feedback from project partners was received upon handover of the project. The system achieved the project’s core goals around clear workflows, validated outputs, and end-to-end report generation. The main refinement identified was that some generated reports could still improve in analytical depth, even though the overall direction and usefulness of the platform were strong.
These sessions provide useful evidence that the interface is workable for its intended roles, but they do not replace a larger formal study with operational users in live humanitarian settings. For that reason, user acceptance testing is treated as a complement to automated verification and integration testing rather than a substitute for them. Given the operational constraints and the system’s primary goal of clear, validated report generation, the remaining limitation in report depth was acknowledged as an area for future architectural refinement.
The testing evidence maps back to the key functional and non-functional requirements used elsewhere in the report.
| Requirement | Main Test Evidence | What Is Being Verified | Coverage Type |
|---|---|---|---|
|
Orchestrator regression contract tests plus report retrieval integration tests | Ordered sections, minimum narrative quality, valid source URLs, and retrievable report artefacts. | Automated |
|
Dual-view schema tests, get_operations_view, and references endpoint tests |
Operations payload contains executive summary, key metrics, priority actions, and supporting sources. | Automated + manual route walkthrough |
|
test_auth.py, create-crisis integration tests, protected /admin route design |
Only authenticated admins can access crisis creation, ingestion, and orchestrator actions. | Automated + manual admin scenario |
|
References endpoint tests and operations/dual-view source assertions | Source links remain attached to report sections and executive outputs. | Automated |
|
Authentication flow and permission tests | Unauthenticated or non-admin users are rejected from protected endpoints. | Automated |
|
Retry logic tests and failing-workflow resilience tests | Transient failures are retried and unrecoverable failures return structured error information. | Automated |
|
Manual analyst, operations, map, and admin walkthroughs | Routes remain understandable and the role-specific views can complete their intended task flow. | Manual |