Evaluation

The project successfully implemented a web-based interface with an intuitive design. Key features include drag-and-drop functionality and visual feedback through a loading bar, enhancing usability. Additionally, comprehensive documentation is available on sightlinks.org, providing users with necessary guidance.

Moreover, our source code has clearly documented README files for more technical users. Users will find our system to be easily installed. We verified this by providing our source code to our partners at team 14, as well as our clients, and several mock users within our class, and instructed them to follow the README to build the project without providing any help. Positive feedback was recieved from all parties, with everybody being able to successfully deploy our application.

The system demonstrates strong functional performance, achieving over 90% overall accuracy. The classification layer achieved 99.7% accuracy on unseen satellite data, which is significantly greater than comparable implementations in literature. The Object Detection layer successfully implemented Rotated Object Detection techniques, which have previously not been used in literature on transport feature detection, and achieved a <50cm precision using 15cm resolution data. This is a significant improvement over comparable solutions in literature.

The system is compatible with multiple common geo-spatial file formats, including the most common .tif and .jgw formats, while maintaining standardised outputs. This makes the system immediately and easily integratable with common GIS services. This was verified through consultion with an expert at the Environmental Sciences Research Institute, working on the ArcGIS service.

The processing pipeline is robust, consisting of multiple well-defined modular stages:

• Image segmentation to divide images into meaningful parts
• Filtering to remove noise and improve detection accuracy
• Crossing detection to identify key landmarks
• Georeferencing for precise spatial alignment

These stages have clearly defined input and output endpoints to ensure a stable pipeline, helping guarantee consistent and reliable performance across different datasets without requirement adjustments to the code. Each stage has unit tests, end-point tests and integration tests, to ensure that errors are immediately and clearly identified during the development process and before deployment.

To the best of our knowledge, the final deployed version does not contain any major bugs for all standard inputs. The website interface incorporates input validation, ensuring that only properly formatted data adhering to a standardized geospatial dataset format is accepted. Any error occuring during the processing of data on the server is handled gracefully, and returns the user to the input screen.

Significant optimizations have been achieved to enhance system efficiency:

• Efficiency Improvements:
  Initially, the classification screening layer utilized a VGG16 model, as it was the approach outlined in the research paper on which our system was originally based [1].
  • Throughout the development process, various models were evaluated and integrated to enhance performance. The final model architecture, MobileNetV3_small, achieved an 8.85× improvement in processing speed, reducing inference time from 0.185 seconds (VGG16) to 0.021 seconds (MobileNetV3_small). All performance evaluations were conducted on an NVIDIA 4070 GPU, with each test repeated three times and averaged to mitigate potential anomalies caused by background subprocesses.
  • In terms of hardware independent statistics, the model size was reduced from 528 Mb to 21.8 Mb, while the FLOP count per inference was reduced from 16 Billion to 60 Million.
  • Initial model: VGG16 (528 MB)
  • Intermediate model: ResNet50 (98 MB)
  • Quantised MobileNetV3 (2.8MB)
  • Final models:
    • MobileNetV3 (21.8 MB)

Our system is compatible with several standardised file formats as both input and output. It has a direct integration with team 14's platform, who are our primary users.

Our system offers multiple access methods to accommodate varying levels of technical expertise and different requirements for development granularity, ranging from a complete abstraction from the underlying code to the tools to retrain the layers to accept different features other than crosswalks:

• Core implementation is open-sourced on GitHub
• A WebApp interface with complete abstraction from the implementation, with a server backend on Azure.
• A robust and well-documented API interface for easily integrate into their systems.
• An open-sourced development toolkit, with proper documentation for recreating our results or retraining for a different feature set.

These fulfill the needs of our three clients (Open-source implementation for Soundscape Community, easily integratable API for GDI-hub and non-technical webapp for the Wheelchair Alliance).

The codebase follows best practices for maintainability, featuring:

• A modular pipeline design, ensuring flexibility for future updates.
• Clear separation of concerns, improving code readability and manageability.

We have very thorough documentation throughout our codebase, including:

• API documentation on sightlinks.org, for users who wish to use our deployed system
• Clearly written README files in our codebase, detailing usage examples and installation methods

Our project was, in both our opinion and the words of our clients, effectively managed, enabling us to meet core MoSCoW requirements well ahead of schedule. We consistently updated our progress and recieved recommendations from clients and experts throughout the development process, ensuring that expectations were clearly defined and aligned.

Key aspects of our project management approach:

• Collaborative Development: Every team member actively participated in all stages of development, facilitating continuous improvement and group support. We maintained transparency by holding biweekly meetings— one during lab hours on Tuesdays and another through weekly client calls.
• Quality Assurance: Implemented thorough code review processes, testing protocols and regular refactoring to maintain high code quality.
• Documentation: Maintained comprehensive documentation throughout development, making it easier for team members to understand and contribute to different parts of the project, as well as allow our clients to offer relevant feedback at each stage.
• Risk Management: Proactively identified and addressed potential issues early in the development cycle. Input and Outputs to each function are clearly documented, and are tested by a set of unit tests. We designed our entire system before any implementation began, and although we added additional features later we did not deviate from this strategy.
• Timeline Management: All critical deadlines were met. Our structured approach provided flexibility to address emerging issues and refine the system beyond initial targets without compromising on our original intended functionality.

Our management approach prioritized:

• Regular communication with clients and partners to maximize transparency and incorporate feedback from people who are experts in their field.
• Clear, but flexible, task delegation and responsibility assignment to optimize efficiency while accomodating people's personal needs.
• Systematic bug tracking and resolution to maintain system stability and performance, and maximise user safety.