Testing

Testing Strategy

As a client side application, we test our code thoroughly with a coverage of over 90%+ (if we can count them). Therefore, we are delivering our product to client with great confidence.

Besides pure testing, we also use Continuous Integration (CI) with Travis, Jazzy documentation coverage and swiftformat code-linter formatting as part of our quality assurance process.

Performance Tests

We used Xcode's built-in XCTest for performance testing, in aspects of launching, loading a video and loading a 4K60FPS video.

Sample code of how are able to measure time is shown above.

With multiple optimisations made during our implementation stage, and making full use of the multi-threading and pre-fetching strategy, we are able to archive better results then a main-stream education resource application.

All data below are collected on Patrick's iPhone XS Max, Apple A12 Bionic@2x2.5GHz, 4x1.6GHz, 4GB RAM, testing X5GON-Mobile v0.3 and Coursera v3.6.

Responsive Design Testing

Although we have tested our application with integration UI testing on multiple platforms, we would also have to deploy them manually and try them out ourselves to make sure these UI elements are placed correctly and clearly visible to the end user.

With our limited knowledge, we didn't find a way to automate this process, nor are we rich enough to own every Apple Device. However, as shown above, we made our best attempt to make full use of the simulator and adapted our application to multiple screen sizes.

Compatibility Testing

As an application developed with Swift 5 and upgraded to Swift 5.1 during the development cycle, we have very limited support for older devices. However, we do aim to support all devices with Operating System version newer than iOS 13.0 and iPadOS 13.0.

To do backwards compatibility testing, we provided CI test scripts so as to test on earlier platforms we do not have.

See actual scripts below for detailed information.

User Acceptance Testing

To gain an overview of how our application is doing and find out about potential improvements, we asked 4 people from various backgrounds to test our application and subsequently recorded their feedbacks.

Testers

Monica

Mark

Bang

John

Tasks

Feedback

Feedback from Client

We are very proud to say that our client is very satisfied with our work, and everything is up to his standard.

Continuous Integration

We utilised Continuous Integration extensively throughout the validation process of our application. Since version v0.1 (prototype 1), we imported Travis as our CI platform and tested almost all PRs with CI before merging.

Since then, we have made more than 350 builds and 40 types of tests. We believe this is also an important part on how we deploy our testing strategies to everyday development.

Documentation Coverage

Technically speaking, as a client-facing application, documentation would not be such an urgent need like an API-based software. Despite this, we are proud to announce that we have archived a documentation coverage of higher than 85%.

Although documentation coverage is not an actual test, we take it as an validation process of making sure we understand every bit of our code, and make them ready to be presented to clients and to be developed in the future.

An online published version of documentation can be found here.

Testing and Formatting at Development Stage with Git Hook

We recognise the importance of eliminating human errors in code development and follow good automation practices. Therefore, we use swiftformat and hook it to Git, so as to force the execution of linting and formatting process before every single code commit

As developer ourselves, we are amazed at how automation could eliminate certain easy but hard-to-notice bugs to be committed into the code base. We believe that this is a key part of how we practice testing and validation in our development cycle.