For testing the web server by integration testing, we used Python’s unittest library (which can also be used for integration tests). We made use of Django’s helper functionalities on top the unittest library, combining the djangorestframework (DRF) to build on top of this further.

Django provides a “test client” which can be used to simulate requests to a URL, as though a user was making the request through a web browser

The DRF extends this with a couple of helpful tweaks, such as the ability to easily specify the format of data being sent (json/multipart/etc.) and the ability to automatically send multipart data with PUT and PATCH, as opposed to just POST.

For each different piece of functionality of the pet API, the test sets up some data for the scenario and makes a request using the test client. It then checks that the returned response from the server is correct, and that the resulting DB state is correct.

As a result of the tests, we were able to be very confident that all our users will be able to interact with this API stably and consistently to ensure a good experience for all. We achieved 100% test coverage within the user facing portions of the app, which is a useful metric in showing that we have covered all bases of the web server with our integration tests.

There are mainly three approaches to UI Testing - Manual Testing, Record-and-Playback Testing, and Model-Based Testing.

We found the manual testing method to be very time-consuming and inefficient, given the large scale of the GUI and the many components for both desktop and mobile. Thus, we did some further reserach and found a UI Automation Test Framework for games and apps called “Airtest” (Airtest Project: https://github.com/AirtestProject), which has a cross-engine test automation framework based on UI inspection called "Poco".

Poco uses a method of testing that is similar to both record-and-playback testing (where the testing tool records the activity of the user and then imitates it) and model-based testing (where the run time behaviour of a project is checked against predictions made by a model), hence covering our needs when testing the UI. Poco has Unity SDK support, so we followed the tutorial given on the GitHub repository step by step, and wrote the scripts to test our project. The full documentation of the tests can be found under the Poco repo, but in short the tests are created in Python. Each component is specified by using a Poco object, instantiated by "poco = UnityPoco()". The poco() object is then instantiated with the specified GUI component and used to carry out the tests on each component. As seen above, poco() imitates the GUI component specified inside and tests its various methods, such as .click() for buttons. When the script is run, the Log at the bottom displays whether or not the tests have passed or whether they've failed for any reason. We can then use this information to get a better idea of where our GUI is failing and where it is implemented correctly.

Using Poco for testing both saved us a lot of time (as we didn't have to manually come up with writing scripts but could instead follow a framework) and greatly improved our testing efficiency.

After we implemented the automated testing provided by Poco, we were able to ensure that the UI functioned smoothly and efficiently without encountering any hiccoughs. Although Poco doesn't provide the test coverage in terms of percentage, we ensured that we tested all components of our GUI (for both desktop and mobile) and made sure that all of them passed, thus obtaining (somewhat indirectly) 100% coverage for our GUI components. All of our UI tests passed, meaning that our GUI -barring any exceptional bugs- functions fine and can be deployed for users to interact with.

After conducting our own test methods, we asked 2 other users to test our project and provide feedback on whether they were satisfied with their experience. The testers' real names are hidden for privacy purposes, however the remaining information of each tester is real.

Our testers were chosen with the intent that they would be in our potential userbase, which would allow us to gain a better understanding of what we should pay attention to more in our project. This in turn also allowed us to improve our existing project and understand the code from a user standpoint rather than a software developer one. With this in mind, we separated our test cases into the following cases stated below.

Test Cases

Our testers were chosen with the intent that they would be in our potential userbase, which would allow us to gain a better understanding of what we should pay attention to more in our project. This in turn also allowed us to improve our existing project and understand the code from a user standpoint rather than a software developer one. With this in mind, we separated our test cases into the following cases stated below.

Test Case 1

We had our testers use the desktop GUI, searching for bus times and then trying to retrieve business-related news.

Test Case 2

We had our testers use the desktop application and interact with the pet, telling them about the different interactions (depending on the mouse clicks) and having them try the different clicks out.

Test Case 3

We made our testers use the mobile GUI and interact with the mobile pet, having them move the pet around.

Test Case 4

We asked our testers to use the application on both mobile and desktop to interact with the pet using their voice, making them try out different commands and have a chat with the pet.

We asked our testers to rate the given acceptance requirements from 1 to 10, where a 1 would serve as "strongly dislike" and a 10 would serve as a "strongly like". Their individual scores are given below.

Conclusion

At the end of our user acceptance testing, both our testers mentioned that they enjoyed interacting with the pet and that the pet would be sufficient to help them with their daily activities. As a result of their feedback, we can conlude that our project has mostly served its purpose, and with a few tweaks based on the comments of our testers, we can make our project appeal in a more comprehensive and engaging way for our target audience. The feedback provided by our testers also reassured our progress thus far and further motivated us to improve upon the project.