Testing


After finishing the implementation of the project, we conducted several tests to make sure that SOTA works as expected:

  • Unit Testing
  • Integration Testing
  • Stress Testing
  • User Acceptance Testing
  • Performance Testing


Unit Testing

To test the functionality of individual unit, we used Unit Testing to validate that each component of the software performs as designed. As our project is written with Java programming language, we decided to use Junit for unit testing.

The benefits to improve our project by doing Unit Testing [1]:

  • Unit testing makes it easier to add new features to existing code and design in further development. When the code is changed, we can check if each component still performs properly by running the tests. It makes the project more agile and increases the confidence in refactoring.
  • Since unit testing is the first step of the testing strategy, issues could be found early so that it won’t impact the other parts of code. Therefore, it reduces the cost of bug fixes. Because once all the unit tests passed, if the program fails in other test, the issues of individual component can be excluded.
  • Unit testing provides us a better way to debug during development process, if a test for a specific component fails, we only need to debug the latest change of that part of code.
We did Unit Testing on all the functions of our program:
  • Conversion of text to speech
  • Conversion of speech to text
  • Conversation flow
  • Dictionary function
  • Database logging


Integration Testing

With all unit tests passed, we carried out the integration testing. It is testing whether the units still perform as expected when combined as a group and to expose the issues in interactions between integrated units. There are several types of Integration Testing: Top-down testing, Bottom-up testing, Big Bang testing and Hybrid Testing. Since the units of our program are at the same level, we decided to do the Big Bang Integration Testing.

To implement Integration Testing, we speak a sentence to SOTA as input, and the output is the response from SOTA. We intended to check if output is correct after processing the input by conversion of speech to text, conversation flow, and conversion of text to speech. And make sure the input is recorded in database correctly. As a result, the integration test pass and units combined as a group work well.


Stress Testing

Stress testing is a software testing activity that determines the robustness of software by testing beyond the limits of normal operation. [2]

In our case, our stress testing is to put SOTA under a noisy environment and test if SOTA can still recognize the user’s word.

The results showed that SOTA can handle most of the interfering noise, however, if the noise is too loud, user’s input will not be recognized and an error message will be given by SOTA.


User Acceptance Testing

The target users of our project are the children aged from 4 to 5. However, we are not allowed to test with children. To simulate a user acceptance testing, we tested with our peers and demonstrated our project to our supervising TA and client every week. We got feedback from the testing to help us improve the project, e.g. Change the voice and pitch of SOTA while telling a story.


Performance Testing


SOTA Response Latency


The graph above is the server condition of Google Could of our project. In the third graph, it shows the median latency of API call by time. The latency is the time taken to parse the audio file to text. The peak at the beginning is because we were using mobile hotspot to connect with SOTA. However, under a stable Internet connection, the average of latency is 5 seconds. Adding the time taken by other data processing together, it takes 6-8 seconds for SOTA to response given by a stable Internet connection.


Reference

[1]: Segue Technologies, The Benefits of Unit Testing [2018], https://www.seguetech.com/the-benefits-of-unit-testing/

[2]: En.wikipedia.org, Stress testing [2018], https://en.wikipedia.org/wiki/Stress_testing