Testing was an important part of our application and it was considered from the very first stages of development. We thoroughly tested our application to ensure all the usability, functionality and stability standards were met as set out in the MoSCoW list and the clients requirements.

Testing Scope

1. The question generation system is a client-side application and so there will be a lot of user engagement. To facilitate this we ensured tests were written and performed with full consideration of the users but without any expectations of their interactions. This meant tests were written to handle all possible erroneous inputs and uses of the application as well as check if the application performs efficiently and correctly with valid appropriate interactions from the user.

2. The system maintains a multitude of models for accurately generating different types of questions. All these functions should be tested independently to ensure they work as required (for example, testing distractors can be generated for any given word as a stand-alone application). Then these functions were tested on how well they integrate within the system and perform their expected tasks.

3. Finally, as requested by the clients, the question generation system must have API endpoints for external programmers to interact with. This was also tested to see how accurately and quickly the API can respond to HTTP requests.

Test Methodology

The question generation’s built-in features were tested by unit testing. The overall performance of the web application was tested by integration testing. The web applications design and UI were tested by user acceptance testing. All of these tests were continually enforced by using continuous integration to ensure all commits did not break the application or introduce any bugs.

Test Principle

The development of the question generation system strictly followed the TDD principle. The system was continually tested by integration testing and unit testing during each stage of the development. We continuously wrote and repeatedly used integration tests to ensure the system worked correctly as a whole and the question generation and answering functions behaved as expected. Unit tests tested each individual python file and had over 80% coverage as requested by the clients. This aided us in discovering hidden problems and ensuring the application worked as smooth as possible.

Automate Testing

Unit Testing

The unit tests were written for the models which contain the SQLAlchemy interface to the database and for the functions that handle the question generation logic engine. The GIVEN-WHEN-THEN structure was used to describe all the unit tests to ensure future developers can be sure of the purposes of each test.

To perform unit testing we used the pyTest framework which is a popular way of writing scalable unit tests for testing databases, APIs and UIs. The reason for using purest over the unit test module was because pytests requires less boilerplate code and so the test suites are more readable. This was especially important because the client wanted the code to be maintainable and scalable.

Integration Testing

Functional tests were written for the controllers and routes of the project. These contain the view functions and the blueprints used in the system. Since the project uses an application factory, it was quite easy to instantiate a testing version of the application through the flask’s built-in test_client helper method. POST and GET calls were then made to this dummy server and their responses would be validated by checking their status code and the contents of the view page returned.