Testing

Testing the scripts

Table of Contents

1. Unit Test

2. System Test

3. User Acceptance

Unit Test

Unit Testing: Ensuring Reliability in NLP Models

Why Unit Testing is Essential

Unit testing is a critical component of software development, especially for systems involving complex logic like NLP models. It ensures that individual parts of the software work as intended, providing a reliable foundation for the overall system. In the context of NLP models for gaming commands, unit tests verify that:

  • Entity recognition is accurate and consistent.
  • Similarity matching algorithms produce expected results.
  • Actions are correctly mapped to user commands.
  • The JSON architecture algorithm structures data as intended.

Passing all unit tests indicates a robust system with well-functioning components, significantly reducing the likelihood of errors in production.

Example Test Cases and Outcomes

Below is a table summarizing key test cases and their outcomes, demonstrating comprehensive coverage and reliability:

Test Case Description Outcome
test_similarities_match Confirms accurate matching based on semantic similarity. ✔
test_motion_to_action_mapping Ensures motions are correctly mapped to game-specific actions. ✔
test_predict_to_json Validates the correct structure and content of the JSON output. ✔
test_preprocess_sentences Tests for accurate sentence segmentation and cleaning. ✔
test_initialize_output_structure Checks for the correct initialization of the output data structure. ✔
test_predict Evaluates the integration of entity processing with prediction logic. ✔
test_similarities_match_none_returned_for_low_similarity Verifies that 'none' is returned for low similarity matches. ✔
test_motion_to_action_mapping_returns_none_for_unknown_game Checks correct handling of unknown game titles. ✔
test_predict_to_json_creates_correct_file_structure Ensures the JSON file is created with the correct structure and data. ✔
test_preprocess_sentences_handles_multiple_delimiters Confirms the ability to process sentences with various delimiters. ✔
test_similarities_match_with_empty_target_phrase Assesses handling of empty target phrases in similarity matching. ✔
test_similarities_match_with_empty_possible_phrases Tests similarity matching functionality with an empty list of possible phrases. ✔
test_motion_to_action_mapping_with_invalid_game_type Ensures type validation for game titles in motion to action mapping. ✔
test_predict_to_json_with_empty_sentences Ensures that if the user is quiet, it will not add anything. ✔

Unit Test Code Example

An example of a unit test from test_predict.py is shown below, testing the similarity matching function:


                def test_similarities_match(self):
                      target_phrase = "mining"
                      possible_phrases = ["place", "mine", "break"]
                      expected = "mine"
                      result = similarities_match(target_phrase, possible_phrases)
                      self.assertEqual(result, expected)

This test case verifies that the similarities_match function correctly identifies 'mine' as the most similar phrase, illustrating how unit tests validate individual components' logic and functionality.

Implications of Passing All Tests

Passing all unit tests signifies a high degree of confidence in the system's reliability and correctness. It indicates that each component functions as expected under various scenarios, laying a strong foundation for further development and integration. For NLP models in gaming commands, it ensures that commands are accurately interpreted and translated into actions, contributing to a seamless user experience.

System Test

Comprehensive System Testing for NLP Model Integration

Purpose of System Testing

System testing assesses the complete and integrated software to ensure it meets its requirements. For NLP models integrated within gaming applications, system tests validate the entire processing pipeline—from recognizing natural language commands and extracting pertinent information to generating actionable JSON outputs for game control.

Key System Test Cases

Below, we outline essential system tests designed to verify the operational integrity and accuracy of the NLP model integration:

Test Case Description Expected Outcome Outcome
Full Pipeline Test Validates the entire NLP processing pipeline from input to JSON output. Output JSON correctly represents the processed commands with all necessary details. ✔
Predict to JSON Output Structure Ensures the predict_to_json function generates the correct output structure. All expected keys (mode, orientation, landmark, poses, gestures) are present in the JSON output. ✔
Game Mode Recognition Accuracy Tests the system's ability to accurately recognize and output the correct game mode from a given sentence. The system correctly identifies and outputs the game mode for various input sentences. ✔
Gesture Recognition Accuracy Assesses the system's accuracy in identifying specific gestures and mapping them to corresponding actions. Accurate identification and mapping of gestures to actions as specified by the user. ✔
Integration with Gaming Environments Tests the integration of the system with actual gaming environments to ensure seamless operation. Smooth operation within a gaming environment with real-time command processing and action execution. ✔

Example Code from System Testing

An illustrative example from the system tests, focusing on the full pipeline test:

def test_full_pipeline(self):
            test_sentence = "I want to play Minecraft using hand gestures, index pinch to jump."
            temp_output_file = Path("./temp_test_output.json")
            predict_to_json(test_sentence, str(temp_output_file))
            with open(temp_output_file, 'r', encoding='utf-8') as file:
                output_data = json.load(file)
            self.assertEqual(output_data["mode"], "Minecraft", "The game mode does not match.")

This test case highlights the practical application of system testing, demonstrating how the model processes input sentences and generates a JSON output that accurately reflects the user's intent.

Implications of Passing System Tests

Passing system tests is a critical milestone, indicating the system operates as intended in a fully integrated environment. For NLP model integration, this means the system can accurately interpret natural language commands, extract the relevant actions, and represent these actions in a structured format suitable for game control. It confirms the system's readiness for deployment and its potential to enhance the gaming experience through intuitive natural language interactions.

User Acceptance Test

Testers

System Architecture Diagram
  • Jeff: 22-year-old avid gamer familiar with voice-command interfaces.
  • Abed: 19-year-old casual player with an understanding of basic tech operations.
  • Britta: 25-year-old experienced gamer with a wide range of gaming knowledge.
  • Pierce: 20-year-old student and occasional gamer who values intuitive design.
    • Case ID Test Case Preconditions Steps to Test Expected Result Status Notes
    1 Basic Command Recognition System launched successfully. Input a basic voice command, e.g., "I want to play Minecraft." The system should recognize the game context and be ready for further input. 5/5/5/5 Every basic command like naming their video game work, this apply to all short sentences, also when game no found or not in MotionInput it return nothing
    2 Action Mapping Game context recognized. Input a command to perform an in-game action, e.g., "Jump when I pinch." The system should recognize any synonym and map to the appropriate certain game action base on that games available action 4/4/5/4 The in game action is map correctly eventhough the synonym is use, there only sometime when the user say some outside context where it misunderstood.
    3 Gesture Recognition System running and game context set. Input a gesture-related command, e.g., "I want to place a block when I do index pinch." The system should recognize the gesture and map it to the game action. 3/4/2/4 Work most of the time however there sometime where user say some vague action like not specifying number of fingers it get confuses
    4 Multi-Action Input Single action recognized successfully. Input multiple actions in one sentence. The system should parse the sentence and map each action correctly. 4/5/4/5 With longer and complete sentence, it work just fine, even sometime the user swap action and gesture it will still recognize which action belong to which gesture or pose
    9 Response Time Measurement System running. Measure the time it takes from issuing a voice command to action recognition and mapping. The system should perform the entire process within an acceptable time frame (specify the time). 4/4/4/4 When short sentence are use it able to performs under 5 seconds, however when the length of sentence increase it take significantly longer to output

    Feedback From MotionInput Members