User Manual

  1. Ensure you have python 3.11 installed
  2. To run PixelPilot, start by running the file “runner.exe” (python scripts bundled into an executable in the future)
  3. The program will ask you whether you want to use the 0.5B, the 1.5B or the 3B model. You must type your answer.
  4. Open up visual studio code.
  5. Ensure you have the extension installed and running.

These are the ways you can use our extension:

USER_MANUAL.TXT

Walkthrough
Asking Questions
Changing Models/Mastery
Image Generation
Replace Image
Qwen Model Features
  1. Press {Ctrl + Shift + P / Cmd + Shift + P} and select 'PixelPilot: Start'
  2. A window will open up that would ask a few basic questions regarding the game such as the game stlye, device the game will run on, controller type and game-making tool
  3. Lastly, the user has to enter a description of what they want their game to be and click 'Submit'.
  1. Open the Copilot chat using the ‘Toggle Chat’ button at the top of the screen
  2. Click on ‘+’ at the top right of the chat window if you wish to start a new chat
  3. Enter ‘@pixel’ followed by your question which could be any prompt
  1. Press Ctrl + Shift + P / Cmd + Shift + P
  2. Select ‘PixelPilot: Select Chat Completion Model’ or 'PixelPilot: Select Code Generation Model' or 'PixelPilot: Select Mastery Level'
  3. Select desired Model/Mastery

OR

  1. Open the Copilot chat using the ‘Toggle Chat’ button at the top of the screen
  2. Click on ‘+’ at the top right of the chat window if you wish to start a new chat
  3. Enter '@pixel /changeChatCompletionModel' or ‘@pixel /changeCodeGenerationModel’ or ‘@pixel /mastery’
  4. Select desired Model/Mastery
  1. Open the Copilot chat using the ‘Toggle Chat’ button at the top of the screen
  2. Click on ‘+’ at the top right of the chat window if you wish to start a new chat
  3. Enter ‘@pixel /image’ followed by your promt for the image you want to generate
  4. Wait until the image generation process is going on
  5. A window pops up with 3 image options generated from the prompt
  6. Select an option by clicking on the ‘Select Image {ImageNum}’ or regenerate the options by clicking on ‘Regenerate Images’
  1. Open the Copilot chat using the ‘Toggle Chat’ button at the top of the screen
  2. Click on ‘+’ at the top right of the chat window if you wish to start a new chat
  3. Enter ‘@pixel /image’ followed by your promt which contains a hint of which image you want to replace and the new image (Example: ‘@pixel /image please replace the red ball with a blue ball’)
  4. Wait until the image generation process is going on
  5. A window pops up with 3 image options generated from the prompt
  6. Select an option by clicking on the ‘Select Image {ImageNum}’ or regenerate the options by clicking on ‘Regenerate Images’

These features particularly work only for the Qwen-2.5-Coder Model

  1. Open the Copilot chat using the ‘Toggle Chat’ button at the top of the screen
  2. Click on ‘+’ at the top right of the chat window if you wish to start a new chat
  3. Enter ‘@pixel’ followed by the prompt which is the description of your game. Remember to explain the game in detail.

    The model will generate a series of sub-prompts based on your prompts

  4. For each sub-prompt, type in “G:” and then type in the prompt

    For each of these, the model will generate a section of the code for the game

Deployment Manual

These are the key steps for our clients to remember while testing and deploying our extension :

Testing the Extension

Requirements

To use PixelPilot, ensure you have the following :

  • Visual Studio Code installed on your system.
  • GitHub CoPilot VS Code Extension installed.
  • A GitHub account to access AI-powered features.
  • Python 3.11 installed on your system.
  • Node.js installed on your system.

Getting Started

  1. Clone the repository.
  2. Open the repository in VS Code.
  3. Use the terminal and install the required dependencies by running: npm install
  4. Start the extension by opening src/extension.ts, then going to the 'Run and Debug' tab using the left panel and then clicking Run Extension.
  5. Open the chat window in the side panel and interact with PixelPilot using the commands listed below.

Commands

PixelPilot supports the following commands :

@pixel /image
  • Description: Generate an image using the PixelPilot extension.
  • Usage: Type @pixel /image followed by a description of the image you want to generate.
@pixel changeChatCompletionModel
  • Description: Change the chat completion model used by PixelPilot.
  • Usage: Type @pixel changeChatCompletionModel and select a model from the list.
@pixel changeCodeGenerationModel
  • Description: Change the code generation model used by PixelPilot.
  • Usage: Type @pixel changeCodeGenerationModel and select a model from the list.
@pixel mastery
  • Description: Set your mastery level to tailor PixelPilot's responses to your skill level.
  • Usage: Type @pixel mastery and select a level (NOVICE, INTERMEDIATE, or PRO).
Walkthrough
  • Description: Start the interactive walkthrough to create a new game project.
  • Usage: Run the command PixelPilot: Start from the Command Palette (Ctrl+Shift+P/Cmd+Shift+P).

Offline Models

PixelPilot supports offline models for image generation and other tasks. Hence before running the extension, follow these instructions :

  1. Enter this in the terminal while in the directory of the repository :
    git submodule init
    git submodule update --recursive
  2. Navigate to the backend directory :
    cd Backend
  3. Create a Python virtual environment (Python 3.11):
    • Windows : py -3.11 -m venv venv
    • Mac/Linux : python3.11 -m venv venv
  4. Activate the virtual environment:
    • Windows :
      • PowerShell : venv\Scripts\Activate.ps1
      • Command Prompt : venv\Scripts\activate
    • Mac/Linux : source venv/bin/activate
  5. Install dependencies :
    pip install -r requirements.txt
  6. Run the python script :
    python runner.py
  7. Once done using the extension, interrupt the python script that is running. Then deactivate the virtual environment :
    deactivate
  8. To use the extension again reperform steps 4 and 6
Additional Notes
  • To break down a game idea into multiple prompts, simply write the game idea in the chat window.
  • To generate code based on these prompts, prefix the prompt with G: and input one prompt at a time.
Troubleshooting

If you encounter any issues:

  1. Ensure all dependencies are installed.
  2. Verify that the GitHub CoPilot extension is active and linked to your GitHub account.
  3. Check the output logs in the Output panel for error messages.

Check the output logs in the Output panel for error messages.

Deploying the extension on Visual Studio Marketplace

  1. Ensure you have node.js installed
  2. Run: npm install -g @vscode/vsce
    • (this is the VSCE tool, which is used to publish extensions)
  3. Ensure you have an organisation. If not, follow these steps:
    • Sign into azure DevOps
    • Select New organization
    • Enter a name for your organisation, select the hosting geography and then select continue
  4. Sign in to your organization by typing “https://dev.azure.com/{yourorganization}”.
  5. Open the User settings dropdown menu
  6. Select Personal access tokens
  7. On the Personal Access Tokens page, select New Token. The website will display this form:
    Create New Personal Access Token
  8. Select the following details for the token:
    • Name: any name you want for the token
    • Organization: All accessible organizations
    • Expiration (optional): set the desired expiration date for the token
    • Scopes - Custom defined: All accessible organizations
      • click Show all scopes link below the Scopes section
      • in the Scopes list, scroll to Marketplace and select Manage scope
  9. Click “Create”
  10. Go to the Visual Studio Marketplace publisher management page.
  11. Log in with the same Microsoft account you used to create the Personal Access Token
  12. Click “Create publisher” in the panel on the left
  13. In the new page, specify the mandatory parameters for a new publisher - identifier and name (ID and Name fields respectively):
    • ID: the unique identifier for your publisher in the VSCode Marketplace. This will be used in your extension URLs, and the ID cannot be changed once created.
    • Name: the unique name of your publisher that will be displayed in the VSCode Marketplace with your extensions. This can be your company or brand name.
  14. You can optionally fill out the rest of the fields.
  15. Click Create
  16. Change your directory to the directory of the extension.
  17. Run the following command in your terminal: You will then be asked to type in the Personal Access Token created in the previous step :
    Deploy Command
  18. To publish the extension automatically, use the following command:
    vsce publish

Backend Deployment

For the backend, the clients should bundle the python scripts into an executable using PyInstaller. This bundling should occur within a Python3.11 virtual environment and the Python3.11 interpreter should be bundled with the python scripts. After that, they can provide a zip folder to the users that contains the executable, the Backend/fastsdcpu and the Backend/offlineModels folders from our repository. The client should make it clear to the user that they should not change anything within the zip folder especially the directory format.

Monthly Videos

Check out our monthly videos showcasing the development process and features of PixelPilot.

December

January

Febuary

March

Development Blog

Follow our journey as we build PixelPilot, share progress updates, challenges, and breakthroughs.

Visit the Blog

GAME_OVER.EXE

Thanks for Exploring Pixel Pilot!

We hope you've enjoyed learning about our VS Code extension for children's game development.