This app will allow a user to navigate with either their nose (NoseNav), Multitouch in the air, or with Eye Gaze. They can either say commands, or use combinations of facial movements to act as mouse presses or keyboard keys.
Either install with the Installer version, or download the Portable Zip version, unzip all of the zip file's contents as it is in the folder, and run the program named MI3-FacialNavigation-3.04.exe

v3.04 for Facial Navigation features new commands for Visual Studio accessibility.

Note that this version is set for Nose Navigation with Speech by default, and you rotate through the modes with the phrase "Next Mode". If however, you wish to permanently change it to start with Eyegaze with Speech, replace the data\mode_controller.json file with mode_controller_eyegaze_(rename).json (rename it to mode_controller.json and backup that file).

From v3.03+, a new In-Range model has been added to detect from the maximum ranges you can move your neck.
Saying "left range" upon starting will set up the leftmost position marker, followed by saying "right range" will set the rightmost position marker. The size of the nose joystick will then shrink to match, and it should be more comfortable in moving within those ranges. Say "reset ranges" to go back to the default.

Head rotation has also been added which are mapped to left and right keyboard keys, by rotating your head left and right.

Here are some of the speech commands that are available when in speech modes. A full list can be seen by pressing "?" when running the software. When not in a speech mode in this microbuild (app) you will be in facial navigation mode (table below). It is currently English only but future versions will be multilingual.

Note that the joypad features are not available in this build (v3.001 - April 2022).

The in-air joypad microbuild is a technical Windows 10 demonstrator to show two handed in-air joypad control of Xbox compatible controller games. It requires ViGEM to be installed (vigem.org) before running it as the software uses vgamepad, the current versions (at time of writing) are in the zip file.

Unzip all of the zip file's contents as it is in the folder, and run the program named MI3-In-Air-Joypad-with-Ask-KITA.exe

Gestures and their assignments:

• To press start - pinch together the right thumb and the right little pinky finger
• Rotating left wrist - analogue left d-circle
• Rotating right wrist - analogue right d-circle
• Right hand fingers down - ABXY buttons
• Left hand fingers down - Up-Down-Left-Right d-pad buttons
• Right handed to left handed switching - edit the data\events.json file where it has args for right hand
• Speech is enabled via Ask-KITA so if there is a chat window, enable with "transcribe" and then you can speak. "Stop transcribe" to stop.

The speech commands as above are in this build.

The in-air inking microbuild (app) is a technical Windows 10/11 demonstrator to show native windows inking in the air, with depth calculations for ink thickness. It works with apps like Krita, OneNote, Fresh Paint, Photoshop and others. Speech is now available as commands and transcribe. You can hold a pen in your fingertips, or you can draw without any pen. The left hand will be used for erasing.

Unzip all of the zip file's contents as it is in the folder, and run the program named MI3-In-Air- Inking-with-Ask-KITA.exe. A calibration window will open. Follow the on-screen instructions to calibrate depth values. Once calibration is done, the main program will automatically start. Open any Windows Inking application such as Paint 3D, OneNote or FreshPaint. Move your hand forwards or back to control depth (ink thickness). Activate the eraser by doing an index-thumb pinch with your non-dominant hand. Calibration runs by default but can be turned off by simply editing the JSON file in data/config.json. To do this, open the config.json file, find the run_calibration variable, and set it to false. Save the file and restart the program.

Digital Inking in-air with Depth

The in-air keyboard microbuild (app) is a technical Windows 10/11 demonstrator to show a two handed keyboard in the air. It will work with up to 6 fingers but ideally 2 are optimal. Keyboard shortcut keys are also assignable.

Unzip all of the zip file's contents as it is in the folder, and run the program named MI3-In-Air-Keyboard-with-Ask-KITA.exe, or if you are using the non-speech version, MI3-In-Air-Keyboard.exe

In-Air Keyboard

This microbuild (app) is a technical Windows 10/11 demonstrator that enables you to play retro games with usual ABXY gamepad buttons, by hitting in the air. It has 3 modes to play; mode 1 - tap d-pad to set walking direction, mode 2 - hold d-pad to walk, and mode 3 - walking is always forward (W key). In order to walk in the game, you have to walk on the spot to enable the directional D-pad buttons.

Speech with Ask-KITA is included. Say Start Transcribe and Stop Transcribe to start and stop general text, otherwise it is in commands mode. Keyboard shortcuts can be added to events.json in the Data directory. Unzip all of the zip file's contents as it is in the folder, and run the program named MI3-RetroGamepad-withWalking.exe

Retro Gamepad with walking on the spot

This microbuild (app) is a technical Windows 10/11 demonstrator that enables you to play First Person and 3rd Person 3D Games while walking. In order to walk in the game, you have to walk on the spot to enable the directional D-pad buttons. The dominant hand controls the mouse cursor (normally the viewpoint in those games).

Unzip all of the zip file's contents as it is in the folder, and run the program named MI3-Walking-FPS-with-Ask-KITA.exe, or MI3-Walking-FPS.exe (without Ask-KITA speech), respectively.

Walking on the spot for FPS games

This is the package for Ask-KITA on its own, as a federated speech application on Windows 10/11. It is powered by VOSK. It should not need installation or admin rights and has both transcribe and command modes available.

Unzip all of the zip file's contents as it is in the folder, and run the program named Ask-Kita-v1.0.exe

This early preview has mouse navigation and some speech commands.

Permissions required to use this MAC version are as follows: the app will prompt the user for permission of microphone and camera, however, for accessibility permission the user needs to go to System preferences -> security and privacy -> accessibility and give permission to the MotionInput application.

If you wish to explore where we started from, download version 2 here.