gesture_auth

gesture_auth is a modular Python 3.12 prototype for recording and recognizing custom hand gestures with MediaPipe Hands.

It is only for gesture recognition and visualization. It does not perform keyboard automation, OS automation, app launching, process control, or system control.

This version can optionally run Windows scripts that you place in the local scripts/ folder. Treat those scripts as your own automation layer and review them before enabling.

Requirements

• Python 3.12
• Webcam
• mediapipe
• opencv-python
• numpy

Install dependencies:

pip install -r requirements.txt

Run

python main.py

The camera preview is mirrored. The target capture rate is 30 FPS, subject to webcam and machine performance.

Controls

• C: Record/calibrate the current gesture
• L: Load the current gesture name, or first available saved gesture
• N: Start a new generated gesture name such as gesture01
• TAB: Cycle through saved gestures
• R: Recalibrate the current gesture
• D: Delete the current saved gesture
• Q: Quit

To save a gesture, press N if you want a new generated name, then press C. The app gives you a short hands-free countdown so you can move both hands into position. When CALIBRATING appears, hold still for 3 seconds.

Folder Layout

gesture_auth/
    main.py
    config.py
    camera.py
    hand_tracker.py
    calibration.py
    matcher.py
    normalizer.py
    script_runner.py
    storage.py
    hud.py
    utils.py
    requirements.txt
    README.md
    gesture_scripts.example.json
    gestures/
    scripts/

Gesture Format

Each gesture is stored as its own JSON file in gestures/.

Example:

{
  "name": "gesture01",
  "creation_date": "2026-06-30T09:00:00+00:00",
  "number_of_samples": 72,
  "normalized_landmarks": [[[0.0, 0.0, 0.0]]],
  "threshold": 80.0
}

The real normalized_landmarks value contains one or two hands, with 21 landmarks per hand and 3 coordinates per landmark.

Windows Scripts

Gesture-to-script mappings live in gesture_scripts.json. Copy gesture_scripts.example.json to gesture_scripts.json when you want to enable local script bindings. Scripts must be inside the local scripts/ folder and can be .ps1, .bat, or .cmd.

Example:

{
  "gesture01": {
    "enabled": true,
    "script": "gesture01.ps1"
  }
}

Then create:

scripts/gesture01.ps1

The app runs the mapped script once after the gesture reaches MATCH. It will not run inline command strings from JSON, and it will reject script paths that point outside scripts/.

The included example starts disabled:

{
  "gesture01": {
    "enabled": false,
    "script": "gesture01.ps1"
  }
}

Matching

matcher.py owns gesture matching. The current implementation uses mean Euclidean landmark distance and converts that distance to a 0-100% similarity score. A gesture must stay above its configured threshold for one second before the HUD displays MATCH.

To add a new matching algorithm later, modify or replace matcher.py while keeping its public GestureMatcher.compare() behavior.

Tracking

hand_tracker.py owns MediaPipe Hands. Replacing MediaPipe with another tracker should only require changing that module, as long as it continues to return HandData objects with (hands, 21, 3) landmark arrays.