Kinect 360 Vision Lab

Experimental real-time computer-vision tools for the Xbox 360 Kinect. The project can display the infrared and depth streams, track motion in the Kinect dot field, build tensor-ready sensor frames, and run a TensorFlow/YOLO dashboard with pose and object detection.

Note

This project currently targets Linux and the original Xbox 360 Kinect (Kinect v1). A Kinect power/USB adapter is required.

Features

• Native IR and 11-bit depth capture through libfreenect
• Motion grid, region tracking, trails, and virtual line-crossing counts
• Normalized two-channel IR/depth tensors for model input
• Full-screen TensorFlow dashboard with pose, object, scene, and motion views
• Optional NVIDIA GPU acceleration for TensorFlow

Requirements

• Linux with Python 3.11 or newer
• Xbox 360 Kinect and its power/USB adapter
• libfreenect and Video4Linux utilities
• A desktop session capable of showing OpenCV windows

The AI dashboard downloads models from TensorFlow Hub when first used. YOLO weights are also downloaded by Ultralytics when the configured model is not already present. These generated files are intentionally excluded from Git.

Quick start

Install the system packages on Debian or Ubuntu:

./scripts/setup_libfreenect.sh

Unplug and reconnect the Kinect, then confirm that it works:

freenect-glview

Create a virtual environment for the lightweight viewers:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r requirements/base.txt

Run one of the viewers:

python kinect_freenect_viewer.py --combined-view
python kinect_grid.py --mode tracker

Press q to quit. The tracker also accepts r to rebuild its background baseline. The native viewer can save its latest tensor with s when --export-tensor PATH.npy is supplied.

AI dashboard

The full dashboard has a larger, separately pinned dependency set:

python3 -m venv .venv-tf
source .venv-tf/bin/activate
python -m pip install --upgrade pip
python -m pip install -r requirements/ai.txt
./start.sh

TensorFlow runs on the CPU by default. To require a configured NVIDIA GPU:

./start.sh --tf-device gpu

Use python tf_runner.py --help to see display, model, camera, and threshold options. python tf_gpu_check.py prints the TensorFlow/CUDA status and performs a small matrix multiplication.

Utilities

Capture and inspect one normalized IR/depth tensor:

python inspect_tensor.py --save-prefix sample

The resulting .npy tensor and preview image are ignored by Git.

Project layout

File	Purpose
kinect_native.py	ctypes wrapper around libfreenect_sync and tensor preprocessing
kinect_freenect_viewer.py	Native IR/depth viewer and tensor export
kinect_grid.py	V4L2 dot-field motion tracker
tf_runner.py	TensorFlow/YOLO fusion dashboard
inspect_tensor.py	Single-frame tensor diagnostic
scripts/	Environment activation and Debian/Ubuntu host setup
requirements/	Lightweight and AI dependency sets

Troubleshooting

• The Kinect cannot be opened: run freenect-glview, reconnect the adapter, and verify your user has access to the USB device. A reboot may be needed after installing udev rules or blacklisting gspca_kinect.
• No display window appears: OpenCV's GUI requires a graphical desktop; it will not work in a headless shell without display forwarding.
• GPU mode exits: confirm python tf_gpu_check.py lists a GPU. CPU mode is the supported fallback.
• Model loading is slow: first use may download and cache model data.

Development

Install the lightweight dependencies, then run the checks:

python -m unittest discover -s tests
python -m compileall -q *.py

See CONTRIBUTING.md before opening a pull request.

For private-data considerations, trusted model usage, and vulnerability reporting, see SECURITY.md.

Project status and license

This is experimental software, not a safety or security system. Depth and AI outputs can be inaccurate. No open-source license has been selected yet; until one is added, the repository remains all rights reserved by its author.