GamePoint

A comprehensive computer vision system for tennis match analysis using advanced object detection, tracking, and court mapping techniques. This backend implementation processes tennis videos to extract real-time telemetry data including player positions, ball tracking, speed calculations, and distance measurements.

Demo Video

Click the video above to play the demo showing tennis tracking with enhanced telemetry

Overview

This system leverages state-of-the-art deep learning models and computer vision algorithms to:

• Detect players and ball using YOLOv8 object detection
• Track movement across video frames with persistent IDs
• Map court coordinates using homography transformations
• Calculate ball speeds between hit events
• Measure player distances traveled during matches
• Generate minimap overlays with enhanced telemetry display

Features

Object Detection & Tracking

• Player Detection: YOLOv8n (yolov8n.pt) for person detection with bounding boxes
• Ball Detection: Fine-tuned YOLOv8 model (models/last.pt) optimized for tennis ball detection
• Persistent Tracking: Multi-object tracking with consistent player IDs across frames
• Ball Interpolation: Smooth ball trajectory filling between detection gaps

Court Analysis & Mapping

• Court Line Detection: Deep learning model (models/keypoints_model.pth) for identifying court keypoints
• Homography Transformation: Maps image coordinates to standardized court coordinates
• Official Court Dimensions: Accurate scaling based on official tennis court measurements (23.77m × 8.23m)
• Minimap Generation: Real-time court overview with player and ball positions

Telemetry & Analytics

• Ball Speed Calculation: Speed measurement between hit events using court distance and time
• Player Distance Tracking: Cumulative distance traveled by each player during the match
• Speed Statistics: Real-time min/max speed tracking throughout the video
• Enhanced Visualization: Semi-transparent backgrounds with emoji-enhanced telemetry display

Architecture

Input Video → Frame Processing → Object Detection → Tracking → Court Mapping → Telemetry → Output Video
                ↓                ↓               ↓         ↓           ↓          ↓          ↓
            read_video()    YOLOv8 Models    Player/Ball  Court Line  Homography  Speed/Distance  save_video()
                              Detection       IDs       Keypoints  Transform  Calculations     Enhanced UI

Prerequisites

• Python 3.9+
• OpenCV (opencv-python)
• PyTorch & TorchVision
• Ultralytics YOLOv8 (ultralytics)
• NumPy
• Pickle for stub caching

Setup

cd tennis-tracker/backend1
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install -r requirements.txt

Usage

Basic Execution

python main.py

The system will:

1. Process the input video (input/vid3.mp4)
2. Generate detection stubs for faster subsequent runs
3. Create output videos with enhanced telemetry (output_videos/output_video.avi & .mp4)

Configuration Options

• Input Video: Modify input_video_path in main.py (default: input/vid3.mp4)
• Stub Usage: Set read_from_stub=True/False to use cached detections or reprocess
• Minimap Settings: Adjust size, position, and orientation in render_minimap_overlay()
• Telemetry Colors: Customize colors in overlay_speeds_on_frames() function

Key Components

Main Processing Pipeline (main.py)

• Video reading and frame extraction
• Multi-model detection coordination
• Telemetry calculation and overlay
• Output video generation

Player Tracking (player_tracker.py)

• YOLOv8-based person detection
• Player selection and filtering
• Bounding box visualization

Ball Tracking (ball_tracker.py)

• Tennis ball detection with fine-tuned model
• Ball position interpolation
• Trajectory smoothing

Court Detection (court_line_detector.py)

• Court keypoint detection
• Homography matrix computation
• Court line visualization

Minimap Generation (minimap.py)

• Court-to-canvas coordinate transformation
• Real-time player/ball position mapping
• Enhanced court drawing with professional styling
• Automatic canvas resizing for frame compatibility

Telemetry System (speed.py)

• Ball speed calculation between hit events
• Player cumulative distance measurement
• Enhanced text overlay with semi-transparent backgrounds
• Real-time statistics display

Output

Generated Files

• Output Videos: output_videos/output_video.avi and output_videos/output_video.mp4
• Detection Stubs: Cached detections in tracker_stubs/ directory
  • player_detections.pkl - Player bounding boxes and IDs
  • ball_detections.pkl - Ball positions and trajectories
  • court_keypoints.pkl - Court keypoint detections

Video Enhancements

• Player Bounding Boxes: Color-coded boxes with persistent IDs
• Ball Tracking: Highlighted ball position with trajectory
• Court Lines: Detected court keypoint overlays
• Minimap: Real-time court overview in bottom-right corner
• Telemetry Overlay: Enhanced speed and distance display with backgrounds

Technical Details

Coordinate Systems

• Image Coordinates: Pixel-based frame coordinates
• Court Coordinates: Meters-based standardized court positions
• Canvas Coordinates: Minimap pixel coordinates for visualization

Performance Optimizations

• Stub Caching: Save/load detections to avoid reprocessing
• Frame-by-Frame Processing: Efficient memory usage for long videos
• Homography Reuse: Single transformation matrix per frame when possible

Accuracy Considerations

• Ball Detection: Fine-tuned model provides better accuracy than generic COCO model
• Court Mapping: Homography accuracy depends on clear court line visibility
• Speed Calculations: Based on court distance between hit events
• Distance Tracking: Cumulative pixel-to-meter conversion using court dimensions

Models Used

• Player Detection: yolov8n.pt (YOLOv8 nano, COCO pretrained)
• Ball Detection: models/last.pt (Fine-tuned YOLOv8 for tennis balls)
• Court Detection: models/keypoints_model.pth (Custom court keypoint detector)

Notes

• For optimal ball detection results, ensure good lighting and clear ball visibility
• Court line detection works best on videos with clearly visible court lines
• Speed calculations are most accurate between distinct hit events
• The system automatically handles video FPS for accurate timing calculations
• Output videos maintain the original FPS and resolution of input footage