HawkeyeVideoSync

HawkeyeVideoSync is a Python tool for synchronizing football match videos (50 fps) with Hawkeye tracking data (player and ball positions at 50 Hz), leveraging also Opta event data.

This project provides tools that enable to:

• visualize the synchronization by plotting data sources both on frame pitch and IFAB 2D pitch;
• navigate through the data sources by game times and offsets.

Video frame from the football match

❌ Hawkeye Data before Synchronization (incorrect)

✅ Hawkeye Data after Synchronization (correct)

Installation

1. Clone the repository.
2. Inside the repository directory, install dependencies:

   pip install -r requirements.txt

3. Set up environment variables: copy .env.example to .env and fill in the required variables.

Usage

Example usage scripts are available in the examples/ folder. Run them to see how to use the main features of the project.

Data Requirements

• Hawkeye data: files must be named as output{0...N}.json and placed in the appropriate directory e.g. hawkeye/.
• Opta data: the file must be named MA1_opta_match.json.

Method Overview

In this section, we explain the used method to achieve syncronization:

1. Game Time Detection: For any video frame, the game time is detected from the scoreboard overlay using an OCR model. This enables free navigation to any moment in the match (e.g., 34:56 of the first half) and is crucial for automation.

2. Player Detection: Players on the pitch are detected using a computer vision model (we used a fine-tuned version of YOLO, available at Roboflow).

3. Pitch Keypoint Detection & Homography: Keypoints of the pitch are detected using a computer vision model (we used a fine-tuned version of YOLO, available at Roboflow), and a homography is computed to map detected players in the video frame to their real positions on the IFAB pitch.

4. Synchronization: For a given video frame at a specific game time, a window of Hawkeye data is selected. The Hawkeye frame that minimizes the RMS error using the ICP (Iterative Closest Point) method is chosen for synchronization.

   

   In the video above, green points represent the players detected in the frame, while red points indicate the player positions from Hawkeye data. ICP applies transformations to make the two point clouds match.

5. Drift Correction: Multiple video-Hawkeye synchronization points are created throughout the match. Drift correction is applied by calculating the internal drift of Hawkeye data (we accumulate drift by checking wheter the next hawkeye data point has more or less than 20ms, which is the expected, ideal, interval).

---

For more details, refer to the example scripts and the source code.