A real-time computer vision system for detecting, tracking, and analyzing vehicle behavior in traffic video feeds. The system uses a custom-trained YOLO object detection model combined with grid-based trajectory analysis to predict vehicle turns and detect overtaking maneuvers with high accuracy.
- Overview
- Features
- System Architecture
- Directory Structure
- Requirements
- Installation
- Usage
- How It Works
- Configuration
- Output Format
- Development and Testing
- License
This project implements an end-to-end autonomous driving perception pipeline focused on two critical driving behaviors:
- Turn Prediction -- Classifying vehicle movements as straight, left turn, right turn, or U-turn based on trajectory analysis through a fine-grained grid system.
- Overtaking Detection -- Identifying when one vehicle passes another traveling in the same direction, using longitudinal position tracking, direction similarity analysis, and bounding box overlap checks.
The system processes traffic surveillance video, runs YOLO-based vehicle detection on each frame, tracks vehicles across frames using grid cells, and outputs an annotated video along with structured CSV and JSON logs.
- Custom-trained YOLO model (Ultralytics YOLOv11) for detecting cars, motorbikes, rickshaws, trucks, and buses.
- Configurable confidence threshold (default 0.25).
- GPU-accelerated inference with automatic CUDA detection and fallback to CPU.
- Ultra-fine grid system dividing the frame into square cells (approximately 4 pixels per cell after subdivision).
- 8-directional movement classification between grid cells.
- Trajectory history maintained over 30 frames per vehicle for stable analysis.
- Angle-based classification using configurable thresholds:
- Straight: less than 10 degrees of cumulative angle change.
- Left/Right Turn: 10 to 25 degrees of angle change.
- U-Turn: greater than 150 degrees of angle change.
- Pattern matching against predefined turn movement sequences.
- Confidence scoring for each classification.
- Direction similarity check (within 15 degrees) to confirm vehicles are traveling the same way.
- Vertical bounding box overlap verification to ensure vehicles are in the same lane region.
- Longitudinal position swap detection across consecutive frames.
- Cooldown mechanism (30 frames) to prevent duplicate counting of the same overtaking event.
- Screen divided into left and right halves; overtaking is only compared between vehicles on the same side.
- Exclusion zones: top 10% and bottom 10% of the frame are excluded from tracking, along with a configurable diagonal boundary line.
- Audio alert (system beep) on overtaking detection (Windows only).
- Annotated output video with:
- Color-coded bounding boxes based on movement direction.
- Grid overlay for debugging trajectory analysis.
- Magenta diagonal boundary line showing the exclusion region.
- Yellow vertical center line separating left and right detection zones.
- Red horizontal lines marking the tracking zone boundaries.
- Cyan lines connecting vehicle pairs being compared for overtaking.
- Thick orange bounding boxes and connection lines highlighting active overtaking events.
- On-screen turn classification labels with probability scores.
- Frame counter and cumulative overtaking event counter.
- Per-frame CSV output with vehicle ID, bounding box coordinates, centroid, direction, turn classification, angle change, confidence, velocity, grid cell information, and overtaking status.
- JSON summary of all overtaking events organized by vehicle and in chronological order.
- Plain text log of overtaking events for quick review.
- Detailed application log saved to
turn_prediction.log.
Input Video
|
v
+-------------------+
| YOLO Detection | -- Custom-trained model (runs/detect/train4/weights/best.pt)
| (GPU or CPU) |
+-------------------+
|
v
+-------------------+
| Vehicle Tracker | -- Grid-based tracking with trajectory history
| (VehicleTracker) |
+-------------------+
|
+-----------------------------+
| |
v v
+-------------------+ +-------------------+
| Turn Analysis | | Overtaking |
| (Grid + Angle) | | Detection |
+-------------------+ +-------------------+
| |
v v
+-------------------+ +-------------------+
| Annotated Video | | CSV / JSON / Log |
| Output | | Data Export |
+-------------------+ +-------------------+
| Class | Purpose |
|---|---|
TurnPredictionSystem |
Main orchestrator. Initializes the model, processes video frames, coordinates detection, tracking, annotation, and data export. |
VehicleTracker |
Manages per-vehicle trajectory history, grid cell mapping, turn pattern classification, and overtaking detection logic. |
DetectionInfo |
Data class holding per-frame detection results including bounding box, direction, turn classification, and overtaking status. |
TurnAnalysis |
Data class encapsulating turn detection results with direction, angle change, confidence, and frame range. |
TrajectoryPoint |
Data class for storing position, timestamp, velocity, acceleration, and angle at each trajectory sample. |
| Enum | Values |
|---|---|
MovementDirection |
Left, Right, Up, Down, Up-Left, Up-Right, Down-Left, Down-Right, Stationary, Unknown |
TurnDirection |
Straight, Left Turn, Right Turn, U-Turn, Stationary, Unknown |
VehicleType |
Car, Truck, Bus, Motorcycle, Bicycle, Unknown |
dc/
|-- 3.py Main application script
|-- requirements.txt Python dependencies
|-- yolo11n.pt Base YOLO model weights
|-- README.md This file
|
|-- models/
| |-- turn_lstm.h5 LSTM model for turn prediction (experimental)
|
|-- runs/ YOLO training output directory
| |-- detect/
| |-- train4/
| |-- weights/
| |-- best.pt Custom-trained vehicle detection model
|
|-- results/ Output files from processing runs
| |-- grid_based_turn_prediction_output.mp4 Annotated output video
| |-- grid_based_turn_predictions.csv Per-frame detection CSV
| |-- overtaking_events.json Overtaking event summary
| |-- overtaking_events_log.txt Plain text overtaking log
| |-- turn_prediction.log Application log
| |-- input_video_*.mp4 Input video segments
|
|-- tests/ Test and demo scripts
| |-- quick_demo.mp4
| |-- turns_detected.mp4
| |-- detection_analysis.png
| |-- training_history.png
|
|-- frames/ Extracted video frames (if applicable)
|
|-- extra/ Supporting scripts and earlier versions
| |-- 1.py - 5.py Iterative development versions
| |-- analyze_results.py Results analysis script
| |-- calculate_accuracy.py Accuracy evaluation
| |-- simple_accuracy.py Simplified accuracy metrics
| |-- final_test.py Final integration test
| |-- quick_demo.py Quick demonstration script
| |-- simple_demo.py Simple demonstration script
| |-- minimal_turn.py Minimal turn detection script
| |-- divide.py Video segmentation utility
| |-- check_pytorch_gpu.py GPU availability check for PyTorch
| |-- check_tf_gpu.py GPU availability check for TensorFlow
| |-- test_gpu_setup.py GPU configuration test
| |-- test_imports.py Dependency import verification
| |-- test_system.py System-level tests
- A machine with a CUDA-compatible NVIDIA GPU is strongly recommended for real-time processing. The system will fall back to CPU if no GPU is available, but performance will be significantly slower.
- Minimum 4 GB GPU memory recommended.
- Python 3.8 or higher
- CUDA Toolkit (if using GPU acceleration)
opencv-python
numpy
ultralytics
torch
pandas
matplotlib
tensorflow
- Clone the repository:
git clone https://github.com/Harshalj2108/Autonomous-Driving.git
cd Autonomous-Driving- Create and activate a virtual environment (recommended):
python -m venv yolovenv
yolovenv\Scripts\activate # Windows
# or
source yolovenv/bin/activate # Linux / macOS- Install dependencies:
pip install -r requirements.txt- Verify GPU availability (optional but recommended):
python extra/check_pytorch_gpu.py- Place your custom-trained YOLO model weights at:
runs/detect/train4/weights/best.pt
If you do not have a custom model, you can use the included yolo11n.pt base model, but detection classes will differ from the custom set (car, motorbike, rickshaw, truck, bus).
python 3.pyThis will:
- Load the custom YOLO model from
runs/detect/train4/weights/best.pt. - Open the input video
input_video_000_000.mp4. - Process each frame with vehicle detection, tracking, turn analysis, and overtaking detection.
- Display a live annotated video window titled "Lane Following Detection -- Bird's Eye View".
- Save the annotated output video to
grid_based_turn_prediction_output.mp4. - Save per-frame detection data to
grid_based_turn_predictions.csv. - Save overtaking event logs to
overtaking_events.jsonandovertaking_events_log.txt.
| Key | Action |
|---|---|
| ESC | Stop processing and save all outputs |
| S | Save a screenshot of the current annotated frame |
Edit the main() function at the bottom of 3.py:
MODEL_PATH = "runs/detect/train4/weights/best.pt"
INPUT_VIDEO = "input_video_000_000.mp4"
OUTPUT_VIDEO = "grid_based_turn_prediction_output.mp4"
CSV_OUTPUT = "grid_based_turn_predictions.csv"Each video frame is passed through a YOLO object detection model. The model outputs bounding boxes with class labels and confidence scores. Detections below the configured confidence threshold are discarded.
The video frame is divided into a dense grid of square cells (approximately 4 pixels wide after subdivision). As vehicles move between frames, the tracker records which grid cell each vehicle occupies. This creates a discrete trajectory through the grid.
Turn direction is determined through two complementary methods:
- Pattern Matching: The sequence of grid cell movements (e.g., right, up-right, up, up-left, left) is compared against predefined turn patterns for left turns, right turns, and U-turns.
- Angle Analysis: The cumulative angle change across the trajectory is computed. The system applies configurable thresholds to classify the movement.
The method with higher confidence is used for the final classification.
The system identifies overtaking events through a multi-step process:
- Filter to active vehicles with at least 3 trajectory points.
- Exclude vehicles outside the tracking zone (top/bottom margins and above the diagonal boundary).
- Only compare vehicles on the same side of the screen (left or right of center).
- Verify vertical bounding box overlap to confirm vehicles are in the same lane region.
- Check that both vehicles are moving in similar directions (within 15 degrees).
- Track longitudinal positions over consecutive frames.
- Detect position swaps: if Vehicle A was behind Vehicle B and is now ahead (consistently for 3 frames), an overtaking event is recorded.
- A 30-frame cooldown prevents the same vehicle pair from triggering duplicate events.
Each processed frame is annotated with tracking visualizations and written to the output video. Detection data is simultaneously written to CSV. At the end of processing, overtaking events are summarized in JSON and plain text formats.
Key parameters can be adjusted in the source code:
| Parameter | Default | Description |
|---|---|---|
confidence_threshold |
0.25 | Minimum YOLO detection confidence |
device |
auto | Compute device: auto, cuda, or cpu |
| Parameter | Default | Description |
|---|---|---|
max_history |
30 | Number of trajectory frames to retain |
min_movement_threshold |
5 | Minimum pixel displacement to register movement |
base_cell_size |
16 | Base grid cell size in pixels before subdivision |
subdivision_factor |
4 | How many times to subdivide each grid cell |
| Parameter | Default | Description |
|---|---|---|
straight |
10 degrees | Maximum angle change for straight classification |
left_turn |
25 degrees | Angle range for left turn classification |
right_turn |
25 degrees | Angle range for right turn classification |
u_turn |
150 degrees | Minimum angle change for U-turn classification |
| Parameter | Default | Description |
|---|---|---|
proximity_threshold |
50 pixels | Maximum distance between compared vehicles |
direction_similarity_threshold |
15 degrees | Maximum angle difference for same-direction check |
min_overtake_confirmation_frames |
3 | Frames required to confirm position swap |
overtake_cooldown_frames |
30 | Cooldown period between events for the same pair |
overtake_visual_duration |
60 | Frames to display overtaking highlight after event |
tracking_margin_top |
10% of frame height | Top exclusion zone |
tracking_margin_bottom |
90% of frame height | Bottom exclusion zone |
Each row represents one vehicle detection in one frame:
| Column | Description |
|---|---|
| frame_id | Sequential frame number |
| vehicle_id | Unique vehicle identifier (e.g., car_1, motorbike_2) |
| class | Vehicle class label |
| bbox_x1, bbox_y1, bbox_x2, bbox_y2 | Bounding box coordinates |
| centroid_x, centroid_y | Center point of the bounding box |
| direction | Movement direction (e.g., Grid-Up, Grid-Right, Stationary) |
| turn_direction | Turn classification (Straight, Left Turn, Right Turn, U-Turn) |
| angle_change | Cumulative trajectory angle change in degrees |
| confidence | Detection confidence score |
| velocity | Average pixel displacement per frame |
| grid_cell | Current grid cell identifier |
| grid_row | Grid row index |
| grid_col | Grid column index |
| overtake_count | Total overtakes performed by this vehicle |
| is_overtaking | Whether this vehicle is currently overtaking (Yes/No) |
| overtaken_vehicle_id | ID of the vehicle being overtaken (if applicable) |
Contains three sections:
- summary: Total event count, frames processed, detection parameters.
- vehicle_overtaking_stats: Per-vehicle overtake count and event details.
- all_events_chronological: Every overtaking event in frame order with overtaker ID, overtaken ID, direction angle, and confidence.
| Script | Purpose |
|---|---|
check_pytorch_gpu.py |
Verify PyTorch GPU/CUDA availability |
check_tf_gpu.py |
Verify TensorFlow GPU availability |
test_gpu_setup.py |
Full GPU configuration diagnostic |
test_imports.py |
Verify all required packages are installed |
test_system.py |
System-level integration tests |
calculate_accuracy.py |
Compute detection accuracy against ground truth |
simple_accuracy.py |
Simplified accuracy metrics |
analyze_results.py |
Analyze and plot detection results |
final_test.py |
Final integration and regression tests |
quick_demo.py |
Quick demonstration with reduced input |
simple_demo.py |
Minimal demonstration script |
minimal_turn.py |
Isolated turn detection testing |
divide.py |
Split long videos into segments for processing |
python extra/test_imports.py
python extra/test_system.py
python extra/test_gpu_setup.pyThis project is developed for academic and research purposes.