PTZApp: Florence-2 detection enhancements and plant identification extension#1
Open
saumya-pailwan wants to merge 16 commits into
Open
PTZApp: Florence-2 detection enhancements and plant identification extension#1saumya-pailwan wants to merge 16 commits into
saumya-pailwan wants to merge 16 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces the extension of PTZApp, developed as part of the Summer 2025 project.
It significantly enhances the existing PTZ-YOLO system, transforming it from a static detection pipeline into a context-aware, self-directing observation system.
Key Highlights
New Intelligent Cascade Workflow
Adds scene captioning and contextual grounding using Florence-2 for dynamic, informed scanning.
Enhanced Model Support
Supports both YOLO (fast predefined detection) and Florence-2 (vision-language scene understanding).
Example Application: PlantNet Integration
Demonstrates an optional use case for species identification using the sharpest image from best-of-N captures.
(Note: PlantNet is provided as an example application and is not required for PTZ-YOLO core functionality.)
Best-of-N Capture & Blur Gate
Automatically selects the clearest image for analysis and retries if all are blurry.
Comprehensive Metadata Logging
Publishes detailed telemetry: scene captions, detections, PlantNet results, and alert streams.
Alert System
Detects target or invasive species via configurable JSON lists and emits high-priority alerts.
Edge-Ready Deployment
Fully containerized for Jetson and GPU servers, supporting offline inference.