Skip to content

PTZApp: Florence-2 detection enhancements and plant identification extension#1

Open
saumya-pailwan wants to merge 16 commits into
waggle-sensor:mainfrom
saumya-pailwan:plantnet
Open

PTZApp: Florence-2 detection enhancements and plant identification extension#1
saumya-pailwan wants to merge 16 commits into
waggle-sensor:mainfrom
saumya-pailwan:plantnet

Conversation

@saumya-pailwan
Copy link
Copy Markdown

This pull request introduces the extension of PTZApp, developed as part of the Summer 2025 project.
It significantly enhances the existing PTZ-YOLO system, transforming it from a static detection pipeline into a context-aware, self-directing observation system.

Key Highlights

  • New Intelligent Cascade Workflow
    Adds scene captioning and contextual grounding using Florence-2 for dynamic, informed scanning.

  • Enhanced Model Support
    Supports both YOLO (fast predefined detection) and Florence-2 (vision-language scene understanding).

  • Example Application: PlantNet Integration
    Demonstrates an optional use case for species identification using the sharpest image from best-of-N captures.
    (Note: PlantNet is provided as an example application and is not required for PTZ-YOLO core functionality.)

  • Best-of-N Capture & Blur Gate
    Automatically selects the clearest image for analysis and retries if all are blurry.

  • Comprehensive Metadata Logging
    Publishes detailed telemetry: scene captions, detections, PlantNet results, and alert streams.

  • Alert System
    Detects target or invasive species via configurable JSON lists and emits high-priority alerts.

  • Edge-Ready Deployment
    Fully containerized for Jetson and GPU servers, supporting offline inference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant