Skip to content

feat(prompts): add camera onboarding agent with discovery workflow - #634

Open
kgmwang1 wants to merge 7 commits into
mainfrom
feature/camera-agent
Open

kgmwang1 wants to merge 7 commits into
mainfrom
feature/camera-agent

Conversation

@kgmwang1

@kgmwang1 kgmwang1 commented Jun 26, 2026 •

Copy link
Copy Markdown
Contributor

Description

Implements a deterministic preflight and selection layer for RTSP/ONVIF camera onboarding. Engineers can discover candidates only within an explicitly approved network scope, authenticate selected ONVIF devices, inspect media profiles, verify a selected RTSP feed by receiving a decoded frame, and generate a reviewable Azure IoT Operations Terraform proposal.

Azure IoT Operations and Azure Device Registry remain the deployment source of truth. The dashboard keeps workflow state in memory, writes confidential local artifacts to a gitignored directory, and never applies Terraform.

Related Issue

Relates to #

Type of Change

  • New feature (non-breaking change which adds functionality)
  • Documentation update

Implementation Details

Camera Onboarding Workflow

The camera dashboard now provides an end-to-end onboarding preflight:

  1. Approved Scope — accepts a single IPv4 address, IPv4 address and port, CIDR, final-octet range, or explicitly selected WS-Discovery multicast.
  2. Candidate Discovery — records candidate evidence without treating an open TCP port as a confirmed camera. Unreachable endpoints are summarized instead of rendering one card per address.
  3. Authenticated Inspection — authenticates selected ONVIF devices and enumerates device information and media profiles.
  4. Live Feed Verification — marks a selected profile verified only after OpenCV receives and decodes a frame.
  5. Selection and Output — lets the engineer select verified cameras and generates deterministic namespaced_devices, namespaced_assets, and Akri Media connector enablement values.

The generated proposal includes stable names, credential-free RTSP endpoints, Kubernetes secret references, selected profile metadata, and only inspected capabilities. Duplicate logical endpoints are removed deterministically.

Dashboard and Output Experience

  • Existing manual RTSP camera addition, dashboard preview, MQTT integration, and PTZ behavior remain available.
  • Discovery and feed verification run without blocking the NiceGUI event loop.
  • Generated artifacts are written under the component's gitignored .camera-onboarding/ directory:
    • camera-discovery-results.json preserves confidential evidence, statuses, and sanitized errors.
    • camera-onboarding.tfvars.example contains the reviewable deployment proposal.
  • Successful generation exposes Copy Terraform Proposal and Download Terraform Proposal actions.
  • The dashboard never runs terraform plan or terraform apply.

Camera Onboarding Agent (.github/agents/camera-onboarding.agent.md)

The agent now guides engineers through the checked-in deterministic dashboard implementation instead of generating speculative HCL. It requires approved scope, authenticated capability inspection, profile-specific frame verification, secure local output, and human review before deployment.

Terraform Contract Alignment

  • Output matches the current namespaced_devices and namespaced_assets types in the full single-node blueprint and 111-assets component.
  • Existing blueprint examples remain unchanged; generated proposals are isolated under the component's gitignored local output directory.

Supporting Materials

File Purpose
services/camera-dashboard/src/camera_onboarding.py Deterministic feed verification, sanitization, naming, deduplication, and Terraform rendering
services/camera-dashboard/src/camera_onboarding_ui.py Approved-scope discovery, inspection, profile selection, verification, and output controls
services/camera-dashboard/src/onvif_discovery.py Bounded target expansion, candidate evidence, threaded probing, and ONVIF profile inspection
.github/agents/camera-onboarding.agent.md Secure workflow guidance using the deterministic implementation
project-adrs/Draft/camera-onboarding-preflight.md Draft architecture decision recording the selected workflow, alternatives, security boundaries, and consequences

Privacy & Security Mitigations

  • Credentials are entered only in the local dashboard and remain in memory.
  • Generated output and logs never contain usernames, passwords, tokens, cookies, private keys, or credential-bearing RTSP URLs.
  • Terraform output contains credential secret names only.
  • IP addresses, device identifiers, and topology are treated as confidential and written under .camera-onboarding/, which is gitignored.
  • Discovery never expands beyond the engineer-approved scope; multicast requires explicit selection.
  • Unreachable, unauthorized, unsupported, unknown, and feed-verification failures remain distinct.
  • Unknown adapter values are not reported as unsupported.
  • No generated vendor code is executed at runtime.

Target Applications

App Generated Config
Azure IoT Operations Akri Media connector Connector enablement plus namespaced devices and assets for verified RTSP profiles
510-onvif-connector Camera Dashboard In-memory discovery, inspection, verification, preview, and proposal workflow

Testing Performed

  • Manual validation

Validation Steps

  • python -m ruff check on all changed camera dashboard Python modules — passed.
  • python -m compileall -q src/500-application/510-onvif-connector/services/camera-dashboard/src — passed.
  • markdownlint on changed component and dashboard documentation — passed.
  • git diff --check — passed.
  • Generated camera-onboarding.tfvars.example with deterministic duplicate inputs and verified that credentials were removed — passed.
  • terraform fmt -check on the generated proposal — passed.
  • terraform plan -refresh=false against extracted authoritative blueprint variable contracts using the generated proposal — passed with no changes.
  • Live dashboard validation on an explicitly approved /24 scope — candidate discovery completed, authenticated ONVIF capability inspection passed, and the selected RTSP profile received a decoded frame.
  • Mixed selected/unselected output generation regression check — passed.
  • Dashboard /health endpoint after each runtime update — passed.

The Docker image build was attempted but the environment's connection to files.pythonhosted.org failed during dependency download with a TLS handshake error. Runtime validation used an isolated local virtual environment populated from the configured package feed.

Checklist

  • I have updated the documentation accordingly
  • I have checked for any sensitive data/tokens that should not be committed
  • Lint checks pass (run applicable linters for changed file types)

Security Review

  • No credentials, secrets, or tokens are hardcoded or logged
  • RBAC and identity changes follow least-privilege principles
  • No new network exposure or public endpoints introduced without justification

Additional Notes

The workflow produces a proposal for review only. Terraform and Azure IoT Operations remain the operational deployment path and source of truth.


🤖 Crafted with precision by ✨Copilot following brilliant human instruction,
then carefully refined by our team of discerning human reviewers.

root and others added 2 commits June 3, 2026 09:21
- add camera-onboarding.agent.md with full discovery and config generation phases
- add example sparse input and full manifest YAML files
- add camera discovery research topic and architecture concept docs
- update agents README with camera-onboarding entry

📷 - Generated by Copilot
- enforce env var reference syntax for credentials in input validation
- mandate secret-reference-only config output in Phase 5
- classify discovery manifests as confidential with gitignore guidance
- add security classification header to generated manifest files
- default output to untracked paths with gitignore recommendations

🔒 - Generated by Copilot
kgmwang1 and others added 5 commits October 8, 2026 21:41
- constrain discovery to explicitly approved scopes and preserve evidence states
- authenticate ONVIF devices, inspect profiles, and verify decoded RTSP frames
- generate schema-valid credential-free AIO Terraform proposals
- add copy and download actions plus aligned agent and component guidance

📷 - Generated by Copilot

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- correct deterministic implementation and artifact paths
- update the agent index to describe approved-scope onboarding
- remove obsolete manifest examples that conflict with confidential local output

📷 - Generated by Copilot

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- document the implemented discovery, verification, and Terraform proposal architecture
- capture rejected alternatives, security boundaries, and operational consequences
- replace superseded RPI research and coaching artifacts

📷 - Generated by Copilot

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- restore the full single-node ONVIF example to the pull request baseline
- keep camera onboarding output isolated from existing blueprint examples

📷 - Generated by Copilot

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@kgmwang1
kgmwang1 marked this pull request as ready for review October 8, 2026 22:38
@kgmwang1
kgmwang1 requested a review from a team October 8, 2026 22:38
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown

📚 Documentation Health Report

Generated on: 2026-10-08 22:39:44 UTC

📈 Documentation Statistics

Category File Count
Main Documentation 224
Infrastructure Components 233
Blueprints 40
GitHub Resources 27
AI Assistant Guides (Copilot) 17
Total 541

🏗️ Three-Tree Architecture Status

  • ✅ Bicep Documentation Tree: Auto-generated navigation
  • ✅ Terraform Documentation Tree: Auto-generated navigation
  • ✅ README Documentation Tree: Manual README organization

🔍 Quality Metrics

  • Frontmatter Validation:
    success
  • Link Validation: success

This report is automatically generated by the Documentation Automation workflow.

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown

📚 Documentation Health Report

Generated on: 2026-10-08 22:42:19 UTC

📈 Documentation Statistics

Category File Count
Main Documentation 224
Infrastructure Components 233
Blueprints 40
GitHub Resources 27
AI Assistant Guides (Copilot) 17
Total 541

🏗️ Three-Tree Architecture Status

  • ✅ Bicep Documentation Tree: Auto-generated navigation
  • ✅ Terraform Documentation Tree: Auto-generated navigation
  • ✅ README Documentation Tree: Manual README organization

🔍 Quality Metrics

  • Frontmatter Validation:
    success
  • Link Validation: success

This report is automatically generated by the Documentation Automation workflow.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant