Skip to content

Conversation

@spompea-cfa
Copy link
Contributor

@spompea-cfa spompea-cfa commented Oct 22, 2025

Add OTel metrics:

  • new otel.py module added
  • Consumer: Senzing add_record -- counter, duration metrics
  • Redoer: Senzing process_redo_record -- counter, duration metrics
  • Redoer: Senzing's redo queue size -- gauge metric
  • Exporter: export + write to S3 bucket (treated together as atomic) -- counter, duration metrics
  • Add pip call to middleware Dockerfiles for OTel libs

Ancillary:

  • Tweak init-db section of docker-compose for stability

@spompea-cfa spompea-cfa changed the title [WIP] Add OTel metrics [WIP] Instrument middleware to emit OTel metrics Oct 23, 2025
@spompea-cfa spompea-cfa changed the title [WIP] Instrument middleware to emit OTel metrics Instrument middleware to emit OTel metrics Oct 31, 2025
@spompea-cfa spompea-cfa marked this pull request as ready for review October 31, 2025 15:00
@spompea-cfa spompea-cfa requested a review from a team as a code owner October 31, 2025 15:00
@jamesiarmes jamesiarmes requested a review from Copilot October 31, 2025 21:03
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds OpenTelemetry (OTel) instrumentation to the consumer, redoer, and exporter services to track metrics for message processing, export operations, and queue counts. The changes also include a minor Docker Compose fix for service health checks.

  • Added OTel metrics collection with configurable exporters (Console or OTLP)
  • Instrumented three middleware services to track operation counts, durations, and status
  • Added environment variable support for runtime environment identification

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
middleware/otel.py New module providing OTel initialization and common constants for status tracking
middleware/redoer.py Added OTel metrics for redo message processing and queue monitoring
middleware/exporter.py Added OTel metrics for export operation tracking
middleware/consumer.py Added OTel metrics for consumer message processing
Dockerfile.redoer Installed OpenTelemetry packages
Dockerfile.exporter Installed OpenTelemetry packages
Dockerfile.consumer Installed OpenTelemetry packages and enabled logging auto-instrumentation
docker-compose.yaml Fixed depends_on syntax to use health check condition
README.md Documented new OTel environment variables

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Member

@jamesiarmes jamesiarmes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks sound. Copilot found a few small things, please address before merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants