-
Notifications
You must be signed in to change notification settings - Fork 899
Samples: add telemetry redaction sample #1824
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds a new sample demonstrating how to redact PII in telemetry data (logs and traces) using Presidio with OpenTelemetry, Grafana, Loki, and Tempo. The sample shows client-side PII masking where applications call Presidio to detect and redact sensitive information before logging.
Key Changes:
- Adds a complete Docker Compose-based demo environment with Presidio services, OpenTelemetry collector, and observability stack
- Provides a FastAPI demo application that generates synthetic PII logs and demonstrates both unredacted and redacted logging
- Includes a Grafana dashboard to visualize the difference between unredacted and redacted logs
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| mkdocs.yml | Adds navigation entry for the new telemetry redaction sample |
| docs/samples/index.md | Adds table entry linking to the telemetry redaction sample |
| docs/samples/deployments/redacting-telemetry/index.md | Main documentation explaining the demo purpose, setup, and usage |
| docs/samples/deployments/redacting-telemetry/docker-compose.yml | Orchestrates Presidio services, demo app, OpenTelemetry collector, and observability stack |
| docs/samples/deployments/redacting-telemetry/otel-collector-config.yaml | Configures OpenTelemetry collector to receive and forward logs/traces |
| docs/samples/deployments/redacting-telemetry/tempo-config.yaml | Configures Tempo for distributed tracing storage |
| docs/samples/deployments/redacting-telemetry/grafana/provisioning/datasources/datasources.yml | Configures Loki and Tempo as Grafana data sources |
| docs/samples/deployments/redacting-telemetry/grafana/provisioning/dashboards/dashboards.yml | Configures dashboard provisioning |
| docs/samples/deployments/redacting-telemetry/grafana/provisioning/dashboards/pii-dashboard.json | Defines dashboard showing unredacted vs redacted logs |
| docs/samples/deployments/redacting-telemetry/app/requirements.txt | Specifies Python dependencies for the demo app |
| docs/samples/deployments/redacting-telemetry/app/presidio_client.py | Reusable client for calling Presidio analyzer and anonymizer APIs |
| docs/samples/deployments/redacting-telemetry/app/main.py | Demo FastAPI application generating PII logs with redaction |
| docs/samples/deployments/redacting-telemetry/app/Dockerfile | Containerizes the demo application |
docs/samples/deployments/redacting-telemetry/app/presidio_client.py
Outdated
Show resolved
Hide resolved
SharonHart
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks amazing, thanks
Left a comment that fixed the dashboards for me, let me know if its valid for all cases
docs/samples/deployments/redacting-telemetry/grafana/provisioning/dashboards/pii-dashboard.json
Outdated
Show resolved
Hide resolved
docs/samples/deployments/redacting-telemetry/grafana/provisioning/dashboards/pii-dashboard.json
Outdated
Show resolved
Hide resolved
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
446e0ea to
f972575
Compare
Change Description
Following discussion with @SharonHart on a sample for redacting telemetry.
PR adding a sample for redacting logs/trace attributes using Presidio. This stems from some work I have been doing for redaction + telemetry, and since there wasn't any examples for telemetry in the repo, this is a simple example for folks to get started.
Details of what's included and how to run can be found in:
docs/samples/deployments/redacting-telemetry/index.mdNote: the sample lacks depth in terms of production readiness, configurations & model selection for telemetry, et cetera, but this is clearly stated in the index.md
Happy to make changes if requested.
Issue reference
No issue
Checklist