Conversation
nikhilNava
left a comment
There was a problem hiding this comment.
Lets move the tests under observability, lets follow the pattern on the python SDK. thanks
There was a problem hiding this comment.
Pull Request Overview
This PR adds integration testing infrastructure for OpenAI agent instrumentation, introduces unit tests for observability components, and implements necessary fixes to support tracing generation spans. The changes establish a comprehensive testing strategy separating unit tests from integration tests that require live Azure OpenAI credentials.
- Integration test setup with Azure OpenAI configuration management
- Unit tests for core observability components (scopes, processors, baggage)
- Enhanced span processing for generation input/output messages
- Separated test execution between unit and integration tests
Reviewed Changes
Copilot reviewed 12 out of 19 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/package.json | Added integration test scripts and required dependencies (dotenv, openai, zod, @openai/agents-openai) |
| tests/observability/integration/setup.ts | Global setup file for integration tests with environment configuration |
| tests/observability/integration/openai-agent-instrument.test.ts | Integration tests validating OpenAI agent instrumentation with live API calls |
| tests/observability/integration/conftest.ts | Configuration utilities for Azure OpenAI and Agent365 environment variables |
| tests/observability/integration/.env.example | Example environment file template for integration test credentials |
| tests/observability/extension/openai/OpenAIAgentsTraceProcessor.test.ts | Unit tests for trace processor lifecycle and span handling |
| tests/observability/extension/openai/OpenAIAgentsTraceInstrumentor.test.ts | Unit tests for instrumentor enable/disable lifecycle |
| tests/observability/core/scopes.test.ts | Unit tests for InvokeAgentScope, ExecuteToolScope, and InferenceScope |
| tests/observability/core/observabilityManager.test.ts | Unit tests for ObservabilityManager configuration and builder pattern |
| tests/observability/core/SpanProcessor.test.ts | Unit tests for baggage-to-span attribute enrichment |
| tests/observability/core/BaggageBuilder.test.ts | Unit tests for BaggageBuilder fluent API and scope management |
| tests-agent/openai-agent-auto-instrument-sample/package.json | Added integration test scripts and Jest dependencies to sample package |
| pnpm-lock.yaml | Lockfile updates for new dependencies (dotenv, openai, node-fetch, etc.) |
| packages/.../OpenAIAgentsTraceProcessor.ts | Fixed generation span processing and added root span tracking |
| packages/.../Constants.ts | Added GEN_AI_EXECUTION_PAYLOAD_KEY constant |
| packages/.../jest.config.json | Updated test roots path to new observability directory structure |
| package.json | Added integration test scripts at root level with path ignoring |
| jest.integration.config.cjs | New Jest configuration for integration tests with 60s timeout |
| .github/workflows/ci.yml | Added integration test step to CI workflow with Azure credentials |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
packages/agents-a365-observability-extensions-openai/src/OpenAIAgentsTraceProcessor.ts
Outdated
Show resolved
Hide resolved
packages/agents-a365-observability-extensions-openai/src/OpenAIAgentsTraceProcessor.ts
Outdated
Show resolved
Hide resolved
Discussed. Moved all our existing tests under observability |
…IAgentsTraceProcessor.ts Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…osoft/Agent365-nodejs into users/pefan/integrationtest
Setup integration job and create initial open ai agent instrument integration tests. Also fixedcouple issues found. More tests will be added in next PR. e.g. agent to agent call.