Skip to content

Setup integration test job and create initial OpenAI agent instrument integration tests#31

Merged
fpfp100 merged 16 commits intomainfrom
users/pefan/integrationtest
Nov 6, 2025
Merged

Setup integration test job and create initial OpenAI agent instrument integration tests#31
fpfp100 merged 16 commits intomainfrom
users/pefan/integrationtest

Conversation

@fpfp100
Copy link
Contributor

@fpfp100 fpfp100 commented Nov 5, 2025

Setup integration job and create initial open ai agent instrument integration tests. Also fixedcouple issues found. More tests will be added in next PR. e.g. agent to agent call.

@fpfp100 fpfp100 changed the title Create initial instrument integration test Setup integration job and create initial open ai agent instrument integration tests Nov 5, 2025
@fpfp100 fpfp100 changed the title Setup integration job and create initial open ai agent instrument integration tests Setup integration test job and create initial open ai agent instrument integration tests Nov 5, 2025
@fpfp100 fpfp100 changed the title Setup integration test job and create initial open ai agent instrument integration tests Setup integration test job and create initial OpenAI agent instrument integration tests Nov 5, 2025
Copy link
Contributor

@nikhilNava nikhilNava left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets move the tests under observability, lets follow the pattern on the python SDK. thanks

@fpfp100 fpfp100 marked this pull request as ready for review November 6, 2025 07:27
@fpfp100 fpfp100 requested a review from a team as a code owner November 6, 2025 07:28
Copilot AI review requested due to automatic review settings November 6, 2025 07:28
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds integration testing infrastructure for OpenAI agent instrumentation, introduces unit tests for observability components, and implements necessary fixes to support tracing generation spans. The changes establish a comprehensive testing strategy separating unit tests from integration tests that require live Azure OpenAI credentials.

  • Integration test setup with Azure OpenAI configuration management
  • Unit tests for core observability components (scopes, processors, baggage)
  • Enhanced span processing for generation input/output messages
  • Separated test execution between unit and integration tests

Reviewed Changes

Copilot reviewed 12 out of 19 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tests/package.json Added integration test scripts and required dependencies (dotenv, openai, zod, @openai/agents-openai)
tests/observability/integration/setup.ts Global setup file for integration tests with environment configuration
tests/observability/integration/openai-agent-instrument.test.ts Integration tests validating OpenAI agent instrumentation with live API calls
tests/observability/integration/conftest.ts Configuration utilities for Azure OpenAI and Agent365 environment variables
tests/observability/integration/.env.example Example environment file template for integration test credentials
tests/observability/extension/openai/OpenAIAgentsTraceProcessor.test.ts Unit tests for trace processor lifecycle and span handling
tests/observability/extension/openai/OpenAIAgentsTraceInstrumentor.test.ts Unit tests for instrumentor enable/disable lifecycle
tests/observability/core/scopes.test.ts Unit tests for InvokeAgentScope, ExecuteToolScope, and InferenceScope
tests/observability/core/observabilityManager.test.ts Unit tests for ObservabilityManager configuration and builder pattern
tests/observability/core/SpanProcessor.test.ts Unit tests for baggage-to-span attribute enrichment
tests/observability/core/BaggageBuilder.test.ts Unit tests for BaggageBuilder fluent API and scope management
tests-agent/openai-agent-auto-instrument-sample/package.json Added integration test scripts and Jest dependencies to sample package
pnpm-lock.yaml Lockfile updates for new dependencies (dotenv, openai, node-fetch, etc.)
packages/.../OpenAIAgentsTraceProcessor.ts Fixed generation span processing and added root span tracking
packages/.../Constants.ts Added GEN_AI_EXECUTION_PAYLOAD_KEY constant
packages/.../jest.config.json Updated test roots path to new observability directory structure
package.json Added integration test scripts at root level with path ignoring
jest.integration.config.cjs New Jest configuration for integration tests with 60s timeout
.github/workflows/ci.yml Added integration test step to CI workflow with Azure credentials
Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

@fpfp100
Copy link
Contributor Author

fpfp100 commented Nov 6, 2025

Lets move the tests under observability, lets follow the pattern on the python SDK. thanks

Discussed. Moved all our existing tests under observability

…IAgentsTraceProcessor.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings November 6, 2025 19:04
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 12 out of 19 changed files in this pull request and generated no new comments.

Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

Copilot AI review requested due to automatic review settings November 6, 2025 19:53
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 12 out of 19 changed files in this pull request and generated 10 comments.

Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

@fpfp100 fpfp100 merged commit df65925 into main Nov 6, 2025
7 checks passed
@pontemonti pontemonti deleted the users/pefan/integrationtest branch December 5, 2025 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants