Tags: github/copilot-sdk
Tags
Add E2E scenario tests/examples for all SDK languages (#512) * Commit base e2e across SDKs * Add C# samples to 8 representative scenarios Add csharp/ subdirectories with Program.cs and csproj for: - transport/stdio, transport/tcp - bundling/fully-bundled - tools/no-tools, tools/custom-agents - sessions/streaming - callbacks/permissions - prompts/system-message All samples reference the local .NET SDK via ProjectReference. Each verify.sh updated with dotnet build/run steps. Added C# build artifacts to .gitignore. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix go.mod replace paths for test scenarios All 30 go.mod files had incorrect relative paths to the SDK Go module. Scenarios live at test/scenarios/<cat>/<scenario>/go/ (5 levels deep), so the replace directive needs ../../../../../go instead of ../../../../go. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Replace copilot-core with Copilot CLI across all test scenarios copilot-core is not a product name. Updated 63 files to use the correct terminology: - Prose/comments: "Copilot CLI" - Binary name in code/commands: copilot - COPILOT_CLI_PATH env var: unchanged (already correct) - Dockerfile ENTRYPOINT/COPY: copilot - docker-compose service name: copilot-cli Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add scenario build verification workflow for PR checks Runs build verification for all test/scenarios across 4 languages (TypeScript, Python, Go, C#) as parallel jobs. Triggered on PRs that touch scenarios or SDK source, and on push to main. Build-only — no E2E execution (no API keys or Copilot CLI needed). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add full language parity: all 34 scenarios × 4 languages Every scenario now has TypeScript, Python, Go, and C# implementations. New additions: - C# for 26 scenarios (auth, bundling, callbacks, modes, prompts, sessions, tools, transport) - Python + Go + C# for 3 BYOK scenarios (anthropic, azure, ollama) - Python + Go + C# stubs for multi-user scenarios (SKIP pattern) All 136 scenario builds verified: 34 TS, 34 PY, 34 GO, 34 CS. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add CI caching and justfile targets for scenario builds - Add npm, Go module, and NuGet caching to scenario-builds.yml - Add just scenario-build, scenario-verify, scenario-build-lang targets Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Quality fixes: C# in all verify.sh, fix stubs, consistent patterns - Add C# build/run steps to all 26 verify.sh scripts that were missing them - Replace infinite-sessions TS stub with real implementation - Rework modes: remove filesystem-preset, rename cli-preset→default and minimal-preset→minimal with real implementations in all 4 languages - Fix C# patterns across all 33 scenarios: consistent await using, StartAsync/StopAsync, proper disposal 33 scenarios × 4 languages = 132 builds, all passing. 2 multi-user scenarios remain as stubs (require memory FS features). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Strengthen Python CI: py_compile + import check instead of AST-only Install the Python SDK and use py_compile for proper compilation checking, plus verify the copilot module is importable. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Update scenario model references to claude-sonnet-4.6 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: remove soft-pass fallbacks in verify.sh scenario scripts Replace else/elif fallback branches that always passed with proper failure reporting. Previously, when the expected grep pattern wasn't found, the test would emit a warning but still count as passed. Now these cases correctly report failure and increment the FAIL counter. Changed 18 files with 21 soft-pass fixes across: - 10 standard else-branch fallbacks (modes, prompts, tools, sessions) - 3 elif-branch fallbacks (callbacks, streaming, virtual-filesystem) - 3 multi-branch fixes with both 'partial' and 'got response' fallbacks (concurrent-sessions, multi-user-long-lived, multi-user-short-lived) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Strengthen tools scenario verifications - tool-filtering: use word-boundary grep (-w) for blacklisted tools to avoid false positives on substrings like 'bashing' - no-tools: change question to directly request bash tool usage; update verify grep to check for inability patterns (can't, cannot, unable) - virtual-filesystem: require both 'Virtual filesystem contents' AND 'plan.md' in output; fix dead elif branch - custom-agents: tighten grep to only match 'researcher' or 'Research' instead of also matching generic tool names - skills: add lowercase 'skill' to grep pattern for broader matching - mcp-servers: replace soft-pass (non-empty output) with meaningful content grep; add separate failure message for pattern mismatch Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Support latest .NET * Move to haiku * Fix scenario tests: paths, verifications, streaming, and parallel execution - Fix relative paths in TS package.json (43 files) and Python requirements.txt (33 files) - Add RollForward to C# csproj files for .NET 8/10 compat - Remove soft-pass fallbacks in verify.sh — tests now hard-fail on missing patterns - Fix Go permissions bug (req.Kind → req.ToolName) and add ToolName to SDK type - Fix Python/Go availableTools: empty list was omitted instead of sent as [] - Fix streaming event names (assistant.message.chunk → assistant.message_delta) - Fix TS streaming subscription (session.on('event') → typed subscription) - Fix Python streaming enum comparison (event.type.value) - Add permission handlers to Go skills scenario - Switch scenarios to claude-haiku-4.5 for faster execution - Parallelize verify.sh with live progress bar and per-SDK status icons - Fix parallel pip install race with pre-install and import check - Remove go.sum files from tracking - Remove hardcoded OAuth client ID from C# gh-app scenario Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Restore go.sum files needed for CI builds Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Revert ToolName addition to Go PermissionRequest — use Extra map instead The ToolName field doesn't exist on PermissionRequest in other SDKs. The scenario test now reads toolName from the Extra map to stay consistent without modifying SDK types. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: use o4-mini for reasoning-effort scenario tests claude-haiku-4.5 does not support the reasoningEffort configuration, causing all 4 SDK scenario tests to fail. Switch to o4-mini which supports reasoning effort. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add E2E scenario tests/examples for all SDK languages (#512) * Commit base e2e across SDKs * Add C# samples to 8 representative scenarios Add csharp/ subdirectories with Program.cs and csproj for: - transport/stdio, transport/tcp - bundling/fully-bundled - tools/no-tools, tools/custom-agents - sessions/streaming - callbacks/permissions - prompts/system-message All samples reference the local .NET SDK via ProjectReference. Each verify.sh updated with dotnet build/run steps. Added C# build artifacts to .gitignore. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix go.mod replace paths for test scenarios All 30 go.mod files had incorrect relative paths to the SDK Go module. Scenarios live at test/scenarios/<cat>/<scenario>/go/ (5 levels deep), so the replace directive needs ../../../../../go instead of ../../../../go. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Replace copilot-core with Copilot CLI across all test scenarios copilot-core is not a product name. Updated 63 files to use the correct terminology: - Prose/comments: "Copilot CLI" - Binary name in code/commands: copilot - COPILOT_CLI_PATH env var: unchanged (already correct) - Dockerfile ENTRYPOINT/COPY: copilot - docker-compose service name: copilot-cli Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add scenario build verification workflow for PR checks Runs build verification for all test/scenarios across 4 languages (TypeScript, Python, Go, C#) as parallel jobs. Triggered on PRs that touch scenarios or SDK source, and on push to main. Build-only — no E2E execution (no API keys or Copilot CLI needed). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add full language parity: all 34 scenarios × 4 languages Every scenario now has TypeScript, Python, Go, and C# implementations. New additions: - C# for 26 scenarios (auth, bundling, callbacks, modes, prompts, sessions, tools, transport) - Python + Go + C# for 3 BYOK scenarios (anthropic, azure, ollama) - Python + Go + C# stubs for multi-user scenarios (SKIP pattern) All 136 scenario builds verified: 34 TS, 34 PY, 34 GO, 34 CS. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add CI caching and justfile targets for scenario builds - Add npm, Go module, and NuGet caching to scenario-builds.yml - Add just scenario-build, scenario-verify, scenario-build-lang targets Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Quality fixes: C# in all verify.sh, fix stubs, consistent patterns - Add C# build/run steps to all 26 verify.sh scripts that were missing them - Replace infinite-sessions TS stub with real implementation - Rework modes: remove filesystem-preset, rename cli-preset→default and minimal-preset→minimal with real implementations in all 4 languages - Fix C# patterns across all 33 scenarios: consistent await using, StartAsync/StopAsync, proper disposal 33 scenarios × 4 languages = 132 builds, all passing. 2 multi-user scenarios remain as stubs (require memory FS features). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Strengthen Python CI: py_compile + import check instead of AST-only Install the Python SDK and use py_compile for proper compilation checking, plus verify the copilot module is importable. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Update scenario model references to claude-sonnet-4.6 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: remove soft-pass fallbacks in verify.sh scenario scripts Replace else/elif fallback branches that always passed with proper failure reporting. Previously, when the expected grep pattern wasn't found, the test would emit a warning but still count as passed. Now these cases correctly report failure and increment the FAIL counter. Changed 18 files with 21 soft-pass fixes across: - 10 standard else-branch fallbacks (modes, prompts, tools, sessions) - 3 elif-branch fallbacks (callbacks, streaming, virtual-filesystem) - 3 multi-branch fixes with both 'partial' and 'got response' fallbacks (concurrent-sessions, multi-user-long-lived, multi-user-short-lived) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Strengthen tools scenario verifications - tool-filtering: use word-boundary grep (-w) for blacklisted tools to avoid false positives on substrings like 'bashing' - no-tools: change question to directly request bash tool usage; update verify grep to check for inability patterns (can't, cannot, unable) - virtual-filesystem: require both 'Virtual filesystem contents' AND 'plan.md' in output; fix dead elif branch - custom-agents: tighten grep to only match 'researcher' or 'Research' instead of also matching generic tool names - skills: add lowercase 'skill' to grep pattern for broader matching - mcp-servers: replace soft-pass (non-empty output) with meaningful content grep; add separate failure message for pattern mismatch Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Support latest .NET * Move to haiku * Fix scenario tests: paths, verifications, streaming, and parallel execution - Fix relative paths in TS package.json (43 files) and Python requirements.txt (33 files) - Add RollForward to C# csproj files for .NET 8/10 compat - Remove soft-pass fallbacks in verify.sh — tests now hard-fail on missing patterns - Fix Go permissions bug (req.Kind → req.ToolName) and add ToolName to SDK type - Fix Python/Go availableTools: empty list was omitted instead of sent as [] - Fix streaming event names (assistant.message.chunk → assistant.message_delta) - Fix TS streaming subscription (session.on('event') → typed subscription) - Fix Python streaming enum comparison (event.type.value) - Add permission handlers to Go skills scenario - Switch scenarios to claude-haiku-4.5 for faster execution - Parallelize verify.sh with live progress bar and per-SDK status icons - Fix parallel pip install race with pre-install and import check - Remove go.sum files from tracking - Remove hardcoded OAuth client ID from C# gh-app scenario Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Restore go.sum files needed for CI builds Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Revert ToolName addition to Go PermissionRequest — use Extra map instead The ToolName field doesn't exist on PermissionRequest in other SDKs. The scenario test now reads toolName from the Extra map to stay consistent without modifying SDK types. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: use o4-mini for reasoning-effort scenario tests claude-haiku-4.5 does not support the reasoningEffort configuration, causing all 4 SDK scenario tests to fail. Switch to o4-mini which supports reasoning effort. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fix MCP env vars: send envValueMode direct across all SDKs (#484) * Use MCP direct env var mode in Node SDK * Equivalent fixes for other languages * Update @github/copilot to 0.0.411-1 and regenerate SDKs Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix Python lint errors (line too long) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add normalizer to strip <reminder> tags from user messages The CLI now injects dynamic <reminder> tags with SQL table state into user messages. These tags vary based on runtime state and should not affect snapshot matching. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fix MCP env vars: send envValueMode direct across all SDKs (#484) * Use MCP direct env var mode in Node SDK * Equivalent fixes for other languages * Update @github/copilot to 0.0.411-1 and regenerate SDKs Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix Python lint errors (line too long) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add normalizer to strip <reminder> tags from user messages The CLI now injects dynamic <reminder> tags with SQL table state into user messages. These tags vary based on runtime state and should not affect snapshot matching. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fix Available SDKs table Location links to point to SDK subfolders (#486 ) The Location column was linking to external cookbook READMEs in github/awesome-copilot instead of the SDK subfolders in this repo. - Changed Location links to point to SDK subfolders (nodejs/, python/, go/, dotnet/) - Added separate Cookbook column for external cookbook links Fixes #474 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fix Available SDKs table Location links to point to SDK subfolders (#486 ) The Location column was linking to external cookbook READMEs in github/awesome-copilot instead of the SDK subfolders in this repo. - Changed Location links to point to SDK subfolders (nodejs/, python/, go/, dotnet/) - Added separate Cookbook column for external cookbook links Fixes #474 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
fix: set executable permission on CLI binary in Python wheel (#419) * fix: set executable permission on CLI binary in Python wheel The repack step in build-wheels.mjs was producing wheels where copilot/bin/copilot had 0o644 (-rw-r--r--) permissions because setuptools strips the executable bit when building package data. After extracting the wheel for repack, restore chmod 0o755 on the CLI binary so that zf.write() captures the correct permissions. This ensures pip/uv install the binary as executable on Unix. * Update python/scripts/build-wheels.mjs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
fix: set executable permission on CLI binary in Python wheel (#419) * fix: set executable permission on CLI binary in Python wheel The repack step in build-wheels.mjs was producing wheels where copilot/bin/copilot had 0o644 (-rw-r--r--) permissions because setuptools strips the executable bit when building package data. After extracting the wheel for repack, restore chmod 0o755 on the CLI binary so that zf.write() captures the correct permissions. This ensures pip/uv install the binary as executable on Unix. * Update python/scripts/build-wheels.mjs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
PreviousNext