## Pre-Launch Enhancements: Testing Infrastructure & Tool Improvements #8
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Scripts for running the NL/T/GO test suites locally against a GUI Unity Editor, complementing the CI workflows in .github/workflows/. Benefits: - 10-100x faster than CI (no Docker startup) - Real-time Unity console debugging - Single test execution for rapid iteration - Auto-detects HTTP vs stdio transport Usage: ./scripts/local-test/setup.sh # One-time setup ./scripts/local-test/quick-test.sh NL-0 # Run single test ./scripts/local-test/run-nl-suite-local.sh # Full suite See scripts/local-test/README.md for details. Also updated .gitignore to: - Allow scripts/local-test/ to be tracked - Ignore generated artifacts (reports/*.xml, .claude/local/, .unity-mcp/)
Move SaveDirtyScenesIfNeeded() call outside the PlayMode conditional so EditMode tests don't get blocked by Unity's "Save Scene" modal dialog. This prevents MCP from timing out when running EditMode tests with unsaved scene changes.
…ure/local-test-harness
The FAST_FAIL_TIMEOUT class attribute was referenced on line 149 but never defined, causing AttributeError on every ping attempt. This error was silently caught by the broad 'except Exception' handler, causing all fast-fail commands (read_console, get_editor_state, ping) to fail after 6 seconds of retries with 'ping not answered' error. Added FAST_FAIL_TIMEOUT = 10 to define a 10-second timeout for fast-fail commands, matching the intent of the existing fast-fail infrastructure.
… and Quaternion
Dry-run validation now validates value formats, not just property existence:
- AnimationCurve: Validates structure ({keys:[...]} or direct array), checks
each keyframe is an object, validates numeric fields (time, value, inSlope,
outSlope, inWeight, outWeight) and integer fields (weightedMode)
- Quaternion: Validates array length (3 for Euler, 4 for raw) or object
structure ({x,y,z,w} or {euler:[x,y,z]}), ensures all components are numeric
Refactored shared validation helpers into appropriate locations:
- ParamCoercion: IsNumericToken, ValidateNumericField, ValidateIntegerField
- VectorParsing: ValidateAnimationCurveFormat, ValidateQuaternionFormat
Added comprehensive XML documentation clarifying keyframe field defaults
(all default to 0 except as noted).
Added 5 new dry-run validation tests covering valid and invalid formats
for both AnimationCurve and Quaternion properties.
Resolved conflict in plugin_hub.py: both branches added FAST_FAIL_TIMEOUT (fixing the same missing constant bug). Kept msanatan's version (2.0s with better comment) as it's more appropriate for fast-fail behavior.
- test_refresh_unity_retry_recovery: Mock now handles both refresh_unity and get_editor_state commands (refresh_unity internally calls get_editor_state when wait_for_ready=True) - test_run_tests_async_forwards_params: Mock response now includes required 'mode' field for RunTestsStartResponse Pydantic validation - test_get_test_job_forwards_job_id: Updated to handle GetTestJobResponse as Pydantic model instead of dict (use model_dump() for assertions)
Follow-up to PR CoplayDev#527: Since SaveDirtyScenesIfNeeded() now runs for all test modes, update the warning message to say 'tests' instead of 'PlayMode tests'.
…p detection When polling for test completion, MCP clients like Cursor can detect the repeated get_test_job calls as 'looping' and terminate the agent. Added wait_timeout parameter that makes the server wait internally for tests to complete (polling Unity every 2s) before returning. This dramatically reduces client-side tool calls from 10-20 down to 1-2, avoiding loop detection. Usage: get_test_job(job_id='xxx', wait_timeout=30) - Returns immediately if tests complete within timeout - Returns current status if timeout expires (client can call again) - Recommended: 30-60 seconds
|
Caution Review failedFailed to post review comments 📝 WalkthroughWalkthroughThis PR comprehensively restructures MCPForUnity's architecture by migrating all resource URIs from Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor Client
participant Server as MCP Server
participant Transport
participant UnityEditor as Unity Editor
Note over Client,UnityEditor: Async Test Execution Flow (New)
Client->>Server: run_tests(mode, filters...)
Server->>Transport: send_with_unity_instance("run_tests", params)
Transport->>UnityEditor: start test job → job_id
UnityEditor-->>Transport: { success: true, job_id, mode }
Transport-->>Server: job_id
Server-->>Client: { job_id, mode, status: "started" }
Note over Client: Poll for results
Client->>Server: get_test_job(job_id, wait_timeout?)
Server->>Transport: send_with_unity_instance("get_test_job", job_id)
Transport->>UnityEditor: query job status
alt Tests Still Running
UnityEditor-->>Transport: { success: true, job_id, status: "running" }
else Tests Complete
UnityEditor-->>Transport: { success: true, job_id, status: "completed", result }
end
Transport-->>Server: status/result
Server-->>Client: GetTestJobResponse
sequenceDiagram
autonumber
participant Client
participant Server as ManageGameObject
participant Resolvers as Helpers
participant UnityEditor as Unity Editor
Note over Client,UnityEditor: Modular GameObject Tool Flow (New)
Client->>Server: HandleCommand({ action: "create", name, ... })
alt action == "create"
Server->>Resolvers: GameObjectCreate.Handle(params)
Resolvers->>Resolvers: Validate name, resolve prefab/parent
Resolvers->>UnityEditor: Undo + instantiate/create
Resolvers->>Resolvers: AddComponentInternal (via ComponentHelpers)
Resolvers->>Resolvers: SetComponentPropertiesInternal (via GameObjectComponentHelpers)
Resolvers-->>Server: SuccessResponse with serialized object
else action == "modify"
Server->>Resolvers: GameObjectModify.Handle(params)
Resolvers->>Resolvers: Find target via ManageGameObjectCommon
Resolvers->>UnityEditor: Apply transform/properties/components
Resolvers-->>Server: SuccessResponse or ErrorResponse
else action == "delete"
Server->>Resolvers: GameObjectDelete.Handle(params)
Resolvers->>Resolvers: Find targets via ManageGameObjectCommon
Resolvers->>UnityEditor: Undo.DestroyObjectImmediate
Resolvers-->>Server: SuccessResponse with deleted list
end
Server-->>Client: SuccessResponse | ErrorResponse
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
msanatan
approved these changes
Jan 7, 2026
msanatan
added a commit
that referenced
this pull request
Jan 7, 2026
* refactor: Split ParseColorOrDefault into two overloads and change default to Color.white * Auto-format Python code * Remove unused Python module * Refactored VFX functionality into multiple files Tested everything, works like a charm * Rename ManageVfx folder to just Vfx We know what it's managing * Clean up whitespace on plugin tools and resources * Make ManageGameObject less of a monolith by splitting it out into different files * Remove obsolete FindObjectByInstruction method We also update the namespace for ManageVFX * refactor: Consolidate editor state resources into single canonical implementation Merged EditorStateV2 into EditorState, making get_editor_state the canonical resource. Updated Unity C# to use EditorStateCache directly. Enhanced Python implementation with advice/staleness enrichment, external changes detection, and instance ID inference. Removed duplicate EditorStateV2 resource and legacy fallback mapping. * Validate editor state with Pydantic models in both C# and Python Added strongly-typed Pydantic models for EditorStateV2 schema in Python and corresponding C# classes with JsonProperty attributes. Updated C# to serialize using typed classes instead of anonymous objects. Python now validates the editor state payload before returning it, catching schema mismatches early. * Consolidate run_tests and run_tests_async into single async implementation Merged run_tests_async into run_tests, making async job-based execution the default behavior. Removed synchronous blocking test execution. Updated RunTests.cs to start test jobs immediately and return job_id for polling. Changed TestJobManager methods to internal visibility. Updated README to reflect single run_tests_async tool. Python implementation now uses async job pattern exclusively. * Validate test job responses with Pydantic models in Python * Change resources URI from unity:// to mcpforunity:// It should reduce conflicts with other Unity MCPs that users try, and to comply with Unity's requests regarding use of their company and product name * Update README with all tools + better listing for resources * Update other references to resources * Updated translated doc - unfortunately I cannot verify * Update the Chinese translation of the dev docks * Change menu item from Setup Window to Local Setup Window We now differentiate whether it's HTTP local or remote * Fix URIs for menu items and tests * Shouldn't have removed it * Minor edits from CodeRabbit feedback * Don't use reflection which takes longer * Fix failing python tests * Add serialization helpers for ParticleSystem curves and MinMaxCurve types Added SerializeAnimationCurve and SerializeMinMaxCurve helper methods to properly serialize Unity's curve types. Updated GetInfo to use these helpers for startLifetime, startSpeed, startSize, gravityModifier, and rateOverTime instead of only reading constant values. * Use ctx param * Update Server/src/services/tools/run_tests.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Minor fixes * Rename anything EditorStateV2 to just EditorState It's the default, there's no old version * Make infer_single_instance_id public by removing underscore prefix * Fix Python tests, again * Replace AI generated .meta files with actual Unity ones * ## Pre-Launch Enhancements: Testing Infrastructure & Tool Improvements (#8) * Add local test harness for fast developer iteration Scripts for running the NL/T/GO test suites locally against a GUI Unity Editor, complementing the CI workflows in .github/workflows/. Benefits: - 10-100x faster than CI (no Docker startup) - Real-time Unity console debugging - Single test execution for rapid iteration - Auto-detects HTTP vs stdio transport Usage: ./scripts/local-test/setup.sh # One-time setup ./scripts/local-test/quick-test.sh NL-0 # Run single test ./scripts/local-test/run-nl-suite-local.sh # Full suite See scripts/local-test/README.md for details. Also updated .gitignore to: - Allow scripts/local-test/ to be tracked - Ignore generated artifacts (reports/*.xml, .claude/local/, .unity-mcp/) * Fix issue CoplayDev#525: Save dirty scenes for all test modes Move SaveDirtyScenesIfNeeded() call outside the PlayMode conditional so EditMode tests don't get blocked by Unity's "Save Scene" modal dialog. This prevents MCP from timing out when running EditMode tests with unsaved scene changes. * fix: add missing FAST_FAIL_TIMEOUT constant in PluginHub The FAST_FAIL_TIMEOUT class attribute was referenced on line 149 but never defined, causing AttributeError on every ping attempt. This error was silently caught by the broad 'except Exception' handler, causing all fast-fail commands (read_console, get_editor_state, ping) to fail after 6 seconds of retries with 'ping not answered' error. Added FAST_FAIL_TIMEOUT = 10 to define a 10-second timeout for fast-fail commands, matching the intent of the existing fast-fail infrastructure. * feat(ScriptableObject): enhance dry-run validation for AnimationCurve and Quaternion Dry-run validation now validates value formats, not just property existence: - AnimationCurve: Validates structure ({keys:[...]} or direct array), checks each keyframe is an object, validates numeric fields (time, value, inSlope, outSlope, inWeight, outWeight) and integer fields (weightedMode) - Quaternion: Validates array length (3 for Euler, 4 for raw) or object structure ({x,y,z,w} or {euler:[x,y,z]}), ensures all components are numeric Refactored shared validation helpers into appropriate locations: - ParamCoercion: IsNumericToken, ValidateNumericField, ValidateIntegerField - VectorParsing: ValidateAnimationCurveFormat, ValidateQuaternionFormat Added comprehensive XML documentation clarifying keyframe field defaults (all default to 0 except as noted). Added 5 new dry-run validation tests covering valid and invalid formats for both AnimationCurve and Quaternion properties. * test: fix integration tests after merge - test_refresh_unity_retry_recovery: Mock now handles both refresh_unity and get_editor_state commands (refresh_unity internally calls get_editor_state when wait_for_ready=True) - test_run_tests_async_forwards_params: Mock response now includes required 'mode' field for RunTestsStartResponse Pydantic validation - test_get_test_job_forwards_job_id: Updated to handle GetTestJobResponse as Pydantic model instead of dict (use model_dump() for assertions) * Update warning message to apply to all test modes Follow-up to PR CoplayDev#527: Since SaveDirtyScenesIfNeeded() now runs for all test modes, update the warning message to say 'tests' instead of 'PlayMode tests'. * feat(run_tests): add wait_timeout to get_test_job to avoid client loop detection When polling for test completion, MCP clients like Cursor can detect the repeated get_test_job calls as 'looping' and terminate the agent. Added wait_timeout parameter that makes the server wait internally for tests to complete (polling Unity every 2s) before returning. This dramatically reduces client-side tool calls from 10-20 down to 1-2, avoiding loop detection. Usage: get_test_job(job_id='xxx', wait_timeout=30) - Returns immediately if tests complete within timeout - Returns current status if timeout expires (client can call again) - Recommended: 30-60 seconds * fix: use Pydantic attribute access in test_run_tests_async for merge compatibility * revert: remove local test harness - will be submitted in separate PR --------- Co-authored-by: Scott Jennings <scott.jennings+CIGINT@cloudimperiumgames.com> --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: dsarno <david@lighthaus.us> Co-authored-by: Scott Jennings <scott.jennings+CIGINT@cloudimperiumgames.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds critical testing infrastructure improvements and tool enhancements to prepare for launch. All changes merge cleanly with no conflicts and pass all test suites (269 Unity EditMode tests, 118 Python integration tests).
Key Features
🚀 Server-Side Test Polling (
wait_timeoutparameter)get_test_jobrapidly were triggering loop detection warningswait_timeoutparameter (30-60s recommended) toget_test_jobtoolget_test_job(job_id="...", wait_timeout=60)- returns immediately if tests finish early🛠️ ScriptableObject Tool Enhancements
AnimationCurveandQuaterniontypesManageScriptableObjectStressTests)💾 Test Mode Improvements (Fix CoplayDev#525)
🐛 Bug Fixes
FAST_FAIL_TIMEOUTconstant in PluginHubTesting
Files Changed
run_tests.py,ManageScriptableObject.cs,TestRunnerService.cs,ParamCoercion.cs,VectorParsing.csManageScriptableObjectStressTests.cs(279 comprehensive tests)Breaking Changes
None - all changes are backward compatible
Dependencies
No new dependencies added
Ready to merge ✅
Summary by CodeRabbit
Release Notes
New Features
Bug Fixes
✏️ Tip: You can customize this high-level summary in your review settings.