Skip to content

Feature: 🔍 Enhanced Component Data Reading via Serialization #93

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

dsarno
Copy link

@dsarno dsarno commented Apr 10, 2025

This PR enhances the ManageGameObject tool to supercharge its ability to read component data accurately and comprehensively! 🚀 Basically, it can read serialized data from a bunch of different kinds of standard and custom components, and can be extended to work with even more components.

Summary:

  • 🔬 Enhanced Component Reading: The new serializer now uses our custom JsonConverter implementations (Vector3Converter, ColorConverter, UnityEngineObjectConverter, etc.). This means the get_components action can reliably pull detailed info for all sorts of component data types.
  • 🧹 Tidy up ManageGameObject: Extracted GameObject and Component serialization logic from the large ManageGameObject.cs into a new, dedicated GameObjectSerializer helper class (UnityMcpBridge.Editor.Helpers.GameObjectSerializer).

Enhanced Capabilities (get_components action):

The get_components action now reliably retrieves detailed information for components attached to GameObjects, including:

  • Public properties and fields ✅
  • Non-public fields marked with [SerializeField] (using includeNonPublicSerialized: true) 👀
  • Common Unity types (Vector2/3/4, Quaternion, Color, Rect, Bounds) 🎨
  • Enum values 🏷️
  • Lists of supported types 📚
  • Nested custom classes/structs 🪆🪆🪆
  • References to other Unity Objects (GameObjects, Components, Materials, Textures, etc.), resolved by direct reference, asset path, or find instructions 🔗

Examples:

  1. Get all component data for 'MCP_TestData':

    • Request:
      {
        "tool_name": "mcp_UnityMCP_manage_gameobject",
        "arguments": {
          "action": "get_components",
          "target": "MCP_TestData"
        }
      }
    • Example Response Snippet (McpTestComponent):
      {
        "typeName": "McpTestComponent",
        "instanceID": 124350, // Instance ID will vary
        "properties": {
          "publicInt": 10,
          "publicFloat": 3.14,
          "publicString": "Hello MCP",
          "publicVector3": {"x": 1.0, "y": 2.0, "z": 3.0},
          "publicColor": {"r": 1.0, "g": 0.0, "b": 0.0, "a": 1.0},
          "publicGameObjectRef": {"name": "DoorwayZone", "instanceID": 124510}, // Instance ID will vary
          "publicMaterialRef": "Assets/Art/Materials/Bun_Tan.mat",
          "publicVectorList": [{"x": 1.0, "y": 1.0, "z": 1.0}, {"x": 0.0, "y": 0.0, "z": 0.0}],
          "nestedData": {"IsEnabled": true, "MoreData": {"Name": "Nested", "Value": 456}, "IntList": [7, 8, 9]}
          // ... other properties ...
        }
      }
  2. Inspect a UI element ('InventoryPanel'):

    • Request:
      {
        "tool_name": "mcp_UnityMCP_manage_gameobject",
        "arguments": {
          "action": "get_components",
          "target": "InventoryPanel",
          "search_method": "by_name"
        }
      }
    • Example Response Snippet (Image Component):
      {
        "typeName": "UnityEngine.UI.Image",
        "instanceID": -137898, // Instance ID will vary
        "properties": {
          "sprite": {"name": "UI_Square_Frame01", "instanceID": 124818}, // Instance ID will vary
          "color": {"r": 0.19, "g": 0.19, "b": 0.19, "a": 1.0},
          "material": "Assets/Settings/URP/Resources/Materials/UI_Default.mat",
          "raycastTarget": true,
          "type": 1, // Sliced
          "pixelsPerUnitMultiplier": 1.0
          // ... other properties ...
        }
      }

Future Considerations:

  • 🌱 The GameObjectSerializer and the UnityMcpBridge.Runtime.Serialization converters provide a solid foundation. We can easily add support for more complex or custom data types in the future by creating new JsonConverter implementations, making this tool even more powerful! 💪

(Note: This PR also implicitly includes commits that added missing essential package files (.asmdef, .meta, runtime scripts) to the repository, which were necessary to resolve compilation errors when loading the package remotely via Git.)

Summary by CodeRabbit

  • New Features

    • Introduced robust serialization for Unity objects, enabling detailed data capture and smoother multi-channel responses.
    • Added enhanced JSON converters for Unity-specific types, ensuring accurate data exchange.
    • Expanded game object management functionality with an option to include additional serialized details.
  • Enhancements

    • Improved error feedback during command processing for clearer user responses.

dsarno added 5 commits April 9, 2025 19:58
…ializer helper

Moved serialization logic (GetGameObjectData, GetComponentData, metadata caching, JSON conversion helpers) from ManageGameObject tool to a dedicated GameObjectSerializer class in the Helpers namespace.

This improves separation of concerns and reduces the size/complexity of ManageGameObject.cs. Updated ManageGameObject to use the new helper class.
@dsarno
Copy link
Author

dsarno commented Apr 10, 2025

You can see the full CodeRabbit of this PR on my fork: dsarno#2

The Unity Editor was crashing with ValidTRS() assertions when attempting to get components from certain GameObjects like the Main Camera.

Investigation revealed the crash occurred during JSON serialization when reflection code accessed specific matrix properties (e.g., Camera.cullingMatrix, Transform.rotation, Transform.lossyScale). Accessing these properties appears to trigger internal Transform state validation failures, potentially due to interactions with the JSON serializer's reflection mechanism.

This fix addresses the issue by:

- Replacing LINQ iteration in GetComponentsFromTarget with a standard loop over a copied list to prevent potential premature serialization interactions.

- Explicitly skipping known problematic Camera matrix properties (cullingMatrix, pixelRect, rect) and generic matrix properties (worldToLocalMatrix, localToWorldMatrix) within GetComponentData's reflection logic.

- Retaining manual serialization for Transform component properties to avoid related reflection issues.
@dsarno
Copy link
Author

dsarno commented Apr 10, 2025

One of my Cursor cleanups deleted a bunch of @justinpbarnett 's comments in ManageGameObject.cs. I tried to restore them from a diff but there were too many and it didn't really work. We could either ask for AI help to regenerate inline comments, or I can try to rebuild the whole file based on your commented version (actually already did try that, and it wasn't that easy). Let me know what you think and thanks!

dsarno added 2 commits April 10, 2025 13:10
…sform and Camera components to prevent serialization crashes
- Add get_components to the list of available actions in the docstring

- Document required and optional parameters for get_components action

- Clarify the return value structure for get_components

- Improve overall documentation clarity for component data retrieval
@dsarno
Copy link
Author

dsarno commented Apr 16, 2025

@justinpbarnett Think you'll have a few minutes to try it? Spent a solid couple days putting it together.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant