get_runtime_output sometimes returns output from the wrong build scheme/target

[drewster99]

get_runtime_output MCP tool usually returns the runtime output of the most recent run of the app, so long as the app has been terminated before the call is made. However, in some cases, it can return the runtime output of the prior run.

I'm not positive but I believe this happened when the MCP server had last built against SchemeA, but then I ran SchemeB in Xcode. claude code called for get_runtime_output but got the results of the last SchemeA app run, rather than the SchemeB app.

[drewster99]

@claude Investigate and report

[claude]

Claude finished @drewster99's task in 1m 31s —— View job

---

Investigation Complete: Runtime Output Scheme/Target Issue

• Read and understand the get_runtime_output implementation
• Examine xcresult finding logic (find_xcresult_for_project())
• Analyze scheme/target tracking mechanism
• Identify root cause of scheme confusion
• Report findings with specific recommendations

---

Root Cause Analysis

I've identified the root cause of the issue where get_runtime_output sometimes returns output from the wrong scheme/target. The problem is in the xcresult file discovery logic.

The Issue

In xcode_mcp_server/utils/xcresult.py:469-507, the find_xcresult_for_project() function locates xcresult files based only on project name matching, not on which scheme was run:

def find_xcresult_for_project(project_path: str) -> Optional[str]:
    # ... 
    project_name = os.path.basename(normalized_path).replace('.xcworkspace', '').replace('.xcodeproj', '')
    
    # Look for directories matching the project name
    # DerivedData directories typically have format: ProjectName-randomhash
    for derived_dir in os.listdir(derived_data_base):
        # More precise matching: must start with project name followed by a dash
        if derived_dir.startswith(project_name + "-"):
            logs_dir = os.path.join(derived_data_base, derived_dir, "Logs", "Launch")
            # Find the most recent .xcresult file
            # ... returns most recent by modification time only

The function returns the most recently modified xcresult file for the project, regardless of which scheme created it.

The User's Scenario

1. MCP server runs SchemeA via run_project_with_user_interaction or run_project_until_terminated → creates xcresult file
2. User manually runs SchemeB in Xcode → creates a newer xcresult file
3. Claude Code calls get_runtime_output without a scheme parameter
4. find_xcresult_for_project() returns the most recent xcresult (from SchemeB)
5. User expects SchemeA's output but gets SchemeB's output

Why This Happens

• Xcode stores all scheme runtime logs in the same directory: ~/Library/Developer/Xcode/DerivedData/ProjectName-hash/Logs/Launch/
• xcresult filenames don't include scheme information: They have timestamps but not scheme names
• The current logic only considers project name and modification time, not scheme/target identity

Technical Details

The wait_for_xcresult_after_timestamp() function (used by run tools) partially mitigates this by checking timestamps, but get_runtime_output() doesn't use this approach—it just calls find_xcresult_for_project() which returns the most recent file by modification time, regardless of which scheme created it.

---

Recommendations

There are several potential solutions, each with trade-offs:

Option 1: Add scheme parameter to get_runtime_output (Most Robust)

• Add optional scheme parameter to get_runtime_output tool
• Parse xcresult metadata using xcrun xcresulttool to extract the scheme name
• Only return xcresult if it matches the requested scheme
• Pros: Accurate, explicit control
• Cons: Requires parsing xcresult metadata (adds complexity), requires users to specify scheme

Option 2: Track last-run scheme in server state (Stateful)

• Maintain server-side state tracking which scheme was last run via MCP tools
• get_runtime_output only returns xcresult from that scheme
• Pros: Seamless user experience
• Cons: Requires stateful server, doesn't work if user manually runs different scheme in Xcode

Option 3: Return scheme info in output (Documentation)

• Extract and include scheme name in get_runtime_output JSON response
• Let users verify they got the right scheme
• Pros: Simple, non-breaking change
• Cons: Doesn't prevent the issue, just makes it observable

Option 4: Use timestamp tracking everywhere (Partial Fix)

• Modify get_runtime_output to accept an optional after_timestamp parameter
• Have run tools return the start timestamp
• Users call get_runtime_output(after_timestamp=...) to get output from a specific run
• Pros: Works across schemes, no metadata parsing needed
• Cons: Requires coordination between tools, doesn't help with manual Xcode runs

---

Recommended Solution

Option 1 (Add scheme parameter) + Option 3 (Include scheme in output) is the most robust approach:

1. Add optional scheme parameter to get_runtime_output
2. Parse xcresult to extract scheme name using xcrun xcresulttool get log metadata --path <xcresult>
3. Filter to only return xcresult matching the scheme (if scheme specified)
4. Always include the detected scheme name in the JSON output
5. Update documentation to recommend specifying scheme when calling get_runtime_output

This provides both accuracy and transparency while maintaining backward compatibility (scheme parameter is optional).

---

[drewster99]

@claude if we include a scheme parameter -- can we do this without changing anything about the way we're fetching the runtime log? I can't tell from your recommended solution if this we are changing the underlying way of fetching the data.

[claude]

Claude finished @drewster99's task in 1m 9s —— View job

---

Investigating scheme parameter implementation details

• Review current get_runtime_output implementation
• Review xcresult fetching logic
• Determine if scheme filtering requires changing fetch mechanism
• Provide clear technical answer

---

Answer: Yes, We Can Add Scheme Filtering Without Changing the Fetch Mechanism

Short answer: We can add a scheme parameter to get_runtime_output without changing how we fetch the xcresult file. The filtering would happen AFTER fetching, by parsing the xcresult metadata to verify it matches the requested scheme.

Current Implementation

Looking at the code:

1. get_runtime_output() in tools/get_runtime_output.py:18-83:

   • Calls find_xcresult_for_project(project_path) to get the most recent xcresult
   • Passes that xcresult to extract_console_logs_from_xcresult() to parse the logs
   • Returns the console output as JSON

2. find_xcresult_for_project() in utils/xcresult.py:469-507:

   • Searches ~/Library/Developer/Xcode/DerivedData/{ProjectName}-*/Logs/Launch/
   • Returns the most recent .xcresult file by modification time
   • Does NOT look at scheme information at all

Proposed Solution (No Fetch Changes Required)

If we add a scheme parameter, here's what would happen:

@mcp.tool()
@apply_config
def get_runtime_output(project_path: str,
                      scheme: Optional[str] = None,  # NEW parameter
                      regex_filter: Optional[str] = None,
                      max_lines: int = 20) -> str:
    # Validate path (unchanged)
    project_path = validate_and_normalize_project_path(project_path, "Getting runtime output for")

    # Find xcresult (UNCHANGED - same fetch logic)
    xcresult_path = find_xcresult_for_project(project_path)
    
    if not xcresult_path:
        return "No xcresult file found..."
    
    # NEW: If scheme specified, verify this xcresult matches
    if scheme:
        actual_scheme = extract_scheme_from_xcresult(xcresult_path)  # NEW helper function
        if actual_scheme != scheme:
            return f"Found xcresult for scheme '{actual_scheme}', but expected '{scheme}'"
    
    # Extract logs (unchanged)
    success, console_output = extract_console_logs_from_xcresult(xcresult_path, regex_filter, max_lines)
    
    return console_output

The New Helper Function

We'd add a new function to extract scheme information from the xcresult:

def extract_scheme_from_xcresult(xcresult_path: str) -> Optional[str]:
    """Extract the scheme name from an xcresult bundle."""
    result = subprocess.run(
        ['xcrun', 'xcresulttool', 'get', 'log', 'metadata', '--path', xcresult_path],
        capture_output=True,
        text=True,
        timeout=10
    )
    # Parse JSON output to find scheme name
    # Return scheme name or None

Key Points

1. Fetch logic stays the same: We still use find_xcresult_for_project() which finds the most recent xcresult by modification time
2. Filtering happens AFTER: Once we have the xcresult path, we check if it matches the requested scheme
3. Backward compatible: If no scheme parameter is provided, behavior is identical to current implementation
4. Transparent: We could also always extract and include the scheme name in the output JSON (even when not filtering)

Trade-offs

Pros:

• No changes to xcresult discovery logic
• Simple validation layer
• Backward compatible
• Clear error messages when scheme doesn't match

Cons:

• If user requests SchemeA but the most recent run was SchemeB, they get an error instead of SchemeA's output
• Doesn't search for older xcresult files that might match the requested scheme

Alternative: Search Multiple xcresults

If you wanted to be more sophisticated, we COULD change the fetch logic to search through multiple xcresult files to find one matching the scheme:

def find_xcresult_for_project_and_scheme(project_path: str, scheme: Optional[str] = None):
    # Get ALL xcresult files, not just the most recent
    all_xcresults = get_all_xcresult_files(project_path)
    
    if not scheme:
        return all_xcresults[0]  # Most recent
    
    # Search through xcresults to find one matching the scheme
    for xcresult in all_xcresults:
        if extract_scheme_from_xcresult(xcresult) == scheme:
            return xcresult
    
    return None

But this would be changing the fetch mechanism, which you wanted to avoid.

---

Recommendation: Start with the simple approach (validate scheme after fetching). It adds the feature without changing the core fetch logic, and we can always enhance it later if needed.

---