Skip to content

Conversation

@ymanor2404
Copy link
Contributor

Updating Ryan the UX research agent to know to access the Google Workspace MCP Server, specifically the research studies folder, to use as context for research-informed requirements.

Updating Ryan the UX research agent to know to access the Google Workspace MCP Server, specifically the research studies folder, to use as context for research-informed requirements.
@github-actions

This comment has been minimized.

@github-actions
Copy link
Contributor

github-actions bot commented Dec 15, 2025

Claude Code Review

Summary

This PR updates the Ryan (UX Researcher) agent configuration to integrate with Google Workspace MCP Server for accessing UX research studies. The changes add significant new instructions for research-informed RFE generation with mandatory citation requirements.

Overall Assessment: ✅ The changes are functional and align with the agent's purpose, but there are several quality, maintainability, and security concerns that should be addressed.


Issues by Severity

🟡 Major Issues

1. Hardcoded Google Drive URL with Unclear Access Control

  • Location: agents/ryan-ux_researcher.md:11
  • Issue: The hardcoded Google Drive folder URL (https://drive.google.com/drive/folders/1yW2GbqKThAskAAKA1UodTWqMzWZbVBo1?usp=drive_link) is embedded directly in the agent prompt without documentation about access permissions
  • Impact:
    • If the folder permissions change, the agent will fail
    • No visibility into who can access this folder
    • May expose internal research data if folder permissions are misconfigured
  • Recommendation:
    • Document the expected folder permissions in a comment
    • Consider adding error handling instructions for when the folder is inaccessible
    • Add a note in the PR description or documentation about folder access requirements

2. Excessive Use of ALL CAPS and Aggressive Tone

  • Locations: Lines 10-12, 17, 23, 24
  • Issue: Heavy use of ALL CAPS ("MUST", "CRITICAL", "ESSENTIAL", "DO NOT", "SOLELY", "EVERY TIME") creates an aggressive tone that may not translate well to agent behavior
  • Impact:
    • May cause the LLM to be overly rigid or defensive
    • Reduces readability for humans reviewing the agent configuration
    • Goes against professional objectivity guidance in CLAUDE.md
  • Recommendation: Use standard instructional language. LLMs respond well to clear, direct instructions without shouting. For example:
    • "MUST leverage this folder" → "Always leverage this folder"
    • "IT IS ESSENTIAL" → "It is essential" or "Required:"
    • "DO NOT PULL IN DATA" → "Do not pull in data" or "Restriction: Only use data from..."

3. Removed WebSearch Tool Without Documentation

  • Location: Line 4 (tools field)
  • Issue: WebSearch was removed from the tools list, but the rationale isn't documented in the PR description or code
  • Impact: This is a significant capability change that should be explained, especially since lines 12 say "DO NOT PULL IN DATA FROM ANY WEB SEARCH TOOL"
  • Recommendation: Either:
    • Document in the PR description why WebSearch was removed
    • Add a comment in the file explaining this restriction
    • Consider if there are edge cases where web search might still be valuable

4. Potential Agent Overload with Conflicting Instructions

  • Locations: Lines 10-12 (restrictive) vs existing research methodology (lines 30-150)
  • Issue: The new strict instructions ("ONLY REFERENCE THE RESEARCH STUDIES FOLDER", "your ONLY context") may conflict with the agent's role as a UX researcher who should synthesize multiple data sources
  • Impact: Agent may refuse to provide valuable UX insights that aren't directly in the research folder
  • Recommendation: Clarify the priority: "When creating RFEs, prioritize research from the 'All UXR Reports' folder. If relevant research doesn't exist, clearly state this and recommend conducting new research before proceeding."

🔵 Minor Issues

5. Missing Documentation About MCP Server Configuration

  • Location: Line 10
  • Issue: References "the Google Workspace MCP server which can be found within the file" but doesn't specify which file or how to configure it
  • Recommendation: Add a comment or reference to where MCP server setup is documented

6. Example Uses Future Date

  • Location: Line 20
  • Issue: Example cites "AI Engineer Workflows Q3 2025 Study" - may cause confusion if this is a placeholder
  • Recommendation: Use a generic example like "Q3 2024 Study" or "Product Usability Study [Month Year]"

7. Inconsistent Formatting

  • Locations: Multiple blank lines (line 9, 14, trailing blank lines removed)
  • Issue: Inconsistent spacing reduces readability
  • Recommendation: Follow markdown best practices: one blank line between sections

8. Unclear Error Behavior

  • Location: Lines 22-25
  • Issue: What should the agent do if it finds partial research but not enough to fully support an RFE?
  • Recommendation: Add guidance: "If partial research exists but is insufficient, summarize what is available and identify specific research gaps that should be addressed."

Positive Highlights

Clear Purpose: The addition of research-informed RFE generation with citation requirements is a valuable enhancement

Concrete Examples: The dark mode toggle example (lines 18-20) provides clear guidance for expected output format

User Advocacy: The instruction to disagree when research doesn't support a request (lines 22-25) demonstrates good UX research practices

Preserved Core Knowledge: The existing UX research methodology (lines 30-150) remains intact, providing comprehensive context


Recommendations (Prioritized)

High Priority

  1. Soften the tone: Replace ALL CAPS with standard case for better readability and more professional agent behavior
  2. Document the Google Drive URL: Add access control expectations and error handling guidance
  3. Clarify the restriction scope: Make it clear when the research folder restriction applies (RFE creation) vs. general UX research work

Medium Priority

  1. Document WebSearch removal: Explain why this tool was removed in PR description or code comments
  2. Add MCP server reference: Link to setup documentation for the Google Workspace MCP server
  3. Define partial research behavior: What should the agent do when research is incomplete?

Low Priority

  1. Fix formatting: Standardize blank line usage throughout the file
  2. Update example dates: Use realistic or clearly-marked placeholder dates

Testing Recommendations

Before merging:

  • Test RFE generation with actual Google Drive access to verify MCP integration works
  • Test what happens when the Google Drive folder is inaccessible
  • Verify the agent correctly refuses unsupported requests (e.g., the OpenShift Mobile Phone example)
  • Confirm citations are properly formatted and traceable

Approval Status

⚠️ Conditional Approval: The core functionality is sound, but addressing the tone/formatting issues (#2, #7) and documenting the Google Drive access requirements (#1) would significantly improve maintainability and agent behavior quality.

The changes are safe to merge if the team accepts the current tone, but I recommend addressing at least the high-priority items for long-term maintainability.


🔍 View AI decision process (logs available for 90 days)

📋 View memory system files loaded (click to expand)

What Amber Loaded for Code Review

Amber automatically loaded these repository standards from the memory system:

  1. CLAUDE.md - Master project instructions, development standards
  2. backend-development.md - Go backend, K8s integration patterns
  3. frontend-development.md - NextJS, Shadcn UI, React Query patterns
  4. security-standards.md - Auth, RBAC, token handling
  5. k8s-client-usage.md - User token vs service account patterns
  6. error-handling.md - Consistent error patterns
  7. react-query-usage.md - Data fetching patterns

Impact: This review used your repository's specific code quality standards, security patterns, and best practices from the memory system (PRs #359, #360) - not just generic code review guidelines.

@bobbravo2 bobbravo2 added this to the v0.0.14 milestone Dec 15, 2025
@bobbravo2 bobbravo2 merged commit 494f993 into ambient-code:main Dec 15, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants