(WiP) Restructure ADK agent architecture and add comprehensive documentation

[dandye]

Summary

This PR introduces a major restructuring of the ADK (Agent Development Kit) agent architecture to improve organization, maintainability, and user experience:

• Directory reorganization: Logical groupings with agents/, scripts/, ui/, and libs/ directories
• Comprehensive documentation: New Google ADK documentation section with quickstart guides for Cloud Run and AgentSpace deployment
• Automation tools: Makefile with deployment, testing, and management commands
• Integration scripts: OAuth, AgentSpace, and Cloud Run deployment automation
• Enhanced structure: Better separation of concerns and cleaner codebase organization

Key Changes

Directory Structure

• agents/ - Contains agent implementations (google_mcp_security_agent, sample_agents)
• scripts/ - Deployment and management scripts
• ui/ - Web UI components
• libs/adk_utils/ - Shared utility functions

Documentation

• Added comprehensive ADK documentation with quickstart guides
• Cloud Run and AgentSpace deployment paths
• Local setup instructions
• Makefile reference guide
• Advanced topics and configuration options

Automation

• Complete Makefile with 30+ commands for deployment and management
• OAuth configuration and AgentSpace integration scripts
• Environment variable management tools
• Agent testing and monitoring utilities

Test Plan

• Verify directory structure is correct
• Test local agent setup with new structure
• Validate Cloud Run deployment path
• Test AgentSpace integration workflow
• Verify documentation accuracy
• Test Makefile commands
• Ensure backward compatibility where needed

This restructuring provides a solid foundation for the ADK agent ecosystem while maintaining all existing functionality.

❯ make
Available targets:

Environment Management:
  env-setup       - Create .env from .env.sample
  env-check       - Validate required environment variables
  config-show     - Display current configuration (masks secrets)
  env-update      - Update environment variable

OAuth Management:
  oauth-client    - Guide for creating OAuth client
  oauth-uri       - Generate OAuth authorization URI
  oauth-link      - Link OAuth to AgentSpace
  oauth-verify    - Verify OAuth configuration
  oauth-setup     - Complete OAuth setup workflow

Agent Management:
  agents-list     - List all deployed Agent Engine instances
  agents-delete   - Delete Agent Engine instance by index
  adk-deploy      - Deploy agent to Agent Engine
  adk-redeploy    - Redeploy agent to existing Agent Engine resource
  test-agent      - Test deployed agent with a message

AgentSpace Integration:
  agentspace-register - Register agent with AgentSpace
  agentspace-update   - Update existing AgentSpace registration
  agentspace-verify   - Verify AgentSpace integration
  agentspace-delete   - Remove agent from AgentSpace
  agentspace-url      - Get AgentSpace UI URL

Cloud Run Deployment:
  cloudrun-deploy - Deploy agent to Cloud Run
  cloudrun-run    - Run Cloud Run service locally
  cloudrun-test   - Test deployed Cloud Run service
  cloudrun-logs   - View Cloud Run service logs
  cloudrun-url    - Get Cloud Run service URL
  cloudrun-delete - Delete Cloud Run service

Development Tools:
  docs            - Build documentation
  gcloud-proxy    - Proxy to Cloud Run service locally

Multi-Agent System:
  multi-agent-setup - Set up multi-agent environment
  multi-agent-run - Run the manager agent
  multi-agent-web - Run the multi-agent web UI

Agent Templates:
  cookiecutter-setup - Set up cookiecutter agent builder
  cookiecutter-run - Run the cookiecutter web builder
  cookiecutter-new-agent - Create new agent from template

Environment variables from .env will be automatically loaded.

Examples:
  make env-check
  make config-show
  make env-update KEY=GOOGLE_CLOUD_PROJECT VALUE=my-project
  make oauth-setup
  make agents-list
  make adk-deploy AGENT_DIR=. DISPLAY_NAME='Security Agent'
  make test-agent MSG='List security tools'
  make agentspace-register
  make agentspace-verify
  make agentspace-url

ToDo

1. De-dupe the 2 requirements.txt files:

run-with-google-adk/requirements.txt
run-with-google-adk/agents/google_mcp_security_agent/requirements.txt

2. Test MCP Tools in Agentspace

Blocked on the scope: https://www.googleapis.com/auth/generative-language.retriever in oauth_manager.py. ToDo: test without that scope.

3. Consider incorporating VeerMuchandi's tools from https://github.com/VeerMuchandi/agent_registration_tool
   3.1 Create Makefile targets for those tools

4. Document the complete workflow:

This is a first draft (needs review):

  make env-check
  make config-show
  make env-update KEY=GOOGLE_CLOUD_PROJECT VALUE=my-project
  make adk-deploy AGENT_DIR=. DISPLAY_NAME='Security Agent'
  make agents-list
  make test-agent MSG='List security tools'
  make oauth-setup
  make agentspace-register
  make agentspace-verify
  make agentspace-url

Each step should indicate which env vars are required as input (in .env and/or in make command) and which are created. Indicate where the created ones should are required in the next step. If there are alternatives to using the make approach, (like cloud console clicking), then note that and point to relevant docs with screenshots.