-
Couldn't load subscription status.
- Fork 1
Add Integrated CI/CD Flow Example #9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| # Linear | ||
| LINEAR_ACCESS_TOKEN="your_linear_access_token" | ||
| LINEAR_SIGNING_SECRET="your_linear_signing_secret" | ||
| LINEAR_TEAM_ID="e30439bb-1547-4d4e-9934-eb757c9beb46" # Zeeeepa team ID | ||
|
|
||
| # GitHub | ||
| GITHUB_TOKEN="your_github_token" | ||
| GITHUB_REPO="codegen-sh/codegen-sdk" # Default repository to work with | ||
|
|
||
| # Slack | ||
| SLACK_BOT_TOKEN="your_slack_bot_token" | ||
| SLACK_SIGNING_SECRET="your_slack_signing_secret" | ||
| SLACK_CHANNEL="general" # Default channel for notifications | ||
|
|
||
| # AI Providers | ||
| ANTHROPIC_API_KEY="your_anthropic_api_key" | ||
| OPENAI_API_KEY="your_openai_api_key" | ||
|
|
||
| # Application | ||
| APP_NAME="codegen-cicd" # Name of the application |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,131 @@ | ||
| # Integrated CI/CD Flow with Codegen | ||
|
|
||
| This example demonstrates a complete CI/CD flow using Codegen's components to automate the entire software development lifecycle. It integrates several existing examples into a cohesive pipeline that handles everything from requirements gathering to deployment. | ||
|
|
||
| ## Architecture | ||
|
|
||
| The integrated CI/CD flow consists of the following components: | ||
|
|
||
| ``` | ||
| [Requirements] → [Planning] → [Development] → [Review] → [Testing] → [Deployment] → [Monitoring] | ||
| ``` | ||
|
|
||
| ### 1. Requirements & Planning Hub (Linear + AI) | ||
|
|
||
| - **Core Component**: Enhanced Linear Webhooks | ||
| - **Purpose**: Capture, analyze, and plan development work | ||
| - **Key Features**: | ||
| - AI-powered requirement analysis and breakdown | ||
| - Automatic task prioritization and dependency mapping | ||
| - Integration with knowledge base for context-aware planning | ||
| - Slack notifications for stakeholder alignment | ||
|
|
||
| ### 2. AI-Assisted Development (Local Checkout + Ticket-to-PR) | ||
|
|
||
| - **Core Component**: Enhanced Ticket-to-PR Bot | ||
| - **Purpose**: Generate high-quality code changes with deep context | ||
| - **Key Features**: | ||
| - Deep code research before implementation | ||
| - Contextual understanding of the entire codebase | ||
| - Intelligent code generation with architectural awareness | ||
| - Automatic PR creation with detailed documentation | ||
|
|
||
| ### 3. Comprehensive Code Review (PR Review Bot + Deep Analysis) | ||
|
|
||
| - **Core Component**: Enhanced PR Review Bot | ||
| - **Purpose**: Thorough code review with multiple perspectives | ||
| - **Key Features**: | ||
| - Static analysis and security scanning | ||
| - Architecture and design pattern validation | ||
| - Performance impact assessment | ||
| - Slack integration for team collaboration | ||
|
|
||
| ### 4. Continuous Knowledge & Assistance (Slack Chatbot) | ||
|
|
||
| - **Core Component**: Enhanced Slack Chatbot | ||
| - **Purpose**: Provide context and assistance throughout the pipeline | ||
| - **Key Features**: | ||
| - Answer questions about any stage of the pipeline | ||
| - Provide insights into codebase and architecture | ||
| - Facilitate team communication and knowledge sharing | ||
| - Serve as a central hub for pipeline status and alerts | ||
|
|
||
| ## Implementation | ||
|
|
||
| The implementation uses Modal for serverless deployment and integrates with Linear, GitHub, and Slack for a complete development workflow. | ||
|
|
||
| ### Environment Variables | ||
|
|
||
| Create a `.env` file with the following variables: | ||
|
|
||
| ``` | ||
| # Linear | ||
| LINEAR_ACCESS_TOKEN="your_token" | ||
| LINEAR_SIGNING_SECRET="your_secret" | ||
| LINEAR_TEAM_ID="your_team_id" | ||
|
|
||
| # GitHub | ||
| GITHUB_TOKEN="your_github_token" | ||
|
|
||
| # Slack | ||
| SLACK_SIGNING_SECRET="your_slack_secret" | ||
| SLACK_BOT_TOKEN="your_slack_token" | ||
|
|
||
| # AI Providers | ||
| ANTHROPIC_API_KEY="your_anthropic_key" | ||
| OPENAI_API_KEY="your_openai_key" | ||
| ``` | ||
|
|
||
| ### Deployment | ||
|
|
||
| Deploy the integrated CI/CD flow using Modal: | ||
|
|
||
| ```bash | ||
| modal deploy app.py | ||
| ``` | ||
|
|
||
| This will create webhook endpoints for Linear and GitHub, and set up the Slack bot. | ||
|
|
||
| ## Usage | ||
|
|
||
| 1. Create a Linear ticket with the "Codegen" label | ||
| 2. The system will: | ||
| - Analyze the ticket and break it down into subtasks | ||
| - Research the codebase to understand the context | ||
| - Generate code changes and create a PR | ||
| - Review the PR and provide feedback | ||
| - Notify stakeholders via Slack | ||
| 3. Interact with the Slack bot to get updates and ask questions | ||
|
|
||
| ## Components | ||
|
|
||
| The integrated CI/CD flow consists of the following components: | ||
|
|
||
| - `app.py`: Main application that integrates all components | ||
| - `requirements_hub.py`: Linear webhook handler for requirements analysis | ||
| - `development_engine.py`: Code generation and PR creation | ||
| - `review_system.py`: PR review and analysis | ||
| - `knowledge_assistant.py`: Slack bot for assistance and knowledge sharing | ||
| - `shared/`: Shared utilities and models | ||
| - `config.py`: Configuration management | ||
| - `models.py`: Shared data models | ||
| - `utils.py`: Utility functions | ||
| - `event_bus.py`: Event communication between components | ||
|
|
||
| ## Extending the Flow | ||
|
|
||
| You can extend the flow by: | ||
|
|
||
| 1. Adding new components to the pipeline | ||
| 2. Enhancing existing components with additional features | ||
| 3. Integrating with other tools and services | ||
| 4. Customizing the AI models and prompts | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| If you encounter issues: | ||
|
|
||
| 1. Check the Modal logs for errors | ||
| 2. Verify that all environment variables are set correctly | ||
| 3. Ensure that the webhooks are properly configured in Linear and GitHub | ||
| 4. Check the Slack bot permissions and event subscriptions |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,164 @@ | ||
| """Main application for the integrated CI/CD flow. | ||
|
|
||
| This module integrates all components of the CI/CD flow and provides | ||
| a unified interface for deployment. | ||
| """ | ||
|
|
||
| import logging | ||
| import os | ||
| from typing import Any, Dict | ||
|
|
||
| import modal | ||
| from codegen.extensions.events.codegen_app import CodegenApp | ||
| from fastapi import FastAPI, Request | ||
|
|
||
| from development_engine import create_development_engine | ||
| from knowledge_assistant import create_knowledge_assistant | ||
| from requirements_hub import create_requirements_hub | ||
| from review_system import create_review_system | ||
| from shared import get_modal_image, get_modal_secret, load_config_from_env | ||
|
|
||
| # Configure logging | ||
| logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(name)s - %(levelname)s - %(message)s") | ||
| logger = logging.getLogger(__name__) | ||
|
|
||
| # Create Modal image and app | ||
| image = get_modal_image() | ||
| app = CodegenApp(name="codegen-cicd") | ||
|
|
||
|
|
||
| @app.cls(secrets=[get_modal_secret()], keep_warm=1, image=image) | ||
| class IntegratedCICDFlow: | ||
| """Integrated CI/CD flow for Codegen.""" | ||
|
|
||
| def __init__(self): | ||
| """Initialize the integrated CI/CD flow.""" | ||
| self.config = None | ||
| self.requirements_hub = None | ||
| self.development_engine = None | ||
| self.review_system = None | ||
| self.knowledge_assistant = None | ||
| self.web_app = None | ||
|
|
||
| @modal.enter() | ||
| def setup(self): | ||
| """Set up the integrated CI/CD flow.""" | ||
| logger.info("Setting up integrated CI/CD flow") | ||
|
|
||
| # Load configuration | ||
| self.config = load_config_from_env() | ||
|
|
||
| # Initialize components | ||
| self.requirements_hub = create_requirements_hub(app) | ||
| self.development_engine = create_development_engine() | ||
| self.review_system = create_review_system() | ||
| self.knowledge_assistant = create_knowledge_assistant() | ||
|
|
||
| # Create FastAPI app | ||
| self.web_app = self.create_fastapi_app() | ||
|
|
||
| # Subscribe to Linear webhooks | ||
| app.linear.subscribe_all_handlers() | ||
|
|
||
| logger.info("Integrated CI/CD flow set up successfully") | ||
|
|
||
| @modal.exit() | ||
| def teardown(self): | ||
| """Clean up resources.""" | ||
| logger.info("Tearing down integrated CI/CD flow") | ||
|
|
||
| # Unsubscribe from Linear webhooks | ||
| app.linear.unsubscribe_all_handlers() | ||
|
|
||
| def create_fastapi_app(self) -> FastAPI: | ||
| """Create a FastAPI app with all webhook handlers. | ||
|
|
||
| Returns: | ||
| FastAPI app | ||
| """ | ||
| web_app = FastAPI() | ||
|
|
||
| # Create Slack app | ||
| slack_app = self.knowledge_assistant.create_fastapi_app() | ||
|
|
||
| # Mount Slack app | ||
| web_app.mount("/slack", slack_app) | ||
|
|
||
| # Add GitHub webhook endpoint | ||
| @web_app.post("/github/webhook") | ||
| async def github_webhook(request: Request): | ||
| """Handle GitHub webhook events.""" | ||
| event = await request.json() | ||
| return app.github.handle(event, request) | ||
|
|
||
| # Add Linear webhook endpoint | ||
| @web_app.post("/linear/webhook") | ||
| async def linear_webhook(request: Request): | ||
| """Handle Linear webhook events.""" | ||
| event = await request.json() | ||
| return app.linear.handle(event, request) | ||
|
|
||
| # Add health check endpoint | ||
| @web_app.get("/health") | ||
| async def health_check(): | ||
| """Health check endpoint.""" | ||
| return {"status": "ok"} | ||
|
|
||
| return web_app | ||
|
|
||
| @modal.method() | ||
| def process_linear_event(self, event: Dict[str, Any]) -> Dict[str, Any]: | ||
| """Process a Linear event. | ||
|
|
||
| Args: | ||
| event: Linear webhook event | ||
|
|
||
| Returns: | ||
| Response data | ||
| """ | ||
| logger.info(f"Processing Linear event: {event.get('action')}") | ||
| return app.linear.handle(event) | ||
|
|
||
| @modal.method() | ||
| def process_github_event(self, event: Dict[str, Any]) -> Dict[str, Any]: | ||
| """Process a GitHub event. | ||
|
|
||
| Args: | ||
| event: GitHub webhook event | ||
|
|
||
| Returns: | ||
| Response data | ||
| """ | ||
| logger.info(f"Processing GitHub event: {event.get('action')}") | ||
| return app.github.handle(event) | ||
|
Comment on lines
+109
to
+133
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The methods Recommendation: |
||
|
|
||
| @modal.web_endpoint(method="GET") | ||
| def index(self): | ||
| """Index page.""" | ||
| return { | ||
| "name": "Codegen Integrated CI/CD Flow", | ||
| "version": "1.0.0", | ||
| "endpoints": { | ||
| "linear_webhook": "/linear/webhook", | ||
| "github_webhook": "/github/webhook", | ||
| "slack": "/slack", | ||
| "health": "/health", | ||
| }, | ||
| } | ||
|
|
||
| @modal.web_endpoint(method="POST") | ||
| def webhook(self, request: Request): | ||
| """Generic webhook endpoint that routes to the appropriate handler.""" | ||
| # Get the webhook source from the path | ||
| path = request.url.path | ||
| if "linear" in path: | ||
| return self.process_linear_event(request.json()) | ||
| elif "github" in path: | ||
| return self.process_github_event(request.json()) | ||
| else: | ||
| return {"error": "Unknown webhook source"} | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| # For local development | ||
| modal.runner.deploy_in_background(app) | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The methods
process_linear_eventandprocess_github_eventlack error handling for the operationsapp.linear.handle(event)andapp.github.handle(event). This could lead to unhandled exceptions if these methods fail or if the event data is malformed, potentially crashing the application or causing unexpected behavior.Recommendation:
Implement try-except blocks around these calls to handle possible exceptions gracefully. Log the errors and return appropriate error responses to ensure the application remains stable and provides useful feedback on failures.