improvement(kb): use trigger.dev for kb tasks

[waleedlatif1]

Summary

use trigger.dev for kb tasks, current setup broke down in serverless envs

trigger > redis > synchronous processing

Type of Change

• New feature

Testing

Tested manually.

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
docs	Ready	Preview	Comment	Aug 28, 2025 7:10pm
sim	Ready	Preview	Comment	Aug 28, 2025 7:10pm

[greptile-apps]

Greptile Summary

This PR integrates Trigger.dev as the primary method for knowledge base document processing, replacing the previous Redis-only approach with a robust three-tier fallback system. The changes introduce a new background task (background/knowledge-processing.ts) that wraps existing document processing functionality with Trigger.dev's task queue system, providing features like retry logic, concurrency limits, and monitoring.

The core integration happens in lib/knowledge/documents/service.ts, where a new priority-based processing system is implemented: Trigger.dev (primary) → Redis queue (fallback) → in-memory processing (final fallback). This approach ensures the system continues to function regardless of which services are available. The isTriggerAvailable() function checks for the necessary environment variables (TRIGGER_SECRET_KEY and TRIGGER_DEV_ENABLED), while processDocumentsWithTrigger() handles the actual job triggering using the tasks.trigger() API.

To support external service access, the presigned URL generation in route.ts was modified to provide direct S3 URLs for knowledge-base files (similar to chat files), enabling Trigger.dev to access documents without going through the application's authentication layer. The package.json was updated to use @trigger.dev/sdk version 4.0.1, likely containing bug fixes or improvements needed for the integration.

This architectural change improves scalability and reliability for document processing workloads while maintaining backward compatibility through the fallback mechanisms. The implementation follows established patterns in the codebase with proper error handling and logging throughout.

PR Description Notes:

• The PR description template is incomplete - no summary, issue reference, type of change, or testing details are provided
• All checkboxes remain unchecked

Confidence score: 3/5

• This PR introduces significant architectural changes but has one notable inconsistency that could cause issues
• Score reflects the solid implementation of the Trigger.dev integration but is lowered due to incomplete Azure Blob storage handling
• Pay close attention to the presigned URL route file where Azure Blob storage doesn't match the S3 behavior for knowledge-base files

_{4 files reviewed, 2 comments}

_{Edit Code Review Bot Settings | Greptile}