feat(ai-powerups): AI personas, projects, and prompt context builder

AGENTS.md

@@ -17,6 +17,7 @@ When new backend features are discovered, update `ai-context/core-features-refer
 
 ## Building
 
+- When type checking, use `yarn check -p <package-name>`, e.g., `yarn check -p @webiny/api-core`
 - When building a single package, use `yarn build -p <package-name>`, e.g., `yarn build -p @webiny/api-core`.
 - To build all packages, simply run `yarn build`.
 - To build all packages without caching, use `yarn build --no-cache`.

ai-context/plans/ai-personas-refactor.md

@@ -0,0 +1,195 @@
+# Architectural Refactor Plan — `feat/ai-personas` Branch
+
+## Context
+
+The `feat/ai-personas` branch introduces AI Projects, Reader/Writer Personas, and enhanced content generation. While the features work, several architectural violations emerged during rapid development: direct S3 usage bypassing DI, near-total code duplication between persona types, growing use-case responsibilities, hardcoded configuration, and silent error swallowing throughout. This plan addresses each with concrete, non-breaking refactors.
+
+---
+
+## Item 1: Extract `GetFileContentsUseCase` — eliminate direct S3 from `ai-powerups` (P0, Size: L)
+
+**Problem**: `loadProjectFiles.ts` directly instantiates `new S3()`, reads `process.env.S3_BUCKET`, and reaches into FileManager's internal KV structure (`FileManager/File/${id}/Metadata`). It's a pure function that can't participate in DI, mixing S3 fetching, KV caching, MIME filtering, and metadata lookup.
+
+**What to do**:
+
+1. **`packages/api-file-manager`** — define the abstraction:
+   - Create `src/features/file/GetFileContents/abstractions.ts`
+   - `GetFileContentsUseCase` with `execute(fileId: string): Promise<Result<{ buffer: Buffer; contentType: string }, Error>>`
+   - Export from package index
+
+2. **`packages/api-file-manager-s3`** — implement:
+   - Create `src/features/GetFileContents/GetFileContentsUseCaseImpl.ts`
+   - Inject `GlobalKeyValueStore` (for `MetadataReader` pattern already used in this package) and S3 client
+   - Read `process.env.S3_BUCKET` (established convention in this package)
+   - Register in the S3 feature
+
+3. **`packages/ai-powerups`** — consume:
+   - Convert `loadProjectFiles.ts` into an injectable `ProjectFileLoader` class
+   - `ProjectFileLoader` injects `GetFileContentsUseCase` and `GlobalKeyValueStore`
+   - MIME filtering and project-context caching stay here (ai-powerups concerns)
+   - `WbGeneratePageContentUseCase` injects `ProjectFileLoader` instead of calling the free function
+
+**Key files**:
+
+- `packages/ai-powerups/src/api/features/WbGeneratePageContent/loadProjectFiles.ts` (replace)
+- `packages/api-file-manager/src/features/file/GetFileContents/` (new)
+- `packages/api-file-manager-s3/src/features/GetFileContents/` (new)
+
+**Dependencies**: None. Foundation for Items 4 and 5.
+
+---
+
+## Item 2: Unify Reader/Writer Personas via factory pattern (P1, Size: S)
+
+**Problem**: 6 files are near-identical between Reader and Writer personas — API handlers, types, and admin settings. Adding a third persona type would mean copying everything again.
+
+**What to do**:
+
+1. **API side** — create `packages/ai-powerups/src/api/features/Personas/`:
+   - `types.ts`: Shared `PersonaPreset { id, name, description, style? }` + `PersistedPersonas`
+   - `createPersonasHandler.ts`: Factory function taking `{ name: string }`, returns handler class with shared Zod schema and mapping logic
+   - `ReaderPersonasHandler.ts` and `WriterPersonasHandler.ts` become one-liner factory calls
+   - Keep separate `declare module` augmentations per type (different keys on `IAiPowerUpsSettings`)
+
+2. **Admin side** — create `packages/ai-powerups/src/admin/presentation/createPersonasSettingsGroup.ts`:
+   - Factory taking config: `{ name, label, description, addItemLabel, itemTitlePrefix, descriptionHint, styleHint }`
+   - `ReaderPersonasSettings.ts` and `WriterPersonasSettings.ts` each become thin wrappers
+
+**Key files**:
+
+- `packages/ai-powerups/src/api/features/ReaderPersonas/ReaderPersonasHandler.ts` (simplify)
+- `packages/ai-powerups/src/api/features/WriterPersonas/WriterPersonasHandler.ts` (simplify)
+- `packages/ai-powerups/src/admin/presentation/ReaderPersonasSettings.ts` (simplify)
+- `packages/ai-powerups/src/admin/presentation/WriterPersonasSettings.ts` (simplify)
+
+**Dependencies**: None. Independent.
+
+---
+
+## Item 3: Make token budget configurable (P2, Size: S)
+
+**Problem**: `ProjectsHandler.ts` hardcodes `TOKEN_BUDGET = 150_000` and `CHARS_PER_TOKEN = 4`. Different models have different context windows. No way to adjust without code changes.
+
+**What to do**:
+
+- Create `TokenBudgetConfig` abstraction in `ai-powerups/src/api/features/Projects/abstractions.ts`
+- Interface: `{ getTokenBudget(): number; getCharsPerToken(): number }`
+- Default implementation returns current values (150K, 4)
+- `ProjectsHandler` injects `TokenBudgetConfig` instead of using constants
+- Consumers can override via `container.register()` for different models
+
+**Key files**:
+
+- `packages/ai-powerups/src/api/features/Projects/ProjectsHandler.ts`
+
+**Dependencies**: None. Independent.
+
+---
+
+## Item 4: Extract `GenerationContextResolver` from use case (P2, Size: M)
+
+**Problem**: `WbGeneratePageContentUseCase` has 5 constructor dependencies and handles settings lookup, persona resolution, project loading, file assembly, prompt building, and AI invocation. Lines 50-78 are pure context resolution logic.
+
+**What to do**:
+
+- Create `GenerationContextResolver` abstraction in `WbGeneratePageContent/abstractions.ts`
+- Interface: `resolve(params): Promise<GenerationContext>` where `GenerationContext = { project?, readerPersona?, writerPersona?, files }`
+- Implementation injects `GetSettingsUseCase` and `ProjectFileLoader`
+- Use case drops to 3 dependencies: `GenerationContextResolver`, `Ai`, `Encryption` (+ `AiSdkTools`)
+
+**Key files**:
+
+- `packages/ai-powerups/src/api/features/WbGeneratePageContent/WbGeneratePageContentUseCase.ts`
+- `packages/ai-powerups/src/api/features/WbGeneratePageContent/abstractions.ts`
+
+**Dependencies**: Depends on Item 1 (`ProjectFileLoader`).
+
+---
+
+## Item 5: Eliminate silent error swallowing (P1, Size: S)
+
+**Problem**: Empty catch blocks in `loadProjectFiles.ts` (lines 91, 104, 150) and `GenerateContentPresenter.ts` (line 133) make production debugging impossible.
+
+**What to do**:
+
+1. **`ProjectFileLoader`** (post-Item 1):
+   - Cache deserialization failures: `console.warn` with cache key and error
+   - Cache write failures: `console.warn` with cache key and error
+   - S3/file retrieval failures: `console.warn` with file ID, bucket key, and error
+   - Return type: `Result<{ files: ProjectFileContent[], warnings: string[] }>`
+
+2. **`GenerateContentPresenter.ts`** (independent):
+   - Add `error?: string` to ViewModel
+   - Set it from catch block instead of just `console.error`
+
+**Key files**:
+
+- `packages/ai-powerups/src/api/features/WbGeneratePageContent/loadProjectFiles.ts`
+- `packages/ai-powerups/src/admin/presentation/WbContentGeneration/GenerateContentPresenter.ts`
+
+**Dependencies**: File loader changes depend on Item 1. Presenter fix is independent.
+
+---
+
+## Item 6: Decouple presenter from raw settings shape (P3, Size: S)
+
+**Problem**: `GenerateContentPresenter` directly navigates `this._settings?.projects?.presets` in 5 private methods. If the settings shape changes, every method breaks.
+
+**What to do**:
+
+- Extract `IContentGenerationOptionsProvider` interface with methods: `getProjectOptions()`, `getProjectFileOptions(projectId)`, `getPersonaOptions(type)`, `getProjectDefaults(projectId)`, `getAllFileIds(projectId)`
+- Implementation wraps `IAiPowerUpsSettings`
+- Presenter injects the provider instead of raw settings
+
+**Key files**:
+
+- `packages/ai-powerups/src/admin/presentation/WbContentGeneration/GenerateContentPresenter.ts`
+
+**Dependencies**: None, but low priority.
+
+---
+
+## Item 7: Export `buildPrompt` helpers for testing (P3, Size: XS)
+
+**Problem**: `buildPersonaSections()` and `buildProjectSection()` in `buildPrompt.ts` are module-private. Non-trivial string assembly that should be independently testable.
+
+**What to do**: Add `export` keyword to both functions.
+
+**Key files**:
+
+- `packages/ai-powerups/src/api/features/WbGeneratePageContent/buildPrompt.ts`
+
+**Dependencies**: None.
+
+---
+
+## Execution Order
+
+```
+Phase 1 (parallel, no dependencies):
+  [ ] Item 7: Export buildPrompt helpers                              [XS]
+  [ ] Item 2: Unify Personas via factories                           [S]
+  [ ] Item 3: Token budget config                                    [S]
+  [ ] Item 5 (presenter part): Fix GenerateContentPresenter error    [XS]
+
+Phase 2 (largest item, no blockers):
+  [ ] Item 1: GetFileContentsUseCase + ProjectFileLoader             [L]
+
+Phase 3 (depends on Item 1):
+  [ ] Item 5 (file loader part): Fix error logging in ProjectFileLoader  [S]
+  [ ] Item 4: Extract GenerationContextResolver                     [M]
+
+Phase 4 (low priority):
+  [ ] Item 6: Decouple presenter from settings shape                 [S]
+```
+
+## Verification
+
+After each item:
+
+- `yarn check -p @webiny/ai-powerups` — type check
+- `yarn check -p @webiny/api-file-manager` — type check (for Item 1)
+- `yarn check -p @webiny/api-file-manager-s3` — type check (for Item 1)
+- `yarn build -p @webiny/ai-powerups 2>&1 | tail -30` — build succeeds
+- `yarn lint` — no new lint errors
+- Existing tests still pass: `yarn test packages/ai-powerups 2>&1 | tail -50`

ai-context/plans/ai-projects.md

@@ -0,0 +1,140 @@
+# Plan: Projects for AI Power-Ups
+
+> Source PRD: `ai-context/prds/ai-projects-prd.md`
+
+## Architectural decisions
+
+Durable decisions that apply across all phases:
+
+- **Data model**: Projects live inside `IAiPowerUpsSettings` as a `projects: { presets: Project[] }` section, following the same settings group pattern as WriterPersonas/ReaderPersonas. Each project has an `id`, `name`, `description`, `instructions`, `defaultReaderPersonaId`, `defaultWriterPersonaId`, `files: ProjectFile[]`, and `version: number`.
+- **Settings group pattern**: Admin side implements `AiPowerUpsSettingsGroup.Interface` (form definition). API side implements `AiPowerUpsSettingsGroupHandler.Interface` (Zod validation, mapFromStorage/mapToStorage). Registration via feature files in admin and api Extension.
+- **Project context cache key**: `project-context:{projectId}:v{version}` in GlobalKeyValueStore with `expiresAt` TTL (30 days).
+- **Prompt ordering** (for prefix-based Anthropic prompt caching): persona system prompt → project instructions → project files → component catalog → tools → user prompt.
+- **ID-based references**: `projectId`, `readerPersonaId`, `writerPersonaId` are passed as string IDs from admin to API. Backend resolves full objects from settings.
+- **File content support**: MD, MDX, TXT, JSON, CSV only (text-readable formats). Files loaded from S3 via DAM file key.
+
+---
+
+NOTE: when a phase is complete, mark it as "done" in the list below.
+
+## [x] Phase 1: Projects CRUD in Settings
+
+**User stories**: Create a project, edit a project's name/description/instructions/default personas, delete a project.
+
+### What to build
+
+A new "Projects" settings group that appears as a fourth tab in AI Power-Ups settings (below Writer Personas). Users can add/edit/remove project presets with name, description, instructions textarea, and default reader/writer persona dropdowns. No file support yet.
+
+The admin side defines the form using the settings group builder pattern (`objectAccordionMultiple` renderer with `list()`). The API side adds a handler with Zod validation, type augmentation on `IAiPowerUpsSettings`, and feature registration. Default persona dropdowns use `.options()` callbacks that read from the same settings object's persona presets.
+
+### Acceptance criteria
+
+- [ ] "Projects" tab appears in AI Power-Ups settings page
+- [ ] Can add a new project with name (required), description, instructions, default reader persona, default writer persona
+- [ ] Can edit and delete existing projects
+- [ ] Projects persist across page reloads (saved to KeyValueStore via settings)
+- [ ] Default persona dropdowns show existing reader/writer persona presets
+- [ ] Zod validation rejects projects without a name
+- [ ] Admin `IAiPowerUpsSettings` type augmented with `projects` section
+
+---
+
+## [x] Phase 2: Project Files (DAM Integration)
+
+**User stories**: Attach reference files from the DAM to a project.
+
+### What to build
+
+Add a "Files" field to the project edit form that lets admins pick files from the DAM. Uses a `file().list()` field with a custom renderer backed by `MultiFilePicker`. Each file reference stores `FileItem` object. Ortder of files is index-based.
+
+Only file metadata is stored in settings — no file content loading happens in this phase.
+
+### Acceptance criteria
+
+- [ ] Project edit form includes a "Files" section with DAM file picker
+- [ ] Can add multiple files from the DAM to a project
+- [ ] Can remove files from a project
+- [ ] File metadata persist with the project
+- [ ] Files display with their DAM name in the project form
+
+---
+
+## [x] Phase 3: Project Picker in Generate Content Dialog
+
+**User stories**: Pick a project in the Generate Content dialog; default personas auto-populate.
+
+### What to build
+
+Add a "Project" select field to the Generate Content dialog form, placed above the persona selects. When a project is selected, auto-populate the reader/writer persona fields with the project's defaults (user can still override). Pass `projectId` through the full chain: admin form → gateway GraphQL mutation → API task input → use case → prompt builder. Backend resolves the project from settings and appends project instructions to the system prompt.
+
+No file content loading — only the project's `instructions` field is used in prompt construction.
+
+### Acceptance criteria
+
+- [ ] "Project" dropdown appears in Generate Content dialog above persona selects
+- [ ] Dropdown lists all configured projects by name
+- [ ] Selecting a project auto-fills reader/writer persona dropdowns with project defaults
+- [ ] User can override auto-filled personas
+- [ ] `projectId` flows through GraphQL mutation, task input, and use case params
+- [ ] Backend resolves project from settings and appends instructions to system prompt
+- [ ] Generation works correctly without a project selected (backward compatible)
+
+---
+
+## [x] Phase 4: File Content Loading + Prompt Assembly
+
+**User stories**: Project files are sent to the LLM; per-file toggle in dialog.
+
+### What to build
+
+Two pieces: (1) Backend file content loading — when a project is selected and has files, load each file's content from S3 via the DAM file key, then assemble into the system prompt using the format from PRD §7.5. (2) Dialog UI — when a project is selected, show a "Files included" section with checkboxes for each file. Unchecked files are passed as `excludedFileIds` through the full chain.
+
+Supported formats: MD, MDX, TXT, JSON, CSV (read as text). The use case filters out excluded files before assembling the prompt.
+
+### Acceptance criteria
+
+- [ ] When a project with files is used for generation, file contents appear in the system prompt
+- [ ] File content is loaded from S3 using DAM file metadata (key)
+- [ ] Prompt format matches PRD §7.5 (project name, instructions, then files with labels/notes)
+- [ ] Dialog shows "Files included" section with per-file checkboxes when project is selected
+- [ ] Unchecked files are excluded from generation (excludedFileIds param)
+- [ ] `excludedFileIds` flows through GraphQL mutation, task input, and use case
+- [ ] Unsupported file types are silently skipped
+- [ ] Generation still works if project has no files
+
+---
+
+## [x] Phase 5: Project Context Cache
+
+**User stories**: Performance optimization — avoid re-fetching files from S3 on every generation.
+
+### What to build
+
+Three pieces: (1) Upgrade KeyValueStore abstraction (§7.8) to support `expiresAt: Date` in options for both `KeyValueStore` and `GlobalKeyValueStore`. Update the DynamoDB implementation to store and check expiration. (2) Build project context cache — on generation, check cache by `project-context:{projectId}:v{version}` key. On miss, assemble from S3, gzip, store with 30-day TTL. On hit, decompress and use. (3) Version bumping — when settings are saved and project data changed, increment the project's `version` field in the handler's `mapToStorage`.
+
+### Acceptance criteria
+
+- [ ] KeyValueStore abstraction supports `expiresAt: Date` option
+- [ ] DynamoDB implementation stores and respects expiration
+- [ ] Assembled project context is cached with version-keyed entries
+- [ ] Cache hit avoids S3 reads
+- [ ] Editing a project bumps `version`, causing cache miss on next generation
+- [ ] Cache entries expire after 30 days via TTL
+
+---
+
+## [x] Phase 6: Anthropic Prompt Cache + Token Budget
+
+**User stories**: Cost/latency optimization via Anthropic prompt caching; token budget guardrails.
+
+### What to build
+
+Two pieces: (1) Mark the entire system block (personas + project block) with `cacheControl: { type: "ephemeral" }` via Anthropic provider options. Ensure prompt ordering is stable (persona → project instructions → project files → catalog → tools) for maximum prefix cache hits. (2) Token budget validation — at project save time, estimate assembled context size and reject if it exceeds 150k tokens with a clear error indicating which files are largest.
+
+### Acceptance criteria
+
+- [ ] System prompt block includes `providerOptions.anthropic.cacheControl` marking
+- [ ] Prompt ordering is stable and cache-friendly
+- [ ] Repeat generations with same project + personas hit Anthropic's prompt cache
+- [ ] Saving a project with assembled context exceeding 150k tokens shows a validation error
+- [ ] Validation error identifies the largest files contributing to the budget overage