Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 88 additions & 0 deletions CHANGES_SUMMARY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# Summary of Changes for Image-to-Image Routing Fix

## Problem Statement
When users sent chat completion requests with both text and images (multimodal content), the system was returning API error 1002 "common error" from the Jimeng API. The issue was that the smart routing wasn't properly handling the transition from multimodal content to the appropriate generation endpoint.

## Root Cause
The routing logic was correct in theory, but lacked validation and safety checks to ensure:
1. Prompts passed to generation functions were always strings (not arrays/objects)
2. Image arrays were non-empty when routing to image-to-image
3. Clear error messages for debugging

## Changes Made

### 1. src/api/controllers/chat.ts
**Both `createCompletion` and `createCompletionStream` functions:**

- Added `String()` wrapper around `finalPrompt` to ensure it's always a string type
- Added validation: throw error if `hasImages=true` but `images.length === 0`
- Added enhanced logging when images are detected (shows image types)
- For stream handler: provides user-friendly error message in stream on validation failure

### 2. src/api/controllers/images.ts

**`generateImageComposition` function (image-to-image):**
- Added validation to ensure `prompt` parameter is a string
- Added validation to ensure `images` array is not empty
- Throws clear error messages: "提示词必须是字符串类型" and "图生图至少需要1张输入图片"

**`generateImagesInternal` function (text-to-image):**
- Added validation to ensure `prompt` parameter is a string
- Throws clear error message: "提示词必须是字符串类型,如需使用图片请使用图生图接口"

### 3. src/lib/message-parser.ts

**`parseMessages` function:**
- Added `finalText` variable for clarity
- Added warning log when images are detected but no text content extracted
- Warning: "检测到图片但没有提取到文本内容,这可能导致生成结果不理想"

## Files Modified
- `src/api/controllers/chat.ts` - Enhanced validation and logging
- `src/api/controllers/images.ts` - Added type validation for prompts
- `src/lib/message-parser.ts` - Added warning for edge cases

## Files Added
- `ROUTING_FIX.md` - Detailed explanation of the fix and how it works
- `TEST_SCENARIO.md` - Test cases and verification checklist
- `CHANGES_SUMMARY.md` - This file

## Build Status
✅ Build succeeds: `npm run build` completes without errors
⚠️ TypeScript type-check has pre-existing errors (not introduced by these changes)

## Benefits

1. **Prevents API Errors**: Catches malformed requests before sending to Jimeng API
2. **Clear Error Messages**: Users and developers get descriptive errors
3. **Better Debugging**: Enhanced logging makes routing decisions transparent
4. **Type Safety**: Ensures prompts are always strings, preventing JSON serialization issues
5. **Consistent Behavior**: Both sync and stream handlers have same validation

## Testing Recommendations

Test these scenarios:
1. Text-only message → should route to text-to-image
2. Text + image message → should route to image-to-image
3. Image-only message → should route to image-to-image with warning

## Expected Log Output (Success Case)

For a text-with-image request:
```
[info] 消息解析结果: 文本长度=27, 图片数量=1
[info] 图片类型: base64
[info] 智能路由请求类型: image-to-image, 提示词长度: 27, 图片数量: 1
[info] 检测到图片输入,图片类型: base64
[info] 开始图生图,模型: jimeng-4.0, 图片数量: 1
```

This replaces the previous error:
```
[error] 即梦API请求失败: ret=1002, errmsg=common error
```

## Backward Compatibility
✅ All changes are backward compatible
✅ Existing text-only and video requests work as before
✅ Only affects error handling and logging
Loading