Skip to content

Commit

Permalink
fix: avoid hang when exporting voice chat (#262)
Browse files Browse the repository at this point in the history
* work on #261: avoid hang

See #261 for the situation. This commit avoids errors caused by multimodal parts we don't understand.

It does NOT add any functionality that understands advanced voice mode parts. Perhaps that should be broken out of #261 into a separate enhancement request.

* address es-lint complaint

eslint error:

error '&&' should be placed at the beginning of the line

solution:

move a trailing `&&` to the front of the next line
  • Loading branch information
peterkaminski authored Oct 13, 2024
1 parent 96c28f4 commit 9924c09
Showing 1 changed file with 7 additions and 3 deletions.
10 changes: 7 additions & 3 deletions src/api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -315,15 +315,19 @@ async function fetchImageFromPointer(uri: string) {
}

/** replaces `file-service://` pointers with data uris containing the image */
/** avoid errors in parsing multimodal parts we don't understand */
async function replaceImageAssets(conversation: ApiConversation): Promise<void> {
const isMultiModalInputImage = (part: string | MultiModalInputImage): part is MultiModalInputImage => {
return typeof part !== 'string' && part.asset_pointer.startsWith('file-service://')
const isMultiModalInputImage = (part: any): part is MultiModalInputImage => {
return typeof part === 'object' && part !== null && 'asset_pointer' in part
&& typeof part.asset_pointer === 'string' && part.asset_pointer.startsWith('file-service://')
}

const imageAssets = Object.values(conversation.mapping).flatMap((node) => {
if (!node.message) return []
if (node.message.content.content_type !== 'multimodal_text') return []

return node.message.content.parts.filter(isMultiModalInputImage)
return (Array.isArray(node.message.content.parts) ? node.message.content.parts : [])
.filter(isMultiModalInputImage)
})

const executionOutputs = Object.values(conversation.mapping).flatMap((node) => {
Expand Down

0 comments on commit 9924c09

Please sign in to comment.