Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat (ai/core): add experimental generateImage function #4056

Merged
merged 29 commits into from
Dec 10, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fx
  • Loading branch information
lgrammel committed Dec 10, 2024
commit b2086cfedbbf638484a7f54cf4c3f18818305e01
15 changes: 15 additions & 0 deletions examples/ai-core/src/generate-image/openai.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
import { openai } from '@ai-sdk/openai';
import { generateImage } from 'ai';
import 'dotenv/config';

async function main() {
const { images } = await generateImage({
model: openai.image('dall-e-3'),
prompt: 'Santa Claus driving a Cadillac',
maxRetries: 0,
});

console.log(images);
}

main().catch(console.error);
9 changes: 7 additions & 2 deletions packages/ai/core/generate-image/generate-image.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ The prompt that should be used to generate the image.
/**
Number of images to generate.
*/
n: number;
n?: number;

/**
Maximum number of retries per embedding model call. Set to 0 to disable retries.
Expand All @@ -58,7 +58,12 @@ Only applicable for HTTP-based providers.
const { retry } = prepareRetries({ maxRetries: maxRetriesArg });

const { images } = await retry(() =>
model.doGenerate({ prompt, n, abortSignal, headers }),
model.doGenerate({
prompt,
n: n ?? 1,
abortSignal,
headers,
}),
);

return new DefaultGenerateImageResult({ images });
Expand Down
6 changes: 4 additions & 2 deletions packages/openai/src/openai-image-model.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,11 +56,13 @@ export class OpenAIImageModel implements ImageModelV1 {
});

return {
images: response.map(item => item.b64_json),
images: response.data.map(item => item.b64_json),
};
}
}

// minimal version of the schema, focussed on what is needed for the implementation
// this approach limits breakages when the API changes and increases efficiency
const openaiImageResponseSchema = z.array(z.object({ b64_json: z.string() }));
const openaiImageResponseSchema = z.object({
data: z.array(z.object({ b64_json: z.string() })),
});
Loading