Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat (ai/core): add experimental generateImage function #4056

Merged
merged 29 commits into from
Dec 10, 2024
Merged

Conversation

lgrammel
Copy link
Collaborator

@lgrammel lgrammel commented Dec 10, 2024

Fixes #2524

Tasks

  • implementation
    • spec
    • openai image generation model
    • openai image generation model test
    • generateImage
    • generateImage test
    • size
    • providerOptions
    • experimental
    • uint8array conversion
    • image
    • redesign result
  • examples
    • basic example
    • next.js example
  • documentation
    • generateImage reference
    • core guide
    • openai model
  • changeset

@lgrammel lgrammel marked this pull request as ready for review December 10, 2024 15:11
import { GeneratedImage, GenerateImageResult } from './generate-image-result';

/**
Embed a value using an embedding model. The type of the value is defined by the embedding model.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

has old content, need to update

@lgrammel lgrammel merged commit 09a9cab into main Dec 10, 2024
9 checks passed
@lgrammel lgrammel deleted the lg/C6dUx840 branch December 10, 2024 16:48
`generateImage` accepts an optional `headers` parameter of type `Record<string, string>`
that you can use to add custom headers to the image generation request.

```ts highlight={"7"}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like it might be intended to highlight 8

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

7 should be correct (headers) but i'll double check when deployed

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes 7 shows as expected

},
{
name: 'headers',
type: 'Record<string, string>',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR-wide: should this be the set of types Resolvable allows? If so, update docs/code in some manner.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, when we introduce resolvable more broadly we need to update this

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wondered while reading:

  • what format(s) are produced
  • how does token usage work

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • no token usage
  • format is currently open, we can just add on-the-fly mimetype detection if people ask

headers?: Record<string, string | undefined>;
}): PromiseLike<{
/**
Generated images as base64 encoded strings.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we want some representation of what the binary format is, or to mention briefly in a doc comment if there is only one currently and no field specifier is needed?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you mean the image type, we can auto-detect this later using magic bytes if needed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

image generation
3 participants