feat(component,ai,gemini): implement automatic format conversion for unsupported media types #1128
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…unsupported media types
pinglin
added a commit
that referenced
this pull request
Sep 26, 2025
…unsupported media types (#1128) Because - Users had to manually convert media formats or use special syntax (`:png`, `:pdf`, etc.) when working with formats not directly supported by Gemini API - The existing validation system only provided error messages suggesting manual conversion, creating friction in the user experience - The complex `formatSupport` struct and validation logic made the codebase harder to maintain and understand This commit - Implements automatic format conversion for unsupported MIME types in Gemini AI component: - **Images**: Automatically converts GIF, BMP, TIFF → PNG - **Videos**: Automatically converts MKV → MP4 - **Audio**: Automatically converts M4A, WMA → FLAC - **Documents**: Automatically converts Office formats (DOC, DOCX, PPT, PPTX, XLS, XLSX) → PDF, and processes text-based formats (HTML, Markdown, CSV) as plain text - Refactors format definitions by removing the complex `formatSupport` struct and replacing it with simple arrays for Gemini-supported and convertible formats - Removes the `validateFormat` function that only provided error messages, replacing it with direct format checking and conversion logic - Provides clear error messages for truly unsupported formats that cannot be converted - Ensures consistent behavior across all media processing paths (both inline data and File API)
pinglin
added a commit
that referenced
this pull request
Sep 26, 2025
…unsupported media types (#1128) Because - Users had to manually convert media formats or use special syntax (`:png`, `:pdf`, etc.) when working with formats not directly supported by Gemini API - The existing validation system only provided error messages suggesting manual conversion, creating friction in the user experience - The complex `formatSupport` struct and validation logic made the codebase harder to maintain and understand This commit - Implements automatic format conversion for unsupported MIME types in Gemini AI component: - **Images**: Automatically converts GIF, BMP, TIFF → PNG - **Videos**: Automatically converts MKV → MP4 - **Audio**: Automatically converts M4A, WMA → FLAC - **Documents**: Automatically converts Office formats (DOC, DOCX, PPT, PPTX, XLS, XLSX) → PDF, and processes text-based formats (HTML, Markdown, CSV) as plain text - Refactors format definitions by removing the complex `formatSupport` struct and replacing it with simple arrays for Gemini-supported and convertible formats - Removes the `validateFormat` function that only provided error messages, replacing it with direct format checking and conversion logic - Provides clear error messages for truly unsupported formats that cannot be converted - Ensures consistent behavior across all media processing paths (both inline data and File API)
pinglin
added a commit
that referenced
this pull request
Sep 26, 2025
…unsupported media types (#1128) Because - Users had to manually convert media formats or use special syntax (`:png`, `:pdf`, etc.) when working with formats not directly supported by Gemini API - The existing validation system only provided error messages suggesting manual conversion, creating friction in the user experience - The complex `formatSupport` struct and validation logic made the codebase harder to maintain and understand This commit - Implements automatic format conversion for unsupported MIME types in Gemini AI component: - **Images**: Automatically converts GIF, BMP, TIFF → PNG - **Videos**: Automatically converts MKV → MP4 - **Audio**: Automatically converts M4A, WMA → FLAC - **Documents**: Automatically converts Office formats (DOC, DOCX, PPT, PPTX, XLS, XLSX) → PDF, and processes text-based formats (HTML, Markdown, CSV) as plain text - Refactors format definitions by removing the complex `formatSupport` struct and replacing it with simple arrays for Gemini-supported and convertible formats - Removes the `validateFormat` function that only provided error messages, replacing it with direct format checking and conversion logic - Provides clear error messages for truly unsupported formats that cannot be converted - Ensures consistent behavior across all media processing paths (both inline data and File API)
pinglin
added a commit
that referenced
this pull request
Sep 26, 2025
…unsupported media types (#1128) Because - Users had to manually convert media formats or use special syntax (`:png`, `:pdf`, etc.) when working with formats not directly supported by Gemini API - The existing validation system only provided error messages suggesting manual conversion, creating friction in the user experience - The complex `formatSupport` struct and validation logic made the codebase harder to maintain and understand This commit - Implements automatic format conversion for unsupported MIME types in Gemini AI component: - **Images**: Automatically converts GIF, BMP, TIFF → PNG - **Videos**: Automatically converts MKV → MP4 - **Audio**: Automatically converts M4A, WMA → FLAC - **Documents**: Automatically converts Office formats (DOC, DOCX, PPT, PPTX, XLS, XLSX) → PDF, and processes text-based formats (HTML, Markdown, CSV) as plain text - Refactors format definitions by removing the complex `formatSupport` struct and replacing it with simple arrays for Gemini-supported and convertible formats - Removes the `validateFormat` function that only provided error messages, replacing it with direct format checking and conversion logic - Provides clear error messages for truly unsupported formats that cannot be converted - Ensures consistent behavior across all media processing paths (both inline data and File API)
pinglin
added a commit
that referenced
this pull request
Sep 26, 2025
…unsupported media types (#1128) Because - Users had to manually convert media formats or use special syntax (`:png`, `:pdf`, etc.) when working with formats not directly supported by Gemini API - The existing validation system only provided error messages suggesting manual conversion, creating friction in the user experience - The complex `formatSupport` struct and validation logic made the codebase harder to maintain and understand This commit - Implements automatic format conversion for unsupported MIME types in Gemini AI component: - **Images**: Automatically converts GIF, BMP, TIFF → PNG - **Videos**: Automatically converts MKV → MP4 - **Audio**: Automatically converts M4A, WMA → FLAC - **Documents**: Automatically converts Office formats (DOC, DOCX, PPT, PPTX, XLS, XLSX) → PDF, and processes text-based formats (HTML, Markdown, CSV) as plain text - Refactors format definitions by removing the complex `formatSupport` struct and replacing it with simple arrays for Gemini-supported and convertible formats - Removes the `validateFormat` function that only provided error messages, replacing it with direct format checking and conversion logic - Provides clear error messages for truly unsupported formats that cannot be converted - Ensures consistent behavior across all media processing paths (both inline data and File API)
pinglin
added a commit
that referenced
this pull request
Sep 26, 2025
…unsupported media types (#1128) Because - Users had to manually convert media formats or use special syntax (`:png`, `:pdf`, etc.) when working with formats not directly supported by Gemini API - The existing validation system only provided error messages suggesting manual conversion, creating friction in the user experience - The complex `formatSupport` struct and validation logic made the codebase harder to maintain and understand This commit - Implements automatic format conversion for unsupported MIME types in Gemini AI component: - **Images**: Automatically converts GIF, BMP, TIFF → PNG - **Videos**: Automatically converts MKV → MP4 - **Audio**: Automatically converts M4A, WMA → FLAC - **Documents**: Automatically converts Office formats (DOC, DOCX, PPT, PPTX, XLS, XLSX) → PDF, and processes text-based formats (HTML, Markdown, CSV) as plain text - Refactors format definitions by removing the complex `formatSupport` struct and replacing it with simple arrays for Gemini-supported and convertible formats - Removes the `validateFormat` function that only provided error messages, replacing it with direct format checking and conversion logic - Provides clear error messages for truly unsupported formats that cannot be converted - Ensures consistent behavior across all media processing paths (both inline data and File API)
pinglin
added a commit
that referenced
this pull request
Sep 26, 2025
…unsupported media types (#1128) Because - Users had to manually convert media formats or use special syntax (`:png`, `:pdf`, etc.) when working with formats not directly supported by Gemini API - The existing validation system only provided error messages suggesting manual conversion, creating friction in the user experience - The complex `formatSupport` struct and validation logic made the codebase harder to maintain and understand This commit - Implements automatic format conversion for unsupported MIME types in Gemini AI component: - **Images**: Automatically converts GIF, BMP, TIFF → PNG - **Videos**: Automatically converts MKV → MP4 - **Audio**: Automatically converts M4A, WMA → FLAC - **Documents**: Automatically converts Office formats (DOC, DOCX, PPT, PPTX, XLS, XLSX) → PDF, and processes text-based formats (HTML, Markdown, CSV) as plain text - Refactors format definitions by removing the complex `formatSupport` struct and replacing it with simple arrays for Gemini-supported and convertible formats - Removes the `validateFormat` function that only provided error messages, replacing it with direct format checking and conversion logic - Provides clear error messages for truly unsupported formats that cannot be converted - Ensures consistent behavior across all media processing paths (both inline data and File API)
pinglin
added a commit
that referenced
this pull request
Sep 26, 2025
…unsupported media types (#1128) Because - Users had to manually convert media formats or use special syntax (`:png`, `:pdf`, etc.) when working with formats not directly supported by Gemini API - The existing validation system only provided error messages suggesting manual conversion, creating friction in the user experience - The complex `formatSupport` struct and validation logic made the codebase harder to maintain and understand This commit - Implements automatic format conversion for unsupported MIME types in Gemini AI component: - **Images**: Automatically converts GIF, BMP, TIFF → PNG - **Videos**: Automatically converts MKV → MP4 - **Audio**: Automatically converts M4A, WMA → FLAC - **Documents**: Automatically converts Office formats (DOC, DOCX, PPT, PPTX, XLS, XLSX) → PDF, and processes text-based formats (HTML, Markdown, CSV) as plain text - Refactors format definitions by removing the complex `formatSupport` struct and replacing it with simple arrays for Gemini-supported and convertible formats - Removes the `validateFormat` function that only provided error messages, replacing it with direct format checking and conversion logic - Provides clear error messages for truly unsupported formats that cannot be converted - Ensures consistent behavior across all media processing paths (both inline data and File API)
pinglin
added a commit
that referenced
this pull request
Sep 26, 2025
…unsupported media types (#1128) Because - Users had to manually convert media formats or use special syntax (`:png`, `:pdf`, etc.) when working with formats not directly supported by Gemini API - The existing validation system only provided error messages suggesting manual conversion, creating friction in the user experience - The complex `formatSupport` struct and validation logic made the codebase harder to maintain and understand This commit - Implements automatic format conversion for unsupported MIME types in Gemini AI component: - **Images**: Automatically converts GIF, BMP, TIFF → PNG - **Videos**: Automatically converts MKV → MP4 - **Audio**: Automatically converts M4A, WMA → FLAC - **Documents**: Automatically converts Office formats (DOC, DOCX, PPT, PPTX, XLS, XLSX) → PDF, and processes text-based formats (HTML, Markdown, CSV) as plain text - Refactors format definitions by removing the complex `formatSupport` struct and replacing it with simple arrays for Gemini-supported and convertible formats - Removes the `validateFormat` function that only provided error messages, replacing it with direct format checking and conversion logic - Provides clear error messages for truly unsupported formats that cannot be converted - Ensures consistent behavior across all media processing paths (both inline data and File API)
pinglin
added a commit
that referenced
this pull request
Sep 26, 2025
…unsupported media types (#1128) Because - Users had to manually convert media formats or use special syntax (`:png`, `:pdf`, etc.) when working with formats not directly supported by Gemini API - The existing validation system only provided error messages suggesting manual conversion, creating friction in the user experience - The complex `formatSupport` struct and validation logic made the codebase harder to maintain and understand This commit - Implements automatic format conversion for unsupported MIME types in Gemini AI component: - **Images**: Automatically converts GIF, BMP, TIFF → PNG - **Videos**: Automatically converts MKV → MP4 - **Audio**: Automatically converts M4A, WMA → FLAC - **Documents**: Automatically converts Office formats (DOC, DOCX, PPT, PPTX, XLS, XLSX) → PDF, and processes text-based formats (HTML, Markdown, CSV) as plain text - Refactors format definitions by removing the complex `formatSupport` struct and replacing it with simple arrays for Gemini-supported and convertible formats - Removes the `validateFormat` function that only provided error messages, replacing it with direct format checking and conversion logic - Provides clear error messages for truly unsupported formats that cannot be converted - Ensures consistent behavior across all media processing paths (both inline data and File API)
pinglin
added a commit
that referenced
this pull request
Sep 26, 2025
…unsupported media types (#1128) Because - Users had to manually convert media formats or use special syntax (`:png`, `:pdf`, etc.) when working with formats not directly supported by Gemini API - The existing validation system only provided error messages suggesting manual conversion, creating friction in the user experience - The complex `formatSupport` struct and validation logic made the codebase harder to maintain and understand This commit - Implements automatic format conversion for unsupported MIME types in Gemini AI component: - **Images**: Automatically converts GIF, BMP, TIFF → PNG - **Videos**: Automatically converts MKV → MP4 - **Audio**: Automatically converts M4A, WMA → FLAC - **Documents**: Automatically converts Office formats (DOC, DOCX, PPT, PPTX, XLS, XLSX) → PDF, and processes text-based formats (HTML, Markdown, CSV) as plain text - Refactors format definitions by removing the complex `formatSupport` struct and replacing it with simple arrays for Gemini-supported and convertible formats - Removes the `validateFormat` function that only provided error messages, replacing it with direct format checking and conversion logic - Provides clear error messages for truly unsupported formats that cannot be converted - Ensures consistent behavior across all media processing paths (both inline data and File API)
jvallesm
pushed a commit
that referenced
this pull request
Oct 7, 2025
🤖 I have created a release *beep* *boop* --- ## [0.61.0](v0.60.0...v0.61.0) (2025-10-06) ### Features * **component,ai,gemini:** add image generation support ([#1122](#1122)) ([d986614](d986614)) * **component,ai,gemini:** add multimedia support with unified format… ([#1114](#1114)) ([291b379](291b379)) * **component,ai,gemini:** add text embeddings task support ([#1129](#1129)) ([d7ca6cf](d7ca6cf)) * **component,ai,gemini:** enhance streaming to output all fields ([#1106](#1106)) ([dfb6b24](dfb6b24)) * **component,ai,gemini:** implement automatic format conversion for unsupported media types ([#1128](#1128)) ([f767b8a](f767b8a)) * **component,ai,gemini:** implement File API support for large files… ([#1118](#1118)) ([b51c8f4](b51c8f4)) * **data:** add comprehensive AVIF image format support ([#1135](#1135)) ([76d6941](76d6941)) * **data:** add HEIC/HEIF image support and normalize MIME types ([#1127](#1127)) ([2dfa254](2dfa254)) * **data:** enhance unmarshaler with JSON string to struct conversion ([#1116](#1116)) ([9e06b7c](9e06b7c)) * **data:** implement time types support with pattern validation ([#1115](#1115)) ([79630c0](79630c0)) ### Bug Fixes * **compogen:** escape curly braces for readme.com compatibility ([#1124](#1124)) ([904992d](904992d)) * **component,ai,gemini:** add operation validation for cache task ([#1130](#1130)) ([9e19255](9e19255)) * **component,ai,gemini:** correct text-based documents logic ([#1103](#1103)) ([ed5a111](ed5a111)) * **component,ai,gemini:** unify InlineData processing and enable images in streaming responses ([#1125](#1125)) ([3117046](3117046)) * **data:** remove duplicate dot in generated filenames ([#1136](#1136)) ([0a74a00](0a74a00)) * **external:** fix Content-Disposition header parsing for filename extraction ([#1132](#1132)) ([869b081](869b081)) * **service:** handle null JSON metadata in pipeline conversion ([#1134](#1134)) ([b244784](b244784)) * **text:** correct positions on duplicate markdown chunks ([#1120](#1120)) ([1b4cd1f](1b4cd1f)) * **usage:** add missing error filtering for users/admin ([#1119](#1119)) ([cd1bd55](cd1bd55)) ### Refactor * **component,ai,gemini:** merge usage and usage-metadata fields into single usage field ([#1126](#1126)) ([a6046cd](a6046cd)) * **component,ai.gemini:** standardize file api timeout and use native embedding type ([#1133](#1133)) ([174f7d6](174f7d6)) * **component,generic,http:** move test functions to test files and improve code legibility ([#1131](#1131)) ([1153a09](1153a09)) * **component,generic,http:** replace env-based URL validation with constructor injection ([#1121](#1121)) ([f1f7d2f](f1f7d2f)) ### Tests * **component,generic,http:** replace external httpbin.org dependency with local test server ([#1101](#1101)) ([a82d155](a82d155)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
donch1989
pushed a commit
that referenced
this pull request
Oct 7, 2025
🤖 I have created a release *beep* *boop* --- ## [0.61.0](v0.60.0...v0.61.0) (2025-10-07) ### Features * **component,ai,gemini:** add image generation support ([#1122](#1122)) ([d986614](d986614)) * **component,ai,gemini:** add multimedia support with unified format… ([#1114](#1114)) ([291b379](291b379)) * **component,ai,gemini:** add text embeddings task support ([#1129](#1129)) ([d7ca6cf](d7ca6cf)) * **component,ai,gemini:** enhance streaming to output all fields ([#1106](#1106)) ([dfb6b24](dfb6b24)) * **component,ai,gemini:** implement automatic format conversion for unsupported media types ([#1128](#1128)) ([f767b8a](f767b8a)) * **component,ai,gemini:** implement File API support for large files… ([#1118](#1118)) ([b51c8f4](b51c8f4)) * **data:** add comprehensive AVIF image format support ([#1135](#1135)) ([76d6941](76d6941)) * **data:** add HEIC/HEIF image support and normalize MIME types ([#1127](#1127)) ([2dfa254](2dfa254)) * **data:** enhance unmarshaler with JSON string to struct conversion ([#1116](#1116)) ([9e06b7c](9e06b7c)) * **data:** implement time types support with pattern validation ([#1115](#1115)) ([79630c0](79630c0)) ### Bug Fixes * **compogen:** escape curly braces for readme.com compatibility ([#1124](#1124)) ([904992d](904992d)) * **component,ai,gemini:** add operation validation for cache task ([#1130](#1130)) ([9e19255](9e19255)) * **component,ai,gemini:** correct text-based documents logic ([#1103](#1103)) ([ed5a111](ed5a111)) * **component,ai,gemini:** unify InlineData processing and enable images in streaming responses ([#1125](#1125)) ([3117046](3117046)) * **component,document:** fix incorrect expected value in the unit test ([#1138](#1138)) ([189dbd6](189dbd6)) * **data:** remove duplicate dot in generated filenames ([#1136](#1136)) ([0a74a00](0a74a00)) * **external:** fix Content-Disposition header parsing for filename extraction ([#1132](#1132)) ([869b081](869b081)) * **service:** handle null JSON metadata in pipeline conversion ([#1134](#1134)) ([b244784](b244784)) * **text:** correct positions on duplicate markdown chunks ([#1120](#1120)) ([1b4cd1f](1b4cd1f)) * **usage:** add missing error filtering for users/admin ([#1119](#1119)) ([cd1bd55](cd1bd55)) ### Miscellaneous * release v0.61.0 ([e1db93c](e1db93c)) ### Refactor * **component,ai,gemini:** merge usage and usage-metadata fields into single usage field ([#1126](#1126)) ([a6046cd](a6046cd)) * **component,ai.gemini:** standardize file api timeout and use native embedding type ([#1133](#1133)) ([174f7d6](174f7d6)) * **component,generic,http:** move test functions to test files and improve code legibility ([#1131](#1131)) ([1153a09](1153a09)) * **component,generic,http:** replace env-based URL validation with constructor injection ([#1121](#1121)) ([f1f7d2f](f1f7d2f)) ### Tests * **component,generic,http:** replace external httpbin.org dependency with local test server ([#1101](#1101)) ([a82d155](a82d155)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
donch1989
added a commit
to instill-ai/instill-core
that referenced
this pull request
Oct 8, 2025
Because - The version of the pipeline-backend service is not updated in the instill-core repository. This commit - updates the `PIPELINE_BACKEND_VERSION` in the `.env` file to `0.61.0`. - updates the `pipelineBackend.image.tag` in the helm chart values.yaml file to `0.61.0`. ## Changes in pipeline-backend - chore(main): release 0.61.0 (instill-ai/pipeline-backend#1137) - chore: release v0.61.0 - fix(component,document): fix incorrect expected value in the unit test (instill-ai/pipeline-backend#1138) - fix(data): remove duplicate dot in generated filenames (instill-ai/pipeline-backend#1136) - feat(data): add comprehensive AVIF image format support (instill-ai/pipeline-backend#1135) - fix(service): handle null JSON metadata in pipeline conversion (instill-ai/pipeline-backend#1134) - refactor(component,ai.gemini): standardize file api timeout and use native embedding type (instill-ai/pipeline-backend#1133) - fix(external): fix Content-Disposition header parsing for filename extraction (instill-ai/pipeline-backend#1132) - refactor(component,generic,http): move test functions to test files and improve code legibility (instill-ai/pipeline-backend#1131) - fix(component,ai,gemini): add operation validation for cache task (instill-ai/pipeline-backend#1130) - feat(component,ai,gemini): add text embeddings task support (instill-ai/pipeline-backend#1129) - feat(component,ai,gemini): implement automatic format conversion for unsupported media types (instill-ai/pipeline-backend#1128) - feat(data): add HEIC/HEIF image support and normalize MIME types (instill-ai/pipeline-backend#1127) - refactor(component,ai,gemini): merge usage and usage-metadata fields into single usage field (instill-ai/pipeline-backend#1126) - fix(component,ai,gemini): unify InlineData processing and enable images in streaming responses (instill-ai/pipeline-backend#1125) - fix(compogen): escape curly braces for readme.com compatibility (instill-ai/pipeline-backend#1124) - ci(workflows): merge sync-component-docs workflows into single push-t… (instill-ai/pipeline-backend#1123) - feat(component,ai,gemini): add image generation support (instill-ai/pipeline-backend#1122) Co-authored-by: donch1989 <441005+donch1989@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Because
:png,:pdf, etc.) when working with formats not directly supported by Gemini APIformatSupportstruct and validation logic made the codebase harder to maintain and understandThis commit
formatSupportstruct and replacing it with simple arrays for Gemini-supported and convertible formatsvalidateFormatfunction that only provided error messages, replacing it with direct format checking and conversion logic