feat: enhance Gemini integration with system instructions and response modalities #64
Conversation
|
Caution Review failedThe pull request is closed. Summary by CodeRabbit
Summary by CodeRabbit
WalkthroughThe Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant GeminiChatRequest
participant BifrostRequest
participant BifrostResponse
Client->>GeminiChatRequest: Create request (with system instruction, content, config)
GeminiChatRequest->>GeminiChatRequest: ConvertToBifrostRequest()
GeminiChatRequest->>BifrostRequest: Build BifrostRequest (convert content, config, tools)
BifrostRequest-->>GeminiChatRequest: BifrostRequest object
BifrostRequest->>BifrostResponse: Send request, receive response
BifrostResponse-->>GeminiChatRequest: Response with metadata
GeminiChatRequest->>GeminiChatRequest: DeriveGenAIFromBifrostResponse()
GeminiChatRequest-->>Client: GenerateContentResponse (includes thoughts, safety ratings)
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (3)
✨ Finishing Touches
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
This stack of pull requests is managed by Graphite. Learn more about stacking. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 7
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
transports/bifrost-http/integrations/genai/types.go(4 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (1)
transports/bifrost-http/integrations/genai/types.go (1)
core/schemas/bifrost.go (11)
Tool(108-112)ModelChatMessageRoleSystem(25-25)ModelChatMessageRole(20-20)ModelParameters(75-89)Function(101-105)BifrostMessage(143-152)ToolCall(266-270)ImageContent(174-179)FunctionCall(260-263)FunctionParameters(92-98)AssistantMessage(166-171)
🔇 Additional comments (4)
transports/bifrost-http/integrations/genai/types.go (4)
37-42: LGTM! System instruction handling is correct.The implementation properly converts system instructions to Bifrost messages with the system role and prepends them to the chat input as expected.
46-49: Improved message conversion with helper method.Good refactoring to extract the content conversion logic into a dedicated helper method, improving code organization and reusability.
280-282: LGTM! Good use of reflection for empty struct check.Using
reflect.DeepEqualto check for empty GenerationConfig is a clean approach that avoids manual field-by-field checking.
421-426: Excellent addition of thinking content support!The implementation correctly handles the "thinking" content from assistant messages by setting the
Thoughtfield in the Part struct.
Pratham-Mishra04
force-pushed
the
06-09-refactor_http_transport_gemini_integration_refactors
branch
from
70efb6c to
9bc0828
Compare
There was a problem hiding this comment.
Actionable comments posted: 4
♻️ Duplicate comments (2)
transports/bifrost-http/integrations/genai/types.go (2)
156-159: Previous feedback not addressed – log marshalling errors.The
json.Marshalerror is still swallowed. At minimum emit a debug log so failures can be traced.
See prior review on the same lines.
441-455: Previous feedback not addressed – filter empty safety ratings and add defensive checks.Safety ratings are appended even when all fields are zero values, and the nested assertions can still panic on malformed payloads.
This mirrors an earlier review that remains unresolved.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
transports/bifrost-http/integrations/genai/types.go(5 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (1)
transports/bifrost-http/integrations/genai/types.go (2)
core/schemas/bifrost.go (16)
BifrostRequest(54-64)ModelParameters(75-89)Tool(108-112)ModelChatMessageRoleSystem(25-25)ModelChatMessageRole(20-20)Function(101-105)BifrostMessage(143-152)ToolCall(266-270)ImageContent(174-179)FunctionCall(260-263)AssistantMessage(166-171)ModelChatMessageRoleUser(24-24)UserMessage(155-157)ModelChatMessageRoleTool(27-27)ToolMessage(160-163)FunctionParameters(92-98)core/utils.go (1)
Ptr(3-5)
Pratham-Mishra04
force-pushed
the
06-09-refactor_http_transport_gemini_integration_refactors
branch
from
9bc0828 to
e339c57
Compare
Pratham-Mishra04
force-pushed
the
06-09-feat_openai_integration_added_to_http_transport
branch
from
b015950 to
111399d
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
♻️ Duplicate comments (3)
transports/bifrost-http/integrations/genai/types.go (3)
282-284:TopKstill hidden inExtraParamsA dedicated
TopKfield exists onModelParameters; stashing it inExtraParamsis inconsistent with howTemperature,TopP, andMaxTokensare handled. (See previous review.)
129-133: Expensivereflect.DeepEqualfor zero-value checkUsing reflection here is overkill and brittle. Prefer explicit field checks for
ToolConfigas previously suggested.
224-229:⚠️ Potential issueGuard tool-call injection to
assistantrole only
AssistantMessage.ToolCallsis attached irrespective of the originating role. Embedding tool calls intouser,tool, orsystemmessages violates the Bifrost schema and risks upstream validation failures.-if len(toolCalls) > 0 { - mainMsg.AssistantMessage = &schemas.AssistantMessage{ - ToolCalls: &toolCalls, - } -} +if len(toolCalls) > 0 && role == schemas.ModelChatMessageRoleAssistant { + mainMsg.AssistantMessage = &schemas.AssistantMessage{ + ToolCalls: &toolCalls, + } +}
Pratham-Mishra04
force-pushed
the
06-09-feat_openai_integration_added_to_http_transport
branch
from
111399d to
7ee161c
Compare
Pratham-Mishra04
force-pushed
the
06-09-refactor_http_transport_gemini_integration_refactors
branch
from
e339c57 to
1618f5d
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (6)
transports/bifrost-http/integrations/genai/types.go (6)
172-180: Inline image data is not base64-encoded
part.InlineData.Datacontains raw bytes. Converting it withstring()embeds arbitrary bytes into the data-URL and produces invalid output.-import ( - "encoding/json" - "fmt" - "reflect" - "strings" +import ( + "encoding/base64" + "encoding/json" + "fmt" + "reflect" + "strings" ... -imageContent := schemas.ImageContent{ - Type: &imageType, - URL: fmt.Sprintf("data:%s;base64,%s", part.InlineData.MIMEType, string(part.InlineData.Data)), - MediaType: &part.InlineData.MIMEType, -} +encoded := base64.StdEncoding.EncodeToString(part.InlineData.Data) +imageContent := schemas.ImageContent{ + Type: &imageType, + URL: fmt.Sprintf("data:%s;base64,%s", part.InlineData.MIMEType, encoded), + MediaType: &part.InlineData.MIMEType, +}
243-254: Assistant & system images are silently droppedThe switch only handles
userandtool. Images supplied by the assistant or in a system instruction are never forwarded.case schemas.ModelChatMessageRoleTool: imageMsg.ToolMessage = &schemas.ToolMessage{ ImageContent: &imageContent } +case schemas.ModelChatMessageRoleAssistant: + imageMsg.AssistantMessage = &schemas.AssistantMessage{ ImageContent: &imageContent } +case schemas.ModelChatMessageRoleSystem: + imageMsg.SystemMessage = &schemas.SystemMessage{ ImageContent: &imageContent }
283-285: Use the dedicatedTopKfield instead of ExtraParams
ModelParametersalready has aTopKpointer. Stashing the value inExtraParamsis inconsistent with howTemperature,TopP, andMaxTokensare handled.-if config.TopK != nil { - params.ExtraParams["top_k"] = *config.TopK -} +if config.TopK != nil { + v := int(*config.TopK) + params.TopK = &v +}
131-134: Avoidreflect.DeepEqualfor zero-value detectionUsing reflection here is slow and brittle. An explicit field check is faster and doesn’t break when new fields are added.
-if !reflect.DeepEqual(r.ToolConfig, genai_sdk.ToolConfig{}) { +if r.ToolConfig.Mode != "" || len(r.ToolConfig.FunctionCallParameters) > 0 {
475-499:convertImageContentToGenAIPartreturns base64 text, not decoded bytes
InlineData.Dataexpects the raw binary payload, but the function currently passes the base64 string as-is.- base64Data := url[idx+1:] - return &genai_sdk.Part{ - InlineData: &genai_sdk.Blob{ - MIMEType: *imageContent.MediaType, - Data: []byte(base64Data), - }, - } + base64Data := url[idx+1:] + decoded, err := base64.StdEncoding.DecodeString(base64Data) + if err != nil { + return nil + } + return &genai_sdk.Part{ + InlineData: &genai_sdk.Blob{ + MIMEType: *imageContent.MediaType, + Data: decoded, + }, + }Remember to keep
encoding/base64in the import block.
225-230:⚠️ Potential issueGuard tool-call injection to assistant messages only
toolCallsare blindly attached even when the original role is notassistant, which violates the Bifrost schema and can trigger upstream validation errors.-if len(toolCalls) > 0 { - mainMsg.AssistantMessage = &schemas.AssistantMessage{ - ToolCalls: &toolCalls, - } -} +if len(toolCalls) > 0 && role == schemas.ModelChatMessageRoleAssistant { + mainMsg.AssistantMessage = &schemas.AssistantMessage{ + ToolCalls: &toolCalls, + } +}
Pratham-Mishra04
force-pushed
the
06-09-feat_openai_integration_added_to_http_transport
branch
from
7ee161c to
4ad600d
Compare
Pratham-Mishra04
force-pushed
the
06-09-refactor_http_transport_gemini_integration_refactors
branch
2 times, most recently
from
bb0425c to
c664c44
Compare
Pratham-Mishra04
force-pushed
the
06-09-feat_openai_integration_added_to_http_transport
branch
2 times, most recently
from
fbd3a73 to
e0e01b8
Compare
Pratham-Mishra04
force-pushed
the
06-09-refactor_http_transport_gemini_integration_refactors
branch
2 times, most recently
from
efe4ba6 to
46def42
Compare
There was a problem hiding this comment.
Actionable comments posted: 4
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (1)
transports/bifrost-http/integrations/genai/types.go(5 hunks)
🧰 Additional context used
🧠 Learnings (1)
transports/bifrost-http/integrations/genai/types.go (1)
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#64
File: transports/bifrost-http/integrations/genai/types.go:383-395
Timestamp: 2025-06-09T16:26:05.727Z
Learning: In the Bifrost schema (core/schemas/bifrost.go), only UserMessage and ToolMessage structs have ImageContent fields. AssistantMessage does not have an ImageContent field and cannot contain images. The schema design intentionally limits image content to user and tool messages only.
🔇 Additional comments (1)
transports/bifrost-http/integrations/genai/types.go (1)
226-235: Main message may be silently dropped when tool-calls originate from non-assistant roles
toolCallsare only attached whenrole == assistant, but the guard that addsmainMsgchecks either content or tool-calls.
If a (mis-labelled) request contains only aFunctionCallpart underuserortool, you will end up discarding the information entirely.Consider either:
- enforcing at parse time that
FunctionCallparts only appear withassistantrole, or- relaxing the guard so the message is inserted whenever a
FunctionCallwas detected, regardless of role.
Pratham-Mishra04
force-pushed
the
06-09-refactor_http_transport_gemini_integration_refactors
branch
from
46def42 to
65a8762
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (2)
transports/bifrost-http/integrations/genai/types.go(5 hunks)transports/bifrost-http/main.go(2 hunks)
🧰 Additional context used
🧠 Learnings (1)
transports/bifrost-http/integrations/genai/types.go (2)
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#64
File: transports/bifrost-http/integrations/genai/types.go:273-313
Timestamp: 2025-06-09T16:35:26.900Z
Learning: In convertGenerationConfigToParams method in transports/bifrost-http/integrations/genai/types.go, pre-allocating the ExtraParams map is preferred over lazy allocation because the method has multiple potential ExtraParams assignments, making the computational overhead of conditional checks exceed the memory savings of an empty map.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#64
File: transports/bifrost-http/integrations/genai/types.go:383-395
Timestamp: 2025-06-09T16:26:05.727Z
Learning: In the Bifrost schema (core/schemas/bifrost.go), only UserMessage and ToolMessage structs have ImageContent fields. AssistantMessage does not have an ImageContent field and cannot contain images. The schema design intentionally limits image content to user and tool messages only.
🧬 Code Graph Analysis (1)
transports/bifrost-http/integrations/genai/types.go (2)
core/schemas/bifrost.go (17)
BifrostRequest(54-64)ModelParameters(75-89)Tool(108-112)ModelChatMessageRoleSystem(25-25)ModelChatMessageRole(20-20)Function(101-105)BifrostMessage(143-152)ToolCall(266-270)ImageContent(174-179)FunctionCall(260-263)ModelChatMessageRoleAssistant(23-23)AssistantMessage(166-171)ModelChatMessageRoleUser(24-24)UserMessage(155-157)ModelChatMessageRoleTool(27-27)ToolMessage(160-163)FunctionParameters(92-98)core/utils.go (1)
Ptr(3-5)
🔇 Additional comments (1)
transports/bifrost-http/main.go (1)
14-16: Documentation update looks goodRe-formatting the example usage and adding an explicit note about the lack of streaming support improves readability and sets correct expectations. No further action required.
Also applies to: 31-32
Pratham-Mishra04
force-pushed
the
06-09-feat_openai_integration_added_to_http_transport
branch
from
e0e01b8 to
4cf3b95
Compare
Pratham-Mishra04
force-pushed
the
06-09-refactor_http_transport_gemini_integration_refactors
branch
from
65a8762 to
4c35179
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
transports/bifrost-http/integrations/genai/types.go (2)
424-447: 🧹 Nitpick (assertive)Skip assigning empty safety-ratings slice
candidate.SafetyRatings = ratingsis executed even whenratingsis empty, emitting[]instead ofnullin JSON.
While harmless, omitting the field reduces payload size and mirrors how other optional fields are handled.- candidate.SafetyRatings = ratings + if len(ratings) > 0 { + candidate.SafetyRatings = ratings + }
234-254: 💡 Verification agent🧩 Analysis chain
Potential silent data-loss for images on unsupported roles
The switch appends image messages only for
userandtool.
If the GenAI SDK ever emits images under other roles (e.g.systemor a future role), they will be silently dropped.case schemas.ModelChatMessageRoleUser: ... case schemas.ModelChatMessageRoleTool: ... +default: + // Consider logging / rejecting so the caller knows the image was ignoredNot urgent given today’s schema, but worth a TODO so future contributors aren’t surprised.
🏁 Script executed:
#!/bin/bash # Search for ModelChatMessageRole type definition and associated constants rg -n "^type ModelChatMessageRole" . rg -n "^const .*ModelChatMessageRole" -A5 .Length of output: 146
🏁 Script executed:
#!/bin/bash # Display the type definition and constants for ModelChatMessageRole rg -n 'type ModelChatMessageRole' -A5 core/schemas/bifrost.go rg -n 'const (' -A20 core/schemas/bifrost.goLength of output: 425
🏁 Script executed:
#!/bin/bash # Show the full constant block for ModelChatMessageRole rg -n 'type ModelChatMessageRole' -A15 core/schemas/bifrost.go # Search for any reference to ModelChatMessageRoleTool in the repo rg -n 'ModelChatMessageRoleTool' -n .Length of output: 1914
Ensure images aren’t silently dropped for unexpected roles
Right now only
userandtoolroles append image messages. If the SDK ever emits images underassistant,system,chatbot, or any future role, they’ll be dropped without notice.Locations to update:
- transports/bifrost-http/integrations/genai/types.go (around lines 234–254)
Suggested diff:
// Set image content based on role switch role { case schemas.ModelChatMessageRoleUser: imageMsg.UserMessage = &schemas.UserMessage{ ImageContent: &imageContent, } messages = append(messages, imageMsg) case schemas.ModelChatMessageRoleTool: imageMsg.ToolMessage = &schemas.ToolMessage{ ImageContent: &imageContent, } messages = append(messages, imageMsg) + default: + // TODO: log or return an error so we don’t silently drop images + log.Warnf("ignoring image for unsupported role %q", role) }Adding a
defaultensures future roles don’t lose data unexpectedly.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (2)
transports/bifrost-http/integrations/genai/types.go(5 hunks)transports/bifrost-http/main.go(2 hunks)
🧰 Additional context used
🧠 Learnings (1)
transports/bifrost-http/integrations/genai/types.go (2)
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#64
File: transports/bifrost-http/integrations/genai/types.go:273-313
Timestamp: 2025-06-09T16:35:26.914Z
Learning: In convertGenerationConfigToParams method in transports/bifrost-http/integrations/genai/types.go, pre-allocating the ExtraParams map is preferred over lazy allocation because the method has multiple potential ExtraParams assignments, making the computational overhead of conditional checks exceed the memory savings of an empty map.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#64
File: transports/bifrost-http/integrations/genai/types.go:383-395
Timestamp: 2025-06-09T16:26:05.777Z
Learning: In the Bifrost schema (core/schemas/bifrost.go), only UserMessage and ToolMessage structs have ImageContent fields. AssistantMessage does not have an ImageContent field and cannot contain images. The schema design intentionally limits image content to user and tool messages only.
🔇 Additional comments (3)
transports/bifrost-http/main.go (1)
14-16: Documentation change looks goodClearer example formatting and an explicit non-streaming disclaimer help avoid confusion.
No further action required.Also applies to: 31-32
transports/bifrost-http/integrations/genai/types.go (2)
223-227: Good guard – tool calls restricted toassistantrole onlyLimiting tool-call injection to the assistant role prevents schema violations for user / system messages.
Nicely done.
261-307: 🧹 Nitpick (assertive)Zero-value GenerationConfig always allocates a params struct
convertGenerationConfigToParamsunconditionally returns aModelParametersinstance with an emptyExtraParamsmap.
That works, but it means every Gemini request – even when no config is supplied – carries an allocation.If you want to avoid the extra object when nothing is set:
+empty := genai_sdk.GenerationConfig{} +if r.GenerationConfig == empty { + return nil +}Then call
ensureExtraParamslater when you first need the map.
Minor; feel free to ignore if the current trade-off (one tiny map vs. severalnilchecks) is deliberate.⛔ Skipped due to learnings
Learnt from: Pratham-Mishra04 PR: maximhq/bifrost#64 File: transports/bifrost-http/integrations/genai/types.go:273-313 Timestamp: 2025-06-09T16:35:26.914Z Learning: In convertGenerationConfigToParams method in transports/bifrost-http/integrations/genai/types.go, pre-allocating the ExtraParams map is preferred over lazy allocation because the method has multiple potential ExtraParams assignments, making the computational overhead of conditional checks exceed the memory savings of an empty map.
Pratham-Mishra04
force-pushed
the
06-09-refactor_http_transport_gemini_integration_refactors
branch
from
4c35179 to
96ee207
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (3)
transports/bifrost-http/integrations/genai/router.go(1 hunks)transports/bifrost-http/integrations/genai/types.go(5 hunks)transports/bifrost-http/main.go(2 hunks)
🧰 Additional context used
🧠 Learnings (1)
transports/bifrost-http/integrations/genai/types.go (2)
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#64
File: transports/bifrost-http/integrations/genai/types.go:273-313
Timestamp: 2025-06-09T16:35:26.914Z
Learning: In convertGenerationConfigToParams method in transports/bifrost-http/integrations/genai/types.go, pre-allocating the ExtraParams map is preferred over lazy allocation because the method has multiple potential ExtraParams assignments, making the computational overhead of conditional checks exceed the memory savings of an empty map.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#64
File: transports/bifrost-http/integrations/genai/types.go:383-395
Timestamp: 2025-06-09T16:26:05.777Z
Learning: In the Bifrost schema (core/schemas/bifrost.go), only UserMessage and ToolMessage structs have ImageContent fields. AssistantMessage does not have an ImageContent field and cannot contain images. The schema design intentionally limits image content to user and tool messages only.
🔇 Additional comments (4)
transports/bifrost-http/integrations/genai/router.go (1)
51-55: Replace hard-coded suffix stripping with a short loop for maintainabilityThe three
strings.TrimSuffixcalls work, but they scale poorly as soon as Google adds yet another verb.
A tiny helper slice + loop keeps the intent obvious and makes the next addition a one-liner.-// Remove Google GenAI API endpoint suffixes if present -modelStr = strings.TrimSuffix(modelStr, ":generateContent") -modelStr = strings.TrimSuffix(modelStr, ":streamGenerateContent") -modelStr = strings.TrimSuffix(modelStr, ":countTokens") +// Strip known API verbs that may trail the model name. +for _, sfx := range []string{ + ":generateContent", + ":streamGenerateContent", + ":countTokens", +} { + modelStr = strings.TrimSuffix(modelStr, sfx) +}No functional change, just easier future upkeep.
[ suggest_optional_refactor ]transports/bifrost-http/integrations/genai/types.go (2)
150-181: 👍 Correct base64 handling for inline imagesSwitching to
base64.StdEncoding.EncodeToStringfixes the RFC-2397 / binary-data bug from earlier iterations.
Looks good and aligns with the reverse conversion path.
236-255: Images for roles other than user/tool are intentionally skipped – please confirmThe switch appends image messages only for
userandtoolroles.
Given the schema (onlyUserMessage&ToolMessagehaveImageContent), this is correct, but any future schema change (e.g. system images) will silently drop data.Request verification that this behaviour is documented & covered by tests.
transports/bifrost-http/main.go (1)
14-17: Comment-only change – no action requiredThe re-flowed example block reads better; nothing else to note.
Pratham-Mishra04
force-pushed
the
06-09-feat_openai_integration_added_to_http_transport
branch
from
4cf3b95 to
72f5791
Compare
Pratham-Mishra04
force-pushed
the
06-09-refactor_http_transport_gemini_integration_refactors
branch
from
96ee207 to
371e003
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (2)
transports/bifrost-http/integrations/genai/types.go (2)
283-309:candidate_countis still written twice – please drop the duplicate blockThis duplication was flagged previously and is still present.
Keeping both writes is harmless at runtime but clutters the param map and invites divergence during future edits.@@ if config.CandidateCount > 0 { params.ExtraParams["candidate_count"] = config.CandidateCount } @@ - if config.CandidateCount > 0 { - params.ExtraParams["candidate_count"] = config.CandidateCount - }
470-475: 🧹 Nitpick (assertive)Use a neutral fallback MIME type
image/pngis image-specific. If non-image blobs (audio, pdf, etc.) ever flow through, the default becomes misleading.
application/octet-streamis the conventional catch-all for unknown binary data.- // Determine MIME type with fallback default - mimeType := "image/png" // Default MIME type for images + // Determine MIME type with neutral fallback + mimeType := "application/octet-stream"
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (3)
transports/bifrost-http/integrations/genai/router.go(1 hunks)transports/bifrost-http/integrations/genai/types.go(5 hunks)transports/bifrost-http/main.go(2 hunks)
🧰 Additional context used
🧠 Learnings (1)
transports/bifrost-http/integrations/genai/types.go (2)
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#64
File: transports/bifrost-http/integrations/genai/types.go:273-313
Timestamp: 2025-06-09T16:35:26.914Z
Learning: In convertGenerationConfigToParams method in transports/bifrost-http/integrations/genai/types.go, pre-allocating the ExtraParams map is preferred over lazy allocation because the method has multiple potential ExtraParams assignments, making the computational overhead of conditional checks exceed the memory savings of an empty map.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#64
File: transports/bifrost-http/integrations/genai/types.go:383-395
Timestamp: 2025-06-09T16:26:05.777Z
Learning: In the Bifrost schema (core/schemas/bifrost.go), only UserMessage and ToolMessage structs have ImageContent fields. AssistantMessage does not have an ImageContent field and cannot contain images. The schema design intentionally limits image content to user and tool messages only.
🧬 Code Graph Analysis (1)
transports/bifrost-http/integrations/genai/types.go (2)
core/schemas/bifrost.go (17)
BifrostRequest(54-64)ModelParameters(75-89)Tool(108-112)ModelChatMessageRoleSystem(25-25)ModelChatMessageRole(20-20)Function(101-105)BifrostMessage(143-152)ToolCall(273-277)ImageContent(181-186)FunctionCall(267-270)ModelChatMessageRoleTool(27-27)ToolMessage(160-163)ModelChatMessageRoleAssistant(23-23)AssistantMessage(166-171)ModelChatMessageRoleUser(24-24)UserMessage(155-157)FunctionParameters(92-98)core/utils.go (1)
Ptr(3-5)
Pratham-Mishra04
force-pushed
the
06-09-refactor_http_transport_gemini_integration_refactors
branch
from
371e003 to
d0f2c73
Compare
Pratham-Mishra04
force-pushed
the
06-09-feat_openai_integration_added_to_http_transport
branch
from
72f5791 to
d68ae37
Compare
Pratham-Mishra04
force-pushed
the
06-09-refactor_http_transport_gemini_integration_refactors
branch
from
d0f2c73 to
e009a8b
Compare
There was a problem hiding this comment.
Caution
Inline review comments failed to post. This is likely due to GitHub's limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.
Actionable comments posted: 1
♻️ Duplicate comments (4)
transports/bifrost-http/integrations/genai/types.go (2)
287-287: 🧹 Nitpick (assertive)Consider using the original struct field for StopSequences.
While the current implementation works, taking the address of the slice from the original struct is clearer and avoids potential allocations.
-if len(config.StopSequences) > 0 { - params.StopSequences = &config.StopSequences +if len(r.GenerationConfig.StopSequences) > 0 { + params.StopSequences = &r.GenerationConfig.StopSequences }
283-309:⚠️ Potential issueRemove duplicate
candidate_countassignment.The
candidate_countparameter is assigned twice - at lines 283-285 and again at lines 307-309. This is redundant and could cause confusion.if config.CandidateCount > 0 { params.ExtraParams["candidate_count"] = config.CandidateCount } if len(config.StopSequences) > 0 { params.StopSequences = &config.StopSequences } if config.PresencePenalty != nil { params.PresencePenalty = bifrost.Ptr(float64(*config.PresencePenalty)) } if config.FrequencyPenalty != nil { params.FrequencyPenalty = bifrost.Ptr(float64(*config.FrequencyPenalty)) } if config.Seed != nil { params.ExtraParams["seed"] = *config.Seed } if config.ResponseMIMEType != "" { params.ExtraParams["response_mime_type"] = config.ResponseMIMEType } if config.ResponseLogprobs { params.ExtraParams["response_logprobs"] = config.ResponseLogprobs } if config.Logprobs != nil { params.ExtraParams["logprobs"] = *config.Logprobs } -if config.CandidateCount > 0 { - params.ExtraParams["candidate_count"] = config.CandidateCount -}transports/bifrost-http/main.go (1)
15-15: Update example command to reference correct filename.The command still references
http.go, but the file ismain.go. Update for documentation consistency.transports/bifrost-http/integrations/genai/router.go (1)
51-54: Refactor hard-coded suffix removal for better maintainability.The additional suffixes work correctly, but the manual approach becomes harder to maintain as more endpoints are added.
Consider using a loop over a slice of suffixes as suggested in the previous review to keep this DRY and extensible.
🛑 Comments failed to post (1)
transports/bifrost-http/integrations/genai/types.go (1)
472-472: 🧹 Nitpick (assertive)
Consider using
application/octet-streamas the default MIME type.While
image/pngworks for images, usingapplication/octet-streamas the default would be more generic and handle non-image binary data correctly if this code path is ever used for other content types.-mimeType := "image/png" // Default MIME type for images +mimeType := "application/octet-stream" // Default MIME type for binary dataAlso update line 514:
-return "image/jpeg" // default +return "application/octet-stream" // defaultAlso applies to: 514-514
🤖 Prompt for AI Agents
In transports/bifrost-http/integrations/genai/types.go at lines 472 and 514, the default MIME type is set to "image/png", which is too specific. Change the default MIME type to "application/octet-stream" to make it more generic and suitable for handling various binary data types. Update both lines accordingly to reflect this change.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
transports/bifrost-http/integrations/genai/types.go (2)
157-160: Error silently swallowed – original remark still applies.
The marshal failure is still ignored without any trace; a debug/trace log would greatly aid troubleshooting malformedArgs.
286-288: Pointer taken from a local copy – original nit unchanged.
params.StopSequences = &config.StopSequencesstill captures the slice header from the localconfigcopy instead ofr.GenerationConfig.
It’s harmless but allocates. Using the original struct is clearer.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (3)
transports/bifrost-http/integrations/genai/router.go(1 hunks)transports/bifrost-http/integrations/genai/types.go(5 hunks)transports/bifrost-http/main.go(2 hunks)
🧰 Additional context used
🧠 Learnings (1)
transports/bifrost-http/integrations/genai/types.go (3)
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#64
File: transports/bifrost-http/integrations/genai/types.go:273-313
Timestamp: 2025-06-09T16:35:26.914Z
Learning: In convertGenerationConfigToParams method in transports/bifrost-http/integrations/genai/types.go, pre-allocating the ExtraParams map is preferred over lazy allocation because the method has multiple potential ExtraParams assignments, making the computational overhead of conditional checks exceed the memory savings of an empty map.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#64
File: transports/bifrost-http/integrations/genai/types.go:383-395
Timestamp: 2025-06-09T16:26:05.777Z
Learning: In the Bifrost schema (core/schemas/bifrost.go), only UserMessage and ToolMessage structs have ImageContent fields. AssistantMessage does not have an ImageContent field and cannot contain images. The schema design intentionally limits image content to user and tool messages only.
Learnt from: Pratham-Mishra04
PR: maximhq/bifrost#64
File: transports/bifrost-http/integrations/genai/types.go:470-515
Timestamp: 2025-06-10T11:06:57.681Z
Learning: Bifrost currently only supports images and does not support non-image blob types like audio, PDF, or other binary data formats.
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: Graphite / mergeability_check
🔇 Additional comments (2)
transports/bifrost-http/integrations/genai/router.go (1)
51-58: Suffix-stripping now DRY & future-proof – looks good.Looping over the slice eliminates the old duplication and keeps the logic maintainable as new endpoints appear.
No functional or performance concerns. 👍transports/bifrost-http/main.go (1)
14-16: Documentation formatting improvement looks good.The reformatting of the example usage section improves readability with proper indentation, and the command reference is correctly updated to
main.go.
Pratham-Mishra04
force-pushed
the
06-09-refactor_http_transport_gemini_integration_refactors
branch
from
e009a8b to
0709d38
Compare
Merge activity
|
akshaydeo
changed the base branch from
06-09-feat_openai_integration_added_to_http_transport
to
graphite-base/64
akshaydeo
force-pushed
the
06-09-refactor_http_transport_gemini_integration_refactors
branch
from
0709d38 to
b440cd4
Compare
There was a problem hiding this comment.
Bug: Image Handling Fails for Non-User Roles
Image content is silently dropped for assistant and system role messages in convertContentToBifrostMessages because the function's switch statement only appends images for user and tool roles, resulting in data loss.
transports/bifrost-http/integrations/genai/types.go#L235-L255
bifrost/transports/bifrost-http/integrations/genai/types.go
Lines 235 to 255 in b440cd4
Bug: Gemini Content Parsing Prematurely Ends
The convertContentToBifrostMessages function returns prematurely when a FunctionResponse part is encountered within a Gemini content object. This causes any subsequent parts (e.g., text, images, or other function calls) in the same content object to be ignored, leading to incomplete message conversion and loss of valid content.
transports/bifrost-http/integrations/genai/types.go#L191-L207
bifrost/transports/bifrost-http/integrations/genai/types.go
Lines 191 to 207 in b440cd4
Was this report helpful? Give feedback by reacting with 👍 or 👎
akshaydeo
deleted the
06-09-refactor_http_transport_gemini_integration_refactors
branch
Enhanced Gemini Chat Integration for Bifrost HTTP Transport
This PR enhances the Gemini AI integration by adding comprehensive support for Gemini's advanced features. The implementation now properly handles:
SystemInstructionfieldCachedContentfieldResponseModalitiesThe changes significantly improve the fidelity of the Gemini API integration by ensuring all request and response fields are properly mapped between Bifrost's internal representation and Gemini's SDK types.