Intro

Gemini is a Google GenAI product that offers another set of GenAI models for users. Since December 2023, our "gemini" branch has seen significant community interest, including the integration of Gemma with autogen earlier this year. We're now excited to bring the experimental Gemini branch into AutoGen's main release. While we currently offer Gemini chat completion and vision models, this roadmap aims to unlock more potential of Google Gemini within AutoGen, laying out TODOs, feature requests, and future directions.

The Roadmap

Merged PR: #2360

Future:

• GenerationConfig Parameters: temperature, max_tokens, etc.
• Multiple Responses: generate several responses at the same time.
• More Accurate Cost/Token Calculation. Great feedback from @joshkyh
• Function Calling.
• JSON Output: structure outputs in JSON.
• Image Generation.
• Other Modalities: audio/video generation and processing.
• Safety Settings: user-customizable safety controls. Feedback from @marklysze chat.send_message("hi", stream=stream, safety_settings=safety). documentation
• Multi-turn Vision Chat: improve conversational context for image tasks.
• Streaming: real-time output from the client.