Help needed: API to summarize Youtube video via Gemini's mutlimodal capabilities?

[tan-yong-sheng]

I believe Google AI studio's has supported native information extraction from youtube video via youtube url, but I failed to find on how to do that in litellm. Could I do this using litellm gemini's pass-through instead?

Documentation: https://ai.google.dev/gemini-api/docs/video-understanding#youtube

response = client.models.generate_content(
    model='models/gemini-2.0-flash',
    contents=types.Content(
        parts=[
            types.Part(
                file_data=types.FileData(file_uri='https://www.youtube.com/watch?v=9hE5-98ZeCg')
            ),
            types.Part(text='Please summarize the video in 3 sentences.')
        ]
    )
)

Hope for help, thanks.

[tan-yong-sheng]

I currently use litellm gemini pass-through to achieve this function, as shown below:

curl "http://litellm:4000/gemini/v1beta/models/gemini-2.5-flash:generateContent?key=$LITELLM_API_KEY" \
  -H "Content-Type: application/json" \
  -X POST \
  -d '{
    "contents": [{
      "parts": [
        {
          "fileData": {
            "fileUri": "https://youtu.be/_5uSbMMOkCQ",
            "mimeType": "video/*"
          }
        },
        {
          "text": "Please provide a full transcript and a 3‑sentence summary."
        }
      ]
    }]
  }'

[ishaan-jaff]

@tan-yong-sheng would you be willig to contribute a tutorial for this ? We have new native /v1beta/models/ routes on litellm proxy

[tan-yong-sheng]

Hi @ishaan-jaff , sorry for late reply. Wanna check with you how to use the new native /v1beta/models using curl requests? Or is it compulsory to use google-genai sdk: https://googleapis.github.io/python-genai/?

I am trying to connect to gemini api studio first (not Vertex AI), but it keeps coming this error (Note: i have added litellm API key behind the key parameter):

Gemini's passthrough work perfectly when i tried...

[ishaan-jaff]

@tan-yong-sheng please pass the litellm virtual key in the authentication headers in your curl. You are missing that

[tan-yong-sheng]

Hi, I have tried to pass LITELLM virtual key in different ways as shown in the image below... I am thinking it's because I set the model_name to be gemini/gemini-2.0-flash in litellm-config.yaml instead of gemini-2.0-flash. Thus, the passthrough works perfectly yet I couldn't authenticate properly.

Just my temporary investigation that haven't been verified, as I might not be able to look deep into it because I have to amend a lot of things in litellm-config.yaml (since I have setup multiple gemini accounts in LiteLLM) and I would need to allocate my free time for this.

Below are my trial:

[ishaan-jaff]

your last request is correct - see you passed x-litellm-api-key. But just make sure the parts in your content has a role