llm-vertex

Plugin for LLM adding support for Google Cloud Vertex AI.

Please note that this plugin is for Vertex AI specifically, not Google AI Studio.

For Gemini support using AI Studio, please see llm-gemini instead.

Supported models:

• gemini-2.5-pro-preview-06-05
• gemini-2.5-pro-preview-05-06
• gemini-2.5-pro-exp-03-25
• gemini-2.5-flash-preview-05-20
• gemini-2.5-flash-preview-04-17
• gemini-2.0-flash-001
• gemini-2.0-flash-lite-001
• gemini-2.0-flash-thinking-exp-01-21
• gemini-1.5-flash
• gemini-1.5-pro
• gemini-1.0-pro
• gemini-1.0-pro-vision

Installation

See Installing Plugins for detailed instructions.

Method 1: Use llm

llm install llm-vertex

Method 2: Use pip

pip install llm-vertex

Use

First, authenticate using gcloud:

gcloud auth application-default login

Export two environment variables for the GCP Project and location you want to use:

export VERTEX_PROJECT_ID=gcp-project-id VERTEX_LOCATION=us-east1

Run llm and specify one of the provided models:

❯ llm -m vertex-gemini-1.5-pro-preview-0409 "What's one clever name for a pet pelican?"
"Gulliver" would be a clever name for a pet pelican, referencing both its large gullet and its potential for long journeys! 🦜

Development

Create and activate a virtual environment:

python -m venv .venv
source .venv/bin/activate

Install the package in development mode with test dependencies:

pip install -e '.[dev]'

Run the tests:

python -m pytest

The tests use mocking to avoid requiring actual Google Cloud credentials during development, but do not really test actual functionality outside of making sure the plugin is installed and can be used.