Skip to content

feat: add google_cloud provider (Vertex AI) for Gemini models - #482

Draft
cloetzi wants to merge 4 commits into
mainfrom
feat/google-cloud-provider
Draft

cloetzi wants to merge 4 commits into
mainfrom
feat/google-cloud-provider

Conversation

@cloetzi

@cloetzi cloetzi commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Adds a google_cloud provider for Gemini models on Google Cloud Vertex AI (now called Gemini Enterprise Agent Platform).

Why not the official gem

google-cloud-ai_platform-v1 is the only official Ruby gem, and neither of its transports works well under fibers:

  • gRPC (its default): blocks the Async reactor. 5 concurrent calls took 5.04s instead of ~1s.
  • REST: streaming starts one thread per stream. That thread leaks, still holding its socket, when the caller stops early with break or Async::Stop.

This provider instead uses its own Net::HTTP client, like Gemini::Client, with googleauth for tokens. Vertex's Gemini request/response JSON matches the Gemini Developer API, so GoogleCloud subclasses Gemini and only changes the transport and URL paths.

Fiber-safe auth

On Cloud Run, GKE and GCE, googleauth gets tokens through google-cloud-env, which spots re-entry by comparing Thread.current. When several fibers refresh a token at once, it raises a false ThreadError: deadlock. The client puts every googleauth call behind a process-wide Mutex, which works per fiber, and shares one set of Application Default Credentials (ADC) across clients.

Checked against a fake metadata server, 5 concurrent fibers:

  • With the lock: 0 errors, and the run took about as long as one call.
  • Without the lock: 8 errors.

A unit test fails if the lock is removed.

Regions

Location is set on the client. The default is "global", and the docs say plainly that it has no data-residency guarantee. They show how to bind a location with a config.google_cloud.client Proc or with a provider subclass per location.

Commits

  1. Extract Riffer::Wire::SSE, a shared SSE decoder that Gemini now uses.
  2. GoogleCloud::Client with fiber-safe token refresh.
  3. google_cloud provider for Gemini models, with VCR cassettes recorded against a real project.
  4. Docs.

Follow-ups

  • Claude models on Vertex, stacked on this PR in feat/google-cloud-claude. It's waiting on Model Garden access to record cassettes.
  • Mapping tags to Vertex labels.
  • Sending gs:// files by URL.

🤖 Generated with Claude Code

cloetzi and others added 4 commits October 8, 2026 13:18
Add a provider-independent incremental Server-Sent Events decoder
(event/data fields, multi-line data, CRLF/CR/LF, chunk boundaries) and
switch the Gemini provider's inline frame parsing to it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Riffer::Providers::GoogleCloud::Client is a Net::HTTP transport for
Vertex AI (Gemini Enterprise Agent Platform), modelled on
Gemini::Client. It is bound to one project and location, derives the
endpoint from the location (global, us/eu multi-region, or regional),
and authenticates with a googleauth credentials object, defaulting to
Application Default Credentials shared process-wide.

googleauth's metadata-server path (GCE, Cloud Run, GKE) runs through
google-cloud-env's process-wide LazyValue cache, which detects
reentrancy by Thread: concurrent fibers refreshing a token raise
ThreadError ("deadlock: tried to call LazyValue#get from its own
computation"), hidden by googleauth's retries until latency surfaces it
as Google::Auth::AuthorizationError. Token access is serialized behind
one fiber-aware Mutex, and apply! refreshes only a missing or
near-expiry token, so concurrent fibers share a single refresh.

googleauth is an optional dependency, added to the Gemfile and loaded
with depends_on only on the ADC path.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Riffer::Providers::GoogleCloud (provider key google_cloud, semconv
gcp.vertex_ai) serves Gemini models on Vertex AI. Vertex accepts the
same generateContent body as the Gemini Developer API, so the provider
subclasses the Gemini provider and swaps only the transport and the
publishers/google/models resource path. Model ids may carry an
@Version suffix.

Configured through Riffer.config.google_cloud: project_id, location
(default "global"), credentials (default ADC) and client (instance or
Proc). Tags are not mapped to Vertex labels yet.

The VCR tests await cassettes recorded with real credentials; the
helper filters the access token and project id.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Add docs/providers/GOOGLE_CLOUD.md (Gemini models for now) with its
site manifest entry, and list the provider wherever the others appear.
The page calls out that the default "global" location carries no
data-residency guarantee and shows how to bind a location through
config.google_cloud.client or a per-location provider subclass.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 8, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant