Skip to content

Conversation

@ItsWendell
Copy link

@ItsWendell ItsWendell commented Dec 23, 2025

Summary

This PR upgrades the @temporalio/ai-sdk package to support AI SDK v6 and adds embedding model support for building RAG pipelines in Temporal workflows.

Key Changes

  • AI SDK v6 Upgrade: Migrate from AI SDK v5 (@ai-sdk/provider v2 types) to AI SDK v6 (@ai-sdk/provider v3 types)
  • Embedding Model Support: Add TemporalEmbeddingModel class and invokeEmbeddingModel activity for durable embedding generation in workflows
  • Enhanced Provider Options: New TemporalProviderOptions interface allows configuring separate ActivityOptions for language vs embedding models
  • Improved MCP Types: Use jsonSchema() helper instead of manual symbol hacks, proper FlexibleSchema typing
  • Updated Tests: All AI SDK tests updated for v6 compatibility with new embedding model tests

Motivation

The current ai-sdk package uses AI SDK v5 and lacks embedding model support. This limits its use in modern AI applications that require:

  1. Embedding models - Essential for RAG pipelines, semantic search, and vector-based retrieval
  2. AI SDK v6 - Latest features and improvements from the Vercel AI SDK
  3. Per-model configuration - Different timeout requirements for fast language model calls vs potentially slower embedding batches

Breaking Changes

This is a breaking change that requires updating peer dependencies:

Dependency Before After
@ai-sdk/provider ^2.0.0 ^3.0.0
ai ^5.0.91 ^6.0.0
@ai-sdk/mcp ^0.0.8 ^1.0.0

Usage Examples

Embedding Models in Workflows

import { embedMany } from 'ai';
import { temporalProvider } from '@temporalio/ai-sdk';

export async function ragWorkflow(documents: string[]) {
  // Generate embeddings through Temporal activities
  const { embeddings } = await embedMany({
    model: temporalProvider.embeddingModel('text-embedding-3-small'),
    values: documents,
  });
  
  // Use embeddings for semantic search...
  return embeddings;
}

Per-Model Activity Options

import { TemporalProvider } from '@temporalio/ai-sdk';

const provider = new TemporalProvider({
  default: {
    startToCloseTimeout: '5 minutes',
    retry: { maximumAttempts: 3 },
  },
  languageModel: {
    startToCloseTimeout: '10 minutes', // LLM calls can take longer
  },
  embeddingModel: {
    startToCloseTimeout: '2 minutes', // Embeddings are faster
  },
});

Test Results ✅

All AI SDK tests pass:

✔ Hello world agent responds in haikus (704ms)
✔ Middleware (530ms)
✔ Generate object (551ms)
✔ Embedding model generates embeddings (511ms)
✔ Tools workflow can use AI tools (591ms)
✔ Telemetry (1.3s)
✔ MCP Use (8.5s) - passes locally, skipped in CI

6 tests passed
1 test skipped (MCP - skipped in CI due to server setup)

Test Plan

  • Update TestModel and TestProvider to v3 interfaces
  • Add TestEmbeddingModel for mock embedding responses
  • Add embedding workflow test (embeddingWorkflow)
  • Verify invokeEmbeddingModel activity scheduling
  • Update tool definition syntax (inputSchema instead of parameters)
  • Update middleware type to LanguageModelV3Middleware
  • Verify MCP test passes locally with jsonSchema() helper
  • All tests pass locally

@CLAassistant
Copy link

CLAassistant commented Dec 23, 2025

CLA assistant check
All committers have signed the CLA.

@ItsWendell ItsWendell changed the title feat(ai-sdk): upgrade to AI SDK v3 with embedding model support feat(ai-sdk): upgrade to AI SDK v6 with embedding model support Dec 23, 2025
This PR upgrades the ai-sdk package to support AI SDK v6 and adds
embedding model support for RAG pipelines in Temporal workflows.

## Changes

### AI SDK v6 Upgrade
- Upgrade from AI SDK v5 (`@ai-sdk/provider` v2 types) to AI SDK v6 (v3 types)
- Update peer dependencies to `@ai-sdk/provider@^3.0.0`, `ai@^6.0.0`, `@ai-sdk/mcp@^1.0.0`

### Embedding Model Support
- Add `TemporalEmbeddingModel` class implementing `EmbeddingModelV3`
- Add `invokeEmbeddingModel` activity for durable embedding generation
- Add `InvokeEmbeddingModelArgs` and `InvokeEmbeddingModelResult` types
- Implement `embeddingModel()` method on `TemporalProvider`

### Enhanced Provider Options
- Add `TemporalProviderOptions` interface for per-model activity configuration
- Support separate `ActivityOptions` for language vs embedding models
- Default options can be overridden by model-specific options

### Type Re-exports
- Re-export common AI SDK types from `@ai-sdk/provider` for convenience
- Includes `LanguageModelV3*`, `EmbeddingModelV3*`, and `ProviderV3` types

### Other Improvements
- Fix timestamp validation in language model response handling
- Remove debug logging from MCP client
- Use `ToolExecutionOptions` instead of `ToolCallOptions`
@ItsWendell ItsWendell force-pushed the feat/ai-sdk-v3-embedding-support branch from 4cdbc98 to be8bf08 Compare December 23, 2025 13:11
Updates all AI SDK tests to use v3 types and adds comprehensive tests
for the new embedding model functionality.

## Changes

### Test Infrastructure Updates
- Update `TestModel` and `TestProvider` to implement v3 interfaces
- Add `TestEmbeddingModel` class for mocking embedding responses
- Update `TestProvider` to support both language and embedding models
- Fix tool call response format (`args` instead of `input`)

### New Embedding Model Tests
- Add `embeddingWorkflow` to test workflows
- Add `embeddingGenerator` for mock embedding responses
- Add test case 'Embedding model generates embeddings'
- Verify `invokeEmbeddingModel` activity is called correctly

### Dependency Updates
- Update `@ai-sdk/provider` to `^3.0.0`
- Update `ai` to `^6.0.0`
- Update `@ai-sdk/openai` to `^1.0.0`
- Update `@ai-sdk/mcp` to `^1.0.0`

### Type Updates
- Update middleware type from `LanguageModelV2Middleware` to `LanguageModelV3Middleware`
- Update tool definition from `inputSchema` to `parameters`
Updates test files to match AI SDK v6 type requirements:

- Use LanguageModelV3FinishReason object format with `unified` and `raw` fields
- Use nested LanguageModelV3Usage structure with inputTokens/outputTokens objects
- Use `input` property for LanguageModelV3ToolCall (not `args`)
- Add required `warnings` field to EmbeddingModelV3Result
- Add `specificationVersion: 'v3'` to LanguageModelV3Middleware
- Use `inputSchema` property in tool() (not `parameters`)
- Cast OpenAI provider to ProviderV3 for remote tests

Also updates pnpm-workspace.yaml to allow new AI SDK packages in
minimumReleaseAgeExclude.
- Remove unnecessary `as any` casts in activities.ts and load-polyfills.ts
- Remove AI SDK type re-exports from index.ts (users should import directly)
- Fix misleading supportsParallelCalls comment in provider.ts
- Update @ai-sdk/openai to v3 for native ProviderV3 support (removes casts)
- Revert pnpm-workspace.yaml minimumReleaseAgeExclude workaround
- Use FlexibleSchema<unknown> for inputSchema instead of any
- Rename args to input in CallToolArgs (AI SDK v6 convention)
- Replace manual symbol hacks with proper jsonSchema() helper
- Change clientArgs and other any types to unknown

The jsonSchema() helper is the same approach used by @ai-sdk/mcp
internally. MCP test passes locally.
@ItsWendell ItsWendell marked this pull request as ready for review December 23, 2025 15:49
@ItsWendell ItsWendell requested a review from a team as a code owner December 23, 2025 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants