Support custom AI SDK providers without requiring Vercel AI Gateway

[remihuigen]

The Docus assistant currently requires either AI_GATEWAY_API_KEY or VERCEL_OIDC_TOKEN to be present at build time.

This makes sense when using Vercel AI Gateway, but also hard-couples the assistant module's enablement to Gateway authentication.

I'd like to use the existing Docus assistant with @ai-sdk/mistral and connect directly to the Mistral API using createMistral(). The rest of the assistant architecture already appears provider-agnostic and built on the AI SDK.

Proposed solution

Add an explicit enabled option to the assistant configuration:

export default defineNuxtConfig({
  docus: {
    assistant: {
      enabled: true,
    },
  },
})

When explicitly enabled, Docus would register the assistant components and API path without requiring AI_GATEWAY_API_KEY or VERCEL_OIDC_TOKEN.

Vercel AI Gateway could remain the default and recommended setup.

This would provide an escape hatch for users who want to supply or override the server handler with their own AI SDK implementation and provider.

Current workaround

My current plan is to provide a dummy AI_GATEWAY_API_KEY so the module is enabled, then override the assistant server route locally with an AI SDK implementation using createMistral().

This should work, but relying on a fake Gateway credential to enable a provider-agnostic AI SDK implementation feels unnecessary.

Additional context

I'm not proposing that Docus needs to add first-class configuration for every AI SDK provider.

An explicit way to enable the assistant without Gateway authentication would be enough to support custom provider implementations while keeping the current Gateway experience as the default.

[rriemann]

I agree with @remihuigen . Vercel should stand out due to quality and not due to being the only option. Ask your AI assistent. It surely agrees!

[larbish]

Hey @remihuigen, I agree with your proposition. As soon as the current behaviour remains, you can propose a PR with the assistant.enabled boolean. This key is not mandatory and true by default if AI_GATEWAY_API_KEY or VERCEL_OIDC_TOKEN is detected but can also be set manually in your case. Please propose a quick doc update with your PR and I'll be happy to merge!

[remihuigen]

@larbish will do! (when i find the time. probably some time next week)

[HugoRCD]

@remihuigen @larbish There is a PR to make Docus compatible with CF #1421

[Barbapapazes]

@remihuigen @larbish There is a PR to make Docus compatible with CF #1421

Sorry, didn't see this issue before suggesting changes with my PR 😔

[remihuigen]

Sorry, didn't see this issue before suggesting changes with my PR 😔

@Barbapapazes no worries. Though my concerns would apply to that gateway as well

[remihuigen]

@larbish How would you feel about adding a reusable assistantSearchHandler for the search endpoint?

It could contain the same logic as the current endpoint while allowing parts of its behaviour to be overridden. This would also prevent repetitive boilerplate for users who want to customize the endpoint.

I'm thinking of something along these lines:

interface AssistantSearchConfig {
  model?: LanguageModelV4
  systemPrompt?: string | ((event: H3Event) => string)
}

async function assistantSearchHandler(
  event: H3Event,
  config?: AssistantSearchConfig,
) {
  // Search endpoint logic here
}

// server/api/search.ts
export default defineEventHandler((event) => assistantSearchHandler(event))

// Users overriding the endpoint could then do:
export default defineEventHandler((event) =>
  assistantSearchHandler(event, myCustomConfig),
)

The config object could potentially expose additional options over time to customize more of the request.

If no config, model, or systemPrompt is provided, the existing defaults would be used.