Warning
This project is in beta. The API is subject to changes and may break.
fluent-ai is a lightweight, type-safe AI toolkit that seamlessly integrates multiple AI providers. It features structured outputs, streaming capabilities, and job serialization support.
Zod is a popular type of validation library for TypeScript and JavaScript that allows developers to define and validate data schemas in a concise and type-safe manner. fluent-ai is built upon zod.
npm install fluent-ai zod@next
fluent-ai includes support for multiple AI providers and modalities.
provider | chat completion | embedding | image generation | list models |
---|---|---|---|---|
anthropic | ✅ | ✅ | ||
fal | ✅ | |||
✅ | ||||
ollama | ✅ | ✅ | ✅ | |
openai | ✅ | ✅ | ✅ | ✅ |
voyage | ✅ |
By default, API keys for providers are read from environment variable (process.env
) following the format <PROVIDER>_API_KEY
(e.g., OPENAI_API_KEY
, ANTHROPIC_API_KEY
).
You can also initialize a provider with manual API key settings:
import { openai } from "fluent-ai";
openai({ apiKey: "<key>" });
For more examples with different AI providers, check out the examples directory.
Don't see your AI providers? Feel free to open an issue or start a discussion to request support. Join our Discord community
Each request to AI providers is wrapped in a Job
. which can also serialized and deserialized. A fluent API with method chaining help create jobs easily.
import { openai, user } from "fluent-ai";
const job = openai()
.chat("gpt-4o-mini")
.messages([user("Hi")])
.temperature(0.5)
.maxTokens(1024);
Alternatively, fluent-ai supports declarative job creation using JSON objects, with full TypeScript autocompletion support.
import { load } from "fluent-ai";
const job = load({
provider: "openai",
type: "chat",
input: {
model: "gpt-4o-mini",
messages: [{ role: "user", content: "hi" }],
temperature: 0.5,
},
});
fluent-ai provides built-in TypeScript type definitions and schema validation for jobs:
import { type Job } from "fluent-ai"; // TypeScript type
import { JobSchema } from "fluent-ai"; // Zod schema
import { jobJSONSchema } from "fluent-ai"; // JSON Schema
To serialize a job to a JSON object, use the dump
method:
const payload = job.dump();
This allows you to save the job's state for later use, such as storing it in a queue or database.
To recreate and execute a job from the JSON object, use the load
function:
import { load } from "fluent-ai";
const job = load(payload);
await job.run();
Chat completion, such as ChatGPT, is the most common AI service. It generates responses in a conversational format based on given inputs, also knows as prompts.
import { openai, system, user, text } from "fluent-ai";
const job = openai()
.chat("gpt-4o-mini")
.messages([system("You are a helpful assistant"), user("Hi")]);
const result = await job.run();
console.log(text(result));
Function calling (or tool calling) is an advanced functionality in chat completions that enhances their ability to interact with external systems and perform specific tasks.
Here's how to create a tool:
import { z } from "zod";
import { tool } from "fluent-ai";
const weatherTool = tool("get_current_weather")
.description("Get the current weather in a given location")
.parameters(
z.object({
location: z.string(),
unit: z.enum(["celsius", "fahrenheit"]).optional(),
})
);
To use the tool, add it to a chat job with a function-calling-enabled model, such as gpt-4o-mini
from openai.
const job = openai().chat("gpt-4o-mini").tool(weatherTool);
await job.messages([user("What is the weather in San Francisco?")]).run();
Rather than waiting for the complete response, streaming enables the model to return portions of the response as they're generated. fluent-ai provides built-in streaming support for text, objects, and tools in chat models.
const job = openai()
.chat("gpt-4o-mini")
.messages([system("You are a helpful assistant"), user("Hi")])
.stream();
for await (const event of await job.run()) {
console.log(text(event));
}
fluent-ai supports streaming text, object and tool calls on demand. For more details, see the streaming docs.
import { openai } from "fluent-ai";
const job = openai().embedding("text-embedding-3-small").value("hello");
const result = await job.run();
import { openai } from "fluent-ai";
const job = openai().image("dalle-2").prompt("a cat").n(1).size("512x512");
const result = await job.run();
fluent-ai provides an easy way to retrieve all available models from supported providers (openai, anthropic, ollama).
import { openai } from "fluent-ai";
const models = await openai().models().run();
Feel free to open an issue or start a discussion if you have any questions. If you would like to request support for a new AI provider, please create an issue with details about the provider's API. Join our Discord community for help and updates.
fluent-ai is licensed under Apache 2.0 as found in the LICENSE file.