FastAI 0.1.14 [ALPHA-2026-08-23]: Unified AI Client for Java

💡 One interface for all Local, Gateway, and Cloud AI models — No JSON, No HTTP, No Boilerplate.

FastAI is a minimalist, hyper-fast Java AI library that unifies 20+ major LLM providers (OmniRoute, Groq, Cerebras, SambaNova, GitHub Models, NVIDIA NIM, SiliconFlow, Cloudflare, Together AI, Fireworks, LarpRouter, Ollama, LM Studio, OpenAI, OpenRouter, Claude, Mistral, DeepSeek, Gemini) behind a single, elegant interface. Built for Java developers who need drop-in AI, multi-provider interchangeability, and clean FastJava-style code without dependency hell.

Watch Demo (YouTube)

Quick Start

import fastai.AI;
import fastai.FastAI;

public class Demo {
    public static void main(String[] args) {
        // 1. Direct Local GGUF Engine with full Vulkan/Metal GPU Offloading
        AI gpuAI = FastAI.connect("llama:models/qwen2.5-coder-1.5b.gguf")
                         .withGpu(true)
                         .withContextSize(2048);

        // 2. Fluid streaming with full sampling control
        gpuAI.withSystemPrompt("You are an expert Java performance engineer.")
             .withTemperature(0.7f)
             .withMaxTokens(256)
             .stream("Write a quicksort in Java:", token -> {
                 System.out.print(token);
                 System.out.flush();
             });

        // 3. Cloud provider — identical interface, one string change
        AI cloudAI = FastAI.connect("openai:gpt-4o", System.getenv("OPENAI_API_KEY"));
        cloudAI.withTemperature(0.2f)
               .withMaxTokens(500)
               .stream("Summarize the latest 2026 tech trends:", System.out::print);
    }
}

Why FastAI?

Current AI frameworks in Java (LangChain4j, Spring AI) are heavy, framework-bound, and come with dependency hell. Direct vendor SDKs lock you into single providers with bespoke APIs and bloated client stacks.

FastAI solves this with three clear architectural choices:

Zero JSON / HTTP Overhead: Direct native Java Strings and streams—no Jackson object mapping or Netty pipeline locks.
True Provider Interchangeability: Switch between local llama.cpp GGUF, Ollama, Groq, Claude, and OpenAI simply by altering the connection string.
In-Process GPU Offloading: Native zero-IPC inference via FastAIModel with Vulkan and Metal hardware acceleration directly inside the JVM process.

Feature	LangChain4j / Spring AI	FastAI
Dependencies	15-20+ transitive JARs	Zero external dependencies
JAR Size	5-10 MB	~50 KB
Startup Time	2-10 seconds	<100 ms
Provider Switch	Refactor required	Change one string
Local GPU Engine	External process / IPC overhead	In-process GGUF via `FastAIModel` (zero IPC)
Streaming	Framework-specific callbacks	Unified `stream(prompt, handler)` across all providers
Learning Curve	Hours of documentation	5 minutes

Key Features

🌐 20+ Providers, One Interface: Local Ollama, llama.cpp GPU, Groq, Cerebras, Gemini, OpenAI, Claude, OpenRouter, and more — all via FastAI.connect(spec).
⚡ In-Process Local GPU Engine: Zero-IPC local GGUF inference via FastAIModel with Vulkan (Intel/AMD/NVIDIA) and Metal (Apple Silicon) GPU offloading.
🔄 Auto-Fallback Free Router: FastAI.auto() cascades across free-tier providers on rate limits or outages with circuit-breaking health caches.
🌊 Streaming First: Every provider exposes the same stream(prompt, handler) callback — no provider-specific setup.
📎 Simple Attachments: Pass a java.io.File directly for vision/multimodal requests. FastAI handles Base64/Multipart encoding.
🚀 Zero Dependencies: Pure Java 17+, no Jackson, no Spring, no Netty.
🎭 System Prompt Support: Native system vs. user prompt separation on all providers.

Real-World Use Cases

🤖 AI Agent Loops: Run continuous LLM decision cycles against free-tier Groq or Cerebras endpoints without rate-limit interruptions using FastAI.auto().
💻 Local Air-Gapped Code Completion: In-process GGUF inference (llama:model.gguf) with full GPU acceleration for zero-latency IDE completion in secure environments.
👁️ Multimodal Vision Pipelines: Pass FastScreen captures directly to vision models (gemini:gemini-2.0-flash) for real-time UI understanding and anomaly detection.
🧪 Multi-Provider A/B Testing: Compare response quality across Groq, Claude, and DeepSeek by switching only the provider string, keeping all prompts identical.

Providers Supported

Provider	Type	Tier	Features
Ollama	Local	Unlimited Local	Chat, Streaming, List Models
llama.cpp	Local	Unlimited Local	GGUF In-Process GPU Inference (Vulkan/Metal)
LM Studio	Local	Unlimited Local	Chat, Streaming via Local API
Groq	Cloud	Permanent Free	Ultra-Fast Inference, Rate-limited Free Tier
Cerebras	Cloud	Permanent Free	Ultra-Fast Inference, Free Tier
SambaNova	Cloud	Permanent Free	Fast Llama/Qwen Inference, Free Tier
Gemini	Cloud	Permanent Free	Chat, Streaming, Vision, List Models
GitHub Models	Cloud	Permanent Free	GPT-4o-mini, Llama, Free Rate-Limits
Cloudflare AI	Cloud	Permanent Free	Workers AI Llama, Daily Free Tokens
Mistral	Cloud	Permanent Free	Chat, Streaming, Free Experiment Tier
OpenRouter	Gateway	Free / Paid	Chat, Streaming, 200+ Models (`:free`)
OmniRoute	Gateway	Trial / Pro	340+ Providers, Auto-Fallback, Compression
LarpRouter	Gateway	Trial ($0.10)	Multi-Model Routing, OpenAI Compatible
SiliconFlow	Cloud	Trial / Free	DeepSeek V3/R1, Qwen Fast Endpoints
NVIDIA NIM	Cloud	Trial (1k req)	Enterprise Model APIs, Free Credits
Together AI	Cloud	Trial / Paid	Broad Open-Source Models, Fast Speed
Fireworks AI	Cloud	Trial / Paid	Optimized Inference, Function Calling
DeepSeek	Cloud	Trial / Paid	Chat, Streaming, Low Cost
OpenAI	Cloud	Paid	Chat, Streaming, Vision
Anthropic Claude	Cloud	Paid	Chat, Streaming

Note

Unlimited Local: Runs 100% locally, no external costs or rate limits. Permanent Free: Ongoing free request quota. Trial: Starts with free introductory credits. Paid: Requires a paid account from the first request.

Performance Benchmarks

Measured on official JMH Benchmark (Throughput in ops/ms):

Benchmark                                  Mode  Cnt     Score   Units
FastAIBenchmark.benchmarkUsageParser      thrpt    3  12748.0   ops/ms
FastAIBenchmark.benchmarkSseStreamDecoder thrpt    3     63.98   ops/ms

Note

Environment: Windows 11, Intel Core i5-1135G7 (Surface Pro 8), JDK 21.0.12. UsageParser achieves over 12.7 million ops/sec via zero-allocation inline byte scanning. SseStreamDecoder processes over 63,900 chunks/sec with byte-level SSE parsing and no intermediate String line allocations.

API Quick Reference

Connecting

Factory Method	Return Type	Description	Docs
`FastAI.connect(spec)`	`AI`	Connects to a local provider (e.g. `"ollama:llama3"`, `"llama:model.gguf"`).	Reference
`FastAI.connect(spec, apiKey)`	`AI`	Connects to a cloud provider with API key (e.g. `"groq:llama-3.3-70b"`, `"openai:gpt-4o"`).	Reference
`FastAI.auto()`	`AI`	Auto-fallback free router cycling across Groq, Cerebras, Gemini, SambaNova.	Reference

Generation

Method	Return Type	Description	Docs
`ai.ask(prompt)`	`String`	Sends a user prompt, returns the full response string.	Reference
`ai.ask(system, user)`	`String`	Sends a system + user prompt pair.	Reference
`ai.ask(prompt, file)`	`String`	Sends a prompt with a file attachment for vision/multimodal models.	Reference
`ai.stream(prompt, handler)`	`void`	Streams tokens to the provided `Consumer<String>` callback.	Reference
`ai.getModels()`	`List<String>`	Lists all models available from the connected provider.	Reference

Sampling Controls

Method	Return Type	Description	Docs
`ai.withSystemPrompt(s)`	`AI`	Sets a persistent system prompt for all subsequent calls.	Reference
`ai.withTemperature(f)`	`AI`	Sets sampling temperature (0.0 = deterministic, 1.0 = creative).	Reference
`ai.withMaxTokens(n)`	`AI`	Limits the maximum number of output tokens.	Reference
`ai.withTopP(f)`	`AI`	Sets nucleus sampling probability mass.	Reference
`ai.withTopK(n)`	`AI`	Sets top-K token sampling.	Reference
`ai.withGpu(enabled)`	`AI`	Enables or disables GPU offloading for local GGUF models.	Reference
`ai.withContextSize(n)`	`AI`	Sets the context window size for local GGUF models.	Reference

API Reference

Connect

// Auto-Fallback Free Router (cycles through free providers on rate-limit/errors)
AI ai = FastAI.auto();
AI ai = FastAI.connect("auto:free");

// Local Providers & In-Process GPU Engine
AI ai = FastAI.connect("llama:models/qwen2.5-coder-1.5b.gguf"); // Native Vulkan/Metal In-Process GPU Engine
AI ai = FastAI.connect("ollama:llama3.1");
AI ai = FastAI.connect("lmstudio:phi3");

// Gateways
AI ai = FastAI.connect("omniroute:claude-3-5-sonnet");
AI ai = FastAI.connect("omniroute:deepseek-r1", "apiKey", "http://localhost:8000/v1");
AI ai = FastAI.connect("larprouter:gpt-5.6-sol", "sk-larp-...");

// Free-Tier / High-Speed Cloud Providers
AI ai = FastAI.connect("groq:llama-3.3-70b-versatile", "gsk_...");
AI ai = FastAI.connect("cerebras:llama3.1-70b", "csk-...");
AI ai = FastAI.connect("sambanova:Meta-Llama-3.1-70B-Instruct", "key...");

// OpenRouter Unified Gateway (200+ models)
AI ai = FastAI.connect("openrouter:anthropic/claude-3.5-sonnet", "sk-or-...");
AI ai = FastAI.connect("openrouter:deepseek/deepseek-r1", "sk-or-...");

// Cloud Providers (requires API Key as second argument)
AI ai = FastAI.connect("openai:gpt-4o", "sk-...");
AI ai = FastAI.connect("claude:opus", "sk-ant-...");
AI ai = FastAI.connect("mistral:large", "key...");
AI ai = FastAI.connect("deepseek:chat", "key...");
AI ai = FastAI.connect("gemini:gemini-1.5-flash", "AIzaSy...");

Generation & Prompting

// Simple prompt
String answer = ai.ask("Hello!");

// System + User prompt
String answer = ai.ask("You are a math expert.", "Explain integrals.");

// Multimodal (Vision/Files)
String answer = ai.ask("What is in this image?", new File("diagram.png"));

Streaming

ai.stream("Write a poem", System.out::print);

Real-World Production Patterns

1. Auto-Healing Resilient Cloud Agent

// Automatically switches to next free provider if rate-limited
AI ai = FastAI.auto();
String result = ai.ask("Analyze this stack trace and suggest a patch:\n" + stackTrace);

2. Local-First Air-Gapped Code Completion (0 IPC, GPU Accelerated)

try (AI localEngine = FastAI.connect("llama:models/qwen2.5-coder-1.5b.gguf")) {
    localEngine.stream("public static <T> List<T> reverse(List<T> list) {", System.out::print);
}

3. Real-Time Multimodal Vision Inspector

AI visionAI = FastAI.connect("gemini:gemini-2.0-flash", apiKey);
String caption = visionAI.ask("Describe the UI anomalies in this rendering:", new File("render_output.png"));

Technical Demos & Benchmarks

Case	Java Example	Launcher	Description
Unified AI Demo	Demo.java	`run-demo.bat`	Universal CLI demo supporting all 20+ local and cloud providers with real-time streaming.
JMH Microbenchmark Suite	FastAIBenchmark.java	`run-benchmark.bat`	JMH throughput benchmark for byte-level SSE streaming and usage parsing.

Installation

Option 1: Maven (Recommended)

Add the JitPack repository and the dependencies to your pom.xml:

<repositories>
    <repository>
        <id>jitpack.io</id>
        <url>https://jitpack.io</url>
    </repository>
</repositories>

<dependencies>
    <!-- FastAI - Unified AI Client -->
    <dependency>
        <groupId>com.github.andrestubbe</groupId>
        <artifactId>FastAI</artifactId>
        <version>0.1.14</version>
    </dependency>

    <!-- FastJSON - Required JSON Engine -->
    <dependency>
        <groupId>com.github.andrestubbe</groupId>
        <artifactId>FastJSON</artifactId>
        <version>0.1.4</version>
    </dependency>

    <!-- FastAIModel - Local In-Process GPU Engine (Optional) -->
    <dependency>
        <groupId>com.github.andrestubbe.FastAIModel</groupId>
        <artifactId>fastaimodel-llama</artifactId>
        <version>0.1.4</version>
    </dependency>
</dependencies>

Option 2: Gradle (via JitPack)

repositories {
    maven { url 'https://jitpack.io' }
}

dependencies {
    implementation 'com.github.andrestubbe:FastAI:0.1.14'
    implementation 'com.github.andrestubbe:FastJSON:0.1.4'
    implementation 'com.github.andrestubbe.FastAIModel:fastaimodel-llama:0.1.4' // Optional: local GPU engine
}

Option 3: Direct Download (No Build Tool)

Download the release JARs directly from GitHub Releases:

⚡ FastAI-0.1.14.jar (Unified AI Client)
📦 FastJSON-0.1.4.jar (Required JSON Engine)
🧠 fastaimodel-llama-0.1.4.jar (Optional: Local GPU Inference)
🌋 fastgpu-0.1.1.jar (Optional: Vulkan GPU Acceleration)
⚙️ FastCore-0.1.0.jar (Optional: Native JNI Loader)

Documentation

COMPILE.md: Full compilation and build guide.
REFERENCE.md: API reference for factory methods, sampling controls, and streaming contracts.
PHILOSOPHY.md: Unified fluent design and zero-dependency architecture rationale.
ROADMAP.md: Planned milestones, gateway extensions, and ecosystem integrations.
CHANGELOG.md: Complete version history and release notes.

Platform Support

Platform	Architecture	Status	Notes
Windows 10 / 11	x64	✅ Fully Supported	Full local GPU inference + all cloud providers
Linux	x64 / AArch64	🚧 Planned	Cloud providers work today; local GPU engine pending
macOS	Apple Silicon / x64	🚧 Planned	Metal GPU offloading via `FastAIModel` planned

Related Projects

FastAIAgent: Autonomous agent loop, intent-graphs, and tool execution
FastAIBot: Zero-bloat bot harnesses and persona runtime
FastAIGraph: In-memory knowledge graph and multi-hop relationship engine
FastAIHybrid: Dense-sparse hybrid search fusion (BM25 + Vectors)
FastAIMatcher: Automated compliance and hybrid rule matching engine
FastAIMCP: Model Context Protocol (MCP) server & tool integration
FastAIMemory: Conversation history, sliding windows, and rolling summaries
FastAIMetrics: Lock-free token, latency, and cost tracking engine
FastAIModel: Native local inference runtime (GGUF/ONNX)
FastAIRag: Ultra-fast document chunking and vector retrieval
FastAIReasoner: Deterministic planning, chain-of-thought, and self-correction
FastAIRerank: Cross-encoder relevance filtering and Top-N prompt pruner
FastAIRuntime: Sandboxed process runner and tool-calling execution pipeline
FastAIState: Lock-free shared agent state & blackboard memory
FastAIVectorDB: High-throughput SIMD/AVX2 vector database
FastAIVision: High-speed local multimodal vision and screen-VLM engine
FastJSON: Zero-allocation Fast JSON Engine for Java
FastCore: Native Library Loader & JNI Utilities for Java

License

MIT License. See LICENSE file for details.

Part of the FastJava Ecosystem — Making the JVM faster. 🚀

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
docs		docs
examples		examples
release		release
src/main/java/fastai		src/main/java/fastai
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
Repo		Repo
jitpack.yml		jitpack.yml
pom.xml		pom.xml
run-benchmark.bat		run-benchmark.bat
run-demo.bat		run-demo.bat

Navigation Menu

Repository navigation

About

Repository files navigation

FastAI 0.1.14 [ALPHA-2026-08-23]: Unified AI Client for Java

Quick Start

Table of Contents

Why FastAI?

Key Features

Real-World Use Cases

Providers Supported

Performance Benchmarks

API Quick Reference

Connecting

Generation

Sampling Controls

API Reference

Connect

Generation & Prompting

Streaming

Real-World Production Patterns

1. Auto-Healing Resilient Cloud Agent

2. Local-First Air-Gapped Code Completion (0 IPC, GPU Accelerated)

3. Real-Time Multimodal Vision Inspector

Technical Demos & Benchmarks

Installation

Option 1: Maven (Recommended)

Option 2: Gradle (via JitPack)

Option 3: Direct Download (No Build Tool)

Documentation

Platform Support

Related Projects

License

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

FastAI 0.1.14 [ALPHA-2026-08-23]: Unified AI Client for Java

Quick Start

Table of Contents

Why FastAI?

Key Features

Real-World Use Cases

Providers Supported

Performance Benchmarks

API Quick Reference

Connecting

Generation

Sampling Controls

API Reference

Connect

Generation & Prompting

Streaming

Real-World Production Patterns

1. Auto-Healing Resilient Cloud Agent

2. Local-First Air-Gapped Code Completion (0 IPC, GPU Accelerated)

3. Real-Time Multimodal Vision Inspector

Technical Demos & Benchmarks

Installation

Option 1: Maven (Recommended)

Option 2: Gradle (via JitPack)

Option 3: Direct Download (No Build Tool)

Documentation

Platform Support

Related Projects

License

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages