Skip to content

FleetFluid is a Python library that simplifies data transformation by letting you use AI-powered functions without writing (and hosting) them from scratch.

License

Notifications You must be signed in to change notification settings

vossmoos/fleetfluid

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

7 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

FleetFluid

AI Agent Functions for Data Processing with Dual-Mode Support

FleetFluid is a Python library that simplifies data transformation by letting you use AI-powered functions without writing them from scratch. Instead of building functions, you invoke ready-made, agent-based functions that handle tasks like text cleaning, information extraction, translation, labeling, anonymization, and moreβ€”just by specifying what you need in natural language.

πŸš€ Dual-Mode Architecture

FleetFluid supports two execution modes:

  • πŸ”“ Open Source Mode: Uses PydanticAI agents directly on your machine
  • ☁️ Cloud Mode: Uses cloud computation for enterprise-grade performance and reliability

The same Python interface works in both modes - just change the initialization to switch between them!

Installation

pip install fleetfluid

Quick Start

Open Source Mode

from fleetfluid.core import FleetFluid

# Initialize with PydanticAI (open source mode)
ff = FleetFluid(
    model="openai:gpt-4",
    temperature=0.7,
    max_tokens=1000
)

# AI Transformation
result = ff.ai("Rewrite this in a more technical tone", "The data processing pipeline needs optimization.")
print(result)

# Multi-label Classification
result = ff.label("Database query is slow", ["Performance Issue", "Feature Request", "Bug Report"], multiple=True)
print(f"Labels: {result.labels}")
print(f"Confidence: {result.confidence_scores}")

Cloud Mode

from fleetfluid.core import FleetFluid

# Initialize with API key (cloud mode)
ff = FleetFluid(
    api_key="your_premium_api_key",
    api_endpoint="https://api.fleetfluid.io"
)

# Same interface, different backend!
result = ff.label("Database query is slow", ["Performance Issue", "Feature Request", "Bug Report"])
print(f"Label: {result.label}")

Environment-Based Configuration

# Set environment variables
export FLEETFLUID-API-KEY="your_api_key"
export FLEETFLUID_API_ENDPOINT="https://api.fleetfluid.io"
# Automatically detects cloud mode
ff = FleetFluid()  # No parameters needed!

πŸ”„ Mode Switching

You can switch between modes at runtime:

# Start in open source mode
ff = FleetFluid(model="openai:gpt-4")

# Switch to cloud mode
ff.switch_to_cloud_mode("new_api_key", "https://api.fleetfluid.io")

# Switch back to open source mode
ff.switch_to_open_source_mode("anthropic:claude-3-sonnet", temperature=0.3)

πŸ“‹ Complete Function Reference

Initialization

Open Source Mode

# Basic initialization
ff = FleetFluid()

# Custom model and parameters
ff = FleetFluid(
    model="anthropic:claude-3-sonnet",
    temperature=0.7,
    max_tokens=1000,
    top_p=0.9
)

Cloud Mode

# Explicit configuration
ff = FleetFluid(
    api_key="your_api_key",
    api_endpoint="https://api.fleetfluid.io"
)

# Environment-based
ff = FleetFluid()  # Uses FLEETFLUID-API-KEY env var

Core Methods

All methods work identically in both modes:

ff.label(text, labels, multiple=False)

Label text using AI with structured output.

# Single label
result = ff.label("Hello world", ["greeting", "statement", "question"])
print(f"Label: {result.label}")
print(f"Confidence: {result.confidence}")

# Multiple labels
result = ff.label("Hello world", ["greeting", "statement", "question"], multiple=True)
print(f"Labels: {result.labels}")
print(f"Confidence Scores: {result.confidence_scores}")

ff.ai(prompt, data)

Apply AI transformation to data.

result = ff.ai("Make this more formal", "hey there, what's up?")
print(result)  # "Hello, how are you doing?"

ff.extract(extraction_type, text)

Extract specific information from text.

skills = ff.extract("skills", "Python developer with ML experience")
print(skills)  # ["Python", "machine learning"]

ff.anonymize(text)

Anonymize personal information.

anonymized = ff.anonymize("My name is John, email: john@example.com")
print(anonymized)  # "My name is [NAME], email: [EMAIL]"

ff.describe(features, style="natural")

Generate descriptions from features.

description = ff.describe(
    {"color": "blue", "size": "large"}, 
    style="marketing"
)
print(description)  # "A stunning large blue item..."

Async Versions

All methods have async counterparts:

# Async versions for use in async contexts
result = await ff.label_async("Hello world", ["greeting", "statement"])
result = await ff.ai_async("Make formal", "hey there")
result = await ff.extract_async("skills", "Python developer")

πŸ—οΈ Architecture Details

Strategy Pattern Implementation

FleetFluid uses the Strategy pattern to seamlessly switch between execution backends:

  • Abstract Base Classes: Define interfaces for all operations
  • Open Source Implementations: Use PydanticAI agents directly
  • Cloud Implementations: Wrap REST API calls
  • Dynamic Resolution: Methods delegate to appropriate implementation

Configuration Priority

  1. Constructor Parameters (highest priority)
  2. Environment Variables
  3. Default Values (lowest priority)
# Constructor overrides environment
ff = FleetFluid(model="openai:gpt-4")  # Always open source mode

# Environment used when no constructor params
ff = FleetFluid()  # Uses FLEETFLUID-API-KEY if set

πŸ”§ Configuration

Environment Variables

Variable Description Default
FLEETFLUID-API-KEY API key for cloud mode None
FLEETFLUID_API_ENDPOINT API endpoint for cloud mode https://api.fleetfluid.io

Open Source Mode Parameters

All PydanticAI parameters are supported:

  • model: Model identifier
  • temperature: Creativity level (0.0-1.0)
  • max_tokens: Maximum response length
  • top_p: Nucleus sampling parameter
  • frequency_penalty: Frequency penalty
  • presence_penalty: Presence penalty

Cloud Mode Parameters

  • api_key: Your premium API key
  • api_endpoint: Custom API endpoint (optional)

🚨 Error Handling

try:
    result = ff.label("Hello world", ["greeting", "statement"])
except RuntimeError as e:
    if "API" in str(e):
        print("Cloud mode error - check your key and endpoint")
    elif "Open Source" in str(e):
        print("Open source mode error - check your model configuration")
    else:
        print(f"General error: {e}")

πŸ”’ Security

  • Open Source Mode: API keys stay on your machine
  • Cloud Mode: Uses Bearer token authentication
  • HTTPS: All API communications are encrypted
  • No Data Logging: Your data is not stored or logged

πŸ“¦ Requirements

  • Python 3.8+
  • PydanticAI (for open source mode)
  • httpx (for cloud mode)
  • API key for your chosen model provider (open source mode) or FleetFluid (cloud mode)

See the test.py file for comprehensive testing and examples of the dual-mode functionality.

πŸš€ Quick Reference

Initialization Cheat Sheet

Mode Code Use Case
Open Source FleetFluid(model="gpt-4") Development, open source
Open Source FleetFluid(model="claude-3", temperature=0.7) Custom AI parameters
Cloud FleetFluid(api_key="key") Production, enterprise
Cloud FleetFluid(api_key="key", api_endpoint="https://api.fleetfluid.io") Custom API endpoint
Auto FleetFluid() Uses environment variables

Function Quick Reference

Function Description Example
ff.label(text, labels) Single label classification ff.label("Hello", ["greeting", "statement"])
ff.label(text, labels, multiple=True) Multiple label classification ff.label("Hello", ["greeting", "statement"], multiple=True)
ff.ai(prompt, data) AI transformation ff.ai("Make formal", "hey there")
ff.extract(type, text) Information extraction ff.extract("skills", "Python developer")
ff.anonymize(text) Text anonymization ff.anonymize("My name is John")
ff.describe(features, style) Feature description ff.describe({"color": "blue"}, "marketing")

Environment Variables

# For cloud mode
export FLEETFLUID-API-KEY="your_key"
export FLEETFLUID_API_ENDPOINT="https://api.fleetfluid.io"

# For open source mode (standard AI provider keys)
export OPENAI_API_KEY="your_openai_key"
export ANTHROPIC_API_KEY="your_anthropic_key"

Mode Detection Priority

  1. Constructor Parameters (highest)
  2. Environment Variables
  3. Default Values (lowest)

Pro Tip: api_key parameter always wins over model parameter!

πŸ“„ License

MIT License. See LICENSE file for details.

πŸ†˜ Support

About

FleetFluid is a Python library that simplifies data transformation by letting you use AI-powered functions without writing (and hosting) them from scratch.

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages