Highflame: Enterprise-Scale LLM Gateway

This is the Python SDK for Highflame - an enterprise-scale, fast LLM gateway that provides unified access to multiple LLM providers with advanced routing, monitoring, and management capabilities.

Package Name: highflame (v2.0.0)
Previous Package: javelin_sdk

For more information about Highflame, see https://highflame.com

Highflame Documentation: https://docs.highflame.ai/

---

Quick Start

Installation

pip install highflame

Basic Usage

from highflame import Highflame, Config
import os

# Initialize client
config = Config(api_key=os.getenv("HIGHFLAME_API_KEY"))
client = Highflame(config)

# Query a route
response = client.query_route(
    route_name="my_route",
    query_body={"messages": [{"role": "user", "content": "Hello!"}], "model": "gpt-4"}
)

---

What Changed

Package & Import Changes

v1:

from javelin_sdk import JavelinClient, JavelinConfig

v2:

from highflame import Highflame, Config

Class Name Changes

v1	v2
JavelinClient	Highflame
JavelinConfig	Config
JavelinClientError	ClientError

Configuration Changes

Environment Variables: All environment variable names have changed from JAVELIN_* to HIGHFLAME_*:

v1	v2
JAVELIN_API_KEY	HIGHFLAME_API_KEY
JAVELIN_VIRTUALAPIKEY	HIGHFLAME_VIRTUALAPIKEY
JAVELIN_BASE_URL	HIGHFLAME_BASE_URL

Configuration Fields:

• javelin_api_key → api_key
• javelin_virtualapikey → virtual_api_key
• Default base_url: https://api-dev.javelin.live → https://api.highflame.app

Example:

v1:

from javelin_sdk import JavelinConfig

config = JavelinConfig(javelin_api_key="your-key")

v2:

from highflame import Config

config = Config(api_key="your-key")

HTTP Headers (Backward Compatible)

v2 sends both old and new headers for backward compatibility during the transition period:

Header Type	v1	v2 (Primary)	v2 (Backward Compat)
API Key	x-javelin-apikey	x-highflame-apikey	x-javelin-apikey
Virtual API Key	x-javelin-virtualapikey	x-highflame-virtualapikey	x-javelin-virtualapikey
Route	x-javelin-route	x-highflame-route	x-javelin-route
Model	x-javelin-model	x-highflame-model	x-javelin-model
Provider	x-javelin-provider	x-highflame-provider	x-javelin-provider
Account ID	x-javelin-accountid	x-highflame-accountid	x-javelin-accountid
User	x-javelin-user	x-highflame-user	x-javelin-user
User Role	x-javelin-userrole	x-highflame-userrole	x-javelin-userrole

Note: Both header formats are sent simultaneously to ensure compatibility with existing backends.

API Endpoint Changes

v1	v2
https://api-dev.javelin.live	https://api.highflame.app

Exception Handling

v1:

from javelin_sdk.exceptions import (
    JavelinClientError,
    GatewayNotFoundError,
    RouteNotFoundError,
    # ... etc
)

v2:

from highflame.exceptions import (
    ClientError,
    GatewayNotFoundError,
    RouteNotFoundError,
    # ... etc
)

All exception classes now inherit from ClientError instead of JavelinClientError.

OpenTelemetry Tracing

Service & Tracer Names:

• Service name: "javelin-sdk" → "highflame"
• Tracer name: "javelin" → "highflame"

Span Attributes:

• javelin.response.body → highflame.response.body
• javelin.error → highflame.error

CLI Changes

Command Name:

• javelin → highflame

Cache Directory:

• ~/.javelin/ → ~/.highflame/

Example:

v1:

javelin auth
javelin routes list

v2:

highflame-cli auth
highflame-cli route list

Exception Handling Changes

v1:

from javelin_sdk.exceptions import (
    JavelinClientError,
    RouteNotFoundError,
    ProviderNotFoundError,
)

try:
    client.query_route(...)
except JavelinClientError as e:
    print(f"Error: {e}")

v2:

from highflame.exceptions import (
    ClientError,
    RouteNotFoundError,
    ProviderNotFoundError,
)

try:
    client.query_route(...)
except ClientError as e:
    print(f"Error: {e}")

Complete Migration Example

v1 Code:

import os
from javelin_sdk import JavelinClient, JavelinConfig
from javelin_sdk.exceptions import RouteNotFoundError

# Get API key from environment
api_key = os.getenv("JAVELIN_API_KEY")

# Create configuration
config = JavelinConfig(
    javelin_api_key=api_key,
    base_url="https://api-dev.javelin.live"
)

# Create client
client = JavelinClient(config)

# Query a route
try:
    response = client.query_route(
        route_name="my_route",
        query_body={
            "messages": [{"role": "user", "content": "Hello"}],
            "model": "gpt-4"
        }
    )
    print(response)
except RouteNotFoundError as e:
    print(f"Route not found: {e}")
finally:
    client.close()

v2 Code:

import os
from highflame import Highflame, Config
from highflame.exceptions import RouteNotFoundError

# Get API key from environment
api_key = os.getenv("HIGHFLAME_API_KEY")

# Create configuration
config = Config(
    api_key=api_key,
    base_url="https://api.highflame.app"
)

# Create client
client = Highflame(config)

# Query a route
try:
    response = client.query_route(
        route_name="my_route",
        query_body={
            "messages": [{"role": "user", "content": "Hello"}],
            "model": "gpt-4"
        }
    )
    print(response)
except RouteNotFoundError as e:
    print(f"Route not found: {e}")
finally:
    client.close()

Async/Await Support

Async support remains unchanged in v2:

async with Highflame(config) as client:
    response = await client.aquery_route(
        route_name="my_route",
        query_body={...}
    )

Migration Checklist

1. ✅ Update package installation: pip install highflame (instead of javelin_sdk or highflame-sdk)
2. ✅ Update imports: from highflame import Highflame, Config
3. ✅ Update class names: JavelinClient → Highflame, JavelinConfig → Config
4. ✅ Update environment variables: JAVELIN_* → HIGHFLAME_*
5. ✅ Update configuration field names: javelin_api_key → api_key
6. ✅ Update API endpoint if using custom base URL
7. ✅ Update exception imports: JavelinClientError → ClientError
8. ✅ Update CLI commands: javelin → highflame-cli
9. ✅ Update cache directory references if any

Note: HTTP headers are backward compatible - both old and new headers are sent automatically, so no immediate changes needed for header handling. The v2 SDK maintains full API compatibility with v1 in terms of functionality - all methods, parameters, and responses remain the same, only naming conventions have changed.

---

Backend Changes Required

To complete the migration from Javelin to Highflame, the following backend changes are required:

1. Database Schema Changes

File: highflame-admin/internal/admin/module/keyvault/schema.go

• Rename field: JavelinSecretKey → HighflameSecretKey
• Database column: api_key_secret_key_javelin → api_key_secret_key_highflame
• Database column: api_key_secret_key_javelin_start → api_key_secret_key_highflame_start

File: highflame-admin/internal/admin/module/apikey/schema.go

• Table name: javelinapikeys → highflameapikeys (or maintain backward compatibility)
• Struct name: JavelinAPIKey → HighflameAPIKey

2. Go Model Changes

File: highflame-admin/internal/admin/model/keyvault.go

// Change:
JavelinSecretKey string `json:"api_key_secret_key_javelin,omitempty"`

// To:
HighflameSecretKey string `json:"api_key_secret_key_highflame,omitempty"`

File: highflame-admin/internal/admin/model/apikey.go

// Change:
type JavelinAPIKey struct { ... }

// To:
type HighflameAPIKey struct { ... }

File: highflame-admin/internal/admin/model/chronicle.go

// Change:
JavelinResponseHeaders json.RawMessage `json:"javelin_response_headers"`

// To:
HighflameResponseHeaders json.RawMessage `json:"highflame_response_headers"`

3. Service Layer Changes

File: highflame-admin/internal/admin/module/keyvault/service.go

• Update all JavelinSecretKey references to HighflameSecretKey
• Update variable names: javelin_key → highflame_key, etc.

File: highflame-admin/internal/admin/module/audit/repo.go

• Update field deletion: "api_key_secret_key_javelin" → "api_key_secret_key_highflame"

4. Swagger/OpenAPI Specification Updates

Files to update:

• highflame-admin/docs/swagger.yaml
• highflame-admin/docs/swagger.json
• highflame-admin/docs/docs.go

Changes required:

• api_key_secret_key_javelin → api_key_secret_key_highflame
• javelin_response_headers → highflame_response_headers
• Model names: JavelinAPIKey → HighflameAPIKey

After updating, regenerate the Swagger spec and sync Python SDK models using swagger/sync_models.py.

5. Database Migration

Create a migration script to:

1. Add new columns alongside old ones (for zero-downtime migration)
2. Migrate existing data from old columns to new columns
3. Update application code to use new field names
4. Deprecate old columns after transition period
5. Remove old columns in a future release

6. Other Backend Services

File: highflame-core/pkg/persist/admin_client/aws_secrets.go

• Update JavelinSecretKey field to HighflameSecretKey

File: highflame-core/tests/e2e/promptfoo/extensions/shared/utils.js

• Update test references from api_key_secret_key_javelin to api_key_secret_key_highflame

7. API Header Support (Backward Compatibility)

The Python SDK currently sends both x-javelin-* and x-highflame-* headers for backward compatibility. The backend should:

• Continue accepting x-javelin-* headers during the transition period
• Prefer x-highflame-* headers when both are present
• Plan deprecation timeline for old headers (recommend 6-12 month transition period)
• Log usage of old headers to track migration progress

Backend Migration Checklist

• Database schema: Add new api_key_secret_key_highflame column
• Database migration: Script to migrate data from old to new columns
• Go models: Rename JavelinSecretKey → HighflameSecretKey
• Go structs: Rename JavelinAPIKey → HighflameAPIKey
• Service layer: Update all field references
• Swagger spec: Regenerate with new field names
• Tests: Update all test references
• Documentation: Update API documentation
• Header support: Ensure both old and new headers are accepted
• Monitoring: Track usage of old vs new headers/fields
• Deprecation plan: Create timeline for removing old fields/headers

---

Development Setup

Setting up Virtual Environment

Windows

# Create virtual environment
python -m venv venv

# Activate virtual environment
venv\Scripts\activate

# Install dependencies
pip install poetry
poetry install

macOS/Linux

# Create virtual environment
python -m venv venv

# Activate virtual environment
source venv/bin/activate

# Install dependencies
pip install poetry
poetry install

Building and Installing the SDK

# Uninstall any existing version
pip uninstall highflame highflame-sdk javelin_sdk -y

# Build the package
poetry build

# Install the newly built package
pip install dist/highflame-<version>-py3-none-any.whl

Development Notes

For local development, change version = "RELEASE_VERSION" with any semantic version (e.g., version = "2.0.1") in pyproject.toml.

⚠️ Important: Make sure to revert pyproject.toml before committing to main.

---

Universal Endpoints

Highflame provides universal endpoints that allow you to use a consistent interface across different LLM providers. Here are the main patterns:

Azure OpenAI

• Basic Azure OpenAI integration
• Universal endpoint implementation
• OpenAI-compatible interface

Bedrock

• Basic Bedrock integration
• Universal endpoint implementation
• OpenAI-compatible interface

Gemini

• Basic Gemini integration
• Universal endpoint implementation
• OpenAI-compatible interface

Agent Examples

• CrewAI integration
• LangGraph integration

Basic Examples

• Asynchronous example
• Synchronous example
• Drop-in replacement example

Advanced Examples

• Document processing
• RAG implementation

---

Additional Integration Patterns

For more detailed examples and integration patterns, check out:

• Azure OpenAI Integration
• AWS Bedrock Integration
• CLI Reference

---

Type Hints & py.typed Marker

This package includes a py.typed marker file, which indicates to type checkers (like mypy, pyright, pylance) that the package supports type checking. This allows IDEs and static analysis tools to provide better autocomplete, type checking, and refactoring support.

Usage:

# With py.typed, type checkers can validate this:
from highflame import Highflame, Config

config: Config = Config(api_key="your-key")
client: Highflame = Highflame(config)

---

Logging

The SDK includes logging support for debugging and observability. Logging is configured at the module level using Python's standard logging module.

Basic Configuration

import logging

# Enable debug logging for the SDK
logging.basicConfig(level=logging.DEBUG)

# Or set logging for specific modules
logging.getLogger("highflame").setLevel(logging.DEBUG)
logging.getLogger("highflame.services").setLevel(logging.DEBUG)

Logging Levels

• DEBUG - Detailed information for diagnosing problems
  • Client initialization
  • Route queries
  • Service operations
  • Tracing configuration
• INFO - General informational messages
• WARNING - Warning messages for potentially problematic situations
• ERROR - Error messages for failures

Example: Full Debug Logging

import logging
from highflame import Highflame, Config

# Configure detailed logging
logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)

# Initialize client
config = Config(api_key="your-key")
client = Highflame(config)

# Debug logs will show:
# - Client initialization with base URL
# - Route queries with route names
# - Tracing configuration (if enabled)
response = client.query_route(
    route_name="my_route",
    query_body={...}
)

Production Configuration

For production, use structured logging:

import logging
import json
from datetime import datetime

# Use JSON logging for better observability
class JSONFormatter(logging.Formatter):
    def format(self, record):
        log_obj = {
            "timestamp": datetime.utcnow().isoformat(),
            "level": record.levelname,
            "logger": record.name,
            "message": record.getMessage(),
        }
        if record.exc_info:
            log_obj["exception"] = self.formatException(record.exc_info)
        return json.dumps(log_obj)

# Configure handler
handler = logging.StreamHandler()
handler.setFormatter(JSONFormatter())

logger = logging.getLogger("highflame")
logger.addHandler(handler)
logger.setLevel(logging.INFO)

Available Loggers

Logger	Purpose
highflame.client	Main Highflame client operations
highflame.services.route_service	Route querying and management
highflame.services.gateway_service	Gateway operations
highflame.tracing_setup	OpenTelemetry tracing configuration

---

TODO

Code Quality Improvements

• Type Hints Coverage: Add comprehensive type hints to all methods. Many internal methods currently return Any or lack return type annotations. Properties like client, aclient, close(), and helper methods need type hints.

• Error Handling: Replace broad except Exception blocks (27+ locations) with specific exception types. This will improve debugging and error context.

• Backward Compatibility: Phase out dual header support (x-javelin-* and x-highflame-*) after transition period. Create deprecation timeline and migration path.

• Request/Response Validation: Add validation layer for requests and responses to catch errors early.

• HTTP Connection Pooling: Add configuration options for HTTP connection pooling to improve performance.

CLI Improvements

• CLI Separation: Separate CLI into its own highflame-cli package. Create separate repository, package, and PyPI distribution.

• CLI Error Messages: Improve CLI error messages with troubleshooting hints and actionable guidance.

• CLI Testing: Add comprehensive test suite for CLI commands and edge cases.

• CLI Documentation: Update CLI documentation to reflect current bundled state vs. future separated state.

Reliability & Performance

• Automatic Retry Logic: Implement retry logic with exponential backoff for transient failures.

• Rate Limit Detection: Add automatic rate limit detection and backoff handling.

• Performance Metrics: Add performance metrics tracking for requests, latency, and throughput.

• Circuit Breaker Pattern: Implement circuit breaker pattern for resilience against failing services.

• Request Caching: Add optional request caching layer for frequently accessed resources.

Developer Experience

• Deprecation Warnings: Add deprecation warning module for v1 → v2 migration to help users transition.

• Structured Logging: Enhance structured JSON logging implementation for production environments.

• Custom Middleware: Add support for custom middleware to allow users to extend SDK functionality.

• Better Error Messages: Implement error messages with troubleshooting steps and links to documentation.

Testing & Release

• Full Test Suite: Run and expand comprehensive test suite covering all SDK functionality.

• Build & Distribution: Test building distribution packages and verify installation process.

• Installation Testing: Test pip install highflame and verify all imports work correctly.

• CLI Functionality Testing: Test all CLI commands and verify they work as expected.

• Performance Testing: Conduct performance testing and optimization.

• PyPI Publishing: Publish highflame v2.0.0 to PyPI with proper release notes.

• Documentation Updates: Update GitHub release notes and publish migration guide on docs site.

Architecture

• Client Class Refactoring: Refactor large Highflame class (1645 lines) into smaller, focused classes for better maintainability.

• Constants Module: Create centralized constants module for HTTP headers and configuration values.

• Service Layer Improvements: Enhance service layer with better error handling and type safety.