Skip to content

Repository files navigation

PTOF — Prompt Token Optimization Framework

Deterministic prompt preprocessing middleware for Large Language Model workflows.

PTOF is a rule-based optimization framework that classifies, structures, validates, and compresses prompts before inference.

Unlike AI-powered prompt rewriting tools, PTOF focuses on:

  • deterministic behavior
  • explainable transformations
  • semantic safety
  • structural optimization
  • benchmark validation
  • reproducible outputs

Why PTOF Exists

Most prompt tooling today behaves like a black box:

input → AI rewrite → output

This creates problems:

  • unpredictable transformations
  • semantic drift
  • inconsistent outputs
  • impossible benchmarking
  • difficult debugging
  • non-deterministic behavior
  • unreliable enterprise workflows

PTOF takes a different approach:

input
→ classify
→ detect complexity
→ detect clarification dependency
→ structurally compress
→ validate safety
→ explain reasoning
→ benchmark deterministically

The framework acts as a preprocessing layer between user input and an LLM.

Instead of replacing prompts with probabilistic rewrites, PTOF focuses on deterministic preprocessing infrastructure.

Short Prompt Handling (v0.9 Evolution)

Original PTOF versions fully bypassed classification and complexity analysis for prompts under 15 tokens.

During real-world testing, this created incorrect behavior for short but semantically obvious prompts such as:

  • "Should we build this?"
  • "Debug this error"
  • "Compare React vs Vue"

The framework now applies:

  • lightweight classification
  • lightweight response shaping

while still bypassing:

  • aggressive stripping
  • structural compression
  • heavy logic-gate processing

This preserves the original low-overhead philosophy while improving practical usability and output accuracy.

Current behavior:

IF token_count(user_prompt) < 15
  → skip structural optimization layers
  → still run lightweight classification
  → apply minimal deterministic formatting
  → flag: short_prompt = TRUE

Core Philosophy

PTOF intentionally avoids:

  • LLM-based optimization
  • embeddings
  • vector similarity
  • probabilistic rewriting
  • black-box transformations
  • semantic hallucination risk

The framework prioritizes:

  • explainability
  • reproducibility
  • auditability
  • semantic preservation
  • deterministic behavior
  • infrastructure-level reliability

This makes PTOF more suitable for:

  • enterprise workflows
  • middleware systems
  • benchmarkable pipelines
  • prompt orchestration systems
  • controlled AI environments

Core Features

Deterministic Classification

PTOF classifies prompts into task families such as:

  • Comparative
  • Technical
  • Analytical
  • Decision
  • Creative
  • Strategic
  • Informational

Example:

Compare GPT and Claude for enterprise architecture reviews.

Detected as:

Comparative

Complexity Detection

PTOF detects:

  • multi-intent prompts
  • trade-off analysis
  • contradictory constraints
  • overloaded workflows
  • conditional reasoning
  • mixed task families

Example:

Compare GPT and Claude, recommend one, explain trade-offs, and create a migration plan.

Detected as:

Comparative complex

Complexity Detection Notes

PTOF intentionally uses conservative complexity detection.

A prompt may be marked as complex when:

  • multiple strong intent families exist
  • conjunction chains are detected
  • competing directives are present

This behavior evolved from real-world semantic preservation testing, especially for multi-intent prompts where aggressive simplification risked intent loss.

The framework currently prioritizes:

semantic preservation > aggressive simplification

Clarification Validation

PTOF identifies when prompts genuinely require additional information before inference.

Examples:

  • uploaded datasets
  • missing code artifacts
  • referenced files
  • unavailable screenshots
  • undefined inputs

Example:

Analyze this uploaded CSV and explain the drop in retention.

Detected as:

Clarification required

Structural Compression

PTOF performs deterministic structural cleanup such as:

  • duplicate removal
  • repeated intent cleanup
  • redundancy detection
  • filler reduction
  • repeated phrase normalization

WITHOUT:

  • semantic rewriting
  • synonym substitution
  • AI-generated paraphrasing

This preserves semantic intent while reducing prompt noise.


Explainability Layer

Every optimization decision can be explained deterministically.

Example:

{
  "classification": [
    "Detected Comparative signal: compare.",
    "Detected Decision signal: recommend."
  ],
  "complexity": [
    "Multiple intent families detected.",
    "Detected comparison or trade-off signal."
  ]
}

This allows PTOF to function as:

  • an optimization engine
  • a debugging tool
  • a prompt analysis system
  • a benchmarkable preprocessing layer

Benchmark Infrastructure

PTOF includes a deterministic benchmark system designed to evaluate:

  • type accuracy
  • semantic safety
  • instruction retention
  • clarification accuracy
  • complexity detection
  • token efficiency

Benchmark suite includes:

  • short prompts
  • noisy prompts
  • emotional prompts
  • contradictory prompts
  • overloaded enterprise prompts
  • multi-intent prompts
  • real-world workflow prompts

Current benchmark status:

PASS: 16
WARN: 0
FAIL: 0

Example Pipeline

Raw Prompt
↓
Classification
↓
Complexity Detection
↓
Clarification Validation
↓
Structural Compression
↓
Semantic Validation
↓
Explainability Output
↓
Optimized Prompt

Example Use Cases

PTOF can function as:

  • prompt preprocessing middleware
  • enterprise AI infrastructure
  • prompt normalization layer
  • benchmarkable optimization pipeline
  • AI orchestration utility
  • structured AI workflow engine
  • safe prompt preprocessing system

Potential integrations:

  • enterprise copilots
  • LLM routing systems
  • prompt gateways
  • benchmark tooling
  • provider-aware optimization
  • AI workflow orchestration systems

CLI Usage

Optimize Prompt

npm run ptof -- optimize "Compare GPT and Claude for code review and recommend one."

Example output:

COMPRESSED PROMPT:
Compare GPT and Claude for code review and recommend one.

TYPE:
Comparative complex

FORMAT RULE:
Use 2-column comparison, max 5 rows.

Explain Optimization

npm run ptof -- explain "Debug this React layout issue on mobile."

Outputs deterministic reasoning layers.


Run Benchmarks

npm run benchmark

or:

node cli/ptof-cli.js benchmark

Demo UI

PTOF includes a React + Vite demo UI for:

  • prompt testing
  • optimization visualization
  • classification inspection
  • token comparison
  • explainability inspection

Run locally:

cd demo-ui
npm install
npm run dev

The UI currently demonstrates:

  • deterministic optimization
  • classification logic
  • complexity detection
  • prompt comparison
  • benchmark-ready outputs

Current Scope

PTOF currently focuses on:

  • deterministic preprocessing
  • structural cleanup
  • classification
  • semantic-safe optimization
  • explainable transformations

Current optimization intentionally avoids:

  • aggressive semantic rewriting
  • probabilistic restructuring
  • AI-generated paraphrasing

This is an intentional architectural decision.


Planned v0.8 Features

Upcoming structural optimization layer:

  • long prompt structuring
  • reference extraction
  • output intent restructuring
  • constraint extraction
  • deterministic prompt organization

Example future transformation:

Before:

messy prompt + links + scattered instructions

After:

TASK:
...

REFERENCE MATERIAL:
- link
- link

CONSTRAINTS:
...

WITHOUT changing semantic meaning.


Roadmap

v0.8

  • long prompt structuring
  • reference extraction
  • output intent restructuring
  • optimization visualization improvements

v0.9

  • integration benchmark layer
  • provider-aware optimization
  • expanded benchmark datasets
  • prompt pipeline visualization

v1.0

  • SaaS architecture
  • provider abstraction layer
  • API middleware support
  • benchmark dashboards
  • orchestration APIs
  • public release

Project Structure

core/
 ├── classification/
 ├── clarification/
 ├── compression/
 ├── complexity/
 ├── explainability/
 ├── formatting/
 ├── structure/
 └── optimizer.js

cli/
 ├── ptof-cli.js
 └── commands/

demo-ui/

test/
 ├── benchmarkCases/
 ├── evaluators/
 └── results/

docs/

Technical Direction

PTOF is evolving toward:

Deterministic preprocessing middleware for LLM systems.

Long-term goals include:

  • provider-aware preprocessing
  • prompt routing intelligence
  • middleware orchestration
  • benchmark-driven optimization
  • enterprise-safe preprocessing layers

The project intentionally prioritizes:

  • correctness before intelligence
  • determinism before creativity
  • explainability before automation

License

MIT


Response Behavior Contracts

PTOF now supports deterministic response behavior presets that shape downstream LLM outputs without modifying semantic intent.

Examples:

  • Thesis mode
  • Challenge my views
  • Simplify all
  • Action plan
  • ELI10
  • Risk analysis

Example:

Should we build this startup?

Final LLM-ready prompt:

Should we build this startup?

RESPONSE CONTRACT:
- Challenge assumptions and identify weaknesses.

This creates:

  • controllable downstream behavior
  • inspectable response shaping
  • deterministic output contracts
  • reusable optimization presets

Final Prompt Rendering

PTOF separates:

compressedPrompt

from:

finalPrompt

This architecture improves:

  • explainability
  • benchmarkability
  • semantic inspection
  • middleware compatibility
  • enterprise auditability

Pipeline:

User Prompt
↓
Optimization
↓
Response Contract
↓
Final LLM-ready Prompt

Current Benchmark Status

Current benchmark categories include:

  • type accuracy
  • semantic safety
  • instruction retention
  • clarification validation
  • response mode accuracy
  • token efficiency
  • long prompt structuring

Current benchmark status:

PASS: 20
WARN: 0
FAIL: 0

Updated Product Positioning

PTOF is evolving into:

Deterministic prompt optimization middleware for LLM systems.

NOT:

  • a black-box rewriting engine
  • an AI paraphrasing system
  • a probabilistic optimizer

The project prioritizes:

  • deterministic behavior
  • semantic preservation
  • explainable transformations
  • benchmark-driven reliability
  • enterprise-safe preprocessing

Releases

Packages

Contributors

Languages