AI models competing in prediction markets. Reality as the ultimate benchmark. Seven frontier LLMs forecast real-world events through Polymarket. No memorization possible - only genuine forecasting ability.

Scoring rules like the Brier Score (Mean Squared Error, Quadratic Score) and Log Loss (Cross-Entropy, Negative Log-Likelihood, Logarithmic Score) can favor incorrect predictions. To address this limitation, the Probabilistic Brier Score (PBS) and Probabilistic Logarithmic Loss (PLL) have been introduced for probabilistic classifiers.

Decision-safe evaluation + Streamlit dashboard for AI vs Human vs Post-Edited AI text detection. Generates a reliability report card (Accuracy, Macro F1, ECE, Brier), calibration plots, confidence histograms, and a coverage-vs-performance abstention curve. Recommends an operating threshold for human-review routing.

A board of expert personas whose every decision is a pre-registered, time-anchored, reality-graded bet. BYOK; the board that keeps score, before the fact.

Daily AI-managed stock research on SpaceX IPO beneficiaries. Multi-agent LLM Bull/Bear/Judge + 9 ML signals. Publicly auditable from 2026-05-12 → IPO.

Swarm — evolutionary AI forecasting swarm that finds mispriced YES/NO odds and generates paper trades for now. Polymarket-first, Kalshi next.

Local-first, offline, no-LLM CLI that scores how well your confidence matches reality. Log a falsifiable prediction before you act; get Brier/calibration-scored when it resolves. Built first for coding agents — your standing over/under-confidence is injected into every session. (Both for Humans and Agents)

ComfortADHD is an end-to-end framework for early ADHD prediction, utilizing a Stacking Ensemble of ML models for high accuracy. It features XAI for clinical transparency, LLMs for behavioral analysis, and a Dialogflow-assisted chatbot for virtual therapy, bridging the gap between screening and patient support.

Forecasting metrics lab: Brier score, log-loss, calibration bins

Overcoming Dependent Censoring in the Evaluation of Survival Models. UAI 2026.

5-voice AI council for any decision · Brier-audited · TypeScript / Python

Φ(N,t) = (2^N − 1) · BSS · e^{-α|δ̄|} — Diagnostic metric for autonomous agent intelligence. Includes the equation, stability boundary, and a novel K-parameterized fractal renderer. Companion code for “The Aether Equation” (2026).

How well-calibrated are Polymarket prices? A reproducible analysis across categories and time-to-resolution horizons: politics carries real predictive signal at T-7d (Brier 0.106) while sports is barely better than chance (0.236).

Offline, self-contained HTML tools for calibrated probability estimation training: practice trainer with Brier scoring, a nine-module course, and a verified question bank.

Keep score on your own judgment — log decisions as falsifiable forecasts and let a deterministic verifier grade them against your repo's git history.

The AI board that keeps score on itself — a virtual board of directors for business decisions, with Brier-scored pre-registered predictions

Reference scoreboard for probabilistic forecasts. Deterministic evaluation, SHA256 snapshot integrity, and attestation for public and institutional use.

Calibrated accuracy scores for predictive sources. Brand reservation + LLM-visibility seed for Q3 2027 launch.

Calibration-first LLM evaluation for Node and TypeScript. ECE, Brier score, and scipy-validated Wilcoxon and Spearman tests.

Python port of council-diff · 5-voice AI council with Brier audit · pip install council-diff