docs(flow): dogfood classify flow docs on 3.25.0

[RapidPoseidon]

What this is

A from-scratch dogfood of the classify flow feature (rapidata==3.25.0, released 2026-09-19 21:28 UTC, docs republished 21:30 UTC) against production, as a brand-new customer would experience it: fresh uv venv, docs.rapidata.ai only (no repo code) for every claim, then the installed SDK against real credentials, then the repo for fixes. Full method: create_classify_flow with defaults and with explicit min/max/time_to_live, both create_new_flow_batch shapes (plain and contexts=), get_status() while running, get_response_count() timed, get_results() blocked to completion, every result field, get_flow_by_id, find_flows, get_flow_items, update_config (ranking and classify), preheat(), and every documented validation/error path (category bounds, duplicate values, TTL bounds, min-above-max, context_assets bounds, batch-level context/context_assets on a classify flow).

Rebased onto #889 (the ranking/classify class split, merged 2026-09-21 10:09 UTC), which moved the guides to docs/flows/classify.md and docs/flows/ranking.md and independently resolved two of the original five findings. Everything below was re-verified against the new classes and pages before re-applying.

Mismatches fixed here

#	Claim (guides, before this PR)	Observed	Fix
1	get_response_count() is documented as a plain getter next to the non-blocking get_status(), with no mention it can block (both guides).	Confirmed live and timed: on a fresh item it blocked 242.5s (until the flow item reached a terminal state) before returning; called again after the item was already terminal, it returned in 0.09s. rapidata_flow_item.py is untouched by #889; its docstring already says "Either way this waits for the flow item to finish."	Both guides now say it "wait[s] for completion the same way get_results() does".
2	data_type, private_metadata and accept_failed_uploads appear in neither guide nor the overview.	Both RapidataClassifyFlow.create_new_flow_batch and RapidataRankingFlow.create_new_flow_batch still take all three with identical semantics. accept_failed_uploads=False (the default) raises FailedUploadException if any datapoint fails to upload instead of continuing.	One sentence in the overview's shared-lifecycle section (docs/flows.md), since the three are common to both flow types.
3	The ranking guide's "Update Flow Configuration" example only shows instruction=.	RapidataRankingFlow.update_config also takes starting_elo, min_responses, max_responses — confirmed in source and ran without error live against a real ranking flow.	Added the other three parameters to the intro sentence and the example.
4	The classify guide's max_responses_per_datapoint/min_responses_per_datapoint bullets state their defaults but not their bounds.	Confirmed live: max < min and min < 1 both raise ValueError client-side, before any network call. Validation unchanged on main.	Added "(must be at least min_responses_per_datapoint)" / "(at least 1)" to the two bullets.

No code or docstring changes — every docstring I checked already matches real behavior; only the guides were out of sync.

Found on 3.25.0, already resolved on main — no action taken here

• Classify results example showed lowercase "yes"/"no" keys against categories=["Yes", "No"]. Real behavior preserves the exact string given (confirmed live: {"Yes": 4, "No": 0}). fix(flow)!: separate ranking and classify flows #889's new docs/flows/classify.md already shows "Yes"/"No".
• context_assets was undocumented. fix(flow)!: separate ranking and classify flows #889 documents it on both guides, including the ranking 1–10 bound and the per-datapoint list[list[str]] shape on classify.
• Flow-level time_to_live and a 2–10 category bound on create_classify_flow. My run (installed 3.25.0) used both successfully against production (flow flw_1VdliMPVSM7Ihk, time_to_live=timedelta(seconds=60) at creation, overridden per-batch to 45s). refactor(flow): trim create_classify_flow parameters #883 (merged 22:01:55 UTC the same evening) deliberately trimmed both to match the deployed backend feature: flow-level time_to_live removed (batch-level untouched), categories back to 2–8. Noted for the record as real production behavior that has since been intentionally superseded.

Confirmed working exactly as documented (on 3.25.0)

• Category-count bound rejected 1/11 categories client-side; duplicate category values rejected for both plain-string and (label, value) forms.
• time_to_live bounds (45s–3600s) rejected client-side at the batch level, both directions.
• context_assets outside 1–10 on a ranking batch raised the documented ValueError before any upload. On 3.25.0, batch-level context/context_assets on a classify flow raised ValueError; after fix(flow)!: separate ranking and classify flows #889 the classify class simply does not take a batch-level context (its context_assets is per-datapoint), so that error path no longer exists.
• get_win_loss_matrix() on a classify item raises ValueError immediately (no wait) — still true on main.
• update_config(): on 3.25.0 it raised ValueError on a classify flow and ran without error on a ranking flow; after fix(flow)!: separate ranking and classify flows #889 it only exists on RapidataRankingFlow, which the new classify guide already states.
• get_status() states and non-blocking behavior; Incomplete-vs-Completed formula confirmed on a real Incomplete item (153/240) and a real Completed item (15/12).
• majority_value is None on an exact tie — observed live (a real 1-1 split).
• distribution includes every category with 0 for ones nobody chose, in the flow's category order.
• get_flow_by_id, find_flows, get_flow_items, preheat() all round-tripped correctly.
• Auth via RAPIDATA_CLIENT_ID/RAPIDATA_CLIENT_SECRET worked immediately as documented.

Known incident — hotfix confirmed live

Per the brief: GET /flow/simple/item/{flowItemId}/results had been returning 400 ('Size' must be less than or equal to '100') for every classify item, hotfix pending. Both my get_results() calls succeeded on the first try (no 400 observed), and zero-vote categories came back correctly (e.g. {"Yes": 4, "No": 0}) — consistent with the parent-reported hotfix (flow spec 2026.09.19.2134-9bb4c01) being live. This is my own independent confirmation, on my own flow items, not a copy of the example the parent verified against.

Not fixed — already flagged by predecessors, still open

• create_classify_flow also accepts validation_set_id and settings — still in the signature on main, still absent from the classify guide. Flagged as a known, deliberately-deferred gap in docs(flow): dogfood classify flow docs on 3.24.1 #881; I didn't exercise either param live, so I'm not adding unverified prose.
• categories also accepts (label, value) tuples — fully documented in the docstring/reference, deliberately excluded from the guide per an explicit reviewer decision in docs(flow): dogfood classify flow docs on 3.24.1 #881. Not re-adding.
• docs/flows/classify.md vs. the separate examples/classify_job.md (older Likert-scale job API) document two different, both-supported APIs with no cross-link — flagged in docs(flow): dogfood classify flow docs on 3.24.1 #881 as a product/IA call, not a docs bug.

Production run: ids and scripts

Account: poseidon@rapidata.ai. Fresh venv: uv venv && uv pip install rapidata==3.25.0 (installed clean on the first try, no PyPI lag). Images: 88 unique https://assets.rapidata.ai/*.webp URLs flattened from asset_a_uri/asset_b_uri in datasets/ai-faces/datapoints.csv of rapidata-rapids-ab-tests.

• Flow (defaults), classify: flw_1Vdkfb2qBLhaCW → item fli_1VdkgusOFWdM3Y (24 images, plain batch, default TTL, ended Incomplete at 153/240 responses).
• Flow (explicit min/max/ttl), classify: flw_1VdliMPVSM7Ihk → item fli_1VdljSXR2SpkCF (6 images, contexts=, batch TTL override 45s, ended Completed at 15/12 responses, one exact tie).
• Temporary flows created only to exercise create_new_flow_batch guard clauses (zero spend — all raise before any upload), soft-deleted after: flw_1VdkfIY9A7XlF8 (classify), flw_1VdkfRfDwn1IS8 (ranking).

The script below targets 3.25.0 and will not run unmodified on main/3.25.2: time_to_live= on create_classify_flow and media_contexts no longer exist after #883/#889, and classify flows no longer expose update_config().

dogfood_classify_flow.py — full script run against production (3.25.0)

"""Phase 1 dogfood script for rapidata 3.25.0 classify flows, run against production.
Exercises every documented path from docs.rapidata.ai/flows/ plus the reference-documented
signatures, and records every mismatch between docs/docstrings and real behaviour.
"""
import json
import time
import traceback
from dataclasses import asdict, is_dataclass
from datetime import timedelta
from pathlib import Path

from rapidata import RapidataClient

RESULTS = {"free_checks": [], "live_run": {}, "errors": []}
IMAGES = Path(__file__).with_name("image_urls.txt").read_text().split()

client = RapidataClient()


def to_jsonable(obj):
    if is_dataclass(obj) and not isinstance(obj, type):
        return {k: to_jsonable(v) for k, v in asdict(obj).items()}
    if isinstance(obj, dict):
        return {k: to_jsonable(v) for k, v in obj.items()}
    if isinstance(obj, list):
        return [to_jsonable(v) for v in obj]
    return obj


def log(step, ok, detail):
    print(f"[{'OK' if ok else 'FAIL'}] {step}: {detail}")
    RESULTS["free_checks"].append({"step": step, "ok": ok, "detail": str(detail)})


def expect_valueerror(step, fn):
    try:
        fn()
        log(step, False, "no exception raised (expected ValueError)")
    except ValueError as e:
        log(step, True, f"ValueError: {e}")
    except Exception as e:
        log(step, False, f"unexpected {type(e).__name__}: {e}")


print("=" * 20, "FREE CHECKS: create_classify_flow validation (no network cost)", "=" * 20)

expect_valueerror("categories too few (1)", lambda: client.flow.create_classify_flow(
    name="poseidon-dogfood-err", instruction="x", categories=["OnlyOne"]))

expect_valueerror("categories too many (11)", lambda: client.flow.create_classify_flow(
    name="poseidon-dogfood-err", instruction="x", categories=[str(i) for i in range(11)]))

expect_valueerror("duplicate category values (plain str)", lambda: client.flow.create_classify_flow(
    name="poseidon-dogfood-err", instruction="x", categories=["Yes", "Yes"]))

expect_valueerror("duplicate category values (tuple form)", lambda: client.flow.create_classify_flow(
    name="poseidon-dogfood-err", instruction="x", categories=[("Yes", "dup"), ("No", "dup")]))

expect_valueerror("min_responses_per_datapoint < 1", lambda: client.flow.create_classify_flow(
    name="poseidon-dogfood-err", instruction="x", categories=["Yes", "No"], min_responses_per_datapoint=0))

expect_valueerror("max_responses_per_datapoint < min", lambda: client.flow.create_classify_flow(
    name="poseidon-dogfood-err", instruction="x", categories=["Yes", "No"],
    max_responses_per_datapoint=2, min_responses_per_datapoint=5))

expect_valueerror("time_to_live too low (44) at flow create", lambda: client.flow.create_classify_flow(
    name="poseidon-dogfood-err", instruction="x", categories=["Yes", "No"], time_to_live=44))

expect_valueerror("time_to_live too high (3601) at flow create", lambda: client.flow.create_classify_flow(
    name="poseidon-dogfood-err", instruction="x", categories=["Yes", "No"], time_to_live=3601))

print("=" * 20, "FREE CHECKS: create_new_flow_batch guard clauses (no network cost)", "=" * 20)

temp_classify = client.flow.create_classify_flow(
    name="poseidon-dogfood-tmp-classify", instruction="temp validation flow, safe to delete", categories=["Yes", "No"])
temp_ranking = client.flow.create_ranking_flow(
    name="poseidon-dogfood-tmp-ranking", instruction="temp validation flow, safe to delete")
log("temp flows created", True, f"classify={temp_classify.id} ranking={temp_ranking.id}")

expect_valueerror("batch time_to_live too low (44)", lambda: temp_classify.create_new_flow_batch(
    datapoints=[IMAGES[0]], time_to_live=44))
expect_valueerror("batch time_to_live too high (3601)", lambda: temp_classify.create_new_flow_batch(
    datapoints=[IMAGES[0]], time_to_live=3601))
expect_valueerror("batch-level context= on classify flow", lambda: temp_classify.create_new_flow_batch(
    datapoints=[IMAGES[0]], context="not allowed on classify"))
expect_valueerror("batch-level context_assets= on classify flow", lambda: temp_classify.create_new_flow_batch(
    datapoints=[IMAGES[0]], context_assets=[IMAGES[1]]))
expect_valueerror("context_assets too many (11) on ranking flow", lambda: temp_ranking.create_new_flow_batch(
    datapoints=[IMAGES[0]], context_assets=["placeholder"] * 11))
expect_valueerror("context_assets empty (0) on ranking flow", lambda: temp_ranking.create_new_flow_batch(
    datapoints=[IMAGES[0]], context_assets=[]))
expect_valueerror("update_config on classify flow", lambda: temp_classify.update_config(instruction="nope"))

try:
    temp_ranking.update_config(instruction="Poseidon dogfood: updated instruction")
    log("update_config on ranking flow", True, "ran without error")
except Exception as e:
    log("update_config on ranking flow", False, f"unexpected {type(e).__name__}: {e}")

temp_classify.delete()
temp_ranking.delete()
log("temp flows deleted", True, "cleanup done")

print("=" * 20, "LIVE RUN 1: classify flow with ALL documented defaults", "=" * 20)

flow_a = client.flow.create_classify_flow(
    name="poseidon-dogfood-classify-defaults",
    instruction="Does this image show a human face?",
    categories=["Yes", "No"],
)
RESULTS["live_run"]["flow_a_id"] = flow_a.id
RESULTS["live_run"]["flow_a_type"] = flow_a._flow_type
log("flow_a created (defaults)", True, flow_a.id)

batch_a_images = IMAGES[:24]
item_a = flow_a.create_new_flow_batch(datapoints=batch_a_images)
RESULTS["live_run"]["item_a_id"] = item_a.id
RESULTS["live_run"]["item_a_image_count"] = len(batch_a_images)
log("item_a created (plain batch, 24 images, default TTL)", True, item_a.id)

status_a_early = item_a.get_status()
log("item_a get_status() shortly after creation", True, str(status_a_early))
RESULTS["live_run"]["item_a_status_early"] = str(status_a_early)

t0 = time.time()
count_a = item_a.get_response_count()
dt = time.time() - t0
log("item_a get_response_count() timing", True, f"{dt:.1f}s -> {count_a}")
RESULTS["live_run"]["item_a_get_response_count"] = {"value": count_a, "seconds_blocked": dt}

status_a_after_count = item_a.get_status()
log("item_a get_status() after get_response_count()", True, str(status_a_after_count))

expect_valueerror("get_win_loss_matrix() on classify item_a", lambda: item_a.get_win_loss_matrix())

results_a = None
try:
    results_a = item_a.get_results()
    log("item_a get_results()", True, "succeeded")
    RESULTS["live_run"]["item_a_results"] = to_jsonable(results_a)
except Exception as e:
    body = getattr(e, "body", None)
    status = getattr(e, "status", None)
    log("item_a get_results()", False, f"{type(e).__name__} status={status} body={body} str={e}")
    RESULTS["errors"].append({
        "step": "item_a get_results()", "type": type(e).__name__,
        "status": status, "body": str(body), "str": str(e), "traceback": traceback.format_exc(),
    })

print("=" * 20, "LIVE RUN 2: classify flow with EXPLICIT min/max/ttl + contexts=", "=" * 20)

flow_b = client.flow.create_classify_flow(
    name="poseidon-dogfood-classify-explicit",
    instruction="Does this image show a human face?",
    categories=["Yes", "No"],
    max_responses_per_datapoint=5,
    min_responses_per_datapoint=2,
    time_to_live=timedelta(seconds=60),
)
RESULTS["live_run"]["flow_b_id"] = flow_b.id
log("flow_b created (explicit min/max, timedelta ttl)", True, flow_b.id)

batch_b_images = IMAGES[24:30]
contexts_b = [f"Sample image #{i + 1} from a public face dataset" for i in range(len(batch_b_images))]
item_b = flow_b.create_new_flow_batch(
    datapoints=batch_b_images,
    contexts=contexts_b,
    time_to_live=45,
)
RESULTS["live_run"]["item_b_id"] = item_b.id
RESULTS["live_run"]["item_b_image_count"] = len(batch_b_images)
log("item_b created (contexts=, batch TTL override=45)", True, item_b.id)

results_b = None
try:
    results_b = item_b.get_results()
    log("item_b get_results()", True, "succeeded")
    RESULTS["live_run"]["item_b_results"] = to_jsonable(results_b)
except Exception as e:
    body = getattr(e, "body", None)
    status = getattr(e, "status", None)
    log("item_b get_results()", False, f"{type(e).__name__} status={status} body={body} str={e}")
    RESULTS["errors"].append({
        "step": "item_b get_results()", "type": type(e).__name__,
        "status": status, "body": str(body), "str": str(e), "traceback": traceback.format_exc(),
    })

print("=" * 20, "Flow management surface", "=" * 20)

fetched_a = client.flow.get_flow_by_id(flow_a.id)
log("get_flow_by_id(flow_a.id)", True, f"id={fetched_a.id} type={fetched_a._flow_type}")

found = client.flow.find_flows(name="poseidon-dogfood", amount=10)
log("find_flows(name='poseidon-dogfood')", True, f"{len(found)} flows: {[f.id for f in found]}")

items_a = flow_a.get_flow_items()
log("flow_a.get_flow_items()", True, f"{len(items_a)} items: {[i.id for i in items_a]}")

try:
    client.flow.preheat()
    log("client.flow.preheat()", True, "ran without error")
except Exception as e:
    log("client.flow.preheat()", False, f"{type(e).__name__}: {e}")

out = Path(__file__).with_name("results.json")
out.write_text(json.dumps(RESULTS, indent=2, default=str))
print("wrote", out)

Local checks (after the rebase onto #889)

• uv run --frozen --group docs mkdocs build — succeeds; only the pre-existing mri.md/box.py warnings, none from the flow pages.
• uv run --frozen pyright src/rapidata/rapidata_client — 0 errors, 0 warnings (no code changed by this PR).
• uv run --frozen pytest tests/rapidata_client/flow/ — 50 passed.
• tests/test_docs_site.py fails to collect (ImportError while importing test module '/data/workspace/rapidata-python-sdk/tests/test_docs_site.py'.) exactly as on unchanged main — pre-existing, also noted in docs(flow): dogfood classify flow docs on 3.24.1 #881; not touched here.
• uv.lock untouched (--frozen). It still reads 3.25.0 on main after the 3.25.2 bump — pre-existing; deliberately not re-synced here to keep this PR docs-only and conflict-free.

Session: https://poseidon.rapidata.internal/chat/node-28d94ad0

🤖 Generated with Claude Code