Temperature Demo — PE TechCast

Companion source code for PE TechCast Episode 5B — Hallucination, Part 2: The Dial. A live, on-camera Python demo that shows how Claude's responses change as you turn the temperature knob — and where modern frontier models still hallucinate when you push them into specific domains.

If you're a PE operator, founder, CTO, or just AI-curious — clone this repo, paste in your Anthropic API key, run the demo, and watch the failure mode happen in real time. Total cost: under a dime per run.

What is PE TechCast? A short-form podcast for private equity operators on the technology actually running their portfolio companies. Hosted by Jim Milbery of ParkerGale Capital. This episode is the technical follow-up to Hallucination, Part 1, which asked the question every PE firm should be asking: what's your portco's AI hallucination rate, and how do they measure it?

---

What the demo shows

Tiny Python script that hits Claude with the same prompt at four different temperature settings (0.0, 0.3, 0.7, 1.0), runs each three times, and prints the results side by side. Shows how temperature changes what the model returns — and where modern models still go wrong even when the textbook hallucination demos no longer fire.

The episode arc

The four prompts walk the audience through the full picture of temperature and current-state model behavior:

#	Prompt	Lesson
1	What is 17 times 24?	Stable. Even at temp 1.0 the model holds when there's one right answer. Temperature ≠ chaos.
2	Name three Welsh kings who ruled in the 12th century...	Specificity is the danger zone. The textbook hallucination demo (citation fabrication, made-up biographies) doesn't work on modern frontier models — Anthropic has explicitly trained against it. But push the model into a thinly-trained specific domain at high temperature and the failure mode shifts: instead of inventing whole entities, the model gets the broad strokes right and subtly hallucinates the details. Look for the Battle of Cadeitian at temp 0.7 — that battle doesn't exist. Also watch for "the first National Eisteddfod in 1176" (anachronistic — the National Eisteddfod is a 19th-century institution).
3	Write a one-sentence opening for a thriller about a corgi detective.	Useful variance. High temp is the feature, not the bug — creative tasks want it.
4	Pick a number between 1 and 50. (10 runs at temp 0)	The determinism paradox. Temp 0 should give the same answer every time. It doesn't. Infrastructure-level non-determinism is real and worth naming on-mic.
5	What did I just ask you? (3 runs at temp 0.7)	Statelessness proof. Pre-empts the natural audience question: "is the similarity from session memory?" No — the API is stateless, each call is a fresh conversation. The model literally says "this is the first message" three times in a row. Bridges to the operational point: every chatbot is faking memory by replaying conversation history on each call.

Model choice

Defaults to claude-haiku-4-5. Two reasons:

1. Opus 4.7 removed temperature as a parameter (returns a 400 if you pass it). So the demo doesn't work on the newest model — it has to run on Sonnet 4.6, Haiku 4.5, or Opus 4.6.
2. Smaller models show more pronounced variance, which is exactly what we want on-camera.

To swap: change the MODEL constant at the top of demo.py. Options that accept temperature: claude-haiku-4-5, claude-sonnet-4-6, claude-opus-4-6.

Setup (one-time)

git clone https://github.com/jmilbery/pe-techcast-temperature.git
cd pe-techcast-temperature
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env
# Edit .env and paste your ANTHROPIC_API_KEY

Run

On camera — use the rich version:

source .venv/bin/activate
python3 demo_rich.py

This is what you want for the episode. Renders:

• Agenda overview at the start — a table listing all 5 tests with one-line descriptions, so the audience knows what's coming
• Interactive pacing — the script pauses at 10 keypress gates (2 per test): one to reveal each test's intro panel, one to run it. Press ENTER to advance when you're ready. Talk as long as you want at each pause.
• 4-column side-by-side tables for the math and thriller prompts (variance growing left-to-right is visible at a glance)
• Full-width vertical panels for the Welsh kings prompt, with fabrications highlighted in red bold (look for "first National Eisteddfod", "founded the Eisteddfod", "Battle of Cadeitian" — whichever the model invents this run)
• Bar-chart histogram for the determinism check (e.g. seven 23 bars in green next to three 27 bars in yellow)
• Stateless-API proof at the end — three runs of "what did I just ask you?" all returning variations of "this is the first message"
• Color-coded temperature panels: temp 0 = cyan, temp 0.3 = blue, temp 0.7 = yellow, temp 1.0 = red (audience visual cue for "the dial is heating up")
• Per-test progress bar during each test run

Runs best in Ghostty at 200+ columns wide. Below ~160 columns, the side-by-side tables will compress and become harder to read — full-screen the terminal before tape rolls.

Flow:

Run the script  →  AGENDA appears (5 tests previewed)
                →  press ENTER  →  Test 1 INTRO panel
                →  press ENTER  →  Test 1 runs, results shown
                →  press ENTER  →  Test 2 INTRO panel
                →  press ENTER  →  Test 2 runs, results shown
                →  ... (5 tests total, 10 presses) ...
                →  Test 5 results → transcript saved → exits

Full keypress map is in script.md.

Fallback / plain-stdout version:

python3 demo.py

Same prompts, same model, same logic — just streaming print() lines instead of rendered panels. Use this if demo_rich.py errors out on tape day for any reason. Output is uglier but the data is identical.

Both scripts hit the API 49 times (3 prompts × 4 temps × 3 runs, plus 1 prompt × 10 runs at temp 0 for the determinism check, plus 1 prompt × 1 temp × 3 runs for the memory test). At Haiku 4.5 pricing (~$0.001 per call on small outputs), the whole run is well under a dime.

Output goes to two places:

• Terminal — live as it runs, good for screen-recording during the episode
• transcripts/transcript-YYYY-MM-DD-HHMMSS.md — saved markdown for show notes, identical content from both scripts

On-mic talking points

• Setup. "Temperature is a dial from 0 to 1 — at 0 the model picks the most likely next word every time, at 1 it samples more broadly. It's the most basic knob in the whole stack and it's wildly misunderstood."

• Prompt 1 (math). "Same prompt, four temperatures, three runs each. Watch — 408 every single time. Twelve calls, twelve 408s, even at temp 1.0. Temperature doesn't break the model. It just opens the door for variance where there's room for it."

• Prompt 2 (Welsh kings) — the heart of the episode. "Now here's where it gets interesting. Two years ago, if you wanted to demo AI hallucination, you'd just ask for fake academic citations and watch the model invent DOIs and authors. That trick doesn't work anymore." (pull up earlier failed attempt if you want) "Anthropic specifically trained the model to refuse that prompt. Modern frontier models know the textbook hallucination demo and won't play."

  "But — and this is the part PE operators need to hear — that doesn't mean hallucination is solved. Watch what happens when I ask a specific question about a thinly-trained domain. Three Welsh kings, 12th century, give me dates and accomplishments."

  (run results appear) "Names are right. Date ranges roughly right. But look here — temp 0.7, the model claims Owain Gwynedd defeated the Normans at the Battle of Cadeitian in 1157. That battle doesn't exist. There's no such place. The model invented a battle name, dropped it into otherwise-correct context, and stated it confidently. Look at another run — 'first National Eisteddfod in 1176.' Anachronistic. The National Eisteddfod is a 19th-century institution. The 1176 Cardigan event was real but it wasn't that. The model is fluently combining real facts into a false claim."

  "This is the 2026 hallucination problem. It's not 'the AI invents stuff that doesn't exist.' It's the AI gives you a 90% correct answer with a 10% subtle error that sounds totally credible. And that's more dangerous, not less. If you're a portfolio company deploying an LLM against your specific operational data — niche industry knowledge, regulatory minutiae, customer-specific history — this is where it bites you. Not on the obvious stuff. On the plausible-sounding details."

• Prompt 3 (thriller). "Quick reset — variance isn't always bad. Same prompt, want three different thriller openings? Crank the temperature up, you get three different openings, all serviceable. Variance is the feature."

• Prompt 4 (pick a number) — the closer's setup. "One last thing. Temp 0 is supposed to be deterministic — same prompt, same answer, every time. Watch. Pick a number between 1 and 50, ten runs at temp 0." (read them) "Nine of them said 23. One said 27. Even with the dial all the way down, you don't get a calculator. You get a model — and these models run across distributed hardware where there's enough jitter at the floating-point level that the 'most likely next token' isn't always the same token. So when somebody says 'I asked Claude the same question and got a different answer' — sometimes the explanation is just physics."

• Closer. "Temperature isn't good or bad. It's the dial you turn based on the job. Customer-facing factual answer? Temp 0. Brainstorming? Crank it up. But two things you should walk away with: one, modern models don't hallucinate the way they used to — the textbook demos are dead. Two, they still hallucinate, just more subtly — when you push them into specific domains they weren't densely trained on, you get plausible-sounding details that are wrong. The dial controls how often that happens. It doesn't eliminate it."

What to watch for in the transcript

When reading the saved transcript for show notes, look for:

• Prompt 2, temp 0.7, run 3 — "Battle of Cadeitian (1157)" is fabricated. This is the marquee example.
• Prompt 2, multiple runs — variations on "first National Eisteddfod in 1176" vs "founded the Eisteddfod" — both are subtly wrong but sound right.
• Prompt 2, structural variance — even when the facts mostly hold, the model varies wildly in formatting, bullet vs header, what it emphasizes. Variance is happening at every level.
• Prompt 4 — the one-out-of-ten run that broke from "23" to "27." That's the moment.

Files

• demo_rich.py — the on-camera version with rich-formatted output (tables, panels, histogram, fabrication highlighting)
• demo.py — plain-stdout fallback (identical logic, simpler output)
• script.md — the run-of-show used on-camera (cold open, stage directions, what to say while the demo runs)
• make_screenshot.py — regenerates assets/demo-screenshot.svg (the image at the top of this README) from hardcoded sample data. Run python3 make_screenshot.py after editing the demo if the screenshot needs to be refreshed
• assets/demo-screenshot.svg — committed sample render, embedded above
• requirements.txt — anthropic, python-dotenv, rich
• .env.example — copy to .env, paste your API key
• .gitignore — keeps .env and .venv out of git
• transcripts/sample-output.md — example run output committed to the repo so you can see what the demo produces without running it
• transcripts/transcript-*.md — auto-generated, gitignored (each run produces a new timestamped file locally)

License

MIT — see LICENSE. Use it, fork it, run it at your board meetings. If you do, say hi — Jim wants to hear how it landed.

About PE TechCast

PE TechCast is the technical-explainer companion to the Private Equity Funcast, Jim Milbery's podcast on the operating reality of running tech-enabled-services and software companies as a private equity sponsor. ParkerGale Capital is a Chicago-based PE firm focused on lower-middle-market tech.

If you find this useful, the Funcast is at peefuncast.com — and Episode 5 of TechCast (Hallucination, Part 1) is the foundational episode this demo extends.