A backstep is the move you make to undo a commitment you have already made. So is a rollback.
An answer to a question that reading about it never settled: what is rollback netcode actually doing, and what does it cost?
Every fighting-game forum will tell you rollback "guesses your opponent's input and rewinds when it is wrong". That sentence is true and explains nothing. How often is the guess right? How deep does the rewind go? What happens when your opponent is on another continent? Those have numbers, and the only way to get them was to build the thing and measure it.
So: a Rust workspace that implements rollback netcode at two levels.
- A deterministic, fully instrumented 2D arena. Every byte of state is auditable, the snapshot is 204 bytes, and the checksum covers all of it.
- The Last Blade 2, a 1998 Neo Geo fighting game, running on the official FBNeo emulator core, driven by exactly the same rollback engine.
The second one is the point. If the engine needed to understand the game, it could not drive an arcade emulator whose state is a 415 KB opaque blob. It does not, so it can.
A person plays P1 through SDL2. A headless EC2 instance plays P2 through a scripted state machine. The peers exchange only inputs, over UDP authenticated with HMAC-SHA256, while Prometheus, Grafana, JSONL logs and an HTML report record every prediction, correction and disagreement.
New to any of this? 00 - Glossary explains every term from scratch, and 17 - How emulation works explains what it means to run a 1998 arcade board in software.
The Last Blade 2 under rollback. The band across the top is the session's own
telemetry, burned in from its log: 142 rollbacks so far, 18.5% of a frame's work
being re-simulated, and ROLLBACK -5 firing on the frame it happened. The game
underneath does not stutter.
Both simulations ran between a desktop in Madrid and EC2 instances in Frankfurt, São Paulo and Tokyo, over the public internet.
| Arena | The Last Blade 2 | |
|---|---|---|
| Snapshot size | 204 bytes | 415 155 bytes |
| CPU, 300 s session | ~2 s | ~116 s |
| Prediction accuracy | 92.7-95.5% | 91.5-92.7% |
| Desyncs | 0 | 0 |
2 997 checksum comparisons agreed, across three continents and zero desyncs. Two different CPUs, two operating systems, two libcs. That is the evidence behind the determinism rules in 05, and two processes of one binary on one machine could never have produced it.
What distance does, at a glance:
| From Madrid to | SRTT | Max depth | Stalls | FPS |
|---|---|---|---|---|
| Frankfurt | 50 ms | 3 | 0 | 60.01 |
| São Paulo | 272 ms | 8 (the limit) | 827 | 57.37 |
| Tokyo | 267 ms | 8 (the limit) | 1 283 | 56.01 |
The default 8-frame prediction window is 133 ms at 60 Hz - exactly one way at 267 ms. Beyond that the session stops speculating and waits, which is the design working correctly and also the point at which it needs retuning. What that retuning costs, and who pays for it, is measured in 08 - Experiments along with all five synthetic profiles.
| Document | Subject |
|---|---|
| 00 - Glossary | Start here. Every technical term from scratch |
| 01 - Theory | What rollback is, why it exists, what it costs |
| 02 - Architecture | The crates, and why the boundary sits where it does |
| 03 - Protocol | Datagram format, authentication, handshake |
| 04 - Running locally | Controls, commands, a session end to end |
| 05 - Determinism | The rules that prevent desync, and how they were checked |
| 06 - AWS | The infrastructure, the threat model, the session key |
| 07 - Dashboard | Prometheus, Grafana, what each panel means |
| 08 - Experiments | Five profiles, the method, the results |
| 09 - The Last Blade 2 | The FBNeo core, the boot script, the pinned commit |
| 10 - Costs | What a session costs, and where money disappears |
| 11 - Cleanup | How to destroy everything and confirm it went |
| 12 - Troubleshooting | Symptoms, causes, what to look at first |
| 13 - Coverage | What was validated, what was not |
| 14 - Video | Recording sessions, and watching rollback happen |
| 15 - Elastic | Per-event analysis: what summary.csv cannot answer |
| 16 - The algorithm | Data structures, invariants, code paths, complexity |
| 17 - How emulation works | What it means to run a 1998 arcade board in software |
| 18 - Dashboards and queries | Kibana panels, ES|QL, and the published dataset |
The dataset is published. Every session log is in
dataset/, 5 MB compressed, so the analysis can be checked
or taken somewhere else without running a session or spending anything on AWS.
Diagrams: system topology and crate graph, a rollback on a timeline, AWS network topology.
What the lab set out to do, and what actually happened. The two deviations are listed with the rest rather than buried.
| Requirement | Status | Where |
|---|---|---|
| Rollback engine: prediction, limit, history, re-simulation | done | 16 |
| Defaults: 1 input delay, 8 prediction, 16 states, 60 Hz | done | SessionConfig::default |
| Stop on window full; end after 3 s of silence | done | 16 |
| Checksums every 60 confirmed frames; desync ends the session | done | 05 |
| UDP/7000, versioned wire, 1 200-byte limit, HMAC-SHA256 | done | 03 |
| Six message types, 8-input redundancy, sequence + ACK | done | 03 |
| Handshake validates version, commit, config, seed, hashes | done | 03 |
| Integer-only 2D arena + FSM bot | done | 02 |
| SDL2 client, overlay, keyboard and gamepad | done, played end to end | 18 |
| FBNeo core in a reproducible container | done, different commit | 09 |
Emulated game via retro_serialize, scripted boot, no ROM offsets |
done | 09 |
| Prometheus, Grafana, JSONL, all listed metrics | done | 07 |
Five profiles, 180 s, summary.csv + self-contained HTML |
done | 08 |
| Terraform VPC, SSM, S3, IMDSv2, 4 h terminate, no SSH | done | 06 |
just test / local-up / aws-up / play / bench / collect / aws-down |
done | 04 |
| Unit, property, 100 k replay, golden protocol, fake core, E2E | done | 13 |
| Gates: fmt, clippy, tests, shellcheck, terraform, docs | done | just test |
| AWS smoke: handshake, session, collect, destroy | done, ×6 across three regions | 08 |
| Didactic documentation of every technical term | done | 00 |
One deviation, forced and documented.
The FBNeo commit differs. The spec pinned
finalburnneo/FBNeo@f1c3545f…, which has no makefile.libretro. The build uses
libretro/FBNeo@0332bb98… and records both hashes in the artefact's
provenance, so the deviation is visible from the binary rather than only from
this table.
Beyond the spec: video recording with burned-in telemetry (14), per-event Elasticsearch analysis (15), an algorithm reference (16), and runs against three regions rather than one.
Every acceptance criterion that was reachable has been met, including the last one to fall: a human on P1, playing The Last Blade 2 against the scripted bot in Frankfurt over the real internet. That session produced the most surprising number in the project, and it contradicted an assumption written into these docs. See 18 - Dashboards.
What still has no measurement is perception. The lab can say a correction was 8 frames deep; it cannot say whether that was noticeable. That is the one question rollback exists to answer and the one this repository does not.
| Tool | For | Check |
|---|---|---|
| Rust ≥ 1.82 | building everything | cargo --version |
| SDL2 ≥ 2.0.20 | the graphical client | pkg-config --modversion sdl2 |
| Docker | FBNeo build, Prometheus, Grafana, Elastic | docker --version |
| ffmpeg | recording sessions (optional) | ffmpeg -version |
just |
the commands below | just --version |
| Terraform ≥ 1.6 | AWS infrastructure | terraform version |
| AWS CLI | aws-up, collect, aws-down |
aws sts get-caller-identity |
| shellcheck | the script lint gate | shellcheck --version |
x86_64. The observability docker-compose uses host networking and is therefore
Linux-only; the reason is written in the file itself.
just test # fmt, clippy, tests (debug and release), shellcheck, terraform, docs
just e2e # two real processes, a real socket, all five profiles
just bench # 180 s per profile, writes summary.csv and report.htmljust bench produces artifacts/report/summary.csv, one row per session across
about 37 columns, and artifacts/report/report.html, which is self-contained:
no CDN, no script, charts as inline SVG.
just local-up
# Grafana http://127.0.0.1:3000
# Prometheus http://127.0.0.1:9090
# Exporter http://127.0.0.1:9898/metricscp terraform/example.tfvars terraform/terraform.tfvars
$EDITOR terraform/terraform.tfvars # allowed_cidr = your address, as a /32
curl -s https://checkip.amazonaws.com # to find it
just aws-up arena
just play arena
just collect # ALWAYS before aws-down
just aws-downYou supply the ROM. It needs neogeo.zip, the Neo Geo BIOS, in
artifacts/system/: a Neo Geo game is only half the code that runs, and the
BIOS is hashed into the handshake alongside the ROM.
just build-core # builds FBNeo in a container
just check-determinism /path/lastbld2.zip # check the core before anything
just e2e 90 lastblade2 /path/lastbld2.zip
just aws-up lastblade2 /path/lastbld2.zip
just play lastblade2 /path/lastbld2.zipjust check-determinism is not fussiness. FBNeo as shipped seeds its RNG and
its emulated calendar clock from the host clock, so two peers that start in
different wall-clock seconds diverge before the first input. just build-core
patches that; the measurement and the fix are in
05 - Determinism.
No ROMs and no BIOS. lastbld2.zip and neogeo.zip are yours, and are never
committed, redistributed, or included in any artefact here.
No savestates or personal logs. All of artifacts/ is gitignored.
No keys. The session key is ephemeral, generated per run, kept in SSM SecureString and in a local file with mode 0600, and never enters Terraform state or a command line.
STUN, relay, matchmaking, spectating, reconnection, state synchronisation, Tekken 3, vision-based AI and memory-reading bots. Fightcade is a reference point, not a dependency.
Multiple regions were originally out of scope and are now covered: Frankfurt,
São Paulo and Tokyo, with ops/scripts/region-run.sh to reproduce any of them.