Skip to content

Add Colab quickstart notebook#12

Open
CodeSarthak wants to merge 1 commit into
simchowitzlabpublic:mainfrom
CodeSarthak:add-colab-quickstart
Open

Add Colab quickstart notebook#12
CodeSarthak wants to merge 1 commit into
simchowitzlabpublic:mainfrom
CodeSarthak:add-colab-quickstart

Conversation

@CodeSarthak

Copy link
Copy Markdown
Contributor

Adds a self-contained colab_quickstart.ipynb at the repo root that runs a pretrained NanoWM checkpoint on free-tier Google Colab end-to-end. Linked from the README's Quick Start section.

Motivation

Getting from git clone to a working rollout on a fresh machine currently hits several rough edges that aren't covered by environment.yml alone — modern huggingface_hub / transformers versions break the pinned diffusers==0.24.0 imports, the released model.safetensors doesn't load via the pickle-based find_model, PyTorch >= 2.6 changed the torch.load(weights_only=...) default, etc. The notebook bakes all of these into a single top-to-bottom flow targeted at free-tier Colab (T4 GPU, 15 minutes total).

What it does

  1. GPU sanity check
  2. git clone the repo
  3. pip install a pinned dep set (huggingface_hub==0.20.3, transformers==4.36.0, diffusers==0.24.0, accelerate==0.25.0, plus the rest from environment.yml)
  4. One required runtime restart (Colab pre-imports huggingface_hub at session start)
  5. Idempotent patch for find_model to pass weights_only=False (see find_model fails on PyTorch >= 2.6 (weights_only default change) #10 / linked PR for the upstream fix; the cell becomes a no-op once that lands)
  6. snapshot_download the Point Maze B/2 checkpoint
  7. Convert model.safetensorsmodel.pt so find_model can load it
  8. Unzip the OSF Point Maze dataset (instructions for the manual download up top)
  9. Run src/sample/rollout.py with known-good DINO-WM Point Maze flags (--history_length 3, --rollout_length 20, --num_sampling_steps 15)
  10. Display the resulting compare.mp4 inline

Each cell is idempotent so a user can re-run individual steps after a transient failure without resetting the session. A troubleshooting table at the bottom maps each common error to which step to re-run.

Tested

Ran the notebook top to bottom on free-tier Colab T4 — produces the expected gen / gt / compare mp4s in ~15 minutes total wall-clock.

Notes

  • Targets Point Maze (smallest demo). Swapping the HF repo id in step 6 + the OSF folder in step 8 generalizes to PushT / Wall / Rope / Granular.
  • CSGO is intentionally out of scope (~675 GB dataset, not feasible on free Colab).
  • The notebook references the find_model weights-only patch issue (find_model fails on PyTorch >= 2.6 (weights_only default change) #10). Once that PR lands, the patch cell will no-op cleanly — no churn needed.

Adds a self-contained notebook that runs a pretrained NanoWM checkpoint
on Google Colab's free T4 GPU, end-to-end:

- Pins a known-good huggingface_hub / transformers / diffusers / accelerate
  set compatible with the existing diffusers==0.24.0 target
- Handles the released .safetensors -> .pt conversion needed by find_model
- Includes a one-liner idempotent patch for the PyTorch >= 2.6
  weights_only default change (see simchowitzlabpublic#10 / PR for an upstream fix)
- Uses known-good rollout flags for DINO-WM Point Maze
  (history_length=3, rollout_length=20, frame_interval=5)
- Idempotent cells: safe to re-run individual steps after a kernel
  restart or transient failure
- Inline troubleshooting table mapping each common error to which step
  to re-run

Linked from the README's Quick Start section.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant