NullSplats

Tkinter + OpenGL desktop app for training and viewing 3D Gaussian splats from casual captures. It wraps COLMAP for camera poses, uses PyTorch + gsplat for training, and stores everything in a reproducible cache tree so scenes can be resumed later.

What it does

nullsplats_720p30_h265_under10MB_noaudio.mp4

• Ingest a video or image folder, extract and score frames, and auto-select a subset.
• Run COLMAP SfM to produce camera poses and sparse points.
• Train Gaussian splats on the GPU with gsplat; export checkpoints as .ply or .splat.
• View splats in an embedded OpenGL viewer inside the app.
• Keep per-scene inputs/outputs under cache for repeatable workflows.

Nullsplats supports 3 methods of creating splats:

1. Traditional colmap + gsplat training.
2. Depth Anything 3 3D Gaussian Estimation.
3. SHARP Monocular View Synthesis.

Here are some sample splats trained using this program.

• 50 views — gsplat, 720p (iter 12k)
• 5 views — Depth Anything 3, 720p
• 1 view — SHARP, 720p

Video Examples

Gsplat 50-view

gsplat_50view_720p.mp4

Gsplat closeup

gsplat_closeup.mp4

DA3 5-view

DA3_5view_720p.mp4

DA3 closeup

DA3_closeup.mp4

SHARP 1-view

SHARP_1view_720p.mp4

SHARP closeup

SHARP_Closeup.mp4

Side-by-side comparison

Comparison_Closeup.mp4

On my RTX pro 6000 Blackwell the gsplat trained in around 5 minutes including colmap time. Depth Anything 3 took around 3-4 minutes but used a significant 16GB of vram. SHARP produced a splat in around 2.5 minutes.

Overall each is pretty good considering the input. The single view SHARP splat is particularly impressive. If DA3 was less weird, was less transparent it would be significantly better. The geometry looks pretty good.

Discord

Want to discuss development or get support? Open an Issue or checkout the discord.

Repository layout

• main.py - app entrypoint (Tk root + tabs).
• nullsplats/
  • ui/ - Tk UI, tabs, OpenGL viewers, shaders (ui/shaders/.vert|.frag).
  • backend/ - frame extraction, COLMAP pipeline, splat training.
  • util/ - logging, config, threading helpers and tool path defaults.
• build.bat - portable bundle builder.
• run.bat - launcher used inside the portable bundle.
• requirements.txt - Python dependencies.

Code Structure

Overview

The app is organized around a small core state object (AppState), four UI tabs (Inputs, COLMAP, Training, Exports), and a backend pipeline that handles frame extraction, COLMAP structure-from-motion, and gsplat training.

High-Level Flow

1. main.py sets up logging, creates AppState, and builds the Tk root in ui/root.py.
2. ui/root.py wires the four tabs and routes scene selection between them.
3. Inputs tab creates scenes, extracts frames, and persists selected/resized frames.
4. COLMAP tab runs SfM to generate camera poses and sparse points.
5. Training tab runs gsplat training and streams live previews.
6. Exports tab lists checkpoints and renders previews/turntables.

Core State and Caching

• nullsplats/app_state.py owns AppState (config + SceneManager + current scene).
• nullsplats/backend/io_cache.py defines ScenePaths and metadata read/write.
• Cache layout (per scene):
  • cache/inputs/<scene_id>/source (original source copy)
  • cache/inputs/<scene_id>/frames_all
  • cache/inputs/<scene_id>/frames_selected
  • cache/inputs/<scene_id>/metadata.json
  • cache/outputs/<scene_id>/sfm
  • cache/outputs/<scene_id>/splats
  • cache/outputs/<scene_id>/renders
• nullsplats/backend/scene_manager.py handles scene discovery, selection persistence, and thumbnail caching (thumbnails.db).

UI Architecture

Root + Tabs

• ui/root.py builds the ttk.Notebook, instantiates tabs, and coordinates tab changes.
• Tabs:
  • Inputs: ui/tab_inputs.py + mixins
  • COLMAP: ui/tab_colmap.py
  • Training: ui/tab_training.py + layout/preview mixins
  • Exports: ui/tab_exports.py
• Wizard flows:
  • Inline wizard in Inputs: ui/tab_inputs_wizard.py (single popup with inputs + training preset + COLMAP options).
  • Standalone wizard window: ui/wizard.py

Inputs Tab

• ui/tab_inputs.py is the coordinator for the Inputs workflow.
• ui/tab_inputs_scenes.py renders the scene sidebar and scene management.
• ui/tab_inputs_grid.py renders the virtualized frame grid and thumbnails.
• Key backend usage: backend/video_frames.py, backend/scene_manager.py.

COLMAP Tab

• ui/tab_colmap.py runs SfM and manages COLMAP settings/logs.
• Backend calls:
  • backend/sfm_pipeline.py (COLMAP CLI)

Training Tab

• ui/tab_training.py orchestrates training runs, manages logging, and owns preview state.
• Training method is selectable (gsplat or DA3). Live preview is gsplat-only.
• ui/tab_training_layout.py builds the UI widgets.
• ui/tab_training_preview.py handles preview polling + in-memory preview queue.
• Backend calls:
  • backend/splat_train.py (gsplat training loop)

Exports Tab

• ui/tab_exports.py lists checkpoints, opens a preview viewer, and renders turntables.
• Uses GLCanvas to display .ply checkpoints and imageio for video output.

Backend Pipeline

Frame Extraction

• backend/video_frames.py handles:
  • ffmpeg/ffprobe extraction
  • sharpness/variance scoring
  • auto-select of best frames
  • cache persistence of selections

SfM (COLMAP)

• backend/sfm_pipeline.py runs COLMAP feature extraction, matching, mapping, and model conversion. Logs go to cache/outputs/<scene_id>/sfm/logs.

Training

• backend/splat_train.py is the training entry point.
• Supporting modules:
  • backend/splat_train_config.py (dataclasses and callbacks)
  • backend/splat_train_io.py (COLMAP text parsing + frame loading)
  • backend/splat_train_ops.py (CUDA config, optimizers, export helpers)
  • backend/gs_utils.py (camera/appearance optimization utilities)
• DA3 backend: backend/splat_backends/depth_anything3_trainer.py (Depth Anything 3 inference + gs_ply export)

Rendering and Viewer Stack

• ui/gl_canvas.py is the main preview surface:
  • Wraps GaussianSplatViewer for live OpenGL display.
  • Uses SplatRenderer (gsplat rasterization) for offline renders and turntables.
  • Supports in-memory previews via PreviewPayload.
• ui/gaussian_splat_viewer.py is the OpenGL renderer (instanced quads + shaders).
• ui/gaussian_splat_camera.py contains camera math helpers.
• Shaders live in ui/shaders/gaussian_splat.vert and ui/shaders/gaussian_splat.frag.
• Control panels:
  • ui/render_controls.py (basic controls)
  • ui/advanced_render_controls.py (debug/scale/camera)
  • ui/colmap_camera_panel.py (apply COLMAP poses)

Threading and Logging

• util/threading.py runs background tasks and marshals callbacks to the Tk thread.
• util/logging.py sets a consistent console + file logger under logs/app.log.
• util/tooling_paths.py resolves default COLMAP and CUDA paths.

Tooling, Build, and Run

• requirements.txt contains core deps (torch, gsplat, PyOpenGL, etc.).
• build.bat creates a portable bundle with venv, COLMAP, and CUDA DLLs.
• run.bat is the launcher inside the portable bundle.

Requirements

• Windows (primary target) or a Linux environment with matching binaries.
• Python 3.10+ with pip/venv.
• GPU with a CUDA-capable driver; PyTorch CUDA build installed.
• ffmpeg/ffprobe on PATH (for video extraction).
• COLMAP binaries (CUDA build recommended) under tools/colmap or user-provided path.
• Optional: GLOMAP binaries under tools/glomap (future use).
• Optional: Depth Anything 3 backend (pip install from GitHub; no submodule).
• Optional: SHARP backend (tools/sharp is included; install editable deps).

Depth Anything 3 (DA3) backend

DA3 uses the official Depth Anything 3 API and emits .ply splats into the cache. It requires installing the project via pip from GitHub:

pip install git+https://github.com/ByteDance-Seed/Depth-Anything-3

DA3 settings live in the Training tab (process resolution, view selection, and COLMAP-based view scoring). If COLMAP confidence is missing, DA3 falls back to evenly spaced views.

SHARP backend

SHARP runs on single images and can optionally use COLMAP intrinsics for multi-view runs. The repo includes the SHARP source under tools/sharp; install it in editable mode:

pip install -e tools/sharp

Install for development

From repo root:

python -m venv .venv
.venv\Scripts\activate.bat
pip install -r requirements.txt

If you need to bootstrap CUDA-friendly PyTorch/gsplat, run tools\setup_cuda_venv.bat (optional helper if present).

For DA3 support:

pip install git+https://github.com/ByteDance-Seed/Depth-Anything-3

For SHARP support (if you skipped it in requirements):

pip install -e tools/sharp

Running the app

With the venv active:

python main.py

Logs go to logs and stdout; cache lives under cache/inputs/<scene_id> and cache/outputs/<scene_id>.

UI workflow

• Inputs tab: choose or create a Scene ID, select video or image folder, set candidate/target frame counts, then Extract Frames. Frames and metadata land in cache/inputs//.
• COLMAP tab: verify COLMAP path, matcher, and camera model, then run SfM. Outputs land in cache/outputs//sfm.
• Training tab: configure CUDA device and training hyperparams, then run training. Outputs land in cache/outputs//splats.
• Exports tab: browse checkpoints and preview them in the viewer.

Portable bundle (Windows)

build.bat creates a self-contained bundle under build\NullSplats-portable and build\NullSplats-portable.zip.

• Prereq: .venv populated with all deps (including CUDA PyTorch/gsplat).
• Optional: set SKIP_CLEAN=1 to reuse an existing bundle; pass a CUDA path as the first arg to override CUDA_PATH/CUDA_HOME for DLL copy. Set REQUIRE_CUDA=0 if you intentionally want to skip bundling CUDA DLLs (otherwise the build fails when CUDA is missing). CUDA copy pulls DLLs from CUDA_SRC\bin (cud*/nv*).
• Optional: set SKIP_ZIP=1 to skip creating the zip (faster). If 7z is on PATH, zipping uses -mx=0 (store-only) for speed; otherwise falls back to PowerShell Compress-Archive -CompressionLevel Fastest.
• The builder prunes unused Python packages (tqdm, tyro, opencv-python, PyYAML) and copies only core CUDA DLLs; COLMAP is bundled, GLOMAP is not.
• If you need to debug CUDA bundling, build.bat prints source and destination paths plus directory listings for the copied DLLs. Set REQUIRE_CUDA=1 to fail fast when none are copied.
• Run from repo root:

build.bat

Inside the bundle, use run.bat to launch; it activates the bundled venv and prepends bundled CUDA and COLMAP paths.

Known issues

1. Download is Windows + CUDA only.
2. COLMAP is not built during build process.
3. Download is large.
4. Thumbnails fail to load sometimes.
5. Splat rendering is slightly off.
6. Camera control is not great.
7. Does not include ffmpeg with the build.
8. May need CUDA SDK on the path, even though it is included.

Paths and defaults

• COLMAP default: bundled tools/colmap/COLMAP.bat if present; otherwise user-specified.
• CUDA default: bundled cuda/ inside the bundle; otherwise CUDA_PATH/CUDA_HOME, then C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.8.
• Cache: created on demand under cache/inputs and cache/outputs.