Fast and memory-efficient webdataset shard reader - Rust core with Python bindings, streaming metadata, and a bucketed dataloader.
-
Updated
Oct 10, 2026 - Rust
#
webdataset
Here are 3 public repositories matching this topic...
A Rust implementation of WebDataset: stream training data from tar shards, with a drop-in Python replacement checked field by field against the reference library.
rust machine-learning deep-learning tar webassembly dataset python-bindings dataloader no-std data-loading pyo3 webdataset
-
Updated
Sep 13, 2026 - Rust
Index, group, sample, resynchronize, and repack uncompressed WebDataset TAR archives
-
Updated
Aug 16, 2026 - Rust
Add this topic to your repo
To associate your repository with the webdataset topic, visit your repo's landing page and select "manage topics."