Skip to content

Commit

Permalink
[Turbopack] add persisting to the new backend (#69668)
Browse files Browse the repository at this point in the history
### What?

Takes regular snapshots of the state of the system.

Stores task cache and task data in a lmdb database. 

Currently active operations are paused, stored and continued on restore.

Task Cache need large keys, so key hashing and lookup is added on top of
LMDB for that.
  • Loading branch information
sokra authored and kdy1 committed Oct 10, 2024
1 parent faefbfd commit 74d35aa
Show file tree
Hide file tree
Showing 34 changed files with 2,060 additions and 262 deletions.
55 changes: 53 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 10 additions & 0 deletions turbopack/crates/turbo-tasks-backend/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,21 +12,31 @@ bench = false
[lints]
workspace = true

[features]
default = []
verify_serialization = []

[dependencies]
anyhow = { workspace = true }
async-trait = { workspace = true }
auto-hash-map = { workspace = true }
byteorder = "1.5.0"
dashmap = { workspace = true, features = ["raw-api"]}
either = { workspace = true }
hashbrown = { workspace = true }
indexmap = { workspace = true }
lmdb-rkv = "0.14.0"
once_cell = { workspace = true }
parking_lot = { workspace = true }
pot = "3.0.0"
rand = { workspace = true }
rayon = { workspace = true }
rustc-hash = { workspace = true }
serde = { workspace = true }
serde_path_to_error = { workspace = true }
smallvec = { workspace = true }
tokio = { workspace = true }
tokio-scoped = "0.2.0"
tracing = { workspace = true }
turbo-prehash = { workspace = true }
turbo-tasks = { workspace = true }
Expand Down
Loading

0 comments on commit 74d35aa

Please sign in to comment.