Skip to content

Commit

Permalink
rustc: Preallocate when building the dep graph
Browse files Browse the repository at this point in the history
This commit alters the `query` function in the dep graph module to preallocate
memory using `with_capacity` instead of relying on automatic growth. Discovered
in rust-lang#44576 it was found that for the syntex_syntax clean incremental benchmark
the peak memory usage was found when the dep graph was being saved, particularly
the `DepGraphQuery` data structure itself. PRs like rust-lang#44142 which add more
queries end up just making this much larger!

I didn't see an immediately obvious way to reduce the size of the
`DepGraphQuery` object, but it turns out that `with_capacity` helps quite a bit!
Locally 831 MB was used [before] this commit, and 770 MB is in use at the peak
of the compiler [after] this commit. That's a nice 7.5% improvement! This won't
quite make up for the losses in rust-lang#44142 but I figured it's a good start.

[before]: https://gist.github.com/alexcrichton/2d2b9c7a65503761925c5a0bcfeb0d1e
[before]: https://gist.github.com/alexcrichton/6da51f2a6184bfb81694cc44f06deb5b
  • Loading branch information
alexcrichton committed Sep 15, 2017
1 parent 2d288a5 commit a7817dd
Show file tree
Hide file tree
Showing 3 changed files with 16 additions and 3 deletions.
5 changes: 2 additions & 3 deletions src/librustc/dep_graph/query.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,10 @@ impl DepGraphQuery {
pub fn new(nodes: &[DepNode],
edges: &[(DepNode, DepNode)])
-> DepGraphQuery {
let mut graph = Graph::new();
let mut graph = Graph::with_capacity(nodes.len(), edges.len());
let mut indices = FxHashMap();
for node in nodes {
indices.insert(node.clone(), graph.next_node_index());
graph.add_node(node.clone());
indices.insert(node.clone(), graph.add_node(node.clone()));
}

for &(ref source, ref target) in edges {
Expand Down
7 changes: 7 additions & 0 deletions src/librustc_data_structures/graph/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,13 @@ impl<N: Debug, E: Debug> Graph<N, E> {
}
}

pub fn with_capacity(nodes: usize, edges: usize) -> Graph<N, E> {
Graph {
nodes: SnapshotVec::with_capacity(nodes),
edges: SnapshotVec::with_capacity(edges),
}
}

// # Simple accessors

#[inline]
Expand Down
7 changes: 7 additions & 0 deletions src/librustc_data_structures/snapshot_vec.rs
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,13 @@ impl<D: SnapshotVecDelegate> SnapshotVec<D> {
}
}

pub fn with_capacity(n: usize) -> SnapshotVec<D> {
SnapshotVec {
values: Vec::with_capacity(n),
undo_log: Vec::new(),
}
}

fn in_snapshot(&self) -> bool {
!self.undo_log.is_empty()
}
Expand Down

0 comments on commit a7817dd

Please sign in to comment.