-
Notifications
You must be signed in to change notification settings - Fork 13.6k
Remove interior mutability in mir predecessors cache #64736
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
bors
merged 47 commits into
rust-lang:master
from
Nashenas88:mir_predecessors_cache_cleanup
Dec 2, 2019
Merged
Changes from 1 commit
Commits
Show all changes
47 commits
Select commit
Hold shift + click to select a range
c16ef6b
Remove interior mutability in mir predecessors cache
Nashenas88 b524059
Inline cache impl into Body, rename predecessor fns, change output of…
Nashenas88 ce29f43
Get rid of old comment
Nashenas88 f534d9f
Stop invalidating predecessors cache when accessing unique basic bloc…
Nashenas88 2b31456
Add pass to ensure predecessors cache is generated after optimization
Nashenas88 e1afa51
Fix Mir visitor macro to ensure it calls the proper method to invalid…
Nashenas88 570e418
Address linting errors caught by CI
Nashenas88 8e8c97e
Ensure predecessors are recomputed at critical points, fixes panics
Nashenas88 ad73468
Move predecessors cache invalidation back to basic_blocks_mut, add a …
Nashenas88 94414ac
Address excessive line length that was triggering warning during linting
Nashenas88 52cc85f
Address nits and remove unneeded pass
Nashenas88 22bc8a0
Add back cache invalidation to basic_blocks_and_local_decls_mut
Nashenas88 9b335ce
Move predecessors cache back to its own type
Nashenas88 c0592fa
Move predecessor cache outside of Body, use wrapper types to manage C…
Nashenas88 649c73f
Simplify Cache wrapper to single type, impl Deref on it, fix all comp…
Nashenas88 30b1d9e
Remove Body from FunctionCx, pass it along during librustc_codegen_ssa
Nashenas88 3d68f5f
Improved BodyCache body impl so it only returns a sharable ref, add n…
Nashenas88 16952cc
Add Body back as field of FunctionCx, but under a different lifetime
Nashenas88 66279d1
Revert back to using FunctionCx's Body
Nashenas88 c8c266a
Convert &mut to & since the reference didn't need to be mutable
Nashenas88 2eed90a
Account for new maybe_sideeffect helper that requires predecessors
Nashenas88 ab98c59
Fix a large number of Body -> (ReadOnly)BodyCache type errors, add pr…
Nashenas88 26f1c01
Add read_only fn to BodyCache<&mut...> impl, fix more Body -> (ReadOn…
Nashenas88 0a19371
Add predecessors fn to ReadOnlyBodyCache, fix more Body -> (ReadOnly)…
Nashenas88 3642a71
Fix typo caused by rebasing
Nashenas88 38c0887
Fix remaining Body -> (ReadOnly)BodyCache type errors in librustc_mir…
Nashenas88 fc6b58d
Simplify BodyCache impl and fix all remaining type errors in librustc…
Nashenas88 4de31b2
Fix remaining compilation issues
Nashenas88 35590b5
Fix typos caused during rebase
Nashenas88 67b7a78
Fix tidy errors
Nashenas88 ab657e3
Fix typo
Nashenas88 e54c610
Fix compilation errors created during rebase
Nashenas88 c42bdb8
Undo minor changes that weren't needed, fix one lifetime typo
Nashenas88 595d161
Remove BodyCache.body and rely on Deref as much as possible for ReadO…
Nashenas88 b2fe254
Remove HasLocalDecls impl from BodyCache's, properly reborrow to Body…
Nashenas88 51b0665
Fix typos caused during rebase
Nashenas88 ed90818
Remove files created during conflict resolution
Nashenas88 05dc5e9
Compute predecessors in mir_build query and use existing cache for ge…
Nashenas88 245abc4
Fix type errors cause during rebasing
Nashenas88 c6354e9
Remove inline attributes that hadn't been profiled, unexport Cache si…
Nashenas88 598797c
Remove unchecked inline attribute, remove unused functions, make chac…
Nashenas88 64654ce
Fix type errors created during rebasing
Nashenas88 9978574
Fix rebasing errors, convert some BodyCache::body() calls to reborrows
Nashenas88 acb90eb
Fix tidy issues
Nashenas88 38bd3a2
Use new HashStable proc macro
Nashenas88 6123478
Fix issues caused during rebasing
Nashenas88 3eaad56
Fix issues caused during rebasing
Nashenas88 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Move predecessor cache outside of Body, use wrapper types to manage C…
…ache and Body (WIP, amend this commit)
- Loading branch information
commit c0592faa67a1fe8fb7425f24899c5538dec23ee1
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,47 +1,255 @@ | ||
use rustc_index::vec::IndexVec; | ||
use rustc_data_structures::stable_hasher::{HashStable, StableHasher}; | ||
use rustc_serialize::{Encodable, Encoder, Decodable, Decoder}; | ||
use crate::ich::StableHashingContext; | ||
use crate::mir::BasicBlock; | ||
//use rustc_data_structures::stable_hasher::{HashStable, StableHasher}; | ||
//use rustc_serialize::{Encodable, Encoder, Decodable, Decoder}; | ||
//use crate::ich::StableHashingContext; | ||
use crate::mir::{BasicBlock, BasicBlockData, Body, LocalDecls, Location, Successors}; | ||
use rustc_data_structures::graph::{self, GraphPredecessors, GraphSuccessors}; | ||
use rustc_data_structures::graph::dominators::{dominators, Dominators}; | ||
use std::iter; | ||
use std::ops::{Index, IndexMut}; | ||
use std::vec::IntoIter; | ||
|
||
#[derive(Clone, Debug)] | ||
pub(in crate::mir) struct Cache { | ||
pub(in crate::mir) predecessors: Option<IndexVec<BasicBlock, Vec<BasicBlock>>> | ||
pub struct Cache { | ||
predecessors: Option<IndexVec<BasicBlock, Vec<BasicBlock>>>, | ||
} | ||
|
||
|
||
impl rustc_serialize::Encodable for Cache { | ||
fn encode<S: Encoder>(&self, s: &mut S) -> Result<(), S::Error> { | ||
Encodable::encode(&(), s) | ||
//impl<'tcx, T> rustc_serialize::Encodable for Cache<'tcx, T> { | ||
// fn encode<S: Encoder>(&self, s: &mut S) -> Result<(), S::Error> { | ||
// Encodable::encode(&(), s) | ||
// } | ||
//} | ||
// | ||
//impl<'tcx, T> rustc_serialize::Decodable for Cache<'tcx, T> { | ||
// fn decode<D: Decoder>(d: &mut D) -> Result<Self, D::Error> { | ||
// Decodable::decode(d).map(|_v: ()| Self::new()) | ||
// } | ||
//} | ||
// | ||
//impl<'a, 'tcx, T> HashStable<StableHashingContext<'a>> for Cache<'tcx, T> { | ||
// fn hash_stable(&self, _: &mut StableHashingContext<'a>, _: &mut StableHasher) { | ||
// // Do nothing. | ||
// } | ||
//} | ||
|
||
impl Cache { | ||
pub fn new() -> Self { | ||
Self { | ||
predecessors: None, | ||
} | ||
} | ||
|
||
#[inline] | ||
pub fn invalidate_predecessors(&mut self) { | ||
// FIXME: consider being more fine-grained | ||
self.predecessors = None; | ||
} | ||
|
||
#[inline] | ||
/// This will recompute the predecessors cache if it is not available | ||
pub fn predecessors(&mut self, body: &Body<'_>) -> &IndexVec<BasicBlock, Vec<BasicBlock>> { | ||
if self.predecessors.is_none() { | ||
let mut result = IndexVec::from_elem(vec![], body.basic_blocks()); | ||
for (bb, data) in body.basic_blocks().iter_enumerated() { | ||
if let Some(ref term) = data.terminator { | ||
for &tgt in term.successors() { | ||
result[tgt].push(bb); | ||
} | ||
} | ||
} | ||
|
||
self.predecessors = Some(result) | ||
} | ||
|
||
self.predecessors.as_ref().unwrap() | ||
} | ||
|
||
#[inline] | ||
pub fn predecessors_for(&mut self, bb: BasicBlock, body: &Body<'_>) -> &[BasicBlock] { | ||
&self.predecessors(body)[bb] | ||
} | ||
|
||
#[inline] | ||
pub fn predecessor_locations<'a>(&'a mut self, loc: Location, body: &'a Body<'a>) -> impl Iterator<Item = Location> + 'a { | ||
let if_zero_locations = if loc.statement_index == 0 { | ||
let predecessor_blocks = self.predecessors_for(loc.block, body); | ||
let num_predecessor_blocks = predecessor_blocks.len(); | ||
Some( | ||
(0..num_predecessor_blocks) | ||
.map(move |i| predecessor_blocks[i]) | ||
.map(move |bb| body.terminator_loc(bb)), | ||
) | ||
} else { | ||
None | ||
}; | ||
|
||
let if_not_zero_locations = if loc.statement_index == 0 { | ||
None | ||
} else { | ||
Some(Location { block: loc.block, statement_index: loc.statement_index - 1 }) | ||
}; | ||
|
||
if_zero_locations.into_iter().flatten().chain(if_not_zero_locations) | ||
} | ||
|
||
#[inline] | ||
pub fn basic_blocks_mut<'a, 'tcx>(&mut self, body: &'a mut Body<'tcx>) -> &'a mut IndexVec<BasicBlock, BasicBlockData<'tcx>> { | ||
debug!("bbm: Clearing predecessors cache for body at: {:?}", body.span.data()); | ||
self.invalidate_predecessors(); | ||
&mut body.basic_blocks | ||
} | ||
|
||
#[inline] | ||
pub fn basic_blocks_and_local_decls_mut<'a, 'tcx>( | ||
&mut self, | ||
body: &'a mut Body<'tcx> | ||
) -> (&'a mut IndexVec<BasicBlock, BasicBlockData<'tcx>>, &'a mut LocalDecls<'tcx>) { | ||
debug!("bbaldm: Clearing predecessors cache for body at: {:?}", body.span.data()); | ||
self.invalidate_predecessors(); | ||
(&mut body.basic_blocks, &mut body.local_decls) | ||
} | ||
} | ||
|
||
pub struct OwningCache<'tcx> { | ||
cache: Cache, | ||
body: Body<'tcx>, | ||
Nashenas88 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
} | ||
|
||
impl<'tcx> OwningCache<'tcx> { | ||
pub fn borrow(&mut self) -> BorrowedCache<'_, 'tcx> { | ||
BorrowedCache { | ||
cache: &mut self.cache, | ||
body: &self.body, | ||
} | ||
} | ||
|
||
pub fn borrow_mut(&mut self) -> MutCache<'_, 'tcx> { | ||
MutCache { | ||
cache: &mut self.cache, | ||
body: &mut self.body, | ||
} | ||
} | ||
} | ||
|
||
pub struct BorrowedCache<'a, 'tcx> { | ||
cache: &'a mut Cache, | ||
body: &'a Body<'tcx> | ||
} | ||
|
||
impl<'a, 'tcx> BorrowedCache<'a, 'tcx> { | ||
#[inline] | ||
pub fn predecessors_for(&mut self, bb: BasicBlock) -> &[BasicBlock] { | ||
self.cache.predecessors_for(bb, self.body) | ||
} | ||
|
||
#[inline] | ||
pub fn body(&self) -> &Body<'tcx> { | ||
self.body | ||
} | ||
|
||
#[inline] | ||
pub fn basic_blocks(&self) -> &IndexVec<BasicBlock, BasicBlockData<'tcx>> { | ||
&self.body.basic_blocks | ||
} | ||
|
||
#[inline] | ||
pub fn dominators(&mut self) -> Dominators<BasicBlock> { | ||
dominators(self) | ||
} | ||
} | ||
|
||
impl rustc_serialize::Decodable for Cache { | ||
fn decode<D: Decoder>(d: &mut D) -> Result<Self, D::Error> { | ||
Decodable::decode(d).map(|_v: ()| Self::new()) | ||
impl<'a, 'tcx> Index<BasicBlock> for BorrowedCache<'a, 'tcx> { | ||
type Output = BasicBlockData<'tcx>; | ||
|
||
#[inline] | ||
fn index(&self, index: BasicBlock) -> &BasicBlockData<'tcx> { | ||
&self.body[index] | ||
} | ||
} | ||
|
||
impl<'a> HashStable<StableHashingContext<'a>> for Cache { | ||
fn hash_stable(&self, _: &mut StableHashingContext<'a>, _: &mut StableHasher) { | ||
// Do nothing. | ||
impl<'a, 'tcx> graph::DirectedGraph for BorrowedCache<'a, 'tcx> { | ||
type Node = BasicBlock; | ||
} | ||
|
||
impl<'a, 'graph, 'tcx> graph::GraphPredecessors<'graph> for BorrowedCache<'a, 'tcx> { | ||
type Item = BasicBlock; | ||
type Iter = IntoIter<BasicBlock>; | ||
} | ||
|
||
impl<'a, 'tcx> graph::WithPredecessors for BorrowedCache<'a, 'tcx> { | ||
fn predecessors( | ||
&mut self, | ||
node: Self::Node, | ||
) -> <Self as GraphPredecessors<'_>>::Iter { | ||
self.predecessors_for(node).to_vec().into_iter() | ||
} | ||
} | ||
|
||
impl Cache { | ||
pub fn new() -> Self { | ||
Cache { | ||
predecessors: None | ||
} | ||
impl<'a, 'tcx> graph::WithNumNodes for BorrowedCache<'a, 'tcx> { | ||
fn num_nodes(&self) -> usize { | ||
self.body.num_nodes() | ||
} | ||
} | ||
|
||
impl<'a, 'tcx> graph::WithStartNode for BorrowedCache<'a, 'tcx> { | ||
fn start_node(&self) -> Self::Node { | ||
self.body.start_node() | ||
} | ||
} | ||
|
||
impl<'a, 'tcx> graph::WithSuccessors for BorrowedCache<'a, 'tcx> { | ||
fn successors( | ||
&self, | ||
node: Self::Node, | ||
) -> <Self as GraphSuccessors<'_>>::Iter { | ||
self.body.successors(node) | ||
} | ||
} | ||
|
||
impl<'a, 'b, 'tcx> graph::GraphSuccessors<'b> for BorrowedCache<'a, 'tcx> { | ||
type Item = BasicBlock; | ||
type Iter = iter::Cloned<Successors<'b>>; | ||
} | ||
|
||
pub struct MutCache<'a, 'tcx> { | ||
cache: &'a mut Cache, | ||
body: &'a mut Body<'tcx>, | ||
} | ||
|
||
impl<'a, 'tcx> MutCache<'a, 'tcx> { | ||
#[inline] | ||
pub fn invalidate_predecessors(&mut self) { | ||
// FIXME: consider being more fine-grained | ||
self.predecessors = None; | ||
pub fn body(&mut self) -> &mut Body<'tcx> { | ||
self.body | ||
} | ||
|
||
#[inline] | ||
pub fn basic_blocks(&self) -> &IndexVec<BasicBlock, BasicBlockData<'tcx>> { | ||
&self.body.basic_blocks | ||
} | ||
|
||
#[inline] | ||
pub fn basic_blocks_mut(&mut self) -> &mut IndexVec<BasicBlock, BasicBlockData<'tcx>> { | ||
self.cache.basic_blocks_mut(&mut self.body) | ||
} | ||
} | ||
|
||
CloneTypeFoldableAndLiftImpls! { | ||
Cache, | ||
impl<'a, 'tcx> Index<BasicBlock> for MutCache<'a, 'tcx> { | ||
type Output = BasicBlockData<'tcx>; | ||
|
||
#[inline] | ||
fn index(&self, index: BasicBlock) -> &BasicBlockData<'tcx> { | ||
&self.body[index] | ||
} | ||
} | ||
|
||
impl<'a, 'tcx> IndexMut<BasicBlock> for MutCache<'a, 'tcx> { | ||
fn index_mut(&mut self, index: BasicBlock) -> &mut Self::Output { | ||
self.cache.invalidate_predecessors(); | ||
&mut self.body.basic_blocks[index] | ||
} | ||
} | ||
|
||
//CloneTypeFoldableAndLiftImpls! { | ||
// Cache, | ||
//} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.