Auto merge of rust-lang#134268 - lqd:polonius-next, r=<try>

Foundations of location-sensitive polonius

I'd to land the prototype I'm describing in the [polonius project goal](rust-lang/rust-project-goals#118). It still is incomplete and naive and terrible but it's working "well enough" to consider landing.

I'd also like to make review easier by not opening a huge PR, but have a couple small-ish ones (the +/- line change summary of this PR looks big, but >80% is moving datalog to a single place).

This PR starts laying the foundation for that work:
- it refactors and collects 99% of the old datalog fact gen, which was spread around everywhere, into a single dedicated module. It's still present at 3 small places (one of which we should revert anyways) that are kinda deep within localized components that are not as easily extractable into the rest of fact gen, so it's fine for now.
- starts introducing the localized constraints, the building blocks of the naive way of implementing the location-sensitive analysis in-tree, which is roughly sketched out in https://smallcultfollowing.com/babysteps/blog/2023/09/22/polonius-part-1/ and https://smallcultfollowing.com/babysteps/blog/2023/09/29/polonius-part-2/ but with a different vibe than per-point environments, just `r1@p: r2@q` constraints.
- sets up the skeleton of generating these localized constraints: converting NLL typeck constraints, and creating liveness constraints
- introduces the polonius dual to NLL MIR to help development and debugging. It doesn't do much original currently but is a way to see these localized constraints: its an NLL MIR dump + a dumb listing of the constraints. It's not intended to be a long-term thing, it's for testing purposes, and I will replace its contents in the future with a different approach (an HTML where we can more easily explore these constraints, have mermaid graphs of the usual graphviz dumps, etc)

I've started documenting the approach in this PR, I'll add more in the future. It's quite simple, and should be very clear when more constraints are introduced anyways.

r? `@matthewjasper`

Best reviewed per commit so that the datalog move is less bothersome to read, but if you'd prefer we separate that into a different PR, I can do that.

Author: bors

compiler/rustc_borrowck/src/lib.rs

@@ -206,6 +206,7 @@ fn do_mir_borrowck<'tcx>(
         polonius_output,
         opt_closure_req,
         nll_errors,
+        localized_outlives_constraints,
     } = nll::compute_regions(
         &infcx,
         free_regions,
@@ -318,6 +319,16 @@ fn do_mir_borrowck<'tcx>(
 
     mbcx.report_move_errors();
 
+    // If requested, dump polonius MIR.
+    polonius::dump_polonius_mir(
+        &infcx,
+        body,
+        &regioncx,
+        &borrow_set,
+        &localized_outlives_constraints,
+        &opt_closure_req,
+    );
+
     // For each non-user used mutable variable, check if it's been assigned from
     // a user-declared local. If so, then put that local into the used_mut set.
     // Note that this set is expected to be small - only upvars from closures

compiler/rustc_borrowck/src/nll.rs

@@ -29,6 +29,7 @@ use crate::consumers::ConsumerOptions;
 use crate::diagnostics::RegionErrors;
 use crate::facts::{AllFacts, AllFactsExt, RustcFacts};
 use crate::location::LocationTable;
+use crate::polonius::LocalizedOutlivesConstraintSet;
 use crate::region_infer::RegionInferenceContext;
 use crate::type_check::{self, MirTypeckResults};
 use crate::universal_regions::UniversalRegions;
@@ -45,6 +46,9 @@ pub(crate) struct NllOutput<'tcx> {
     pub polonius_output: Option<Box<PoloniusOutput>>,
     pub opt_closure_req: Option<ClosureRegionRequirements<'tcx>>,
     pub nll_errors: RegionErrors<'tcx>,
+
+    /// When using `-Zpolonius=next`: the localized typeck and liveness constraints.
+    pub localized_outlives_constraints: LocalizedOutlivesConstraintSet,
 }
 
 /// Rewrites the regions in the MIR to use NLL variables, also scraping out the set of universal
@@ -116,14 +120,15 @@ pub(crate) fn compute_regions<'a, 'tcx>(
     let var_origins = infcx.get_region_var_origins();
 
     // If requested, emit legacy polonius facts.
-    polonius::emit_facts(
+    polonius::legacy::emit_facts(
         &mut all_facts,
         infcx.tcx,
         location_table,
         body,
         borrow_set,
         move_data,
         &universal_region_relations,
+        &constraints,
     );
 
     let mut regioncx = RegionInferenceContext::new(
@@ -134,6 +139,18 @@ pub(crate) fn compute_regions<'a, 'tcx>(
         elements,
     );
 
+    // If requested for `-Zpolonius=next`, convert NLL constraints to localized outlives
+    // constraints.
+    let mut localized_outlives_constraints = LocalizedOutlivesConstraintSet::default();
+    if infcx.tcx.sess.opts.unstable_opts.polonius.is_next_enabled() {
+        polonius::create_localized_constraints(
+            &mut regioncx,
+            infcx.infcx.tcx,
+            body,
+            &mut localized_outlives_constraints,
+        );
+    }
+
     // If requested: dump NLL facts, and run legacy polonius analysis.
     let polonius_output = all_facts.as_ref().and_then(|all_facts| {
         if infcx.tcx.sess.opts.unstable_opts.nll_facts {
@@ -174,6 +191,7 @@ pub(crate) fn compute_regions<'a, 'tcx>(
         polonius_output,
         opt_closure_req: closure_region_requirements,
         nll_errors,
+        localized_outlives_constraints,
     }
 }
 
@@ -214,40 +232,7 @@ pub(super) fn dump_nll_mir<'tcx>(
         &0,
         body,
         |pass_where, out| {
-            match pass_where {
-                // Before the CFG, dump out the values for each region variable.
-                PassWhere::BeforeCFG => {
-                    regioncx.dump_mir(tcx, out)?;
-                    writeln!(out, "|")?;
-
-                    if let Some(closure_region_requirements) = closure_region_requirements {
-                        writeln!(out, "| Free Region Constraints")?;
-                        for_each_region_constraint(tcx, closure_region_requirements, &mut |msg| {
-                            writeln!(out, "| {msg}")
-                        })?;
-                        writeln!(out, "|")?;
-                    }
-
-                    if borrow_set.len() > 0 {
-                        writeln!(out, "| Borrows")?;
-                        for (borrow_idx, borrow_data) in borrow_set.iter_enumerated() {
-                            writeln!(
-                                out,
-                                "| {:?}: issued at {:?} in {:?}",
-                                borrow_idx, borrow_data.reserve_location, borrow_data.region
-                            )?;
-                        }
-                        writeln!(out, "|")?;
-                    }
-                }
-
-                PassWhere::BeforeLocation(_) => {}
-
-                PassWhere::AfterTerminator(_) => {}
-
-                PassWhere::BeforeBlock(_) | PassWhere::AfterLocation(_) | PassWhere::AfterCFG => {}
-            }
-            Ok(())
+            emit_nll_mir(tcx, regioncx, closure_region_requirements, borrow_set, pass_where, out)
         },
         options,
     );
@@ -265,6 +250,51 @@ pub(super) fn dump_nll_mir<'tcx>(
     };
 }
 
+/// Produces the actual NLL MIR sections to emit during the dumping process.
+pub(crate) fn emit_nll_mir<'tcx>(
+    tcx: TyCtxt<'tcx>,
+    regioncx: &RegionInferenceContext<'tcx>,
+    closure_region_requirements: &Option<ClosureRegionRequirements<'tcx>>,
+    borrow_set: &BorrowSet<'tcx>,
+    pass_where: PassWhere,
+    out: &mut dyn io::Write,
+) -> io::Result<()> {
+    match pass_where {
+        // Before the CFG, dump out the values for each region variable.
+        PassWhere::BeforeCFG => {
+            regioncx.dump_mir(tcx, out)?;
+            writeln!(out, "|")?;
+
+            if let Some(closure_region_requirements) = closure_region_requirements {
+                writeln!(out, "| Free Region Constraints")?;
+                for_each_region_constraint(tcx, closure_region_requirements, &mut |msg| {
+                    writeln!(out, "| {msg}")
+                })?;
+                writeln!(out, "|")?;
+            }
+
+            if borrow_set.len() > 0 {
+                writeln!(out, "| Borrows")?;
+                for (borrow_idx, borrow_data) in borrow_set.iter_enumerated() {
+                    writeln!(
+                        out,
+                        "| {:?}: issued at {:?} in {:?}",
+                        borrow_idx, borrow_data.reserve_location, borrow_data.region
+                    )?;
+                }
+                writeln!(out, "|")?;
+            }
+        }
+
+        PassWhere::BeforeLocation(_) => {}
+
+        PassWhere::AfterTerminator(_) => {}
+
+        PassWhere::BeforeBlock(_) | PassWhere::AfterLocation(_) | PassWhere::AfterCFG => {}
+    }
+    Ok(())
+}
+
 #[allow(rustc::diagnostic_outside_of_impl)]
 #[allow(rustc::untranslatable_diagnostic)]
 pub(super) fn dump_annotation<'tcx, 'infcx>(

compiler/rustc_borrowck/src/polonius/constraints.rs

@@ -0,0 +1,45 @@
+use rustc_middle::ty::RegionVid;
+use rustc_mir_dataflow::points::PointIndex;
+
+/// A localized outlives constraint reifies the CFG location where the outlives constraint holds,
+/// within the origins themselves as if they were different from point to point: from `a: b`
+/// outlives constraints to `a@p: b@p`, where `p` is the point in the CFG.
+///
+/// This models two sources of constraints:
+/// - constraints that traverse the subsets between regions at a given point, `a@p: b@p`. These
+///   depend on typeck constraints generated via assignments, calls, etc. (In practice there are
+///   subtleties where a statement's effect only starts being visible at the successor point, via
+///   the "result" of that statement).
+/// - constraints that traverse the CFG via the same region, `a@p: a@q`, where `p` is a predecessor
+///   of `q`. These depend on the liveness of the regions at these points, as well as their
+///   variance.
+///
+/// The `source` origin at `from` flows into the `target` origin at `to`.
+///
+/// This dual of NLL's [crate::constraints::OutlivesConstraint] therefore encodes the
+/// position-dependent outlives constraints used by Polonius, to model the flow-sensitive loan
+/// propagation via reachability within a graph of localized constraints.
+#[derive(Copy, Clone, PartialEq, Eq, Debug, Hash)]
+pub(crate) struct LocalizedOutlivesConstraint {
+    pub source: RegionVid,
+    pub from: PointIndex,
+    pub target: RegionVid,
+    pub to: PointIndex,
+}
+
+/// A container of [LocalizedOutlivesConstraint]s that can be turned into a traversable
+/// `rustc_data_structures` graph.
+#[derive(Clone, Default, Debug)]
+pub(crate) struct LocalizedOutlivesConstraintSet {
+    pub outlives: Vec<LocalizedOutlivesConstraint>,
+}
+
+impl LocalizedOutlivesConstraintSet {
+    pub(crate) fn push(&mut self, constraint: LocalizedOutlivesConstraint) {
+        if constraint.source == constraint.target && constraint.from == constraint.to {
+            // 'a@p: 'a@p is pretty uninteresting
+            return;
+        }
+        self.outlives.push(constraint);
+    }
+}

compiler/rustc_borrowck/src/polonius/dump.rs

@@ -0,0 +1,101 @@
+use std::io;
+
+use rustc_middle::mir::pretty::{PrettyPrintMirOptions, dump_mir_with_options};
+use rustc_middle::mir::{Body, ClosureRegionRequirements, PassWhere};
+use rustc_middle::ty::TyCtxt;
+use rustc_session::config::MirIncludeSpans;
+
+use crate::borrow_set::BorrowSet;
+use crate::polonius::{LocalizedOutlivesConstraint, LocalizedOutlivesConstraintSet};
+use crate::{BorrowckInferCtxt, RegionInferenceContext};
+
+/// `-Zdump-mir=polonius` dumps MIR annotated with NLL and polonius specific information.
+// Note: this currently duplicates most of NLL MIR, with some additions for the localized outlives
+// constraints. This is ok for now as this dump will change in the near future to an HTML file to
+// become more useful.
+pub(crate) fn dump_polonius_mir<'tcx>(
+    infcx: &BorrowckInferCtxt<'tcx>,
+    body: &Body<'tcx>,
+    regioncx: &RegionInferenceContext<'tcx>,
+    borrow_set: &BorrowSet<'tcx>,
+    localized_outlives_constraints: &LocalizedOutlivesConstraintSet,
+    closure_region_requirements: &Option<ClosureRegionRequirements<'tcx>>,
+) {
+    let tcx = infcx.tcx;
+    if !tcx.sess.opts.unstable_opts.polonius.is_next_enabled() {
+        return;
+    }
+
+    // We want the NLL extra comments printed by default in NLL MIR dumps (they were removed in
+    // #112346). Specifying `-Z mir-include-spans` on the CLI still has priority: for example,
+    // they're always disabled in mir-opt tests to make working with blessed dumps easier.
+    let options = PrettyPrintMirOptions {
+        include_extra_comments: matches!(
+            tcx.sess.opts.unstable_opts.mir_include_spans,
+            MirIncludeSpans::On | MirIncludeSpans::Nll
+        ),
+    };
+
+    dump_mir_with_options(
+        tcx,
+        false,
+        "polonius",
+        &0,
+        body,
+        |pass_where, out| {
+            emit_polonius_mir(
+                tcx,
+                regioncx,
+                closure_region_requirements,
+                borrow_set,
+                localized_outlives_constraints,
+                pass_where,
+                out,
+            )
+        },
+        options,
+    );
+}
+
+/// Produces the actual NLL + Polonius MIR sections to emit during the dumping process.
+fn emit_polonius_mir<'tcx>(
+    tcx: TyCtxt<'tcx>,
+    regioncx: &RegionInferenceContext<'tcx>,
+    closure_region_requirements: &Option<ClosureRegionRequirements<'tcx>>,
+    borrow_set: &BorrowSet<'tcx>,
+    localized_outlives_constraints: &LocalizedOutlivesConstraintSet,
+    pass_where: PassWhere,
+    out: &mut dyn io::Write,
+) -> io::Result<()> {
+    // Emit the regular NLL front-matter
+    crate::nll::emit_nll_mir(
+        tcx,
+        regioncx,
+        closure_region_requirements,
+        borrow_set,
+        pass_where.clone(),
+        out,
+    )?;
+
+    let liveness = regioncx.liveness_constraints();
+
+    // Add localized outlives constraints
+    match pass_where {
+        PassWhere::BeforeCFG => {
+            if localized_outlives_constraints.outlives.len() > 0 {
+                writeln!(out, "| Localized constraints")?;
+
+                for constraint in &localized_outlives_constraints.outlives {
+                    let LocalizedOutlivesConstraint { source, from, target, to } = constraint;
+                    let from = liveness.location_from_point(*from);
+                    let to = liveness.location_from_point(*to);
+                    writeln!(out, "| {source:?} at {from:?} -> {target:?} at {to:?}")?;
+                }
+                writeln!(out, "|")?;
+            }
+        }
+        _ => {}
+    }
+
+    Ok(())
+}