generalize search graph to enable fuzzing #127627

lcnr · 2024-07-11T21:20:26Z

I do not believe it to be feasible to correctly implement the search graph without fuzzing. This PR enables this by requiring a fuzzer to only implement three new traits:

Cx: implemented by all I: Interner
ProofTreeBuilder: implemented by struct ProofTreeBuilder<D> for all D: SolverDelegate
Delegate: implemented for a new struct SearchGraphDelegate<D> for all D: SolverDelegate

It also moves the evaluation cache implementation into rustc_type_ir, requiring Interner to provide methods to create and access arbitrary WithDepNode<T> and to provide mutable access to a given GlobalCache. It otherwise does not change the API surface for users of the shared library.

This change should not impact behavior in any way.

r? @compiler-errors

compiler/rustc_next_trait_solver/src/solve/search_graph.rs

compiler-errors

r=me after nits and comments

compiler/rustc_next_trait_solver/src/solve/inspect/build.rs

compiler/rustc_type_ir/src/search_graph/global_cache.rs

compiler/rustc_type_ir/src/search_graph/mod.rs

lcnr · 2024-07-12T10:04:34Z

@bors r=compiler-errors rollup

bors · 2024-07-12T10:04:36Z

📌 Commit 208d709 has been approved by compiler-errors

It is now in the queue for this repository.

compiler-errors · 2024-07-12T10:19:16Z

compiler/rustc_next_trait_solver/src/solve/search_graph.rs

 use crate::delegate::SolverDelegate;
-use crate::solve::inspect::{self, ProofTreeBuilder};
-use crate::solve::{
-    CacheData, CanonicalInput, Certainty, QueryResult, SolverMode, FIXPOINT_STEP_LIMIT,
-};
-
-#[derive(Copy, Clone, PartialEq, Eq, Debug)]
-pub struct SolverLimit(usize);
-
-rustc_index::newtype_index! {
-    #[orderable]
-    #[gate_rustc_only]
-    pub struct StackDepth {}
-}
-
-bitflags::bitflags! {
-    /// Whether and how this goal has been used as the root of a
-    /// cycle. We track the kind of cycle as we're otherwise forced
-    /// to always rerun at least once.
-    #[derive(Debug, Clone, Copy, PartialEq, Eq)]
-    struct HasBeenUsed: u8 {
-        const INDUCTIVE_CYCLE = 1 << 0;
-        const COINDUCTIVE_CYCLE = 1 << 1;
-    }
-}
-
-#[derive(derivative::Derivative)]
-#[derivative(Debug(bound = ""))]
-struct StackEntry<I: Interner> {
-    input: CanonicalInput<I>,
-
-    available_depth: SolverLimit,
-
-    /// The maximum depth reached by this stack entry, only up-to date
-    /// for the top of the stack and lazily updated for the rest.
-    reached_depth: StackDepth,
-
-    /// Whether this entry is a non-root cycle participant.
-    ///
-    /// We must not move the result of non-root cycle participants to the
-    /// global cache. We store the highest stack depth of a head of a cycle
-    /// this goal is involved in. This necessary to soundly cache its
-    /// provisional result.
-    non_root_cycle_participant: Option<StackDepth>,
-
-    encountered_overflow: bool,
-
-    has_been_used: HasBeenUsed,
-
-    /// We put only the root goal of a coinductive cycle into the global cache.
-    ///
-    /// If we were to use that result when later trying to prove another cycle
-    /// participant, we can end up with unstable query results.
-    ///
-    /// See tests/ui/next-solver/coinduction/incompleteness-unstable-result.rs for
-    /// an example of where this is needed.
-    ///
-    /// There can  be multiple roots on the same stack, so we need to track
-    /// cycle participants per root:
-    /// ```plain
-    /// A :- B
-    /// B :- A, C
-    /// C :- D
-    /// D :- C
-    /// ```
-    nested_goals: HashSet<CanonicalInput<I>>,
-    /// Starts out as `None` and gets set when rerunning this
-    /// goal in case we encounter a cycle.
-    provisional_result: Option<QueryResult<I>>,
-}
-
-/// The provisional result for a goal which is not on the stack.
-#[derive(Debug)]
-struct DetachedEntry<I: Interner> {
-    /// The head of the smallest non-trivial cycle involving this entry.
-    ///
-    /// Given the following rules, when proving `A` the head for
-    /// the provisional entry of `C` would be `B`.
-    /// ```plain
-    /// A :- B
-    /// B :- C
-    /// C :- A + B + C
-    /// ```
-    head: StackDepth,
-    result: QueryResult<I>,
-}
+use rustc_type_ir::inherent::Predicate;
+use rustc_type_ir::search_graph::{self, CycleKind, UsageKind};
+use rustc_type_ir::solve::{CanonicalInput, Certainty, QueryResult};
+use rustc_type_ir::Interner;
+use std::marker::PhantomData;


Imports are still not cleaned up :(

compiler-errors · 2024-07-12T10:20:30Z

compiler/rustc_next_trait_solver/src/solve/search_graph.rs

-    head: StackDepth,
-    result: QueryResult<I>,
-}
+use rustc_type_ir::inherent::Predicate;


We should always just glob-import inherent::* as a convention.

fully move it into `rustc_type_ir` and make it independent of `Interner`.

compiler-errors · 2024-07-12T10:34:47Z

I've fixed the imports.

Ideally we'd enforce via lint at least that inherent::* is preferred over manually importing traits from that module, since almost certainly never want to import specific trait names from there, but that seems like a lot of effort. Seems easy to mess up with rust-analyzer's auto-imports unfortunately :(

Would prefer if we try to keep these imports clean in general 🤔 I've tried very hard to clean them up in the process of uplifting and I think it does a lot to help the code maintainability.

@bors r+

bors · 2024-07-12T10:34:49Z

📌 Commit 15f770b has been approved by compiler-errors

It is now in the queue for this repository.

…r-errors generalize search graph to enable fuzzing I do not believe it to be feasible to correctly implement the search graph without fuzzing. This PR enables this by requiring a fuzzer to only implement three new traits: - `Cx`: implemented by all `I: Interner` - `ProofTreeBuilder`: implemented by `struct ProofTreeBuilder<D>` for all `D: SolverDelegate` - `Delegate`: implemented for a new `struct SearchGraphDelegate<D>` for all `D: SolverDelegate` It also moves the evaluation cache implementation into `rustc_type_ir`, requiring `Interner` to provide methods to create and access arbitrary `WithDepNode<T>` and to provide mutable access to a given `GlobalCache`. It otherwise does not change the API surface for users of the shared library. This change should not impact behavior in any way. r? `@compiler-errors`

…iaskrgr Rollup of 9 pull requests Successful merges: - rust-lang#124980 (Generalize `fn allocator` for Rc/Arc.) - rust-lang#126639 (Add AMX target-features and `x86_amx_intrinsics` feature flag) - rust-lang#126827 (Use pidfd_spawn for faster process spawning when a PidFd is requested) - rust-lang#127153 (Initial implementation of anonymous_pipe API) - rust-lang#127433 (Stabilize const_cstr_from_ptr (CStr::from_ptr, CStr::count_bytes)) - rust-lang#127552 (remove unnecessary `git` usages) - rust-lang#127613 (Update dist-riscv64-linux to binutils 2.40) - rust-lang#127627 (generalize search graph to enable fuzzing) - rust-lang#127648 (Lower timeout of CI jobs to 4 hours) r? `@ghost` `@rustbot` modify labels: rollup

…iaskrgr Rollup of 8 pull requests Successful merges: - rust-lang#124980 (Generalize `fn allocator` for Rc/Arc.) - rust-lang#126639 (Add AMX target-features and `x86_amx_intrinsics` feature flag) - rust-lang#126827 (Use pidfd_spawn for faster process spawning when a PidFd is requested) - rust-lang#127433 (Stabilize const_cstr_from_ptr (CStr::from_ptr, CStr::count_bytes)) - rust-lang#127552 (remove unnecessary `git` usages) - rust-lang#127613 (Update dist-riscv64-linux to binutils 2.40) - rust-lang#127627 (generalize search graph to enable fuzzing) - rust-lang#127648 (Lower timeout of CI jobs to 4 hours) r? `@ghost` `@rustbot` modify labels: rollup

Rollup merge of rust-lang#127627 - lcnr:rustc_search_graph, r=compiler-errors generalize search graph to enable fuzzing I do not believe it to be feasible to correctly implement the search graph without fuzzing. This PR enables this by requiring a fuzzer to only implement three new traits: - `Cx`: implemented by all `I: Interner` - `ProofTreeBuilder`: implemented by `struct ProofTreeBuilder<D>` for all `D: SolverDelegate` - `Delegate`: implemented for a new `struct SearchGraphDelegate<D>` for all `D: SolverDelegate` It also moves the evaluation cache implementation into `rustc_type_ir`, requiring `Interner` to provide methods to create and access arbitrary `WithDepNode<T>` and to provide mutable access to a given `GlobalCache`. It otherwise does not change the API surface for users of the shared library. This change should not impact behavior in any way. r? ``@compiler-errors``

…t-lint, r=compiler-errors Add internal lint for detecting non-glob imports of `rustc_type_ir::inherent` rust-lang#127627 (comment) r? compiler-errors

…lint, r=<try> Add internal lint for detecting non-glob imports of `rustc_type_ir::inherent` rust-lang#127627 (comment) r? compiler-errors

…t-lint, r=compiler-errors Add internal lint for detecting non-glob imports of `rustc_type_ir::inherent` rust-lang#127627 (comment) r? compiler-errors

Rollup merge of rust-lang#127854 - fmease:glob-import-type_ir_inherent-lint, r=compiler-errors Add internal lint for detecting non-glob imports of `rustc_type_ir::inherent` rust-lang#127627 (comment) r? compiler-errors

Update Rust toolchain from nightly-2024-07-12 to nightly-2024-07-13 without any other source changes. This is an automatically generated pull request. If any of the CI checks fail, manual intervention is required. In such a case, review the changes at https://github.com/rust-lang/rust from rust-lang@5315cbe up to rust-lang@c6727fc. The log for this commit range is: rust-lang@c6727fc9b5 Auto merge of rust-lang#123351 - beetrees:x86-ret-snan-rust, r=nikic,workingjubilee rust-lang@62c068feea Auto merge of rust-lang#127636 - nnethercote:fix-Parser-look_ahead, r=oli-obk rust-lang@5d76a13bbe Auto merge of rust-lang#127653 - matthiaskrgr:rollup-72bqgvp, r=matthiaskrgr rust-lang@f11c2c8e95 Rollup merge of rust-lang#127648 - Kobzol:ci-lower-timeout, r=pietroalbini rust-lang@526da2366a Rollup merge of rust-lang#127627 - lcnr:rustc_search_graph, r=compiler-errors rust-lang@f5fa6fb602 Rollup merge of rust-lang#127613 - nikic:riscv-update, r=cuviper rust-lang@b4f002d2e5 Rollup merge of rust-lang#127552 - onur-ozkan:unnecessary-git-usage, r=Kobzol rust-lang@8ceb4e49ff Rollup merge of rust-lang#127433 - dtolnay:conststrlen, r=workingjubilee rust-lang@f9b3e8b387 Rollup merge of rust-lang#126827 - the8472:pidfd-spawn, r=workingjubilee rust-lang@18152d72a4 Rollup merge of rust-lang#126639 - sayantn:amx, r=Amanieu rust-lang@65ea92d4a1 Rollup merge of rust-lang#124980 - zachs18:rc-allocator, r=Amanieu rust-lang@05eac57ef6 Auto merge of rust-lang#127479 - Urgau:rustc-stable-hash, r=michaelwoerister rust-lang@15f770b143 enable fuzzing of `SearchGraph` rust-lang@cae9d480bf Adjust tests for x86 "Rust" ABI changes rust-lang@3f4b9dd463 Lower timeout of CI jobs to 4 hours rust-lang@7f1518bddd Add instability attribute on private const_strlen function rust-lang@b286722878 Auto merge of rust-lang#127635 - matthiaskrgr:rollup-foopajr, r=matthiaskrgr rust-lang@100f3fd133 Add a new special case to `Parser::look_ahead`. rust-lang@ebe1305b1e Remove the bogus special case from `Parser::look_ahead`. rust-lang@dad95578b0 Add unit tests for `Parser::look_ahead`. rust-lang@ec05c4ea3f Add the feature gate and target-features rust-lang@c2b7842555 Rollup merge of rust-lang#127625 - SkiFire13:revert-comment-deletion, r=workingjubilee rust-lang@ca576eae4e Rollup merge of rust-lang#127622 - compiler-errors:builtin-internal, r=lqd rust-lang@fe564c10ab Rollup merge of rust-lang#127607 - Zalathar:normalize-hint, r=wesleywiser rust-lang@83d1a1b252 Rollup merge of rust-lang#127596 - tesuji:help-unwrap-or, r=compiler-errors rust-lang@1e7ad4c3ed Rollup merge of rust-lang#127422 - greaka:master, r=workingjubilee rust-lang@58fe37f2c3 Rollup merge of rust-lang#127164 - Nadrieril:clean-lowering-loop, r=matthewjasper rust-lang@4a31a6c32a Auto merge of rust-lang#127382 - estebank:const-let, r=compiler-errors rust-lang@5e311f933d Auto merge of rust-lang#127614 - matthiaskrgr:rollup-8geziwi, r=matthiaskrgr rust-lang@a776e5f922 Add doc for deconstruct_option_or_result rust-lang@872d7b82e1 Add suggestion for `Option<&Vec<T>> -> Option<&[T]` rust-lang@d9170dc666 Add regression test for issue 127545 rust-lang@4df75140dd Fix aarch64 test rust-lang@cbe75486f7 Account for `let foo = expr`; to suggest `const foo: Ty = expr;` rust-lang@b56dc8ee90 Use verbose style when suggesting changing `const` with `let` rust-lang@d9021791eb Revert accidental comment deletion rust-lang@b77d3ef7c4 Mark builtin syntax as internal rust-lang@fa3ce50f0b Rollup merge of rust-lang#127605 - nikic:remove-extern-wasm, r=oli-obk rust-lang@d433f176ef Rollup merge of rust-lang#127601 - trevyn:issue-127600, r=compiler-errors rust-lang@47ab86653e Rollup merge of rust-lang#127599 - tgross35:lazy_cell_consume-rename, r=workingjubilee rust-lang@a10b4d1463 Rollup merge of rust-lang#127598 - weiznich:diagnostic_do_not_recommend_also_skips_help, r=compiler-errors rust-lang@73c500b3a7 Rollup merge of rust-lang#127591 - compiler-errors:label-after-primary, r=lcnr rust-lang@380c78741e Rollup merge of rust-lang#127588 - uweigand:s390x-f16-doctests, r=tgross35 rust-lang@6fd955549a Rollup merge of rust-lang#127572 - tbu-:pr_debug_event_nonpacked, r=jhpratt rust-lang@8de487fdbd Rollup merge of rust-lang#124599 - estebank:issue-41708, r=wesleywiser rust-lang@55256c5a18 Update dist-riscv64-linux to binutils 2.40 rust-lang@977439d9b8 Use uplifted `rustc-stable-hash` crate in `rustc_data_structures` rust-lang@f56b2074c6 solve -> solve/mod rust-lang@08a2992d6b compiletest: Better error message for bad `normalize-*` headers rust-lang@8a50bcbdce Remove extern "wasm" ABI rust-lang@a01f49e7f3 check is_ident before parse_ident rust-lang@ab56fe2053 Rename `lazy_cell_consume` to `lazy_cell_into_inner` rust-lang@27d5db166e Allows `#[diagnostic::do_not_recommend]` to supress trait impls in suggestions as well rust-lang@12ae282987 Fix diagnostic and add a test for it rust-lang@df72e478b0 Make sure that labels are defined after the primary span in diagnostics rust-lang@0065763950 core: Limit remaining f16 doctests to x86_64 linux rust-lang@45ad522e87 Don't mark `DEBUG_EVENT` struct as `repr(packed)` rust-lang@0134bd2e67 remove unnecessary `git` usages rust-lang@42772e98e0 Address review comments rust-lang@3e030b38ef Return the `otherwise_block` instead of passing it as argument rust-lang@fc40247c6b Factor out the "process remaining candidates" cases rust-lang@8a222ffd6b Don't try to save an extra block rust-lang@c5062f7318 Move or-pattern expansion inside the main part of the algorithm rust-lang@bff4d213fa Factor out the special handling of or-patterns rust-lang@5bf50e66f9 Move a function rust-lang@53d3e6217b Stabilize const_cstr_from_ptr (CStr::from_ptr, CStr::count_bytes) rust-lang@585ca16e0b as_simd: fix comment to be in line with 507583a (rust-lang#121201) rust-lang@0f643c449a Ensure tests don't fail on i586 in CI rust-lang@ec0c755704 Check that we get somewhat sane PIDs when spawning with pidfds rust-lang@3e4e31b7bf more fine-grained feature-detection for pidfd spawning rust-lang@0ce361938e document safety properties of the internal Process::new constructor rust-lang@6687a3f7da use pidfd_spawn for faster process creation when pidfds are requested rust-lang@5c46acac04 document the cvt methods rust-lang@0e1c832dbd Update `platform-support.md` to reflect improvements in returning floats on 32-bit x86 rust-lang@952becc0bd Ensure floats are returned losslessly by the Rust ABI on 32-bit x86 rust-lang@a1ad6346d6 Add fn allocator method to rc/sync::Weak. Relax Rc<T>/Arc<T>::allocator to allow unsized T. rust-lang@2df4f7dd8c Suggest borrowing on fn argument that is `impl AsRef` Co-authored-by: celinval <35149715+celinval@users.noreply.github.com>

solve -> solve/mod

f56b207

rustbot assigned compiler-errors Jul 11, 2024

lcnr force-pushed the rustc_search_graph branch from 174c513 to b9cf0ab Compare July 11, 2024 21:24

compiler-errors reviewed Jul 11, 2024

View reviewed changes

compiler/rustc_next_trait_solver/src/solve/search_graph.rs Outdated Show resolved Hide resolved

lcnr force-pushed the rustc_search_graph branch from b9cf0ab to cc8ce8d Compare July 11, 2024 21:49

compiler-errors reviewed Jul 12, 2024

View reviewed changes

lcnr force-pushed the rustc_search_graph branch 2 times, most recently from 809acd2 to 208d709 Compare July 12, 2024 10:04

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 12, 2024

compiler-errors reviewed Jul 12, 2024

View reviewed changes

enable fuzzing of SearchGraph

15f770b

fully move it into `rustc_type_ir` and make it independent of `Interner`.

compiler-errors force-pushed the rustc_search_graph branch from 208d709 to 15f770b Compare July 12, 2024 10:32

matthiaskrgr mentioned this pull request Jul 12, 2024

Rollup of 9 pull requests #127650

Closed

matthiaskrgr mentioned this pull request Jul 12, 2024

Rollup of 8 pull requests #127653

Merged

bors merged commit 526da23 into rust-lang:master Jul 12, 2024
6 checks passed

rustbot added this to the 1.81.0 milestone Jul 12, 2024

lcnr deleted the rustc_search_graph branch July 15, 2024 07:59

fmease mentioned this pull request Jul 17, 2024

Add internal lint for detecting non-glob imports of rustc_type_ir::inherent #127854

Merged

lcnr mentioned this pull request Jul 23, 2024

refactor search graph even more! #128115

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

generalize search graph to enable fuzzing #127627

generalize search graph to enable fuzzing #127627

lcnr commented Jul 11, 2024 •

edited

Loading

compiler-errors left a comment

lcnr commented Jul 12, 2024

bors commented Jul 12, 2024

compiler-errors Jul 12, 2024 •

edited

Loading

compiler-errors Jul 12, 2024

compiler-errors commented Jul 12, 2024 •

edited

Loading

bors commented Jul 12, 2024

generalize search graph to enable fuzzing #127627

generalize search graph to enable fuzzing #127627

Conversation

lcnr commented Jul 11, 2024 • edited Loading

compiler-errors left a comment

Choose a reason for hiding this comment

lcnr commented Jul 12, 2024

bors commented Jul 12, 2024

compiler-errors Jul 12, 2024 • edited Loading

Choose a reason for hiding this comment

compiler-errors Jul 12, 2024

Choose a reason for hiding this comment

compiler-errors commented Jul 12, 2024 • edited Loading

bors commented Jul 12, 2024

lcnr commented Jul 11, 2024 •

edited

Loading

compiler-errors Jul 12, 2024 •

edited

Loading

compiler-errors commented Jul 12, 2024 •

edited

Loading