A more generic interface for dataflow analysis #64566

ecstatic-morse · 2019-09-18T00:38:04Z

#64470 requires a transfer function that is slightly more complex than the typical gen/kill one. Namely, it must copy state bits between locals when assignments occur (see #62547 for an attempt to make this fit into the existing framework). This PR contains a dataflow interface that allows for arbitrary transfer functions. The trade-off is efficiency: we can no longer coalesce transfer functions for blocks and must visit each statement individually while iterating to fixpoint.

Another issue is that poorly behaved transfer functions can result in an analysis that fails to converge. gen/kill sets do not have this problem. I believe that, in order to guarantee convergence, flipping a bit from false to true in the entry set cannot cause an output bit to go from true to false (negate all preceding booleans when true is the bottom value). Perhaps someone with a more formal background can confirm and we can add a section to the docs?

This approach is not maximally generic: it still requires that the lattice used for analysis is the powerset of values of Analysis::Idx for the mir::Body of interest. This can be done at a later date. Also, this is the bare minimum to get #64470 working. I've not adapted the existing debug framework to work with the new analysis, so there are no rustc_peek tests either. I'm planning to do this after #64470 is merged.

Finally, my ultimate plan is to make the existing, gen/kill-based BitDenotation a special case of generic::Analysis. Currently they share a ton of code. I should be able to do this without changing any implementers of BitDenotation. Something like:

struct GenKillAnalysis<A: BitDenotation> {
    trans_for_block: IndexVec<BasicBlock, GenKillSet<A::Idx>>,
    analysis: A,
}

impl<A> generic::Analysis for GenKillAnalysis<A> {
    // specializations of `apply_{partial,whole}_block_effect`...
}

r? @pnkfelix

Centril · 2019-09-18T00:43:03Z

src/librustc_mir/dataflow/mod.rs

btw, as drive-by improvement potential, it would be great if this module had a brief explanation of what dataflow is and if it pointed to some good beginner->less-beginner resources for understanding dataflow.

@ecstatic-morse Since @oli-obk r+ed, maybe this can be done as a follow up?

Sounds good. I tentatively plan to do this in the rustc guide, then link to it from dataflow/mod.rs.

There's currently a "to be written" entry in the appendix

Sounds great. 👍

This can be removed once dataflow-based const validation is merged.

oli-obk

I like it. This is much clearer (to me) than the existing dataflow framework.

src/librustc_mir/dataflow/generic.rs

oli-obk · 2019-09-18T15:48:02Z

@bors r+

bors · 2019-09-18T15:48:03Z

📌 Commit a7f5252 has been approved by oli-obk

ecstatic-morse · 2019-09-18T16:22:52Z

@oli-obk Can you r- and pick up the latest commit. There was a bug in the logic of seek_after_assume_call_return.

ecstatic-morse · 2019-09-18T16:39:34Z

@bors r-

(worth a shot)

oli-obk · 2019-09-18T16:39:54Z

@bors r+

bors · 2019-09-18T16:39:55Z

📌 Commit b4e94d9 has been approved by oli-obk

@pnkfelix

…oli-obk A more generic interface for dataflow analysis rust-lang#64470 requires a transfer function that is slightly more complex than the typical `gen`/`kill` one. Namely, it must copy state bits between locals when assignments occur (see rust-lang#62547 for an attempt to make this fit into the existing framework). This PR contains a dataflow interface that allows for arbitrary transfer functions. The trade-off is efficiency: we can no longer coalesce transfer functions for blocks and must visit each statement individually while iterating to fixpoint. Another issue is that poorly behaved transfer functions can result in an analysis that fails to converge. `gen`/`kill` sets do not have this problem. I believe that, in order to guarantee convergence, flipping a bit from `false` to `true` in the entry set cannot cause an output bit to go from `true` to `false` (negate all preceding booleans when `true` is the bottom value). Perhaps someone with a more formal background can confirm and we can add a section to the docs? This approach is not maximally generic: it still requires that the lattice used for analysis is the powerset of values of `Analysis::Idx` for the `mir::Body` of interest. This can be done at a later date. Also, this is the bare minimum to get rust-lang#64470 working. I've not adapted the existing debug framework to work with the new analysis, so there are no `rustc_peek` tests either. I'm planning to do this after rust-lang#64470 is merged. Finally, my ultimate plan is to make the existing, `gen`/`kill`-based `BitDenotation` a special case of `generic::Analysis`. Currently they share a ton of code. I should be able to do this without changing any implementers of `BitDenotation`. Something like: ```rust struct GenKillAnalysis<A: BitDenotation> { trans_for_block: IndexVec<BasicBlock, GenKillSet<A::Idx>>, analysis: A, } impl<A> generic::Analysis for GenKillAnalysis<A> { // specializations of `apply_{partial,whole}_block_effect`... } ``` r? @pnkfelix

@pnkfelix

…oli-obk A more generic interface for dataflow analysis rust-lang#64470 requires a transfer function that is slightly more complex than the typical `gen`/`kill` one. Namely, it must copy state bits between locals when assignments occur (see rust-lang#62547 for an attempt to make this fit into the existing framework). This PR contains a dataflow interface that allows for arbitrary transfer functions. The trade-off is efficiency: we can no longer coalesce transfer functions for blocks and must visit each statement individually while iterating to fixpoint. Another issue is that poorly behaved transfer functions can result in an analysis that fails to converge. `gen`/`kill` sets do not have this problem. I believe that, in order to guarantee convergence, flipping a bit from `false` to `true` in the entry set cannot cause an output bit to go from `true` to `false` (negate all preceding booleans when `true` is the bottom value). Perhaps someone with a more formal background can confirm and we can add a section to the docs? This approach is not maximally generic: it still requires that the lattice used for analysis is the powerset of values of `Analysis::Idx` for the `mir::Body` of interest. This can be done at a later date. Also, this is the bare minimum to get rust-lang#64470 working. I've not adapted the existing debug framework to work with the new analysis, so there are no `rustc_peek` tests either. I'm planning to do this after rust-lang#64470 is merged. Finally, my ultimate plan is to make the existing, `gen`/`kill`-based `BitDenotation` a special case of `generic::Analysis`. Currently they share a ton of code. I should be able to do this without changing any implementers of `BitDenotation`. Something like: ```rust struct GenKillAnalysis<A: BitDenotation> { trans_for_block: IndexVec<BasicBlock, GenKillSet<A::Idx>>, analysis: A, } impl<A> generic::Analysis for GenKillAnalysis<A> { // specializations of `apply_{partial,whole}_block_effect`... } ``` r? @pnkfelix

@pnkfelix

…oli-obk A more generic interface for dataflow analysis rust-lang#64470 requires a transfer function that is slightly more complex than the typical `gen`/`kill` one. Namely, it must copy state bits between locals when assignments occur (see rust-lang#62547 for an attempt to make this fit into the existing framework). This PR contains a dataflow interface that allows for arbitrary transfer functions. The trade-off is efficiency: we can no longer coalesce transfer functions for blocks and must visit each statement individually while iterating to fixpoint. Another issue is that poorly behaved transfer functions can result in an analysis that fails to converge. `gen`/`kill` sets do not have this problem. I believe that, in order to guarantee convergence, flipping a bit from `false` to `true` in the entry set cannot cause an output bit to go from `true` to `false` (negate all preceding booleans when `true` is the bottom value). Perhaps someone with a more formal background can confirm and we can add a section to the docs? This approach is not maximally generic: it still requires that the lattice used for analysis is the powerset of values of `Analysis::Idx` for the `mir::Body` of interest. This can be done at a later date. Also, this is the bare minimum to get rust-lang#64470 working. I've not adapted the existing debug framework to work with the new analysis, so there are no `rustc_peek` tests either. I'm planning to do this after rust-lang#64470 is merged. Finally, my ultimate plan is to make the existing, `gen`/`kill`-based `BitDenotation` a special case of `generic::Analysis`. Currently they share a ton of code. I should be able to do this without changing any implementers of `BitDenotation`. Something like: ```rust struct GenKillAnalysis<A: BitDenotation> { trans_for_block: IndexVec<BasicBlock, GenKillSet<A::Idx>>, analysis: A, } impl<A> generic::Analysis for GenKillAnalysis<A> { // specializations of `apply_{partial,whole}_block_effect`... } ``` r? @pnkfelix

@ghost

Rollup of 5 pull requests Successful merges: - #63630 (Update installed compiler dependencies) - #64536 (Update Cargo) - #64554 (Polonius: more `ui` test suite fixes) - #64566 (A more generic interface for dataflow analysis) - #64591 (Fix a minor grammar nit, update UI tests) Failed merges: r? @ghost

…phviz, r=oli-obk Graphviz debug output for generic dataflow analysis A follow up to rust-lang#64566. This outputs graphviz diagrams in the generic dataflow engine when `#[rustc_mir(borrowck_graphviz_postflow="suffix.dot")]` is set on an item. This code is based on [`dataflow/graphviz.rs`](https://github.com/rust-lang/rust/blob/master/src/librustc_mir/dataflow/graphviz.rs), but displays different information since there are no "gen"/"kill" sets and transfer functions cannot be coalesced in the generic analysis. As a result, we show the dataflow state at the start and end of each block, as well as the changes resulting from each statement. I also render the state bitset in full (`{_1,_2}`) instead of hex-encoded as the current renderer does (`06`).

Add generic dataflow impl

ceb72e4

rust-highfive assigned pnkfelix Sep 18, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 18, 2019

ecstatic-morse mentioned this pull request Sep 18, 2019

Implement dataflow-based const validation #64470

Merged

Centril reviewed Sep 18, 2019

View reviewed changes

ecstatic-morse added 3 commits September 17, 2019 17:52

Document new dataflow analysis

5496a1f

Temporarily add #[allow(unused)] for CI

2886037

This can be removed once dataflow-based const validation is merged.

Add ignore reason to placate tidy

d583fef

ecstatic-morse force-pushed the generic-dataflow branch from ec4283a to d583fef Compare September 18, 2019 00:53

oli-obk reviewed Sep 18, 2019

View reviewed changes

ecstatic-morse added 4 commits September 18, 2019 07:32

Fix Analysis example

cbaaf05

Use an associated const for name

a526e4f

Fix typo

19bf0f4

Publish rustc_mir::dataflow and remove #[allow(unused)]

08aff1a

ecstatic-morse changed the title ~~Add a more generic interface for dataflow analysis~~ A more generic interface for dataflow analysis Sep 18, 2019

Add summary of the current state and future plans

a7f5252

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Sep 18, 2019

Fix bug where is_call_return_effect_applied was never set

b4e94d9

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Sep 18, 2019

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 18, 2019

Centril mentioned this pull request Sep 18, 2019

Rollup of 5 pull requests #64594

Closed

Centril mentioned this pull request Sep 19, 2019

Rollup of 5 pull requests #64597

Closed

Centril mentioned this pull request Sep 19, 2019

Rollup of 5 pull requests #64598

Merged

bors merged commit b4e94d9 into rust-lang:master Sep 19, 2019

ecstatic-morse mentioned this pull request Sep 27, 2019

Graphviz debug output for generic dataflow analysis #64828

Merged

ecstatic-morse mentioned this pull request Oct 16, 2019

A unified framework for dataflow analysis rust-lang/compiler-team#202

Closed

ecstatic-morse mentioned this pull request May 1, 2020

Use existing framework for backward dataflow analyses #71006

Merged

ecstatic-morse deleted the generic-dataflow branch October 6, 2020 01:42

Uh oh!

A more generic interface for dataflow analysis #64566

A more generic interface for dataflow analysis #64566

Uh oh!

Conversation

ecstatic-morse commented Sep 18, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Centril Sep 18, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Centril Sep 18, 2019

Choose a reason for hiding this comment

Uh oh!

ecstatic-morse Sep 18, 2019

Choose a reason for hiding this comment

Uh oh!

ecstatic-morse Sep 18, 2019

Choose a reason for hiding this comment

Uh oh!

Centril Sep 18, 2019

Choose a reason for hiding this comment

Uh oh!

oli-obk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

oli-obk commented Sep 18, 2019

Uh oh!

bors commented Sep 18, 2019

Uh oh!

ecstatic-morse commented Sep 18, 2019

Uh oh!

ecstatic-morse commented Sep 18, 2019

Uh oh!

oli-obk commented Sep 18, 2019

Uh oh!

bors commented Sep 18, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

ecstatic-morse commented Sep 18, 2019 •

edited

Loading

Centril Sep 18, 2019 •

edited

Loading