Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store HIR attributes in a side table #79519

Merged
merged 38 commits into from
Mar 10, 2021
Merged

Store HIR attributes in a side table #79519

merged 38 commits into from
Mar 10, 2021

Conversation

cjgillot
Copy link
Contributor

@cjgillot cjgillot commented Nov 28, 2020

Same idea as #72015 but for attributes.
The objective is to reduce incr-comp invalidations due to modified attributes.
Notably, those due to modified doc comments.

Implementation:

  • collect attributes during AST->HIR lowering, in LocalDefId -> ItemLocalId -> &[Attributes] nested tables;
  • access the attributes through a hir_owner_attrs query;
  • local refactorings to use this access;
  • remove attrs from HIR data structures one-by-one.

Change in behaviour:

  • the HIR visitor traverses all attributes at once instead of parent-by-parent;
  • attribute arrays are sometimes duplicated: for statements and variant constructors;
  • as a consequence, attributes are marked as used after unused-attribute lint emission to avoid duplicate lints.

Current bug: the lint level is not correctly applied in std::backtrace_rs, triggering an unused attribute warning on #![no_std]. I welcome suggestions.

@rust-highfive
Copy link
Collaborator

r? @matthewjasper

(rust-highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Nov 28, 2020
@cjgillot cjgillot force-pushed the noattr branch 2 times, most recently from b80b682 to 3b47e6a Compare December 5, 2020 20:41
@cjgillot cjgillot marked this pull request as ready for review December 5, 2020 23:11
@jonas-schievink
Copy link
Contributor

@bors try @rust-timer queue

@rust-timer
Copy link
Collaborator

Awaiting bors try build completion

@bors
Copy link
Contributor

bors commented Dec 5, 2020

⌛ Trying commit 3b47e6a134e12e60f2357b2a31f74315a1cacdf4 with merge b7675636fff6dc80b3b4d552a8c497ce56522491...

@bors
Copy link
Contributor

bors commented Dec 6, 2020

☀️ Try build successful - checks-actions
Build commit: b7675636fff6dc80b3b4d552a8c497ce56522491 (b7675636fff6dc80b3b4d552a8c497ce56522491)

@rust-timer
Copy link
Collaborator

Queued b7675636fff6dc80b3b4d552a8c497ce56522491 with parent e792288, future comparison URL.

@rust-timer
Copy link
Collaborator

Finished benchmarking try commit (b7675636fff6dc80b3b4d552a8c497ce56522491): comparison url.

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. Please note that if the perf results are neutral, you should likely undo the rollup=never given below by specifying rollup- to bors.

Importantly, though, if the results of this run are non-neutral do not roll this PR up -- it will mask other regressions or improvements in the roll up.

@bors rollup=never
@rustbot modify labels: +S-waiting-on-review -S-waiting-on-perf

@bjorn3
Copy link
Member

bjorn3 commented Dec 6, 2020

A few crates got huge incr patched improvements of up to 65%. Some others got a regression of up to ~7%.

@bors

This comment has been minimized.

@bors

This comment has been minimized.

@cjgillot
Copy link
Contributor Author

cjgillot commented Dec 8, 2020

This PR is prone to bitrot. I wait for an initial review before rebasing again.

@cjgillot
Copy link
Contributor Author

@rustbot modify labels: +S-waiting-on-review -S-waiting-on-author

@bors

This comment has been minimized.

@bors

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@bors

This comment has been minimized.

@cjgillot
Copy link
Contributor Author

cjgillot commented Mar 9, 2021

@wesleywiser : just rebased.

@wesleywiser
Copy link
Member

@bors r+ rollup=never

Mixed perf results.

@bors
Copy link
Contributor

bors commented Mar 9, 2021

📌 Commit 77c0f21 has been approved by wesleywiser

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 9, 2021
@bors
Copy link
Contributor

bors commented Mar 9, 2021

⌛ Testing commit 77c0f21 with merge 0eeeb96aa48c8f2d187b92bee2b37525bd5e9134...

@bors
Copy link
Contributor

bors commented Mar 10, 2021

💥 Test timed out

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Mar 10, 2021
@rust-log-analyzer
Copy link
Collaborator

A job failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

@cjgillot
Copy link
Contributor Author

@bors retry

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 10, 2021
@bors
Copy link
Contributor

bors commented Mar 10, 2021

⌛ Testing commit 77c0f21 with merge dff1edf...

@bors
Copy link
Contributor

bors commented Mar 10, 2021

☀️ Test successful - checks-actions
Approved by: wesleywiser
Pushing dff1edf to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Mar 10, 2021
@bors bors merged commit dff1edf into rust-lang:master Mar 10, 2021
@rustbot rustbot added this to the 1.52.0 milestone Mar 10, 2021
@cjgillot cjgillot deleted the noattr branch March 10, 2021 19:33
flip1995 pushed a commit to flip1995/rust that referenced this pull request Mar 11, 2021
Store HIR attributes in a side table

Same idea as rust-lang#72015 but for attributes.
The objective is to reduce incr-comp invalidations due to modified attributes.
Notably, those due to modified doc comments.

Implementation:
- collect attributes during AST->HIR lowering, in `LocalDefId -> ItemLocalId -> &[Attributes]` nested tables;
- access the attributes through a `hir_owner_attrs` query;
- local refactorings to use this access;
- remove `attrs` from HIR data structures one-by-one.

Change in behaviour:
- the HIR visitor traverses all attributes at once instead of parent-by-parent;
- attribute arrays are sometimes duplicated: for statements and variant constructors;
- as a consequence, attributes are marked as used after unused-attribute lint emission to avoid duplicate lints.

~~Current bug: the lint level is not correctly applied in `std::backtrace_rs`, triggering an unused attribute warning on `#![no_std]`. I welcome suggestions.~~
@cjgillot cjgillot mentioned this pull request Mar 16, 2021
6 tasks
wip-sync pushed a commit to NetBSD/pkgsrc-wip that referenced this pull request May 9, 2021
Package changes:
 * bump bootstraps to 1.51.0.
 * adjust patches and cargo checksums as required
 * 1.51 failed to build natively on 32-bit armv7, there is hope
   that this is fixed with 1.52.  (1.51 can be built with netbsd32
   emulation on a aarch64 system).

Upsteream changes:

Version 1.52.0 (2021-05-06)
============================

Language
--------
- [Added the `unsafe_op_in_unsafe_fn` lint, which checks whether the unsafe
  code in an `unsafe fn` is wrapped in a `unsafe` block.][79208] This lint
  is allowed by default, and may become a warning or hard error in a
  future edition.
- [You can now cast mutable references to arrays to a pointer of the same
  type as the element.][81479]

Compiler
--------
- [Upgraded the default LLVM to LLVM 12.][81451]

Added tier 3\* support for the following targets.

- [`s390x-unknown-linux-musl`][82166]
- [`riscv32gc-unknown-linux-musl` & `riscv64gc-unknown-linux-musl`][82202]
- [`powerpc-unknown-openbsd`][82733]

\* Refer to Rust's [platform support page][platform-support-doc] for more
information on Rust's tiered platform support.

Libraries
---------
- [`OsString` now implements `Extend` and `FromIterator`.][82121]
- [`cmp::Reverse` now has `#[repr(transparent)]` representation.][81879]
- [`Arc<impl Error>` now implements `error::Error`.][80553]
- [All integer division and remainder operations are now `const`.][80962]

Stabilised APIs
-------------
- [`Arguments::as_str`]
- [`char::MAX`]
- [`char::REPLACEMENT_CHARACTER`]
- [`char::UNICODE_VERSION`]
- [`char::decode_utf16`]
- [`char::from_digit`]
- [`char::from_u32_unchecked`]
- [`char::from_u32`]
- [`slice::partition_point`]
- [`str::rsplit_once`]
- [`str::split_once`]

The following previously stable APIs are now `const`.

- [`char::len_utf8`]
- [`char::len_utf16`]
- [`char::to_ascii_uppercase`]
- [`char::to_ascii_lowercase`]
- [`char::eq_ignore_ascii_case`]
- [`u8::to_ascii_uppercase`]
- [`u8::to_ascii_lowercase`]
- [`u8::eq_ignore_ascii_case`]

Rustdoc
-------
- [Rustdoc lints are now treated as a tool lint, meaning that lints are
  now prefixed with `rustdoc::`
  (e.g. `#[warn(rustdoc::non_autolinks)]`).][80527]

  Using the old style is still allowed, and will become a warning in
  a future release.
- [Rustdoc now supports argument files.][82261]
- [Rustdoc now generates smart punctuation for documentation.][79423]
- [You can now use "task lists" in Rustdoc Markdown.][81766] E.g.
  ```markdown
  - [x] Complete
  - [ ] Todo
  ```

Misc
----
- [You can now pass multiple filters to tests.][81356] E.g.
  `cargo test -- foo bar` will run all tests that match `foo` and `bar`.
- [Rustup now distributes PDB symbols for the `std` library on Windows,
  allowing you to see `std` symbols when debugging.][82218]

Internal Only
-------------
These changes provide no direct user facing benefits, but represent significant
improvements to the internals and overall performance of rustc and
related tools.

- [Check the result cache before the DepGraph when ensuring queries][81855]
- [Try fast_reject::simplify_type in coherence before doing full check][81744]
- [Only store a LocalDefId in some HIR nodes][81611]
- [Store HIR attributes in a side table][79519]

Compatibility Notes
-------------------
- [Cargo build scripts are now forbidden from setting `RUSTC_BOOTSTRAP`.]
  [cargo/9181]
- [Removed support for the `x86_64-rumprun-netbsd` target.][82594]
- [Deprecated the `x86_64-sun-solaris` target in favor of `x86_64-pc-solaris`.]
  [82216]
- [Rustdoc now only accepts `,`, ` `, and `\t` as delimiters for specifying
  languages in code blocks.][78429]
- [Rustc now catches more cases of `pub_use_of_private_extern_crate`][80763]
- [Changes in how proc macros handle whitespace may lead to panics when used
  with older `proc-macro-hack` versions. A `cargo update` should be sufficient
  to fix this in all cases.][84136]

[84136]: rust-lang/rust#84136
[80763]: rust-lang/rust#80763
[82166]: rust-lang/rust#82166
[82121]: rust-lang/rust#82121
[81879]: rust-lang/rust#81879
[82261]: rust-lang/rust#82261
[82218]: rust-lang/rust#82218
[82216]: rust-lang/rust#82216
[82202]: rust-lang/rust#82202
[81855]: rust-lang/rust#81855
[81766]: rust-lang/rust#81766
[81744]: rust-lang/rust#81744
[81611]: rust-lang/rust#81611
[81479]: rust-lang/rust#81479
[81451]: rust-lang/rust#81451
[81356]: rust-lang/rust#81356
[80962]: rust-lang/rust#80962
[80553]: rust-lang/rust#80553
[80527]: rust-lang/rust#80527
[79519]: rust-lang/rust#79519
[79423]: rust-lang/rust#79423
[79208]: rust-lang/rust#79208
[78429]: rust-lang/rust#78429
[82733]: rust-lang/rust#82733
[82594]: rust-lang/rust#82594
[cargo/9181]: rust-lang/cargo#9181
[`char::MAX`]: https://doc.rust-lang.org/std/primitive.char.html#associatedconstant.MAX
[`char::REPLACEMENT_CHARACTER`]: https://doc.rust-lang.org/std/primitive.char.html#associatedconstant.REPLACEMENT_CHARACTER
[`char::UNICODE_VERSION`]: https://doc.rust-lang.org/std/primitive.char.html#associatedconstant.UNICODE_VERSION
[`char::decode_utf16`]: https://doc.rust-lang.org/std/primitive.char.html#method.decode_utf16
[`char::from_u32`]: https://doc.rust-lang.org/std/primitive.char.html#method.from_u32
[`char::from_u32_unchecked`]: https://doc.rust-lang.org/std/primitive.char.html#method.from_u32_unchecked
[`char::from_digit`]: https://doc.rust-lang.org/std/primitive.char.html#method.from_digit
[`Peekable::next_if`]: https://doc.rust-lang.org/stable/std/iter/struct.Peekable.html#method.next_if
[`Peekable::next_if_eq`]: https://doc.rust-lang.org/stable/std/iter/struct.Peekable.html#method.next_if_eq
[`Arguments::as_str`]: https://doc.rust-lang.org/stable/std/fmt/struct.Arguments.html#method.as_str
[`str::split_once`]: https://doc.rust-lang.org/stable/std/primitive.str.html#method.split_once
[`str::rsplit_once`]: https://doc.rust-lang.org/stable/std/primitive.str.html#method.rsplit_once
[`slice::partition_point`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.partition_point
[`char::len_utf8`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.len_utf8
[`char::len_utf16`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.len_utf16
[`char::to_ascii_uppercase`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.to_ascii_uppercase
[`char::to_ascii_lowercase`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.to_ascii_lowercase
[`char::eq_ignore_ascii_case`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.eq_ignore_ascii_case
[`u8::to_ascii_uppercase`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.to_ascii_uppercase
[`u8::to_ascii_lowercase`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.to_ascii_lowercase
[`u8::eq_ignore_ascii_case`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.eq_ignore_ascii_case
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request May 31, 2021
Pkgsrc changes:
 * Bump bootstrap kit version to 1.51.0.
 * Adjust patches as needed.
 * Update checksum adjustments.
 * Fix syntax error in commands adjusting libserde_derive for Darwin

Upstream changes:

Version 1.52.1 (2021-05-10)
============================

This release disables incremental compilation, unless the user has explicitly
opted in via the newly added RUSTC_FORCE_INCREMENTAL=1 environment variable.

This is due to the widespread, and frequently occuring, breakage encountered by
Rust users due to newly enabled incremental verification in 1.52.0. Notably,
Rust users **should** upgrade to 1.52.0 or 1.52.1: the bugs that are detected by
newly added incremental verification are still present in past stable versions,
and are not yet fixed on any channel. These bugs can lead to miscompilation of
Rust binaries.

These problems only affect incremental builds, so release builds with Cargo
should not be affected unless the user has explicitly opted into incremental.
Debug and check builds are affected.

See [84970] for more details.

[84970]: rust-lang/rust#84970

Version 1.52.0 (2021-05-06)
============================

Language
--------
- [Added the `unsafe_op_in_unsafe_fn` lint, which checks whether
  the unsafe code in an `unsafe fn` is wrapped in a `unsafe`
  block.][79208] This lint is allowed by default, and may become
  a warning or hard error in a future edition.

- [You can now cast mutable references to arrays to a pointer of
  the same type as the element.][81479]

Compiler
--------
- [Upgraded the default LLVM to LLVM 12.][81451]

Added tier 3\* support for the following targets.

- [`s390x-unknown-linux-musl`][82166]
- [`riscv32gc-unknown-linux-musl` & `riscv64gc-unknown-linux-musl`][82202]
- [`powerpc-unknown-openbsd`][82733]

\* Refer to Rust's [platform support page][platform-support-doc] for more
information on Rust's tiered platform support.

Libraries
---------
- [`OsString` now implements `Extend` and `FromIterator`.][82121]
- [`cmp::Reverse` now has `#[repr(transparent)]` representation.][81879]
- [`Arc<impl Error>` now implements `error::Error`.][80553]
- [All integer division and remainder operations are now `const`.][80962]

Stabilised APIs
-------------
- [`Arguments::as_str`]
- [`char::MAX`]
- [`char::REPLACEMENT_CHARACTER`]
- [`char::UNICODE_VERSION`]
- [`char::decode_utf16`]
- [`char::from_digit`]
- [`char::from_u32_unchecked`]
- [`char::from_u32`]
- [`slice::partition_point`]
- [`str::rsplit_once`]
- [`str::split_once`]

The following previously stable APIs are now `const`.

- [`char::len_utf8`]
- [`char::len_utf16`]
- [`char::to_ascii_uppercase`]
- [`char::to_ascii_lowercase`]
- [`char::eq_ignore_ascii_case`]
- [`u8::to_ascii_uppercase`]
- [`u8::to_ascii_lowercase`]
- [`u8::eq_ignore_ascii_case`]

Rustdoc
-------
- [Rustdoc lints are now treated as a tool lint, meaning that
  lints are now prefixed with `rustdoc::` (e.g.
  `#[warn(rustdoc::non_autolinks)]`).][80527] Using the old style
  is still allowed, and will become a warning in a future release.
- [Rustdoc now supports argument files.][82261]
- [Rustdoc now generates smart punctuation for documentation.][79423]
- [You can now use "task lists" in Rustdoc Markdown.][81766] E.g.
  ```markdown
  - [x] Complete
  - [ ] Todo
  ```

Misc
----
- [You can now pass multiple filters to tests.][81356] E.g.
  `cargo test -- foo bar` will run all tests that match `foo` and `bar`.
- [Rustup now distributes PDB symbols for the `std` library on Windows,
  allowing you to see `std` symbols when debugging.][82218]

Internal Only
-------------
These changes provide no direct user facing benefits, but represent significant
improvements to the internals and overall performance of rustc and
related tools.

- [Check the result cache before the DepGraph when ensuring queries][81855]
- [Try fast_reject::simplify_type in coherence before doing full check][81744]
- [Only store a LocalDefId in some HIR nodes][81611]
- [Store HIR attributes in a side table][79519]

Compatibility Notes
-------------------
- [Cargo build scripts are now forbidden from setting
  `RUSTC_BOOTSTRAP`.][cargo/9181]
- [Removed support for the `x86_64-rumprun-netbsd` target.][82594]
- [Deprecated the `x86_64-sun-solaris` target in favor of
  `x86_64-pc-solaris`.][82216]
- [Rustdoc now only accepts `,`, ` `, and `\t` as delimiters for specifying
  languages in code blocks.][78429]
- [Rustc now catches more cases of `pub_use_of_private_extern_crate`][80763]
- [Changes in how proc macros handle whitespace may lead to panics
  when used with older `proc-macro-hack` versions. A `cargo update` should
  be sufficient to fix this in all cases.][84136]

[84136]: rust-lang/rust#84136
[80763]: rust-lang/rust#80763
[82166]: rust-lang/rust#82166
[82121]: rust-lang/rust#82121
[81879]: rust-lang/rust#81879
[82261]: rust-lang/rust#82261
[82218]: rust-lang/rust#82218
[82216]: rust-lang/rust#82216
[82202]: rust-lang/rust#82202
[81855]: rust-lang/rust#81855
[81766]: rust-lang/rust#81766
[81744]: rust-lang/rust#81744
[81611]: rust-lang/rust#81611
[81479]: rust-lang/rust#81479
[81451]: rust-lang/rust#81451
[81356]: rust-lang/rust#81356
[80962]: rust-lang/rust#80962
[80553]: rust-lang/rust#80553
[80527]: rust-lang/rust#80527
[79519]: rust-lang/rust#79519
[79423]: rust-lang/rust#79423
[79208]: rust-lang/rust#79208
[78429]: rust-lang/rust#78429
[82733]: rust-lang/rust#82733
[82594]: rust-lang/rust#82594
[cargo/9181]: rust-lang/cargo#9181
[`char::MAX`]: https://doc.rust-lang.org/std/primitive.char.html#associatedconstant.MAX
[`char::REPLACEMENT_CHARACTER`]: https://doc.rust-lang.org/std/primitive.char.html#associatedconstant.REPLACEMENT_CHARACTER
[`char::UNICODE_VERSION`]: https://doc.rust-lang.org/std/primitive.char.html#associatedconstant.UNICODE_VERSION
[`char::decode_utf16`]: https://doc.rust-lang.org/std/primitive.char.html#method.decode_utf16
[`char::from_u32`]: https://doc.rust-lang.org/std/primitive.char.html#method.from_u32
[`char::from_u32_unchecked`]: https://doc.rust-lang.org/std/primitive.char.html#method.from_u32_unchecked
[`char::from_digit`]: https://doc.rust-lang.org/std/primitive.char.html#method.from_digit
[`Peekable::next_if`]: https://doc.rust-lang.org/stable/std/iter/struct.Peekable.html#method.next_if
[`Peekable::next_if_eq`]: https://doc.rust-lang.org/stable/std/iter/struct.Peekable.html#method.next_if_eq
[`Arguments::as_str`]: https://doc.rust-lang.org/stable/std/fmt/struct.Arguments.html#method.as_str
[`str::split_once`]: https://doc.rust-lang.org/stable/std/primitive.str.html#method.split_once
[`str::rsplit_once`]: https://doc.rust-lang.org/stable/std/primitive.str.html#method.rsplit_once
[`slice::partition_point`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.partition_point
[`char::len_utf8`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.len_utf8
[`char::len_utf16`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.len_utf16
[`char::to_ascii_uppercase`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.to_ascii_uppercase
[`char::to_ascii_lowercase`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.to_ascii_lowercase
[`char::eq_ignore_ascii_case`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.eq_ignore_ascii_case
[`u8::to_ascii_uppercase`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.to_ascii_uppercase
[`u8::to_ascii_lowercase`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.to_ascii_lowercase
[`u8::eq_ignore_ascii_case`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.eq_ignore_ascii_case
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-HIR Area: The high-level intermediate representation (HIR) A-incr-comp Area: Incremental compilation I-compiletime Issue: Problems and improvements with respect to compile times. merged-by-bors This PR was explicitly merged by bors. relnotes-perf Performance improvements that should be mentioned in the release notes. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.