Skip to content

Commit 23d77ab

Browse files
committed
Add Section How queries interact with external crate metadata
Signed-off-by: xizheyin <xizheyin@smail.nju.edu.cn>
1 parent e03ee80 commit 23d77ab

File tree

1 file changed

+42
-5
lines changed

1 file changed

+42
-5
lines changed

src/query.md

Lines changed: 42 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -78,9 +78,6 @@ during compiler initialization. The macro system generates both structs,
7878
which act as function tables for all query implementations, where each
7979
field is a function pointer to the actual provider.
8080

81-
[providers_struct]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/query/struct.Providers.html
82-
[extern_providers_struct]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/query/struct.ExternProviders.html
83-
8481
**Note:** Both the `Providers` and `ExternProviders` structs are generated by macros and act as function tables for all query implementations.
8582
They are **not** Rust traits, but plain structs with function pointer fields.
8683

@@ -174,11 +171,51 @@ Suppose you want to add a new query called `fubar`. You would:
174171
```rust,ignore
175172
pub fn provide(providers: &mut rustc_middle::util::Providers) {
176173
providers.queries.fubar = fubar;
177-
// If you need an external provider:
178-
providers.extern_queries.fubar = extern_fubar;
179174
}
180175
```
181176
177+
### How queries interact with external crate metadata
178+
179+
When a query is made for an external crate (i.e., a dependency), the query system needs to load the information from that crate's metadata.
180+
This is handled by the [`rustc_metadata` crate][rustc_metadata], which is responsible for decoding and providing the information stored in the `.rmeta` files.
181+
182+
The process works like this:
183+
184+
1. When a query is made, the query system first checks if the `DefId` refers to a local or external crate by checking if `def_id.krate == LOCAL_CRATE`.
185+
This determines whether to use the local provider from [`Providers`][providers_struct] or the external provider from [`ExternProviders`][extern_providers_struct].
186+
187+
2. For external crates, the query system will look for a provider in the [`ExternProviders`][extern_providers_struct] struct.
188+
The `rustc_metadata` crate registers these external providers through the `provide_extern` function in `rustc_metadata/src/rmeta/decoder/cstore_impl.rs`. Just like:
189+
```rust
190+
pub fn provide_extern(providers: &mut ExternProviders) {
191+
providers.foo = |tcx, def_id| {
192+
// Load and decode metadata for external crate
193+
let cdata = CStore::from_tcx(tcx).get_crate_data(def_id.krate);
194+
cdata.foo(def_id.index)
195+
};
196+
// Register other external providers...
197+
}
198+
```
199+
200+
3. The metadata is stored in a binary format in `.rmeta` files that contains pre-computed information about the external crate, such as types, function signatures, trait implementations, and other information needed by the compiler. When an external query is made, the `rustc_metadata` crate:
201+
- Loads the `.rmeta` file for the external crate
202+
- Decodes the metadata using the `Decodable` trait
203+
- Returns the decoded information to the query system
204+
205+
This approach avoids recompiling external crates, allows for faster compilation of dependent crates, and enables incremental compilation to work across crate boundaries.
206+
Here is a simplified example, when you call `tcx.type_of(def_id)` for a type defined in an external crate, the query system will:
207+
1. Detect that the `def_id` refers to an external crate by checking `def_id.krate != LOCAL_CRATE`
208+
2. Call the appropriate provider from `ExternProviders` which was registered by `rustc_metadata`
209+
3. The provider will load and decode the type information from the external crate's metadata
210+
4. Return the decoded type to the caller
211+
212+
This is why most `rustc_*` crates only need to provide local providers - the external providers are handled by the metadata system.
213+
The only exception is when a crate needs to provide special handling for external queries, in which case it would implement both local and external providers.
214+
215+
[rustc_metadata]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_metadata/index.html
216+
[providers_struct]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/query/struct.Providers.html
217+
[extern_providers_struct]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/query/struct.ExternProviders.html
218+
182219
---
183220
184221
## Adding a new query

0 commit comments

Comments
 (0)