Arbitrary self types v2: use Receiver trait

In this new version of Arbitrary Self Types, we no longer use the Deref trait
exclusively when working out which self types are valid. Instead, we follow a
chain of Receiver traits. This enables methods to be called on smart pointer
types which fundamentally cannot support Deref (for instance because they are
wrappers for pointers that don't follow Rust's aliasing rules).

This includes:
* Changes to tests appropriately
* New tests for:
  * The basics of the feature
  * Ensuring lifetime elision works properly
  * Generic Receivers
  * A copy of the method subst test enhanced with Receiver

This is really the heart of the 'arbitrary self types v2' feature, and
is the most critical commit in the current PR.

Subsequent commits are focused on:
* Detecting "shadowing" problems, where a smart pointer type can hide
  methods in the pointee.
* Diagnostics and cleanup.

Naming: in this commit, the "Autoderef" type is modified so that it no
longer solely focuses on the "Deref" trait, but can now consider the
"Receiver" trait instead. Should it be renamed, to something like
"TraitFollower"? This was considered, but rejected, because
* even in the Receiver case, it still considers built-in derefs
* the name Autoderef is short and snappy.

Author: adetaylor

compiler/rustc_error_codes/src/error_codes/E0307.md

@@ -65,8 +65,10 @@ impl Trait for Foo {
 ```
 
 The nightly feature [Arbitrary self types][AST] extends the accepted
-set of receiver types to also include any type that can dereference to
-`Self`:
+set of receiver types to also include any type that can implements the
+`Receiver` trait and can follow its chain of `Target` types to `Self`.
+There's a blanket implementation of `Receiver` for `T: Deref`, so any
+type which dereferences to `Self` can be used.
 
 ```
 #![feature(arbitrary_self_types)]

compiler/rustc_hir_analysis/messages.ftl

@@ -241,10 +241,10 @@ hir_analysis_invalid_generic_receiver_ty_help =
     use a concrete type such as `self`, `&self`, `&mut self`, `self: Box<Self>`, `self: Rc<Self>`, `self: Arc<Self>`, or `self: Pin<P>` (where P is one of the previous types except `Self`)
 
 hir_analysis_invalid_receiver_ty = invalid `self` parameter type: `{$receiver_ty}`
-    .note = type of `self` must be `Self` or a type that dereferences to it
+    .note = type of `self` must be `Self` or some type implementing Receiver
 
 hir_analysis_invalid_receiver_ty_help =
-    consider changing to `self`, `&self`, `&mut self`, `self: Box<Self>`, `self: Rc<Self>`, `self: Arc<Self>`, or `self: Pin<P>` (where P is one of the previous types except `Self`)
+    consider changing to `self`, `&self`, `&mut self`, or a type implementing `Receiver` such as `self: Box<Self>`, `self: Rc<Self>`, or `self: Arc<Self>`
 
 hir_analysis_invalid_union_field =
     field must implement `Copy` or be wrapped in `ManuallyDrop<...>` to be used in a union

compiler/rustc_hir_analysis/src/autoderef.rs

@@ -18,7 +18,6 @@ pub enum AutoderefKind {
     /// A type which must dispatch to a `Deref` implementation.
     Overloaded,
 }
-
 struct AutoderefSnapshot<'tcx> {
     at_start: bool,
     reached_recursion_limit: bool,
@@ -27,6 +26,10 @@ struct AutoderefSnapshot<'tcx> {
     obligations: PredicateObligations<'tcx>,
 }
 
+/// Recursively dereference a type, considering both built-in
+/// dereferences (`*`) and the `Deref` trait.
+/// Although called `Autoderef` it can be configured to use the
+/// `Receiver` trait instead of the `Deref` trait.
 pub struct Autoderef<'a, 'tcx> {
     // Meta infos:
     infcx: &'a InferCtxt<'tcx>,
@@ -39,6 +42,7 @@ pub struct Autoderef<'a, 'tcx> {
 
     // Configurations:
     include_raw_pointers: bool,
+    use_receiver_trait: bool,
     silence_errors: bool,
 }
 
@@ -69,6 +73,14 @@ impl<'a, 'tcx> Iterator for Autoderef<'a, 'tcx> {
         }
 
         // Otherwise, deref if type is derefable:
+        // NOTE: in the case of self.use_receiver_trait = true, you might think it would
+        // be better to skip this clause and use the Overloaded case only, since &T
+        // and &mut T implement Receiver. (They don't implement Deref). We can't do that
+        // for a couple of reasons - first, there is magic in the builtin * to allow
+        // calls to fn a(&mut self) if there's a: &mut T even if 'a' itself is not mutable.
+        // Secondly the builtin deref can work in constant contexts.
+        // So even for Receiver-based autodereferencing, we need to retain
+        // use of the built-in dereference *.
         let (kind, new_ty) =
             if let Some(ty) = self.state.cur_ty.builtin_deref(self.include_raw_pointers) {
                 debug_assert_eq!(ty, self.infcx.resolve_vars_if_possible(ty));
@@ -111,7 +123,7 @@ impl<'a, 'tcx> Autoderef<'a, 'tcx> {
         body_def_id: LocalDefId,
         span: Span,
         base_ty: Ty<'tcx>,
-    ) -> Autoderef<'a, 'tcx> {
+    ) -> Self {
         Autoderef {
             infcx,
             span,
@@ -125,6 +137,7 @@ impl<'a, 'tcx> Autoderef<'a, 'tcx> {
                 reached_recursion_limit: false,
             },
             include_raw_pointers: false,
+            use_receiver_trait: false,
             silence_errors: false,
         }
     }
@@ -137,8 +150,13 @@ impl<'a, 'tcx> Autoderef<'a, 'tcx> {
             return None;
         }
 
-        // <ty as Deref>
-        let trait_ref = ty::TraitRef::new(tcx, tcx.lang_items().deref_trait()?, [ty]);
+        // <ty as Deref>, or whatever the equivalent trait is that we've been asked to walk.
+        let (trait_def_id, trait_target_def_id) = if self.use_receiver_trait {
+            (tcx.lang_items().receiver_trait()?, tcx.lang_items().receiver_target()?)
+        } else {
+            (tcx.lang_items().deref_trait()?, tcx.lang_items().deref_target()?)
+        };
+        let trait_ref = ty::TraitRef::new(tcx, trait_def_id, [ty]);
         let cause = traits::ObligationCause::misc(self.span, self.body_id);
         let obligation = traits::Obligation::new(
             tcx,
@@ -151,11 +169,8 @@ impl<'a, 'tcx> Autoderef<'a, 'tcx> {
             return None;
         }
 
-        let (normalized_ty, obligations) = self.structurally_normalize(Ty::new_projection(
-            tcx,
-            tcx.lang_items().deref_target()?,
-            [ty],
-        ))?;
+        let (normalized_ty, obligations) =
+            self.structurally_normalize(Ty::new_projection(tcx, trait_target_def_id, [ty]))?;
         debug!("overloaded_deref_ty({:?}) = ({:?}, {:?})", ty, normalized_ty, obligations);
         self.state.obligations.extend(obligations);
 
@@ -234,6 +249,14 @@ impl<'a, 'tcx> Autoderef<'a, 'tcx> {
         self
     }
 
+    /// Use `core::ops::Receiver` and `core::ops::Receiver::Target` as
+    /// the trait and associated type to iterate, instead of
+    /// `core::ops::Deref` and `core::ops::Deref::Target`
+    pub fn use_receiver_trait(mut self) -> Self {
+        self.use_receiver_trait = true;
+        self
+    }
+
     pub fn silence_errors(mut self) -> Self {
         self.silence_errors = true;
         self

compiler/rustc_hir_analysis/src/check/wfcheck.rs

@@ -1805,15 +1805,17 @@ fn receiver_is_valid<'tcx>(
 
     confirm_type_is_not_a_method_generic_param(receiver_ty, method_generics)?;
 
-    let mut autoderef = Autoderef::new(infcx, wfcx.param_env, wfcx.body_def_id, span, receiver_ty);
+    let hard_coded_receiver_trait_def_id =
+        tcx.require_lang_item(LangItem::LegacyReceiver, Some(span));
+
+    let mut autoderef = Autoderef::new(infcx, wfcx.param_env, wfcx.body_def_id, span, receiver_ty)
+        .use_receiver_trait();
 
     // The `arbitrary_self_types_pointers` feature allows raw pointer receivers like `self: *const Self`.
     if arbitrary_self_types_enabled == Some(ArbitrarySelfTypesLevel::WithPointers) {
         autoderef = autoderef.include_raw_pointers();
     }
 
-    let receiver_trait_def_id = tcx.require_lang_item(LangItem::LegacyReceiver, Some(span));
-
     // Keep dereferencing `receiver_ty` until we get to `self_ty`.
     while let Some((potential_self_ty, _)) = autoderef.next() {
         debug!(
@@ -1839,7 +1841,7 @@ fn receiver_is_valid<'tcx>(
         if arbitrary_self_types_enabled.is_none() {
             if !receiver_is_implemented(
                 wfcx,
-                receiver_trait_def_id,
+                hard_coded_receiver_trait_def_id,
                 cause.clone(),
                 potential_self_ty,
             ) {
@@ -1852,7 +1854,7 @@ fn receiver_is_valid<'tcx>(
                 cause.clone(),
                 wfcx.param_env,
                 potential_self_ty,
-                receiver_trait_def_id,
+                hard_coded_receiver_trait_def_id,
             );
         }
     }

compiler/rustc_hir_typeck/src/method/probe.rs

@@ -366,6 +366,7 @@ impl<'a, 'tcx> FnCtxt<'a, 'tcx> {
                         autoderefs: 0,
                         from_unsafe_deref: false,
                         unsize: false,
+                        reachable_via_deref: true,
                     }]),
                     opt_bad_ty: None,
                     reached_recursion_limit: false,
@@ -516,36 +517,90 @@ fn method_autoderef_steps<'tcx>(
     let (ref infcx, goal, inference_vars) = tcx.infer_ctxt().build_with_canonical(DUMMY_SP, &goal);
     let ParamEnvAnd { param_env, value: self_ty } = goal;
 
-    let mut autoderef =
+    // If arbitrary self types is not enabled, we follow the chain of
+    // `Deref<Target=T>`. If arbitrary self types is enabled, we instead
+    // follow the chain of `Receiver<Target=T>`, but we also record whether
+    // such types are reachable by following the (potentially shorter)
+    // chain of `Deref<Target=T>`. We will use the first list when finding
+    // potentially relevant function implementations (e.g. relevant impl blocks)
+    // but the second list when determining types that the receiver may be
+    // converted to, in order to find out which of those methods might actually
+    // be callable.
+    let mut autoderef_via_receiver =
+        Autoderef::new(infcx, param_env, hir::def_id::CRATE_DEF_ID, DUMMY_SP, self_ty)
+            .include_raw_pointers()
+            .use_receiver_trait()
+            .silence_errors();
+
+    let mut autoderef_via_deref =
         Autoderef::new(infcx, param_env, hir::def_id::CRATE_DEF_ID, DUMMY_SP, self_ty)
             .include_raw_pointers()
             .silence_errors();
-    let mut reached_raw_pointer = false;
-    let mut steps: Vec<_> = autoderef
-        .by_ref()
-        .map(|(ty, d)| {
-            let step = CandidateStep {
-                self_ty: infcx.make_query_response_ignoring_pending_obligations(inference_vars, ty),
-                autoderefs: d,
-                from_unsafe_deref: reached_raw_pointer,
-                unsize: false,
-            };
-            if let ty::RawPtr(_, _) = ty.kind() {
-                // all the subsequent steps will be from_unsafe_deref
-                reached_raw_pointer = true;
-            }
-            step
-        })
-        .collect();
 
-    let final_ty = autoderef.final_ty(true);
+    let mut reached_raw_pointer = false;
+    let arbitrary_self_types_enabled =
+        tcx.features().arbitrary_self_types() || tcx.features().arbitrary_self_types_pointers();
+    let mut final_ty_reachable_by_deref = None;
+    let mut steps: Vec<_> = if arbitrary_self_types_enabled {
+        let reachable_via_deref =
+            autoderef_via_deref.by_ref().map(|_| true).chain(std::iter::repeat(false));
+        autoderef_via_receiver
+            .by_ref()
+            .zip(reachable_via_deref)
+            .map(|((ty, d), reachable_via_deref)| {
+                if reachable_via_deref {
+                    final_ty_reachable_by_deref = Some(ty);
+                }
+                let step = CandidateStep {
+                    self_ty: infcx
+                        .make_query_response_ignoring_pending_obligations(inference_vars, ty),
+                    autoderefs: d,
+                    from_unsafe_deref: reached_raw_pointer,
+                    unsize: false,
+                    reachable_via_deref,
+                };
+                if let ty::RawPtr(_, _) = ty.kind() {
+                    // all the subsequent steps will be from_unsafe_deref
+                    reached_raw_pointer = true;
+                }
+                step
+            })
+            .collect()
+    } else {
+        let steps = autoderef_via_deref
+            .by_ref()
+            .map(|(ty, d)| {
+                let step = CandidateStep {
+                    self_ty: infcx
+                        .make_query_response_ignoring_pending_obligations(inference_vars, ty),
+                    autoderefs: d,
+                    from_unsafe_deref: reached_raw_pointer,
+                    unsize: false,
+                    reachable_via_deref: true,
+                };
+                if let ty::RawPtr(_, _) = ty.kind() {
+                    // all the subsequent steps will be from_unsafe_deref
+                    reached_raw_pointer = true;
+                }
+                step
+            })
+            .collect();
+        final_ty_reachable_by_deref = Some(autoderef_via_deref.by_ref().final_ty(false));
+        steps
+    };
+    let final_ty =
+        final_ty_reachable_by_deref.expect("Should be at least one type in any autoderef");
+    let final_ty = infcx.resolve_vars_if_possible(final_ty);
     let opt_bad_ty = match final_ty.kind() {
         ty::Infer(ty::TyVar(_)) | ty::Error(_) => Some(MethodAutoderefBadTy {
             reached_raw_pointer,
             ty: infcx.make_query_response_ignoring_pending_obligations(inference_vars, final_ty),
         }),
+
         ty::Array(elem_ty, _) => {
             let dereferences = steps.len() - 1;
+            let reachable_via_deref =
+                steps.last().map(|cs| cs.reachable_via_deref).unwrap_or_default();
 
             steps.push(CandidateStep {
                 self_ty: infcx.make_query_response_ignoring_pending_obligations(
@@ -557,19 +612,29 @@ fn method_autoderef_steps<'tcx>(
                 // a *mut/const [T; N]
                 from_unsafe_deref: reached_raw_pointer,
                 unsize: true,
+                reachable_via_deref,
             });
 
             None
         }
         _ => None,
     };
 
+    // FIXME - the previous section may emit an error if the final type reachable
+    // by the Deref chain is ty::Infer(ty::TyVar(_)). We should consider emitting a warning
+    // if the final type reachable by the Receiver chain is similarly problematic.
+
     debug!("method_autoderef_steps: steps={:?} opt_bad_ty={:?}", steps, opt_bad_ty);
 
     MethodAutoderefStepsResult {
         steps: tcx.arena.alloc_from_iter(steps),
         opt_bad_ty: opt_bad_ty.map(|ty| &*tcx.arena.alloc(ty)),
-        reached_recursion_limit: autoderef.reached_recursion_limit(),
+        reached_recursion_limit: if arbitrary_self_types_enabled {
+            autoderef_via_receiver
+        } else {
+            autoderef_via_deref
+        }
+        .reached_recursion_limit(),
     }
 }
 
@@ -1065,6 +1130,10 @@ impl<'a, 'tcx> ProbeContext<'a, 'tcx> {
     ) -> Option<PickResult<'tcx>> {
         self.steps
             .iter()
+            // At this point we're considering the types to which the receiver can be converted,
+            // so we want to follow the `Deref` chain not the `Receiver` chain. Filter out
+            // steps which can only be reached by following the (longer) `Receiver` chain.
+            .filter(|step| step.reachable_via_deref)
             .filter(|step| {
                 debug!("pick_all_method: step={:?}", step);
                 // skip types that are from a type error or that would require dereferencing

compiler/rustc_middle/src/traits/query.rs

@@ -149,11 +149,21 @@ pub struct CandidateStep<'tcx> {
     /// `foo.by_raw_ptr()` will work and `foo.by_ref()` won't.
     pub from_unsafe_deref: bool,
     pub unsize: bool,
+    /// We will generate CandidateSteps which are reachable via a chain
+    /// of following `Receiver`. The first 'n' of those will be reachable
+    /// by following a chain of 'Deref' instead (since there's a blanket
+    /// implementation of Receiver for Deref).
+    /// We use the entire set of steps when identifying method candidates
+    /// (e.g. identifying relevant `impl` blocks) but only those that are
+    /// reachable via Deref when examining what the receiver type can
+    /// be converted into by autodereffing.
+    pub reachable_via_deref: bool,
 }
 
 #[derive(Copy, Clone, Debug, HashStable)]
 pub struct MethodAutoderefStepsResult<'tcx> {
-    /// The valid autoderef steps that could be found.
+    /// The valid autoderef steps that could be found by following a chain
+    /// of `Receiver<Target=T>` trait implementations.
     pub steps: &'tcx [CandidateStep<'tcx>],
     /// If Some(T), a type autoderef reported an error on.
     pub opt_bad_ty: Option<&'tcx MethodAutoderefBadTy<'tcx>>,

tests/ui/async-await/inference_var_self_argument.stderr

@@ -4,8 +4,8 @@ error[E0307]: invalid `self` parameter type: `&dyn Foo`
 LL |     async fn foo(self: &dyn Foo) {
    |                        ^^^^^^^^
    |
-   = note: type of `self` must be `Self` or a type that dereferences to it
-   = help: consider changing to `self`, `&self`, `&mut self`, `self: Box<Self>`, `self: Rc<Self>`, `self: Arc<Self>`, or `self: Pin<P>` (where P is one of the previous types except `Self`)
+   = note: type of `self` must be `Self` or some type implementing Receiver
+   = help: consider changing to `self`, `&self`, `&mut self`, or a type implementing `Receiver` such as `self: Box<Self>`, `self: Rc<Self>`, or `self: Arc<Self>`
 
 error[E0038]: the trait `Foo` cannot be made into an object
   --> $DIR/inference_var_self_argument.rs:5:5

tests/ui/async-await/issue-66312.stderr

@@ -4,8 +4,8 @@ error[E0307]: invalid `self` parameter type: `T`
 LL |     fn is_some(self: T);
    |                      ^
    |
-   = note: type of `self` must be `Self` or a type that dereferences to it
-   = help: consider changing to `self`, `&self`, `&mut self`, `self: Box<Self>`, `self: Rc<Self>`, `self: Arc<Self>`, or `self: Pin<P>` (where P is one of the previous types except `Self`)
+   = note: type of `self` must be `Self` or some type implementing Receiver
+   = help: consider changing to `self`, `&self`, `&mut self`, or a type implementing `Receiver` such as `self: Box<Self>`, `self: Rc<Self>`, or `self: Arc<Self>`
 
 error[E0308]: mismatched types
   --> $DIR/issue-66312.rs:9:8