core: add a limited form of the linked-list callsite registry for v0.1.x (tokio-rs#2083)

## Motivation

Currently on `v0.1.x`, the global callsite registry is implemented as a
`Mutex<Vec<&'static dyn Callsite>>`. This has a few downsides:

* Every time a callsite is registered, the `Mutex` must be locked. This
  can cause a deadlock when a `register_callsite` implementation calls
  into code that _also_ registers a callsite. See tokio-rs#2020 for details.

* Registering callsites is relatively slow and forces the program to
  synchronize on the callsite registry (and not on *individual*
  callsites). This means that if two threads are both registering
  totally different callsites, both threads must wait for the lock.
  Although this overhead is amortized over the entire rest of the
  program, it does have an impact in short-running applications where
  any given callsite may only be hit once or twice.

* Memory allocation may occur while the registry is locked. This makes
  the use of `tracing` inside of memory allocators difficult or
  impossible.

On the `master` branch (v0.2.x), PR tokio-rs#988 rewrote the callsite registry
to use an intrusive atomic singly-linked list of `Callsite`s. This
removed the need for locking the callsite registry, and made it possible
to register callsites without ever allocating memory. Because the
callsite registry on v0.2 will *never* allocate, this also makes it
possible to compile `tracing-core` for `no_std` platforms without
requiring `liballoc`. Unfortunately, however, we cannot use an identical
implementation on v0.1.x, because using the intrusive linked-list
registry for *all* callsites requires a breaking change to the
`Callsite` trait (in order to add a way to get the callsite's
linked-list node), which we cannot make on v0.1.x.

## Solution

This branch adds a linked-list callsite registry for v0.1.x in a way
that allows us to benefit from *some* of the advantages of this approach
in a majority of cases. The trick is introducing a new `DefaultCallsite`
type in `tracing-core` that implements the `Callsite` trait. This type
can contain an intrusive list node, and *when a callsite is a
`DefaultCallsite`*, we can register it without having to push it to the
`Mutex<Vec<...>>`. The locked vec still _exists_, for `Callsite`s that
are *not* `DefaultCallsite`s, so we can't remove the `liballoc`
dependency, but in most cases, we can avoid the mutex and allocation.

Naturally, `tracing` has been updated to use the `DefaultCallsite` type
from `tracing-core`, so the `Vec` will only be used in the following
cases:

* User code has a custom implementation of the `Callsite` trait, which
  is [not terribly common][1].
* An older version of the `tracing` crate is in use.

This fixes the deadlock described in tokio-rs#2020 when `DefaultCallsite`s are
used. Additionally, it should reduce the performance impact and memory
usage of the callsite registry.

Furthermore, I've changed the subscriber registry so that a
`RwLock<Vec<dyn Dispatch>>` is only used when there actually are
multiple subscribers in use. When there's only a global default
subscriber, we can once again avoid locking for the registry of
subscribers. When the `std` feature is disabled, thread-local scoped
dispatchers are unavailable, so the single global subscriber will always
be used on `no_std`.

We can make additional changes, such as the ones from tokio-rs#2020, to _also_
resolve potential deadlocks when non-default callsites are in use, but
since this branch rewrites a lot of the callsite registry code, that
should probably be done in a follow-up.

[1]: https://cs.github.com/?scopeName=All+repos&scope=&q=%28%2Fimpl+.*Callsite%2F+AND+language%3Arust%29+NOT+%28path%3A%2Ftracing%2F**+OR+path%3A%2Ftracing-*%2F**+OR+path%3A%2Ftokio-trace*%2F**%29%29

Signed-off-by: Eliza Weisman <eliza@buoyant.io>

Author: hawkw

tracing-core/src/callsite.rs

@@ -124,6 +124,28 @@ pub trait Callsite: Sync {
     ///
     /// [metadata]: super::metadata::Metadata
     fn metadata(&self) -> &Metadata<'_>;
+
+    /// This method is an *internal implementation detail* of `tracing-core`. It
+    /// is *not* intended to be called or overridden from downstream code.
+    ///
+    /// The `Private` type can only be constructed from within `tracing-core`.
+    /// Because this method takes a `Private` as an argument, it cannot be
+    /// called from (safe) code external to `tracing-core`. Because it must
+    /// *return* a `Private`, the only valid implementation possible outside of
+    /// `tracing-core` would have to always unconditionally panic.
+    ///
+    /// THIS IS BY DESIGN. There is currently no valid reason for code outside
+    /// of `tracing-core` to override this method.
+    // TODO(eliza): this could be used to implement a public downcasting API
+    // for `&dyn Callsite`s in the future.
+    #[doc(hidden)]
+    #[inline]
+    fn private_type_id(&self, _: private::Private<()>) -> private::Private<TypeId>
+    where
+        Self: 'static,
+    {
+        private::Private(TypeId::of::<Self>())
+    }
 }
 
 /// Uniquely identifies a [`Callsite`]

tracing-core/src/dispatcher.rs

@@ -134,14 +134,15 @@ use crate::stdlib::{
     fmt,
     sync::{
         atomic::{AtomicBool, AtomicUsize, Ordering},
-        Arc, Weak,
+        Arc,
     },
 };
 
 #[cfg(feature = "std")]
 use crate::stdlib::{
     cell::{Cell, RefCell, RefMut},
     error,
+    sync::Weak,
 };
 
 /// `Dispatch` trace data to a [`Subscriber`].
@@ -387,6 +388,7 @@ fn get_global() -> Option<&'static Dispatch> {
     }
 }
 
+#[cfg(feature = "std")]
 pub(crate) struct Registrar(Weak<dyn Subscriber + Send + Sync>);
 
 impl Dispatch {
@@ -412,6 +414,7 @@ impl Dispatch {
         me
     }
 
+    #[cfg(feature = "std")]
     pub(crate) fn registrar(&self) -> Registrar {
         Registrar(Arc::downgrade(&self.subscriber))
     }
@@ -651,14 +654,8 @@ where
     }
 }
 
+#[cfg(feature = "std")]
 impl Registrar {
-    pub(crate) fn try_register(
-        &self,
-        metadata: &'static Metadata<'static>,
-    ) -> Option<subscriber::Interest> {
-        self.0.upgrade().map(|s| s.register_callsite(metadata))
-    }
-
     pub(crate) fn upgrade(&self) -> Option<Dispatch> {
         self.0.upgrade().map(|subscriber| Dispatch { subscriber })
     }

tracing/src/lib.rs

@@ -1164,20 +1164,16 @@ pub mod __macro_support {
         }
     }
 
-    impl Callsite for MacroCallsite {
-        fn set_interest(&self, interest: Interest) {
-            let interest = match () {
-                _ if interest.is_never() => 0,
-                _ if interest.is_always() => 2,
-                _ => 1,
-            };
-            self.interest.store(interest, Ordering::SeqCst);
-        }
-
-        #[inline(always)]
-        fn metadata(&self) -> &Metadata<'static> {
-            self.meta
-        }
+    /// /!\ WARNING: This is *not* a stable API! /!\
+    /// This function, and all code contained in the `__macro_support` module, is
+    /// a *private* API of `tracing`. It is exposed publicly because it is used
+    /// by the `tracing` macros, but it is not part of the stable versioned API.
+    /// Breaking changes to this module may occur in small-numbered versions
+    /// without warning.
+    #[inline]
+    #[cfg(not(feature = "log"))]
+    pub fn __disabled_span(_: &'static Metadata<'static>) -> crate::Span {
+        crate::Span::none()
     }
 
     impl fmt::Debug for MacroCallsite {

tracing/src/macros.rs

@@ -3090,13 +3090,14 @@ macro_rules! __tracing_log {
             use $crate::log;
             let level = $crate::level_to_log!($level);
             if level <= log::max_level() {
+                let meta = $callsite.metadata();
                 let log_meta = log::Metadata::builder()
                     .level(level)
                     .target(__CALLSITE.metadata().target())
                     .build();
                 let logger = log::logger();
                 if logger.enabled(&log_meta) {
-                    $callsite.log(logger, log_meta, $value_set)
+                    $crate::__macro_support::__tracing_log(meta, logger, log_meta, $value_set)
                 }
             }
         }}

tracing/tests/register_callsite_deadlock.rs

@@ -0,0 +1,47 @@
+use std::{sync::mpsc, thread, time::Duration};
+use tracing::{
+    metadata::Metadata,
+    span,
+    subscriber::{self, Interest, Subscriber},
+    Event,
+};
+
+#[test]
+fn register_callsite_doesnt_deadlock() {
+    pub struct EvilSubscriber;
+
+    impl Subscriber for EvilSubscriber {
+        fn register_callsite(&self, meta: &'static Metadata<'static>) -> Interest {
+            tracing::info!(?meta, "registered a callsite");
+            Interest::always()
+        }
+
+        fn enabled(&self, _: &Metadata<'_>) -> bool {
+            true
+        }
+        fn new_span(&self, _: &span::Attributes<'_>) -> span::Id {
+            span::Id::from_u64(1)
+        }
+        fn record(&self, _: &span::Id, _: &span::Record<'_>) {}
+        fn record_follows_from(&self, _: &span::Id, _: &span::Id) {}
+        fn event(&self, _: &Event<'_>) {}
+        fn enter(&self, _: &span::Id) {}
+        fn exit(&self, _: &span::Id) {}
+    }
+
+    subscriber::set_global_default(EvilSubscriber).unwrap();
+
+    // spawn a thread, and assert it doesn't hang...
+    let (tx, didnt_hang) = mpsc::channel();
+    let th = thread::spawn(move || {
+        tracing::info!("hello world!");
+        tx.send(()).unwrap();
+    });
+
+    didnt_hang
+        // Note: 60 seconds is *way* more than enough, but let's be generous in
+        // case of e.g. slow CI machines.
+        .recv_timeout(Duration::from_secs(60))
+        .expect("the thread must not have hung!");
+    th.join().expect("thread should join successfully");
+}