Port LogContext to Rust

[erikjohnston]

Ports the logcontext classes to Rust, and gives tokio tasks a captured logcontext so that work running in (or spawned from) Rust is attributed to the request that caused it.

Best reviewed commit by commit:

1. Add characterization tests for logcontext error messages and the filter — pins the exact logcontext_error message shapes, the abuse-detection code paths and LoggingContextFilter's observable behaviour, against the existing Python implementation (this commit is green on its own). These are the behavioural contract the port has to satisfy.
2. Port ContextResourceUsage to a Rust pyclass — self-contained: the new synapse_rust.logcontext module, the class, its stub and the re-export.
3. Move the logcontext storage and LoggingContext to Rust — the core change; the commit message carries detailed design notes. Highlights:
   • The slot is typed Option<Py<LoggingContext>>, with None representing the sentinel. _Sentinel/SENTINEL_CONTEXT stay pure Python (unchanged); thin wrappers on current_context/set_current_context convert at the boundary, and pyo3's extraction enforces the type (TypeError otherwise).
   • The accounting is native: one getrusage(RUSAGE_THREAD) read per switch via libc, inline stop/start bookkeeping for base LoggingContexts, Python dispatch only for subclasses (BackgroundProcessLoggingContext) so their overrides run. The thread id comes from PyThread_get_thread_ident (the exact threading.get_ident()
     value) without calling into Python.
   • The hot paths avoid per-operation allocation: names are Py<PyString> (the per-log-record str(context)/server_name reads are INCREF-only), error branches materialise strings only when hit.
4. Attribute Rust-spawned work to the caller's logcontext — create_deferred captures the caller's context and scopes it onto the spawned task via a tokio task-local (LogContextHandle); current_context() gives the task-local read precedence, so LoggingContextFilter/pyo3-log resolve the right context on worker
   threads with no per-record stamping. run_python_awaitable restores the captured context (via a with_logcontext helper, the Rust PreserveLoggingContext) around Python called back from Rust, so e.g. runInteraction from the Rust /versions handler accounts its DB usage against the right request. Integration tests exercise both guarantees through real production code paths.

Follow-up work on top of this (separate PR): porting BackgroundProcessLoggingContext natively and removing further Py<_> indirections. The fact that BackgroundProcessLoggingContext is a subclass is what forces some of the warts in this PR: e.g. having to use Py<LoggingContext> everywhere, etc.

We don't try (yet) to make this pure Rust, instead we see this as simply maintaining the Python logcontext machinery when crossing, rather than trying to make a Rust equivalent that can be used by pure Rust dependencies. We probably do want to do that in future, as well as wire up e.g. CPU recording on Rust side, but that is unnecessary for now.

[Copilot]

Pull request overview

Ports Synapse’s logcontext machinery to a Rust-backed implementation so that the “current logcontext” is visible to both Python (reactor + threadpool) and Rust (tokio tasks), enabling correct attribution of logs and resource accounting across the language boundary.

Changes:

• Introduces a Rust implementation of LoggingContext, ContextResourceUsage, and current-context storage with both OS-thread-local and tokio-task-local sources of truth.
• Updates Python-facing wrappers (synapse.logging.context) and related types/docs to use the Rust extension while preserving existing observable behavior.
• Adds characterization and integration tests to pin error-message shapes, filter behavior, and cross-language attribution guarantees.

Reviewed changes

Copilot reviewed 15 out of 16 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/util/test_logcontext.py	Adds characterization tests for logcontext_error messages and LoggingContextFilter behavior.
tests/test_state.py	Updates test setup to provide hs.hostname expected by updated logcontext usage.
tests/synapse_rust/test_logcontext.py	Adds integration tests validating Rust↔Python logcontext attribution via real code paths.
tests/storage/test_base.py	Adjusts DatabasePool test construction to provide a hostname.
tests/handlers/test_appservice.py	Updates mocked homeserver to provide hostname for background process/logcontext usage.
synapse/synapse_rust/logcontext.pyi	Adds typing stubs for the Rust logcontext module surface.
synapse/metrics/background_process_metrics.py	Updates BackgroundProcessLoggingContext.start rusage typing to the new (utime, stime) tuple shape.
synapse/logging/context.py	Switches Python logcontext implementation to wrappers over the Rust extension and re-exports Rust-backed classes/constants.
rust/src/logging/mod.rs	Adds Rust logging module namespace aligned with Python logger naming.
rust/src/logging/context.rs	Implements Rust storage, task-local scoping, and Rust-backed LoggingContext/ContextResourceUsage.
rust/src/lib.rs	Registers the Rust logcontext module with Python.
rust/src/deferred.rs	Captures/scopes logcontext onto spawned tokio tasks and restores it for Rust→Python callbacks.
rust/Cargo.toml	Adds libc dependency for getrusage(RUSAGE_THREAD) access.
docs/log_contexts.md	Documents the Rust-backed storage model and how to correctly spawn tokio tasks with captured logcontext.
changelog.d/19979.misc	Adds Towncrier newsfragment for the port.
Cargo.lock	Locks libc dependency addition.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.