Skip to content

Registry debug logs leak to console via ZenML root logger initialization #380

@AgentDosaku

Description

@AgentDosaku

Summary

Using Registry can emit verbose debug messages to stdout/stderr even when callers never explicitly configure Mindtrace/Registry logging for console debug output.

Example output observed during load(...):

  • Loading metadata from: ...
  • Loaded metadata: ...
  • Downloading directory from ...
  • Download complete. Contents: ...

These are self.logger.debug(...) calls in the local registry backend, but they still reach the terminal due to propagation and root logger handlers installed by ZenML.

Root cause chain

From local investigation in this repo:

  1. Registry / backend classes inherit Mindtrace and obtain loggers via get_logger(...).
  2. Mindtrace logger setup defaults to propagate=True in mindtrace.core.logging.logger.get_logger.
  3. mindtrace.registry imports ZenML materializer codepaths.
  4. ZenML imports call zenml.init_logging() at import time.
    • zenml/__init__.py calls init_logging().
    • zenml/logger.py:init_logging() sets root verbosity and adds root handlers (console + ZenML handler).
  5. With propagation enabled, Registry/backend DEBUG records bubble to root and appear in console.

Reproduction

import logging

print("before import:", logging.getLogger().level, logging.getLogger().handlers)

from mindtrace.registry import Registry

print("after import:", logging.getLogger().level, logging.getLogger().handlers)

r = Registry()  # default local backend
r.save("a/shared:item", 1)
print(r.load("a/shared:item"))

Expected by many users: no backend debug spam on console unless explicitly enabled.

Actual: debug lines from local registry backend appear on console in environments where ZenML root logging is initialized.

Notes from local run

  • Before importing mindtrace.registry, root logger had no handlers.
  • After import, root had a StreamHandler and ZenMLLoggingHandler, with root level changed.
  • Registry/backend logger effective level allowed debug, and propagated to root.

Suggested direction

Short-term mitigation for Registry UX:

  • Default Registry logger propagation to False (while allowing explicit opt-in).

Longer-term logging architecture:

  • Clarify library-vs-application logging ownership and avoid surprising import-time global logging side effects.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions