Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setting a basicConfig before importing distributed changes the scheduler logging level. #8805

Open
Thomas-Z opened this issue Jul 27, 2024 · 0 comments

Comments

@Thomas-Z
Copy link

Hello,

Some kind of follow-up to the #7527 issue.

Describe the issue:

Setting a basicConfig before the distributed import changes the logging level of the scheduler to INFO.
Doing the same with the distributed import before the logging configuration does work as intended.

Minimal Complete Verifiable Example:

import logging

DEFAULT_FORMAT = "%(asctime)s %(levelname)-7s %(message)s"
DEFAULT_TIME_FORMAT = "%Y-%m-%d %H:%M:%S"

logging.basicConfig(format=DEFAULT_FORMAT, datefmt=DEFAULT_TIME_FORMAT)


import distributed

loc_cluster = distributed.LocalCluster(
    processes=False, n_workers=1, silence_logs="ERROR"
)

This code output the following logs:

2024-07-27 09:52:13 INFO    To route to workers diagnostics web server please install jupyter-server-proxy: python -m pip install jupyter-server-proxy
2024-07-27 09:52:13 INFO    State start
2024-07-27 09:52:13 INFO      Scheduler at: inproc://10.40.206.22/2275623/1
2024-07-27 09:52:13 INFO      dashboard at:  http://10.40.206.22:8787/status
2024-07-27 09:52:13 INFO    Registering Worker plugin shuffle
2024-07-27 09:52:13 INFO          Start worker at: inproc://10.40.206.22/2275623/4
2024-07-27 09:52:13 INFO             Listening to:         inproc10.40.206.22
2024-07-27 09:52:13 INFO              Worker name:                          0
2024-07-27 09:52:13 INFO             dashboard at:         10.40.206.22:41269
2024-07-27 09:52:13 INFO    Waiting to connect to: inproc://10.40.206.22/2275623/1
2024-07-27 09:52:13 INFO    -------------------------------------------------
2024-07-27 09:52:13 INFO                  Threads:                         96
2024-07-27 09:52:13 INFO                   Memory:                 629.52 GiB
2024-07-27 09:52:13 INFO          Local Directory: /tmp/dask-scratch-space-10238/worker-frdknfmy
2024-07-27 09:52:13 INFO    -------------------------------------------------
2024-07-27 09:52:13 INFO    Register worker <WorkerState 'inproc://10.40.206.22/2275623/4', name: 0, status: init, memory: 0, processing: 0>
2024-07-27 09:52:13 INFO    Starting worker compute stream, inproc://10.40.206.22/2275623/4
2024-07-27 09:52:13 INFO    Starting established connection to inproc://10.40.206.22/2275623/5
2024-07-27 09:52:13 INFO    Starting Worker plugin shuffle
2024-07-27 09:52:13 INFO            Registered to: inproc://10.40.206.22/2275623/1
2024-07-27 09:52:13 INFO    -------------------------------------------------
2024-07-27 09:52:13 INFO    Starting established connection to inproc://10.40.206.22/2275623/1

Environment:

  • Dask version: 2024.7.1
  • Python version: 3.12.4
  • Operating System: Ubuntu 22.04
  • Install method (conda, pip, source): conda
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant