Skip to content

Make writing cluster state dumps to buckets easier #5659

@gjoseph92

Description

@gjoseph92

Client.dump_cluster_state is a useful method for debugging. But having to download all the state to your local machine can be slow. And as we consider having more automation around it (auto-dump on exception, on close, etc.), being able to write the dump directly to cloud/network storage would be useful.

Add support for dump_cluster_state to write using fsspec. We should support passing in any string that fsspec.open can handle, as well as a storage_options dict kwarg.

Metadata

Metadata

Assignees

Labels

diagnosticsgood second issueClearly described, educational, but less trivial than "good first issue".

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions