Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make list_chunkmanagers more resilient to broken entrypoints #8736

Merged
merged 4 commits into from
Mar 13, 2024

Conversation

hmaarrfk
Copy link
Contributor

@hmaarrfk hmaarrfk commented Feb 11, 2024

As I'm a developing my custom chunk manager, I'm often checking out between my development branch and production branch breaking the entrypoint.

This made xarray impossible to import unless I re-ran pip install -e . -vv which is somewhat tiring.

This should help xarray be more resilient in other software's bugs in case they install malformed entrypoints

Example:

>>> from xarray.core.parallelcompat import list_chunkmanagers

>>> list_chunkmanagers()
<ipython-input-3-19326f4950bc>:1: UserWarning: Failed to load entrypoint MyChunkManager due to No module named 'my.array._chunkmanager'. Skipping.
  list_chunkmanagers()
{'dask': <xarray.core.daskmanager.DaskManager at 0x7f5b826231c0>}

Thank you for considering.

  • Closes #xxxx
  • Tests added
  • User visible changes (including notable bug fixes) are documented in whats-new.rst
  • New functions/methods are listed in api.rst

This is mostly a quality of life thing for developers, I don't see this as a user visible change.

@hmaarrfk hmaarrfk force-pushed the more_resilient_to_bad_entrypoints branch from 1fc7460 to 034dc96 Compare February 11, 2024 21:39
@TomNicholas TomNicholas added the topic-chunked-arrays Managing different chunked backends, e.g. dask label Feb 12, 2024
@TomNicholas
Copy link
Member

This looks great! Is it possible to test this error message somehow?

@hmaarrfk hmaarrfk force-pushed the more_resilient_to_bad_entrypoints branch 2 times, most recently from bfb7cdd to 54e3324 Compare February 13, 2024 00:08
@hmaarrfk
Copy link
Contributor Author

and rebased. please see suggested test.

@TomNicholas
Copy link
Member

Amazing! I've just pushed a commit to try to appease mypy.

@TomNicholas
Copy link
Member

You can also put an entry in what's new if you want :)

@TomNicholas
Copy link
Member

hmm - maybe adding -> None to utils.emit_user_level_warning will work?

@hmaarrfk
Copy link
Contributor Author

You can also put an entry in what's new if you want :

let’s keep the scope of this fix small. I don’t think it warrants much of a message.

let’s keep the message for when we tackle the underlying challenge that brought me here ^_^

@hmaarrfk hmaarrfk force-pushed the more_resilient_to_bad_entrypoints branch from 4ede31d to 83fa5e8 Compare February 13, 2024 02:01
hmaarrfk and others added 3 commits February 12, 2024 21:01
As I'm a developing my custom chunk manager, I'm often checking out
between my development branch and production branch breaking the
entrypoint.

This made xarray impossible to import unless I re-ran `pip install -e . -vv`
which is somewhat tiring.
@Illviljan Illviljan added the plan to merge Final call for comments label Mar 10, 2024
@dcherian dcherian merged commit a3f7774 into pydata:main Mar 13, 2024
30 of 33 checks passed
dcherian added a commit to dcherian/xarray that referenced this pull request Mar 15, 2024
* main: (31 commits)
  correctly encode/decode _FillValues/missing_values/dtypes for packed data (pydata#8713)
  Expand use of `.oindex` and `.vindex` (pydata#8790)
  Return a dataclass from Grouper.factorize (pydata#8777)
  [skip-ci] Fix upstream-dev env (pydata#8839)
  Add dask-expr for windows envs (pydata#8837)
  [skip-ci] Add dask-expr dependency to doc.yml (pydata#8835)
  Add `dask-expr` to environment-3.12.yml (pydata#8827)
  Make list_chunkmanagers more resilient to broken entrypoints (pydata#8736)
  Do not attempt to broadcast when global option ``arithmetic_broadcast=False`` (pydata#8784)
  try to get the `upstream-dev` CI to complete again (pydata#8823)
  Bump the actions group with 1 update (pydata#8818)
  Update documentation for clarity (pydata#8817)
  DOC: link to zarr.convenience.consolidate_metadata (pydata#8816)
  Refactor Grouper objects (pydata#8776)
  Grouper object design doc (pydata#8510)
  Bump the actions group with 2 updates (pydata#8804)
  tokenize() should ignore difference between None and {} attrs (pydata#8797)
  fix: remove Coordinate from __all__ in xarray/__init__.py (pydata#8791)
  Fix non-nanosecond casting behavior for `expand_dims` (pydata#8782)
  Migrate treenode module. (pydata#8757)
  ...
dcherian added a commit to dcherian/xarray that referenced this pull request Mar 16, 2024
* main: (42 commits)
  correctly encode/decode _FillValues/missing_values/dtypes for packed data (pydata#8713)
  Expand use of `.oindex` and `.vindex` (pydata#8790)
  Return a dataclass from Grouper.factorize (pydata#8777)
  [skip-ci] Fix upstream-dev env (pydata#8839)
  Add dask-expr for windows envs (pydata#8837)
  [skip-ci] Add dask-expr dependency to doc.yml (pydata#8835)
  Add `dask-expr` to environment-3.12.yml (pydata#8827)
  Make list_chunkmanagers more resilient to broken entrypoints (pydata#8736)
  Do not attempt to broadcast when global option ``arithmetic_broadcast=False`` (pydata#8784)
  try to get the `upstream-dev` CI to complete again (pydata#8823)
  Bump the actions group with 1 update (pydata#8818)
  Update documentation for clarity (pydata#8817)
  DOC: link to zarr.convenience.consolidate_metadata (pydata#8816)
  Refactor Grouper objects (pydata#8776)
  Grouper object design doc (pydata#8510)
  Bump the actions group with 2 updates (pydata#8804)
  tokenize() should ignore difference between None and {} attrs (pydata#8797)
  fix: remove Coordinate from __all__ in xarray/__init__.py (pydata#8791)
  Fix non-nanosecond casting behavior for `expand_dims` (pydata#8782)
  Migrate treenode module. (pydata#8757)
  ...
dcherian added a commit to kmsquire/xarray that referenced this pull request Mar 21, 2024
* upstream/main: (765 commits)
  increase typing annotations coverage in `xarray/core/indexing.py` (pydata#8857)
  pandas 3 MultiIndex fixes (pydata#8847)
  FIX: adapt handling of copy keyword argument in scipy backend for numpy >= 2.0dev (pydata#8851)
  FIX: do not cast _FillValue/missing_value in CFMaskCoder if _Unsigned is provided (pydata#8852)
  Implement setitem syntax for `.oindex` and `.vindex` properties (pydata#8845)
  Support pandas copy-on-write behaviour (pydata#8846)
  correctly encode/decode _FillValues/missing_values/dtypes for packed data (pydata#8713)
  Expand use of `.oindex` and `.vindex` (pydata#8790)
  Return a dataclass from Grouper.factorize (pydata#8777)
  [skip-ci] Fix upstream-dev env (pydata#8839)
  Add dask-expr for windows envs (pydata#8837)
  [skip-ci] Add dask-expr dependency to doc.yml (pydata#8835)
  Add `dask-expr` to environment-3.12.yml (pydata#8827)
  Make list_chunkmanagers more resilient to broken entrypoints (pydata#8736)
  Do not attempt to broadcast when global option ``arithmetic_broadcast=False`` (pydata#8784)
  try to get the `upstream-dev` CI to complete again (pydata#8823)
  Bump the actions group with 1 update (pydata#8818)
  Update documentation for clarity (pydata#8817)
  DOC: link to zarr.convenience.consolidate_metadata (pydata#8816)
  Refactor Grouper objects (pydata#8776)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
plan to merge Final call for comments topic-chunked-arrays Managing different chunked backends, e.g. dask
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants