Generalize delayed #7811

TomNicholas · 2023-05-02T18:34:26Z

A small follow-on to #7019 to allow using non-dask implementations of delayed.

(Builds off of #7019)

Closes Generalize dask.delayed calls to go through ChunkManager #7810
Tests added
User visible changes (including notable bug fixes) are documented in whats-new.rst
New functions/methods are listed in api.rst

for more information, see https://pre-commit.ci

…rray into cubed_integration

This reverts commit cdcb3fb.

for more information, see https://pre-commit.ci

Co-authored-by: Deepak Cherian <dcherian@users.noreply.github.com>

for more information, see https://pre-commit.ci

Co-authored-by: Illviljan <14371165+Illviljan@users.noreply.github.com>

for more information, see https://pre-commit.ci

Co-authored-by: Illviljan <14371165+Illviljan@users.noreply.github.com>

for more information, see https://pre-commit.ci

Co-authored-by: Illviljan <14371165+Illviljan@users.noreply.github.com>

for more information, see https://pre-commit.ci

…rray into cubed_integration

for more information, see https://pre-commit.ci

…rray into cubed_integration

…chunks arguments

TomNicholas and others added 30 commits September 10, 2022 17:44

generalise chunk methods to allow cubed

15fc2b8

[pre-commit.ci] auto fixes from pre-commit.com hooks

5e05b71

for more information, see https://pre-commit.ci

fic typing typo

cff89ee

Merge branch 'cubed_integration' of https://github.com/TomNicholas/xa…

039973b

…rray into cubed_integration

fixed circular import

60d44bc

fix some mypy errors

5ddba7e

added cubed to mypy ignore list

37d0d66

Merge branch 'main' into cubed_integration

67d7efc

simplify __array_ufunc__ check

cdcb3fb

Revert "simplify __array_ufunc__ check" as I pushed to wrong branch

73e4563

This reverts commit cdcb3fb.

update cubed array type

5995685

Merge branch 'main' into cubed_integration

46223ae

fix missed conflict

320b09f

sketch for ChunkManager adapter class

3facfd6

[pre-commit.ci] auto fixes from pre-commit.com hooks

c616a85

for more information, see https://pre-commit.ci

Remove erroneous docstring about usage of map_blocks

ecabaa4

Co-authored-by: Deepak Cherian <dcherian@users.noreply.github.com>

apply_ufunc -> apply_gufunc

e53a588

Co-authored-by: Deepak Cherian <dcherian@users.noreply.github.com>

chunk -> from_array

fe21edd

Co-authored-by: Deepak Cherian <dcherian@users.noreply.github.com>

remove staticmethods

3f6aedc

attempt to type methods of ABC

ea8f482

from_array

c49ab8e

attempt to specify types

26d1868

method for checking array type

e9b4a33

Merge branch 'main' into pr/7019

3a43b00

Update pyproject.toml

c7c9589

Merge branch 'main' into cubed_integration

fc051e3

[pre-commit.ci] auto fixes from pre-commit.com hooks

56e9d0f

for more information, see https://pre-commit.ci

fixed import errors

3b16cca

generalize .chunk method kwargs

7ac3323

used dask functions in dask chunkmanager

e732b87

TomNicholas and others added 19 commits April 17, 2023 14:42

From future import annotations

5a46294

Co-authored-by: Illviljan <14371165+Illviljan@users.noreply.github.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

d6b56c6

for more information, see https://pre-commit.ci

From functools import annotations

471d22a

Co-authored-by: Illviljan <14371165+Illviljan@users.noreply.github.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

8378f43

for more information, see https://pre-commit.ci

From future import annotations

f8b1020

Co-authored-by: Illviljan <14371165+Illviljan@users.noreply.github.com>

Merge branch 'main' into cubed_integration

907c15b

[pre-commit.ci] auto fixes from pre-commit.com hooks

11676ab

for more information, see https://pre-commit.ci

defined type for NormalizedChunks

76ce09e

Merge branch 'cubed_integration' of https://github.com/TomNicholas/xa…

127c184

…rray into cubed_integration

[pre-commit.ci] auto fixes from pre-commit.com hooks

604bbf3

for more information, see https://pre-commit.ci

standardized capitalization of ChunkManagerEntrypoint

7604594

Merge branch 'cubed_integration' of https://github.com/TomNicholas/xa…

97537dd

…rray into cubed_integration

Merge branch 'main' into cubed_integration

026fe17

ensure ruff doesn't remove import

355555f

ignore remaining typing errors stemming from unclear dask typing for …

6eac87a

…chunks arguments

Merge branch 'main' into cubed_integration

f4224f6

rename store_kwargs->chunkmanager_store_kwargs

4ec8370

missed return value

316c63d

redirect dask.delayed in open_mfdataset through ChunkManager

c183e73

github-actions bot added io topic-arrays related to flexible array support topic-backends topic-dask topic-indexing topic-rolling topic-typing topic-zarr Related to zarr storage library labels May 2, 2023

TomNicholas mentioned this pull request May 2, 2023

Generalize handling of chunked array types #7019

Merged

15 tasks

tomwhite mentioned this pull request Sep 20, 2023

Delayed-like functionality cubed-dev/cubed#311

Open

TomNicholas added the topic-chunked-arrays Managing different chunked backends, e.g. dask label Apr 23, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generalize delayed #7811

Generalize delayed #7811

TomNicholas commented May 2, 2023

Generalize delayed #7811

Are you sure you want to change the base?

Generalize delayed #7811

Conversation

TomNicholas commented May 2, 2023