Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GPU support for ndfilters & imread modules #151

Merged
merged 84 commits into from
Sep 1, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
c64224d
Add local threshold methods that laxily return dask arrays
GenevieveBuckley Mar 31, 2019
6ffad36
Add tests for local thresholding
GenevieveBuckley Mar 31, 2019
a1ccd93
All local_threshold operations should be float
GenevieveBuckley Apr 10, 2019
169e85f
Import numpy
GenevieveBuckley Apr 10, 2019
022d908
Set module name for threshold_local function
GenevieveBuckley Apr 10, 2019
5776baa
Fix docstring link formatting
GenevieveBuckley Apr 11, 2019
893dc67
Try flake8 ignore docstring (url can't be shortened)
GenevieveBuckley Apr 11, 2019
d33b4cc
threshold_local input argument type checking
GenevieveBuckley Apr 11, 2019
b90f901
Tests for ndfilters.threshold_local() functions
GenevieveBuckley Apr 11, 2019
d95300a
Make compatible with older dask versions
GenevieveBuckley Apr 11, 2019
5a5c358
Add test for invalid method name in threhsold_local()
GenevieveBuckley Apr 11, 2019
0a08f7d
Remove if statement that is never run
GenevieveBuckley Apr 11, 2019
57941bd
Older versions of dask.array.from_array() can't handle list input
GenevieveBuckley Apr 11, 2019
e05de7b
Neater way to handle gaussian sigma value in threshold_local()
GenevieveBuckley Apr 11, 2019
00c08dd
Make it work with dask and python 3.5
GenevieveBuckley Apr 11, 2019
a414ced
threshold_local 'mean' compatible with python 3.5 and 3.7
GenevieveBuckley Apr 11, 2019
9ac951e
Simplify mean filtering in threshold_local()
GenevieveBuckley Apr 11, 2019
db2308c
Remove unused import from ndfilters._thresholding
GenevieveBuckley Apr 12, 2019
22da2f6
Merge branch 'master' into dask-image-thresholds
GenevieveBuckley Aug 5, 2019
429c886
Trigger CircleCI builds to run again.
GenevieveBuckley Aug 5, 2019
176da77
GPU dispatch for convolve
GenevieveBuckley Aug 3, 2020
bcf01a2
Make convolve function generic for dispatch
GenevieveBuckley Aug 3, 2020
aac1e9a
Add forgotten numpy import
GenevieveBuckley Aug 3, 2020
4ec4360
Fix convolve dispatcher name
GenevieveBuckley Aug 3, 2020
246277d
Lazy cupyx import
GenevieveBuckley Aug 3, 2020
3a7879e
Lazy import of cupyx.scipy.ndimage
GenevieveBuckley Aug 3, 2020
bbba7c6
Fix up flake8 warnings
GenevieveBuckley Aug 3, 2020
2bd4933
Flake8 noqa added
GenevieveBuckley Aug 3, 2020
66e063f
Update minimum dask dependency to dask==1.1.1
GenevieveBuckley Aug 3, 2020
c6be63a
Remove reference to closed dask issue 6442
GenevieveBuckley Aug 4, 2020
5cd162c
Dispatch from within functions (#5)
GenevieveBuckley Aug 4, 2020
00c7a95
Merge branch 'dispatch' of https://github.com/GenevieveBuckley/dask-i…
GenevieveBuckley Aug 4, 2020
2c48b3d
John's suggestions for streamlining dispatch
GenevieveBuckley Aug 7, 2020
f6ce8ae
Use meta kwarg to fix array chunktype repr
GenevieveBuckley Aug 7, 2020
6d5c87c
This shouldn't make a difference, but checking just in case
GenevieveBuckley Aug 7, 2020
b858192
Fix up dispatch lazy registration, and output array metadata
GenevieveBuckley Aug 8, 2020
39d678d
Non-lazy cupy dispatcher
GenevieveBuckley Aug 8, 2020
dc92c49
Lazy cupy dispatching again
GenevieveBuckley Aug 8, 2020
1e06d8a
Propagate array type metadata through map_overlap
GenevieveBuckley Aug 17, 2020
ad58e45
Try being more specific about the cupy array type
GenevieveBuckley Aug 17, 2020
75981d6
Merge branch 'dispatch-edits' of https://github.com/GenevieveBuckley/…
GenevieveBuckley Aug 17, 2020
3c4b7a0
Remove commented trial & error
GenevieveBuckley Aug 17, 2020
c7f087e
flake8 - remove unused import
GenevieveBuckley Aug 17, 2020
e385895
Add docstrings for dispatch helper functions
GenevieveBuckley Aug 17, 2020
e1c76a7
Fix lazy dispatch with map_overlap
GenevieveBuckley Aug 17, 2020
8354a0f
Roll out GPU/CPU dispatch for all of dask-image
GenevieveBuckley Aug 17, 2020
1f01f83
GPU support for ndmorph submodule
GenevieveBuckley Aug 17, 2020
ab52cba
GPU support for ndfilters submodule
GenevieveBuckley Aug 17, 2020
bfbd781
Add __all__ to ndfourier and ndmeasure modules
GenevieveBuckley Aug 19, 2020
e79e059
Add option to read in images to cupy arrays
GenevieveBuckley Aug 21, 2020
2932ed6
Make arraytype a keyword only argument in imread
GenevieveBuckley Aug 21, 2020
02e173b
Flake8 formatting fix
GenevieveBuckley Aug 21, 2020
309e072
Bump minimum dask version to 2.8.1
GenevieveBuckley Aug 21, 2020
90b1841
Exclude dispatch module form coverage measurements, can't use cupy on CI
GenevieveBuckley Aug 29, 2020
999eb43
Add test forcupy arrays with dask-image convolve
GenevieveBuckley Aug 29, 2020
c5674fb
Add basic cupy tests for all ndfilter functions
GenevieveBuckley Aug 29, 2020
f5df597
Minimum cupy version is 7.7.0 or greater, for access to cupyx.scipy.n…
GenevieveBuckley Aug 29, 2020
70c2e84
Add basic cupy tests for ndmorph binary morphological operations
GenevieveBuckley Aug 29, 2020
cded99a
Skip coverage for cupy specific lines (can't test cupy on CI)
GenevieveBuckley Aug 30, 2020
0887aea
No cupyx binary_dilation or binary_erosion, dispatch to grey_dilation…
GenevieveBuckley Aug 30, 2020
acce479
Add dispatch function for numpy.array(), cupy.array(), etc.
GenevieveBuckley Aug 30, 2020
ac1b68d
Fix _get_sigmas for use with cupy arrays
GenevieveBuckley Aug 30, 2020
de44acf
Register pytest custom 'cupy' marker in pytest.ini
GenevieveBuckley Aug 30, 2020
f3dcb36
flake8 formatting in tests
GenevieveBuckley Aug 30, 2020
4881072
Merge branch 'gpu-support' of https://github.com/GenevieveBuckley/das…
GenevieveBuckley Aug 30, 2020
01ad673
ndfilter rank filter tests now passing
GenevieveBuckley Aug 30, 2020
e1d4c57
Add cupy tests for imread
GenevieveBuckley Aug 30, 2020
ec57fbb
Merge branch 'gpu-support' of github.com:GenevieveBuckley/dask-image …
GenevieveBuckley Aug 30, 2020
7618d32
Must use sigma=1 in cupy gaussian tests because our arraychunks are s…
GenevieveBuckley Aug 31, 2020
9f9067c
Use pathlib Path syntax for cupy imread test
GenevieveBuckley Aug 31, 2020
18e8b0a
Restone cupy binary_erosion, binary_dilation
GenevieveBuckley Aug 31, 2020
cd0bc7a
Restore gaussian _get_border() & _get_sigmas()
GenevieveBuckley Aug 31, 2020
fdb65ab
Remove ndmorph GPU dispatch, since binary dilation/erosion do not yet…
GenevieveBuckley Aug 31, 2020
e48f93d
Merge branch 'gpu-support' into gpu-local-threshold
GenevieveBuckley Aug 31, 2020
16bfdab
GPU ndfilters tests, add size= kwarg to generic_filter call
GenevieveBuckley Aug 31, 2020
32d4ac8
Fix kwargs in cupy generic_filter tests
GenevieveBuckley Aug 31, 2020
3eb3ec2
Fix correlate dispatching for ndfilters
GenevieveBuckley Aug 31, 2020
0d16f59
Cupy generic_filter won't support extra_arguments or extra_keywords
GenevieveBuckley Aug 31, 2020
9fe0e4b
Must use cupy.ReductionKernel as function for cupy generic_filter tests
GenevieveBuckley Aug 31, 2020
8e2ca03
Merge branch 'gpu-support' into gpu-local-threshold
GenevieveBuckley Sep 1, 2020
e344595
GPU compatible local_threshold
GenevieveBuckley Sep 1, 2020
84af72b
Remove unused 'dispatch_array'
GenevieveBuckley Sep 1, 2020
06a7ff1
Merge branch 'master' into gpu-support
GenevieveBuckley Sep 1, 2020
a24a77b
Make threshold_local() work for GPU
GenevieveBuckley Sep 1, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix lazy dispatch with map_overlap
  • Loading branch information
GenevieveBuckley committed Aug 17, 2020
commit e1c76a775fbb81f2e352680ed4d9cff212b74cfd
2 changes: 1 addition & 1 deletion dask_image/ndfilters/_conv.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ def convolve(image,
depth, boundary = _utils._get_depth_boundary(image.ndim, depth, "none")

result = image.map_overlap(
dispatch_convolve,
dispatch_convolve(image),
depth=depth,
boundary=boundary,
dtype=image.dtype,
Expand Down
7 changes: 3 additions & 4 deletions dask_image/utils/_dispatcher.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,15 +47,14 @@ def __call__(self, arg, *args, **kwargs):

@dispatch_convolve.register(np.ndarray)
def numpy_convolve(*args, **kwargs):
return scipy.ndimage.filters.convolve(*args, **kwargs)
return scipy.ndimage.filters.convolve


@dispatch_convolve.register_lazy("cupy")
def register_cupy():
import cupy
import cupy.core.core
import cupyx.scipy.ndimage

@dispatch_convolve.register(cupy.core.core.ndarray)
@dispatch_convolve.register(cupy.ndarray)
def cupy_convolve(*args, **kwargs):
return cupyx.scipy.ndimage.filters.convolve(*args, **kwargs)
return cupyx.scipy.ndimage.filters.convolve