Refactor haversine_distance to a header-only API #477

harrism · 2022-01-19T04:24:28Z

This is an exploration of what a refactoring of libcuspatial to a header-only API would look like. This PR refactors a single API, haversine_distance. The new API is iterator-based, and is completely independent of libcudf and libcudf_test.

The design is based on std/Thrust algorithms. It is an iterator-based API, which enables flexibility to use "fancy" iterators such as transform iterators.

There is no support for nulls. We didn't allow cudf::column inputs with NULLS in the cuDF-based API anyway.

trxcllnt · 2022-01-19T05:05:26Z

cpp/include/cuspatial/experimental/detail/haversine.hpp

@@ -0,0 +1,125 @@
+/*


Shouldn't this be named haversine.cuh?

Depends on your style. Thrust doesn't follow that convention.

Actually, in the future we may want to enable both host and device execution of these algorithms, a la Thrust.

I thought anything with CUDA C++ in it was supposed to either be .cu or .cuh? If someone includes this from a .cpp file, GCC won't know what to do with the __device__ functions?

True. Thrust uses .h for all headers, and makes sure to protect everything so that it can be portable to different backends. I don't know if we want to go that direction. I just have a particular aversion to ".cuh" used in non-detail headers (not sure why, it's just ugly I guess).

Generally my rule is that .h/.hpp files should be able to included in a host-only TU without issue.

I'd go with .cuh unless we decide to make cuSpatial work on both CPU/GPU via Thrust.

Done. It would be nice to support CPU and GPU backends but I'm not sure how feasible that is with the more complicated algorithms. Also it would have a large impact on tests, which currently use device vectors. So this is definitely future work.

cpp/tests/experimental/spatial/haversine_test.cu

cpp/include/cuspatial/experimental/haversine.hpp

jrhemstad

Since this is experimenting with a new API/library design, some bigger pictures things to consider:

Do we want to provide per-algorithm control over the temporary allocations like Thrust? Or will cuSpatial continue to depend on RMM and a user can control temp allocations via per_device_resource?

Looking at the implementation, this algorithm could conceivably run on CPUs with the thrust::host backend. Is cuSpatial interested in providing the flexibility to run on both CPU and GPU?

harrism · 2022-01-19T22:33:24Z

Do we want to provide per-algorithm control over the temporary allocations like Thrust? Or will cuSpatial continue to depend on RMM and a user can control temp allocations via per_device_resource?

yeah, this is what I meant when I asked you if you thought it should take an exec policy argument. This algorithm doesn't allocate, so I removed the MR argument. But if it did, it could get the allocator from the exec_policy like Thrust does.

harrism · 2022-01-19T22:36:35Z

Looking at the implementation, this algorithm could conceivably run on CPUs with the thrust::host backend. Is cuSpatial interested in providing the flexibility to run on both CPU and GPU?

That requires further discussion. We have not received any requirements for this yet that I know of.

Co-authored-by: Jake Hemstad <jhemstad@nvidia.com> Co-authored-by: Paul Taylor <paul.e.taylor@me.com>

…ial into fea-header-only-haversine

cpp/include/cuspatial/experimental/type_utils.hpp

cpp/include/cuspatial/types.hpp

cpp/include/cuspatial/experimental/detail/haversine.hpp

harrism · 2022-04-07T01:38:41Z

I made a significant changed based on my experimentation with refactoring other APIs which use both LonLat and XY coordinates. I replaced location_2d with a more generic vec_2d with x and y members. This can be used for both LonLat and XY which reduces code duplication especially in type utilities and arithmetic operators.

cpp/doc/libcuspatial_refactoring_guide.md

cpp/include/cuspatial/types.hpp

cpp/doc/libcuspatial_refactoring_guide.md

isVoid

lgtm. 3 small nits in doc.

cpp/doc/libcuspatial_refactoring_guide.md

vyasr

CMake approval

jrhemstad · 2022-04-27T14:18:01Z

cpp/doc/libcuspatial_refactoring_guide.md

+  3. Longitude/Latitude data is passed as array of structures, using the `cuspatial::lonlat_2d`
+     type (include/cuspatial/types.hpp)


The code doesn't currently tell me this. Maybe add a static_assert to demonstrate it.

Concepts would be so nice for this :(

I don't understand, I already have that static assert.

static_assert( std::conjunction_v<std::is_same<lonlat_2d<T>, Location>, std::is_same<lonlat_2d<T>, LocationB>>, "Inputs must be cuspatial::lonlat_2d");

Mentioned the static asserts in the refactoring guide.

Reading the code on lines33-43 and then reading on it describes the important things about this code...

"There are a few key points to notice...

Longitude/Latitude data is passed as array of structures, using the cuspatial::lonlat_2d type"

My only point is that it's impossible to "notice" this point just from the code as is on 33-43.

Got it. I added this clarification to that point:

Longitude/Latitude data is passed as array of structures, using the `cuspatial::lonlat_2d` type (include/cuspatial/types.hpp). This is enforced using a `static_assert` in the function body (discussed later).

cpp/doc/libcuspatial_refactoring_guide.md

harrism · 2022-04-28T00:22:26Z

Thanks everyone for reviewing! @gpucibot merge

harrism · 2022-04-28T00:59:47Z

@gpucibot merge

Following #477, implements the header-only API for `cuspatial::lonlat_to_cartesian` and implements existing C++ API on top of it. Follows the refactoring guide introduced in #477. Note this branch is based on the branch for #477 so diff includes all changes from that PR as well. Once #477 is merged this PR will simplify a lot. Authors: - Mark Harris (https://github.com/harrism) Approvers: - Vyas Ramasubramani (https://github.com/vyasr) - Michael Wang (https://github.com/isVoid) URL: #514

harrism added 2 commits January 19, 2022 15:19

Create header-only refactoring of cuspatial::haversine_distance

314580d

Merge branch 'branch-22.04' into fea-header-only-haversine

0736356

github-actions bot added cmake Related to CMake code or build configuration libcuspatial Relates to the cuSpatial C++ library labels Jan 19, 2022

harrism added breaking Breaking change improvement Improvement / enhancement to an existing function labels Jan 19, 2022

harrism added the 5 - DO NOT MERGE Hold off on merging; see PR for details label Jan 19, 2022

trxcllnt reviewed Jan 19, 2022

View reviewed changes

cpp/tests/experimental/spatial/haversine_test.cu Outdated Show resolved Hide resolved

trxcllnt reviewed Jan 19, 2022

View reviewed changes

cpp/tests/experimental/spatial/haversine_test.cu Outdated Show resolved Hide resolved

jrhemstad reviewed Jan 19, 2022

View reviewed changes

cpp/include/cuspatial/experimental/haversine.hpp Outdated Show resolved Hide resolved

jrhemstad reviewed Jan 19, 2022

View reviewed changes

cpp/include/cuspatial/experimental/haversine.hpp Outdated Show resolved Hide resolved

jrhemstad reviewed Jan 19, 2022

View reviewed changes

cpp/include/cuspatial/experimental/haversine.hpp Outdated Show resolved Hide resolved

jrhemstad reviewed Jan 19, 2022

View reviewed changes

harrism and others added 5 commits January 20, 2022 11:10

Apply suggestions from code review

5830b66

Co-authored-by: Jake Hemstad <jhemstad@nvidia.com> Co-authored-by: Paul Taylor <paul.e.taylor@me.com>

require RandomAccessIterator

4c16cd6

Merge branch 'fea-header-only-haversine' of github.com:harrism/cuspat…

7580661

…ial into fea-header-only-haversine

Convert haversine API to use AOS inputs.

073e2d7

Revert cosmetic changes to top-level haversine.hpp

e37d61e

jrhemstad reviewed Feb 16, 2022

View reviewed changes

cpp/include/cuspatial/experimental/type_utils.hpp Outdated Show resolved Hide resolved

jrhemstad reviewed Feb 16, 2022

View reviewed changes

cpp/include/cuspatial/types.hpp Outdated Show resolved Hide resolved

jrhemstad reviewed Feb 16, 2022

View reviewed changes

cpp/include/cuspatial/types.hpp Outdated Show resolved Hide resolved

trxcllnt reviewed Feb 16, 2022

View reviewed changes

cpp/include/cuspatial/experimental/detail/haversine.hpp Outdated Show resolved Hide resolved

harrism added 4 commits February 17, 2022 08:46

Align location_2d and remove unused location_3d and coord_2d.

9d8e3eb

__device__ only

7f2bbad

"" --> <>

2448677

Remove unused macro.

0d954e0

Simplify coordinate types to a single vec_2d

c208144

Clean up haversine_test.cu includes

21db279

harrism mentioned this pull request Apr 7, 2022

Refactor lonlat_to_cartesian to header-only API #514

Merged

jrhemstad reviewed Apr 8, 2022

View reviewed changes

cpp/doc/libcuspatial_refactoring_guide.md Outdated Show resolved Hide resolved

isVoid reviewed Apr 13, 2022

View reviewed changes

cpp/doc/libcuspatial_refactoring_guide.md Outdated Show resolved Hide resolved

cpp/include/cuspatial/types.hpp Show resolved Hide resolved

isVoid reviewed Apr 13, 2022

View reviewed changes

cpp/doc/libcuspatial_refactoring_guide.md Show resolved Hide resolved

harrism added 4 commits April 26, 2022 11:45

type-safe vectors

be1226c

Fix typo

8134f12

vec_2d --> lonlat_2d in docs

42f909e

Merge branch 'branch-22.06' into fea-header-only-haversine

e5cb703

isVoid approved these changes Apr 27, 2022

View reviewed changes

cpp/doc/libcuspatial_refactoring_guide.md Outdated Show resolved Hide resolved

cpp/doc/libcuspatial_refactoring_guide.md Outdated Show resolved Hide resolved

cpp/doc/libcuspatial_refactoring_guide.md Outdated Show resolved Hide resolved

Review suggestions

0b78d87

vyasr approved these changes Apr 27, 2022

View reviewed changes

jrhemstad reviewed Apr 27, 2022

View reviewed changes

cpp/doc/libcuspatial_refactoring_guide.md Outdated Show resolved Hide resolved

jrhemstad reviewed Apr 27, 2022

View reviewed changes

cpp/doc/libcuspatial_refactoring_guide.md Outdated Show resolved Hide resolved

Clarified documentation / refactoring guide.

17569c6

jrhemstad approved these changes Apr 27, 2022

View reviewed changes

style

d5ef3b7

harrism added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed 3 - Ready for Review Ready for review by team labels Apr 28, 2022

harrism mentioned this pull request Apr 28, 2022

C++ pairwise linestring distance #510

Merged

rapids-bot bot merged commit 1986a18 into rapidsai:branch-22.06 Apr 28, 2022

harrism added this to the header-only C++ API milestone May 12, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor haversine_distance to a header-only API #477

Refactor haversine_distance to a header-only API #477

harrism commented Jan 19, 2022 •

edited

Loading

trxcllnt Jan 19, 2022

harrism Jan 19, 2022

harrism Feb 16, 2022

trxcllnt Feb 20, 2022

harrism Mar 29, 2022

jrhemstad Mar 29, 2022

harrism Mar 31, 2022

jrhemstad left a comment

harrism commented Jan 19, 2022

harrism commented Jan 19, 2022

harrism commented Apr 7, 2022

isVoid left a comment

vyasr left a comment

jrhemstad Apr 27, 2022

jrhemstad Apr 27, 2022

harrism Apr 27, 2022

harrism Apr 27, 2022

jrhemstad Apr 27, 2022

harrism Apr 28, 2022

harrism commented Apr 28, 2022

harrism commented Apr 28, 2022

		3. Longitude/Latitude data is passed as array of structures, using the `cuspatial::lonlat_2d`
		type (include/cuspatial/types.hpp)

Refactor haversine_distance to a header-only API #477

Refactor haversine_distance to a header-only API #477

Conversation

harrism commented Jan 19, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jrhemstad left a comment

Choose a reason for hiding this comment

harrism commented Jan 19, 2022

harrism commented Jan 19, 2022

harrism commented Apr 7, 2022

isVoid left a comment

Choose a reason for hiding this comment

vyasr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

harrism commented Apr 28, 2022

harrism commented Apr 28, 2022

harrism commented Jan 19, 2022 •

edited

Loading