Fix 23.08 -> 23.10 automerge #1251
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Updates the devcontainers to v23.08. The latest images now include Python and automatically generate the VSCode workspace on startup, so I was able to simplify the cuSpatial devcontainers a bit. Authors: - Paul Taylor (https://github.com/trxcllnt) - H. Thomson Comer (https://github.com/thomcom) Approvers: - Mark Harris (https://github.com/harrism) - H. Thomson Comer (https://github.com/thomcom) - AJ Schmidt (https://github.com/ajschmidt8) URL: rapidsai#1214
Closes rapidsai#1215 Depends on rapidsai#1191 New benchmark for C++ API for WGS84->UTM transform. Below results were run on a single CPU core for Proj (ignore gbench's claims to the contrary), and on a single H100 80GB GPU. Machine was a DGX H100. ``` (rapids) coder ➜ ~/cuspatial $ CUDA_VISIBLE_DEVICES=1 cpp/build/latest/cuproj/benchmarks/WGS_TO_UTM_BENCH 2023-08-02T00:08:52+00:00 Running cpp/build/latest/cuproj/benchmarks/WGS_TO_UTM_BENCH Run on (224 X 3800 MHz CPU s) CPU Caches: L1 Data 48 KiB (x112) L1 Instruction 32 KiB (x112) L2 Unified 2048 KiB (x112) L3 Unified 107520 KiB (x2) Load Average: 1.70, 6.21, 14.43 --------------------------------------------------------------------------------------------------------------------- Benchmark Time CPU Iterations UserCounters... --------------------------------------------------------------------------------------------------------------------- proj_utm_benchmark/forward_double/100 0.013 ms 0.013 ms 57756 items_per_second=7.95314M/s proj_utm_benchmark/forward_double/1000 0.114 ms 0.114 ms 6118 items_per_second=8.73369M/s proj_utm_benchmark/forward_double/10000 1.24 ms 1.24 ms 588 items_per_second=8.07697M/s proj_utm_benchmark/forward_double/100000 12.0 ms 12.0 ms 58 items_per_second=8.35593M/s proj_utm_benchmark/forward_double/1000000 120 ms 120 ms 6 items_per_second=8.36301M/s proj_utm_benchmark/forward_double/10000000 1213 ms 1213 ms 1 items_per_second=8.24563M/s proj_utm_benchmark/forward_double/100000000 11977 ms 11976 ms 1 items_per_second=8.35038M/s proj_utm_benchmark/forward_double/1000000000 119680 ms 119677 ms 1 items_per_second=8.35579M/s cuproj_utm_benchmark/forward_float/100/manual_time 0.011 ms 0.040 ms 66283 items_per_second=9.47605M/s cuproj_utm_benchmark/forward_float/1000/manual_time 0.012 ms 0.041 ms 56799 items_per_second=81.2019M/s cuproj_utm_benchmark/forward_float/10000/manual_time 0.013 ms 0.042 ms 55571 items_per_second=793.482M/s cuproj_utm_benchmark/forward_float/100000/manual_time 0.013 ms 0.042 ms 53048 items_per_second=7.5779G/s cuproj_utm_benchmark/forward_float/1000000/manual_time 0.027 ms 0.056 ms 25842 items_per_second=36.9063G/s cuproj_utm_benchmark/forward_float/10000000/manual_time 0.170 ms 0.198 ms 4130 items_per_second=58.8554G/s cuproj_utm_benchmark/forward_float/100000000/manual_time 1.60 ms 1.62 ms 439 items_per_second=62.6581G/s cuproj_utm_benchmark/forward_float/1000000000/manual_time 15.9 ms 15.9 ms 44 items_per_second=63.0518G/s cuproj_utm_benchmark/forward_double/100/manual_time 0.012 ms 0.041 ms 57960 items_per_second=8.30297M/s cuproj_utm_benchmark/forward_double/1000/manual_time 0.015 ms 0.044 ms 47605 items_per_second=68.0791M/s cuproj_utm_benchmark/forward_double/10000/manual_time 0.015 ms 0.044 ms 47353 items_per_second=676.864M/s cuproj_utm_benchmark/forward_double/100000/manual_time 0.016 ms 0.045 ms 43394 items_per_second=6.19684G/s cuproj_utm_benchmark/forward_double/1000000/manual_time 0.042 ms 0.070 ms 16621 items_per_second=23.7599G/s cuproj_utm_benchmark/forward_double/10000000/manual_time 0.304 ms 0.332 ms 2302 items_per_second=32.863G/s cuproj_utm_benchmark/forward_double/100000000/manual_time 2.93 ms 2.96 ms 240 items_per_second=34.087G/s cuproj_utm_benchmark/forward_double/1000000000/manual_time 29.3 ms 29.3 ms 24 items_per_second=34.13G/s Authors: - Mark Harris (https://github.com/harrism) Approvers: - Michael Wang (https://github.com/isVoid) URL: rapidsai#1216
The previous logic for handling the case of a LineString that shares boundary points with a Polygon and has no interior points was wrong, it was just finding the center point of such a LineString and testing that for containment. The correct logic is as follows: A LineString is contained in a polygon if: The sum of the points of the LineString that are contained in the Polygon, plus the number of vertices of the linestring that are on the polygon boundary, is equal to the length of the LineString less the number of points in the LineString that are contained in the boundary of the Polygon. ``` polygon.contains(linestring) = contains + point_intersection_count == rhs.sizes - linestring_intersection_count ``` The new logic counts the number of point intersections and linestring intersection that the LineString shares with the polygon. Point intersections occur when the LineString shares an edge point with the Polygon exterior, implying either a LineString segment that approaches from the exterior or the interior. An interior point will be counted, an exterior point will be left in the size of the LineString during comparison, with the final count being less than the size of the LineString. The hardest part of this implementation was in writing `_pli_features_rebuild_offsets` that takes the results of a `pairwise_linestring_intersection` result and splits them into row-appropriate points in one set and linestrings in another set. This is potentially a good place to move the logic into C++, though I don't think it will be a major profiling issue initially. Also improves `touches` and `crosses` where the predicate had been written too tightly to the test cases. --- In addition, a bug with `pairwise_multipoint_equals_count` is fixed when `lhs` contains more than 1 multipoints, but all multipoints are empty. The bug causes the API to raise a cuda invalid configuration error. Authors: - H. Thomson Comer (https://github.com/thomcom) - Michael Wang (https://github.com/isVoid) Approvers: - Mark Harris (https://github.com/harrism) - Michael Wang (https://github.com/isVoid) URL: rapidsai#1186
Closes rapidsai#1213 This benchmark is run on a DGX H100. See the cuproj_benchmark.ipynb included with this PR. PyProj results use a single Xeon core from this machine, and cuProj results use a single H100 GPU. cuProj speedup for double precision, data on device (see notebook): ``` cuProj Speedup for 1,000,000,000 points: 4103.17x ```  Note this PR also includes a number of C++ changes that were necessary to enable Python/Cython bindings, and/or to enable compatibility with PyProj (e.g. refactored EPSG string parsing class). TODO: - [x] Support arrays - [x] More comprehensive tests, including a grid of coordinates - [x] Test inverse transforms - [x] Support `__cuda_array_interface__` - [x] Support 32-bit floats - [x] Update CI scripts to build cuProj Python bindings and run pytests - [x] ~Documentation~ -- in follow-up PR rapidsai#1237 - [ ] Support interleaved coordinates - [x] Support axis order the way PyProj does (e.g. not always lat, lon) (actually, this does now work the way PyProj does for the transformation we support, which requires (lat, lon) ordering for WGS84, and outputs (Easting, Northing) order. - [ ] But we could add support for `always_xy` parameter that PyProj has. - [x] Support integer epsg code arguments in `Transformer.from_crs` - [x] Support mixed integer and string epsg code arguments in `Transformer.from_crs` - [x] Support tuples of ("EPSG", code) in `Transformer.from_crs` - [ ] Fix projection factory to not return a raw pointer - [x] cuprojshim clang-format - [x] Benchmark notebook Authors: - Mark Harris (https://github.com/harrism) - Michael Wang (https://github.com/isVoid) - AJ Schmidt (https://github.com/ajschmidt8) Approvers: - H. Thomson Comer (https://github.com/thomcom) - Ben Jarmak (https://github.com/jarmak-nv) - AJ Schmidt (https://github.com/ajschmidt8) - Vyas Ramasubramani (https://github.com/vyasr) - Michael Wang (https://github.com/isVoid) - Bradley Dice (https://github.com/bdice) URL: rapidsai#1217
…eries (rapidsai#1212) This PR aims to solve point-in-polygon queries using three-dimensional cartesian coordinates on spherical surfaces. Users may prefer to use that approach for more accurate computations. This PR "closes rapidsai#1202 " Authors: - https://github.com/ayasar70 - Mark Harris (https://github.com/harrism) - Michael Wang (https://github.com/isVoid) Approvers: - Michael Wang (https://github.com/isVoid) - Mark Harris (https://github.com/harrism) URL: rapidsai#1212
closes rapidsai#1205 This PR updates the cuSpatial Readme per requirements in rapidsai#1205 (as well as the update-version.sh). Depends on rapidsai#1237 and rapidsai#1217 Authors: - Ben Jarmak (https://github.com/jarmak-nv) Approvers: - Mark Harris (https://github.com/harrism) URL: rapidsai#1232
Adds doxygen configuration for libcuproj, and Python documentation. Depends on rapidsai#1217 Fixes rapidsai#1239 Authors: - Mark Harris (https://github.com/harrism) - Michael Wang (https://github.com/isVoid) - AJ Schmidt (https://github.com/ajschmidt8) Approvers: - Ben Jarmak (https://github.com/jarmak-nv) - H. Thomson Comer (https://github.com/thomcom) - Paul Taylor (https://github.com/trxcllnt) - Michael Wang (https://github.com/isVoid) - Ray Douglass (https://github.com/raydouglass) URL: rapidsai#1237
Follow up to rapidsai#1217 to enable wheels testing for `cuproj` for nightly tests. Should fix issue seen [here](https://github.com/rapidsai/cuspatial/actions/runs/5748659541/job/15581971500) Authors: - Ray Douglass (https://github.com/raydouglass) Approvers: - AJ Schmidt (https://github.com/ajschmidt8) - Vyas Ramasubramani (https://github.com/vyasr) URL: rapidsai#1250
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
github-actions
bot
added
conda
thomcom
force-pushed
the
bug/fix-23.08-23.10-automerge
branch
2 times, most recently
from
efeef92
to
96e8827
Compare
thomcom
added
improvement
thomcom
force-pushed
the
bug/fix-23.08-23.10-automerge
branch
from
ccb7eec
to
203dd0e
Compare
Closes rapidsai#1138 Closes rapidsai#1141 [here](https://github.com/rapidsai/cuspatial/pull/1156/files#diff-c522c9afb3364b1aed2b2589c0d0c260dbc634bc54844536b1d382cecb92bf29R112) Depends on rapidsai#1152 Depends on rapidsai#1064 Please direct your attention [to the notebook](https://github.com/rapidsai/cuspatial/pull/1156/files#diff-cc4c516f63efa822793d75315c1b28a04bad6c9efc6fd2bdcac5cc30b05d14dd) since the dependencies and delayed state of CI issues over this week have put a lot of files into this PR. This notebook demonstrates cuSpatial's new binary predicates on large datasets, benchmarking and comparing against the host implementation on GeoPandas. In order to support the large inputs for these comparisons I had to reactivate the `pairwise_point_in_polygon` functionality that I'd previously written off. This is because quadtree doesn't support large N for NxN operations, since it is many-to-many, and brute-force would require a huge number of iterations to support such large dataframes. There are some more optimizations that can be made to speed up `pairwise_point_in_polygon`, but the algorithm itself isn't sufficiently fast. It is detailed fairly closely in the notebook. Please take a look and let's have some conversations about steps forward. Authors: - H. Thomson Comer (https://github.com/thomcom) Approvers: - Michael Wang (https://github.com/isVoid) - Mark Harris (https://github.com/harrism) - Ray Douglass (https://github.com/raydouglass) - AJ Schmidt (https://github.com/ajschmidt8) URL: rapidsai#1156
thomcom
force-pushed
the
bug/fix-23.08-23.10-automerge
branch
from
203dd0e
to
278de03
Compare
ajschmidt8
approved these changes
Aug 7, 2023
3 tasks
rapids-bot bot
pushed a commit
that referenced
this pull request
Aug 19, 2023
The yaml files in conda/environments were accidentally deleted in #1251 . This PR adds them back. Authors: - Mark Harris (https://github.com/harrism) Approvers: - Ray Douglass (https://github.com/raydouglass) - H. Thomson Comer (https://github.com/thomcom) URL: #1261
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Checklist