Branch 23.06 merge 23.04 (2) #1035
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…rapidsai#926) This PR implements binary predicates that depend only on equality, which is implemented here using columnar comparison in python. I'm playing with benchmarks of this feature now. On only Point geometries, we begin to outperform geopandas at 50k points, with 60x performance at 10m points. Authors: - H. Thomson Comer (https://github.com/thomcom) Approvers: - Michael Wang (https://github.com/isVoid) URL: rapidsai#926
This PR closes rapidsai#756 , add `pairwise_point_polygon_distance` for python. Depend on rapidsai#984 rapidsai#976 Authors: - Michael Wang (https://github.com/isVoid) Approvers: - H. Thomson Comer (https://github.com/thomcom) URL: rapidsai#988
Similarly to these [cudf](rapidsai/cudf#12819) and [cuml](rapidsai/cuml#5246) PRs, this PR adds an entry to `.pre-commit-config.yaml` to run the [dependency-file-generator](https://github.com/rapidsai/dependency-file-generator). It also adds an argument to the `rapidsai/shared-action-workflows/.github/workflows/checks.yaml` shared workflow to disable the `dependency-file-generator` from running in that shared workflow. This avoids having the `dependency-file-generator` run in two places since pre-commit is run in CI [here](https://github.com/rapidsai/cuspatial/blob/branch-23.04/ci/check_style.sh#L23). Authors: - AJ Schmidt (https://github.com/ajschmidt8) Approvers: - Ray Douglass (https://github.com/raydouglass) URL: rapidsai#1008
This PR adds a notebook that demonstrate the use of quadtree PiP with a custom `QuadTree` structure and joins dataframe. We can use this PR to discuss the possibility of adding the custom structure to the codebase. This also updates the dependency list to include notebook environment in "all" targeted conda envs. Authors: - Michael Wang (https://github.com/isVoid) Approvers: - H. Thomson Comer (https://github.com/thomcom) - Mark Harris (https://github.com/harrism) - AJ Schmidt (https://github.com/ajschmidt8) URL: rapidsai#919
Closes rapidsai#985 Also contains cleanup of docs for other spatial join functions, correct ordering of stream and MR parameters, and adds missing C++17 property from tests cmake configuration. Authors: - Mark Harris (https://github.com/harrism) Approvers: - Michael Wang (https://github.com/isVoid) - Paul Taylor (https://github.com/trxcllnt) URL: rapidsai#979
Fixes rapidsai#1017. Reduces C++ gtest total time (on my PC) from 47.9 seconds to 20.08 seconds. Several tests were running large datasets and combinations of size parameters that would be better to run as benchmarks rather than gtests. Reducing these by a factor of 10-100 saves a lot of development time and still exercises the code. In the case of `HausdorffTest/1.10000Spaces10Points (4850 ms)`, reducing it to 1000 spaces, 10 points reduced the time by nearly 100x, likely because it's $O(N^2)$. I modified any test that used close to 1s or more total time, since most column-API tests use under that, and most header-only tests use under 0.2s. | Test | Time Before (s) | Time After (s) | Speedup | |---|---|---|---| | DERIVE_TRAJECTORIES_TEST_EXP | 14.49 | 0.27 | 53.7x | | HAUSDORFF_TEST_EXP | 9.21 | 0.26 | 35.4x | | UTILITY_TEST | 1.86 | 0.30 | 6.2x | | POINT_BOUNDING_BOXES_TEST_EXP | 1.35 | 0.15 | 9x | | TRAJECTORY_DISTANCES_AND_SPEEDS_TEST_EXP | 0.80 | 0.13 | 6.2x | | TOTAL | 47.9 | 20.08 | 2.4x | Before: ``` (rapids) coder ➜ ~/cuspatial/cpp/build/release $ ninja test [0/1] Running tests... Test project /home/coder/cuspatial/cpp/build/release Start 1: SINUSOIDAL_PROJECTION_TEST 1/45 Test #1: SINUSOIDAL_PROJECTION_TEST ................. Passed 0.81 sec Start 2: HAVERSINE_TEST 2/45 Test #2: HAVERSINE_TEST ............................. Passed 0.77 sec Start 3: HAUSDORFF_TEST 3/45 Test #3: HAUSDORFF_TEST ............................. Passed 0.75 sec Start 4: JOIN_POINT_TO_LINESTRING_SMALL_TEST 4/45 Test rapidsai#4: JOIN_POINT_TO_LINESTRING_SMALL_TEST ........ Passed 0.73 sec Start 5: JOIN_POINT_IN_POLYGON_TEST 5/45 Test rapidsai#5: JOIN_POINT_IN_POLYGON_TEST ................. Passed 0.79 sec Start 6: POINT_IN_POLYGON_TEST 6/45 Test rapidsai#6: POINT_IN_POLYGON_TEST ...................... Passed 0.80 sec Start 7: PAIRWISE_POINT_IN_POLYGON_TEST 7/45 Test rapidsai#7: PAIRWISE_POINT_IN_POLYGON_TEST ............. Passed 0.76 sec Start 8: POINT_QUADTREE_TEST 8/45 Test rapidsai#8: POINT_QUADTREE_TEST ........................ Passed 0.76 sec Start 9: LINESTRING_BOUNDING_BOXES_TEST 9/45 Test rapidsai#9: LINESTRING_BOUNDING_BOXES_TEST ............. Passed 0.76 sec Start 10: POLYGON_BOUNDING_BOXES_TEST 10/45 Test rapidsai#10: POLYGON_BOUNDING_BOXES_TEST ................ Passed 0.80 sec Start 11: POINT_DISTANCE_TEST 11/45 Test rapidsai#11: POINT_DISTANCE_TEST ........................ Passed 0.79 sec Start 12: POINT_LINESTRING_DISTANCE_TEST 12/45 Test rapidsai#12: POINT_LINESTRING_DISTANCE_TEST ............. Passed 0.78 sec Start 13: LINESTRING_DISTANCE_TEST 13/45 Test rapidsai#13: LINESTRING_DISTANCE_TEST ................... Passed 0.78 sec Start 14: POINT_POLYGON_DISTANCE_TEST 14/45 Test rapidsai#14: POINT_POLYGON_DISTANCE_TEST ................ Passed 0.76 sec Start 15: LINESTRING_INTERSECTION_TEST 15/45 Test rapidsai#15: LINESTRING_INTERSECTION_TEST ............... Passed 0.83 sec Start 16: POINT_LINESTRING_NEAREST_POINT_TEST 16/45 Test rapidsai#16: POINT_LINESTRING_NEAREST_POINT_TEST ........ Passed 0.77 sec Start 17: QUADTREE_POLYGON_FILTERING_TEST 17/45 Test rapidsai#17: QUADTREE_POLYGON_FILTERING_TEST ............ Passed 0.79 sec Start 18: QUADTREE_LINESTRING_FILTERING_TEST 18/45 Test rapidsai#18: QUADTREE_LINESTRING_FILTERING_TEST ......... Passed 0.76 sec Start 19: TRAJECTORY_DISTANCES_AND_SPEEDS_TEST 19/45 Test rapidsai#19: TRAJECTORY_DISTANCES_AND_SPEEDS_TEST ....... Passed 0.79 sec Start 20: DERIVE_TRAJECTORIES_TEST 20/45 Test rapidsai#20: DERIVE_TRAJECTORIES_TEST ................... Passed 0.76 sec Start 21: TRAJECTORY_BOUNDING_BOXES_TEST 21/45 Test rapidsai#21: TRAJECTORY_BOUNDING_BOXES_TEST ............. Passed 0.75 sec Start 22: SPATIAL_WINDOW_POINT_TEST 22/45 Test rapidsai#22: SPATIAL_WINDOW_POINT_TEST .................. Passed 0.75 sec Start 23: UTILITY_TEST 23/45 Test rapidsai#23: UTILITY_TEST ............................... Passed 1.86 sec Start 24: HAVERSINE_TEST_EXP 24/45 Test rapidsai#24: HAVERSINE_TEST_EXP ......................... Passed 0.14 sec Start 25: POINT_DISTANCE_TEST_EXP 25/45 Test rapidsai#25: POINT_DISTANCE_TEST_EXP .................... Passed 0.11 sec Start 26: POINT_LINESTRING_DISTANCE_TEST_EXP 26/45 Test rapidsai#26: POINT_LINESTRING_DISTANCE_TEST_EXP ......... Passed 0.11 sec Start 27: POINT_POLYGON_DISTANCE_TEST_EXP 27/45 Test rapidsai#27: POINT_POLYGON_DISTANCE_TEST_EXP ............ Passed 0.13 sec Start 28: HAUSDORFF_TEST_EXP 28/45 Test rapidsai#28: HAUSDORFF_TEST_EXP ......................... Passed 9.21 sec Start 29: LINESTRING_DISTANCE_TEST_EXP 29/45 Test rapidsai#29: LINESTRING_DISTANCE_TEST_EXP ............... Passed 0.17 sec Start 30: LINESTRING_INTERSECTION_TEST_EXP 30/45 Test rapidsai#30: LINESTRING_INTERSECTION_TEST_EXP ........... Passed 0.19 sec Start 31: POINT_LINESTRING_NEAREST_POINT_TEST_EXP 31/45 Test rapidsai#31: POINT_LINESTRING_NEAREST_POINT_TEST_EXP .... Passed 0.12 sec Start 32: SINUSOIDAL_PROJECTION_TEST_EXP 32/45 Test rapidsai#32: SINUSOIDAL_PROJECTION_TEST_EXP ............. Passed 0.12 sec Start 33: POINTS_IN_RANGE_TEST_EXP 33/45 Test rapidsai#33: POINTS_IN_RANGE_TEST_EXP ................... Passed 0.11 sec Start 34: POINT_IN_POLYGON_TEST_EXP 34/45 Test rapidsai#34: POINT_IN_POLYGON_TEST_EXP .................. Passed 0.12 sec Start 35: PAIRWISE_POINT_IN_POLYGON_TEST_EXP 35/45 Test rapidsai#35: PAIRWISE_POINT_IN_POLYGON_TEST_EXP ......... Passed 0.11 sec Start 36: DERIVE_TRAJECTORIES_TEST_EXP 36/45 Test rapidsai#36: DERIVE_TRAJECTORIES_TEST_EXP ............... Passed 14.49 sec Start 37: POINT_BOUNDING_BOXES_TEST_EXP 37/45 Test rapidsai#37: POINT_BOUNDING_BOXES_TEST_EXP .............. Passed 1.35 sec Start 38: POLYGON_BOUNDING_BOXES_TEST_EXP 38/45 Test rapidsai#38: POLYGON_BOUNDING_BOXES_TEST_EXP ............ Passed 0.11 sec Start 39: LINESTRING_BOUNDING_BOXES_TEST_EXP 39/45 Test rapidsai#39: LINESTRING_BOUNDING_BOXES_TEST_EXP ......... Passed 0.11 sec Start 40: TRAJECTORY_DISTANCES_AND_SPEEDS_TEST_EXP 40/45 Test rapidsai#40: TRAJECTORY_DISTANCES_AND_SPEEDS_TEST_EXP ... Passed 0.80 sec Start 41: POINT_QUADTREE_TEST_EXP 41/45 Test rapidsai#41: POINT_QUADTREE_TEST_EXP .................... Passed 0.12 sec Start 42: OPERATOR_TEST_EXP 42/45 Test rapidsai#42: OPERATOR_TEST_EXP .......................... Passed 0.14 sec Start 43: FIND_TEST_EXP 43/45 Test rapidsai#43: FIND_TEST_EXP .............................. Passed 0.13 sec Start 44: JOIN_POINT_IN_POLYGON_SMALL_TEST_EXP 44/45 Test rapidsai#44: JOIN_POINT_IN_POLYGON_SMALL_TEST_EXP ....... Passed 0.11 sec Start 45: JOIN_POINT_IN_POLYGON_LARGE_TEST_EXP 45/45 Test rapidsai#45: JOIN_POINT_IN_POLYGON_LARGE_TEST_EXP ....... Passed 0.13 sec 100% tests passed, 0 tests failed out of 45 Total Test time (real) = 47.07 sec ``` After: ``` (rapids) coder ➜ ~/cuspatial/cpp/build/release $ ninja test [0/1] Running tests... Test project /home/coder/cuspatial/cpp/build/release Start 1: SINUSOIDAL_PROJECTION_TEST 1/45 Test #1: SINUSOIDAL_PROJECTION_TEST ................. Passed 0.78 sec Start 2: HAVERSINE_TEST 2/45 Test #2: HAVERSINE_TEST ............................. Passed 0.75 sec Start 3: HAUSDORFF_TEST 3/45 Test #3: HAUSDORFF_TEST ............................. Passed 0.74 sec Start 4: JOIN_POINT_TO_LINESTRING_SMALL_TEST 4/45 Test rapidsai#4: JOIN_POINT_TO_LINESTRING_SMALL_TEST ........ Passed 0.77 sec Start 5: JOIN_POINT_IN_POLYGON_TEST 5/45 Test rapidsai#5: JOIN_POINT_IN_POLYGON_TEST ................. Passed 0.76 sec Start 6: POINT_IN_POLYGON_TEST 6/45 Test rapidsai#6: POINT_IN_POLYGON_TEST ...................... Passed 0.78 sec Start 7: PAIRWISE_POINT_IN_POLYGON_TEST 7/45 Test rapidsai#7: PAIRWISE_POINT_IN_POLYGON_TEST ............. Passed 0.74 sec Start 8: POINT_QUADTREE_TEST 8/45 Test rapidsai#8: POINT_QUADTREE_TEST ........................ Passed 0.75 sec Start 9: LINESTRING_BOUNDING_BOXES_TEST 9/45 Test rapidsai#9: LINESTRING_BOUNDING_BOXES_TEST ............. Passed 0.75 sec Start 10: POLYGON_BOUNDING_BOXES_TEST 10/45 Test rapidsai#10: POLYGON_BOUNDING_BOXES_TEST ................ Passed 0.73 sec Start 11: POINT_DISTANCE_TEST 11/45 Test rapidsai#11: POINT_DISTANCE_TEST ........................ Passed 0.73 sec Start 12: POINT_LINESTRING_DISTANCE_TEST 12/45 Test rapidsai#12: POINT_LINESTRING_DISTANCE_TEST ............. Passed 0.74 sec Start 13: LINESTRING_DISTANCE_TEST 13/45 Test rapidsai#13: LINESTRING_DISTANCE_TEST ................... Passed 0.76 sec Start 14: POINT_POLYGON_DISTANCE_TEST 14/45 Test rapidsai#14: POINT_POLYGON_DISTANCE_TEST ................ Passed 0.76 sec Start 15: LINESTRING_INTERSECTION_TEST 15/45 Test rapidsai#15: LINESTRING_INTERSECTION_TEST ............... Passed 0.78 sec Start 16: POINT_LINESTRING_NEAREST_POINT_TEST 16/45 Test rapidsai#16: POINT_LINESTRING_NEAREST_POINT_TEST ........ Passed 0.77 sec Start 17: QUADTREE_POLYGON_FILTERING_TEST 17/45 Test rapidsai#17: QUADTREE_POLYGON_FILTERING_TEST ............ Passed 0.75 sec Start 18: QUADTREE_LINESTRING_FILTERING_TEST 18/45 Test rapidsai#18: QUADTREE_LINESTRING_FILTERING_TEST ......... Passed 0.77 sec Start 19: TRAJECTORY_DISTANCES_AND_SPEEDS_TEST 19/45 Test rapidsai#19: TRAJECTORY_DISTANCES_AND_SPEEDS_TEST ....... Passed 0.74 sec Start 20: DERIVE_TRAJECTORIES_TEST 20/45 Test rapidsai#20: DERIVE_TRAJECTORIES_TEST ................... Passed 0.75 sec Start 21: TRAJECTORY_BOUNDING_BOXES_TEST 21/45 Test rapidsai#21: TRAJECTORY_BOUNDING_BOXES_TEST ............. Passed 0.74 sec Start 22: SPATIAL_WINDOW_POINT_TEST 22/45 Test rapidsai#22: SPATIAL_WINDOW_POINT_TEST .................. Passed 0.75 sec Start 23: UTILITY_TEST 23/45 Test rapidsai#23: UTILITY_TEST ............................... Passed 0.30 sec Start 24: HAVERSINE_TEST_EXP 24/45 Test rapidsai#24: HAVERSINE_TEST_EXP ......................... Passed 0.12 sec Start 25: POINT_DISTANCE_TEST_EXP 25/45 Test rapidsai#25: POINT_DISTANCE_TEST_EXP .................... Passed 0.12 sec Start 26: POINT_LINESTRING_DISTANCE_TEST_EXP 26/45 Test rapidsai#26: POINT_LINESTRING_DISTANCE_TEST_EXP ......... Passed 0.12 sec Start 27: POINT_POLYGON_DISTANCE_TEST_EXP 27/45 Test rapidsai#27: POINT_POLYGON_DISTANCE_TEST_EXP ............ Passed 0.13 sec Start 28: HAUSDORFF_TEST_EXP 28/45 Test rapidsai#28: HAUSDORFF_TEST_EXP ......................... Passed 0.26 sec Start 29: LINESTRING_DISTANCE_TEST_EXP 29/45 Test rapidsai#29: LINESTRING_DISTANCE_TEST_EXP ............... Passed 0.14 sec Start 30: LINESTRING_INTERSECTION_TEST_EXP 30/45 Test rapidsai#30: LINESTRING_INTERSECTION_TEST_EXP ........... Passed 0.19 sec Start 31: POINT_LINESTRING_NEAREST_POINT_TEST_EXP 31/45 Test rapidsai#31: POINT_LINESTRING_NEAREST_POINT_TEST_EXP .... Passed 0.11 sec Start 32: SINUSOIDAL_PROJECTION_TEST_EXP 32/45 Test rapidsai#32: SINUSOIDAL_PROJECTION_TEST_EXP ............. Passed 0.11 sec Start 33: POINTS_IN_RANGE_TEST_EXP 33/45 Test rapidsai#33: POINTS_IN_RANGE_TEST_EXP ................... Passed 0.13 sec Start 34: POINT_IN_POLYGON_TEST_EXP 34/45 Test rapidsai#34: POINT_IN_POLYGON_TEST_EXP .................. Passed 0.11 sec Start 35: PAIRWISE_POINT_IN_POLYGON_TEST_EXP 35/45 Test rapidsai#35: PAIRWISE_POINT_IN_POLYGON_TEST_EXP ......... Passed 0.14 sec Start 36: DERIVE_TRAJECTORIES_TEST_EXP 36/45 Test rapidsai#36: DERIVE_TRAJECTORIES_TEST_EXP ............... Passed 0.27 sec Start 37: POINT_BOUNDING_BOXES_TEST_EXP 37/45 Test rapidsai#37: POINT_BOUNDING_BOXES_TEST_EXP .............. Passed 0.15 sec Start 38: POLYGON_BOUNDING_BOXES_TEST_EXP 38/45 Test rapidsai#38: POLYGON_BOUNDING_BOXES_TEST_EXP ............ Passed 0.13 sec Start 39: LINESTRING_BOUNDING_BOXES_TEST_EXP 39/45 Test rapidsai#39: LINESTRING_BOUNDING_BOXES_TEST_EXP ......... Passed 0.12 sec Start 40: TRAJECTORY_DISTANCES_AND_SPEEDS_TEST_EXP 40/45 Test rapidsai#40: TRAJECTORY_DISTANCES_AND_SPEEDS_TEST_EXP ... Passed 0.13 sec Start 41: POINT_QUADTREE_TEST_EXP 41/45 Test rapidsai#41: POINT_QUADTREE_TEST_EXP .................... Passed 0.14 sec Start 42: OPERATOR_TEST_EXP 42/45 Test rapidsai#42: OPERATOR_TEST_EXP .......................... Passed 0.14 sec Start 43: FIND_TEST_EXP 43/45 Test rapidsai#43: FIND_TEST_EXP .............................. Passed 0.15 sec Start 44: JOIN_POINT_IN_POLYGON_SMALL_TEST_EXP 44/45 Test rapidsai#44: JOIN_POINT_IN_POLYGON_SMALL_TEST_EXP ....... Passed 0.12 sec Start 45: JOIN_POINT_IN_POLYGON_LARGE_TEST_EXP 45/45 Test rapidsai#45: JOIN_POINT_IN_POLYGON_LARGE_TEST_EXP ....... Passed 0.13 sec 100% tests passed, 0 tests failed out of 45 Total Test time (real) = 20.08 sec ``` Authors: - Mark Harris (https://github.com/harrism) Approvers: - Michael Wang (https://github.com/isVoid) - Paul Taylor (https://github.com/trxcllnt) URL: rapidsai#1018
Closes rapidsai#986. Depends on rapidsai#979. Authors: - Mark Harris (https://github.com/harrism) Approvers: - Michael Wang (https://github.com/isVoid) - Paul Taylor (https://github.com/trxcllnt) URL: rapidsai#1005
The last couple of PRs were very brittle w.r.t updating their functionality. This PR eliminates some of that brittleness by adding object inheritance for every combination of features for every binary predicate. Class choice is handled by type dispatching on the input columns. The operation stack (`_preprocess`, `_op`, and `_postprocess`) is now chained to make debugging easier. Instead of calling the operations in sequence, they call one another. Refactoring the existing modules to use the new architecture was trivial. I'm still interested in modifying the architecture so that the set operations that determine the final result are identifiable, and possibly composable. Authors: - H. Thomson Comer (https://github.com/thomcom) Approvers: - Michael Wang (https://github.com/isVoid) - Mark Harris (https://github.com/harrism) URL: rapidsai#1009
Fixes rapidsai#1032 by using `<tt>\"</tt>` instead of `` `"` ``. Also reflows some Markdown. <!-- Provide a standalone description of changes in this PR. --> <!-- Reference any issues closed by this PR with "closes rapidsai#1234". --> Authors: - Mark Harris (https://github.com/harrism) Approvers: - Michael Wang (https://github.com/isVoid) URL: rapidsai#1034
github-actions
bot
added
cmake
isVoid
approved these changes
Apr 5, 2023
harrism
added
non-breaking
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fix failed automerger
Checklist