Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update strings contains benchmarks to nvbench #15495

Merged
merged 27 commits into from
May 16, 2024

Conversation

davidwendt
Copy link
Contributor

@davidwendt davidwendt commented Apr 10, 2024

Description

Reference #15405
Updates the benchmarks for cudf::strings::contains() to use nvbench and also introduce a hit-test axis.
The logic has been updated to remove the unneeded fill() call for long strings.
Also cleaned up code and updated logic to process 4 bytes per warp thread.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@davidwendt davidwendt added 2 - In Progress Currently a work in progress libcudf Affects libcudf (C++/CUDA) code. strings strings issues (C++ and Python) improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Apr 10, 2024
@davidwendt davidwendt self-assigned this Apr 10, 2024
@github-actions github-actions bot added the CMake CMake build issue label Apr 10, 2024
d_target.empty());

if (!d_target.empty()) {
if (d_target.empty()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 This didn't occur to me, and I've been staring at this code for a bit now.

@davidwendt
Copy link
Contributor Author

This also includes some performance improvement for long strings

| width |   rows   | hr |   Ref Time |   Cmp Time |         Diff |   %Diff |
|-------|----------|----|------------|------------|--------------|---------|
|  32   |  260000  | 20 |  93.142 us |  88.831 us |    -4.310 us |  -4.63% |
|  64   |  260000  | 20 | 203.619 us | 203.895 us |     0.276 us |   0.14% |
|  128  |  260000  | 20 | 242.877 us | 204.080 us |   -38.797 us | -15.97% |
|  256  |  260000  | 20 | 385.031 us | 286.262 us |   -98.769 us | -25.65% |
|  512  |  260000  | 20 | 667.015 us | 460.092 us |  -206.923 us | -31.02% |
| 1024  |  260000  | 20 |   1.239 ms | 805.203 us |  -433.541 us | -35.00% |
|  32   | 1953000  | 20 | 359.205 us | 359.754 us |     0.549 us |   0.15% |
|  64   | 1953000  | 20 |   1.133 ms |   1.134 ms |     1.066 us |   0.09% |
|  128  | 1953000  | 20 |   1.611 ms |   1.328 ms |  -282.885 us | -17.56% |
|  256  | 1953000  | 20 |   2.666 ms |   1.941 ms |  -724.912 us | -27.19% |
|  512  | 1953000  | 20 |   4.754 ms |   3.229 ms | -1524.398 us | -32.07% |
| 1024  | 1953000  | 20 |   8.998 ms |   5.792 ms | -3205.510 us | -35.63% |
|  32   | 16777216 | 20 |   2.747 ms |   2.747 ms |     0.334 us |   0.01% |
|  64   | 16777216 | 20 |   9.149 ms |   9.148 ms |    -1.015 us |  -0.01% |
|  32   |  260000  | 80 |  95.208 us |  94.667 us |    -0.541 us |  -0.57% |
|  64   |  260000  | 80 | 201.054 us | 202.984 us |     1.930 us |   0.96% |
|  128  |  260000  | 80 | 269.381 us | 234.155 us |   -35.225 us | -13.08% |
|  256  |  260000  | 80 | 411.775 us | 314.839 us |   -96.935 us | -23.54% |
|  512  |  260000  | 80 | 688.695 us | 489.677 us |  -199.018 us | -28.90% |
| 1024  |  260000  | 80 |   1.266 ms | 850.385 us |  -415.672 us | -32.83% |
|  32   | 1953000  | 80 | 427.888 us | 428.361 us |     0.473 us |   0.11% |
|  64   | 1953000  | 80 |   1.130 ms |   1.134 ms |     4.034 us |   0.36% |
|  128  | 1953000  | 80 |   1.801 ms |   1.557 ms |  -243.390 us | -13.52% |
|  256  | 1953000  | 80 |   2.874 ms |   2.160 ms |  -714.607 us | -24.86% |
|  512  | 1953000  | 80 |   4.918 ms |   3.445 ms | -1472.601 us | -29.94% |
| 1024  | 1953000  | 80 |   9.206 ms |   6.152 ms | -3054.411 us | -33.18% |
|  32   | 16777216 | 80 |   3.350 ms |   3.355 ms |     4.182 us |   0.12% |
|  64   | 16777216 | 80 |   9.117 ms |   9.122 ms |     5.016 us |   0.06% |

From #15536 (comment)

@davidwendt davidwendt added 3 - Ready for Review Ready for review by team and removed 2 - In Progress Currently a work in progress labels May 13, 2024
@davidwendt davidwendt marked this pull request as ready for review May 13, 2024 20:58
@davidwendt davidwendt requested a review from a team as a code owner May 13, 2024 20:58
cpp/src/strings/search/find.cu Outdated Show resolved Hide resolved
cpp/src/strings/search/find.cu Outdated Show resolved Hide resolved
cpp/benchmarks/string/find.cpp Outdated Show resolved Hide resolved
cpp/benchmarks/string/find.cpp Outdated Show resolved Hide resolved
@davidwendt davidwendt requested a review from PointKernel May 14, 2024 15:10
Copy link
Member

@PointKernel PointKernel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Thanks!

Comment on lines 66 to 77
state.exec(nvbench::exec_tag::sync, [&](nvbench::launch& launch) {
if (api == "find") {
cudf::strings::find(input, target);
} else if (api == "find_multi") {
cudf::strings::find_multiple(input, cudf::strings_column_view(targets));
} else if (api == "contains") {
cudf::strings::contains(input, target);
} else if (api == "starts_with") {
cudf::strings::starts_with(input, target);
} else if (api == "ends_with") {
cudf::strings::ends_with(input, target);
}
Copy link
Contributor

@ttnghia ttnghia May 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will multiple if/else inside one state.exec lambda affect the benchmark run (as this will be executed in thousands of iterations)?
How about placing separate state.exec in each if/else branch?

Comment on lines +373 to +376
if (((i + j + d_target.size_bytes()) <= d_str.size_bytes()) &&
d_target.compare(d_str.data() + i + j, d_target.size_bytes()) == 0) {
found = true;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will any early termination be helpful here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I benchmarked that and it did not help -- technically a bit slower.

@davidwendt
Copy link
Contributor Author

/merge

@rapids-bot rapids-bot bot merged commit 49af261 into rapidsai:branch-24.06 May 16, 2024
71 checks passed
@davidwendt davidwendt deleted the perf-contains branch May 16, 2024 18:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3 - Ready for Review Ready for review by team CMake CMake build issue improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change strings strings issues (C++ and Python)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants