Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement batch construction for strings columns #17035

Merged
merged 40 commits into from
Oct 17, 2024
Merged
Changes from 1 commit
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
e252fb4
Complete `make_strings_column_batch`
ttnghia Oct 9, 2024
11d09ef
Update tests
ttnghia Oct 10, 2024
7e253f2
Update benchmark
ttnghia Oct 10, 2024
0f22d4b
Update docs
ttnghia Oct 10, 2024
326d73e
Rewrite `valid_if_n_kernel`
ttnghia Oct 10, 2024
d314669
Cleanup
ttnghia Oct 10, 2024
7453ef4
Fix tests
ttnghia Oct 10, 2024
ee5b7fd
Discard null mask if there is no nulls
ttnghia Oct 10, 2024
629516b
Change benchmark
ttnghia Oct 10, 2024
27d474d
Debugging performance
ttnghia Oct 10, 2024
6f7c6f3
Cleanup and fix `valid_if_batch_async`
ttnghia Oct 10, 2024
99107f6
Combine function
ttnghia Oct 10, 2024
42391fb
Implement `make_chars` using `cub::DeviceMemcpy::Batched`
ttnghia Oct 11, 2024
9e23b41
Move the implementation of `make_chars` to `strings_column_factories.…
ttnghia Oct 11, 2024
f8957f1
Revert unrelated changes
ttnghia Oct 11, 2024
239a816
Merge branch 'branch-24.12' into strings_batch_construction
ttnghia Oct 11, 2024
cfadc8d
Rewrite tests
ttnghia Oct 11, 2024
f13ab82
Fix null
ttnghia Oct 11, 2024
de1a716
Fix stream, and rename function
ttnghia Oct 11, 2024
22d6dac
Use `string_index_pair`
ttnghia Oct 11, 2024
0d95af5
Rewrite benchmark using range based for loop
ttnghia Oct 11, 2024
05155a3
Remove unused var
ttnghia Oct 11, 2024
7d4ed69
Disable test by default since it needs very much memory
ttnghia Oct 11, 2024
de5ebab
Change tests
ttnghia Oct 11, 2024
3f55a20
Add stream test
ttnghia Oct 11, 2024
c66c2d2
Change from using `make_host_vector_async` to `make_std_vector_async`
ttnghia Oct 11, 2024
03c3e1a
Using `DISABLED_` prefix for test
ttnghia Oct 11, 2024
60f7e3a
Add `CUDF_CUDA_TRY`
ttnghia Oct 11, 2024
6dccdc7
Change comments
ttnghia Oct 11, 2024
8e190ad
Fix spell
ttnghia Oct 11, 2024
c981e99
Test stream sync
ttnghia Oct 12, 2024
8dfa8dd
Revert "Test stream sync"
ttnghia Oct 12, 2024
4190cfd
Test disabling kvikio
ttnghia Oct 13, 2024
8fd05d1
Revert "Test disabling kvikio"
ttnghia Oct 14, 2024
a1a36c6
Merge branch 'branch-24.12' into strings_batch_construction
ttnghia Oct 14, 2024
b73240b
Move `make_chars_buffer` to `strings_children.cuh`
ttnghia Oct 14, 2024
d5dfa31
Merge branch 'branch-24.12' into strings_batch_construction
ttnghia Oct 14, 2024
3ec7b5a
Fix header with `device_memcpy.cuh`
ttnghia Oct 14, 2024
6d43f4c
Fix headers
ttnghia Oct 16, 2024
7764849
Merge branch 'branch-24.12' into strings_batch_construction
ttnghia Oct 16, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Disable test by default since it needs very much memory
Signed-off-by: Nghia Truong <nghiat@nvidia.com>
  • Loading branch information
ttnghia committed Oct 11, 2024
commit 7d4ed6980cff032f052610afddaa919fd7ac4d5b
3 changes: 3 additions & 0 deletions cpp/tests/strings/factories_test.cu
Original file line number Diff line number Diff line change
Expand Up @@ -364,6 +364,8 @@ TEST_F(StringsBatchConstructionTest, CreateColumnsFromPairs)
}
}

// The test below requires a huge amount of memory, thus it is disabled by default.
#ifdef LOCAL_TEST
TEST_F(StringsBatchConstructionTest, CreateLongStringsColumns)
{
auto constexpr num_columns = 2;
Expand Down Expand Up @@ -434,3 +436,4 @@ TEST_F(StringsBatchConstructionTest, CreateLongStringsColumns)
CUDF_TEST_EXPECT_COLUMNS_EQUAL(expected->view(), output[i]->view());
}
}
#endif // LOCAL_TEST
Loading