Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add TPC-H inspired examples for Libcudf #16088

Merged
merged 125 commits into from
Jul 17, 2024
Merged
Changes from 1 commit
Commits
Show all changes
125 commits
Select commit Hold shift + click to select a range
6e34bf8
Add basic cudf code to project / select
JayjeetAtGithub Jun 19, 2024
b20a8e6
wip q1
JayjeetAtGithub Jun 25, 2024
c36cc16
q1 done
JayjeetAtGithub Jun 25, 2024
90fbcb3
filtering using timestamp
JayjeetAtGithub Jun 25, 2024
6ec4748
Finish q1
JayjeetAtGithub Jun 25, 2024
18a9209
Finish q1
JayjeetAtGithub Jun 25, 2024
4f1aad8
Add column names
JayjeetAtGithub Jun 25, 2024
c86ce5e
Remove unnecessary memory copies
JayjeetAtGithub Jun 25, 2024
8174b99
Add a query plan diagram
JayjeetAtGithub Jun 25, 2024
b02cd03
Add helper functions
JayjeetAtGithub Jun 26, 2024
2fb3198
Convert scalar to device buffer
JayjeetAtGithub Jun 26, 2024
08e6e7e
Extract create metadata into utils
JayjeetAtGithub Jun 26, 2024
22bda7c
Add copyright to tpch q6
JayjeetAtGithub Jun 26, 2024
e68fcaa
Add initial README
JayjeetAtGithub Jun 26, 2024
143d32c
Extract order by into utils
JayjeetAtGithub Jun 26, 2024
7dd2cdb
Use make_column_from_scalar factory function
JayjeetAtGithub Jun 26, 2024
06f02c1
Add data gen instructions in README
JayjeetAtGithub Jun 26, 2024
62f4960
Fixes
JayjeetAtGithub Jun 26, 2024
eb03a3b
Fixes
JayjeetAtGithub Jun 26, 2024
d06ac1c
Move append_col_to_table to utils
JayjeetAtGithub Jun 26, 2024
c3631d8
Fix scale errors in q1
JayjeetAtGithub Jun 26, 2024
60e8fef
misc fixes
JayjeetAtGithub Jun 26, 2024
5b5237e
Remove useless headers
JayjeetAtGithub Jun 26, 2024
512b0ec
Finish q6
JayjeetAtGithub Jun 27, 2024
210b3d8
Update README
JayjeetAtGithub Jun 27, 2024
f420cf2
Cleanup q1/q6
JayjeetAtGithub Jun 27, 2024
182d9c2
measure query exec time
JayjeetAtGithub Jun 27, 2024
b6b5985
start working on q5
JayjeetAtGithub Jun 27, 2024
79cd0ea
Add q5
JayjeetAtGithub Jun 27, 2024
bcee9d8
Update README
JayjeetAtGithub Jun 27, 2024
3c95cc2
Implement fixed point scalar
JayjeetAtGithub Jun 28, 2024
e85cf41
Fix fixed_point_scalar init in q1
JayjeetAtGithub Jun 28, 2024
3d50688
Remove fixed point scalar
JayjeetAtGithub Jun 28, 2024
46387c4
Extract groupby into utils
JayjeetAtGithub Jun 28, 2024
b0ac3ee
Clean up Q5
JayjeetAtGithub Jun 28, 2024
4ff6293
Remove test.cp
JayjeetAtGithub Jun 29, 2024
d20bac0
Add copyright notice
JayjeetAtGithub Jun 29, 2024
b297e98
Clean up Q5
JayjeetAtGithub Jun 29, 2024
1cd06f8
Fix naming of variables
JayjeetAtGithub Jun 29, 2024
6d786c4
Remove more headers from Q5
JayjeetAtGithub Jun 29, 2024
38997ac
Refactor Q1
JayjeetAtGithub Jun 29, 2024
dc625c4
Add finished q9
JayjeetAtGithub Jun 29, 2024
e398abc
Add finished q9
JayjeetAtGithub Jun 29, 2024
0f4516c
Update README
JayjeetAtGithub Jun 29, 2024
5ca3420
Fix sql query in Q5
JayjeetAtGithub Jun 29, 2024
dd3a1d5
Fix comments
JayjeetAtGithub Jun 29, 2024
6e87bc9
Refactor Q6
JayjeetAtGithub Jun 29, 2024
12ddb78
Extract base dataset dir into utils
JayjeetAtGithub Jun 30, 2024
c45577f
Remove unnecessary rmm imports
JayjeetAtGithub Jun 30, 2024
07cfd36
Rearrange utils.hpp
JayjeetAtGithub Jul 1, 2024
da87ff4
Push down projections into read parquet
JayjeetAtGithub Jul 2, 2024
a423ea2
Add comments to utils.hpp
JayjeetAtGithub Jul 2, 2024
5312f34
Remove implementation status from README
JayjeetAtGithub Jul 1, 2024
6ef9d12
Add nvtx ranges to helper functions
JayjeetAtGithub Jul 2, 2024
903cb67
Add more nvtx ranges
JayjeetAtGithub Jul 2, 2024
c0be319
Add script to run the benchmarks
JayjeetAtGithub Jul 2, 2024
9ed2a09
Fix README
JayjeetAtGithub Jul 2, 2024
7145fe8
Pass the base dir as a cli arg
JayjeetAtGithub Jul 2, 2024
bbf40e2
Pass dataset path as an argument to tpch/run.sh
JayjeetAtGithub Jul 2, 2024
a8b8255
Use memory resource for q1
JayjeetAtGithub Jul 2, 2024
8f66bfb
Use a memory pool
JayjeetAtGithub Jul 2, 2024
1335ff9
Add info to check args
JayjeetAtGithub Jul 3, 2024
e105141
Measure the query execution time using the Timer implementation from …
JayjeetAtGithub Jul 3, 2024
33b90ba
Turn on/off memory pol usage
JayjeetAtGithub Jul 3, 2024
5dda317
Add view creation sql to q1/q6
JayjeetAtGithub Jul 3, 2024
684e5e2
Push down filters in Q5
JayjeetAtGithub Jul 3, 2024
45a4b62
remove run.sh
JayjeetAtGithub Jul 3, 2024
68377ce
Cleanup parsing cli arguments
JayjeetAtGithub Jul 3, 2024
dec52ae
Rename groupby_context to groupby_context_t
JayjeetAtGithub Jul 3, 2024
f7ec78d
Add support for managed memory
JayjeetAtGithub Jul 3, 2024
4b9972e
Refactor mem management code
JayjeetAtGithub Jul 3, 2024
e69c34b
Fix indentation of queries
JayjeetAtGithub Jul 3, 2024
47199d5
Rename predicates
JayjeetAtGithub Jul 3, 2024
b7a57a7
Fix comments
JayjeetAtGithub Jul 3, 2024
2cb21ed
Dynamically determine scale of bin op
JayjeetAtGithub Jul 4, 2024
057f54a
use east const
JayjeetAtGithub Jul 4, 2024
87648c5
Rename rmm utilties
JayjeetAtGithub Jul 5, 2024
4798527
Fix append function
JayjeetAtGithub Jul 5, 2024
3032572
Address col id by name
JayjeetAtGithub Jul 5, 2024
2f1defa
Fix col id addressing
JayjeetAtGithub Jul 5, 2024
adde65b
Add name to col_id addressing
JayjeetAtGithub Jul 5, 2024
b805c41
Add comments
JayjeetAtGithub Jul 5, 2024
b7e25c9
Remove plot.png
JayjeetAtGithub Jul 5, 2024
43733e2
Fix the calc functions
JayjeetAtGithub Jul 5, 2024
4ae4538
Run clang-format
JayjeetAtGithub Jul 5, 2024
f0d6f6a
Change q5 for benchmarks in dt04
JayjeetAtGithub Jul 5, 2024
91ac503
Fix q5
JayjeetAtGithub Jul 5, 2024
cb0550c
Use float64 instead of decimal64
JayjeetAtGithub Jul 5, 2024
545cfb9
Add stream/mr params to new col calc functions
JayjeetAtGithub Jul 6, 2024
cadc195
Add stream / mr params
JayjeetAtGithub Jul 6, 2024
c552510
Fix the SQL queries
JayjeetAtGithub Jul 6, 2024
adf9456
Fix the join order for Q9
JayjeetAtGithub Jul 6, 2024
48d108a
Fix trailing whitespace
JayjeetAtGithub Jul 10, 2024
1bb2793
Update cpp/examples/tpch/CMakeLists.txt
JayjeetAtGithub Jul 11, 2024
174f998
Update cpp/examples/tpch/README.md
JayjeetAtGithub Jul 11, 2024
94e3b4e
Add docstring to remaining functions in utils
JayjeetAtGithub Jul 11, 2024
e8c8abb
Make the one scalars const
JayjeetAtGithub Jul 11, 2024
d830680
Add docstrings to column calculation functions
JayjeetAtGithub Jul 11, 2024
f75430d
Merge branch 'branch-24.08' into tpch-bench
mhaseeb123 Jul 11, 2024
e297deb
Add file-level docstring to Q1
JayjeetAtGithub Jul 11, 2024
146d45b
Add file-level docstring to Q5
JayjeetAtGithub Jul 11, 2024
90903f2
Add file-level docstring to Q6
JayjeetAtGithub Jul 11, 2024
0c78691
Add file-level docstring to Q9
JayjeetAtGithub Jul 11, 2024
194f08f
Add docstring to join_and_gather function
JayjeetAtGithub Jul 11, 2024
6b12762
Add consts to join_and_gather
JayjeetAtGithub Jul 11, 2024
9dd31a7
Add more const literals
JayjeetAtGithub Jul 11, 2024
0e98994
Add more const literals
JayjeetAtGithub Jul 11, 2024
792f33e
Add consts, nodiscards, and other qualifiers
JayjeetAtGithub Jul 11, 2024
5e103b9
Add more const references
JayjeetAtGithub Jul 11, 2024
a32e899
More improvements
JayjeetAtGithub Jul 11, 2024
781a460
Add [[nodiscard]] to append
JayjeetAtGithub Jul 12, 2024
4578a4b
Use std transform
JayjeetAtGithub Jul 12, 2024
fcdde79
Change to append in place
JayjeetAtGithub Jul 15, 2024
b0e764f
Address reviews
JayjeetAtGithub Jul 16, 2024
2435179
Add consts
JayjeetAtGithub Jul 16, 2024
d02cf08
Add consts
JayjeetAtGithub Jul 16, 2024
750d6a9
Misc fixes
JayjeetAtGithub Jul 16, 2024
0f984f0
Extract timer into cudf/utilities/timer.hpp
JayjeetAtGithub Jul 16, 2024
14be022
Add a cudf::timer
JayjeetAtGithub Jul 16, 2024
0ae2863
Fix the invalid cli args message
JayjeetAtGithub Jul 16, 2024
2ce182d
Allow chaining of appends
JayjeetAtGithub Jul 16, 2024
38f33ed
Move timer to a utilities dir in examples
JayjeetAtGithub Jul 16, 2024
c368953
Fix parquet_io and add the utilities dir
JayjeetAtGithub Jul 16, 2024
0fdf2f3
Move timer into cudf::example namespacE
JayjeetAtGithub Jul 16, 2024
c153636
Fix the consts for table with names fn
JayjeetAtGithub Jul 17, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Use std transform
  • Loading branch information
JayjeetAtGithub committed Jul 12, 2024
commit 4578a4b1dcc343c1cb3ad03f343b732065127e45
14 changes: 8 additions & 6 deletions cpp/examples/tpch/utils.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -229,12 +229,14 @@ std::unique_ptr<table_with_names> apply_inner_join(
CUDF_FUNC_RANGE();
std::vector<cudf::size_type> left_on_indices;
std::vector<cudf::size_type> right_on_indices;
for (auto& col_name : left_on) {
left_on_indices.push_back(left_input->col_id(col_name));
}
for (auto& col_name : right_on) {
right_on_indices.push_back(right_input->col_id(col_name));
}
std::transform(
left_on.begin(), left_on.end(), std::back_inserter(left_on_indices), [&](auto const& col_name) {
return left_input->col_id(col_name);
});
std::transform(right_on.begin(),
right_on.end(),
std::back_inserter(right_on_indices),
[&](auto const& col_name) { return right_input->col_id(col_name); });
auto table = join_and_gather(
left_input->table(), right_input->table(), left_on_indices, right_on_indices, compare_nulls);
return std::make_unique<table_with_names>(
Expand Down
Loading