Skip to content

GH-45254: [C++][Acero] Fix the row offset truncation in row table merge #45255

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 14, 2025
Merged

GH-45254: [C++][Acero] Fix the row offset truncation in row table merge #45255

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion cpp/src/arrow/acero/hash_join_node_test.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3370,8 +3370,10 @@ TEST(HashJoin, LARGE_MEMORY_TEST(BuildSideOver4GBVarLength)) {
constexpr int value_no_match_length_min = 128;
constexpr int value_no_match_length_max = 129;
constexpr int value_match_length = 130;
// The value "DDD..." will be hashed to the partition over 4GB of the hash table.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note this really depends on the hash algorithm which is opaque, but this as well is the best I can do so far.

// Matching at this area gives us more coverage.
const auto value_match =
std::make_shared<StringScalar>(std::string(value_match_length, 'X'));
std::make_shared<StringScalar>(std::string(value_match_length, 'D'));
constexpr int16_t num_rows_per_batch_left = 128;
constexpr int16_t num_rows_per_batch_right = 4096;
const int64_t num_batches_left = 8;
Expand Down
4 changes: 2 additions & 2 deletions cpp/src/arrow/acero/swiss_join.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -439,11 +439,11 @@ Status RowArrayMerge::PrepareForMerge(RowArray* target,
num_rows = 0;
num_bytes = 0;
for (size_t i = 0; i < sources.size(); ++i) {
target->rows_.mutable_offsets()[num_rows] = static_cast<uint32_t>(num_bytes);
target->rows_.mutable_offsets()[num_rows] = num_bytes;
num_rows += sources[i]->rows_.length();
num_bytes += sources[i]->rows_.offsets()[sources[i]->rows_.length()];
}
target->rows_.mutable_offsets()[num_rows] = static_cast<uint32_t>(num_bytes);
target->rows_.mutable_offsets()[num_rows] = num_bytes;
}

return Status::OK();
Expand Down
Loading