Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-34535: [C++] Move ChunkResolver to the public API #44357

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

anjakefala
Copy link
Collaborator

@anjakefala anjakefala commented Oct 9, 2024

Rationale for this change

Adopting #40226.

The creation and return of a shared_ptr does result in some performance overhead, that makes a difference for a performance-sensitive application.

If someone could use ChunkResolver to learn the indices, they could then instead access the data directly.

What changes are included in this PR?

  • Updates to documentation (thanks to @SChakravorti21 )
  • Moving ChunkResolver to public API, and updating all references to it in the code

Are these changes tested?

There seemed to be comprehensive tests already: https://github.com/apache/arrow/blob/main/cpp/src/arrow/chunked_array_test.cc#L324 If an edgecase is missing, I'd be happy to add it.

Are there any user-facing changes?

ChunkResolver and TypedChunkLocation are now in the public API.

Co-authored-by: SChakravorti21<schakravorti@bloomberg.net>
@anjakefala anjakefala marked this pull request as draft October 9, 2024 21:22
@anjakefala
Copy link
Collaborator Author

I'm aware of the build failures and actively resolving them!

@anjakefala
Copy link
Collaborator Author

For the failing R builds, this conversation needs to be resolved: #43623

@anjakefala anjakefala force-pushed the chunkresolver branch 2 times, most recently from 375ef57 to 994ffc8 Compare October 10, 2024 18:01
@anjakefala anjakefala marked this pull request as ready for review October 11, 2024 00:22
cpp/src/arrow/chunk_resolver.h Show resolved Hide resolved
cpp/src/arrow/chunk_resolver.h Outdated Show resolved Hide resolved
cpp/src/arrow/chunk_resolver_benchmark.cc Outdated Show resolved Hide resolved
cpp/src/arrow/chunked_array_test.cc Outdated Show resolved Hide resolved
cpp/src/arrow/compute/kernels/vector_sort.cc Outdated Show resolved Hide resolved
@github-actions github-actions bot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels Oct 12, 2024
@felipecrv
Copy link
Contributor

felipecrv commented Oct 12, 2024

Yes, I agree this API should be made public. I've added tests and benchmarks recently [1].

[1] #43954

@github-actions github-actions bot removed the awaiting changes Awaiting changes label Oct 15, 2024
@github-actions github-actions bot added the awaiting change review Awaiting change review label Oct 15, 2024
r/src/altrep.cpp Outdated Show resolved Hide resolved
@github-actions github-actions bot added awaiting merge Awaiting merge and removed awaiting change review Awaiting change review labels Oct 17, 2024
@anjakefala
Copy link
Collaborator Author

anjakefala commented Oct 18, 2024

Please add a Co-authored-by: SChakravorti21 <schakravorti@bloomberg.net> in the squashed commit. I tried to add it, but I seem to had a typo, and it didn't register.

(Edit: Turns out PRs are merged via a script, and not a manual squash.)

@anjakefala
Copy link
Collaborator Author

I amended @SChakravorti21 as an author in a recent commit, so that the merge script will pick them up as a co-author.

…_job

Guard ChunkResolver use in altrep.cpp
@@ -80,20 +80,26 @@ void DeletePointer(std::shared_ptr<T>* ptr) {
template <typename T>
using Pointer = cpp11::external_pointer<std::shared_ptr<T>, DeletePointer<T>>;

#if ARROW_MAJOR_VERSION >= 18
Copy link
Member

@assignUser assignUser Oct 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#if ARROW_MAJOR_VERSION >= 18
#if ARROW_VERSION_MAJOR >= 18

atm the wrong branch get's picked: https://github.com/apache/arrow/actions/runs/11407714122/job/31744347738?pr=44357#step:7:29940

See arrow/util/config.h

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants