Skip to content

Conversation

@lidavidm
Copy link
Member

This repurposes the code from ScalarVectorToArray. This doesn't support dictionary scalars.

@github-actions
Copy link

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also suggest the overload AppendScalar(const Scalar& scalat, int64_t nrepeats) to append the same scalar a number of times.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I've added the overload.

@pitrou
Copy link
Member

pitrou commented Jun 28, 2021

Hmm, it looks like the implementation could be more optimized. When appending multiple scalars (or multiple times the same scalars), the required space could be reserved. Also, at least for primitive types, I would expect appending multiple scalars to be much more streamlined than this.

(it looks like such optimizations could be crafted more easily if AppendScalar was a bunch of virtual methods?)

Perhaps create a TODO JIRA for those optimizations, unless you would like to work on them on this PR.

@lidavidm
Copy link
Member Author

I changed it to at least call Reserve() since that's simple. I filed ARROW-13197 and also noted the dictionary scalar support there.

@lidavidm
Copy link
Member Author

(Note: Ben left feedback on this in #10511 so I'll address that here.)

@lidavidm
Copy link
Member Author

Now fixed to not require allocations/not use a vector as Ben suggested.

lidavidm and others added 2 commits June 29, 2021 11:04
namespace {
struct AppendScalarImpl {
template <typename T>
enable_if_t<has_c_type<T>::value || is_decimal_type<T>::value, Status> Visit(const T&) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't include FixedSizeBinaryType, which doesn't require a ReserveData call

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FixedSizeBinaryType didn't work with this specialization anyways, I'll go add the required Builder::Append(Buffer) call that's needed.

@lidavidm
Copy link
Member Author

This got included in #10511.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants