Skip to content

Conversation

@lidavidm
Copy link
Member

This likely needs more testing, especially where I had to implement functionality in (Basic)Decimal256. Also, we may want to extend the scalar cast benchmarks to cover decimals. There's also potentially some redundancy to eliminate in the tests.

@github-actions
Copy link

@lidavidm lidavidm changed the title ARROW-10606: [C++] WIP: implement Decimal256 casts ARROW-10606: [C++] Implement Decimal256 casts Mar 19, 2021
@lidavidm lidavidm marked this pull request as ready for review March 22, 2021 15:07
@lidavidm
Copy link
Member Author

CC @bkietz and @emkornfield.

@bkietz bkietz requested review from bkietz and pitrou and removed request for pitrou March 22, 2021 16:54
Copy link
Member

@pitrou pitrou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for doing this. Here are some comments.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, do we want to avoid repeating those kernels using a helper function? e.g.

template <typename OutDecimal, typename InDecimal>
OutDecimal ConvertDecimalType(InDecimal&& val, KernelContext*);

template <>
Decimal128 ConvertDecimalType<Decimal256, Decimal128>(Decimal256&& val, KernelContext*) {
  return Decimal128(val.little_endian_array()[1],
                    val.little_endian_array()[0]);
}

template <typename OutDecimal>
OutDecimal ConvertDecimalType<OutDecimal, OutDecimal>(OutDecimal&& val, KernelContext*) {
  return val;
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't the CastFunctor signature <Out, In>? Here it seems you're implementing the converse (Decimal256 to Decimal128).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment here: should be CastFunctor<Decimal256Type, Arg0Type>?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be Type::DECIMAL256?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note the signature of CastFunction::AddKernel:

  Status AddKernel(Type::type in_type_id, std::vector<InputType> in_types,
                   OutputType out_type, ArrayKernelExec exec,
                   NullHandling::type = NullHandling::INTERSECTION,
                   MemAllocation::type = MemAllocation::PREALLOCATE);

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be Type::DECIMAL128?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decimal128ToDecimal256

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decimal256ToDecimal128

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't you want to avoid the code duplication here? This looks like exactly the same test as above.

@lidavidm lidavidm marked this pull request as draft March 23, 2021 14:37
@lidavidm
Copy link
Member Author

Thanks, looks like I got something backwards…will rework this.

@emkornfield
Copy link
Contributor

@lidavidm is it still worth me taking a look or is the rework going to change things substantially?

@lidavidm lidavidm marked this pull request as ready for review March 24, 2021 11:42
@lidavidm
Copy link
Member Author

@emkornfield this should be ready again (was just waiting for CI - looks like MacOS flaked)

Copy link
Member

@pitrou pitrou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, just one comment

auto func = std::make_shared<CastFunction>("cast_decimal256", Type::DECIMAL256);
// Needed for Parquet conversion. Full implementation is ARROW-10606
// tracks full implementation.
// Needed for Parquet conversion.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this comment is still useful. These are just the same casts as for Decimal128.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants