-
Couldn't load subscription status.
- Fork 3.9k
ARROW-10606: [C++] Implement Decimal256 casts #9751
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
CC @bkietz and @emkornfield. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot for doing this. Here are some comments.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, do we want to avoid repeating those kernels using a helper function? e.g.
template <typename OutDecimal, typename InDecimal>
OutDecimal ConvertDecimalType(InDecimal&& val, KernelContext*);
template <>
Decimal128 ConvertDecimalType<Decimal256, Decimal128>(Decimal256&& val, KernelContext*) {
return Decimal128(val.little_endian_array()[1],
val.little_endian_array()[0]);
}
template <typename OutDecimal>
OutDecimal ConvertDecimalType<OutDecimal, OutDecimal>(OutDecimal&& val, KernelContext*) {
return val;
}There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't the CastFunctor signature <Out, In>? Here it seems you're implementing the converse (Decimal256 to Decimal128).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same comment here: should be CastFunctor<Decimal256Type, Arg0Type>?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should be Type::DECIMAL256?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note the signature of CastFunction::AddKernel:
Status AddKernel(Type::type in_type_id, std::vector<InputType> in_types,
OutputType out_type, ArrayKernelExec exec,
NullHandling::type = NullHandling::INTERSECTION,
MemAllocation::type = MemAllocation::PREALLOCATE);There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should be Type::DECIMAL128?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Decimal128ToDecimal256
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Decimal256ToDecimal128
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't you want to avoid the code duplication here? This looks like exactly the same test as above.
|
Thanks, looks like I got something backwards…will rework this. |
|
@lidavidm is it still worth me taking a look or is the rework going to change things substantially? |
|
@emkornfield this should be ready again (was just waiting for CI - looks like MacOS flaked) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1, just one comment
| auto func = std::make_shared<CastFunction>("cast_decimal256", Type::DECIMAL256); | ||
| // Needed for Parquet conversion. Full implementation is ARROW-10606 | ||
| // tracks full implementation. | ||
| // Needed for Parquet conversion. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this comment is still useful. These are just the same casts as for Decimal128.
This likely needs more testing, especially where I had to implement functionality in (Basic)Decimal256. Also, we may want to extend the scalar cast benchmarks to cover decimals. There's also potentially some redundancy to eliminate in the tests.