feat: support reading DeltaByteArray encoding for Decimals #1037
Conversation
fix decimal avg function precision issue Alchemy-item: [[6020] Spark sql avg agg function support decimal](oap-project#511 (comment)) commit 1/1 - 475f069
Alchemy-item: [[oap] Register merge extract companion agg functions without suffix](oap-project#512 (comment)) commit 1/1 - c1e41a3
Signed-off-by: Yuan <yuanzhou@apache.org> Alchemy-item: [[11771] [11772] Fix smj result mismatch issue](oap-project#514 (comment)) commit 1/1 - 8c77615
Alchemy-item: [[11067] Support scan filter for decimal in ORC](oap-project#513 (comment)) commit 1/1 - 60fce69
Alchemy-item: [ [5962] Support struct schema evolution matching by name](oap-project#805 (comment)) commit 1/1 - e3d77fd
…rtType Alchemy-item: [[13620] fix: Add config for requested type check in ReaderBase::conve…](oap-project#855 (comment)) commit 1/1 - 924fc2e
prestodb-ci
force-pushed
the
update
branch
4 times, most recently
from
62f96b6 to
ec7dc4e
Compare
prestodb-ci
force-pushed
the
update
branch
2 times, most recently
from
198f2b4 to
6d118a5
Compare
|
There're some legal restrictions to do it in Russia. It would be great if you merge it in your fork first. So it would be published in repo that has no direct relations with Meta. Then you are able to share it to any other repo on your own. I'm not interested in direct commit into Meta's repo. |
prestodb-ci
force-pushed
the
update
branch
3 times, most recently
from
7ea7352 to
151e9ef
Compare
|
@4ertus2 Thanks for extending the support of decimal reader.
Would you mind providing a test with this type of Parquet file? It could be added in the https://github.com/oap-project/velox/blob/update/velox/dwio/parquet/tests/reader/ParquetTableScanTest.cpp. Thanks. |
|
I cannot share samples I tested on. They have confidential info inside. I also asked for a sample here apache/parquet-testing#89 But there's no-one who want to help with it yet. |
prestodb-ci
force-pushed
the
update
branch
2 times, most recently
from
351ef91 to
6e833ba
Compare
|
@4ertus2 Thanks for sharing the context. |
prestodb-ci
force-pushed
the
update
branch
6 times, most recently
from
344fde6 to
7a31679
Compare
prestodb-ci
force-pushed
the
update
branch
9 times, most recently
from
c5d9aac to
9f0ef60
Compare
jinchengchenghh
force-pushed
the
update
branch
2 times, most recently
from
704b7d6 to
9f74a76
Compare
prestodb-ci
force-pushed
the
update
branch
9 times, most recently
from
07f098d to
537e695
Compare
prestodb-ci
force-pushed
the
update
branch
3 times, most recently
from
e963905 to
b228025
Compare
prestodb-ci
force-pushed
the
update
branch
5 times, most recently
from
cc6eb06 to
7dfc0d2
Compare
5 participants
Decimal could be represented as FixedLengthByteArrays or ByteArrays in parquet files. FixedLengthByteArrays and ByteArrays could be encoded with DeltaByteArray encoding.
PR adds support for reading such Decimal columns.