Skip to content

Commit

Permalink
apacheGH-42016: [Python] Expose new FLOAT16 logical type in the pyarr…
Browse files Browse the repository at this point in the history
…ow.parquet bindings (apache#42103)

### Rationale for this change
Resolves apache#42016.

### What changes are included in this PR?
Expose new `FLOAT16` logical type in the `pyarrow.parquet` bindings

### Are these changes tested?
Unit test added.

### Are there any user-facing changes?
No.

* GitHub Issue: apache#42016

Authored-by: Tai Le Manh <manhtai.lmt@gmail.com>
Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
  • Loading branch information
tlm365 authored Jun 20, 2024
1 parent d261a82 commit ec28590
Show file tree
Hide file tree
Showing 4 changed files with 6 additions and 1 deletion.
1 change: 1 addition & 0 deletions python/pyarrow/_parquet.pxd
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ cdef extern from "parquet/api/schema.h" namespace "parquet" nogil:
ParquetLogicalType_TIME" parquet::LogicalType::Type::TIME"
ParquetLogicalType_TIMESTAMP" parquet::LogicalType::Type::TIMESTAMP"
ParquetLogicalType_INT" parquet::LogicalType::Type::INT"
ParquetLogicalType_FLOAT16" parquet::LogicalType::Type::FLOAT16"
ParquetLogicalType_JSON" parquet::LogicalType::Type::JSON"
ParquetLogicalType_BSON" parquet::LogicalType::Type::BSON"
ParquetLogicalType_UUID" parquet::LogicalType::Type::UUID"
Expand Down
1 change: 1 addition & 0 deletions python/pyarrow/_parquet.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -1299,6 +1299,7 @@ cdef logical_type_name_from_enum(ParquetLogicalTypeId type_):
ParquetLogicalType_TIME: 'TIME',
ParquetLogicalType_TIMESTAMP: 'TIMESTAMP',
ParquetLogicalType_INT: 'INT',
ParquetLogicalType_FLOAT16: 'FLOAT16',
ParquetLogicalType_JSON: 'JSON',
ParquetLogicalType_BSON: 'BSON',
ParquetLogicalType_UUID: 'UUID',
Expand Down
1 change: 1 addition & 0 deletions python/pyarrow/tests/parquet/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,7 @@ def alltypes_sample(size=10000, seed=0, categorical=False):
'int16': np.arange(size, dtype=np.int16),
'int32': np.arange(size, dtype=np.int32),
'int64': np.arange(size, dtype=np.int64),
'float16': np.arange(size, dtype=np.float16),
'float32': np.arange(size, dtype=np.float32),
'float64': np.arange(size, dtype=np.float64),
'bool': np.random.randn(size) > 0,
Expand Down
4 changes: 3 additions & 1 deletion python/pyarrow/tests/parquet/test_metadata.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,10 +81,12 @@ def test_parquet_metadata_api():
assert col.max_definition_level == 1
assert col.max_repetition_level == 0
assert col.max_repetition_level == 0

assert col.physical_type == 'BOOLEAN'
assert col.converted_type == 'NONE'

col_float16 = schema[5]
assert col_float16.logical_type.type == 'FLOAT16'

with pytest.raises(IndexError):
schema[ncols + 1] # +1 for index

Expand Down

0 comments on commit ec28590

Please sign in to comment.