Skip to content

Commit

Permalink
Python: ManifestWriter and ManifestListWriter (#8622)
Browse files Browse the repository at this point in the history
* add ManifestWriter and ManifestListWriter

* fix lint issue

* remove assert, fix format issue

* add prepare to ManifestWriter, remove TODO, fix format issue

* fix some nit issue, add prepare... to ensure the correctness of data written

* fix format issue

* fix lint issue

* avoid creating too much objects, handling v1, v2 data_file_type and DataFile class properly.

* modify tests

* refactor the way of handling two version of DataFile record

* add integration tests, fix bugs, change PartitionSummary to a function

* fix format issue

* make data_type_v2 constants
  • Loading branch information
HonahX authored Sep 29, 2023
1 parent 6172f5c commit 8062aef
Show file tree
Hide file tree
Showing 6 changed files with 855 additions and 9 deletions.
4 changes: 2 additions & 2 deletions python/pyiceberg/avro/writer.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
from uuid import UUID

from pyiceberg.avro.encoder import BinaryEncoder
from pyiceberg.types import StructType
from pyiceberg.typedef import Record
from pyiceberg.utils.decimal import decimal_required_bytes, decimal_to_bytes
from pyiceberg.utils.singleton import Singleton

Expand Down Expand Up @@ -160,7 +160,7 @@ def write(self, encoder: BinaryEncoder, val: Any) -> None:
class StructWriter(Writer):
field_writers: Tuple[Writer, ...] = dataclassfield()

def write(self, encoder: BinaryEncoder, val: StructType) -> None:
def write(self, encoder: BinaryEncoder, val: Record) -> None:
for writer, value in zip(self.field_writers, val.record_fields()):
writer.write(encoder, value)

Expand Down
Loading

0 comments on commit 8062aef

Please sign in to comment.