Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor schema impl #210

Merged
merged 27 commits into from
Aug 7, 2024
Merged

Refactor schema impl #210

merged 27 commits into from
Aug 7, 2024

Conversation

chmp
Copy link
Owner

@chmp chmp commented Aug 4, 2024 •
edited
Loading

  • Fix serde_arrow\src\internal\schema\test.rs
  • Check that tests in serde_arrow\src\internal\schema\serde\test.rs still make sense
  • Delete serde_arrow\src\internal\schema\deserialization.rs
  • Add test to schema roundtrip (serialize - deserialize), in particular maps
  • Include type_ids in UnionArray and UnionArrayView (and move checks for consecutive type ids into deserializer)
  • Add Serialization Tests for TensorField, VariableShapeTensorField
  • Update changelog

@chmp chmp changed the title Feature/refactor schema impl Refactor schema impl Aug 4, 2024
chmp added 22 commits August 5, 2024 11:06
@chmp
Broken. Switch API to use arrow::Field internally
f04554b
@chmp
Remove ArrowOrCustomField
e07641b
@chmp
Implement Arrow -> Arrow field conversion
f6779f6
@chmp
Initial impl of deserialization logic
0646dc2
@chmp
Implement more type conversions
26964a7
@chmp
Fully implement Dictionary
904164d
@chmp
Fix schema serialization tests
ad38848
@chmp
Fix various copy and paste errors
d6f6df8
@chmp
Fix copy and paste error
dde9ed1
@chmp
Fix type id handling for UnionArrays
2ac156f
@chmp
Ensure nullable fields are nullable, even without explicit nullable
f9e90ab
@chmp
Fix more copy and paste errors
894ab0c
@chmp
Fix tensor field extension types
6602a14
@chmp
Fix schema tests (+ Bug in serialize)
8c6991b
@chmp
Delete unused code
8617372
@chmp
Move all tests into a single file
5659f74
@chmp
Include type ids in union arrays, similar to arrow
bb74c18
@chmp
Move ext types into own modules, add serialization test for FixedShap…
3102fcf 
…eTensor
@chmp
Add serialization test for VariableShapeTensorField
deaf588
@chmp
Update changelog
31d544c
@chmp
Validate map entries
c95d5f0
@chmp
Fix todos, remove unused code
cca0efe
@chmp chmp marked this pull request as ready for review August 7, 2024 19:14
@chmp chmp merged commit f038a3f into main Aug 7, 2024
1 check passed
@chmp chmp mentioned this pull request Aug 7, 2024
Closed
5 tasks
@chmp chmp deleted the feature/refactor-schema-impl branch September 22, 2024 19:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@chmp