Skip to content

Conversation

@hrodmn
Copy link
Collaborator

@hrodmn hrodmn commented Nov 16, 2025

Related to

Description

This adds the max_row_group_size parameter to geoparquet::WriterBuilder and sets the default value to 150_000 (per recommendations in cholmes' guide.

The rustac cli has a new option --parquet-max-row-group-size that can be used to set the max_row_group_size on output geoparquet files.

Checklist

Delete any checklist items that do not apply (e.g. if your change is minor, it may not require documentation updates).

  • Unit tests
  • Documentation, including doctests
  • Git history is linear
  • Commit messages are descriptive
  • (optional) Git commit messages follow conventional commits
  • Code is formatted (cargo fmt)
  • cargo test
  • Changes are added to the CHANGELOG

@hrodmn hrodmn requested a review from gadomski as a code owner November 16, 2025 19:29
@hrodmn hrodmn force-pushed the feat/row-group-size branch from 1b78bd8 to 31fe450 Compare November 16, 2025 19:31
@gadomski gadomski linked an issue Nov 17, 2025 that may be closed by this pull request
Copy link
Member

@gadomski gadomski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the options fix, this feels nice. Just one tweak that might let us do fewer changes.

@gadomski gadomski self-requested a review November 18, 2025 17:17
@hrodmn hrodmn requested a review from gadomski November 18, 2025 18:57
Copy link
Member

@gadomski gadomski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@gadomski gadomski enabled auto-merge (squash) November 18, 2025 19:08
@gadomski gadomski merged commit 2bde538 into main Nov 18, 2025
11 checks passed
@gadomski gadomski deleted the feat/row-group-size branch November 18, 2025 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

make max_row_group_size configurable when writing geoparquet

3 participants