How to handle PRs with breaking API changes between "breaking API" releases

[alamb]

Our release schedule is here https://github.com/apache/arrow-rs?tab=readme-ov-file#release-versioning-and-schedule

This implies we need to wait until after our August 2024 release to have any new breaking API changes (aka arrow 53.0.0).

The question is "what do we do between now, and when we release arrow 53.0.0 with PRs that have breaking API changes such as #5845

I can see three options:

Option 1: Hold Them

• Continue to do all releases off the master branch.
• Keep any PRs with breaking API changes in draft (or mark it with a label or something) and wait until master opens for the next breaking API release (in this case after the August 2024 release)

Option 2: Merge to a 53.0.0 feature branch

• Continue to do all releases off the master branch
• Merge any PRs with breaking API changes to a new integration branch (53.0.0-dev or something)
• When master opens for 53.0.0 merge all changes from the 53.0.0-dev branch to master

Option 3: Merge to main branch

• In this model we would do minor releases off separate branches and main would remain open for all PRs
• Merge all PRs to master (including those with breaking changes)
• Create branches for minor releases (e.g. a 52.1.0-dev branch) and backport any commits from master that should be included in the release

Analysis of options

I think Option 1 is the simplest but runs the risk of PRs sitting for months, bitrotting, and then a merge-tastrophe when we try to merge many outstanding PRs at once. This would be especially bad if there are PRs with large API changes prone to accumulating conflicts

Option 2 might make the bit-rot problem better, but requires someone to actively keep that branch sync'd with main / resolve any conflicts that arise. This would likely fall on the maintainers to do

Option 3 keeps the workflow simple, but now requires any contributors to create two PRs (one to main and one to the dev branch) to get things out in minor releases

[alamb]

I think after the analysis above, I suggest we go with Option 1

[Xuanwo]

I think after the analysis above, I suggest we go with Option 1

Given the current maintenance status, I recommend choosing Option 1. Please ensure that the contributor is informed about the status in advance.

[alamb]

Please ensure that the contributor is informed about the status in advance.

That is a good point -- if we come to consensus on an approach, I will make a PR that updates https://github.com/apache/arrow-rs?tab=readme-ov-file#release-versioning-and-schedule / the contributor guide with the information on breaking API changes

[tustvold]

Option 1 I think makes the right tradeoff of making the process for non-breaking changes as simple as possible

[alamb]

Here is a PR documenting Option 1 as the policy: #5953