Skip to content

Commit 2dd17b9

Browse files
compheadmartin-g
comphead
and
martin-g
authored
chore: Add script to protect RC branches during the release (apache#18660)
## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123. --> - Closes apache#17134 ## Rationale for this change ASF Infra works in tricky way, to apply a protection for the new branch its needed to get: - branch created in repo - add rules to `.asf.yaml` in `main` - tricky part ASF reapply policies ONLY if `.asf.yaml` changed in PR, means that "future" branches won't be covered until the file changed and pushed. Adding a script to add automatically a block with new RC branch protection rules and updated documentation <!-- Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed. Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes. --> ## What changes are included in this PR? <!-- There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR. --> ## Are these changes tested? <!-- We typically require tests for all PRs in order to: 1. Prevent the code from being accidentally broken by subsequent changes 2. Serve as another way to document the expected behavior of the code If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> ## Are there any user-facing changes? <!-- If there are user-facing changes then we may require documentation to be updated before approving the PR. --> <!-- If there are any breaking changes to public APIs, please add the `api change` label. --> --------- Co-authored-by: Martin Grigorov <martin-g@users.noreply.github.com>
1 parent 486c5d8 commit 2dd17b9

File tree

3 files changed

+190
-80
lines changed

3 files changed

+190
-80
lines changed

‎.asf.yaml‎

Lines changed: 2 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -54,72 +54,14 @@ github:
5454
# needs to be updated as part of the release process
5555
# .asf.yaml doesn't support wildcard branch protection rules, only exact branch names
5656
# https://github.com/apache/infrastructure-asfyaml?tab=readme-ov-file#branch-protection
57-
# Keeping set of protected branches for future releases
58-
# Meanwhile creating a prerelease script that will update the branch protection names
59-
# automatically. Keep track on it https://github.com/apache/datafusion/issues/17134
57+
# these branches protection blocks autogenerated during release process which is described in
58+
# https://github.com/apache/datafusion/tree/main/dev/release#2-add-a-protection-to-release-candidate-branch
6059
branch-50:
6160
required_pull_request_reviews:
6261
required_approving_review_count: 1
6362
branch-51:
6463
required_pull_request_reviews:
6564
required_approving_review_count: 1
66-
# branch-52:
67-
# required_pull_request_reviews:
68-
# required_approving_review_count: 1
69-
# branch-53:
70-
# required_pull_request_reviews:
71-
# required_approving_review_count: 1
72-
# branch-54:
73-
# required_pull_request_reviews:
74-
# required_approving_review_count: 1
75-
# branch-55:
76-
# required_pull_request_reviews:
77-
# required_approving_review_count: 1
78-
# branch-56:
79-
# required_pull_request_reviews:
80-
# required_approving_review_count: 1
81-
# branch-57:
82-
# required_pull_request_reviews:
83-
# required_approving_review_count: 1
84-
# branch-58:
85-
# required_pull_request_reviews:
86-
# required_approving_review_count: 1
87-
# branch-59:
88-
# required_pull_request_reviews:
89-
# required_approving_review_count: 1
90-
# branch-60:
91-
# required_pull_request_reviews:
92-
# required_approving_review_count: 1
93-
# branch-61:
94-
# required_pull_request_reviews:
95-
# required_approving_review_count: 1
96-
# branch-62:
97-
# required_pull_request_reviews:
98-
# required_approving_review_count: 1
99-
# branch-63:
100-
# required_pull_request_reviews:
101-
# required_approving_review_count: 1
102-
# branch-64:
103-
# required_pull_request_reviews:
104-
# required_approving_review_count: 1
105-
# branch-65:
106-
# required_pull_request_reviews:
107-
# required_approving_review_count: 1
108-
# branch-66:
109-
# required_pull_request_reviews:
110-
# required_approving_review_count: 1
111-
# branch-67:
112-
# required_pull_request_reviews:
113-
# required_approving_review_count: 1
114-
# branch-68:
115-
# required_pull_request_reviews:
116-
# required_approving_review_count: 1
117-
# branch-69:
118-
# required_pull_request_reviews:
119-
# required_approving_review_count: 1
120-
# branch-70:
121-
# required_pull_request_reviews:
122-
# required_approving_review_count: 1
12365
pull_requests:
12466
# enable updating head branches of pull requests
12567
allow_update_branch: true

‎dev/release/README.md‎

Lines changed: 28 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -26,11 +26,11 @@ Patch releases are made on an adhoc basis, but we try and avoid them given the f
2626
## Release Process Overview
2727

2828
New development happens on the `main` branch.
29-
Releases are made from branches, e.g. `branch-37` for the `37.x.y` release series.
29+
Releases are made from branches, e.g. `branch-50` for the `50.x.y` release series.
3030

3131
To prepare for a new release series, we:
3232

33-
- Create a new branch from `main`, such as `branch-37` in the Apache repository (not in a fork)
33+
- Create a new branch from `main`, such as `branch-50` in the Apache repository (not in a fork)
3434
- Continue merging new features changes to `main` branch
3535
- Prepare the release branch for release:
3636
- Update version numbers in `Cargo.toml` files and create `CHANGELOG.md`
@@ -55,7 +55,7 @@ these steps:
5555

5656
1. Find (or create) the issue for the incremental release ([example release issue]) and discuss the proposed change there with the maintainers.
5757
2. Follow normal workflow to create PR to `main` branch and wait for its approval and merge.
58-
3. After PR is squash merged to `main`, branch from most recent release branch (e.g. `branch-37`), cherry-pick the commit and create a PR targeting the release branch [example backport PR].
58+
3. After PR is squash merged to `main`, branch from most recent release branch (e.g. `branch-50`), cherry-pick the commit and create a PR targeting the release branch [example backport PR].
5959

6060
For example, to backport commit `12345` from `main` to `branch-50`:
6161

@@ -127,16 +127,35 @@ We then publish the code in the approved artifacts to crates.io.
127127

128128
First create a new release branch from `main` in the apache repository.
129129

130-
For example, to create the `branch-51` branch for the `51.x.y` release series:
130+
For example, to create the `branch-50` branch for the `50.x.y` release series:
131131

132132
```shell
133133
git fetch apache # make sure we are up to date
134134
git checkout apache/main # checkout current latest development branch
135-
git checkout -b branch-51 # create local branch
136-
git push -u apache branch-51 # push branch to apache remote
135+
git checkout -b branch-50 # create local branch
136+
git push -u apache branch-50 # push branch to apache remote
137137
```
138138

139-
### 2. Prepare PR to Update Changelog and the Release Version
139+
### 2. Add a protection to release candidate branch
140+
141+
To protect a release candidate branch from accidental merges, run:
142+
143+
```shell
144+
./dev/release/add-branch-protection.sh 50
145+
```
146+
147+
The script will modify `.asf.yaml` and add following block:
148+
149+
```yaml
150+
branch-50:
151+
required_pull_request_reviews:
152+
required_approving_review_count: 1
153+
```
154+
155+
- Create a PR.
156+
- Merge to `main`.
157+
158+
### 3. Prepare PR to Update Changelog and the Release Version
140159

141160
First, prepare a PR to update the changelog and versions to reflect the planned
142161
release. See [#18173](https://github.com/apache/datafusion/pull/18173) for an example.
@@ -202,17 +221,6 @@ git commit -a -m 'Update version'
202221

203222
Remember to merge any fixes back to `main` branch as well.
204223

205-
### 3. Prepare a PR to Modify `asf.yaml`
206-
207-
Modify `asf.yaml` to protect future release candidate branch to prevent accidental merges:
208-
209-
```yaml
210-
# needs to be updated as part of the release process
211-
branch-50:
212-
required_pull_request_reviews:
213-
required_approving_review_count: 1
214-
```
215-
216224
### 4. Prepare Release Candidate Artifacts
217225

218226
After the PR gets merged, you are ready to create release artifacts based off the
@@ -425,7 +433,7 @@ svn ls https://dist.apache.org/repos/dist/dev/datafusion
425433
Delete a release candidate:
426434

427435
```shell
428-
svn delete -m "delete old DataFusion RC" https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-38.0.0-rc1/
436+
svn delete -m "delete old DataFusion RC" https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-50.0.0-rc1/
429437
```
430438

431439
#### Deleting old releases from `release` svn
@@ -441,5 +449,5 @@ svn ls https://dist.apache.org/repos/dist/release/datafusion
441449
Delete a release:
442450

443451
```shell
444-
svn delete -m "delete old DataFusion release" https://dist.apache.org/repos/dist/release/datafusion/datafusion-37.0.0
452+
svn delete -m "delete old DataFusion release" https://dist.apache.org/repos/dist/release/datafusion/datafusion-50.0.0
445453
```

‎dev/release/add-branch-protection.sh‎

Lines changed: 160 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,160 @@
1+
#!/usr/bin/env bash
2+
# Licensed to the Apache Software Foundation (ASF) under one
3+
# or more contributor license agreements. See the NOTICE file
4+
# distributed with this work for additional information
5+
# regarding copyright ownership. The ASF licenses this file
6+
# to you under the Apache License, Version 2.0 (the
7+
# "License"); you may not use this file except in compliance
8+
# with the License. You may obtain a copy of the License at
9+
#
10+
# http://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing,
13+
# software distributed under the License is distributed on an
14+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
# KIND, either express or implied. See the License for the
16+
# specific language governing permissions and limitations
17+
# under the License.
18+
19+
set -eu
20+
21+
# Script to add branch protection for a new release branch in .asf.yaml
22+
#
23+
# This script automates the process of adding branch protection rules to .asf.yaml
24+
# for new release branches. It ensures the branch protection block doesn't already
25+
# exist before adding it.
26+
#
27+
# Usage:
28+
# ./dev/release/add-branch-protection.sh <release_number>
29+
#
30+
# Examples:
31+
# ./dev/release/add-branch-protection.sh 52
32+
# ./dev/release/add-branch-protection.sh 53
33+
#
34+
# The script will:
35+
# 1. Validate the release number is a positive integer
36+
# 2. Check if branch protection already exists for branch-<release_number>
37+
# 3. Add the branch protection block to .asf.yaml if it doesn't exist
38+
# 4. Error out if the block already exists
39+
40+
# Check if release number is provided
41+
if [ $# -eq 0 ]; then
42+
echo "Error: Release number is required"
43+
echo "Usage: $0 <release_number>"
44+
echo "Example: $0 52"
45+
exit 1
46+
fi
47+
48+
RELEASE_NUM=$1
49+
BRANCH_NAME="branch-${RELEASE_NUM}"
50+
ASF_YAML_FILE=".asf.yaml"
51+
52+
# Validate release number is a positive integer
53+
if ! [[ "$RELEASE_NUM" =~ ^[0-9]+$ ]]; then
54+
echo "Error: Release number must be a positive integer"
55+
echo "Provided: $RELEASE_NUM"
56+
echo "Example: ./dev/release/add-branch-protection.sh 52"
57+
exit 1
58+
fi
59+
60+
# Check if .asf.yaml exists
61+
if [ ! -f "$ASF_YAML_FILE" ]; then
62+
echo "Error: $ASF_YAML_FILE not found in current directory"
63+
echo "Please run this script from the repository root"
64+
exit 1
65+
fi
66+
67+
# Check if the branch exists in the official Apache DataFusion repository
68+
GITHUB_API_URL="https://api.github.com/repos/apache/datafusion/branches/${BRANCH_NAME}"
69+
HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" "$GITHUB_API_URL")
70+
71+
if [ "$HTTP_STATUS" != "200" ]; then
72+
echo "Error: Branch ${BRANCH_NAME} does not exist in the official Apache DataFusion repository"
73+
echo "Please create the branch '${BRANCH_NAME}' first before adding branch protection"
74+
echo ""
75+
echo "To check existing branches, visit:"
76+
echo " https://github.com/apache/datafusion/branches"
77+
exit 1
78+
fi
79+
80+
# Check if branch protection already exists for this release
81+
if grep -q "^[[:space:]]*${BRANCH_NAME}:" "$ASF_YAML_FILE"; then
82+
echo "Error: Branch protection for ${BRANCH_NAME} already exists in $ASF_YAML_FILE"
83+
exit 1
84+
fi
85+
86+
# Create a temporary file
87+
TEMP_FILE=$(mktemp)
88+
89+
# Read the file and insert the new branch protection block
90+
# We'll insert it after the last branch-XX block
91+
awk -v branch="$BRANCH_NAME" '
92+
/^[[:space:]]*branch-[0-9]+:/ {
93+
last_branch_line = NR
94+
last_branch_content = $0
95+
}
96+
{
97+
lines[NR] = $0
98+
}
99+
END {
100+
if (last_branch_line == 0) {
101+
print "Error: No existing branch protection blocks found" > "/dev/stderr"
102+
exit 1
103+
}
104+
105+
# Print all lines up to and including the last branch block
106+
for (i = 1; i <= last_branch_line; i++) {
107+
print lines[i]
108+
}
109+
110+
# Print the required_pull_request_reviews lines after the last branch
111+
for (i = last_branch_line + 1; i <= NR; i++) {
112+
print lines[i]
113+
# After printing the required_approving_review_count line, insert new branch
114+
if (lines[i] ~ /required_approving_review_count:/) {
115+
# Check if this belongs to the last branch block by looking ahead
116+
next_non_empty = i + 1
117+
while (next_non_empty <= NR && lines[next_non_empty] ~ /^[[:space:]]*$/) {
118+
next_non_empty++
119+
}
120+
# If next non-empty line is not indented more than branch level, we found the end
121+
if (next_non_empty > NR || lines[next_non_empty] !~ /^[[:space:]]{6,}/) {
122+
print " " branch ":"
123+
print " required_pull_request_reviews:"
124+
print " required_approving_review_count: 1"
125+
# Skip to next iteration to avoid double printing
126+
for (j = i + 1; j <= NR; j++) {
127+
i = j
128+
if (j <= NR) print lines[j]
129+
}
130+
break
131+
}
132+
}
133+
}
134+
}
135+
' "$ASF_YAML_FILE" > "$TEMP_FILE"
136+
137+
# Check if awk succeeded
138+
if [ $? -ne 0 ]; then
139+
rm -f "$TEMP_FILE"
140+
exit 1
141+
fi
142+
143+
# Verify the new content was added
144+
if ! grep -q "^[[:space:]]*${BRANCH_NAME}:" "$TEMP_FILE"; then
145+
echo "Error: Failed to add branch protection block"
146+
rm -f "$TEMP_FILE"
147+
exit 1
148+
fi
149+
150+
# Replace the original file with the modified version
151+
mv "$TEMP_FILE" "$ASF_YAML_FILE"
152+
153+
echo "Successfully added branch protection for ${BRANCH_NAME} to $ASF_YAML_FILE"
154+
echo ""
155+
echo "Added block:"
156+
echo " ${BRANCH_NAME}:"
157+
echo " required_pull_request_reviews:"
158+
echo " required_approving_review_count: 1"
159+
echo ""
160+
echo "Please review the changes and commit them."

0 commit comments

Comments
 (0)