Skip to content

chore: update hermetic build workflow #11114

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JoeWang1127 merged 1 commit into from
Sep 3, 2024
Merged

chore: update hermetic build workflow #11114

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 17 additions & 17 deletions .github/scripts/hermetic_library_generation.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,11 @@ set -e
# 1. git
# 2. gh
# 3. docker
# 4. mvn

# The parameters of this script is:
# 1. target_branch, the branch into which the pull request is merged.
# 2. current_branch, the branch with which the pull request is associated.
# 3. [optional] image_tag, the tag of gcr.io/cloud-devrel-public-resources/java-library-generation.
# The value will be parsed from the generation configuration if not specified.
# 4. [optional] generation_config, the path to the generation configuration,
# 3. [optional] generation_config, the path to the generation configuration,
# the default value is generation_config.yaml in the repository root.
while [[ $# -gt 0 ]]; do
key="$1"
Expand All @@ -37,10 +34,6 @@ case "${key}" in
current_branch="$2"
shift
;;
--image_tag)
image_tag="$2"
shift
;;
--generation_config)
generation_config="$2"
shift
Expand All @@ -54,22 +47,18 @@ shift
done

if [ -z "${target_branch}" ]; then
echo "missing required argument --target_branch"
echo "Missing required argument --target_branch"
exit 1
fi

if [ -z "${current_branch}" ]; then
echo "missing required argument --current_branch"
echo "Missing required argument --current_branch"
exit 1
fi

if [ -z "${generation_config}" ]; then
generation_config=generation_config.yaml
echo "Use default generation config: ${generation_config}"
Copy link
Contributor

@blakeli0 blakeli0 Sep 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we keep the image tag as a parameter? In long term, we do want to pass it in and remove the generator version from the generation_config.yaml.

Copy link
Contributor Author

@JoeWang1127 JoeWang1127 Sep 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I missed this comment. I'll create a follow up.

fi

if [ -z "${image_tag}" ]; then
image_tag=$(grep "gapic_generator_version" "${generation_config}" | cut -d ':' -f 2 | xargs)
echo "Using default generation config: ${generation_config}"
fi

workspace_name="/workspace"
Expand All @@ -85,10 +74,14 @@ if [[ ! ("${change_of_last_commit}" == *"${generation_config}"*) ]]; then
echo "The last commit doesn't contain any changes to the generation_config.yaml, skipping the whole generation process." || true
exit 0
fi

# copy generation configuration from target branch to current branch.
git show "${target_branch}":"${generation_config}" > "${baseline_generation_config}"
config_diff=$(diff "${generation_config}" "${baseline_generation_config}" || true)

# parse image tag from the generation configuration.
image_tag=$(grep "gapic_generator_version" "${generation_config}" | cut -d ':' -f 2 | xargs)

# get .m2 folder so it's mapped into the docker container
m2_folder=$(dirname "$(mvn help:evaluate -Dexpression=settings.localRepository -q -DforceStdout)")

Expand All @@ -101,8 +94,15 @@ docker run \
gcr.io/cloud-devrel-public-resources/java-library-generation:"${image_tag}" \
--baseline-generation-config-path="${workspace_name}/${baseline_generation_config}" \
--current-generation-config-path="${workspace_name}/${generation_config}"

# commit the change to the pull request.
git add java-* pom.xml gapic-libraries-bom/pom.xml versions.txt
if [[ $(basename "$(pwd)") == "google-cloud-java" ]]; then
Copy link
Contributor

@blakeli0 blakeli0 Sep 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have to differentiate between google-cloud-java and java-* repos? Can we just do git add .?

git add java-* pom.xml gapic-libraries-bom/pom.xml versions.txt
else
# The image leaves intermediate folders and files it works with. Here we remove them
rm -rdf output googleapis "${baseline_generation_config}"
git add --all -- ':!pr_description.txt'
fi
changed_files=$(git diff --cached --name-only)
if [[ "${changed_files}" == "" ]]; then
echo "There is no generated code change with the generation config change ${config_diff}."
Expand All @@ -118,4 +118,4 @@ git push
if [[ -f "pr_description.txt" ]]; then
pr_num=$(gh pr list -s open -H "${current_branch}" -q . --json number | jq ".[] | .number")
gh pr edit "${pr_num}" --body "$(cat pr_description.txt)"
fi
fi
11 changes: 8 additions & 3 deletions .github/workflows/hermetic_library_generation.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,11 @@ name: Hermetic library generation upon generation config change through pull req
on:
pull_request:

env:
REPO_FULL_NAME: ${{ github.event.pull_request.head.repo.full_name }}
GITHUB_REPOSITORY: ${{ github.repository }}
jobs:
library_generation:
# skip pull requests come from a forked repository
if: github.event.pull_request.head.repo.full_name == github.repository
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
Expand All @@ -31,6 +32,10 @@ jobs:
shell: bash
run: |
set -x
if [[ "${GITHUB_REPOSITORY}" != "${REPO_FULL_NAME}" ]]; then
echo "This PR comes from a fork. Skip library generation."
exit 0
fi
[ -z "$(git config user.email)" ] && git config --global user.email "cloud-java-bot@google.com"
[ -z "$(git config user.name)" ] && git config --global user.name "cloud-java-bot"
bash .github/scripts/hermetic_library_generation.sh \
Expand All @@ -39,4 +44,4 @@ jobs:
env:
base_ref: ${{ github.base_ref }}
head_ref: ${{ github.head_ref }}
GH_TOKEN: ${{ secrets.CLOUD_JAVA_BOT_TOKEN }}
GH_TOKEN: ${{ secrets.CLOUD_JAVA_BOT_TOKEN }}