-
Notifications
You must be signed in to change notification settings - Fork 18
Add Automation of Python SDK Release Process. #168
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
MOmarMiraj
wants to merge
67
commits into
main
Choose a base branch
from
omar/automate-wheel-build
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
67 commits
Select commit
Hold shift + click to select a range
88398fb
add wheel building action
MOmarMiraj 8eac2b7
fix manylinux image and dont build wheels for PyPi
MOmarMiraj c2627e6
go to 2_34 manylinux image as theres no 2_32 image
MOmarMiraj ea82680
fix linux image name
MOmarMiraj 9742677
add full url for linux image
MOmarMiraj 16a9cb4
update setup.py to build in correct folder
MOmarMiraj cd8b06f
update test cmd path
MOmarMiraj 6f659f6
checkout only the example folder
MOmarMiraj deaeb15
checkout example folder correctly
MOmarMiraj efe387e
edit path
MOmarMiraj 670175f
add support for shared libs packaging
MOmarMiraj 3ad27e9
add cryptography as a wheel
MOmarMiraj 5e84ed0
make it manual
MOmarMiraj 35d2f0d
don't stop if fail
MOmarMiraj 67edcbb
debug working dir
MOmarMiraj 145758d
check file permissions
MOmarMiraj f316654
change to python instead of python3
MOmarMiraj 881497d
remove other jobs and see files path
MOmarMiraj f7a1083
try hardcoded path
MOmarMiraj 1cc21ce
debug
MOmarMiraj f94d10b
see python version and go to 3.12
MOmarMiraj c777afd
try
MOmarMiraj 521efd0
check here
MOmarMiraj bacb8c7
try bilding wheels now
MOmarMiraj efc4ed4
fix path of test and removei nstallation of python
MOmarMiraj 8f19e0d
add python3
MOmarMiraj 1a3e9bb
revert path
MOmarMiraj e3ffc3e
add script
MOmarMiraj ea0c161
add permission
MOmarMiraj fce070e
fix name
MOmarMiraj 8bba2eb
delete script
MOmarMiraj 028d32b
fix test command
MOmarMiraj 2e6941f
pass sa token to linux
MOmarMiraj 4e944bd
add morgan change
MOmarMiraj cd9b562
fix rules of when this job wil run
MOmarMiraj 98f4832
update sa token
MOmarMiraj 8bf9ccb
test on test_client instead of example.py
MOmarMiraj e88f750
skip musllinux wheels
MOmarMiraj 976c276
fix rules for when the job is ran
MOmarMiraj 2a09d16
check if its manual and its a release branch
MOmarMiraj 1fc3895
add input
MOmarMiraj c3f0211
remove the push
MOmarMiraj cc5cf76
fix up yml file
MOmarMiraj 5690c32
revert example file
MOmarMiraj 2345ad1
clean up setup.py
MOmarMiraj a16e535
add build sdist and fix EOL
MOmarMiraj 94fe548
add test job for source distro
MOmarMiraj 1a487ca
test sdist job
MOmarMiraj f7b1d8c
install python dep for source distro
MOmarMiraj 3593dfb
clean up .yml file
MOmarMiraj 212f8a6
add publishing of pypi to workflow
MOmarMiraj fa1c3ec
fix wheels to correctly build and publish wheels to pypi and add pypr…
MOmarMiraj 314cdc7
add new line at EOF
MOmarMiraj a004bf2
Add prep-release and release and update wheels according to fork of P…
MOmarMiraj 43548d1
remove release notes and add comments
MOmarMiraj 2474110
Merge branch 'main' into omar/automate-wheel-build
MOmarMiraj 5750b30
remove unnecessary c omments
MOmarMiraj 95c3aee
update setting up git user to use person who commited to be the one w…
MOmarMiraj 0a8e17a
Apply suggestions from code review
MOmarMiraj 55c7792
Merge branch 'main' into omar/automate-wheel-build
MOmarMiraj c63f59b
Clean up files and change version.txt to .VERSION
MOmarMiraj 6891a34
remove wheels.yml and add building+publishing to release
MOmarMiraj 62c6f61
add passphrase and token env variables for signing commits
MOmarMiraj 2e6d5e5
Remove prep release action and combine it with the release action and…
MOmarMiraj 1debd81
add new line at EOF of version file and fix script to trim it. Addres…
MOmarMiraj 6b1d7eb
fail prep release if on CI to avoid inf loop
MOmarMiraj 4699a2e
update wheels script to match CI/CD incase of error
MOmarMiraj File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
0.3.1 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,172 @@ | ||
name: Release Python SDKs | ||
|
||
on: | ||
workflow_dispatch: | ||
inputs: | ||
version: | ||
description: "New version for the SDKs in the format of M.m.p" | ||
required: true | ||
type: string | ||
build_number: | ||
description: "New build number for the SDKs in the format of Mmmppbb " | ||
required: true | ||
type: string | ||
|
||
jobs: | ||
prepare-release: | ||
runs-on: ubuntu-latest | ||
if: startsWith(github.ref, 'refs/heads/sdk-core/') # Only run on branches that start with sdk-core/ | ||
env: | ||
SDK_VERSION: ${{ github.event.inputs.version }} | ||
SDK_BUILD_NUMBER: ${{ github.event.inputs.build_number }} | ||
steps: | ||
- name: Checkout the code | ||
uses: actions/checkout@v4 | ||
|
||
- name: Import GPG key | ||
uses: crazy-max/ghaction-import-gpg@v6 | ||
with: | ||
gpg_private_key: ${{ secrets.GPG_PRIVATE_KEY }} | ||
passphrase: ${{ secrets.PASSPHRASE }} | ||
git_user_signingkey: true | ||
git_commit_gpgsign: true | ||
MOmarMiraj marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
- name: Run the Prep Release Script | ||
env: | ||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
SDK_CI: 1 | ||
run: | | ||
make prep-release VERSION=${{ env.SDK_VERSION }} BUILD_NUMBER=${{ env.SDK_BUILD_NUMBER }} | ||
shell: bash | ||
|
||
build-wheels: | ||
name: Build wheels for Python SDK on ${{ matrix.os }} | ||
runs-on: ${{ matrix.os }} | ||
if: startsWith(github.ref, 'refs/heads/sdk-core/') | ||
needs: [prepare-release] | ||
strategy: | ||
# we don't want all of them failing if one fails | ||
fail-fast: false | ||
matrix: | ||
# macOS 13 is an Intel runner and macOS 14 is an Apple Silicon runner | ||
os: [ubuntu-22.04, ubuntu-22.04-arm, windows-latest, macos-13, macos-14] | ||
steps: | ||
- uses: actions/checkout@v4 | ||
with: | ||
ref: ${{ github.ref }} | ||
|
||
- name: Upgrade build dependencies | ||
run: python -m pip install --upgrade pip setuptools wheel | ||
MOmarMiraj marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
# Need to grab the SDK version for the wheel name | ||
- name: Extract SDK Version | ||
run: echo "SDK_VERSION=$(cat .VERSION)" >> "$GITHUB_ENV" | ||
shell: bash | ||
|
||
- name: Install cibuildwheel | ||
run: | | ||
python -m pip install cibuildwheel | ||
|
||
- name: Build wheels | ||
env: | ||
# Skip all the unneeded images like PyPy and musllinux images | ||
CIBW_SKIP: pp* *-musllinux_* | ||
# Set the manylinux X86_64/aarch64 image to this specific 2.34 image | ||
CIBW_MANYLINUX_X86_64_IMAGE: "quay.io/pypa/manylinux_2_34_x86_64" | ||
CIBW_MANYLINUX_AARCH64_IMAGE: "quay.io/pypa/manylinux_2_34_aarch64" | ||
# Set the wheel to the native archtiecture (output of platform.machine() which we use in the setup.py script) | ||
CIBW_ARCHS: "native" | ||
# Windows reparing of wheels are not supported so manually install it. | ||
CIBW_BEFORE_BUILD_WINDOWS: "pip install delvewheel" | ||
CIBW_REPAIR_WHEEL_COMMAND_WINDOWS: "delvewheel repair -w {dest_dir} {wheel}" | ||
# Dependencies required for testing each wheel | ||
CIBW_TEST_REQUIRES: "pydantic pytest pytest-asyncio" | ||
# The minimum MacOS version for darwin wheels (matches OPH) | ||
MACOSX_DEPLOYMENT_TARGET: "12.0" | ||
# The command to test every wheel | ||
CIBW_TEST_COMMAND: "python -m pytest {project}/src/onepassword/test_client.py" | ||
# Pass the service account token in all wheel buildings for testing of the wheels. | ||
CIBW_ENVIRONMENT: OP_SERVICE_ACCOUNT_TOKEN=${{ secrets.TEST_SERVICE_ACCOUNT_TOKEN }} | ||
run: | | ||
python -m cibuildwheel --output-dir dist | ||
|
||
- uses: actions/upload-artifact@v4 | ||
with: | ||
name: onepassword-sdk-${{ env.SDK_VERSION }}-${{ matrix.os }} | ||
path: ./dist/*.whl | ||
|
||
build-sdist: | ||
name: Build source distribution for Python SDK | ||
runs-on: ubuntu-latest | ||
if: startsWith(github.ref, 'refs/heads/sdk-core/') | ||
needs: [prepare-release] | ||
steps: | ||
- uses: actions/checkout@v4 | ||
with: | ||
ref: ${{ github.ref }} | ||
# Need to grab the SDK version for the wheel name | ||
- name: Extract SDK Version | ||
run: echo "SDK_VERSION=$(cat .VERSION)" >> "$GITHUB_ENV" | ||
shell: bash | ||
|
||
- name: Install dependencies | ||
run: pip3 install build pydantic pytest pytest-asyncio | ||
|
||
- name: Build source distribution | ||
run: python3 -m build --sdist | ||
|
||
- name: Test Source Distribution | ||
env: | ||
OP_SERVICE_ACCOUNT_TOKEN: ${{ secrets.TEST_SERVICE_ACCOUNT_TOKEN }} | ||
run: | | ||
python3 -m pip install dist/*.tar.gz | ||
python3 -m pytest src/onepassword/test_client.py | ||
|
||
- uses: actions/upload-artifact@v4 | ||
with: | ||
name: onepassword-sdk-${{ env.SDK_VERSION }} | ||
path: ./dist/*.tar.gz | ||
|
||
Release-SDK: | ||
MOmarMiraj marked this conversation as resolved.
Show resolved
Hide resolved
|
||
runs-on: ubuntu-latest | ||
if: startsWith(github.ref, 'refs/heads/sdk-core/') # Only run on branches that start with sdk-core/ | ||
needs: [build-wheels, build-sdist] | ||
steps: | ||
- name: Checkout the code | ||
uses: actions/checkout@v4 | ||
with: | ||
ref: ${{ github.ref }} | ||
|
||
- name: Import GPG key | ||
uses: crazy-max/ghaction-import-gpg@v6 | ||
with: | ||
gpg_private_key: ${{ secrets.GPG_PRIVATE_KEY }} | ||
passphrase: ${{ secrets.PASSPHRASE }} | ||
git_user_signingkey: true | ||
git_commit_gpgsign: true | ||
git_tag_gpgsign: true | ||
|
||
- name: Run the Release Script | ||
env: | ||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
run: make release | ||
shell: bash | ||
|
||
publish-to-pypi: | ||
name: Publish to PyPI | ||
runs-on: ubuntu-latest | ||
if: startsWith(github.ref, 'refs/heads/sdk-core/') | ||
environment: | ||
name: pypi | ||
url: https://pypi.org/project/onepassword-sdk/ | ||
permissions: | ||
id-token: write # IMPORTANT: this permission is mandatory for trusted publishing | ||
needs: [Release-SDK] | ||
steps: | ||
- uses: actions/download-artifact@v4 | ||
with: | ||
pattern: onepassword-sdk-* | ||
path: ./dist | ||
merge-multiple: true | ||
- name: Publish package distributions to PyPi | ||
uses: pypa/gh-action-pypi-publish@release/v1.12 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
[build-system] | ||
requires = ["setuptools>=66", "wheel"] | ||
build-backend = "setuptools.build_meta" | ||
|
||
[project] | ||
name = "onepassword-sdk" | ||
MOmarMiraj marked this conversation as resolved.
Show resolved
Hide resolved
|
||
dynamic = ["version"] | ||
description = "The 1Password Python SDK offers programmatic read access to your secrets in 1Password in an interface native to Python." | ||
authors = [{ name = "1Password" }] | ||
license = { file = "LICENSE" } | ||
readme = "README.md" | ||
requires-python = ">=3.9" | ||
classifiers = [ | ||
"Development Status :: 5 - Production/Stable", | ||
"Operating System :: MacOS", | ||
"Operating System :: POSIX :: Linux", | ||
"Operating System :: Microsoft :: Windows", | ||
"Programming Language :: Python :: 3.9", | ||
"Programming Language :: Python :: 3.10", | ||
"Programming Language :: Python :: 3.11", | ||
"Programming Language :: Python :: 3.12", | ||
"Programming Language :: Python :: 3.13", | ||
"License :: OSI Approved :: MIT License", | ||
] | ||
dependencies = [ | ||
"pydantic>=2.5", | ||
] | ||
|
||
[project.urls] | ||
Homepage = "https://github.com/1Password/onepassword-sdk-python" | ||
|
||
[tool.setuptools.dynamic] | ||
version = {file = "./.VERSION"} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: are we able to enforce this more? E.g. with a regex, also matching on the part that comes after sdk-core/?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we can but we Github Actions don't support regex so we would have to create a whole job to check this which I don't think its necessary. In our case, we want to ensure the branches are in forms of the sdk-core/yyyy-mm-dd-hash but regardless we don't have any checks to ensure that the RC branch are in this form or we verify the commit sha so this additional regex check can add additional verification but for very little upside.