forked from kubeflow/pipelines
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Define a script to close obsolete PRs to update an application. * In the event there are multiple PRs open to update a Kubeflow application we want to close the older PRs; so there is a single open PR updating the application to the newest code. Related to kubeflow#571 * Define a script to close obsolete PRs to update an application. * In the event there are multiple PRs open to update a Kubeflow application we want to close the older PRs; so there is a single open PR updating the application to the newest code. Related to kubeflow#571 * Setup a def namespace for use with apps-cd. * Update update_kf_apps.py to close old PRs on each sync. * Bake the source code into the docker image rather than using a wrapper script to sync the code from git. * Sync'ing the code from git became to difficult to reason about once we start splitting the source code across multiple repositories * We now depend on github/kubeflow/code-intelligence for utilities for working with GitHub Apps * Using a docker image also ensures we don't get broken suddenly when new changes are in place * In the future we could use github actions to automate updating the deployment on postsubmits * Turn app-pipeline.template.yaml into a ConfigMap * This allows better versioning * We can rely on kustomize to create a configmap with a hash based on the contents * kustomize will then reference the config map using its hash. As a result a rolling update is triggered whenever the hash contents changes. * This makes it easier to handle rollous and updates. Define a dev instance of the update KF apps infrastructure to facilitate development * Use profiles in skaffold. * update_kf_apps.py in dev uses a config map now to ubtain app-pipeline.template.yaml rather than fetching it from git * This makes it much easier to test changes in the dev instance Fix a bunch of bugs preventing update_kf_apps.py from working * Update requirements.txt with a bunch of missing packages. * Fix some imports in update_kf_apps.py * Need to set resource requests for the build pods otherwise builds get CPU starved and take forever. Miscellaneous * Create a tool to copy secrets between namespaces from GCS * Fix lint. * Due to kubeflow#460 we need to disable pylint.
- Loading branch information
1 parent
7464a22
commit 65c84fd
Showing
22 changed files
with
645 additions
and
358 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
apiVersion: apps/v1 | ||
kind: Deployment | ||
metadata: | ||
name: server | ||
spec: | ||
template: | ||
spec: | ||
containers: | ||
- name: app | ||
image: gcr.io/kubeflow-releasing/update_kf_apps | ||
command: | ||
# In the dev environment we run with auto_restart for compatibility with | ||
# skaffold | ||
- python | ||
- run_with_auto_restart.py | ||
- --dir=/app | ||
- -- | ||
- python | ||
- -m | ||
- kubeflow.testing.cd.update_kf_apps | ||
- sync | ||
# repo is the code to check out to get the configuration as well as code for | ||
# launching the applications | ||
# | ||
# To test changes before they are checked in you can set this to the repo containing your PR | ||
# e.g. | ||
# Extra arguments to be passed to update_kf_apps.py | ||
- --namespace=kf-releasing-dev | ||
- --config=https://raw.githubusercontent.com/jlewi/testing/cicd_delete_old_prs/apps-cd/applications.yaml | ||
- --output_dir=/tmp/runs | ||
- --src_dir=/src | ||
- --template=/app/config/app-pipeline.template.yaml | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
apiVersion: kustomize.config.k8s.io/v1beta1 | ||
kind: Kustomization | ||
bases: | ||
- ../../base | ||
commonLabels: | ||
environment: dev | ||
namespace: kf-releasing-dev | ||
patchesStrategicMerge: | ||
- deployment.yaml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
apiVersion: kustomize.config.k8s.io/v1beta1 | ||
kind: Kustomization | ||
bases: | ||
- ../../base | ||
commonLabels: | ||
environment: prod | ||
namespace: kf-releasing |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
-i https://pypi.org/simple | ||
asn1crypto==0.24.0 | ||
cffi==1.13.2 | ||
fire==0.2.1 | ||
github3.py==1.3.0 | ||
google-api-core==1.14.2 | ||
google-api-python-client==1.7.10 | ||
google-auth==1.6.3 | ||
google-auth-httplib2==0.0.3 | ||
google-cloud-core==1.0.3 | ||
google-cloud-storage==1.17.0 | ||
json-log-formatter==0.2.0 | ||
jwcrypto==0.6.0 | ||
kubernetes==9.0.0 | ||
pyjwt==1.7.1 | ||
requests==2.22.0 | ||
requests-oauthlib==1.2.0 | ||
requests-toolbelt==0.9.1 | ||
tqdm==4.42.0 | ||
watchdog==0.9.0 |
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.