gcp cred bug fix for multiple credentials in single pipeline #1384

aakashbajaj · 2019-05-24T09:47:58Z

Fix for #1376 #1379
secret_file_path_in_volume was not updated if admin-gcp-sa was passed in secret_name.
Different volume mounts are now created for different service account credentials.

In newer versions, volume names will be automatically generated as mentioned in beec660
So that can be removed later.

This change is

googlebot · 2019-05-24T09:48:04Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here (e.g. I signed it!) and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

k8s-ci-robot · 2019-05-24T09:48:12Z

Hi @aakashbajaj. Thanks for your PR.

I'm waiting for a kubeflow member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

aakashbajaj · 2019-05-24T09:50:16Z

I signed it!

googlebot · 2019-05-24T09:50:19Z

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

aakashbajaj · 2019-05-24T10:07:08Z

/assign @gaoning777

Ark-kun · 2019-07-24T01:28:25Z

I think that users should still be able to override secret_file_path_in_volume. With your change it's ignored.

aakashbajaj · 2019-07-26T09:52:36Z

@Ark-kun fixed that.

sdk/python/kfp/gcp.py

Ark-kun · 2019-08-01T22:09:37Z

/ok-to-test

googlebot · 2019-08-02T05:26:06Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

googlebot · 2019-08-02T05:26:06Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for all the commit author(s) or Co-authors. If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.
In order to pass this check, please resolve this problem and then comment @googlebot I fixed it.. If the bot doesn't comment, it means it doesn't think anything has changed.

ℹ️ Googlers: Go here for more info.

aakashbajaj · 2019-08-02T05:28:26Z

@googlebot I fixed it.

Ark-kun · 2019-08-03T02:15:18Z

/lgtm
/approve

k8s-ci-robot · 2019-08-03T02:15:21Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Ark-kun

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~sdk/OWNERS~~ [Ark-kun]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

aakashbajaj · 2019-08-06T17:44:01Z

@googlebot I fixed it.

Ark-kun · 2019-08-12T21:02:14Z

Sorry. Can you please squash your commits and force-push so that my e-mail is not part of them (hope that squashing will get rid of it)? It's confusing the CLA bot.

googlebot · 2019-08-16T19:36:44Z

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

aakashbajaj · 2019-08-16T20:17:14Z

/retest

…ntial_fix

aakashbajaj · 2019-08-17T14:37:10Z

@Ark-kun can you take a look at the issues?
Is it some permission issue or is it because this branch is behind the master that its not compatible with the latest deployment process?

aakashbajaj · 2019-08-17T14:37:47Z

I should create a new PR with the just the changes to gcp.py file

Ark-kun · 2019-08-20T00:36:05Z

/retest

Ark-kun · 2019-08-20T00:41:08Z

@Ark-kun can you take a look at the issues?

Just flaky tests. It should be merged soon.

aakashbajaj · 2019-08-20T03:26:04Z

/retest

Ark-kun · 2019-08-20T06:39:06Z

/lgtm

@Ark-kun

commit 2592307 Author: Christian Clauss <cclauss@me.com> Date: Wed Aug 21 04:15:19 2019 +0200 Undefined name 'e' in openvino (kubeflow#1876) Discovered in kubeflow#1721 ``` ./contrib/components/openvino/ovms-deployer/containers/evaluate.py:62:16: F821 undefined name 'e' except e: ^ ./contrib/components/openvino/ovms-deployer/containers/evaluate.py:63:50: F821 undefined name 'e' print("Can not read the image file", e) ^ ``` Your review please @Ark-kun commit 7b442f4 Author: Kirin Patel <kirinpatel@gmail.com> Date: Tue Aug 20 18:27:19 2019 -0700 Created extensible code editor based on react-ace (kubeflow#1855) * Created extensible code editor based on react-ace * Installed dependencies * Updated unit tests for Editor.tsx to test placeholder and value in simplified manner * Updated Editor unit tests to use snapshot testing where applicable commit d11fae7 Author: Yuan (Bob) Gong <gongyuan94@gmail.com> Date: Wed Aug 21 08:25:20 2019 +0800 Use KFP lite deployment for presubmit tests (kubeflow#1808) * Refactor presubmit-tests-with-pipeline-deployment.sh so that it can be run from a different project * Simplify getting service account from cluster. * Migrate presubmit-tests-with-pipeline-deployment.sh to use kfp lightweight deployment. * Add option to cache built images to make debugging faster. * Fix cluster set up * Copy image builder image instead of granting permission * Add missed yes command * fix stuff * Let other usages of image-builder image become configurable * let test workflow use image builder image * Fix permission issue * Hide irrelevant error logs * Use shared service account key instead * Move test manifest to test folder * Move build-images.sh to a different script file * Update README.md * add cluster info dump * Use the same cluster resources as kubeflow deployment * Remove cluster info dump * Add timing to test log * cleaned up code * fix tests * address cr comments * Address cr comments * Enable image caching to improve retest speed commit 0864faf Author: IronPan <yangpa@google.com> Date: Tue Aug 20 14:09:19 2019 -0700 Use single part as default (kubeflow#1893) The data stored in artifact storage are usually small. Using multi-part is not strictly a requirement. Change the default to true to better support more platform out of box. commit 6284dc1 Author: Christian Clauss <cclauss@me.com> Date: Tue Aug 20 22:01:18 2019 +0200 IBM Watson samples: from six.moves import xrange (kubeflow#1877) Discovered in kubeflow#1721 __xrange()__ was removed in Python 3 in favor of an improved version of __range()__. This PR ensures equivalent functionality in both Python 2 and Python 3. ``` ./samples/contrib/ibm-samples/watson/source/model-source-code/tf-model/input_data.py:100:40: F821 undefined name 'xrange' fake_image = [1.0 for _ in xrange(784)] ^ ./samples/contrib/ibm-samples/watson/source/model-source-code/tf-model/input_data.py:102:41: F821 undefined name 'xrange' return [fake_image for _ in xrange(batch_size)], [ ^ ./samples/contrib/ibm-samples/watson/source/model-source-code/tf-model/input_data.py:103:37: F821 undefined name 'xrange' fake_label for _ in xrange(batch_size)] ^ ``` @gaoning777 @Ark-kun Your reviews please. commit 79c7bda Author: Ning <ngao@google.com> Date: Tue Aug 20 09:24:56 2019 -0700 fix unit tests and address some comments (kubeflow#1892) commit 6a7b28f Author: Aakash Bajaj <aakashbajaj5589@gmail.com> Date: Tue Aug 20 14:07:33 2019 +0530 gcp cred bug fix for multiple credentials in single pipeline (kubeflow#1384) * gcp cred bug fix for multiple credentials in single pipeline * squash to remove conflict commit 60fd70c Author: Andy Wei <jwwandy@gmail.com> Date: Tue Aug 20 15:33:33 2019 +0800 Let backend apiserver mysql dbname configurable (kubeflow#1714) * let mysql dbname configurable * solve conflict * move declaration out to fix scope commit 101a346 Author: olegchorny <oleg.chorny@gmail.com> Date: Tue Aug 20 08:33:47 2019 +0300 'core' folder included to parameters related On-Premise cluster (kubeflow#1751) * 'core' folder included to parameters related On-Premise cluster Update is required because this sample was migrated to the samples/core folder * Update README.md commit 4c5d34f Author: dushyanthsc <43390008+dushyanthsc@users.noreply.github.com> Date: Mon Aug 19 18:57:33 2019 -0700 test/project-cleanup: Support to cleanup gke-clusters in test project (kubeflow#1857) Change to add base framework for cleaning up resources in a GCP project. The resource specification is specified declaratively using a YAML file. As per current requirements this change only adds cleaning up of GKE clusters.

@hongye-sun

commit 41d39c1 Author: Kirin Patel <kirinpatel@gmail.com> Date: Wed Aug 21 19:38:31 2019 -0700 Add run with json data as input within fixed-data.ts for UI testing and development (kubeflow#1895) * Added run with json data as input * Changed run and uid to not be duplicates of hello-world-runtime commit 851e7c8 Author: Kirin Patel <kirinpatel@gmail.com> Date: Wed Aug 21 19:04:31 2019 -0700 Replace codemirror editor react component with react-ace editor component (kubeflow#1890) * Replaced CodeMirror with Editor in PipelineDetails.tsx * Replaced codemirror in DetailsTable with react-ace * Removed codemirror * Updated unit tests for Editor.tsx to test placeholder and value in simplified manner * Updated DetailsTable.test.tsx.snap to reflect changes made in DetailsTable.tsx * Updated PipelineDetails test snapshot * Changed width of Editor in DetailsTable to be 100% instead of 300px * Revert "Updated unit tests for Editor.tsx to test placeholder and value in simplified manner" This reverts commit 40103f2. commit 8c3d6fe Author: Kirin Patel <kirinpatel@gmail.com> Date: Wed Aug 21 18:30:33 2019 -0700 Add visualization-server service to lightweight deployment (kubeflow#1844) * Add visualization-server service to lightweight deployment * Addressed PR suggestions * Added field to determine if visualization service is active and fixed unit tests for visualization_server.go * Additional small fixes * port change from 88888 -> 8888 * version change from 0.1.15 -> 0.1.26 * removed visualization-server from base/kustomization.yaml * Fixed visualization_server_test.go to reflect new changes * Changed implementation to be fail fast * Changed host name to be constant provided by environment * Added retry and extracted isVisualizationServiceAlive logic to function * Fixed deployment.yaml file * Fixed serviceURL configuration issuse serviceURL is now properly obtained from the environment, the service ip address and port are used rather than service name and namespace * Added log message to indicate when visualization service is unreachable * Addressed PR comments * Removed _HTTP commit ad307db Author: Eterna2 <eterna2@hotmail.com> Date: Thu Aug 22 08:52:32 2019 +0800 [Bug Fix] Delete ResourceOp should not have output parameters (kubeflow#1822) * Fix bug where delete resource op should not have success_condition, failure_condition, and output parameters * remove unnecessary whitespace * compiler test for delete resource ops should retrieve templates from spec instead of root commit 593f25a Author: Alexey Volkov <avolkov@google.com> Date: Wed Aug 21 17:16:33 2019 -0700 Collecting coverage when running python tests (kubeflow#898) * Collecting coiverage when running python tests * Added coveralls to python unit tests * Try removing the PATH modification * Specifying coverage run --source * Using the installed package * Try getting the correct coverage paths commit 553885f Author: Alexey Volkov <avolkov@google.com> Date: Wed Aug 21 16:38:12 2019 -0700 SDK - Components - Fixed ModelBase comparison bug (kubeflow#1874) commit 2622c67 Author: IronPan <yangpa@google.com> Date: Wed Aug 21 16:37:53 2019 -0700 cleanup test dir (kubeflow#1914) commit 203307d Author: Alexey Volkov <avolkov@google.com> Date: Wed Aug 21 16:37:21 2019 -0700 SDK - Lightweight - Fixed custom types in multi-output case (kubeflow#1875) The type was mistakenly serialized as `_ForwardRef('CustomType')`. The input parameter types and single-output types were not affected. commit 2e7f2d4 Author: IronPan <yangpa@google.com> Date: Wed Aug 21 16:36:35 2019 -0700 Add cloud sql and gcs connection for pipeline-lite deployment (kubeflow#1910) * restructure * working example * working example * move mysql * moving minio and mysql out * add gcp * add files * fix test commit 9adf163 Author: Alexey Volkov <avolkov@google.com> Date: Wed Aug 21 16:29:54 2019 -0700 SDK - Airflow - Fixed bug in airflow op creation (kubeflow#1911) This PR fixes a bug in AirFlow op creation. The `_run_airflow_op` helper function was not captured along with the `_run_airflow_op_closure` function, because they belong to different modules (`_run_airflow_op_closure` was module-less). This was not discovered during the notebook testing of the code since in that environment the `_run_airflow_op` was also module-less as it was defined in a notebook (not in .py file). commit 7ec5697 Author: hongye-sun <43763191+hongye-sun@users.noreply.github.com> Date: Wed Aug 21 16:06:31 2019 -0700 Release 151c534 (kubeflow#1916) * Updated component images to version 151c534 * Updated components to version a97f1d0 * Update setup.py * Update setup.py commit 7e062ce Author: IronPan <yangpa@google.com> Date: Wed Aug 21 15:14:31 2019 -0700 Update README.md commit 8e1e823 Author: Christian Clauss <cclauss@me.com> Date: Thu Aug 22 00:04:31 2019 +0200 Lint Python code for undefined names (kubeflow#1721) * Lint Python code for undefined names * Lint Python code for undefined names * Exclude tfdv.py to workaround an overzealous pytest * Fixup for tfdv.py * Fixup for tfdv.py * Fixup for tfdv.py commit 2399348 Author: dushyanthsc <43390008+dushyanthsc@users.noreply.github.com> Date: Wed Aug 21 13:44:31 2019 -0700 apiserver: Remove TFX output artifact recording to metadatastore (kubeflow#1904) commit 151c534 Author: Alexey Volkov <avolkov@google.com> Date: Wed Aug 21 12:56:33 2019 -0700 Simplified the build_docker_image function (kubeflow#1887) * Simplified the build_docker_image function * Extracted 'Dockerfile' into a variable Addressed PR feedback * Fixed conflict with updated master. * Addressed the feedback commit 3b7340f Author: IronPan <yangpa@google.com> Date: Wed Aug 21 12:28:31 2019 -0700 Change the type of resource reference payload column (kubeflow#1905) Gorm doesn't automatically change the type of a column. This changes introduced a column type change which might not be effective for an existing cluster doing upgrade. kubeflow@4e43750#diff-c4afa92d7e54eecff0a482cf57490aa8R40 /assign @hongye-sun commit a4991fd Author: Kirin Patel <kirinpatel@gmail.com> Date: Wed Aug 21 11:56:31 2019 -0700 Enable error propagation from nbconvert to frontend (kubeflow#1909) commit 94aa078 Author: Ning <ngao@google.com> Date: Wed Aug 21 10:22:31 2019 -0700 remove tfx notebook sample in favor of the TFX OSS sample (kubeflow#1908) commit 93e3121 Author: Ning <ngao@google.com> Date: Wed Aug 21 09:20:39 2019 -0700 remove kubeflow training (kubeflow#1902) commit 2592307 Author: Christian Clauss <cclauss@me.com> Date: Wed Aug 21 04:15:19 2019 +0200 Undefined name 'e' in openvino (kubeflow#1876) Discovered in kubeflow#1721 ``` ./contrib/components/openvino/ovms-deployer/containers/evaluate.py:62:16: F821 undefined name 'e' except e: ^ ./contrib/components/openvino/ovms-deployer/containers/evaluate.py:63:50: F821 undefined name 'e' print("Can not read the image file", e) ^ ``` Your review please @Ark-kun commit 7b442f4 Author: Kirin Patel <kirinpatel@gmail.com> Date: Tue Aug 20 18:27:19 2019 -0700 Created extensible code editor based on react-ace (kubeflow#1855) * Created extensible code editor based on react-ace * Installed dependencies * Updated unit tests for Editor.tsx to test placeholder and value in simplified manner * Updated Editor unit tests to use snapshot testing where applicable commit d11fae7 Author: Yuan (Bob) Gong <gongyuan94@gmail.com> Date: Wed Aug 21 08:25:20 2019 +0800 Use KFP lite deployment for presubmit tests (kubeflow#1808) * Refactor presubmit-tests-with-pipeline-deployment.sh so that it can be run from a different project * Simplify getting service account from cluster. * Migrate presubmit-tests-with-pipeline-deployment.sh to use kfp lightweight deployment. * Add option to cache built images to make debugging faster. * Fix cluster set up * Copy image builder image instead of granting permission * Add missed yes command * fix stuff * Let other usages of image-builder image become configurable * let test workflow use image builder image * Fix permission issue * Hide irrelevant error logs * Use shared service account key instead * Move test manifest to test folder * Move build-images.sh to a different script file * Update README.md * add cluster info dump * Use the same cluster resources as kubeflow deployment * Remove cluster info dump * Add timing to test log * cleaned up code * fix tests * address cr comments * Address cr comments * Enable image caching to improve retest speed commit 0864faf Author: IronPan <yangpa@google.com> Date: Tue Aug 20 14:09:19 2019 -0700 Use single part as default (kubeflow#1893) The data stored in artifact storage are usually small. Using multi-part is not strictly a requirement. Change the default to true to better support more platform out of box. commit 6284dc1 Author: Christian Clauss <cclauss@me.com> Date: Tue Aug 20 22:01:18 2019 +0200 IBM Watson samples: from six.moves import xrange (kubeflow#1877) Discovered in kubeflow#1721 __xrange()__ was removed in Python 3 in favor of an improved version of __range()__. This PR ensures equivalent functionality in both Python 2 and Python 3. ``` ./samples/contrib/ibm-samples/watson/source/model-source-code/tf-model/input_data.py:100:40: F821 undefined name 'xrange' fake_image = [1.0 for _ in xrange(784)] ^ ./samples/contrib/ibm-samples/watson/source/model-source-code/tf-model/input_data.py:102:41: F821 undefined name 'xrange' return [fake_image for _ in xrange(batch_size)], [ ^ ./samples/contrib/ibm-samples/watson/source/model-source-code/tf-model/input_data.py:103:37: F821 undefined name 'xrange' fake_label for _ in xrange(batch_size)] ^ ``` @gaoning777 @Ark-kun Your reviews please. commit 79c7bda Author: Ning <ngao@google.com> Date: Tue Aug 20 09:24:56 2019 -0700 fix unit tests and address some comments (kubeflow#1892) commit 6a7b28f Author: Aakash Bajaj <aakashbajaj5589@gmail.com> Date: Tue Aug 20 14:07:33 2019 +0530 gcp cred bug fix for multiple credentials in single pipeline (kubeflow#1384) * gcp cred bug fix for multiple credentials in single pipeline * squash to remove conflict commit 60fd70c Author: Andy Wei <jwwandy@gmail.com> Date: Tue Aug 20 15:33:33 2019 +0800 Let backend apiserver mysql dbname configurable (kubeflow#1714) * let mysql dbname configurable * solve conflict * move declaration out to fix scope commit 101a346 Author: olegchorny <oleg.chorny@gmail.com> Date: Tue Aug 20 08:33:47 2019 +0300 'core' folder included to parameters related On-Premise cluster (kubeflow#1751) * 'core' folder included to parameters related On-Premise cluster Update is required because this sample was migrated to the samples/core folder * Update README.md commit 4c5d34f Author: dushyanthsc <43390008+dushyanthsc@users.noreply.github.com> Date: Mon Aug 19 18:57:33 2019 -0700 test/project-cleanup: Support to cleanup gke-clusters in test project (kubeflow#1857) Change to add base framework for cleaning up resources in a GCP project. The resource specification is specified declaratively using a YAML file. As per current requirements this change only adds cleaning up of GKE clusters. commit 0ed5819 Author: Yuan (Bob) Gong <gongyuan94@gmail.com> Date: Tue Aug 20 08:13:31 2019 +0800 Refactor presubmit-tests-with-pipeline-deployment.sh to run in other projects (kubeflow#1732) * Refactor presubmit-tests-with-pipeline-deployment.sh so that it can be run from a different project * Simplify getting service account from cluster. * Copy image builder image instead of granting permission * Add missed yes command * fix stuff * Let other usages of image-builder image become configurable * let test workflow use image builder image commit 0369a4c Author: Ajay Gopinathan <ajaygopinathan@google.com> Date: Fri Aug 16 21:02:07 2019 -0700 Update manifests to point to 0.26 release. (kubeflow#1870) commit 2b246bc Author: Alexey Volkov <avolkov@google.com> Date: Fri Aug 16 19:54:07 2019 -0700 SDK - Tests - Improved the "ContainerOp.set_retry" test (kubeflow#1843) Properly testing the feature isntead of just comparing with golden data. commit d66508d Author: Ajay Gopinathan <ajaygopinathan@google.com> Date: Fri Aug 16 18:42:48 2019 -0700 Update changelog for 0.1.26 (kubeflow#1872) * Update changelog for 0.1.26 * Update changelog. commit 1d99704 Author: Alexey Volkov <avolkov@google.com> Date: Fri Aug 16 18:24:09 2019 -0700 SDK - Fixed string comparisons (kubeflow#1756) commit 6d5ffa2 Author: Ajay Gopinathan <ajaygopinathan@google.com> Date: Fri Aug 16 16:57:33 2019 -0700 Remove copying of tfx data for cloudbuild release steps. (kubeflow#1871) commit de538aa Author: Ning <ngao@google.com> Date: Fri Aug 16 15:53:07 2019 -0700 add compile step in the samples to generate zip files (kubeflow#1866) * add compile step in the samples to generate zip files commit 5bcc665 Author: Ajay Gopinathan <ajaygopinathan@google.com> Date: Fri Aug 16 13:58:23 2019 -0700 Update Python SDK versions for release. (kubeflow#1860) commit 0d898cb Author: Ajay Gopinathan <ajaygopinathan@google.com> Date: Fri Aug 16 13:16:11 2019 -0700 Release 0517114 (kubeflow#1859) * Updated component images to version 0517114 * Updated components to version 48dd338 commit 7dbca1a Author: Ning <ngao@google.com> Date: Fri Aug 16 12:06:37 2019 -0700 update gcloud ml-engine to ai-platform (kubeflow#1863) commit e849f22 Author: Ryan Dawson <ryandawson@cantab.net> Date: Fri Aug 16 19:21:23 2019 +0100 Seldon examples (kubeflow#1405) commit a6f3f40 Author: sina chavoshi <sina.chavoshi@gmail.com> Date: Fri Aug 16 09:56:09 2019 -0700 Adding a sample for serving component (kubeflow#1830) * Adding a sample for serving component * removed typo / updated based on PR feedback * fixing the jupyter rendering issue * adding pip3 for tensorflow * Fixed spelling error in VERSION * fix indentation based on review feedback commit 54ff3e6 Author: Alexey Volkov <avolkov@google.com> Date: Fri Aug 16 01:22:31 2019 -0700 SDK - Cleanup - Serialized PipelineParamTuple does not need value or type (kubeflow#1469) * SDK - Refactoring - Serialized PipelineParam does not need type Only the types in non-serialized PipelineParams are ever used. * SDK - Refactoring - Serialized PipelineParam does not need value Default values are only relevant when PipelineParam is used in the pipeline function signature and even in this case compiler captures them explicitly from the pipelineParam objects in the signature. There is no other uses for them. commit d2e94e4 Author: Riley Bauer <34456002+rileyjbauer@users.noreply.github.com> Date: Thu Aug 15 19:52:34 2019 -0700 Fix run duration bug (kubeflow#1827) * Allows durations >=24h and renames 'showLink' in RunList * Update, fix tests commit d8eaeaa Author: Alexey Volkov <avolkov@google.com> Date: Thu Aug 15 17:25:59 2019 -0700 SDK - Preserving the pipeline input information in the compiled Workflow (kubeflow#1381) * SDK - Preserving the pipeline metadata in the compiled Workflow * Stabilizing the DSL compiler tests commit afe8a69 Author: Kirin Patel <kirinpatel@gmail.com> Date: Thu Aug 15 17:00:28 2019 -0700 Reduce API usage by utilizing reference name in reference resource API (kubeflow#1824) * Regenerated run api for frontend * Added support for reference name to resource reference API in frontend * Revert "Regenerated run api for frontend" * Addressed PR comments * Removed extra if statement by setting default value of parameter * Removed the whole comment * Addressed PR feedback * Addressed PR feedback * Simplified logic after offline discussion commit ea67c99 Author: Kirin Patel <kirinpatel@gmail.com> Date: Thu Aug 15 16:18:35 2019 -0700 Change how Variables are Provided to Visualizations (kubeflow#1754) * Changed way visualization variables are passed from request to NotebookNode Visualization variables are now saved to a json file and loaded by a NotebookNode upon execution. * Updated roc_curve visualization to reflect changes made to dependency injection * Fixed bug where checking if is_generated is provided to roc_curve visualization would crash visualizaiton Also changed ' -> " * Changed text_exporter to always sort variables by key for testing * Addressed PR suggestions commit d238bef Author: Jiaxiao Zheng <jxzheng@google.com> Date: Thu Aug 15 12:55:29 2019 -0700 Add back coveralls. (kubeflow#1849) * Remove redundant import. * Simplify sample_test.yaml by using withItem syntax. * Simplify sample_test.yaml by using withItem syntax. * Change dict to str in withItems. * Add back coveralls. commit 0517114 Author: Riley Bauer <34456002+rileyjbauer@users.noreply.github.com> Date: Thu Aug 15 12:28:35 2019 -0700 Reduce getPipeline calls in RunList (kubeflow#1852) * Skips calling getPipeline in RunList if the pipeline name is in the pipeline_spec * Update fixed data to include pipeline names in pipeline specs * Remove redundant getRuns call commit 39e5840 Author: IronPan <yangpa@google.com> Date: Thu Aug 15 11:04:34 2019 -0700 Add retry button in Pipeline UI (kubeflow#1782) * add retry button * add retry button * add retry button * address comments * fix tests * fix tests * update image * Update StatusUtils.test.tsx * Update RunDetails.test.tsx * Update Buttons.ts * update test * update frontend * update * update * addrerss comments * update test

@Ark-kun

) * Remove redundant import. * Simplify sample_test.yaml by using withItem syntax. * Simplify sample_test.yaml by using withItem syntax. * Change dict to str in withItems. * Add back coveralls. * Squashed commit of the following: commit 7dbca1a Author: Ning <ngao@google.com> Date: Fri Aug 16 12:06:37 2019 -0700 update gcloud ml-engine to ai-platform (#1863) commit e849f22 Author: Ryan Dawson <ryandawson@cantab.net> Date: Fri Aug 16 19:21:23 2019 +0100 Seldon examples (#1405) commit a6f3f40 Author: sina chavoshi <sina.chavoshi@gmail.com> Date: Fri Aug 16 09:56:09 2019 -0700 Adding a sample for serving component (#1830) * Adding a sample for serving component * removed typo / updated based on PR feedback * fixing the jupyter rendering issue * adding pip3 for tensorflow * Fixed spelling error in VERSION * fix indentation based on review feedback commit 54ff3e6 Author: Alexey Volkov <avolkov@google.com> Date: Fri Aug 16 01:22:31 2019 -0700 SDK - Cleanup - Serialized PipelineParamTuple does not need value or type (#1469) * SDK - Refactoring - Serialized PipelineParam does not need type Only the types in non-serialized PipelineParams are ever used. * SDK - Refactoring - Serialized PipelineParam does not need value Default values are only relevant when PipelineParam is used in the pipeline function signature and even in this case compiler captures them explicitly from the pipelineParam objects in the signature. There is no other uses for them. commit d2e94e4 Author: Riley Bauer <34456002+rileyjbauer@users.noreply.github.com> Date: Thu Aug 15 19:52:34 2019 -0700 Fix run duration bug (#1827) * Allows durations >=24h and renames 'showLink' in RunList * Update, fix tests commit d8eaeaa Author: Alexey Volkov <avolkov@google.com> Date: Thu Aug 15 17:25:59 2019 -0700 SDK - Preserving the pipeline input information in the compiled Workflow (#1381) * SDK - Preserving the pipeline metadata in the compiled Workflow * Stabilizing the DSL compiler tests commit afe8a69 Author: Kirin Patel <kirinpatel@gmail.com> Date: Thu Aug 15 17:00:28 2019 -0700 Reduce API usage by utilizing reference name in reference resource API (#1824) * Regenerated run api for frontend * Added support for reference name to resource reference API in frontend * Revert "Regenerated run api for frontend" * Addressed PR comments * Removed extra if statement by setting default value of parameter * Removed the whole comment * Addressed PR feedback * Addressed PR feedback * Simplified logic after offline discussion commit ea67c99 Author: Kirin Patel <kirinpatel@gmail.com> Date: Thu Aug 15 16:18:35 2019 -0700 Change how Variables are Provided to Visualizations (#1754) * Changed way visualization variables are passed from request to NotebookNode Visualization variables are now saved to a json file and loaded by a NotebookNode upon execution. * Updated roc_curve visualization to reflect changes made to dependency injection * Fixed bug where checking if is_generated is provided to roc_curve visualization would crash visualizaiton Also changed ' -> " * Changed text_exporter to always sort variables by key for testing * Addressed PR suggestions commit d238bef Author: Jiaxiao Zheng <jxzheng@google.com> Date: Thu Aug 15 12:55:29 2019 -0700 Add back coveralls. (#1849) * Remove redundant import. * Simplify sample_test.yaml by using withItem syntax. * Simplify sample_test.yaml by using withItem syntax. * Change dict to str in withItems. * Add back coveralls. commit 0517114 Author: Riley Bauer <34456002+rileyjbauer@users.noreply.github.com> Date: Thu Aug 15 12:28:35 2019 -0700 Reduce getPipeline calls in RunList (#1852) * Skips calling getPipeline in RunList if the pipeline name is in the pipeline_spec * Update fixed data to include pipeline names in pipeline specs * Remove redundant getRuns call commit 39e5840 Author: IronPan <yangpa@google.com> Date: Thu Aug 15 11:04:34 2019 -0700 Add retry button in Pipeline UI (#1782) * add retry button * add retry button * add retry button * address comments * fix tests * fix tests * update image * Update StatusUtils.test.tsx * Update RunDetails.test.tsx * Update Buttons.ts * update test * update frontend * update * update * addrerss comments * update test * WIP: rewrite sample test infra step 1, presubmit check triggered by fire. * WIP: rewrite sample test infra step 2, add ComponentTest class * WIP: rewrite sample test infra Rewrite dockerfile * Fixing * Refactor the logic of run_test.sh into sample_test_launcher.py * Fix exit_code * Fix imports and add todo * Squashed commit of the following: commit 0ed5819 Author: Yuan (Bob) Gong <gongyuan94@gmail.com> Date: Tue Aug 20 08:13:31 2019 +0800 Refactor presubmit-tests-with-pipeline-deployment.sh to run in other projects (#1732) * Refactor presubmit-tests-with-pipeline-deployment.sh so that it can be run from a different project * Simplify getting service account from cluster. * Copy image builder image instead of granting permission * Add missed yes command * fix stuff * Let other usages of image-builder image become configurable * let test workflow use image builder image * Clean up. * Fix launcher flag. * switch to sys.execxutable. * Fix bugs * Add copy blob util function fix issues, and Squashed commit of the following: commit 0ed5819 Author: Yuan (Bob) Gong <gongyuan94@gmail.com> Date: Tue Aug 20 08:13:31 2019 +0800 Refactor presubmit-tests-with-pipeline-deployment.sh to run in other projects (#1732) * Refactor presubmit-tests-with-pipeline-deployment.sh so that it can be run from a different project * Simplify getting service account from cluster. * Copy image builder image instead of granting permission * Add missed yes command * fix stuff * Let other usages of image-builder image become configurable * let test workflow use image builder image commit 0369a4c Author: Ajay Gopinathan <ajaygopinathan@google.com> Date: Fri Aug 16 21:02:07 2019 -0700 Update manifests to point to 0.26 release. (#1870) commit 2b246bc Author: Alexey Volkov <avolkov@google.com> Date: Fri Aug 16 19:54:07 2019 -0700 SDK - Tests - Improved the "ContainerOp.set_retry" test (#1843) Properly testing the feature isntead of just comparing with golden data. commit d66508d Author: Ajay Gopinathan <ajaygopinathan@google.com> Date: Fri Aug 16 18:42:48 2019 -0700 Update changelog for 0.1.26 (#1872) * Update changelog for 0.1.26 * Update changelog. commit 1d99704 Author: Alexey Volkov <avolkov@google.com> Date: Fri Aug 16 18:24:09 2019 -0700 SDK - Fixed string comparisons (#1756) commit 6d5ffa2 Author: Ajay Gopinathan <ajaygopinathan@google.com> Date: Fri Aug 16 16:57:33 2019 -0700 Remove copying of tfx data for cloudbuild release steps. (#1871) commit de538aa Author: Ning <ngao@google.com> Date: Fri Aug 16 15:53:07 2019 -0700 add compile step in the samples to generate zip files (#1866) * add compile step in the samples to generate zip files commit 5bcc665 Author: Ajay Gopinathan <ajaygopinathan@google.com> Date: Fri Aug 16 13:58:23 2019 -0700 Update Python SDK versions for release. (#1860) commit 0d898cb Author: Ajay Gopinathan <ajaygopinathan@google.com> Date: Fri Aug 16 13:16:11 2019 -0700 Release 0517114 (#1859) * Updated component images to version 0517114 * Updated components to version 48dd338 commit 7dbca1a Author: Ning <ngao@google.com> Date: Fri Aug 16 12:06:37 2019 -0700 update gcloud ml-engine to ai-platform (#1863) commit e849f22 Author: Ryan Dawson <ryandawson@cantab.net> Date: Fri Aug 16 19:21:23 2019 +0100 Seldon examples (#1405) commit a6f3f40 Author: sina chavoshi <sina.chavoshi@gmail.com> Date: Fri Aug 16 09:56:09 2019 -0700 Adding a sample for serving component (#1830) * Adding a sample for serving component * removed typo / updated based on PR feedback * fixing the jupyter rendering issue * adding pip3 for tensorflow * Fixed spelling error in VERSION * fix indentation based on review feedback commit 54ff3e6 Author: Alexey Volkov <avolkov@google.com> Date: Fri Aug 16 01:22:31 2019 -0700 SDK - Cleanup - Serialized PipelineParamTuple does not need value or type (#1469) * SDK - Refactoring - Serialized PipelineParam does not need type Only the types in non-serialized PipelineParams are ever used. * SDK - Refactoring - Serialized PipelineParam does not need value Default values are only relevant when PipelineParam is used in the pipeline function signature and even in this case compiler captures them explicitly from the pipelineParam objects in the signature. There is no other uses for them. commit d2e94e4 Author: Riley Bauer <34456002+rileyjbauer@users.noreply.github.com> Date: Thu Aug 15 19:52:34 2019 -0700 Fix run duration bug (#1827) * Allows durations >=24h and renames 'showLink' in RunList * Update, fix tests commit d8eaeaa Author: Alexey Volkov <avolkov@google.com> Date: Thu Aug 15 17:25:59 2019 -0700 SDK - Preserving the pipeline input information in the compiled Workflow (#1381) * SDK - Preserving the pipeline metadata in the compiled Workflow * Stabilizing the DSL compiler tests commit afe8a69 Author: Kirin Patel <kirinpatel@gmail.com> Date: Thu Aug 15 17:00:28 2019 -0700 Reduce API usage by utilizing reference name in reference resource API (#1824) * Regenerated run api for frontend * Added support for reference name to resource reference API in frontend * Revert "Regenerated run api for frontend" * Addressed PR comments * Removed extra if statement by setting default value of parameter * Removed the whole comment * Addressed PR feedback * Addressed PR feedback * Simplified logic after offline discussion commit ea67c99 Author: Kirin Patel <kirinpatel@gmail.com> Date: Thu Aug 15 16:18:35 2019 -0700 Change how Variables are Provided to Visualizations (#1754) * Changed way visualization variables are passed from request to NotebookNode Visualization variables are now saved to a json file and loaded by a NotebookNode upon execution. * Updated roc_curve visualization to reflect changes made to dependency injection * Fixed bug where checking if is_generated is provided to roc_curve visualization would crash visualizaiton Also changed ' -> " * Changed text_exporter to always sort variables by key for testing * Addressed PR suggestions commit d238bef Author: Jiaxiao Zheng <jxzheng@google.com> Date: Thu Aug 15 12:55:29 2019 -0700 Add back coveralls. (#1849) * Remove redundant import. * Simplify sample_test.yaml by using withItem syntax. * Simplify sample_test.yaml by using withItem syntax. * Change dict to str in withItems. * Add back coveralls. commit 0517114 Author: Riley Bauer <34456002+rileyjbauer@users.noreply.github.com> Date: Thu Aug 15 12:28:35 2019 -0700 Reduce getPipeline calls in RunList (#1852) * Skips calling getPipeline in RunList if the pipeline name is in the pipeline_spec * Update fixed data to include pipeline names in pipeline specs * Remove redundant getRuns call commit 39e5840 Author: IronPan <yangpa@google.com> Date: Thu Aug 15 11:04:34 2019 -0700 Add retry button in Pipeline UI (#1782) * add retry button * add retry button * add retry button * address comments * fix tests * fix tests * update image * Update StatusUtils.test.tsx * Update RunDetails.test.tsx * Update Buttons.ts * update test * update frontend * update * update * addrerss comments * update test * Fix default image prefix. * Fix dependencies. * Fix gs op * Lint * Fix papermill exec * Init for componentTest * Debugging. * Try to fix the issue. Possible cause: nteract/papermill#410 * Fix exit code * Another fix. * Squashed commit of the following: commit 2592307 Author: Christian Clauss <cclauss@me.com> Date: Wed Aug 21 04:15:19 2019 +0200 Undefined name 'e' in openvino (#1876) Discovered in #1721 ``` ./contrib/components/openvino/ovms-deployer/containers/evaluate.py:62:16: F821 undefined name 'e' except e: ^ ./contrib/components/openvino/ovms-deployer/containers/evaluate.py:63:50: F821 undefined name 'e' print("Can not read the image file", e) ^ ``` Your review please @Ark-kun commit 7b442f4 Author: Kirin Patel <kirinpatel@gmail.com> Date: Tue Aug 20 18:27:19 2019 -0700 Created extensible code editor based on react-ace (#1855) * Created extensible code editor based on react-ace * Installed dependencies * Updated unit tests for Editor.tsx to test placeholder and value in simplified manner * Updated Editor unit tests to use snapshot testing where applicable commit d11fae7 Author: Yuan (Bob) Gong <gongyuan94@gmail.com> Date: Wed Aug 21 08:25:20 2019 +0800 Use KFP lite deployment for presubmit tests (#1808) * Refactor presubmit-tests-with-pipeline-deployment.sh so that it can be run from a different project * Simplify getting service account from cluster. * Migrate presubmit-tests-with-pipeline-deployment.sh to use kfp lightweight deployment. * Add option to cache built images to make debugging faster. * Fix cluster set up * Copy image builder image instead of granting permission * Add missed yes command * fix stuff * Let other usages of image-builder image become configurable * let test workflow use image builder image * Fix permission issue * Hide irrelevant error logs * Use shared service account key instead * Move test manifest to test folder * Move build-images.sh to a different script file * Update README.md * add cluster info dump * Use the same cluster resources as kubeflow deployment * Remove cluster info dump * Add timing to test log * cleaned up code * fix tests * address cr comments * Address cr comments * Enable image caching to improve retest speed commit 0864faf Author: IronPan <yangpa@google.com> Date: Tue Aug 20 14:09:19 2019 -0700 Use single part as default (#1893) The data stored in artifact storage are usually small. Using multi-part is not strictly a requirement. Change the default to true to better support more platform out of box. commit 6284dc1 Author: Christian Clauss <cclauss@me.com> Date: Tue Aug 20 22:01:18 2019 +0200 IBM Watson samples: from six.moves import xrange (#1877) Discovered in #1721 __xrange()__ was removed in Python 3 in favor of an improved version of __range()__. This PR ensures equivalent functionality in both Python 2 and Python 3. ``` ./samples/contrib/ibm-samples/watson/source/model-source-code/tf-model/input_data.py:100:40: F821 undefined name 'xrange' fake_image = [1.0 for _ in xrange(784)] ^ ./samples/contrib/ibm-samples/watson/source/model-source-code/tf-model/input_data.py:102:41: F821 undefined name 'xrange' return [fake_image for _ in xrange(batch_size)], [ ^ ./samples/contrib/ibm-samples/watson/source/model-source-code/tf-model/input_data.py:103:37: F821 undefined name 'xrange' fake_label for _ in xrange(batch_size)] ^ ``` @gaoning777 @Ark-kun Your reviews please. commit 79c7bda Author: Ning <ngao@google.com> Date: Tue Aug 20 09:24:56 2019 -0700 fix unit tests and address some comments (#1892) commit 6a7b28f Author: Aakash Bajaj <aakashbajaj5589@gmail.com> Date: Tue Aug 20 14:07:33 2019 +0530 gcp cred bug fix for multiple credentials in single pipeline (#1384) * gcp cred bug fix for multiple credentials in single pipeline * squash to remove conflict commit 60fd70c Author: Andy Wei <jwwandy@gmail.com> Date: Tue Aug 20 15:33:33 2019 +0800 Let backend apiserver mysql dbname configurable (#1714) * let mysql dbname configurable * solve conflict * move declaration out to fix scope commit 101a346 Author: olegchorny <oleg.chorny@gmail.com> Date: Tue Aug 20 08:33:47 2019 +0300 'core' folder included to parameters related On-Premise cluster (#1751) * 'core' folder included to parameters related On-Premise cluster Update is required because this sample was migrated to the samples/core folder * Update README.md commit 4c5d34f Author: dushyanthsc <43390008+dushyanthsc@users.noreply.github.com> Date: Mon Aug 19 18:57:33 2019 -0700 test/project-cleanup: Support to cleanup gke-clusters in test project (#1857) Change to add base framework for cleaning up resources in a GCP project. The resource specification is specified declaratively using a YAML file. As per current requirements this change only adds cleaning up of GKE clusters. * Fix working dir * Fix exit code format * Remove kfp notebook sample using tfx:oss components. * Refactor to reduce potential dup code. * Clear unused const. * Fix redundant check * Add image injection for component test. * Fix unused import * Squash and merge from master * Squashed commit of the following: commit 41d39c1 Author: Kirin Patel <kirinpatel@gmail.com> Date: Wed Aug 21 19:38:31 2019 -0700 Add run with json data as input within fixed-data.ts for UI testing and development (#1895) * Added run with json data as input * Changed run and uid to not be duplicates of hello-world-runtime commit 851e7c8 Author: Kirin Patel <kirinpatel@gmail.com> Date: Wed Aug 21 19:04:31 2019 -0700 Replace codemirror editor react component with react-ace editor component (#1890) * Replaced CodeMirror with Editor in PipelineDetails.tsx * Replaced codemirror in DetailsTable with react-ace * Removed codemirror * Updated unit tests for Editor.tsx to test placeholder and value in simplified manner * Updated DetailsTable.test.tsx.snap to reflect changes made in DetailsTable.tsx * Updated PipelineDetails test snapshot * Changed width of Editor in DetailsTable to be 100% instead of 300px * Revert "Updated unit tests for Editor.tsx to test placeholder and value in simplified manner" This reverts commit 40103f2. commit 8c3d6fe Author: Kirin Patel <kirinpatel@gmail.com> Date: Wed Aug 21 18:30:33 2019 -0700 Add visualization-server service to lightweight deployment (#1844) * Add visualization-server service to lightweight deployment * Addressed PR suggestions * Added field to determine if visualization service is active and fixed unit tests for visualization_server.go * Additional small fixes * port change from 88888 -> 8888 * version change from 0.1.15 -> 0.1.26 * removed visualization-server from base/kustomization.yaml * Fixed visualization_server_test.go to reflect new changes * Changed implementation to be fail fast * Changed host name to be constant provided by environment * Added retry and extracted isVisualizationServiceAlive logic to function * Fixed deployment.yaml file * Fixed serviceURL configuration issuse serviceURL is now properly obtained from the environment, the service ip address and port are used rather than service name and namespace * Added log message to indicate when visualization service is unreachable * Addressed PR comments * Removed _HTTP commit ad307db Author: Eterna2 <eterna2@hotmail.com> Date: Thu Aug 22 08:52:32 2019 +0800 [Bug Fix] Delete ResourceOp should not have output parameters (#1822) * Fix bug where delete resource op should not have success_condition, failure_condition, and output parameters * remove unnecessary whitespace * compiler test for delete resource ops should retrieve templates from spec instead of root commit 593f25a Author: Alexey Volkov <avolkov@google.com> Date: Wed Aug 21 17:16:33 2019 -0700 Collecting coverage when running python tests (#898) * Collecting coiverage when running python tests * Added coveralls to python unit tests * Try removing the PATH modification * Specifying coverage run --source * Using the installed package * Try getting the correct coverage paths commit 553885f Author: Alexey Volkov <avolkov@google.com> Date: Wed Aug 21 16:38:12 2019 -0700 SDK - Components - Fixed ModelBase comparison bug (#1874) commit 2622c67 Author: IronPan <yangpa@google.com> Date: Wed Aug 21 16:37:53 2019 -0700 cleanup test dir (#1914) commit 203307d Author: Alexey Volkov <avolkov@google.com> Date: Wed Aug 21 16:37:21 2019 -0700 SDK - Lightweight - Fixed custom types in multi-output case (#1875) The type was mistakenly serialized as `_ForwardRef('CustomType')`. The input parameter types and single-output types were not affected. commit 2e7f2d4 Author: IronPan <yangpa@google.com> Date: Wed Aug 21 16:36:35 2019 -0700 Add cloud sql and gcs connection for pipeline-lite deployment (#1910) * restructure * working example * working example * move mysql * moving minio and mysql out * add gcp * add files * fix test commit 9adf163 Author: Alexey Volkov <avolkov@google.com> Date: Wed Aug 21 16:29:54 2019 -0700 SDK - Airflow - Fixed bug in airflow op creation (#1911) This PR fixes a bug in AirFlow op creation. The `_run_airflow_op` helper function was not captured along with the `_run_airflow_op_closure` function, because they belong to different modules (`_run_airflow_op_closure` was module-less). This was not discovered during the notebook testing of the code since in that environment the `_run_airflow_op` was also module-less as it was defined in a notebook (not in .py file). commit 7ec5697 Author: hongye-sun <43763191+hongye-sun@users.noreply.github.com> Date: Wed Aug 21 16:06:31 2019 -0700 Release 151c534 (#1916) * Updated component images to version 151c534 * Updated components to version a97f1d0 * Update setup.py * Update setup.py commit 7e062ce Author: IronPan <yangpa@google.com> Date: Wed Aug 21 15:14:31 2019 -0700 Update README.md commit 8e1e823 Author: Christian Clauss <cclauss@me.com> Date: Thu Aug 22 00:04:31 2019 +0200 Lint Python code for undefined names (#1721) * Lint Python code for undefined names * Lint Python code for undefined names * Exclude tfdv.py to workaround an overzealous pytest * Fixup for tfdv.py * Fixup for tfdv.py * Fixup for tfdv.py * # Conflicts: # sdk/python/tests/components/test_python_op.py * Fix version for papermill * Squashed commit of the following: commit af456bc Author: Eterna2 <eterna2@hotmail.com> Date: Fri Aug 23 02:08:33 2019 +0800 [front-end-server] Allow viewer:tensorboard podTemplateSpec to be customizable (#1906) * Allow front-end server to provide custom viewer podTemplateSpec via path/configmap * Fix JSON.parse input to string commit 06b7ad6 Author: IronPan <yangpa@google.com> Date: Thu Aug 22 10:03:28 2019 -0700 Cleanup pipeline-lite deployment (#1921) * restructure * working example * working example * move mysql * moving minio and mysql out * add gcp * add files * fix test * extract parameters to single place * update * update readme * update readme * address pr comment commit 56160f1 Author: Kirin Patel <kirinpatel@gmail.com> Date: Thu Aug 22 09:26:33 2019 -0700 Added support for environment specified kernel timeout (#1920) * Fix google-cloud-storage and google fire version info. * Delete run_test.sh * Merge from kfp master * Squashed commit of the following: commit 7b6825d Author: hongye-sun <43763191+hongye-sun@users.noreply.github.com> Date: Thu Aug 22 18:23:18 2019 -0700 Update changelog for 0.1.27 (#1935) * Update changelog for 0.1.27 * Revert unexpected changes. commit 336760c Author: IronPan <yangpa@google.com> Date: Thu Aug 22 17:47:18 2019 -0700 sync namespaced install file (#1932) commit 55d62fe Author: Hamed <hamedhsn@gmail.com> Date: Fri Aug 23 01:09:18 2019 +0100 Support Affinity for ContainerOps (#1886) commit d15697b Author: Animesh Singh <singhan@us.ibm.com> Date: Thu Aug 22 16:29:17 2019 -0700 Initial kfserving pipeline component (#1838) * initial kfserving pipeline component * changing some descriptions * adding action as an input, and pinning versions * moving Dockerfile in correct location * adding namespace to delete commit 119405d Author: hongye-sun <43763191+hongye-sun@users.noreply.github.com> Date: Thu Aug 22 15:49:29 2019 -0700 upgrade backend image versions (#1918) commit c01315a Author: Alexey Volkov <avolkov@google.com> Date: Thu Aug 22 15:31:24 2019 -0700 SDK - Refactoring - Replaced the TypeMeta class (#1930) * SDK - Refactoring - Replaced the TypeMeta class The PipelineParam no longer exposes the private TypeMeta class Fixes #1420 The refactoring PR is part of a series of PR which unifies the metadata and specification types. commit 736c7a5 Author: IronPan <yangpa@google.com> Date: Thu Aug 22 15:29:19 2019 -0700 clean up owner file (#1928) commit 5abba66 Author: Kirin Patel <kirinpatel@gmail.com> Date: Thu Aug 22 14:45:18 2019 -0700 Add pipeline id to pipeline summary card (#1927) * Added pipeline id to pipeline summary card * Updated PipelineDetails.test.tsx.snap commit 3c8952e Author: Kirin Patel <kirinpatel@gmail.com> Date: Thu Aug 22 13:57:20 2019 -0700 Add TFDV, TFMA, and Table visualization support for Python based visualizations (#1898) * Added table and tfdv visualization Also fixed issue surrounding ApiVisualizationType enum * Fixed table visualization * Removed byte limit * Fixed issue where headers would not properly be applied * Fixed issue where table would not be intractable * Updated table visualizaiton to reflect changes made to dependency injection * Fixed bug where checking if headers is provided to table visualizations could crash visualization * Added TFMA visualization * Updated new visualizations to match syntax of #1878 * Updated test snapshots to account for TFMA visualization * Small if statement synax changes * Add flake8 noqa comments to table.py and tfma.py commit db6c9b4 Author: dushyanthsc <43390008+dushyanthsc@users.noreply.github.com> Date: Thu Aug 22 13:11:18 2019 -0700 pipeline-lite: Introduce metadata component to pipeline-lite (#1840) This change introduces the metadata component to pipeline-lite installation. This installation: 1. Does not include metadata-ui 2. mysql installation is used instead of metadata-db 3. Replica count has been reduced 1 instead of 3 * Fix check_notebook_results args. * Fix. * Fix work dir path * Fix global const naming convention * Remove redundant check. * Extract project name from result gcs dir. * Add error catching * Name change. * Improve error catching. * Improve error catching. * naming correction.

* Use latest image for default overlay * Rename to irisv2 * Fix release gen script * Move local gateway section * Update quick install * Add kubeflow release assets * Update quick install for 0.5 * Remove crd config * Move sklearn examples * Fix local gateway * fix sklearn iris v2 example * Update docs/samples/v1beta1/sklearn/v1/README.md Co-authored-by: yuzliu <55463421+yuzliu@users.noreply.github.com> Co-authored-by: yuzliu <55463421+yuzliu@users.noreply.github.com>

…eflow#1385) * fix aix mnist example * adapt the aixserver to accept inputs on the explainer's parameters * Update main README to use v1beta1 sklearn as example (kubeflow#1384) * Use latest image for default overlay * Rename to irisv2 * Fix release gen script * Move local gateway section * Update quick install * Add kubeflow release assets * Update quick install for 0.5 * Remove crd config * Move sklearn examples * Fix local gateway * fix sklearn iris v2 example * Update docs/samples/v1beta1/sklearn/v1/README.md Co-authored-by: yuzliu <55463421+yuzliu@users.noreply.github.com> Co-authored-by: yuzliu <55463421+yuzliu@users.noreply.github.com> * the original parameters change will not modify parameters from the request * Update MMS readme (kubeflow#1380) * update MMS readme * format * Update docs/MULTIMODELSERVING_GUIDE.md Co-authored-by: Dan Sun <dsun20@bloomberg.net> * Update docs/MULTIMODELSERVING_GUIDE.md Co-authored-by: Dan Sun <dsun20@bloomberg.net> * Update docs/MULTIMODELSERVING_GUIDE.md Co-authored-by: Dan Sun <dsun20@bloomberg.net> * Update docs/MULTIMODELSERVING_GUIDE.md Co-authored-by: Dan Sun <dsun20@bloomberg.net> * Update docs/MULTIMODELSERVING_GUIDE.md Co-authored-by: Dan Sun <dsun20@bloomberg.net> * Update docs/MULTIMODELSERVING_GUIDE.md Co-authored-by: Dan Sun <dsun20@bloomberg.net> * Update docs/MULTIMODELSERVING_GUIDE.md Co-authored-by: Dan Sun <dsun20@bloomberg.net> * update readme * remove duplicate Co-authored-by: Dan Sun <dsun20@bloomberg.net> * Feature to include headers in http/https service requests (non-MMS) (kubeflow#1372) * Added http(s) header support extracted from env var + tar format * Created http(s) uri secret support to extract headers * Created tests for http(s) secret extraction * Added condition to extract secret volume from uri * Added how to set headers for http(s) uri request in secret * Added minor note on formatting * Added minor note on how to convert to base64 * refactored variable name and introduced constant * Secret Data Key Header -> Headers * Changed base-uri -> host-uri for consistency * Checked secret data for https-host-uri instead of annotation * Moved host uri to secret data * Added parsing logic for headers/host uri in secret data * Fixed test to match host uri in secret data * Updated variables for clarity * Stored header as a json string in env * Parse header as json * Update headers to store json * Removed unnecessary code * Update docs/samples/explanation/aix/mnist/README.md * fix aix mnist example * adapt the aixserver to accept inputs on the explainer's parameters * the original parameters change will not modify parameters from the request * Update docs/samples/explanation/aix/mnist/README.md Co-authored-by: Dan Sun <dsun20@bloomberg.net> Co-authored-by: yuzliu <55463421+yuzliu@users.noreply.github.com> Co-authored-by: abchoo <77467773+abchoo@users.noreply.github.com> Co-authored-by: Animesh Singh <singhan@us.ibm.com>

gcp cred bug fix for multiple credentials in single pipeline

7198241

k8s-ci-robot requested review from Ark-kun and gaoning777 May 24, 2019 09:48

k8s-ci-robot added the size/XS label May 24, 2019

k8s-ci-robot added the needs-ok-to-test label May 24, 2019

k8s-ci-robot assigned gaoning777 May 24, 2019

Merge branch 'master' into gcp_credential_fix

483dbca

k8s-ci-robot added size/S and removed size/XS labels Jul 26, 2019

Ark-kun reviewed Aug 1, 2019

View reviewed changes

sdk/python/kfp/gcp.py Outdated Show resolved Hide resolved

Ark-kun reviewed Aug 1, 2019

View reviewed changes

sdk/python/kfp/gcp.py Outdated Show resolved Hide resolved

k8s-ci-robot added ok-to-test and removed needs-ok-to-test labels Aug 1, 2019

k8s-ci-robot assigned Ark-kun Aug 3, 2019

k8s-ci-robot added the lgtm label Aug 3, 2019

k8s-ci-robot added the approved label Aug 3, 2019

squash to remove conflict

f14d225

aakashbajaj force-pushed the gcp_credential_fix branch from 7baf043 to f14d225 Compare August 16, 2019 19:36

k8s-ci-robot removed the lgtm label Aug 16, 2019

Merge branch 'master' of github.com:kubeflow/pipelines into gcp_crede…

97e7b72

…ntial_fix

k8s-ci-robot added the lgtm label Aug 20, 2019

k8s-ci-robot merged commit 6a7b28f into kubeflow:master Aug 20, 2019

aakashbajaj deleted the gcp_credential_fix branch August 20, 2019 08:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gcp cred bug fix for multiple credentials in single pipeline #1384

gcp cred bug fix for multiple credentials in single pipeline #1384

aakashbajaj commented May 24, 2019 •

edited by jlewi

Loading

googlebot commented May 24, 2019

k8s-ci-robot commented May 24, 2019

aakashbajaj commented May 24, 2019

googlebot commented May 24, 2019

aakashbajaj commented May 24, 2019

Ark-kun commented Jul 24, 2019

aakashbajaj commented Jul 26, 2019

Ark-kun commented Aug 1, 2019

googlebot commented Aug 2, 2019

googlebot commented Aug 2, 2019

aakashbajaj commented Aug 2, 2019

Ark-kun commented Aug 3, 2019

k8s-ci-robot commented Aug 3, 2019

aakashbajaj commented Aug 6, 2019

Ark-kun commented Aug 12, 2019

googlebot commented Aug 16, 2019

aakashbajaj commented Aug 16, 2019

aakashbajaj commented Aug 17, 2019

aakashbajaj commented Aug 17, 2019

Ark-kun commented Aug 20, 2019

Ark-kun commented Aug 20, 2019

aakashbajaj commented Aug 20, 2019

Ark-kun commented Aug 20, 2019

gcp cred bug fix for multiple credentials in single pipeline #1384

gcp cred bug fix for multiple credentials in single pipeline #1384

Conversation

aakashbajaj commented May 24, 2019 • edited by jlewi Loading

googlebot commented May 24, 2019

What to do if you already signed the CLA

Individual signers

Corporate signers

k8s-ci-robot commented May 24, 2019

aakashbajaj commented May 24, 2019

googlebot commented May 24, 2019

aakashbajaj commented May 24, 2019

Ark-kun commented Jul 24, 2019

aakashbajaj commented Jul 26, 2019

Ark-kun commented Aug 1, 2019

googlebot commented Aug 2, 2019

googlebot commented Aug 2, 2019

aakashbajaj commented Aug 2, 2019

Ark-kun commented Aug 3, 2019

k8s-ci-robot commented Aug 3, 2019

aakashbajaj commented Aug 6, 2019

Ark-kun commented Aug 12, 2019

googlebot commented Aug 16, 2019

aakashbajaj commented Aug 16, 2019

aakashbajaj commented Aug 17, 2019

aakashbajaj commented Aug 17, 2019

Ark-kun commented Aug 20, 2019

Ark-kun commented Aug 20, 2019

aakashbajaj commented Aug 20, 2019

Ark-kun commented Aug 20, 2019

aakashbajaj commented May 24, 2019 •

edited by jlewi

Loading