Squashed commit of the following:

commit 2592307 Author: Christian Clauss <cclauss@me.com> Date: Wed Aug 21 04:15:19 2019 +0200 Undefined name 'e' in openvino (kubeflow#1876) Discovered in kubeflow#1721 ``` ./contrib/components/openvino/ovms-deployer/containers/evaluate.py:62:16: F821 undefined name 'e' except e: ^ ./contrib/components/openvino/ovms-deployer/containers/evaluate.py:63:50: F821 undefined name 'e' print("Can not read the image file", e) ^ ``` Your review please @Ark-kun commit 7b442f4 Author: Kirin Patel <kirinpatel@gmail.com> Date: Tue Aug 20 18:27:19 2019 -0700 Created extensible code editor based on react-ace (kubeflow#1855) * Created extensible code editor based on react-ace * Installed dependencies * Updated unit tests for Editor.tsx to test placeholder and value in simplified manner * Updated Editor unit tests to use snapshot testing where applicable commit d11fae7 Author: Yuan (Bob) Gong <gongyuan94@gmail.com> Date: Wed Aug 21 08:25:20 2019 +0800 Use KFP lite deployment for presubmit tests (kubeflow#1808) * Refactor presubmit-tests-with-pipeline-deployment.sh so that it can be run from a different project * Simplify getting service account from cluster. * Migrate presubmit-tests-with-pipeline-deployment.sh to use kfp lightweight deployment. * Add option to cache built images to make debugging faster. * Fix cluster set up * Copy image builder image instead of granting permission * Add missed yes command * fix stuff * Let other usages of image-builder image become configurable * let test workflow use image builder image * Fix permission issue * Hide irrelevant error logs * Use shared service account key instead * Move test manifest to test folder * Move build-images.sh to a different script file * Update README.md * add cluster info dump * Use the same cluster resources as kubeflow deployment * Remove cluster info dump * Add timing to test log * cleaned up code * fix tests * address cr comments * Address cr comments * Enable image caching to improve retest speed commit 0864faf Author: IronPan <yangpa@google.com> Date: Tue Aug 20 14:09:19 2019 -0700 Use single part as default (kubeflow#1893) The data stored in artifact storage are usually small. Using multi-part is not strictly a requirement. Change the default to true to better support more platform out of box. commit 6284dc1 Author: Christian Clauss <cclauss@me.com> Date: Tue Aug 20 22:01:18 2019 +0200 IBM Watson samples: from six.moves import xrange (kubeflow#1877) Discovered in kubeflow#1721 __xrange()__ was removed in Python 3 in favor of an improved version of __range()__. This PR ensures equivalent functionality in both Python 2 and Python 3. ``` ./samples/contrib/ibm-samples/watson/source/model-source-code/tf-model/input_data.py:100:40: F821 undefined name 'xrange' fake_image = [1.0 for _ in xrange(784)] ^ ./samples/contrib/ibm-samples/watson/source/model-source-code/tf-model/input_data.py:102:41: F821 undefined name 'xrange' return [fake_image for _ in xrange(batch_size)], [ ^ ./samples/contrib/ibm-samples/watson/source/model-source-code/tf-model/input_data.py:103:37: F821 undefined name 'xrange' fake_label for _ in xrange(batch_size)] ^ ``` @gaoning777 @Ark-kun Your reviews please. commit 79c7bda Author: Ning <ngao@google.com> Date: Tue Aug 20 09:24:56 2019 -0700 fix unit tests and address some comments (kubeflow#1892) commit 6a7b28f Author: Aakash Bajaj <aakashbajaj5589@gmail.com> Date: Tue Aug 20 14:07:33 2019 +0530 gcp cred bug fix for multiple credentials in single pipeline (kubeflow#1384) * gcp cred bug fix for multiple credentials in single pipeline * squash to remove conflict commit 60fd70c Author: Andy Wei <jwwandy@gmail.com> Date: Tue Aug 20 15:33:33 2019 +0800 Let backend apiserver mysql dbname configurable (kubeflow#1714) * let mysql dbname configurable * solve conflict * move declaration out to fix scope commit 101a346 Author: olegchorny <oleg.chorny@gmail.com> Date: Tue Aug 20 08:33:47 2019 +0300 'core' folder included to parameters related On-Premise cluster (kubeflow#1751) * 'core' folder included to parameters related On-Premise cluster Update is required because this sample was migrated to the samples/core folder * Update README.md commit 4c5d34f Author: dushyanthsc <43390008+dushyanthsc@users.noreply.github.com> Date: Mon Aug 19 18:57:33 2019 -0700 test/project-cleanup: Support to cleanup gke-clusters in test project (kubeflow#1857) Change to add base framework for cleaning up resources in a GCP project. The resource specification is specified declaratively using a YAML file. As per current requirements this change only adds cleaning up of GKE clusters.
numerology · Aug 21, 2019 · 5e7a8c5 · 5e7a8c5
1 parent 748e9d9
commit 5e7a8c5
Show file tree

Hide file tree

Showing 27 changed files with 807 additions and 94 deletions.
diff --git a/backend/src/apiserver/client_manager.go b/backend/src/apiserver/client_manager.go
@@ -41,15 +41,15 @@ import (
 )
 
 const (
-	minioServiceHost = "MINIO_SERVICE_SERVICE_HOST"
-	minioServicePort = "MINIO_SERVICE_SERVICE_PORT"
-	mysqlServiceHost = "MYSQL_SERVICE_HOST"
-	mysqlUser        = "DBConfig.User"
-	mysqlPassword    = "DBConfig.Password"
-	mysqlServicePort = "MYSQL_SERVICE_PORT"
+	minioServiceHost      = "MINIO_SERVICE_SERVICE_HOST"
+	minioServicePort      = "MINIO_SERVICE_SERVICE_PORT"
+	mysqlServiceHost      = "MYSQL_SERVICE_HOST"
+	mysqlServicePort      = "MYSQL_SERVICE_PORT"
+	mysqlUser             = "DBConfig.User"
+	mysqlPassword         = "DBConfig.Password"
+	mysqlDBName           = "DBConfig.DBName"
 
 	podNamespace          = "POD_NAMESPACE"
-	dbName                = "mlpipeline"
 	initConnectionTimeout = "InitConnectionTimeout"
 )
 
@@ -255,6 +255,7 @@ func initMysql(driverName string, initConnectionTimeout time.Duration) string {
 	util.TerminateIfError(err)
 
 	// Create database if not exist
+	dbName := getStringConfig(mysqlDBName)
 	operation = func() error {
 		_, err = db.Exec(fmt.Sprintf("CREATE DATABASE IF NOT EXISTS %s", dbName))
 		if err != nil {
@@ -280,7 +281,7 @@ func initMinioClient(initConnectionTimeout time.Duration) storage.ObjectStoreInt
 	accessKey := getStringConfig("ObjectStoreConfig.AccessKey")
 	secretKey := getStringConfig("ObjectStoreConfig.SecretAccessKey")
 	bucketName := getStringConfig("ObjectStoreConfig.BucketName")
-	disableMultipart := getBoolConfigWithDefault("ObjectStoreConfig.Multipart.Disable", false)
+	disableMultipart := getBoolConfigWithDefault("ObjectStoreConfig.Multipart.Disable", true)
 
 	minioClient := client.CreateMinioClientOrFatal(minioServiceHost, minioServicePort, accessKey,
 		secretKey, initConnectionTimeout)

diff --git a/backend/src/apiserver/config/config.json b/backend/src/apiserver/config/config.json
@@ -1,7 +1,8 @@
 {
   "DBConfig": {
     "DriverName": "mysql",
-    "DataSourceName": ""
+    "DataSourceName": "",
+    "DBName": "mlpipeline"
   },
   "ObjectStoreConfig":{
     "AccessKey": "minio",

diff --git a/contrib/components/openvino/ovms-deployer/containers/evaluate.py b/contrib/components/openvino/ovms-deployer/containers/evaluate.py
@@ -59,7 +59,7 @@ def getJpeg(path, size, path_prefix):
             img = img.astype('float32')
             img = img.transpose(2,0,1).reshape(1,3,size,size)
             print(path, img.shape, "; data range:",np.amin(img),":",np.amax(img))
-        except e:
+        except Exception as e:
             print("Can not read the image file", e)
             img = None
     else:
@@ -138,4 +138,4 @@ def getJpeg(path, size, path_prefix):
     json.dump(metrics, f)
 f.close
 print("\nOverall accuracy=",matched/i*100,"%")
-print("Average latency=",latency,"ms")
+print("Average latency=",latency,"ms")
diff --git a/frontend/package-lock.json b/frontend/package-lock.json
diff --git a/frontend/package.json b/frontend/package.json
@@ -6,6 +6,7 @@
     "@material-ui/core": "3.7.1",
     "@material-ui/icons": "^3.0.1",
     "@types/js-yaml": "^3.11.2",
+    "brace": "^0.11.1",
     "codemirror": "^5.40.2",
     "d3": "^5.7.0",
     "d3-dsv": "^1.0.10",
@@ -18,6 +19,7 @@
     "portable-fetch": "^3.0.0",
     "re-resizable": "^4.9.0",
     "react": "^16.7.0",
+    "react-ace": "^7.0.2",
     "react-codemirror2": "^5.1.0",
     "react-dom": "^16.5.2",
     "react-dropzone": "^5.1.0",

diff --git a/frontend/src/components/Editor.test.tsx b/frontend/src/components/Editor.test.tsx
@@ -0,0 +1,52 @@
+/*
+ * Copyright 2019 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import * as React from 'react';
+import { mount } from 'enzyme';
+import Editor from './Editor';
+
+/*
+  These tests mimic https://github.com/securingsincity/react-ace/blob/master/tests/src/ace.spec.js
+  to ensure that editor properties (placeholder and value) can be properly
+  tested.
+*/
+
+describe('Editor', () => {
+  it('renders without a placeholder and value', () => {
+    const tree = mount(<Editor />);
+    expect(tree.html()).toMatchSnapshot();
+  });
+
+  it('renders with a placeholder', () => {
+    const placeholder = 'I am a placeholder.';
+    const tree = mount(<Editor placeholder={placeholder} />);
+    expect(tree.html()).toMatchSnapshot();
+  });
+
+  it ('renders a placeholder that contains HTML', () => {
+    const placeholder = 'I am a placeholder with <strong>HTML</strong>.';
+    const tree = mount(<Editor placeholder={placeholder} />);
+    expect(tree.html()).toMatchSnapshot();
+  });
+
+  it('has its value set to the provided value', () => {
+    const value = 'I am a value.';
+    const tree = mount(<Editor value={value} />);
+    expect(tree).not.toBeNull();
+    const editor = (tree.instance() as any).editor;
+    expect(editor.getValue()).toBe(value);
+  });
+});
diff --git a/frontend/src/components/Editor.tsx b/frontend/src/components/Editor.tsx
@@ -0,0 +1,45 @@
+/*
+ * Copyright 2019 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import AceEditor from 'react-ace';
+
+// Modified AceEditor that supports HTML within provided placeholder. This is
+// important because it allows for the usage of multi-line placeholders.
+class Editor extends AceEditor {
+  public updatePlaceholder(): void {
+    const editor = this.editor;
+    const { placeholder } = this.props;
+
+    const showPlaceholder = !editor.session.getValue().length;
+    let node = editor.renderer.placeholderNode;
+    if (!showPlaceholder && node) {
+      editor.renderer.scroller.removeChild(editor.renderer.placeholderNode);
+      editor.renderer.placeholderNode = null;
+    } else if (showPlaceholder && !node) {
+      node = editor.renderer.placeholderNode = document.createElement('div');
+      node.innerHTML = placeholder || '';
+      node.className = 'ace_comment ace_placeholder';
+      node.style.padding = '0 9px';
+      node.style.position = 'absolute';
+      node.style.zIndex = '3';
+      editor.renderer.scroller.appendChild(node);
+    } else if (showPlaceholder && node) {
+      node.innerHTML = placeholder;
+    }
+  }
+}
+
+export default Editor;
diff --git a/frontend/src/components/__snapshots__/Editor.test.tsx.snap b/frontend/src/components/__snapshots__/Editor.test.tsx.snap
@@ -0,0 +1,7 @@
+// Jest Snapshot v1, https://goo.gl/fbAQLP
+
+exports[`Editor renders a placeholder that contains HTML 1`] = `"<div id=\\"brace-editor\\" style=\\"width: 500px; height: 500px;\\" class=\\" ace_editor ace-tm\\"><textarea class=\\"ace_text-input\\" wrap=\\"off\\" autocorrect=\\"off\\" autocapitalize=\\"off\\" spellcheck=\\"false\\" style=\\"opacity: 0;\\"></textarea><div class=\\"ace_gutter\\" aria-hidden=\\"true\\"><div class=\\"ace_layer ace_gutter-layer ace_folding-enabled\\"></div><div class=\\"ace_gutter-active-line\\"></div></div><div class=\\"ace_scroller\\"><div class=\\"ace_content\\"><div class=\\"ace_layer ace_print-margin-layer\\"><div class=\\"ace_print-margin\\" style=\\"left: 4px; visibility: visible;\\"></div></div><div class=\\"ace_layer ace_marker-layer\\"></div><div class=\\"ace_layer ace_text-layer\\" style=\\"padding: 0px 4px;\\"></div><div class=\\"ace_layer ace_marker-layer\\"></div><div class=\\"ace_layer ace_cursor-layer ace_hidden-cursors\\"><div class=\\"ace_cursor\\"></div></div></div><div class=\\"ace_comment ace_placeholder\\" style=\\"padding: 0px 9px; position: absolute; z-index: 3;\\">I am a placeholder with <strong>HTML</strong>.</div></div><div class=\\"ace_scrollbar ace_scrollbar-v\\" style=\\"display: none; width: 20px;\\"><div class=\\"ace_scrollbar-inner\\" style=\\"width: 20px;\\"></div></div><div class=\\"ace_scrollbar ace_scrollbar-h\\" style=\\"display: none; height: 20px;\\"><div class=\\"ace_scrollbar-inner\\" style=\\"height: 20px;\\"></div></div><div style=\\"height: auto; width: auto; top: 0px; left: 0px; visibility: hidden; position: absolute; white-space: pre; overflow: hidden;\\"><div style=\\"height: auto; width: auto; top: 0px; left: 0px; visibility: hidden; position: absolute; white-space: pre; overflow: visible;\\"></div><div style=\\"height: auto; width: auto; top: 0px; left: 0px; visibility: hidden; position: absolute; white-space: pre; overflow: visible;\\">XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX</div></div></div>"`;
+
+exports[`Editor renders with a placeholder 1`] = `"<div id=\\"brace-editor\\" style=\\"width: 500px; height: 500px;\\" class=\\" ace_editor ace-tm\\"><textarea class=\\"ace_text-input\\" wrap=\\"off\\" autocorrect=\\"off\\" autocapitalize=\\"off\\" spellcheck=\\"false\\" style=\\"opacity: 0;\\"></textarea><div class=\\"ace_gutter\\" aria-hidden=\\"true\\"><div class=\\"ace_layer ace_gutter-layer ace_folding-enabled\\"></div><div class=\\"ace_gutter-active-line\\"></div></div><div class=\\"ace_scroller\\"><div class=\\"ace_content\\"><div class=\\"ace_layer ace_print-margin-layer\\"><div class=\\"ace_print-margin\\" style=\\"left: 4px; visibility: visible;\\"></div></div><div class=\\"ace_layer ace_marker-layer\\"></div><div class=\\"ace_layer ace_text-layer\\" style=\\"padding: 0px 4px;\\"></div><div class=\\"ace_layer ace_marker-layer\\"></div><div class=\\"ace_layer ace_cursor-layer ace_hidden-cursors\\"><div class=\\"ace_cursor\\"></div></div></div><div class=\\"ace_comment ace_placeholder\\" style=\\"padding: 0px 9px; position: absolute; z-index: 3;\\">I am a placeholder.</div></div><div class=\\"ace_scrollbar ace_scrollbar-v\\" style=\\"display: none; width: 20px;\\"><div class=\\"ace_scrollbar-inner\\" style=\\"width: 20px;\\"></div></div><div class=\\"ace_scrollbar ace_scrollbar-h\\" style=\\"display: none; height: 20px;\\"><div class=\\"ace_scrollbar-inner\\" style=\\"height: 20px;\\"></div></div><div style=\\"height: auto; width: auto; top: 0px; left: 0px; visibility: hidden; position: absolute; white-space: pre; overflow: hidden;\\"><div style=\\"height: auto; width: auto; top: 0px; left: 0px; visibility: hidden; position: absolute; white-space: pre; overflow: visible;\\"></div><div style=\\"height: auto; width: auto; top: 0px; left: 0px; visibility: hidden; position: absolute; white-space: pre; overflow: visible;\\">XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX</div></div></div>"`;
+
+exports[`Editor renders without a placeholder and value 1`] = `"<div id=\\"brace-editor\\" style=\\"width: 500px; height: 500px;\\" class=\\" ace_editor ace-tm\\"><textarea class=\\"ace_text-input\\" wrap=\\"off\\" autocorrect=\\"off\\" autocapitalize=\\"off\\" spellcheck=\\"false\\" style=\\"opacity: 0;\\"></textarea><div class=\\"ace_gutter\\" aria-hidden=\\"true\\"><div class=\\"ace_layer ace_gutter-layer ace_folding-enabled\\"></div><div class=\\"ace_gutter-active-line\\"></div></div><div class=\\"ace_scroller\\"><div class=\\"ace_content\\"><div class=\\"ace_layer ace_print-margin-layer\\"><div class=\\"ace_print-margin\\" style=\\"left: 4px; visibility: visible;\\"></div></div><div class=\\"ace_layer ace_marker-layer\\"></div><div class=\\"ace_layer ace_text-layer\\" style=\\"padding: 0px 4px;\\"></div><div class=\\"ace_layer ace_marker-layer\\"></div><div class=\\"ace_layer ace_cursor-layer ace_hidden-cursors\\"><div class=\\"ace_cursor\\"></div></div></div></div><div class=\\"ace_scrollbar ace_scrollbar-v\\" style=\\"display: none; width: 20px;\\"><div class=\\"ace_scrollbar-inner\\" style=\\"width: 20px;\\"></div></div><div class=\\"ace_scrollbar ace_scrollbar-h\\" style=\\"display: none; height: 20px;\\"><div class=\\"ace_scrollbar-inner\\" style=\\"height: 20px;\\"></div></div><div style=\\"height: auto; width: auto; top: 0px; left: 0px; visibility: hidden; position: absolute; white-space: pre; overflow: hidden;\\"><div style=\\"height: auto; width: auto; top: 0px; left: 0px; visibility: hidden; position: absolute; white-space: pre; overflow: visible;\\"></div><div style=\\"height: auto; width: auto; top: 0px; left: 0px; visibility: hidden; position: absolute; white-space: pre; overflow: visible;\\">XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX</div></div></div>"`;