Sample tests build kfp_server_api SDK

[IronPan]

As of today, updating the KFP API interface won't trigger a rebuild of kfp_server_api for tests to use.

For example this PR #1781 add a new field in resource_reference.proto. Here is the error message I saw

[33mrun-sample-tests-loop(0:kubeflow_training_classification)�[0m:	HTTP response body: {"error":"InternalServerError: Failed to get run: InternalServerError: Failed to parse resource reference.: failed unmarshal resource references '[{\"ResourceUUID\":\"788dedca-6ec2-4e45-9f6e-2a5182726121\",\"ResourceType\":\"Run\",\"ReferenceUUID\":\"9cebcad4-8a7e-42c1-a725-c5240ad87d43\",\"ReferenceName\":\"kubeflow_training_classification sample experiment\",\"ReferenceType\":\"Experiment\",\"Relationship\":\"Owner\",\"Pa]'. error: unexpected end of JSON input: InternalServerError: Failed to parse resource reference.: failed unmarshal resource references '[{\"ResourceUUID\":\"788dedca-6ec2-4e45-9f6e-2a5182726121\",\"ResourceType\":\"Run\",\"ReferenceUUID\":\"9cebcad4-8a7e-42c1-a725-c5240ad87d43\",\"ReferenceName\":\"kubeflow_training_classification sample experiment\",\"ReferenceType\":\"Experiment\",\"Relationship\":\"Owner\",\"Pa]'. error: unexpected end of JSON input","message":"InternalServerError: Failed to get run: InternalServerError: Failed to parse resource reference.: failed unmarshal resource references '[{\"ResourceUUID\":\"788dedca-6ec2-4e45-9f6e-2a5182726121\",\"ResourceType\":\"Run\",\"ReferenceUUID\":\"9cebcad4-8a7e-42c1-a725-c5240ad87d43\",\"ReferenceName\":\"kubeflow_training_classification sample experiment\",\"ReferenceType\":\"Experiment\",\"Relationship\":\"Owner\",\"Pa]'. error: unexpected end of JSON input: InternalServerError: Failed to parse resource reference.: failed unmarshal resource references '[{\"ResourceUUID\":\"788dedca-6ec2-4e45-9f6e-2a5182726121\",\"ResourceType\":\"Run\",\"ReferenceUUID\":\"9cebcad4-8a7e-42c1-a725-c5240ad87d43\",\"ReferenceName\":\"kubeflow_training_classification sample experiment\",\"ReferenceType\":\"Experiment\",\"Relationship\":\"Owner\",\"Pa]'. error: unexpected end of JSON input","code":13,"details":[{"@type":"type.googleapis.com/api.Error","error_message":"Internal Server Error","error_details":"InternalServerError: Failed to get run: InternalServerError: Failed to parse resource reference.: failed unmarshal resource references '[{\"ResourceUUID\":\"788dedca-6ec2-4e45-9f6e-2a5182726121\",\"ResourceType\":\"Run\",\"ReferenceUUID\":\"9cebcad4-8a7e-42c1-a725-c5240ad87d43\",\"ReferenceName\":\"kubeflow_training_classification sample experiment\",\"ReferenceType\":\"Experiment\",\"Relationship\":\"Owner\",\"Pa]'. error: unexpected end of JSON input: InternalServerError: Failed to parse resource reference.: failed unmarshal resource references '[{\"ResourceUUID\":\"788dedca-6ec2-4e45-9f6e-2a5182726121\",\"ResourceType\":\"Run\",\"ReferenceUUID\":\"9cebcad4-8a7e-42c1-a725-c5240ad87d43\",\"ReferenceName\":\"kubeflow_training_classification sample experiment\",\"ReferenceType\":\"Experiment\",\"Relationship\":\"Owner\",\"Pa]'. error: unexpected end of JSON input"}]}
�[33mrun-sample-tests-loop(0:kubeflow_training_classification)�[0m:	During handling of the above exception, another exception occurred:
�[33mrun-sample-tests-loop(0:kubeflow_training_classification)�[0m:	Traceback (most recent call last):
�[33mrun-sample-tests-loop(0:kubeflow_training_classification)�[0m:	  File "run_sample_test.py", line 190, in <module>
�[33mrun-sample-tests-loop(0:kubeflow_training_classification)�[0m:	    main()
�[33mrun-sample-tests-loop(0:kubeflow_training_classification)�[0m:	  File "run_sample_test.py", line 140, in main
�[33mrun-sample-tests-loop(0:kubeflow_training_classification)�[0m:	    workflow_json = client._get_workflow_json(run_id)
�[33mrun-sample-tests-loop(0:kubeflow_training_classification)�[0m:	  File "/usr/local/lib/python3.5/dist-packages/kfp-0.1.25-py3.5.egg/kfp/_client.py", line 413, in _get_workflow_json
�[33mrun-sample-tests-loop(0:kubeflow_training_classification)�[0m:	    get_run_response = self._run_api.get_run(run_id=run_id)
�[33mrun-sample-tests-loop(0:kubeflow_training_classification)�[0m:	  File "/usr/local/lib/python3.5/dist-packages/kfp_server_api-0.1.18.3-py3.5.egg/kfp_server_api/api/run_service_api.py", line 345, in get_run
�[33mrun-sample-tests-loop(0:kubeflow_training_classification)�[0m:	  File "/usr/local/lib/python3.5/dist-packages/kfp_server_api-0.1.18.3-py3.5.egg/kfp_server_api/api/run_service_api.py", line 422, in get_run_with_http_info
�[33mrun-sample-tests-loop(0:kubeflow_training_classification)�[0m:	  File "/usr/local/lib/python3.5/dist-packages/kfp_server_api-0.1.18.3-py3.5.egg/kfp_server_api/api_client.py", line 330, in call_api
�[33mrun-sample-tests-loop(0:kubeflow_training_classification)�[0m:	  File "/usr/local/lib/python3.5/dist-packages/kfp_server_api-0.1.18.3-py3.5.egg/kfp_server_api/api_client.py", line 161, in __call_api
�[33mrun-sample-tests-loop(0:kubeflow_training_classification)�[0m:	  File "/usr/local/lib/python3.5/dist-packages/kfp_server_api-0.1.18.3-py3.5.egg/kfp_server_api/api_client.py", line 351, in request
�[33mrun-sample-tests-loop(0:kubeflow_training_classification)�[0m:	  File "/usr/local/lib/python3.5/dist-packages/kfp_server_api-0.1.18.3-py3.5.egg/kfp_server_api/rest.py", line 238, in GET
�[33mrun-sample-tests-loop(0:kubeflow_training_classification)�[0m:	  File "/usr/local/lib/python3.5/dist-packages/kfp_server_api-0.1.18.3-py3.5.egg/kfp_server_api/rest.py", line 228, in request
�[33mrun-sample-tests-loop(0:kubeflow_training_classification)�[0m:	kfp_server_api.rest.ApiException: (500)
�[33mrun-sample-tests-loop(0:kubeflow_training_classification)�[0m:	Reason: Internal Server Error
�[33mrun-sample-tests-loop(0:kubeflow_training_classification)�[0m:	HTTP response headers: HTTPHeaderDict({'Transfer-Encoding': 'chunked', 'Trailer': 'Grpc-Trailer-Content-Type', 'Content-Type': 'application/json', 'Date': 'Sat, 10 Aug 2019 19:04:20 GMT'})
�[33mrun-sample-tests-loop(0:kubeflow_training_classification)�[0m:	HTTP response body: {"error":"InternalServerError: Failed to get run: InternalServerError: Failed to parse resource reference.: failed unmarshal resource references '[{\"ResourceUUID\":\"788dedca-6ec2-4e45-9f6e-2a5182726121\",\"ResourceType\":\"Run\",\"ReferenceUUID\":\"9cebcad4-8a7e-42c1-a725-c5240ad87d43\",\"ReferenceName\":\"kubeflow_training_classification sample experiment\",\"ReferenceType\":\"Experiment\",\"Relationship\":\"Owner\",\"Pa]'. error: unexpected end of JSON input: InternalServerError: Failed to parse resource reference.: failed unmarshal resource references '[{\"ResourceUUID\":\"788dedca-6ec2-4e45-9f6e-2a5182726121\",\"ResourceType\":\"Run\",\"ReferenceUUID\":\"9cebcad4-8a7e-42c1-a725-c5240ad87d43\",\"ReferenceName\":\"kubeflow_training_classification sample experiment\",\"ReferenceType\":\"Experiment\",\"Relationship\":\"Owner\",\"Pa]'. error: unexpected end of JSON input","message":"InternalServerError: Failed to get run: InternalServerError: Failed to parse resource reference.: failed unmarshal resource references '[{\"ResourceUUID\":\"788dedca-6ec2-4e45-9f6e-2a5182726121\",\"ResourceType\":\"Run\",\"ReferenceUUID\":\"9cebcad4-8a7e-42c1-a725-c5240ad87d43\",\"ReferenceName\":\"kubeflow_training_classification sample experiment\",\"ReferenceType\":\"Experiment\",\"Relationship\":\"Owner\",\"Pa]'. error: unexpected end of JSON input: InternalServerError: Failed to parse resource reference.: failed unmarshal resource references '[{\"ResourceUUID\":\"788dedca-6ec2-4e45-9f6e-2a5182726121\",\"ResourceType\":\"Run\",\"ReferenceUUID\":\"9cebcad4-8a7e-42c1-a725-c5240ad87d43\",\"ReferenceName\":\"kubeflow_training_classification sample experiment\",\"ReferenceType\":\"Experiment\",\"Relationship\":\"Owner\",\"Pa]'. error: unexpected end of JSON input","code":13,"details":[{"@type":"type.googleapis.com/api.Error","error_message":"Internal Server Error","error_details":"InternalServerError: Failed to get run: InternalServerError: Failed to parse resource reference.: failed unmarshal resource references '[{\"ResourceUUID\":\"788dedca-6ec2-4e45-9f6e-2a5182726121\",\"ResourceType\":\"Run\",\"ReferenceUUID\":\"9cebcad4-8a7e-42c1-a725-c5240ad87d43\",\"ReferenceName\":\"kubeflow_training_classification sample experiment\",\"ReferenceType\":\"Experiment\",\"Relationship\":\"Owner\",\"Pa]'. error: unexpected end of JSON input: InternalServerError: Failed to parse resource reference.: failed unmarshal resource references '[{\"ResourceUUID\":\"788dedca-6ec2-4e45-9f6e-2a5182726121\",\"ResourceType\":\"Run\",\"ReferenceUUID\":\"9cebcad4-8a7e-42c1-a725-c5240ad87d43\",\"ReferenceName\":\"kubeflow_training_classification sample experiment\",\"ReferenceType\":\"Experiment\",\"Relationship\":\"Owner\",\"Pa]'. error: unexpected end of JSON input"}]}
�[37mrun-sample-tests-loop(5:dsl_static_type_checking)�[0m:	[NbConvertApp] Writing 17018 bytes to dsl_static_type_checking.py

To make the tests pass, I have to run backend/api/build_kfp_server_api_python_package.sh and publish a kfp_server_api SDK to public. In theory we shouldn't publish a new version before doing a release. The sample tests should build the private SDK and use it for kfp SDK.

[IronPan]

It turned out the issue was not the SDK but instead the database default size limit on string. The json is truncated and result in parsing failure.

[Ark-kun]

the database default size limit on string. The json is truncated and result in parsing failure.

Oops. We should probably issue an easy to debug error when the string size is bigger.