input parameter for creating pipelines - does not allow camelCase

[rayymlai]

When creating a Kubeflow pipeline, the input parameter cannot allow camel case.

Example:

Accepted: params = {'mystate': 'abc.csv', 'myfile': 'gs://smartpipe/pipestate01.txt'}
Error: params = {'myState': 'abc.csv', 'myFile': 'gs://smartpipe/pipestate01.txt'}

import kfp
client = kfp.Client()
experiment = client.create_experiment('create ML model training pipeline03')
run_name = 'Extract-Train process04'
params = {'myState': 'abc.csv', 'myFile': 'gs://smartpipe/pipestate01.txt'}
client.run_pipeline(experiment.id, run_name, 'smartMLTrainingPipeline.tar.gz', params)

is lack of support of camel case a restriction (e.g. argo restriction) or a software bug?

h1. Error message details

---------------------------------------------------------------------------
ApiException                              Traceback (most recent call last)
<ipython-input-15-d81ac76ab677> in <module>
      2 import kfp
      3 client = kfp.Client()
----> 4 experiment = client.create_experiment('create ML model training pipeline03')
      5 run_name = 'Extract-Train process04'
      6 params = {'myState': 'abc.csv', 'myFile': 'gs://smartpipe/pipestate01.txt'}

/opt/conda/lib/python3.6/site-packages/kfp/_client.py in create_experiment(self, name)
     75 
     76     exp = kfp_experiment.models.ApiExperiment(name=name)
---> 77     response = self._experiment_api.create_experiment(body=exp)
     78 
     79     if self._is_ipython():

/opt/conda/lib/python3.6/site-packages/kfp_experiment/api/experiment_service_api.py in create_experiment(self, body, **kwargs)
     52             return self.create_experiment_with_http_info(body, **kwargs)  # noqa: E501
     53         else:
---> 54             (data) = self.create_experiment_with_http_info(body, **kwargs)  # noqa: E501
     55             return data
     56 

/opt/conda/lib/python3.6/site-packages/kfp_experiment/api/experiment_service_api.py in create_experiment_with_http_info(self, body, **kwargs)
    129             _preload_content=params.get('_preload_content', True),
    130             _request_timeout=params.get('_request_timeout'),
--> 131             collection_formats=collection_formats)
    132 
    133     def get_experiment(self, id, **kwargs):  # noqa: E501

/opt/conda/lib/python3.6/site-packages/kfp_experiment/api_client.py in call_api(self, resource_path, method, path_params, query_params, header_params, body, post_params, files, response_type, auth_settings, async, _return_http_data_only, collection_formats, _preload_content, _request_timeout)
    320                                    response_type, auth_settings,
    321                                    _return_http_data_only, collection_formats,
--> 322                                    _preload_content, _request_timeout)
    323         else:
    324             thread = self.pool.apply_async(self.__call_api, (resource_path,

/opt/conda/lib/python3.6/site-packages/kfp_experiment/api_client.py in __call_api(self, resource_path, method, path_params, query_params, header_params, body, post_params, files, response_type, auth_settings, _return_http_data_only, collection_formats, _preload_content, _request_timeout)
    151             post_params=post_params, body=body,
    152             _preload_content=_preload_content,
--> 153             _request_timeout=_request_timeout)
    154 
    155         self.last_response = response_data

/opt/conda/lib/python3.6/site-packages/kfp_experiment/api_client.py in request(self, method, url, query_params, headers, post_params, body, _preload_content, _request_timeout)
    363                                          _preload_content=_preload_content,
    364                                          _request_timeout=_request_timeout,
--> 365                                          body=body)
    366         elif method == "PUT":
    367             return self.rest_client.PUT(url,

/opt/conda/lib/python3.6/site-packages/kfp_experiment/rest.py in POST(self, url, headers, query_params, post_params, body, _preload_content, _request_timeout)
    273                             _preload_content=_preload_content,
    274                             _request_timeout=_request_timeout,
--> 275                             body=body)
    276 
    277     def PUT(self, url, headers=None, query_params=None, post_params=None,

/opt/conda/lib/python3.6/site-packages/kfp_experiment/rest.py in request(self, method, url, query_params, headers, body, post_params, _preload_content, _request_timeout)
    226 
    227         if not 200 <= r.status <= 299:
--> 228             raise ApiException(http_resp=r)
    229 
    230         return r

ApiException: (400)
Reason: Bad Request
HTTP response headers: HTTPHeaderDict({'Content-Type': 'application/json', 'Trailer': 'Grpc-Trailer-Content-Type', 'Date': 'Mon, 03 Dec 2018 21:21:15 GMT', 'Transfer-Encoding': 'chunked'})
HTTP response body: {"error":"Create experiment failed.: Invalid input error: Failed to create a new experiment. The name create ML model training pipeline03 already exist. Please specify a new name.","code":3,"details":[{"@type":"type.googleapis.com/api.Error","error_message":"Failed to create a new experiment. The name create ML model training pipeline03 already exist. Please specify a new name.","error_details":"Create experiment failed.: Invalid input error: Failed to create a new experiment. The name create ML model training pipeline03 already exist. Please specify a new name."}]}

[yebrahim]

Seems you already have an experiment with this name. Experiment names have to be unique. From the error message: The name create ML model training pipeline03 already exist.

[rayymlai]

i did use a new name but it still throws a different error. once I removed the camel style in the input parameters, the problem goes away. I have verified with Sina Chavoshi from Google team during my EAP for Kubeflow Pipelines.

[yebrahim]

I see. Can you please include the other error here to help us debug the issue? I'll re-open it for now, but please update the description since the error mentioned here is just a name conflict.

[gaoning777]

The problem results from the DSL parameter sanitation while the python SDK does not.

[rayymlai]

this is reasonable diagnosis. is there any solution or fix? thanks

[gaoning777]

solved by this PR: #466

[gaoning777]

The next release will contain this fix. However, if you want, you could build the sdk yourself:
cd $PIPELINE/sdk/python
wget http://central.maven.org/maven2/io/swagger/swagger-codegen-cli/2.3.1/swagger-codegen-cli-2.3.1.jar -O /tmp/swagger-codegen-cli.jar
apt-get install --no-install-recommends -y -q default-jdk
./build.sh /tmp/kfp.tar.gz
pip3 install /tmp/kfp.tar.gz

the swagger and jdk are dependent softwares to build the sdk.

[gaoning777]

/close

[k8s-ci-robot]

@gaoning777: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

@gaoning777: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.