Cannot create job for experiment via Pipelines Go CLI

[berndverst]

I have an existing pipeline and created a new experiment via the Go CLI. Now I wish to run a new job based on this pipeline and CLI. This does not seem possible as there is no way to specify the required experiment ID in the Go CLI (https://github.com/kubeflow/pipelines/tree/master/backend/src/cmd/ml).

ml job --kubeconfig ~/.kube/config --namespace kubeflow create --pipeline-id af7aedfb-4fff-41bf-b697-78cee95fae5c --debug

I get the following error:

{"error":"The job must have a valid experiment resource reference.: Invalid input error: The resource reference is empty. Please specify which experiment owns this resource.","message":"The job must have a valid experiment resource reference.: Invalid input error: The resource reference is empty. Please specify which experiment owns this resource.","code":3,"details":[{"@type":"type.googleapis.com/api.Error","error_message":"The resource reference is empty. Please specify which experiment owns this resource.","error_details":"The job must have a valid experiment resource reference.: Invalid input error: The resource reference is empty. Please specify which experiment owns this resource."}]}

Here is the verbose request by the way:

POST /api/v1/namespaces/kubeflow/services/ml-pipeline:8888/proxy/apis/v1beta1/jobs HTTP/1.1
Host: SOMESERVERNAMEHERE.bernd.dev:443
User-Agent: Go-http-client/1.1
Content-Length: 290
Accept: application/json
Content-Type: application/json
Accept-Encoding: gzip

{"created_at":"0001-01-01T00:00:00.000Z","description":"No description provided","enabled":true,"max_concurrency":"1","name":"default","pipeline_spec":{"parameters":[],"pipeline_id":"af7aedfb-4fff-41bf-b697-78cee95fae5c"},"resource_references":null,"updated_at":"0001-01-01T00:00:00.000Z"}

There does not seem to be a way to pass in the experiment ID. Even all tests I could find set resource_references to null (which is happening in my request above). https://github.com/kubeflow/pipelines/blob/master/backend/src/cmd/ml/cmd/run_test.go

However, from intercepting the Kubeflow Dashboard I can see that I'd want something like:
[{key: {id: "5f4d75ab-c3cc-4fbf-80a2-130c95af3284", type: "EXPERIMENT"}, relationship: "OWNER"}]

There are no raw data, resource-references or experiment-id flags anywhere.

How should I specify the experiment ID?

cc @vicaire

[vicaire]

The API has been changed quite a bit and the CLI has not been updated to match the API.

Some work is needed and tracked here:

#812

Contributions are welcome.

[aronchick]

Hi @vicaire Can you help us what we're supposed to do? We need to programatically trigger a pipeline and there's no guidance for how to handle auth. Can you help us figure this out?