Error message: Invalid resource references for experiment. ListExperiment requires filtering by namespace kubeflow

[ShivamGarg2910]

I have deployed Kubeflow 1.2 on on-prem cluster.
Used this docker image for getting access to Kale: gcr.io/arrikto/jupyter-kale:v0.5.0
On creating a notebook, getting this error:

Message: (400)
Reason: Bad Request
HTTP response headers: HTTPHeaderDict({'content-type': 'application/json', 'trailer': 'Grpc-Trailer-Content-Type', 'date': 'Wed, 10 Feb 2021 06:36:05 GMT', 'x-envoy-upstream-service-time': '8', 'server': 'envoy', 'transfer-encoding': 'chunked'})
HTTP response body: {"error":"Invalid input error: Invalid resource references for experiment. ListExperiment requires filtering by namespace.","message":"Invalid input error: Invalid resource references for experiment. ListExperiment requires filtering by namespace.","code":3,"details":[{"@type":"type.googleapis.com/api.Error","error_message":"Invalid resource references for experiment. ListExperiment requires filtering by namespace.","error_details":"Invalid input error: Invalid resource references for experiment. ListExperiment requires filtering by namespace."}]}

No idea on how to proceed.
Any help is appreciated.

Thanks,
Shivam

[elikatsis]

Hello @ShivamGarg2910,

from the error message I understand that you have deployed multi-user KFP. However, the KFP backend doesn't currently support authentication using the KFP client, nor the client has any functionality for that yet.

This is something we will be pushing to KFP (kubeflow/pipelines#5138).

In the latest MiniKF (AWS/GCP), we have implemented this feature of the KFP backend. Also, the jupyter-kale image that comes with that MiniKF has a variation of the KFP client to properly authenticate with this backend.