Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metadata Writer stopped getting the artifact types #3971

Closed
Ark-kun opened this issue Jun 11, 2020 · 2 comments · Fixed by #4231
Closed

Metadata Writer stopped getting the artifact types #3971

Ark-kun opened this issue Jun 11, 2020 · 2 comments · Fixed by #4231
Assignees
Labels
area/metadata-writer status/triaged Whether the issue has been explicitly triaged

Comments

@Ark-kun
Copy link
Contributor

Ark-kun commented Jun 11, 2020

All outputs are getting recorded as NoType.

annotations:
    pipelines.kubeflow.org/component_spec: >-
      {"name": "Create visualizations", "outputs": [{"name": "Some Url 1",
      "type": "URI"}, {"name": "Some Url 2", "type": {"URI": {"data_type":
      "something"}}}, {"name": "Some HTML", "type": "HTML"}, {"name": "Some
      Markdown", "type": "Markdown"}, {"name": "Some metrics", "type":
      "Metrics"}]}
    pipelines.kubeflow.org/metadata_input_artifact_ids: '[]'
    pipelines.kubeflow.org/metadata_output_artifact_ids: >-
      [{"id": 32, "name": "some_html", "uri":
      "minio://mlpipeline/artifacts/create-visualizations-2-7sxmn/create-visualizations-2-7sxmn-3549616354/create-visualizations-some_html.tgz",
      "type": "NoType"}, {"id": 33, "name": "some_markdown", "uri":
      "minio://mlpipeline/artifacts/create-visualizations-2-7sxmn/create-visualizations-2-7sxmn-3549616354/create-visualizations-some_markdown.tgz",
      "type": "NoType"}, {"id": 34, "name": "some_metrics", "uri":
      "minio://mlpipeline/artifacts/create-visualizations-2-7sxmn/create-visualizations-2-7sxmn-3549616354/create-visualizations-some_metrics.tgz",
      "type": "NoType"}, {"id": 35, "name": "some_url_1", "uri":
      "minio://mlpipeline/artifacts/create-visualizations-2-7sxmn/create-visualizations-2-7sxmn-3549616354/create-visualizations-some_url_1.tgz",
      "type": "NoType"}, {"id": 36, "name": "some_url_2", "uri":
      "minio://mlpipeline/artifacts/create-visualizations-2-7sxmn/create-visualizations-2-7sxmn-3549616354/create-visualizations-some_url_2.tgz",
      "type": "NoType"}, {"id": 37, "name": "main-logs", "uri":
      "minio://mlpipeline/artifacts/create-visualizations-2-7sxmn/create-visualizations-2-7sxmn-3549616354/main.log",
      "type": "NoType"}]
    sidecar.istio.io/inject: 'false'
    workflows.argoproj.io/node-name: create-visualizations-2-7sxmn.create-visualizations
    workflows.argoproj.io/outputs: >-
      {"artifacts":[{"name":"create-visualizations-some_html","path":"/tmp/outputs/Some_HTML/data","s3":{"endpoint":"minio-service.default:9000","bucket":"mlpipeline","insecure":true,"accessKeySecret":{"name":"mlpipeline-minio-artifact","key":"accesskey"},"secretKeySecret":{"name":"mlpipeline-minio-artifact","key":"secretkey"},"key":"artifacts/create-visualizations-2-7sxmn/create-visualizations-2-7sxmn-3549616354/create-visualizations-some_html.tgz"}},{"name":"create-visualizations-some_markdown","path":"/tmp/outputs/Some_Markdown/data","s3":{"endpoint":"minio-service.default:9000","bucket":"mlpipeline","insecure":true,"accessKeySecret":{"name":"mlpipeline-minio-artifact","key":"accesskey"},"secretKeySecret":{"name":"mlpipeline-minio-artifact","key":"secretkey"},"key":"artifacts/create-visualizations-2-7sxmn/create-visualizations-2-7sxmn-3549616354/create-visualizations-some_markdown.tgz"}},{"name":"create-visualizations-some_metrics","path":"/tmp/outputs/Some_metrics/data","s3":{"endpoint":"minio-service.default:9000","bucket":"mlpipeline","insecure":true,"accessKeySecret":{"name":"mlpipeline-minio-artifact","key":"accesskey"},"secretKeySecret":{"name":"mlpipeline-minio-artifact","key":"secretkey"},"key":"artifacts/create-visualizations-2-7sxmn/create-visualizations-2-7sxmn-3549616354/create-visualizations-some_metrics.tgz"}},{"name":"create-visualizations-some_url_1","path":"/tmp/outputs/Some_Url_1/data","s3":{"endpoint":"minio-service.default:9000","bucket":"mlpipeline","insecure":true,"accessKeySecret":{"name":"mlpipeline-minio-artifact","key":"accesskey"},"secretKeySecret":{"name":"mlpipeline-minio-artifact","key":"secretkey"},"key":"artifacts/create-visualizations-2-7sxmn/create-visualizations-2-7sxmn-3549616354/create-visualizations-some_url_1.tgz"}},{"name":"create-visualizations-some_url_2","path":"/tmp/outputs/Some_Url_2/data","s3":{"endpoint":"minio-service.default:9000","bucket":"mlpipeline","insecure":true,"accessKeySecret":{"name":"mlpipeline-minio-artifact","key":"accesskey"},"secretKeySecret":{"name":"mlpipeline-minio-artifact","key":"secretkey"},"key":"artifacts/create-visualizations-2-7sxmn/create-visualizations-2-7sxmn-3549616354/create-visualizations-some_url_2.tgz"}},{"name":"main-logs","archiveLogs":true,"s3":{"endpoint":"minio-service.default:9000","bucket":"mlpipeline","insecure":true,"accessKeySecret":{"name":"mlpipeline-minio-artifact","key":"accesskey"},"secretKeySecret":{"name":"mlpipeline-minio-artifact","key":"secretkey"},"key":"artifacts/create-visualizations-2-7sxmn/create-visualizations-2-7sxmn-3549616354/main.log"}}]}
@Ark-kun Ark-kun self-assigned this Jun 11, 2020
@rmgogogo rmgogogo added the status/triaged Whether the issue has been explicitly triaged label Jun 22, 2020
@Ark-kun
Copy link
Contributor Author

Ark-kun commented Jul 17, 2020

It's still recording types in some cases, but not in mall cases. There is some regression.

@Ark-kun
Copy link
Contributor Author

Ark-kun commented Jul 17, 2020

I've investigated the issue. It's caused by relaxation of Argo input/output name sanitization rules in the DSL compiler.

k8s-ci-robot pushed a commit that referenced this issue Jul 17, 2020
…etrieval. Fixes #3971 (#4231)

* Metadata Writer - Fixed regression with artifact type retrieval

The DSL compiler has changed the output name sanitization rules, so we should change them here accordingly.

* Added the code link
Bobgy pushed a commit that referenced this issue Jul 20, 2020
…etrieval. Fixes #3971 (#4231)

* Metadata Writer - Fixed regression with artifact type retrieval

The DSL compiler has changed the output name sanitization rules, so we should change them here accordingly.

* Added the code link
Jeffwan pushed a commit to Jeffwan/pipelines that referenced this issue Dec 9, 2020
…etrieval. Fixes kubeflow#3971 (kubeflow#4231)

* Metadata Writer - Fixed regression with artifact type retrieval

The DSL compiler has changed the output name sanitization rules, so we should change them here accordingly.

* Added the code link
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/metadata-writer status/triaged Whether the issue has been explicitly triaged
Projects
None yet
2 participants