Skip to content

SageMakerTransformOperator does not deduplicate model name #39502

@mxuan0

Description

@mxuan0

Apache Airflow Provider(s)

amazon

Versions of Apache Airflow Providers

8.7.1

Apache Airflow version

2.7.2

Operating System

amazon linux AMI

Deployment

Amazon (AWS) MWAA

Deployment details

No response

What happened

The SageMakerTransformOperator deduplicates job name before creating the job, but does not deduplicate the model name before creating the model. Upon retry, the operator would fail due to model already exist error.

What you think should happen instead

The operator should apply the same behavior (fail or add timestamp) to both the job name and the model name.

How to reproduce

create a DAG containing a SageMakerTransformOperator step. In the operator config, fix both ModelName and TransformJobName. In airflow UI, run the task until completion in airflow once, and the run the same task for the second time by "clear task".

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions