Trigger AWS Step Function Workflow

[cicdw]

We should add a Task to our Task Library (in src/prefect/tasks/aws) for triggering the execution of AWS Step Function workflows. Presumably this can be done from the Step Function API (e.g., https://docs.aws.amazon.com/step-functions/latest/apireference/API_StartExecution.html) with some authentication parameters that can be stored as Prefect Secrets.

[jhagege]

Hi, is it possible to fetch a result at the end of the execution of the step function ?
I was thinking doing so with a PubSub / SNS message sent from the Step function job, but then how to fetch the result back in the Prefect pipeline ?
Thanks for any insight.

[cicdw]

Hi @jhagege - yes, it is possible using the describe_execution call from boto3 which will return the JSON output from the step function.

It's not immediately clear to me whether boto3 has a Waiter for step functions, but if it did then it would be fairly easy to wait for job completion and return that result back into your Prefect Flow