-
Notifications
You must be signed in to change notification settings - Fork 16.4k
AIP-67 - Multi-team: AwsBatchExecutor per team executor config #60920
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AIP-67 - Multi-team: AwsBatchExecutor per team executor config #60920
Conversation
d8e705d to
538035c
Compare
|
Thanks for the PR, I will review it in the next day or so! |
o-nikolas
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code looks good. Did you do any manual testing? If so can you describe the testing you did and the results you saw?
|
Hi @o-nikolas, thanks for the review! Yes, I did some manual testing. Set up AWS Batch (Fargate) to verify the changes work end-to-end. Built a Docker image with the PR changes and ran it on AWS Batch. Job succeeded:
Logs output:
The logs show: Image in ECR:
Everything works - the executor loads correctly with The Let me know if you'd like me to test anything else! Thank you. |
o-nikolas
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good! Thanks for testing and contribution!
vincbeck
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good!
ferruzzi
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks reasonable.



Summary
Add multi-team support to the AWS Batch Executor,
Changes
supports_multi_team: bool = Trueflag toAwsBatchExecutorMAX_SUBMIT_JOB_ATTEMPTSto instance-levelmax_submit_job_attemptsconf.get()calls to useself.conf.get()for team-specific configuration_load_submit_kwargs()from static method to instance methodbuild_submit_kwargs(conf)to takeconfas required parametertest_team_configtest to verify team isolation works{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.