Skip to content

Conversation

@yash1thsa
Copy link
Contributor

Feel free to ping committers for the review!

In case of an existing issue, reference it using one of the following:

closes: #54561

Issue:
AWS BatchOperator does not fetch log entries for deferred jobs. The issue is prevelant in MWAA and other deployment as well.

Steps:

  • Reproduced the issue in local with AWS connection
  • Code: For deferred path in batch operator module, add monitor_job function as a downstream step
  • Tested the fix with both test cases and running in local with AWS connection

Tests:

  • Happy path with awslog_enabled set to True
  • Happy path with awslog_enabled set to False
  • Failure step for validation error

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@boring-cyborg boring-cyborg bot added area:providers provider:amazon AWS/Amazon - related issues labels Sep 16, 2025
@eladkal eladkal requested a review from vincbeck September 16, 2025 02:31
@vincbeck
Copy link
Contributor

Static checks are failing, please fix them. If you're not familiar, you can read the doc about it.

@yash1thsa
Copy link
Contributor Author

Static checks are failing, please fix them. If you're not familiar, you can read the doc about it.

@vincbeck Thanks for document. Let me review.

@yash1thsa yash1thsa force-pushed the fix/54561 branch 2 times, most recently from 5bc47fb to b489407 Compare September 16, 2025 18:54
@yash1thsa
Copy link
Contributor Author

Cleaned up all CI errors and rebased to master. Please review and let me know for any issues

@vincbeck vincbeck requested a review from o-nikolas September 17, 2025 16:38
@o-nikolas o-nikolas merged commit d18e3dc into apache:main Sep 17, 2025
78 checks passed
@boring-cyborg
Copy link

boring-cyborg bot commented Sep 17, 2025

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Sep 30, 2025
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 1, 2025
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 2, 2025
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 3, 2025
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 4, 2025
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 5, 2025
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 5, 2025
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 7, 2025
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 8, 2025
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 9, 2025
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 10, 2025
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 11, 2025
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 12, 2025
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 14, 2025
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 15, 2025
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 17, 2025
abdulrahman305 bot pushed a commit to abdulrahman305/airflow that referenced this pull request Oct 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:amazon AWS/Amazon - related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AWS BatchOperator does not fetch log entries for deferred jobs

3 participants