Skip to content

Conversation

@Nataneljpwd
Copy link
Contributor

@Nataneljpwd Nataneljpwd commented Jul 2, 2025


As of now, each action performed with the sftp, requires a new connection, this pr allows for all the actions to be optionally done in 1 connection while you have to manage the connection manually

Related: #52289

@Nataneljpwd
Copy link
Contributor Author

Need to add some tests before it can be merged

@Nataneljpwd
Copy link
Contributor Author

@Mottimo here I added the feature which allows for only 1 connection to be used with the hook

@Nataneljpwd Nataneljpwd force-pushed the feature/add-optional-managed-conn branch 2 times, most recently from 58a35ef to 9d9c06b Compare July 8, 2025 16:15
@eladkal eladkal requested a review from Lee-W July 8, 2025 17:26
@Nataneljpwd Nataneljpwd force-pushed the feature/add-optional-managed-conn branch from c374239 to 0559075 Compare July 9, 2025 18:41
@Nataneljpwd
Copy link
Contributor Author

@Lee-W fixed the comments, if you can resolve the one's I fixed, otherwise, reply to them and I will fix them

@Nataneljpwd Nataneljpwd force-pushed the feature/add-optional-managed-conn branch from 0559075 to d9d95a9 Compare July 14, 2025 19:40
@Nataneljpwd
Copy link
Contributor Author

@Lee-W, I have updated it according to everything you said

@Lee-W
Copy link
Member

Lee-W commented Jul 15, 2025

we'll need to fix the CI as well. Also, I'll be out for the conference for the following 2 weeks and might need others help to look into the PR.

@Nataneljpwd
Copy link
Contributor Author

Sure, I will fix the ci, it seems to not find the tests for some reason

@Nataneljpwd
Copy link
Contributor Author

@Lee-W I have fixed the issue, it was that the ci required a test file for the exceptions file, I added it to the overlooked tests file list that is in the core tests as per convention on other providers

@Nataneljpwd Nataneljpwd force-pushed the feature/add-optional-managed-conn branch from d9d95a9 to 9d9b581 Compare July 16, 2025 17:14
@eladkal eladkal requested a review from Lee-W July 23, 2025 06:35
@Lee-W
Copy link
Member

Lee-W commented Jul 28, 2025

Just came back. Will take a look this week. Please resolve the comments if you think you've changed it and no discussion is needed. Thanks!

@Nataneljpwd
Copy link
Contributor Author

@Lee-W resolved all comments that do not require a discussion, only 1 left

@Nataneljpwd Nataneljpwd force-pushed the feature/add-optional-managed-conn branch from 01d0b98 to 00386fa Compare August 5, 2025 07:43
Copy link
Member

@Lee-W Lee-W left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks better now. We're closer to merge than the previous iteration!

@Nataneljpwd
Copy link
Contributor Author

@Lee-W, fixed all comments

Copy link
Member

@Lee-W Lee-W left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No major things need to be changed. most of these are nitpicks, but I feel we probably could improve the test coverage

@Nataneljpwd
Copy link
Contributor Author

@Lee-W fixed all comments, sorry it took me so long, I am just working in parallel on the scheduler starvation bug, not sure if you heard of it, PR number 53492 (in case you want to check it out).

Copy link
Member

@Lee-W Lee-W left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mostly good, a few nitpicks.

@eladkal
Copy link
Contributor

eladkal commented Aug 21, 2025

@Nataneljpwd can you fix the above issues?

@Nataneljpwd
Copy link
Contributor Author

@eladkal Sure, will fix it today and push

@Nataneljpwd
Copy link
Contributor Author

@Lee-W fixed all issues, including merge conflicts

Copy link
Member

@Lee-W Lee-W left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will need to fix the CI failure

@Nataneljpwd
Copy link
Contributor Author

@Lee-W it seems to be a faulty run

Fatal Python error: _enter_buffered_busy: could not acquire lock for <_io.BufferedWriter name='<stdout>'> at interpreter shutdown, possibly due to daemon threads

If you could rerun the job it should fix it

@eladkal
Copy link
Contributor

eladkal commented Aug 24, 2025

Looks like tests passed. cc @Lee-W

@Lee-W
Copy link
Member

Lee-W commented Aug 25, 2025

yep, let's merge it!

@Lee-W Lee-W merged commit bc35370 into apache:main Aug 25, 2025
141 of 142 checks passed
mangal-vairalkar pushed a commit to mangal-vairalkar/airflow that referenced this pull request Aug 30, 2025
Co-authored-by: Natanel Rudyuklakir <natanelrudyuklakir@gmail.com>
nothingmin pushed a commit to nothingmin/airflow that referenced this pull request Sep 2, 2025
Co-authored-by: Natanel Rudyuklakir <natanelrudyuklakir@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants