Skip to content

Conversation

@fopina
Copy link
Contributor

@fopina fopina commented May 13, 2025

Description

This PR allows specifying an explicit proxy to the underlying requests.Session, overriding environment configuration.
This is useful when a specific proxy is required to access OpenSearch but nothing else (such as local tunnels).

Workaround

I've been using a very small/clean subclass across different projects for this, however I think it's clean and useful enough to be pushed to the main upstream class 😄

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@fopina fopina changed the title Added support for explicit proxy to RequestsHttpConnection Explicit proxy support in RequestsHttpConnection May 13, 2025
@fopina fopina force-pushed the requestshttpconnection/explicit_proxy_support branch 3 times, most recently from b7c1dbc to e0e40bf Compare May 13, 2025 10:02
@codecov
Copy link

codecov bot commented May 13, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.40%. Comparing base (ba715b9) to head (e31ace0).
Report is 90 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #908      +/-   ##
==========================================
- Coverage   71.95%   70.40%   -1.56%     
==========================================
  Files          91      125      +34     
  Lines        8001     9291    +1290     
==========================================
+ Hits         5757     6541     +784     
- Misses       2244     2750     +506     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@fopina fopina force-pushed the requestshttpconnection/explicit_proxy_support branch 4 times, most recently from 1dba2cc to f308948 Compare May 13, 2025 10:50
@fopina
Copy link
Contributor Author

fopina commented May 13, 2025

Sorry for the push force spam, I did not expect so many linting complains off a single lint 😓

Ready for review now

@dblock
Copy link
Member

dblock commented May 13, 2025

Thanks. This needs tests, please.

Integ test CI is failing, needs to be looked at. Likely the 3.0 upgrade in CI to be done first in a separate PR (please help).

@fopina
Copy link
Contributor Author

fopina commented May 16, 2025

@dblock I looked at the integration tests but they are failing for the same reason as every other PR I guess. I assume #905 is trying to fix them?

I'll look into writing a unit test sure

@fopina fopina force-pushed the requestshttpconnection/explicit_proxy_support branch from 64d6147 to e31ace0 Compare May 16, 2025 10:46
@fopina
Copy link
Contributor Author

fopina commented May 16, 2025

Unit test I added. I suppose it's ok like this (as http_auth) as testing further would be testing requests proxy implementation.

codecov is doing something funky:

  • it passed in the previous commit - this last one only adds a test, impossible to decrease coverage...
  • seems it compared to main on ba715b9 (very old main) for some reason...

Thanks, looking forward to see this merged and stop using an external package 🙏

@dblock
Copy link
Member

dblock commented May 18, 2025

We can ignore codecov for now.

@nathaliellenaa are you working on #905? @fopina feel free to take it over

@fopina fopina closed this May 18, 2025
@fopina fopina reopened this May 18, 2025
@fopina
Copy link
Contributor Author

fopina commented May 18, 2025

@dblock I have little to no context in this project setup, already took forever just to get tests to run locally, so I’ll have to pass.
I’ll rebase once it’s fixed, if it cannot be merged as is

@nathaliellenaa
Copy link
Contributor

@dblock @fopina I'm waiting for this PR to get merged, then I will rebase and rerun the integ tests to identify and resolve any remaining issues.

@dblock
Copy link
Member

dblock commented May 20, 2025

Looks like it's 2 weeks old, afaik nobody is working on it.

fopina added 2 commits July 15, 2025 17:23
Signed-off-by: Filipe Pina <hzlu1ot0@duck.com>
Signed-off-by: Filipe Pina <hzlu1ot0@duck.com>
@fopina fopina force-pushed the requestshttpconnection/explicit_proxy_support branch from e31ace0 to f98a3ed Compare July 15, 2025 17:23
@fopina
Copy link
Contributor Author

fopina commented Jul 16, 2025

Hi @dblock just noticed #905 was merged and fixed the tests so I rebased et voila, all green 👍

Copy link
Member

@dblock dblock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can merge, but we have a base class with the intent to supporting these features across all connection types if I am not mistaken. Before I do mind checking if all the connection class parameters align and whether this needs to be added to the others? It also seems like we may need tests to ensure that's the case.

@fopina
Copy link
Contributor Author

fopina commented Jul 20, 2025

It could be added to base class but then isn't it expected for all connections to actually support specifying proxies?
Or would the parameter just be ignored?
I just assumed it'd be better to have it at subclass level and, if at some point, all (or majority) do make any use of similar parameter, to move to base?

@dblock
Copy link
Member

dblock commented Jul 21, 2025

Ok, I'll merge. Someone should refactor this whole interface and instead of passing a million different parameters it should take separate session options that allow passing anything to self.session that it supports.

@dblock dblock merged commit d1fa749 into opensearch-project:main Jul 21, 2025
33 checks passed
@fopina
Copy link
Contributor Author

fopina commented Jul 21, 2025

That's a good point, "trust_env" is, sadly, something that has to be used quite often with requests.Session as he refused to fix a clear bug
There might be other edge cases / attributes.
I'll try to find the time for a proposal
Thanks for merging!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants