-
Notifications
You must be signed in to change notification settings - Fork 16.4k
Improve Mongo connection validation and UI #41717
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
4468ad9 to
629768a
Compare
629768a to
e7f8123
Compare
jscheffl
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM... @uranusjr any objections?
6b18ab6 to
dc7f540
Compare
uranusjr
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One nit, otherwise looks good to me.
Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>
a2d1ce6 to
6963415
Compare
|
I simplified the failing sensor test which was added by someone else in a different PR. Now it mocks the MongoHook and only tests the interactions between the sensor code and the hook instance inside the sensor, the details of connections etc are not necessary to confirm the sensor is working as designed. (IMO if we do want to test more realistically, we should use mongomock to mock the Mongo endpoint, and have real non-mocked MongoHook and Connection objects, but that's beyond the scope of this PR). |
|
Hi, I believe that this PR is now breaking the functionality of connections that are imported from env vars for example. When parsing the connection from a uri, in case of mongo, the scheme == conn_type and it will now fail in validations as schema is either airflow/airflow/models/connection.py Line 215 in ab3429c
|
Can you elaborate some details, provide an example and post a separate issue entry as provider bug report? Maybe linking to this PR. |
* hook changes * test changes * UI * test and alert * use AirflowConfigException * fix new test * Update airflow/providers/mongo/hooks/mongo.py Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com> * tweak sensor test --------- Co-authored-by: Elad Kalif <45845474+eladkal@users.noreply.github.com> Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>
This change tightens up the Mongo connection UX:
MongoHookconstructor validates the connection object to ensure it is of the correctconn_type(mongo), raising an exception otherwisemongodb+srv, the constructor exception provides an informative message telling the user to use themongoconn_type and setsrv=truein theextrassrv,sslandallow_insecureextra parametersThis is a replacement for #41371 which aimed to improve the Mongo connection parameters in a different and less-functional way.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.