Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new SQLCheckOperators #23915

Closed

Commits on May 25, 2022

  1. Add new SQLCheckOperators

    This commit adds two new SQL Check Operators to the core sql.py file,
    one called SQLColumnCheckOperator and one called SQLTableCheckOperator.
    Corresponding tests are added as well. The current functionality of
    these operators is minimal, but expected to grow over time. These
    operators, unlike current ones, save information about the results
    of the check queries that is then parseable by third party libraries.
    denimalpaca committed May 25, 2022
    Configuration menu
    Copy the full SHA
    f9d3d3e View commit details
    Browse the repository at this point in the history
  2. Use list comprehension in _get_failed_tests()

    Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>
    denimalpaca and uranusjr authored May 25, 2022
    Configuration menu
    Copy the full SHA
    a58a3c5 View commit details
    Browse the repository at this point in the history
  3. Remove repeated lines of code

    Accidentally copied the last line of the file several times, this
    commit remedies that change.
    denimalpaca committed May 25, 2022
    Configuration menu
    Copy the full SHA
    c6950de View commit details
    Browse the repository at this point in the history
  4. Fix doc strings.

    denimalpaca committed May 25, 2022
    Configuration menu
    Copy the full SHA
    1818691 View commit details
    Browse the repository at this point in the history

Commits on Jun 3, 2022

  1. Update SQLColumnCheckOperator

    Improved the column check operator to include support for
    greater than, geq, less than, and leq functions.
    Updated logic and tests to reflect this change.
    denimalpaca committed Jun 3, 2022
    Configuration menu
    Copy the full SHA
    fc8380e View commit details
    Browse the repository at this point in the history
  2. Update SQLTableCheckOperator

    Improved the table check operator by refactoring
    how checks are performed; this makes it much more customizeable
    at the expense of knowing the specific result of checks, as
    the results are now all just 0 or 1.
    denimalpaca committed Jun 3, 2022
    Configuration menu
    Copy the full SHA
    0f56b08 View commit details
    Browse the repository at this point in the history

Commits on Jun 6, 2022

  1. Fix doc errors in docstrings

    New docstrings were using example dictionaries in the param list,
    and the way they were written before caused doc build errors. The
    fix was to update the formatting and add missing words to the word list.
    denimalpaca committed Jun 6, 2022
    Configuration menu
    Copy the full SHA
    14afffb View commit details
    Browse the repository at this point in the history
  2. Reorder logging

    Moves record logging below the AirflowException for no records,
    and makes the Exception message more clear.
    denimalpaca committed Jun 6, 2022
    Configuration menu
    Copy the full SHA
    eba96f3 View commit details
    Browse the repository at this point in the history
  3. Apply suggestions from code review

    Co-authored-by: Josh Fell <48934154+josh-fell@users.noreply.github.com>
    denimalpaca and josh-fell authored Jun 6, 2022
    Configuration menu
    Copy the full SHA
    2388dd8 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    cf89bf5 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    c83c79e View commit details
    Browse the repository at this point in the history

Commits on Jun 8, 2022

  1. Configuration menu
    Copy the full SHA
    7136397 View commit details
    Browse the repository at this point in the history

Commits on Jun 9, 2022

  1. Fix SQL rendering bug in SQLTableCheckOperator

    SQLTableCheckOperator was rendering an invalid SQL query,
    this fix changes the way the query template is built from the
    passed in strings.
    denimalpaca committed Jun 9, 2022
    Configuration menu
    Copy the full SHA
    cbc76c1 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    fa083ea View commit details
    Browse the repository at this point in the history
  3. Remove quotes around column in null_check

    After some research, it seems the null_check query does *not* need
    quotes around the column name.
    denimalpaca committed Jun 9, 2022
    Configuration menu
    Copy the full SHA
    7439986 View commit details
    Browse the repository at this point in the history

Commits on Jun 13, 2022

  1. Configuration menu
    Copy the full SHA
    99158b9 View commit details
    Browse the repository at this point in the history