Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: improve logic in is_select #17329

Merged
merged 2 commits into from
Nov 3, 2021
Merged

Conversation

betodealmeida
Copy link
Member

@betodealmeida betodealmeida commented Nov 2, 2021

SUMMARY

sqlparse fails to identify this query as a SELECT:

WITH foo AS(
SELECT
  FLOOR(__time TO WEEK) AS "week",
  name,
  COUNT(DISTINCT user_id) AS "unique_users"
FROM "druid"."my_table"
GROUP BY 1,2
)
SELECT
  f.week,
  f.name,
  f.unique_users
FROM foo f

This happens because AS( (no space) is identified as a function, and because of that the statement has type "UKNOWN".

I improved the logic in is_select so that when the token is "UNKNOWN" we check for DDL and DML, returning true only if there's no DDL and the only DML is a SELECT.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

N/A

TESTING INSTRUCTIONS

I added unit tests covering the bug and a few more.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

Copy link
Member

@mistercrunch mistercrunch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@codecov
Copy link

codecov bot commented Nov 2, 2021

Codecov Report

Merging #17329 (ffce826) into master (7c6d6f4) will decrease coverage by 0.06%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #17329      +/-   ##
==========================================
- Coverage   77.15%   77.09%   -0.07%     
==========================================
  Files        1037     1037              
  Lines       55639    55647       +8     
  Branches     7612     7612              
==========================================
- Hits        42930    42902      -28     
- Misses      12459    12495      +36     
  Partials      250      250              
Flag Coverage Δ
hive 81.55% <100.00%> (+<0.01%) ⬆️
mysql 81.98% <100.00%> (+<0.01%) ⬆️
postgres 81.99% <100.00%> (+<0.01%) ⬆️
presto ?
python 82.34% <100.00%> (-0.12%) ⬇️
sqlite 81.66% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/sql_parse.py 99.42% <100.00%> (+0.02%) ⬆️
superset/db_engine_specs/presto.py 84.30% <0.00%> (-6.07%) ⬇️
superset/connectors/sqla/models.py 85.99% <0.00%> (-1.41%) ⬇️
superset/models/core.py 89.26% <0.00%> (-0.74%) ⬇️
superset/utils/cache.py 74.52% <0.00%> (+0.94%) ⬆️
superset/result_set.py 98.38% <0.00%> (+1.61%) ⬆️
superset/utils/celery.py 89.65% <0.00%> (+3.44%) ⬆️
superset/connectors/sqla/utils.py 92.15% <0.00%> (+3.92%) ⬆️
superset/db_engine_specs/sqlite.py 97.29% <0.00%> (+5.40%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7c6d6f4...ffce826. Read the comment docs.

@betodealmeida betodealmeida merged commit 93bafa0 into apache:master Nov 3, 2021
AAfghahi pushed a commit that referenced this pull request Jan 10, 2022
* feat: improve logic in is_select

* Add more edge cases
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.5.0 labels Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/M 🚢 1.5.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants