Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(server/pypika/filter): Fix "notmatches" operator for backends using REGEXP #1507

Merged
merged 1 commit into from
Oct 4, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
fix(server/pypika/filter): Fix "notmatches" operator for backends usi…
…ng REGEXP

closes TCTC-3808

Signed-off-by: Luka Peschke <luka.peschke@toucantoco.com>
  • Loading branch information
lukapeschke committed Oct 4, 2022
commit 1432095beea138df72f3d9aed071ec98b22dcc97
Original file line number Diff line number Diff line change
Expand Up @@ -954,7 +954,7 @@ def _get_single_condition_criterion(
elif condition.operator == "notmatches":
match self.REGEXP_OP:
case RegexOp.REGEXP:
return column_field.regexp(condition.value).negate()
return column_field.regexp(compliant_regex).negate()
case RegexOp.SIMILAR_TO:
return BasicCriterion(
RegexpMatching.not_similar_to,
Expand Down
30 changes: 30 additions & 0 deletions server/tests/backends/fixtures/filter/notmatches_pypika.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
exclude:
- mongo
- pandas
- snowflake
step:
pipeline:
- condition:
column: name
operator: notmatches
value: beer
name: filter
- columns:
- name
name: select
expected:
data:
- name: Ninkasi Ploploplop
- name: Brewdog Nanny State Alcoholvrij
- name: Ardwen Blonde
- name: "Cuv\xE9e des Trolls"
- name: Weihenstephan Hefe Weizen Alcoholarm
- name: Bellfield Lawless Village IPA
- name: Pauwel Kwak
- name: Brasserie De Sutter Brin de Folie
- name: Brugse Zot blonde
schema:
fields:
- name: name
type: string
pandas_version: 1.4.0
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ def test_notmatches_regexp_filter(
ctx = regexp_translator.filter(step=step, columns=selected_columns, **default_step_kwargs)
expected_query = (
Query.from_(previous_step)
.where(Field(column).regexp(regex).negate())
.where(Field(column).regexp(f"%{regex}%").negate())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I disagree. Here, I want to test that the function is called with the regex surrounded by % characters. The test form you suggest would not catch an error if the get_compliant_regex function was bogus

.select(*selected_columns)
)

Expand Down Expand Up @@ -358,7 +358,7 @@ def test_notmatches_regexp__like_filter(
ctx = regexp_translator.filter(step=step, columns=selected_columns, **default_step_kwargs)
expected_query = (
Query.from_(previous_step)
.where(Field(column).regexp(regex).negate())
.where(Field(column).regexp(f"%{regex}%").negate())
.select(*selected_columns)
)

Expand Down Expand Up @@ -417,7 +417,7 @@ def test_notmatches_regexp__contains_filter(
ctx = regexp_translator.filter(step=step, columns=selected_columns, **default_step_kwargs)
expected_query = (
Query.from_(previous_step)
.where(Field(column).regexp(regex).negate())
.where(Field(column).regexp(f"%{regex}%").negate())
.select(*selected_columns)
)

Expand Down