-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Makes operator checking deterministic in Query.filter. #294
Makes operator checking deterministic in Query.filter. #294
Conversation
I'm -1 to making it filter(field, operator, value) -- the reason being that this is how google.appengine.ext.db/ndb handled filtering (I just modeled it after them). Another option to fix this is to use a regex: I'd also feel much better if there were failing test to reproduce this as it's kind of a tricky bug... Any chance you could toss one in? Or add a ticket to make sure we have one to avoid regressing? |
@jgeewax RE: emulating qry = Account.query(Account.userid == 42) I would also guess that they were like that in As for a regression test, #281 is what made us discover this bug in the first place and I just commited fc23837. |
👍 -- I think for this library, For a higher-level library (maybe we'll even pull |
I think Another option that doesn't require a full ORM is just adding a Property class with operator overloading: |
@Alfus that is on the roadmap, but as part of a higher-level abstraction than this one. Here we essentially want to mimic the API. In that vein, the |
The order of dictionary keys is not guaranteed between runs of code so `expression`s ending in '>=' or '<=' were sometimes matching for '='.
fc23837
to
799b6b7
Compare
Rebased after #292 |
@@ -132,10 +134,12 @@ def filter(self, expression, value): | |||
property_name, operator = None, None | |||
expression = expression.strip() | |||
|
|||
for operator_string in self.OPERATORS: | |||
for operator_string, pb_op_enum in self.OPERATORS: | |||
if expression.endswith(operator_string): |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
LGTM pending Travis. I don't know if we want to default to assuming equality in the case that there is no operator, but that shouldn't be a blocker. |
This was to get test coverage back to 100%.
Missing a few lines of test coverage so added 1fb43bd Will wait out Travis again then merge. Somewhat surprised |
…rder Makes operator checking deterministic in Query.filter.
f_pb, = list(q_pb.filter.composite_filter.filter) | ||
p_pb = f_pb.property_filter | ||
self.assertEqual(p_pb.property.name, 'firstname') | ||
self.assertEqual(p_pb.value.string_value, u'John') | ||
self.assertEqual(p_pb.operator, datastore_pb.PropertyFilter.EQUAL) | ||
|
||
def test_filter_w_all_operators(self): |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
…ed (googleapis#330 and googleapis#294) Updated Readme to reflect this change
* chore: Update gapic-generator-python to v1.8.5 PiperOrigin-RevId: 511892190 Source-Link: googleapis/googleapis@a45d9c0 Source-Link: googleapis/googleapis-gen@1907294 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMTkwNzI5NGIxZDgzNjVlYTI0ZjhjNWYyZTA1OWE2NDEyNGM0ZWQzYiJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <partheniou@google.com>
* feat: add `content` field in TextAnchor PiperOrigin-RevId: 435142243 Source-Link: googleapis/googleapis@721b367 Source-Link: googleapis/googleapis-gen@b627aae Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiYjYyN2FhZWIzYWZhZmU3ZDk0ZDYwOTlhMmQxOTQ1YWE5YjRkYWRhZCJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
- [ ] Regenerate this pull request now. fix: resolve DuplicateCredentialArgs error when using credentials_file committer: parthea PiperOrigin-RevId: 425964861 Source-Link: googleapis/googleapis@84b1a5a Source-Link: googleapis/googleapis-gen@4fb761b Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNGZiNzYxYmJkODUwNmFjMTU2ZjQ5YmFjNWYxODMwNmFhOGViM2FhOCJ9
* chore(deps): update all dependencies * revert Co-authored-by: Anthonios Partheniou <partheniou@google.com>
* chore: exclude requirements.txt file from renovate-bot Source-Link: googleapis/synthtool@f58d313 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:7a40313731a7cb1454eef6b33d3446ebb121836738dc3ab3d2d3ded5268c35b6 * update constraints files * fix(deps): require protobuf 3.20.2 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <partheniou@google.com>
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <partheniou@google.com>
Source-Link: googleapis/synthtool@f15cc72 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:bc5eed3804aec2f05fad42aacf973821d9500c174015341f721a984a0825b6fd
* docs: minor wording update PiperOrigin-RevId: 442267451 Source-Link: googleapis/googleapis@4445d18 Source-Link: googleapis/googleapis-gen@4e175b7 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNGUxNzViNzllYzQxNThmMmU3ZmFiOWY0ZGVhMGViZjc2M2E1Y2FhNiJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * docs: minor wording update PiperOrigin-RevId: 442267541 Source-Link: googleapis/googleapis@740f072 Source-Link: googleapis/googleapis-gen@29eb892 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMjllYjg5MmNjYTViMWIwMTM0YjRlYzE4NWRkM2QyZWY1ZDJhZjFhNCJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
PiperOrigin-RevId: 373895026 Source-Link: googleapis/googleapis@0d68bbb Source-Link: googleapis/googleapis-gen@6acf4a0
…ic enums (#294) * feat: enable "rest" transport in Python for services supporting numeric enums PiperOrigin-RevId: 508143576 Source-Link: googleapis/googleapis@7a702a9 Source-Link: googleapis/googleapis-gen@6ad1279 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNmFkMTI3OWMwZTdhYTc4N2FjNmI2NmM5ZmQ0YTIxMDY5MmVkZmZjZCJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * use REST transport in samples/tests * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * revert changes to fixtures * revert * revert --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Anthonios Partheniou <partheniou@google.com>
* chore(deps): update all dependencies * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
- [ ] Regenerate this pull request now. PiperOrigin-RevId: 473833416 Source-Link: googleapis/googleapis@565a550 Source-Link: googleapis/googleapis-gen@1ee1a06 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiMWVlMWEwNmM2ZGUzY2E4Yjg0MzU3MmMxZmRlMDU0OGY4NDIzNjk4OSJ9
#294) Source-Link: googleapis/synthtool@694118b Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:ec49167c606648a063d1222220b48119c912562849a0528f35bfb592a9f72737 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Source-Link: googleapis/synthtool@c4dd595 Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:ce3c1686bc81145c81dd269bd12c4025c6b275b22d14641358827334fddb1d72
Co-authored-by: Anthonios Partheniou <partheniou@google.com>
* docs(samples): adding sample for local SSD disk * Updating region tag * Regenerating * Updating snippets Co-authored-by: Anthonios Partheniou <partheniou@google.com>
* chore(deps): update dependency google-cloud-storage to v2 * 🦉 Updates from OwlBot See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
* chore: update gapic-generator-python-1.4.4 with unit tests generation fixes PiperOrigin-RevId: 475683078 Source-Link: googleapis/googleapis@df791ce Source-Link: googleapis/googleapis-gen@ee0ce41 Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZWUwY2U0MWNiOWJmN2JiNTEwMDVlOTkxMDIyYjAwY2UxYmM2NjlkOCJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: Anthonios Partheniou <partheniou@google.com>
* feat: Add Cloud Trace v2 retry defaults for BatchWriteSpans PiperOrigin-RevId: 504068544 Source-Link: googleapis/googleapis@685d359 Source-Link: googleapis/googleapis-gen@43aa9da Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNDNhYTlkYWNmYzQxN2I2MjVhZDhiZTFiZWQyYTlmYzM2YzI3N2RiNiJ9 * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
The order of dictionary keys is not guaranteed between runs of
code so
expression
s ending in '>=' or '<=' were sometimesmatching for '='.
@pcostell the issue with flakiness was non-deterministic behavior of Python internals. Yay for finding the bug!
This was holding up #281, but it also forced us to use ancestor queries, which is a good thing.
@tseaver This is the quick-and-dirtiest way to fix this issue but I am a fan of changing from
to
WDYT?
Also @jgeewax WDYT since your original design. (Maybe you had reasons for combination property name and operator?)