Relative time queries #4289

marvelm · 2017-10-23T19:20:16Z

Context

I would like to be able to create queries where the current time is injected by Parse.

For example:

const recentInactiveQ = new Parse.Query(Parse.Installation);
recentInactiveQ.greaterThan('createdAt', { $relativeTime: '2 weeks ago' });
recentInactiveQ.lessThan('updatedAt', { $relativeTime: '3 days ago' });

Parse.Push.send({
  where: recentInactiveQ,
  data: {
    alert: "Here's what you missed this week...",
    url: "https://example.org/this-weeks-content"
  },
});

The future

This type of query can be serialized and re-executed at any time. I plan on adding support for recurring push notifications and relative time queries are essential for this feature.
It would look something like this (#4105):

Parse.Push.createCampaign({
  where: recentInactiveQ,
  data: {
    alert: "Here's what you missed this week...",
    url: "https://example.org/this-weeks-content"
  },
  interval: 'daily',
  sendTime: '17:30',
});

flovilmart · 2017-10-24T19:35:51Z

I've added all of you as @marvelm is working with me @AmpMe on those features.

montymxb · 2017-10-24T22:14:21Z

spec/ParseQuery.spec.js

+      })
+      .then((results) => {
+        expect(results.length).toBe(0);
+      })


Considering that we have a test for getting at least one of each object, and one for neither, can we add one that returns both objects? Something using q.greaterThan('ttl', { $relativeTime: '3 days ago' }); is what comes to mind.

montymxb · 2017-10-24T22:17:13Z

src/Adapters/Storage/Mongo/MongoTransform.js

+  parts = parts.filter((part) => part !== '');
+
+  const future = parts[0] === 'in';
+  const past = parts[parts.length - 1] === 'ago';


Hmm, wondering if these should be case insensitive. We could lowercase the entire string in advance before processing.

lowercased on https://github.com/parse-community/parse-server/pull/4289/files#diff-82f0219ca721372e8f2bdbc45c83bbdcR537

montymxb · 2017-10-24T22:19:27Z

src/Adapters/Storage/Mongo/MongoTransform.js

+  // strip the 'ago' or 'in'
+  if (future) {
+    parts = parts.slice(1);
+  } else if (past) {


Do we need to do a check for past? Unless there's something else we're planning to add you could just use else.

flovilmart · 2017-10-24T19:36:22Z

spec/ParseQuery.spec.js

+      ttl: new Date(now - 2 * 24 * 60 * 60 * 1000), // 2 days ago
+    });
+
+    dropDatabase()


you don't need that, the DB is dropped between each test

flovilmart · 2017-10-24T19:36:36Z

spec/helper.js

@@ -451,3 +451,26 @@ jasmine.restoreLibrary = function(library, name) {
  }
  require(library)[name] = libraryCache[library][name];
 }
+
+const { MongoClient } = require('mongodb');


not needed.

montymxb · 2017-10-24T22:25:20Z

src/Adapters/Storage/Mongo/MongoTransform.js

+    switch(interval) {
+    case 'day':
+    case 'days':
+      seconds += val * 24 * 60 * 60;


We can just use ~~2880~~ (~~3600, oops...~~, wow 86400...) rather than performing the multiplication each time, but you could leave a comment to detail it if needed.

montymxb · 2017-10-24T22:27:33Z

src/Adapters/Storage/Mongo/MongoTransform.js

+
+    case 'hour':
+    case 'hours':
+      seconds += val * 60 * 60;


Same as above with 3600

montymxb · 2017-10-24T22:29:34Z

src/Adapters/Storage/Mongo/MongoTransform.js

+    case 'seconds':
+      seconds += val;
+      break;
+


Just a thought, but we could support shorthand interval values like sec, min, hr and their plurals. Not necessarily needed however.

montymxb · 2017-10-24T22:33:29Z

src/Adapters/Storage/Mongo/MongoTransform.js

+  let seconds = 0;
+  for (const [num, interval] of pairs) {
+    const val = Number(num);
+    if (!Number.isInteger(val)) {


With only whole integer values allowed perhaps we should add a test that expects to fail when someone tries something like 2.5 days.

From the code, as interpolated to seconds, it seems that it should work with non Integer values no?

Makes sense it would work, but we would want to test for it if we decide to allow it (or disallow it as currently).

let's stick with integers for now.

montymxb

Some recommendations and ideas for moving this along. The coverage is kind of low too, before this goes in we'll probably have addressed that however.

Overall this is super cool! The logic is pretty clean for converting from relative time textually on top of it, nice job.

flovilmart · 2017-10-24T22:47:20Z

src/Adapters/Storage/Mongo/MongoTransform.js

@@ -565,9 +663,22 @@ function transformConstraint(constraint, field) {
    case '$gte':
    case '$exists':
    case '$ne':
-    case '$eq':
+    case '$eq': {


we may enter on some weird expectations here: $ne { $relativeTime: "1 day ago" } would you expect all object that were created the full day before to be excluded or just the one that matches perfectly the current time?

@montymxb what do you think? Should we limit to $lt/$lte/$gt/$gte to be safe?

Hmm, it would make the most sense with the query methods you've mentioned above, but not sure if we should simply rule out $ne/$eq. Although they may not be the most effective query methods with relative date matching they could still work, especially if we allow setting the now date in the future (but still not that great).

I would just say exclude those, but if someone expects it to work what would we return? Just an empty response like the query matched nothing, or provide some sort of error clarifying that equal/notEqual ops are not allowed using this param, or another approach perhaps.

Or throw an error, that relativeDates operators don’t work with those operators

That would work for me 👍 . Simplifying the initial implementation may be best, we can consider it later if someone actually needs to use it.

https://github.com/parse-community/parse-server/blame/bd2ea87c1d508efe337a1e8880443b1a52a8fb81/CONTRIBUTING.md#L23

hr, hrs min, mins sec, secs

marvelm · 2017-10-25T14:49:02Z

Thanks for reviewing.

I had to rebase against master because of a failing test. I've addressed the issues raised in the review in "'Add integration test for multiple results' f8ce556" and the following commits.

Unnest query object

flovilmart · 2017-10-26T18:29:51Z

@montymxb this all look good to me!

montymxb · 2017-10-26T19:53:01Z

Changes look good, but the CI looks out of date. I'm going to bring this up to date with the master to retrigger build/coverage checking. I'm expecting it to come up as a ✓ (or close) but I want to see for certain before I give this an approval.

codecov · 2017-10-26T20:16:29Z

Codecov Report

Merging #4289 into master will decrease coverage by 0.03%.
The diff coverage is 86.27%.

@@            Coverage Diff             @@
##           master    #4289      +/-   ##
==========================================
- Coverage   92.53%   92.49%   -0.04%     
==========================================
  Files         118      118              
  Lines        8196     8246      +50     
==========================================
+ Hits         7584     7627      +43     
- Misses        612      619       +7

Impacted Files	Coverage Δ
src/Adapters/Storage/Mongo/MongoTransform.js	`85.22% <86.27%> (+0.24%)`	⬆️
src/RestWrite.js	`93.21% <0%> (-0.19%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1dd58b7...314bac5. Read the comment docs.

montymxb

Alright, looks good!

flovilmart requested review from dplewis, montymxb and acinader October 24, 2017 19:35

montymxb reviewed Oct 24, 2017

View reviewed changes

flovilmart reviewed Oct 24, 2017

View reviewed changes

montymxb reviewed Oct 24, 2017

View reviewed changes

montymxb suggested changes Oct 24, 2017

View reviewed changes

flovilmart reviewed Oct 24, 2017

View reviewed changes

montymxb mentioned this pull request Oct 24, 2017

How can I query dateTime field by Month and Year only parse-community/parse-php-sdk#331

Closed

Marvel Mathew added 15 commits October 25, 2017 10:13

Add relative time queries

5d9b227

Encode successful result

9b88e13

Add integration test

205fc35

Add more error cases

712c495

Remove unnecessary new Date

f3fce46

Error when time has both 'in' and 'ago'

b837e2a

naturalTimeToDate -> relativeTimeToDate

f4723a6

Add $relativeTime operator

477bbab

Throw error if $relativeTime is invalid

f28f3fd

Add integration test for invalid relative time

ced46cb

Exclude $exists query

b1e2594

Only run integration tests on MongoDB

db94a73

Add it_only_db test helper

5e503d1

https://github.com/parse-community/parse-server/blame/bd2ea87c1d508efe337a1e8880443b1a52a8fb81/CONTRIBUTING.md#L23

Handle where val might be null or undefined

6a09f17

Add integration test for multiple results

f8ce556

Marvel Mathew added 7 commits October 25, 2017 10:13

Lowercase text before processing

68641c4

Always past if not future

f82fff2

Precompute seconds multiplication

6a0e2ce

Add shorthand for interval

50dc4ba

hr, hrs min, mins sec, secs

Throw error if $relativeTime is used with $exists, $ne, and $eq

5f7d26b

Improve coverage for relativeTimeToDate

af0f831

Add test for erroring on floating point units

dc61f91

marvelm force-pushed the relative-time-queries branch from 0ccf228 to dc61f91 Compare October 25, 2017 14:47

Marvel Mathew and others added 5 commits October 25, 2017 10:51

Remove unnecessary dropDatabase function

1995052

Unit test $ne, $exists, $eq

91149e9

Verify field type

a46dadd

Fix unit test for $exists

6aa7442

Unnest query object

Merge branch 'master' into relative-time-queries

08d0743

flovilmart approved these changes Oct 26, 2017

View reviewed changes

flovilmart mentioned this pull request Oct 26, 2017

⚡ Release 2.6.5 #4298

Merged

Merge branch 'master' into relative-time-queries

314bac5

parse-community deleted a comment from codecov bot Oct 26, 2017

montymxb approved these changes Oct 26, 2017

View reviewed changes

flovilmart merged commit 6f1fe89 into parse-community:master Oct 26, 2017

montymxb mentioned this pull request Oct 30, 2017

Relative Time Queries parse-community/parse-php-sdk#360

Merged

dplewis mentioned this pull request Nov 3, 2019

Roadmap parse-community/Parse-SDK-iOS-OSX#1332

Closed

8 tasks

cbaker6 mentioned this pull request Dec 28, 2021

feat: support relativeTime query constraint on Postgres #7747

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Relative time queries #4289

Relative time queries #4289

marvelm commented Oct 23, 2017 •

edited

Loading

flovilmart commented Oct 24, 2017

montymxb Oct 24, 2017

montymxb Oct 24, 2017

flovilmart Oct 26, 2017

montymxb Oct 26, 2017

montymxb Oct 24, 2017

flovilmart Oct 24, 2017

flovilmart Oct 24, 2017

montymxb Oct 24, 2017 •

edited

Loading

montymxb Oct 24, 2017

montymxb Oct 24, 2017

montymxb Oct 24, 2017

flovilmart Oct 24, 2017

montymxb Oct 24, 2017

flovilmart Oct 26, 2017

montymxb left a comment

flovilmart Oct 24, 2017

flovilmart Oct 24, 2017

montymxb Oct 24, 2017

flovilmart Oct 24, 2017

montymxb Oct 24, 2017

marvelm commented Oct 25, 2017 •

edited

Loading

flovilmart commented Oct 26, 2017

montymxb commented Oct 26, 2017

codecov bot commented Oct 26, 2017 •

edited

Loading

montymxb left a comment

Relative time queries #4289

Relative time queries #4289

Conversation

marvelm commented Oct 23, 2017 • edited Loading

Context

The future

flovilmart commented Oct 24, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

montymxb Oct 24, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

montymxb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marvelm commented Oct 25, 2017 • edited Loading

flovilmart commented Oct 26, 2017

montymxb commented Oct 26, 2017

codecov bot commented Oct 26, 2017 • edited Loading

Codecov Report

montymxb left a comment

Choose a reason for hiding this comment

marvelm commented Oct 23, 2017 •

edited

Loading

montymxb Oct 24, 2017 •

edited

Loading

marvelm commented Oct 25, 2017 •

edited

Loading

codecov bot commented Oct 26, 2017 •

edited

Loading