model WHERE clause #68

jgellin-sf · 2020-11-11T14:50:59Z

What does this PR do?

This PR models the WHERE clause. All types of conditions are modeled except for the following, which are modeled as UnmodeledSyntax:

conditions with semi-joins, such as field IN (SELECT A FROM B)
distance conditions, such as DISTANCE(field,GEOLOCATION(37,122),'mi') < 100
calculated conditions, such as A + B > 10
conditions with colon expressions, such as field = :var

This PR also updates the soql parser to version 0.17. This update includes better handling of WHERE expressions (timely!) and several other updates. Some rules were renamed, resulting in some minor code changes in this PR.

What issues does this PR fix or reference?

W-8295457

codecov · 2020-11-11T14:51:07Z

Codecov Report

Merging #68 (8bd8fb3) into develop (98474e7) will increase coverage by 0.02%.
The diff coverage is 94.27%.

@@             Coverage Diff             @@
##           develop      #68      +/-   ##
===========================================
+ Coverage    94.12%   94.14%   +0.02%     
===========================================
  Files           33       42       +9     
  Lines          851     1076     +225     
  Branches       177      227      +50     
===========================================
+ Hits           801     1013     +212     
- Misses          48       61      +13     
  Partials         2        2

Flag	Coverage Δ
language-server	`100.00% <ø> (ø)`
soql-builder-ui	`92.57% <ø> (ø)`
soql-model	`94.99% <94.27%> (-0.40%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...kages/soql-model/src/serialization/deserializer.ts	`92.59% <87.00%> (-2.55%)`	⬇️
...es/soql-model/src/model/impl/andOrConditionImpl.ts	`100.00% <100.00%> (ø)`
...-model/src/model/impl/fieldCompareConditionImpl.ts	`100.00% <100.00%> (ø)`
...s/soql-model/src/model/impl/inListConditionImpl.ts	`100.00% <100.00%> (ø)`
...soql-model/src/model/impl/includesConditionImpl.ts	`100.00% <100.00%> (ø)`
packages/soql-model/src/model/impl/index.ts	`100.00% <100.00%> (ø)`
...ges/soql-model/src/model/impl/likeConditionImpl.ts	`100.00% <100.00%> (ø)`
packages/soql-model/src/model/impl/literalImpl.ts	`100.00% <100.00%> (ø)`
...s/soql-model/src/model/impl/nestedConditionImpl.ts	`100.00% <100.00%> (ø)`
...ages/soql-model/src/model/impl/notConditionImpl.ts	`100.00% <100.00%> (ø)`
... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 98474e7...8bd8fb3. Read the comment docs.

jgellin-sf · 2020-11-11T14:51:54Z

packages/soql-model/scripts/getquery.js

+  new model.Impl.WhereImpl(
+    new model.Impl.FieldCompareConditionImpl(
+      new model.Impl.FieldRefImpl('field1'),
+      model.Soql.CompareOperator.EQ,
+      new model.Impl.LiteralImpl(model.Soql.LiteralType.Number, '5')
+    )
+  ),


Sample script update

jgellin-sf · 2020-11-11T14:52:07Z

packages/soql-model/scripts/getsobject.js

+var where = queryModel.where ? queryModel.where.toSoqlSyntax() : undefined;
 var orderBy = queryModel.orderBy.orderByExpressions.map(
  (expr) => `${expr.field.fieldName} ${expr.order} ${expr.nullsOrder}`
 );
 var limit = queryModel.limit ? queryModel.limit.limit : 'undefined';
 console.log(`Query:    ${query}`);
 console.log(`SObject:  ${sObject}`);
 console.log(`Fields:   ${fields}`);
+console.log(`Where:    ${where}`);


Sample script update

Would line 32 be good to look at to get an idea of whats available to the UI for a where clause?

No that's just the where clause transformed back to SOQL syntax. Line 16 of getquery.js is indicative of how the model is created. For raw JSON, deserializer.test.ts has lots of examples of what these different conditions look like when modeled in a WHERE clause.

jgellin-sf · 2020-11-11T14:54:06Z

packages/soql-model/src/serialization/deserializer.ts

  protected isIncompleteLimitError(error: ParserError): boolean {
    const context = this.matchErrorToContext(error);
-    return context instanceof Parser.SoqlIntegerContext
+    return context instanceof Parser.SoqlIntegerValueContext


This change is due to a parser rule rename in the new soql parser version

jgellin-sf · 2020-11-11T14:57:52Z

packages/soql-model/src/serialization/deserializer.ts

+  protected toLiteral(ctx: ParserRuleContext): Soql.Literal {
+    if (ctx instanceof Parser.SoqlLiteralLiteralValueContext) {
+      ctx = ctx.soqlLiteral();
+    }
+    if (ctx instanceof Parser.SoqlLiteralCommonLiteralsContext) {
+      ctx = ctx.soqlCommonLiterals();
+    }
+    if (ctx instanceof Parser.SoqlDateLiteralContext) {
+      return new Impl.LiteralImpl(Soql.LiteralType.Date, ctx.getText());
+    } else if (ctx instanceof Parser.SoqlDateTimeLiteralContext) {
+      return new Impl.LiteralImpl(Soql.LiteralType.Date, ctx.getText());
+    } else if (ctx instanceof Parser.SoqlTimeLiteralContext) {
+      return new Impl.LiteralImpl(Soql.LiteralType.Date, ctx.getText());
+    } else if (ctx instanceof Parser.SoqlDateFormulaLiteralContext) {
+      return new Impl.LiteralImpl(Soql.LiteralType.Date, ctx.getText());
+    } else if (ctx instanceof Parser.SoqlNumberLiteralContext) {
+      return new Impl.LiteralImpl(Soql.LiteralType.Number, ctx.getText());
+    } else if (ctx instanceof Parser.SoqlNullLiteralContext) {
+      return new Impl.LiteralImpl(Soql.LiteralType.Null, ctx.getText());
+    } else if (ctx instanceof Parser.SoqlBooleanLiteralContext) {
+      return new Impl.LiteralImpl(Soql.LiteralType.Boolean, ctx.getText());
+    } else if (ctx instanceof Parser.SoqlMultiCurrencyContext) {
+      return new Impl.LiteralImpl(Soql.LiteralType.Currency, ctx.getText());
+    }
+    return new Impl.LiteralImpl(Soql.LiteralType.String, ctx.getText());
+  }


Literals are stored as a type (date, number, null, boolean, currency, string) and a value. The value is the string value of the token. So a literal of type string has a value that includes the quotes. Storing the literal values this way, as opposed to storing as a Javascript version of the type, avoids any messy type transformations.

jgellin-sf · 2020-11-11T20:16:46Z

The latest commit make IN and INCLUDES conditions unmodeled syntax removes support for these conditions.

jgellin-sf · 2020-11-11T21:17:29Z

The commit implement condition simplicity utils adds utilities to determine whether a condition can be expressed as a flat list of AND's ("All") or OR's ("Any").

…into jg/where

dobladez · 2020-11-12T14:46:35Z

Looks good to me 👍

For future PRs: Given that model is growing, Have you thought of providing "builders" to make it easier to build model structures? (for unit tests and for the UI editor). Something like:

 QueryBuilder.select(["a", "b"])
             .from("Account")
             .where( andExpr(fieldExpr("name", EQUAL, "John"),
                             fieldExpr("deleted", EQUAL, false))).build();

* model WHERE clause (#68) * Remote SOQL Errors - Language Server Part (#93) * remove code builder (#92) * Move from antlr4JS to antlr4TS (#96) * SOQL code completion for SELECT fields (#99) * publish language server 0.2.9 (#102) * Fix LSP dependency (#104) * Disabled 'Running...' button while query running (#106) * Add basic code-completion for ORDER BY and other improvements (#107) * Dehru and JG telemetry - Send GDPR clean data to telemetry when errors/unsupported syntax present (#110) * Disable WHERE in model (#113) * Bump versions for publishing (#114) * Code-completion for WHERE clause expressions (#112) Co-authored-by: Jonathan Gellin <jgellin@salesforce.com> Co-authored-by: jgellin-sf <55159130+jgellin-sf@users.noreply.github.com> Co-authored-by: Chase McCarthy <charles.mccarthy@heroku.com> Co-authored-by: Jonny Hork <jhork@salesforce.com> Co-authored-by: Fernando Dobladez <fernandodobladez@salesforce.com> Co-authored-by: Jonathan Gellin <jgellin@salesforce.com>

model WHERE clause

d3d784c

jgellin-sf requested review from amphro, code0100fun, dehru, jonnyhork, mysticflute, praksb and tmock12 as code owners November 11, 2020 14:50

jgellin-sf commented Nov 11, 2020

View reviewed changes

make IN and INCLUDES conditions unmodeled syntax

4e32d91

implement condition simplicity utils

18e19f3

Dehru Cromer added 2 commits November 11, 2020 15:27

added where complexity to script statement

63f5713

Merge branch 'jg/where' of https://github.com/forcedotcom/soql-tooling …

8bd8fb3

…into jg/where

dehru approved these changes Nov 18, 2020

View reviewed changes

dehru merged commit 65eb027 into develop Nov 19, 2020

tmock12 deleted the jg/where branch November 20, 2020 14:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

model WHERE clause #68

model WHERE clause #68

Uh oh!

jgellin-sf commented Nov 11, 2020

Uh oh!

codecov bot commented Nov 11, 2020 •

edited

Loading

Uh oh!

jgellin-sf Nov 11, 2020

Uh oh!

jgellin-sf Nov 11, 2020

Uh oh!

jonnyhork Nov 11, 2020

Uh oh!

jgellin-sf Nov 11, 2020

Uh oh!

jgellin-sf Nov 11, 2020

Uh oh!

jgellin-sf Nov 11, 2020

Uh oh!

jgellin-sf commented Nov 11, 2020

Uh oh!

jgellin-sf commented Nov 11, 2020

Uh oh!

dobladez commented Nov 12, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

model WHERE clause #68

model WHERE clause #68

Uh oh!

Conversation

jgellin-sf commented Nov 11, 2020

What does this PR do?

What issues does this PR fix or reference?

Uh oh!

codecov bot commented Nov 11, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

jgellin-sf Nov 11, 2020

Choose a reason for hiding this comment

Uh oh!

jgellin-sf Nov 11, 2020

Choose a reason for hiding this comment

Uh oh!

jonnyhork Nov 11, 2020

Choose a reason for hiding this comment

Uh oh!

jgellin-sf Nov 11, 2020

Choose a reason for hiding this comment

Uh oh!

jgellin-sf Nov 11, 2020

Choose a reason for hiding this comment

Uh oh!

jgellin-sf Nov 11, 2020

Choose a reason for hiding this comment

Uh oh!

jgellin-sf commented Nov 11, 2020

Uh oh!

jgellin-sf commented Nov 11, 2020

Uh oh!

dobladez commented Nov 12, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov bot commented Nov 11, 2020 •

edited

Loading