Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .prettierrc
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
{
"printWidth": 120,
"parser": "typescript",
"semi": true,
"tabWidth": 2,
"useTabs": false,
Expand Down
17 changes: 14 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,35 @@
# Changelog

## 1.1.0

- Updated `Contributing.md` with more detailed instructions on grammar updates
- Added support for `WITH SECURITY_ENFORCED` (#61)

## 1.0.2
- If a field in a query happened to have a function reserved word, such as `Format`, then parsing the query failed. (#59)

- If a field in a query happened to have a function reserved word, such as `Format`, then parsing the query failed. (#59)

## 1.0.1

- Ensured that nothing is logged directly to the console unless logging is enabled

## 1.0.0

### Changed

**!BREAKING CHANGES!**

- Added literal type information to fields to provide additional information about the field type. (#51)
- WHERE clause fields have one of the following types `'STRING' | 'INTEGER' | 'DECIMAL' | 'BOOLEAN' | 'NULL' | 'DATE_LITERAL' | 'DATE_N_LITERAL';` stored in the condition.
- For date literal fields that have variables, `dateLiteralVariable` will be populated with the value
- For date literal fields that have variables, `dateLiteralVariable` will be populated with the value
- Modified Field data structure to have explicit type information. (#46, #52)
- The data structure for fields has been modified to include specific information about the structure of a given field to ease making sense of a parsed query,
- To aid in creating compose fields, a new helper method is available - `getComposedField()`. This takes in a simple data structure (or even a string) and will return the structure needed to compose a query.

### New

- An additional `queryUtils` object is available with the following functions:
- `function getComposedField(input: string | ComposeFieldInput): FieldType`
- `function getFlattenedFields(query: Query, isAggregateResult?: boolean): string[]`
- `function isSubquery(query: Query | Subquery): query is Subquery`
- Look at the README and refer to the unit tests for example usage.
- Look at the README and refer to the unit tests for example usage.
45 changes: 39 additions & 6 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,46 @@
# How to contribute

Contributions of any kind are welcome and appreciated.

# What features are allowed

Any feature that is part of the core project and does not deviate from what the product is will be considered t obe accepted.

# Making Changes
* For the repository
* Create a new branch from master (usually) and with a meaningful name for your changes
* Make you changes
* Commit and push your change
* Please run `prettier` on all modified files prior to opening your pull request
* open a Pull Request for the master branch

- For the repository
- Create a new branch from master (usually) and with a meaningful name for your changes
- Make you changes
- Commit and push your change
- Please run `prettier` on all modified files prior to opening your pull request
- open a Pull Request for the master branch

# Re-generating parse after grammar change

:beetle: There is a bug in the generated ANTLR output that must be corrected by hand in the generated output to make this work.

- Note: I am not an expert with language parsing/antler and I did not create the grammar, so I was not able to figure this bug out. I believe the problem relies with `antlr4ts-cli` and the way imports are always added to the end of the file

## Generate lexer and parser

- Run `npm run antlr`
- This will remove and re-generate all of the lexer and parser files required to parse SOQL
- :beetle: Manual steps:
- Open `SOQLParser.ts` after the file generation
- Find the class `Keywords_alias_allowedContext` somewhere near line ~44K
- Cut this class and all remaining classes through the end of the file and paste right underneath the `import` statements
- Run all unit tests afterwards `npm run test` - if anything is broken, please troubleshoot or create a PR and ask for assistance

## Update code to support new grammar

- Start with adding a new unit test
- This will ensure the parser and composer support the syntax
- Update the parser to listen for the new grammar (this depends on grammar change that was implemented)
- Assuming there is a new clause somewhere, look in `lib/generated/SOQLListener.ts` to find the new enter/exit methods that were added
- Implement these methods in `lib/SOQLListener.ts` to handle parsing the logic
- Update `lib/SOQLComposer.ts` to handle creating the SOQL query from the parsed query

## Unit tests

- ensure your new unit tests cover all the cases
- Ensure unit tests are all passing, or open a PR and ask for assistance
4 changes: 2 additions & 2 deletions CONTRIBUTORS.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
* Austin Turner @paustint
* Alfredo Benassi @ABenassi87
- Austin Turner @paustint
- Alfredo Benassi @ABenassi87
69 changes: 48 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,30 +5,37 @@
[![dependencies](https://david-dm.org/paustint/soql-parser-js.svg)](https://david-dm.org/paustint/soql-parser-js)

## Description

SOQL Parser JS provides the following capabilities:

1. Parse a SOQL query into a usable data structure.
2. Turn a parsed query data structure back into well a SOQL query with various format options.
3. Check if a SOQL query is syntactically valid (**note**: some cases may be structurally sound but not allowed by SFDC).

This library is written in Typescript and all type definitions are included with the library for your benefit if you choose to use Typescript or use VSCode's automatic type checking.

*Warning*: antlr4 is dependency for this library and is a rather large library (~600 KB) and is required for the parser to function, use in the browser with care.
_Warning_: antlr4 is dependency for this library and is a rather large library (~600 KB) and is required for the parser to function, use in the browser with care.

## Examples

For an example of the parser, check out the [example application](https://paustint.github.io/soql-parser-js/).

Have a look through the unit tests for many more examples.

# Usage

## Parsing

Parsing a SOQL query can be completed by calling `parseQuery(soqlQueryString, options)` and a `Query` data structure will be returned;

#### Typescript / ES6

```typescript
import { parseQuery } from 'soql-parser-js'; // TS / ES6 imports
// var soqlParserJs = require('soql-parser-js'); // node's require format - usage: soqlParserJs.parseQuery()

const soql = 'SELECT UserId, COUNT(Id) from LoginHistory WHERE LoginTime > 2010-09-20T22:16:30.000Z AND LoginTime < 2010-09-21T22:16:30.000Z GROUP BY UserId';
const soql =
'SELECT UserId, COUNT(Id) from LoginHistory WHERE LoginTime > 2010-09-20T22:16:30.000Z AND LoginTime < 2010-09-21T22:16:30.000Z GROUP BY UserId';

const soqlQuery = parseQuery(soql);
// const soqlQuery = soqlParserJs.parseQuery(soql); // using require()
Expand All @@ -37,6 +44,7 @@ console.log(JSON.stringify(soqlQuery, null, 2));
```

#### Options

```typescript
export interface SoqlQueryConfig {
continueIfErrors?: boolean; // default=false
Expand All @@ -45,16 +53,17 @@ export interface SoqlQueryConfig {
```

## Composing

Composing a query will turn a Query object back to a SOQL query. The exact same data structure returned from `parseQuery()` can be used,
but there are many use-cases where you may need to build your own data structure to compose a query.
These examples show building your own Query object with the minimum required fields.

**Note:** For some operators, they may be converted to upper case (e.x. NOT, AND)

**Note:** There are a number of fields populated on the Query object when `parseQuery()` is called that are not required to compose a query. Look at the examples below and the comments in the data model for more information.

**Note:** There are a number of fields populated on the Query object when `parseQuery()` is called that are not required to compose a query. Look at the examples below and the comments in the data model for more information.

**The base query object is shaped like this:**

```typescript
export interface QueryBase {
fields: FieldType[];
Expand All @@ -74,7 +83,9 @@ export interface QueryBase {
The easiest way to build the fields is to call the utility function `getComposedField()`.

### Example

This is the query that will be composed

```sql
SELECT Id, Name, FORMAT(Amount) MyFormattedAmount,
(SELECT Quantity, ListPrice, PricebookEntry.UnitPrice, PricebookEntry.Name FROM OpportunityLineItems)
Expand Down Expand Up @@ -139,11 +150,11 @@ const soqlQuery = {
const composedQuery = composeQuery(soqlQuery, { format: true });

console.log(composedQuery);

```

In the above examples, we made use of `getComposedField(input: string | ComposeFieldInput)` to help easily compose the fields. The input expects a string or one of the following shapes of data below. An error will be thrown if the data passed in is not one of the following shapes:
In the above examples, we made use of `getComposedField(input: string | ComposeFieldInput)` to help easily compose the fields. The input expects a string or one of the following shapes of data below. An error will be thrown if the data passed in is not one of the following shapes:
and will return a `FieldType` object.

```typescript
export interface ComposeField {
field: string;
Expand Down Expand Up @@ -173,10 +184,12 @@ export interface ComposeFieldTypeof {
```

## Checking if a Query is Valid

This will parse the AST tree to confirm the syntax is valid, but will not parse the tree into a data structure.
This method is faster than parsing the full query.

Options:

```typescript
export interface ConfigBase {
logging: boolean; // default=false
Expand All @@ -186,26 +199,29 @@ export interface ConfigBase {
```typescript
import { isQueryValid } from 'soql-parser-js';

const soql = 'SELECT UserId, COUNT(Id) from LoginHistory WHERE LoginTime > 2010-09-20T22:16:30.000Z AND LoginTime < 2010-09-21T22:16:30.000Z GROUP BY UserId';
const soql =
'SELECT UserId, COUNT(Id) from LoginHistory WHERE LoginTime > 2010-09-20T22:16:30.000Z AND LoginTime < 2010-09-21T22:16:30.000Z GROUP BY UserId';

const isValid = isQueryValid(soql);

console.log('isValid', isValid);

```

#### Node

```javascript
var soqlParserJs = require('soql-parser-js');

const soql = 'SELECT UserId, COUNT(Id) from LoginHistory WHERE LoginTime > 2010-09-20T22:16:30.000Z AND LoginTime < 2010-09-21T22:16:30.000Z GROUP BY UserId';
const soql =
'SELECT UserId, COUNT(Id) from LoginHistory WHERE LoginTime > 2010-09-20T22:16:30.000Z AND LoginTime < 2010-09-21T22:16:30.000Z GROUP BY UserId';

const isValid = isQueryValid(soql);

console.log('isValid', isValid);
```

## Format Query

This function is provided as a convenience and just calls parse and compose under the hood.

```typescript
Expand All @@ -214,9 +230,12 @@ import { formatQuery } from 'soql-parser-js';
const query = `SELECT Id, Name, AccountNumber, AccountSource, AnnualRevenue, BillingAddress, BillingCity, BillingCountry, BillingGeocodeAccuracy, ShippingStreet, Sic, SicDesc, Site, SystemModstamp, TickerSymbol, Type, Website, (SELECT Id, Name, AccountId, Amount, CampaignId, CloseDate, CreatedById, Type FROM Opportunities), (SELECT Id, Name, AccountNumber, AccountSource, AnnualRevenue, BillingAddress, Website FROM ChildAccounts) FROM Account WHERE Name LIKE 'a%' OR Name LIKE 'b%' OR Name LIKE 'c%'`;

const formattedQuery1 = formatQuery(query);
const formattedQuery2 = formatQuery(query, { fieldMaxLineLen: 20, fieldSubqueryParensOnOwnLine: false, whereClauseOperatorsIndented: true });
const formattedQuery2 = formatQuery(query, {
fieldMaxLineLen: 20,
fieldSubqueryParensOnOwnLine: false,
whereClauseOperatorsIndented: true,
});
const formattedQuery3 = formatQuery(query, { fieldSubqueryParensOnOwnLine: true, whereClauseOperatorsIndented: true });

```

```sql
Expand Down Expand Up @@ -310,18 +329,22 @@ export interface FormatOptions {
```

## Utility Functions

The following utility functions are available:
1. `getComposedField(input: string | ComposeFieldInput)`
1. Convenience method to construct fields in the correct data format. See example usage in the Compose example.
2. `isSubquery(query: Query | Subquery)`
1. Returns true if the data passed in is a subquery
3. `getFlattenedFields(query: Query)`
1. This provides a list of fields that are stringified and flattened in order to access data from a returned API call from Salesforce. Refer to `tests/publicUtils.spec.ts` for usage examples.

1. `getComposedField(input: string | ComposeFieldInput)`
1. Convenience method to construct fields in the correct data format. See example usage in the Compose example.
1. `isSubquery(query: Query | Subquery)`
1. Returns true if the data passed in is a subquery
1. `getFlattenedFields(query: Query)`
1. This provides a list of fields that are stringified and flattened in order to access data from a returned API call from Salesforce. Refer to `tests/publicUtils.spec.ts` for usage examples.

## Data Models

### Query

These are all available for import in your typescript projects

```typescript
export type LogicalOperator = 'AND' | 'OR';
export type Operator = '=' | '!=' | '<=' | '>=' | '>' | '<' | 'LIKE' | 'IN' | 'NOT IN' | 'INCLUDES' | 'EXCLUDES';
Expand Down Expand Up @@ -473,11 +496,10 @@ export interface WithDataCategoryCondition {
selector: GroupSelector;
parameters: string[];
}

```


### Utils

```typescript
export interface ComposeField {
field: string;
Expand Down Expand Up @@ -514,9 +536,11 @@ export type ComposeFieldInput =
```

## CLI Usage

The CLI can be used to parse a query or compose a previously parsed query back to SOQL.

**Examples:**

```shell
$ npm install -g soql-parser-js
$ soql --help
Expand All @@ -531,6 +555,7 @@ $ soql -compose some-input-file.json -output some-output-file.json
```

**Arguments:**

```
--query, -q A SOQL query surrounded in quotes or a file path to a text file containing a SOQL query.
--compose, -c An escaped and quoted parsed SOQL JSON string or a file path to a text file containing a parsed query JSON object.
Expand All @@ -541,8 +566,10 @@ $ soql -compose some-input-file.json -output some-output-file.json
```

## Contributing

All contributions are welcome on the project. Please read the [contribution guidelines](https://github.com/paustint/soql-parser-js/blob/master/CONTRIBUTING.md).

## Special Thanks
* This library is based on the ANTLR4 grammar file [produced by Mulesoft](https://github.com/mulesoft/salesforce-soql-parser/blob/antlr4/SOQL.g4).
* The following repository also was a help to get things started: https://github.com/petermetz/antlr-4-ts-test

- This library is based on the ANTLR4 grammar file [produced by Mulesoft](https://github.com/mulesoft/salesforce-soql-parser/blob/antlr4/SOQL.g4).
- The following repository also was a help to get things started: https://github.com/petermetz/antlr-4-ts-test
Loading