-
Notifications
You must be signed in to change notification settings - Fork 50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: Parse expression with NBSP character #914
Conversation
9bb774a
to
bacc0d8
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @saig0 🚀 Please have a look a my comment, otherwise all good!
// FEEL-specific whitespace characters from the DMN specification | ||
List( | ||
'\u0009', '\u0020', '\u0085', '\u00A0', '\u1680', '\u180E', '\u2000', '\u2001', '\u2002', | ||
'\u2003', '\u2004', '\u2005', '\u2006', '\u2007', '\u2008', '\u2009', '\u200A', '\u200B', | ||
'\u2028', '\u2029', '\u202F', '\u205F', '\u3000', '\uFEFF', '\u000A', '\u000B', '\u000C', | ||
'\u000D' | ||
).foreach { character => | ||
it should f"be parsed (\\u${character.toInt}%04X)" in { | ||
evaluateExpression(s"${character}1") should returnResult(1) | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔧 Have you considered putting this in a parameterized test and running all the scenarios in this class for all whitespace characters?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great idea. 💡
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After migrating to a parameterized test, I found glitches with some whitespace characters. 👍
Add new test cases to verify that a NBSP and other space characters can be parsed.
Adjust the whitespace handling of the parser to ignore FEEL-specific whitespaces as defined in the DMN specification. The JavaWhitespace class worked very well because it supported also Java-like comments, as required by the DMN specification. Instead of adding additional whitespace rules for the individual parsers, extend the JavaWhitespace class by making a copy of it. This is less error-prone, easier to maintain, and more efficient.
Some whitespace characters were treated as part of the list iteration variable name. As a result, the parsing failed because the keyword "in" was not found. Fix the parsing issue by excluding whitespace characters as part of the variable name.
206ec2b
to
08ef5a4
Compare
Successfully created backport PR for |
Successfully created backport PR for |
#21884) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [org.camunda.feel:feel-engine](http://www.camunda.org) ([source](https://redirect.github.com/camunda/feel-scala)) | `1.18.0-alpha1` -> `1.18.0` | [![age](https://developer.mend.io/api/mc/badges/age/maven/org.camunda.feel:feel-engine/1.18.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/maven/org.camunda.feel:feel-engine/1.18.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/maven/org.camunda.feel:feel-engine/1.18.0-alpha1/1.18.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/maven/org.camunda.feel:feel-engine/1.18.0-alpha1/1.18.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>camunda/feel-scala (org.camunda.feel:feel-engine)</summary> ### [`v1.18.0`](https://redirect.github.com/camunda/feel-scala/releases/tag/1.18.0) [Compare Source](https://redirect.github.com/camunda/feel-scala/compare/1.18.0-alpha1...1.18.0) #### What's Changed ##### Features ✨ - feat: Add built-in function to strip whitespace by [@​skayliu](https://redirect.github.com/skayliu) in [https://github.com/camunda/feel-scala/pull/826](https://redirect.github.com/camunda/feel-scala/pull/826) - feat: Add built-in function to generate a UUID string by [@​skayliu](https://redirect.github.com/skayliu) in [https://github.com/camunda/feel-scala/pull/834](https://redirect.github.com/camunda/feel-scala/pull/834) - feat: Add built-in function to encode a string as base64 by [@​sbuettner](https://redirect.github.com/sbuettner) in [https://github.com/camunda/feel-scala/pull/847](https://redirect.github.com/camunda/feel-scala/pull/847) - feat: Add built-in function to check if a list is empty by [@​skayliu](https://redirect.github.com/skayliu) in [https://github.com/camunda/feel-scala/pull/851](https://redirect.github.com/camunda/feel-scala/pull/851) - feat: Make the FeelEngineBuilder available from Java/Kotlin by [@​vicmosin](https://redirect.github.com/vicmosin) in [https://github.com/camunda/feel-scala/pull/832](https://redirect.github.com/camunda/feel-scala/pull/832) - feat: I can interrupt an expression evaluation by [@​vicmosin](https://redirect.github.com/vicmosin) in [https://github.com/camunda/feel-scala/pull/824](https://redirect.github.com/camunda/feel-scala/pull/824) ##### Fixes 🐛 - fix: Regression in string() function for string and null values by [@​saig0](https://redirect.github.com/saig0) in [https://github.com/camunda/feel-scala/pull/749](https://redirect.github.com/camunda/feel-scala/pull/749) - fix: String escape characters by [@​nicpuppa](https://redirect.github.com/nicpuppa) in [https://github.com/camunda/feel-scala/pull/750](https://redirect.github.com/camunda/feel-scala/pull/750) - fix: Double quotes in string literal are escaped by [@​saig0](https://redirect.github.com/saig0) in [https://github.com/camunda/feel-scala/pull/780](https://redirect.github.com/camunda/feel-scala/pull/780) - fix: Variable resolution for custom context by [@​saig0](https://redirect.github.com/saig0) in [https://github.com/camunda/feel-scala/pull/786](https://redirect.github.com/camunda/feel-scala/pull/786) - fix: `string()` function can handle a context with custom value types by [@​mustafadagher](https://redirect.github.com/mustafadagher) in [https://github.com/camunda/feel-scala/pull/833](https://redirect.github.com/camunda/feel-scala/pull/833) - fix: Invoke `context()` function with a list containing a custom context by [@​saig0](https://redirect.github.com/saig0) in [https://github.com/camunda/feel-scala/pull/858](https://redirect.github.com/camunda/feel-scala/pull/858) - fix: `substring()` with exceeding length by [@​saig0](https://redirect.github.com/saig0) in [https://github.com/camunda/feel-scala/pull/893](https://redirect.github.com/camunda/feel-scala/pull/893) - fix: `number()` to return null if the given string is invalid by [@​saig0](https://redirect.github.com/saig0) in [https://github.com/camunda/feel-scala/pull/892](https://redirect.github.com/camunda/feel-scala/pull/892) - fix: Return `null` if a date is invalid by [@​saig0](https://redirect.github.com/saig0) in [https://github.com/camunda/feel-scala/pull/889](https://redirect.github.com/camunda/feel-scala/pull/889) - fix: Function `context put()` works for more than two keys by [@​saig0](https://redirect.github.com/saig0) in [https://github.com/camunda/feel-scala/pull/902](https://redirect.github.com/camunda/feel-scala/pull/902) - fix: Unary-test behavior with special input variable `?` by [@​saig0](https://redirect.github.com/saig0) in [https://github.com/camunda/feel-scala/pull/887](https://redirect.github.com/camunda/feel-scala/pull/887) - fix: List operation returns null if the value is not a list by [@​saig0](https://redirect.github.com/saig0) in [https://github.com/camunda/feel-scala/pull/895](https://redirect.github.com/camunda/feel-scala/pull/895) - fix: Variable name starting with "in" by [@​saig0](https://redirect.github.com/saig0) in [https://github.com/camunda/feel-scala/pull/909](https://redirect.github.com/camunda/feel-scala/pull/909) - fix: Parse expression with NBSP character by [@​saig0](https://redirect.github.com/saig0) in [https://github.com/camunda/feel-scala/pull/914](https://redirect.github.com/camunda/feel-scala/pull/914) - fix: Detect duplicated context values in `distinct values()` + `union()` + `duplicate values()` by [@​saig0](https://redirect.github.com/saig0) in [https://github.com/camunda/feel-scala/pull/908](https://redirect.github.com/camunda/feel-scala/pull/908) ##### Dependencies 🤖 - chore(deps): bump org.apache.maven.plugins:maven-shade-plugin from 3.5.0 to 3.5.1 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/733](https://redirect.github.com/camunda/feel-scala/pull/733) - build: Update mvn-scalafmt plugin by [@​saig0](https://redirect.github.com/saig0) in [https://github.com/camunda/feel-scala/pull/721](https://redirect.github.com/camunda/feel-scala/pull/721) - chore(deps-dev): bump version.log4j from 2.20.0 to 2.21.0 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/745](https://redirect.github.com/camunda/feel-scala/pull/745) - chore(deps): bump org.apache.maven.plugins:maven-surefire-plugin from 3.1.2 to 3.2.1 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/755](https://redirect.github.com/camunda/feel-scala/pull/755) - chore(deps-dev): bump version.log4j from 2.21.0 to 2.21.1 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/754](https://redirect.github.com/camunda/feel-scala/pull/754) - chore(deps): bump org.apache.maven.plugins:maven-surefire-plugin from 3.2.1 to 3.2.2 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/760](https://redirect.github.com/camunda/feel-scala/pull/760) - chore(deps-dev): bump version.log4j from 2.21.1 to 2.22.0 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/764](https://redirect.github.com/camunda/feel-scala/pull/764) - chore(deps): bump org.apache.maven.plugins:maven-project-info-reports-plugin from 3.4.5 to 3.5.0 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/766](https://redirect.github.com/camunda/feel-scala/pull/766) - chore(deps): bump org.apache.maven.plugins:maven-surefire-plugin from 3.2.2 to 3.2.3 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/775](https://redirect.github.com/camunda/feel-scala/pull/775) - chore(deps-dev): bump version.log4j from 2.22.0 to 2.22.1 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/787](https://redirect.github.com/camunda/feel-scala/pull/787) - chore(deps): bump org.apache.maven.plugins:maven-surefire-plugin from 3.2.3 to 3.2.5 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/793](https://redirect.github.com/camunda/feel-scala/pull/793) - chore(deps): bump actions/cache from 3 to 4 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/795](https://redirect.github.com/camunda/feel-scala/pull/795) - chore(deps): bump org.apache.maven.plugins:maven-shade-plugin from 3.5.1 to 3.5.2 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/812](https://redirect.github.com/camunda/feel-scala/pull/812) - chore(deps-dev): bump version.log4j from 2.22.1 to 2.23.0 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/811](https://redirect.github.com/camunda/feel-scala/pull/811) - chore(deps): bump org.scala-lang:scala-library from 2.13.12 to 2.13.13 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/813](https://redirect.github.com/camunda/feel-scala/pull/813) - chore(deps-dev): bump version.log4j from 2.23.0 to 2.23.1 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/816](https://redirect.github.com/camunda/feel-scala/pull/816) - chore(deps): bump org.apache.maven.plugins:maven-assembly-plugin from 3.6.0 to 3.7.0 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/817](https://redirect.github.com/camunda/feel-scala/pull/817) - chore(deps): bump org.apache.maven.plugins:maven-assembly-plugin from 3.7.0 to 3.7.1 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/819](https://redirect.github.com/camunda/feel-scala/pull/819) - chore(deps): bump com.lihaoyi:fastparse\_2.13 from 3.0.2 to 3.1.0 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/827](https://redirect.github.com/camunda/feel-scala/pull/827) - chore(deps): bump org.apache.maven.plugins:maven-jar-plugin from 3.3.0 to 3.4.0 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/828](https://redirect.github.com/camunda/feel-scala/pull/828) - chore(deps): bump org.apache.maven.plugins:maven-jar-plugin from 3.4.0 to 3.4.1 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/837](https://redirect.github.com/camunda/feel-scala/pull/837) - chore(deps): bump org.apache.maven.plugins:maven-shade-plugin from 3.5.2 to 3.5.3 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/842](https://redirect.github.com/camunda/feel-scala/pull/842) - chore(deps): bump net.alchim31.maven:scala-maven-plugin from 4.8.1 to 4.9.0 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/841](https://redirect.github.com/camunda/feel-scala/pull/841) - chore(deps): bump org.scala-lang:scala-library from 2.13.13 to 2.13.14 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/845](https://redirect.github.com/camunda/feel-scala/pull/845) - chore(deps): bump net.alchim31.maven:scala-maven-plugin from 4.9.0 to 4.9.1 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/849](https://redirect.github.com/camunda/feel-scala/pull/849) - chore(deps): bump org.apache.maven.plugins:maven-shade-plugin from 3.5.3 to 3.6.0 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/862](https://redirect.github.com/camunda/feel-scala/pull/862) - chore(deps): bump com.fasterxml.uuid:java-uuid-generator from 5.0.0 to 5.1.0 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/863](https://redirect.github.com/camunda/feel-scala/pull/863) - chore(deps): bump org.apache.maven.plugins:maven-project-info-reports-plugin from 3.5.0 to 3.6.0 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/868](https://redirect.github.com/camunda/feel-scala/pull/868) - chore(deps): bump org.apache.maven.plugins:maven-surefire-plugin from 3.2.5 to 3.3.0 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/866](https://redirect.github.com/camunda/feel-scala/pull/866) - chore(deps): bump org.apache.maven.plugins:maven-jar-plugin from 3.4.1 to 3.4.2 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/869](https://redirect.github.com/camunda/feel-scala/pull/869) - chore(deps): bump org.apache.maven.plugins:maven-project-info-reports-plugin from 3.6.0 to 3.6.1 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/871](https://redirect.github.com/camunda/feel-scala/pull/871) - chore(deps): bump net.alchim31.maven:scala-maven-plugin from 4.9.1 to 4.9.2 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/872](https://redirect.github.com/camunda/feel-scala/pull/872) - chore(deps): bump com.lihaoyi:fastparse\_2.13 from 3.1.0 to 3.1.1 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/875](https://redirect.github.com/camunda/feel-scala/pull/875) - chore(deps): bump org.apache.maven.plugins:maven-project-info-reports-plugin from 3.6.1 to 3.6.2 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/878](https://redirect.github.com/camunda/feel-scala/pull/878) - chore(deps): bump org.apache.maven.plugins:maven-surefire-plugin from 3.3.0 to 3.3.1 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/876](https://redirect.github.com/camunda/feel-scala/pull/876) - chore(deps): bump org.apache.maven.plugins:maven-site-plugin from 3.12.1 to 3.20.0 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/886](https://redirect.github.com/camunda/feel-scala/pull/886) - chore(deps): bump org.apache.maven.plugins:maven-surefire-plugin from 3.3.1 to 3.4.0 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/885](https://redirect.github.com/camunda/feel-scala/pull/885) - chore(deps): bump org.apache.maven.plugins:maven-project-info-reports-plugin from 3.6.2 to 3.7.0 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/894](https://redirect.github.com/camunda/feel-scala/pull/894) - chore(deps): bump org.apache.maven.plugins:maven-surefire-plugin from 3.4.0 to 3.5.0 by [@​dependabot](https://redirect.github.com/dependabot) in [https://github.com/camunda/feel-scala/pull/910](https://redirect.github.com/camunda/feel-scala/pull/910) #### New Contributors - [@​lzgabel](https://redirect.github.com/lzgabel) made their first contribution in [https://github.com/camunda/feel-scala/pull/734](https://redirect.github.com/camunda/feel-scala/pull/734) - [@​jonathanlukas](https://redirect.github.com/jonathanlukas) made their first contribution in [https://github.com/camunda/feel-scala/pull/794](https://redirect.github.com/camunda/feel-scala/pull/794) - [@​vicmosin](https://redirect.github.com/vicmosin) made their first contribution in [https://github.com/camunda/feel-scala/pull/824](https://redirect.github.com/camunda/feel-scala/pull/824) - [@​skayliu](https://redirect.github.com/skayliu) made their first contribution in [https://github.com/camunda/feel-scala/pull/826](https://redirect.github.com/camunda/feel-scala/pull/826) - [@​mustafadagher](https://redirect.github.com/mustafadagher) made their first contribution in [https://github.com/camunda/feel-scala/pull/833](https://redirect.github.com/camunda/feel-scala/pull/833) - [@​sbuettner](https://redirect.github.com/sbuettner) made their first contribution in [https://github.com/camunda/feel-scala/pull/847](https://redirect.github.com/camunda/feel-scala/pull/847) - [@​andromaqui](https://redirect.github.com/andromaqui) made their first contribution in [https://github.com/camunda/feel-scala/pull/870](https://redirect.github.com/camunda/feel-scala/pull/870) **Full Changelog**: camunda/feel-scala@1.17.0...1.18.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/camunda/camunda). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOC41OS4yIiwidXBkYXRlZEluVmVyIjoiMzguNTkuMiIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiYXV0b21lcmdlIl19-->
Description
Adjust the parser's whitespace handling to ignore FEEL-specific whitespace characters, such as NBSP (i.e. Non-breaking space /
\u00A0
), as defined in the DMN specification.The
JavaWhitespace
class worked very well because it also supported Java-like comments, as required by the DMN specification. Instead of adding additional whitespace rules for the individual parsers, extend theJavaWhitespace
class by making a copy of it. 🙈 This should be less error-prone, easier to maintain, and more efficient. 👼Related issues
closes #840