Support Lambda and add related array functions

[qianheng-aws]

Description

• Support parsing lambda expression.

• Add below array related function which need a lambda function as its argument:

  • json_array_any_match() -> exists(ARRAY, Lambda)
  • json_array_all_match() -> forAll(ARRAY, Lambda)
  • json_array_filter() -> filter(ARRAY, Lambda)
  • json_array_map() -> transform(ARRAY, Lambda)
  • json_array_reduce() -> reduce(ARRAY, start, Lambda, Lambda)

Examples:
Exists:

... | eval a = json_array(1, -1, 2), b = exists(a, x -> x > 0) | head 1 | fields b

# return True

ForAll:

... | eval a = json_array(1, -1, 2), b = forall(a, x -> x > 0) | head 1 | fields b

# return False

Filter:

... | eval a = json_array(1, -1, 2), b = filter(a, x -> x > 0) | head 1 | fields b

# return `[1, 2]`

Transform:

... | eval a = json_array(1, 2, 3), b = transform(a, x -> x + 1) | head 1 | fields b

# return `[2, 3, 4]`

... | eval a = json_array(1, 2, 3), b = transform(a, (x + i) -> x + i) | head 1 | fields b

# return `[1, 3, 5]`

Reduce:

... | eval a = json_array(1, 2, 3), b = reduce(a, 0, (acc, x) -> acc + x) | head 1 | fields b

# return `6`

... | eval a = json_array(1, 2, 3), b = reduce(a, 10, (acc, x) -> acc + x) | head 1 | fields b

# return `16`

... | eval a = json_array(1, 2, 3), b = reduce(a, 0, (acc, x) -> acc + x, acc -> acc * 10) | head 1 | fields b

# return `60`

Related Issues

Partial resolve #863

Check List

• Updated documentation (docs/ppl-lang/README.md)
• Implemented unit tests
• Implemented tests for combination with other commands
• New added source code should include a copyright header
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[LantaoJin]

Need a test for a = json_array(json_object(..), json_object(..), json_object(..)).
For example:

a = [
  {id:8, uid:1},
  {id:7, uid:9},
  {id:7, uid:1},
  ...
]

[qianheng-aws]

Added in IT

[LantaoJin]

a = [
  {id:8, uid:1},
  {id:7, uid:9},
  {id:7, uid:1},
  ...
]

Could you add this json structure example to user doc ppl-lambda.md? it could help user to understand the case.

(PS: please keep review conversations open, it could help reviewers to remember the context and join conversation)

[LantaoJin]

Can we accept both ARRAY and JSON ARRAY STRING?

[LantaoJin]

Em, I think we can keep this lambda functions and remove the JSON_ARRAY_ prefix since they seem common functions.

[qianheng-aws]

Changed to use the original function name as spark

[LantaoJin]

Thanks @qianheng-aws , I think we could separate the lambda functions and json functions (from tests to documentations). Another thought is will we add json_array_xx(jsonString, lambdaFunction) for the original request?

[LantaoJin]

Another thought is will we add json_array_xx(jsonString, lambdaFunction) for the original request?

Discussed offline. Let's enhance the json_array() instead of adding json_array_xx for now. As an example:

| eval a = json_array("[1, -1, 2]"), b = filter(a, x -> x > 0)

[LantaoJin]

@YANG-DB please review this PR and its description, the function names are not exactly same with the original ones in original issue. please double confirm that's fine to you.

[LantaoJin]

To avoid conflicts with #870, I merged this first. @YANG-DB please provide your review and feedback, we still could discuss and address them later.