Update PPL Command Documentation #4562

ritvibhatt · 2025-10-15T01:24:18Z

This PR standardizes the structure and content of PPL command documentation files to improve consistency and user experience.

Changes Made

Documentation Structure Standardization

Implemented consistent section ordering across all PPL command files:

Description - Clear explanation of command functionality
Syntax - Command syntax with parameter details
Optional sections - Behavior notes, configuration details, or usage guidance as needed
Examples - Practical usage examples with expected output
Limitations - Known constraints or considerations

Content Cleanup and Modernization

Removed version information - Eliminated outdated version references (e.g., "Version:
3.3.0") to keep documentation current and reduce maintenance overhead
Extracted aggregation functions - Moved aggregation function documentation to dedicated
functions file (/functions/aggregations.rst) for better organization

Related Issues

Resolves #[Issue number to be closed when this PR is merged]
#4220

Check List

New functionality includes testing.
New functionality has been documented.
New functionality has javadoc added.
New functionality has a user manual doc added.
New PPL command checklist all confirmed.
API changes companion pull request created.
Commits are signed per the DCO using --signoff or -s.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com>

Signed-off-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com>

Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com>

RyanL1997

Hi @ritvibhatt , thanks for the change. And I can signoff for rex, regex, and parse. And I just left some minor comments.

RyanL1997 · 2025-10-15T23:46:55Z

docs/user/ppl/cmd/rex.rst

 ================================== ============ ============


 Limitations


I saw that in the regex.rst, there are some formatting changes in the "Limitation" section, and I was wondering do we need to apply the same here?

I think the format is good, I removed the first sentence in limitations but let me know if you think format can be changed to match regex better

LantaoJin · 2025-10-16T03:10:15Z

docs/user/ppl/cmd/append.rst

 ============
-| Using ``append`` command to append the result of a sub-search and attach it as additional rows to the bottom of the input search results (The main search).
+| The ``append`` command appends the result of a sub-search and attaches it as additional rows to the bottom of the input search results (The main search).
 The command aligns columns with the same field names and types. For different column fields between the main search and sub-search, NULL values are filled in the respective rows.


Need a | before head of this line.

LantaoJin · 2025-10-16T03:15:17Z

docs/user/ppl/cmd/appendcol.rst

-Version
-=======
-3.1.0
+| The ``appendcol`` command appends the result of a sub-search and attaches it alongside with the input search results (The main search).


no need to keep the | any more if there is single line in description. | means starting from a new line

Removed all the | where there was only a single line thanks!

LantaoJin · 2025-10-16T03:15:24Z

docs/user/ppl/cmd/dedup.rst

 ============
-| Using ``dedup`` command to remove identical document defined by field from the search result.
-
+| The ``dedup`` command removes duplicate documents defined by specified fields from the search result.


LantaoJin · 2025-10-16T03:19:31Z

docs/user/ppl/cmd/describe.rst

 Syntax
 ============
-describe <dataSource>.<schema>.<tablename>
+describe [dataSource].[schema].<tablename>


should be describe [dataSource.][schema.]<tablename>

LantaoJin · 2025-10-16T03:23:56Z

docs/user/ppl/cmd/expand.rst

+1. It generates one row per element in the specified array field.
+2. The specified array field is converted into individual rows.
+3. If an alias is provided, the expanded values appear under the alias instead of the original field name.
+4. If the specified field is an empty array, the row is retained with the expanded field set to null.


why not bullet mark here?

You're right bullet points make more sense here, updated

LantaoJin · 2025-10-16T03:27:31Z

docs/user/ppl/cmd/fillnull.rst

+======

 fillnull with <replacement> [in <field-list>]
-


if you delete new line, please add | at the beginning of each line.

LantaoJin · 2025-10-16T03:27:33Z

docs/user/ppl/cmd/fillnull.rst

 fillnull with <replacement> [in <field-list>]
-
 fillnull using <field> = <replacement> [, <field> = <replacement>]
-


LantaoJin · 2025-10-16T03:27:37Z

docs/user/ppl/cmd/fillnull.rst


 Example 1: Replace null values with a specified value on one field
-------------------------------------------------------------------
+===================================================================


delete last =

Updated all of the = and - underlines to match

LantaoJin · 2025-10-16T03:27:43Z

docs/user/ppl/cmd/fillnull.rst


 Example 2: Replace null values with a specified value on multiple fields
-------------------------------------------------------------------------
+=========================================================================


ditto and following

LantaoJin · 2025-10-16T03:33:37Z

docs/user/ppl/cmd/join.rst


-For extended syntax (join with field list) in 3.3.0, when duplicate-named fields in output results are deduplicated, the fields in output determined by the value of 'overwrite' option.
-
-Since 3.3.0, join types ``inner``, ``left``, ``outer`` (alias of ``left``), ``semi`` and ``anti`` are supported by default. ``right``, ``full``, ``cross`` are performance sensitive join types which are disabled by default. Set config ``plugins.calcite.all_join_types.allowed = true`` to enable.


this information (how to enable full capability of join types) totally lost

That was mistakenly deleted thanks for catching that! Added back

LantaoJin · 2025-10-16T06:23:11Z

docs/user/ppl/cmd/lookup.rst

-Example 3: no inputField
-========================
+Example 3: No inputField specified
+====================================


delete last two =

LantaoJin · 2025-10-16T06:23:27Z

docs/user/ppl/cmd/lookup.rst

-Example 4: outputField as a new field
-=====================================
+Example 4: OutputField as a new field
+======================================


delete last one =

LantaoJin · 2025-10-16T06:25:37Z

docs/user/ppl/cmd/patterns.rst

-Change default pattern method
-============
+Change the default pattern method
+=============================


LantaoJin · 2025-10-16T06:25:48Z

docs/user/ppl/cmd/patterns.rst

    +-----------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------+

 Simple Pattern Example 3: Extract log patterns with custom regex pattern
 =========================================================


add more =

LantaoJin · 2025-10-16T06:26:31Z

docs/user/ppl/cmd/patterns.rst

 Configuration
 -------------
-With Calcite specific option ``show_numbered_token`` enabled, the output can detect numbered variable tokens from the pattern field.
+With  option ``show_numbered_token`` enabled, the output can detect numbered variable tokens from the pattern field.


delete one space between With and option

LantaoJin · 2025-10-16T06:29:49Z

docs/user/ppl/cmd/sort.rst


-Example 8: Sort with specifying field type
+Example 9: Sort with specifying field type
 ==================================


LantaoJin · 2025-10-16T06:30:05Z

docs/user/ppl/cmd/spath.rst

    +----------+---+

 Example 2: Lists & Nesting
 ============================


= problem

LantaoJin · 2025-10-16T06:31:10Z

docs/user/ppl/cmd/stats.rst

-        "plugins.calcite.enabled" : true
-      }
-    }'
+* aggregation: mandatory. An aggregation function. The argument of aggregation must be field.


The argument of aggregation must be field. can be removed

LantaoJin · 2025-10-16T06:31:48Z

docs/user/ppl/cmd/subquery.rst

 =============
-subquery (aka subsearch)
+subquery
 =============


= problem

LantaoJin · 2025-10-16T06:34:25Z

docs/user/ppl/cmd/timechart.rst

-3.3.0
-
 Syntax
 ============


= problem

Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com>

Signed-off-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com>

Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com>

aalva500-prog · 2025-10-17T00:18:38Z

Hi @ritvibhatt, thanks for the changes! Quick question: should we also update the documentation in condition.rst for recently added functions like coalesce?

ritvibhatt added 3 commits October 13, 2025 11:43

remove version info

8445ebb

Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com>

fix defaults

81e2da1

Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com>

update main docs and move aggregate functions

3ea70a5

Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com>

ritvibhatt marked this pull request as ready for review October 15, 2025 04:04

Merge branch 'main' into update-docs

0dc4a62

Signed-off-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com>

ritvibhatt marked this pull request as draft October 15, 2025 04:13

fix format

3c67613

Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com>

ritvibhatt changed the title ~~Update Documentation~~ Update PPL Command Documentation Oct 15, 2025

ritvibhatt marked this pull request as ready for review October 15, 2025 15:54

RyanL1997 reviewed Oct 15, 2025

View reviewed changes

LantaoJin reviewed Oct 16, 2025

View reviewed changes

LantaoJin added the documentation Improvements or additions to documentation label Oct 16, 2025

LantaoJin reviewed Oct 16, 2025

View reviewed changes

ritvibhatt and others added 6 commits October 16, 2025 11:11

fix heading underlines

7dd7db9

Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com>

fix typos/content

66a3e97

Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com>

Merge branch 'main' into update-docs

06f0f8d

Signed-off-by: ritvibhatt <53196324+ritvibhatt@users.noreply.github.com>

fix formatting

657f38e

Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com>

update rex limitations

055030a

Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com>

update underlines and bullet points

a422a2d

Signed-off-by: Ritvi Bhatt <ribhatt@amazon.com>

		================================== ============ ============


		Limitations

		fillnull with <replacement> [in <field-list>]

		fillnull using <field> = <replacement> [, <field> = <replacement>]


		For extended syntax (join with field list) in 3.3.0, when duplicate-named fields in output results are deduplicated, the fields in output determined by the value of 'overwrite' option.

		Since 3.3.0, join types ``inner``, ``left``, ``outer`` (alias of ``left``), ``semi`` and ``anti`` are supported by default. ``right``, ``full``, ``cross`` are performance sensitive join types which are disabled by default. Set config ``plugins.calcite.all_join_types.allowed = true`` to enable.

Update PPL Command Documentation #4562

Are you sure you want to change the base?

Update PPL Command Documentation #4562

Conversation

ritvibhatt commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes Made

Documentation Structure Standardization

Content Cleanup and Modernization

Related Issues

Check List

Uh oh!

RyanL1997 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aalva500-prog commented Oct 17, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ritvibhatt commented Oct 15, 2025 •

edited

Loading