-
Notifications
You must be signed in to change notification settings - Fork 176
[Doc] Enable doctest with Calcite #4379
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Doc] Enable doctest with Calcite #4379
Conversation
Signed-off-by: Lantao Jin <ltjin@amazon.com>
Signed-off-by: Lantao Jin <ltjin@amazon.com>
Signed-off-by: Lantao Jin <ltjin@amazon.com>
Signed-off-by: Lantao Jin <ltjin@amazon.com>
Signed-off-by: Lantao Jin <ltjin@amazon.com>
Signed-off-by: Lantao Jin <ltjin@amazon.com>
Signed-off-by: Lantao Jin <ltjin@amazon.com>
Signed-off-by: Lantao Jin <ltjin@amazon.com>
| FLOOR(282474973688888.022) | FLOOR(9223372036854775807.022) | FLOOR(9223372036854775807.0000001) | | ||
|----------------------------+--------------------------------+------------------------------------| | ||
| 282474973688888 | 9223372036854775807 | 9223372036854775807 | | ||
| 282474973688888.0 | 9.223372036854776e+18 | 9.223372036854776e+18 | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For the double values which are out of the double's precision. Keep the behavior same as Java's Math.floor(double)
and Splunk's floor()
math function.
Signed-off-by: Lantao Jin <ltjin@amazon.com>
Signed-off-by: Lantao Jin <ltjin@amazon.com>
] as b | ||
| stats avg(salary) by span(age, 10) as age_span, b.country; | ||
fetched rows / total rows = 5/5 | ||
PPL> source = state_country as a | where country = 'USA' OR country = 'England' | left join ON a.name = b.name [ source = occupation | where salary > 0 | fields name, country, salary | sort salary | head 3 ] as b | stats avg(salary) by span(age, 10) as age_span, b.country; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not sure why this query failed in linux/doctest only. I cannot reproduce in local and IT. Will keep the PPL>
to avoid failure.
ppl_cmd.process("source = state_country as a | where country = 'USA' OR country = 'England' | left join ON a.name = b.name [ source = occupation | where salary > 0 | fields name, country, salary | sort salary | head 3 ] as b | stats avg(salary) by span(age, 10) as age_span, b.country")Expected:
fetched rows / total rows = 3/3
+-------------+----------+-----------+
| avg(salary) | age_span | b.country |
|-------------+----------+-----------|
| null | 40 | null |
| 70000.0 | 30 | USA |
| 100000.0 | 70 | England |
+-------------+----------+-----------+
Got:
fetched rows / total rows = 2/2
+-------------+----------+-----------+
| avg(salary) | age_span | b.country |
|-------------+----------+-----------|
| 70000.0 | 30 | USA |
| 100000.0 | 70 | England |
+-------------+----------+-----------+
----------------------------------------------------------------------
"user/ppl/cmd/grok.rst", | ||
"user/ppl/cmd/head.rst", | ||
"user/ppl/cmd/parse.rst", | ||
"user/ppl/cmd/patterns.rst", | ||
"user/ppl/cmd/rare.rst", | ||
"user/ppl/cmd/search.rst", | ||
"user/ppl/cmd/sort.rst", | ||
"user/ppl/cmd/syntax.rst", | ||
"user/ppl/cmd/trendline.rst", | ||
"user/ppl/cmd/top.rst", | ||
"user/ppl/cmd/where.rst", | ||
"user/ppl/cmd/join.rst", | ||
"user/ppl/cmd/lookup.rst", | ||
"user/ppl/cmd/subquery.rst", | ||
"user/ppl/general/identifiers.rst", | ||
"user/ppl/general/datatypes.rst", | ||
"user/ppl/functions/datetime.rst", | ||
"user/ppl/functions/expressions.rst", | ||
"user/ppl/functions/ip.rst", | ||
"user/ppl/functions/json.rst", | ||
"user/ppl/functions/math.rst", | ||
"user/ppl/functions/relevance.rst", | ||
"user/ppl/functions/string.rst" | ||
"user/ppl/cmd/patterns.rst" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see the #4380 closed. but grok, parse, patterns are still not able to 100% match the result of doctest in v2. @songkant-aws can you fix it or link an issue. Or it just needs to update the doc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reopen #4380
| 2024-07-01 00:00:00 | web-01 | 2 | | ||
| 2024-07-01 00:00:00 | web-02 | 2 | | ||
+---------------------+--------+-------+ | ||
os> source=events | timechart span=1h count() by host |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@selsong , when I change PPL>
to os>
in timechart.rst. I found many results cannot not match the expected results added in doc. timechart
command is a Calcite enabled only command, which means the results in this file shouldn't be impacted by the action of setting calcite enabled as default. Please check why the original expected results cannot match the actual ones. cc @penghuo
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add issue to track. #4409
|
||
PPL query:: | ||
|
||
PPL> source=events | timechart span=1m avg(cpu_usage) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The backport to
To backport manually, run these commands in your terminal: # Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/sql/backport-2.19-dev 2.19-dev
# Navigate to the new working tree
pushd ../.worktrees/sql/backport-2.19-dev
# Create a new branch
git switch --create backport/backport-4379-to-2.19-dev
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 9143a449f38bf08470ecc25826c474d8fd38d6c8
# Push it to GitHub
git push --set-upstream origin backport/backport-4379-to-2.19-dev
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/sql/backport-2.19-dev Then, create a pull request where the |
(cherry picked from commit 9143a44)
(cherry picked from commit 9143a44) Signed-off-by: Peng Huo <penghuo@gmail.com>
(cherry picked from commit 9143a44)
(cherry picked from commit 9143a44) Signed-off-by: Lantao Jin <ltjin@amazon.com>
* main-apple: (218 commits) Add ignorePrometheus Flag for integTest and docTest (opensearch-project#4442) Create fab-radar.yml PPL `fillnull` command enhancement (opensearch-project#4421) reverting to _doc + _id (opensearch-project#4435) Support `multisearch` command in calcite (opensearch-project#4332) Add 3.3 release notes (opensearch-project#4422) (opensearch-project#4423) [SQL/PPL] Fix the `count(*)` and `dc(field)` to be capped at MAX_INTEGER opensearch-project#4416 (opensearch-project#4418) Change the default search sort tiebreaker to `_shard_doc` for PIT search (opensearch-project#4378) [Enhancement] Add error handling for known limitation of sql `JOIN` (opensearch-project#4344) Bugfix: SQL type mapping for legacy JDBC output (opensearch-project#3613) Version bump: 3.3 (opensearch-project#4417) Add max/min eval functions (opensearch-project#4333) Support time modifiers in search command (opensearch-project#4224) Fix numbered token bug and make it optional output in patterns command (opensearch-project#4402) refactor span (opensearch-project#4334) Move release notes categories (opensearch-project#3818) [Doc] Enable doctest with Calcite (opensearch-project#4379) Mod function should return decimal instead of float when handle the operands are decimal literal (opensearch-project#4407) Scale of decimal literal should always be positive in Calcite (opensearch-project#4401) Enable Calcite by default and implicit fallback the unsupported commands (opensearch-project#4372) ...
Description
Followup of #4372
Related Issues
Resolves #[Issue number to be closed when this PR is merged]
Check List
--signoff
or-s
.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.