Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature-wip](multi-catalog)(fix) federation query failed #10602

Merged
merged 8 commits into from
Jul 8, 2022

Conversation

AshinGau
Copy link
Member

@AshinGau AshinGau commented Jul 4, 2022

Proposed changes

Problem Summary:

Fix #10521, multi-catalog query failed for two reasons:

  1. The SelectStmt does not get the correct catalog.
  2. External table should have three level aliases.

Disable querying external views.
Support show create table for external table&view.

Usage

Without ambiguity, four formats of select columns and three formats of from tables are supported.

select columns

  1. select col from
  2. select tbl.col from
  3. select db.tbl.col from
  4. select ctl.db.tbl.col from

from tables

  1. select col from tbl
  2. select col from db.tbl
  3. select col from ctl.db.tbl

q3 in tpch

take q3 in tpch as an example:

SELECT
  l_orderkey, -- no table prefix
  sum(l_extendedprice * (1 - l_discount)) AS revenue,
  o_orderdate,
  o_shippriority
FROM
  dtest.customer,
  dtest.orders,
  hive.hdb.lineitem
WHERE
  c_mktsegment = 'BUILDING'
  AND c_custkey = o_custkey
  AND l_orderkey = o_orderkey
  AND o_orderdate < DATE '1995-03-15'
  AND l_shipdate > DATE '1995-03-15'
GROUP BY
  l_orderkey,
  o_orderdate,
  o_shippriority
ORDER BY
  revenue DESC,
  o_orderdate
LIMIT 10;
SELECT
  lineitem.l_orderkey, -- with table prefix
  sum(l_extendedprice * (1 - l_discount)) AS revenue,
  o_orderdate,
  o_shippriority
FROM
  dtest.customer,
  dtest.orders,
  hive.hdb.lineitem
WHERE
  c_mktsegment = 'BUILDING'
  AND c_custkey = o_custkey
  AND l_orderkey = o_orderkey
  AND o_orderdate < DATE '1995-03-15'
  AND l_shipdate > DATE '1995-03-15'
GROUP BY
  l_orderkey,
  o_orderdate,
  o_shippriority
ORDER BY
  revenue DESC,
  o_orderdate
LIMIT 10;
SELECT
  hdb.lineitem.l_orderkey, -- with database prefix
  sum(l_extendedprice * (1 - l_discount)) AS revenue,
  o_orderdate,
  o_shippriority
FROM
  dtest.customer,
  dtest.orders,
  hive.hdb.lineitem
WHERE
  c_mktsegment = 'BUILDING'
  AND c_custkey = o_custkey
  AND l_orderkey = o_orderkey
  AND o_orderdate < DATE '1995-03-15'
  AND l_shipdate > DATE '1995-03-15'
GROUP BY
  l_orderkey,
  o_orderdate,
  o_shippriority
ORDER BY
  revenue DESC,
  o_orderdate
LIMIT 10;
SELECT
  hive.hdb.lineitem.l_orderkey, -- with catalog prefix
  sum(hive.hdb.lineitem.l_extendedprice * (1 - hive.hdb.lineitem.l_discount)) AS revenue,
  internal.dtest.orders.o_orderdate,
  internal.dtest.orders.o_shippriority
FROM
  internal.dtest.customer,
  internal.dtest.orders,
  hive.hdb.lineitem
WHERE
  c_mktsegment = 'BUILDING'
  AND internal.dtest.customer.c_custkey = internal.dtest.orders.o_custkey
  AND hive.hdb.lineitem.l_orderkey = internal.dtest.orders.o_orderkey
  AND internal.dtest.orders.o_orderdate < DATE '1995-03-15'
  AND hive.hdb.lineitem.l_shipdate > DATE '1995-03-15'
GROUP BY
  hive.hdb.lineitem.l_orderkey,
  internal.dtest.orders.o_orderdate,
  internal.dtest.orders.o_shippriority
ORDER BY
  revenue DESC,
  internal.dtest.orders.o_orderdate
LIMIT 10;

Checklist(Required)

  1. Does it affect the original behavior: (No)
  2. Has unit tests been added: (No)
  3. Has document been added or modified: (No)
  4. Does it need to update dependencies: (No)
  5. Are there any changes that cannot be rolled back: (No)

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@AshinGau AshinGau changed the title [feature-wip](multi-catalog)(followup) federation query failed [feature-wip](multi-catalog)(fix) federation query failed Jul 4, 2022
@github-actions github-actions bot added the area/planner Issues or PRs related to the query planner label Jul 4, 2022
@AshinGau AshinGau force-pushed the multi-cat-fix branch 2 times, most recently from 0dd8b8d to 9b8db72 Compare July 6, 2022 06:28
morningman
morningman previously approved these changes Jul 6, 2022
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 6, 2022
@github-actions
Copy link
Contributor

github-actions bot commented Jul 6, 2022

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Jul 6, 2022

PR approved by anyone and no changes requested.

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jul 6, 2022
morningman
morningman previously approved these changes Jul 6, 2022
@github-actions
Copy link
Contributor

github-actions bot commented Jul 6, 2022

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 6, 2022
@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jul 6, 2022
@morningman morningman merged commit 874299f into apache:master Jul 8, 2022
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 8, 2022
@github-actions
Copy link
Contributor

github-actions bot commented Jul 8, 2022

PR approved by at least one committer and no changes requested.

@AshinGau AshinGau deleted the multi-cat-fix branch July 8, 2022 03:19
eldenmoon pushed a commit to eldenmoon/incubator-doris that referenced this pull request Aug 1, 2022
Fix apache#10521, multi-catalog query failed for two reasons:
1. The `SelectStmt` does not get the correct catalog.
2. External table should have three level aliases.

Disable querying external views.
Support show create table for external table&view.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. area/multi-catalog area/nereids area/planner Issues or PRs related to the query planner reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants