-
Notifications
You must be signed in to change notification settings - Fork 884
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[MD] onboard TSVB to support multiple data source #2572
Conversation
Signed-off-by: Su <szhongna@amazon.com>
@zhongnansu A High level question about the abstraction here. Since multi datasource is tightly coupled with the data plugin, wouldnt it make sense to have the data plugin directly responsible for determining the datasource given the existing data? Or am i missing something about why this is not possible? My naive thinking is that the datasource integration happens at the lowest level of the data plugin where it knows which datasource is related to any given index pattern. So other plugins (including TSVB) can operate under the assumption that there are only index patterns without worrying about what the underlying datasource is. |
@ashwin-pc Like I mentioned in the PR description. TSVB is different with other vis in the way it queries data.
|
@zhongnansu Thanks for the context! I didn't know the difference between the low and high level data plugin api's. I know the answer to my question is probably no, but did you look to see if you could change TSVB to use the high level api instead? If you have, just ignore my comment 😄 Also, not for this PR, but given all the recent investigations and work that has gone into the data plugin and how it works, can we also update its readme? For what it does, its quite limited in its documentation. https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/plugins/data/README.md. |
Thanks @ashwin-pc and @zhongnansu for great discussion. Good suggestion to keep the update to date |
@zhongnansu is this still in the works or can we close it? |
Closing the PR for now. There are some blockers and prioritization change to this task. @ashwin-pc |
Signed-off-by: Su szhongna@amazon.com
Description
[require UX/UI changes to go along]
[MD] onboard TSVB to support multiple data source
Unlike other vis types, TSVB has its own backend. From the expressions TSVB generated, it doesn't go through
opensearchaggs
to create the searchSource obj, and call high level search API. Instead, they retrieve the index pattern, build the OpenSearch query from their plugin and directly query OpenSearch in 2 ways.legacyOpenSearchClient
to query OS to get runtime fields and fields capabilities.This PR targets to refactor above 2 flows by integrating with datasource params. That also requires adding
MD
as an optional plugin of TSVBSee issue below for more details
Issues Resolved
#2153
part of #1990
Check List
yarn test:jest
yarn test:jest_integration
yarn test:ftr