[WIP][discover][s3] fetch options with fields and caches it #8304
Conversation
If the dataset type config passes a meta object with `updateAt` within the data structure we know that the config opts into the TTL. If TTL over configuring UI setting then refetch in the case of S3 to get a new session ID in the cache. Signed-off-by: Kawika Avilla <kavilla414@gmail.com>
Currently does not work I believe. With sample it did send `DESCRIBE default.http_logs_partitioned` But failed with: Total size of serialized results of 6 task is bigger than maxResult size. Otherwise it should be calling the same helper functions that cache the data structure. Signed-off-by: Kawika Avilla <kavilla414@gmail.com>
❌ Empty Changelog SectionThe Changelog section in your PR description is empty. Please add a valid changelog entry or entries. If you did add a changelog entry, check to make sure that it was not accidentally included inside the comment block in the Changelog section. |
❌ Empty Changelog SectionThe Changelog section in your PR description is empty. Please add a valid changelog entry or entries. If you did add a changelog entry, check to make sure that it was not accidentally included inside the comment block in the Changelog section. |
1 similar comment
❌ Empty Changelog SectionThe Changelog section in your PR description is empty. Please add a valid changelog entry or entries. If you did add a changelog entry, check to make sure that it was not accidentally included inside the comment block in the Changelog section. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #8304 +/- ##
==========================================
- Coverage 64.14% 64.12% -0.02%
==========================================
Files 3743 3743
Lines 88833 88860 +27
Branches 13852 13860 +8
==========================================
+ Hits 56979 56980 +1
- Misses 31239 31264 +25
- Partials 615 616 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
| path: trimEnd(`${API.DATA_SOURCE.ASYNC_JOBS}`), | ||
| body: JSON.stringify({ | ||
| lang: 'sql', | ||
| query: `DESCRIBE ${database?.title}.${table.title}`, |
There was a problem hiding this comment.
Should be DESCRIBE TABLE ${connection?.title}.${database?.title}.${table.title}
|
closing in favor of: https://github.com/opensearch-project/OpenSearch-Dashboards/pull/8337/files |
Description
NOTE: Contains commit from #8271. For this PR just look at: a9fa1df
NOTE: Currently does not work. Could it be the wrong query being sent?
Follow-up item should be what happens in the case where fetch fields fails? Or takes too long? Should we put a spinner on the configurator to not display until fetch fields has return a result (regardless of success or fail). Should we disable the
Select dataif fields have not returned a result?With sample it did send
DESCRIBE default.http_logs_partitionedBut failed with: Total size of serialized results of 6 task is bigger than maxResult size.
Otherwise it should be calling the same helper functions that cache the data structure.
Issues Resolved
n/a
Screenshot
Testing the changes
Changelog
Check List
yarn test:jestyarn test:jest_integration