Add datasource field in accelerations cache

[ps48]

Description

Updating the accelerations cache structure from:

export interface CachedAccelerations {
  flintIndexName: string;
  type: AccelerationIndexType;
  database: string;
  table: string;
  indexName: string;
  autoRefresh: boolean;
  status: string;
}

export interface AccelerationsCacheData {
  version: string;
  accelerations: CachedAccelerations[];
  lastUpdated: string; // date string in UTC format
  status: CachedDataSourceStatus;
}

To:

export interface CachedAccelerations {
  flintIndexName: string;
  type: AccelerationIndexType;
  database: string;
  table: string;
  indexName: string;
  autoRefresh: boolean;
  status: string;
}

export interface CachedAcclerationByDataSource {
  name: string;
  accelerations: CachedAccelerations[];
  lastUpdated: string; // date string in UTC format
  status: CachedDataSourceStatus;
}

export interface AccelerationsCacheData {
  version: string;
  dataSources: CachedAcclerationByDataSource[];
}

Issues Resolved

#1484

• Add relevant helper functions in the cache manager
• Increase cache test coverage

Check List

• New functionality includes testing.
  • All tests pass, including unit test, integration test and doctest
• New functionality has been documented.
  • New functionality has javadoc added
  • New functionality has user manual doc added
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

Attention: Patch coverage is 95.83333% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 56.94%. Comparing base (d17cad3) to head (47f0768).

Files	Patch %	Lines
public/framework/catalog_cache/cache_loader.tsx	85.71%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1525      +/-   ##
==========================================
+ Coverage   56.62%   56.94%   +0.31%     
==========================================
  Files         348      348              
  Lines       12656    12675      +19     
  Branches     3202     3206       +4     
==========================================
+ Hits         7167     7218      +51     
+ Misses       5436     5404      -32     
  Partials       53       53              

Flag	Coverage Δ
dashboards-observability	56.94% <95.83%> (+0.31%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[RyanL1997]

LGTM. Lets merge this, so that I can use the changes in #1521.

[RyanL1997]

This is because the dataSources: [] contains the above?

        accelerations: [],
        lastUpdated: '',
        status: CachedDataSourceStatus.Empty,

[ps48]

yes!

[RyanL1997]

Why optional here?

[ps48]

Cause columns are lazy loaded for tables. These will be loaded in cache when users open table flyout or use a table in create acceleration flyout.

[RyanL1997]

is this meaning that, SCHEDULED means loading? For example, instead of checking if it is SUCCESS, FAILED, or CANCELLED, I can use this as an indicator to see if it is at a loading/refreshing stage?

[ps48]

this is added as we want to start with status as scheduled whenever the startpolling is called. Otherwise, it keep the old calls status.

[sejli]

SCHEDULED is the initial state, it may not necessarily be loading.

const { loadStatus, startLoading, stopLoading } = useLoadAccelerationsToCache();

If startLoading isn't called, then loadStatus will remain as SCHEDULED.

[mengweieric]

LGTM.

[sejli]

LGTM