[Secuity Solution][DQD] add historical results (Phase 1) (elastic#191898

) addresses elastic#185882 leverages changes introduced in elastic#188468 # Data Quality Dashboard Historical Results (Phase 1) This PR introduces new functionality to the Data Quality Dashboard History tab (new): - view last 30 days of check results by default: - filter by historical checks by outcome (PASS/FAIL/ALL) - paginate all results (10 per page by default) - each result can be viewed in individually and independently expandable/collapsible accordion panel (collapsed by default) - each result contains an extended index stats panel with (custom, ecs and all fields counts) - each result contains index properties tabs (incompatible and same family) - check now checks and redirects to latest check tab with latest check info - switching from initial historical tab to latest check tab triggers latest check - subsequent switching back and forth between already open history or latest check tabs doesn't trigger a check - legacy data (before release of this elastic#185025) is supported with degraded view (same family tab is disabled with warning tooltip), incompatible tab tables are statically rendered from markdown Latest checks list view (changes): - remove check index button icon from list view - add historical results button icon instead of check index button - historical results button icon directly opens history tab without going through latest check # UI Changes (before/after): ## ESS Changes ### Latest check expand icon - expand icon is replaced with check now icon (functionality is the same) - tooltip text is updated - this new icon is still opening the index check flyout tab (latest check tab) ![ess_before_after_0](https://github.com/user-attachments/assets/795af721-6867-4f56-882e-2a0f52793560) ### Historical check icon **(NEW)** - inline check now functionality is removed - view history icon is added in its stead to open a flyout with history tab - tooltip text is updated ![ess_before_after_1](https://github.com/user-attachments/assets/7f2c6009-35c3-488c-87ac-3048f4bded7b) ### Flyout Header - "checked at" subheader is now shorter (milliseconds are removed) - Tabline with Latest check and History tabs is added **(NEW)** ![ess_before_after_2](https://github.com/user-attachments/assets/728ff743-500e-435a-a07e-4287647a0af5) ### History tab **(NEW)** - top left: filter by check outcome - top right: filter by date range - list of checks collapsed by default (individually separately controlled, multiple can be open at a time) - pagination (10,25,50). 10 by default ![ess_after_3](https://github.com/user-attachments/assets/36fc0cee-b103-483d-ba79-d583bba89acf) ### Individual check result view **(NEW)** - topline: extended index stats including new "custom", "ecs compliant" & "all fields". - incompatible fields and same family fields view (custom, ecs compliant and all fields view is unavailable in history tab) ![ess_after_4](https://github.com/user-attachments/assets/57e6d5a1-1470-4c4b-9272-ccc872d80dc5) ### Legacy check result view **(NEW)** - before this PR went to production elastic#185025 check result data contained information allowing to recreate detailed view of incompatible fields from markdown only (without same family fields) - we recreate incompatible field tables in degraded view from markdown - same family tab is permanently disabled with an explanation tooltip - action buttons still work as is for incompatible fields view - index stats panel is showing as for non-legacy result ![SCR-20241009-lmcu](https://github.com/user-attachments/assets/cd11435e-7335-40f3-a0b8-4e5c6bcc2f38) ### No results ![SCR-20241009-llzw](https://github.com/user-attachments/assets/a942ce8e-6e0e-46d3-9104-c30648a18208) ### Loading view ![ess_after_8](https://github.com/user-attachments/assets/1411ccc2-4978-41f6-a02d-2ca404a01c16) ### Error view ![ess_after_9](https://github.com/user-attachments/assets/adc80e19-0005-46f9-a667-ffd3bf8ecb4f) ## Serverless Changes ### Empty checks result badge **(FIX)** - **previously empty pattern check result badge was marked as `PASS` which was incorrect. It was removed.** ![serverless_before_after_0](https://github.com/user-attachments/assets/67e02e9c-cd7f-46d7-9b7a-9bdaa0abfc6c) ### Latest check expand icon - expand icon is replaced with check now icon (functionality is the same) - tooltip text is updated - this new icon is still opening the index check flyout tab (latest check tab) ![serverless_before_after_1](https://github.com/user-attachments/assets/dfac9aad-158b-4863-b719-47d50b06bda3) ### Historical check icon **(NEW)** - inline check now functionality is removed - view history icon is added in its stead to open a flyout with history tab - tooltip text is updated ![serverless_before_after_2](https://github.com/user-attachments/assets/c688c28c-2d86-4669-a9bb-ffc297d21bbf) ### Flyout Header and Body Topline - "checked at" subheader is now shorter (milliseconds are removed) - Tabline with Latest check and History tabs is added **(NEW)** - **Index Stats Panel is now also showing here just like in latest check tab (but without phase label as ilm is not available in serverless)** **(NEW)** ![serverless_before_after_3](https://github.com/user-attachments/assets/c3ae4160-d07c-4049-b8b4-4b66faa50320) ### History tab **(NEW)** - top left: filter by check outcome - top right: filter by date range - list of checks collapsed by default (individually separately controlled, multiple can be open at a time) - pagination (10,25,50). 10 by default ![serverless_after_4](https://github.com/user-attachments/assets/8b767de3-1ab1-4b9f-b0b8-84754a3776ae) ### Individual check result view **(NEW)** - topline: extended index stats including new "custom", "ecs compliant" & "all fields" but **excluding ilm phase label section**. - incompatible fields and same family fields view (custom, ecs compliant and all fields view is unavailable in history tab) ![serverless_after_5](https://github.com/user-attachments/assets/d8fdd48f-63f2-48f2-8ede-3613bffaa157) ### Legacy check result view **(NEW)** - before this PR went to production elastic#185025 check result data contained information allowing to recreate detailed view of incompatible fields from markdown only (without same family fields) - we recreate incompatible field tables in degraded view from markdown - same family tab is permanently disabled with an explanation tooltip - action buttons still work as is for incompatible fields view - index stats panel is showing as for non-legacy result ![SCR-20241009-lkhi](https://github.com/user-attachments/assets/10adee1c-c11a-428a-9c56-ecc20a37f97f) ### No results ![SCR-20241009-ljwg](https://github.com/user-attachments/assets/8bf48778-98d6-4a96-a713-b49d4cc5165a) ### Loading view ![serverless_after_9](https://github.com/user-attachments/assets/5ba1f2cc-cbd9-4cfa-964c-962be150016f) ### Error view ![serverless_after_10](https://github.com/user-attachments/assets/b5c33ded-4ee5-46ff-9e13-f9e5dfc7546e) (cherry picked from commit e5f7739)
kibanamachine · Oct 11, 2024 · 80c7080 · 80c7080
1 parent 77e8ad4
commit 80c7080
Show file tree

Hide file tree

Showing 194 changed files with 10,022 additions and 5,057 deletions.
diff --git a/...shboard/impl/data_quality_panel/data_quality_details/indices_details/pattern/constants.ts b/...shboard/impl/data_quality_panel/data_quality_details/indices_details/pattern/constants.ts
@@ -6,3 +6,6 @@
  */
 
 export const MIN_PAGE_SIZE = 10;
+
+export const HISTORY_TAB_ID = 'history';
+export const LATEST_CHECK_TAB_ID = 'latest_check';
diff --git a/...ata_quality_details/indices_details/pattern/contexts/historical_results_context/index.tsx b/...ata_quality_details/indices_details/pattern/contexts/historical_results_context/index.tsx
@@ -0,0 +1,21 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+import { createContext, useContext } from 'react';
+import { HistoricalResultsValue } from './types';
+
+export const HistoricalResultsContext = createContext<HistoricalResultsValue | null>(null);
+
+export const useHistoricalResultsContext = () => {
+  const context = useContext(HistoricalResultsContext);
+  if (context == null) {
+    throw new Error(
+      'useHistoricalResultsContext must be used inside the HistoricalResultsContextProvider.'
+    );
+  }
+  return context;
+};
diff --git a/...data_quality_details/indices_details/pattern/contexts/historical_results_context/types.ts b/...data_quality_details/indices_details/pattern/contexts/historical_results_context/types.ts
@@ -0,0 +1,19 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+import { HistoricalResult } from '../../../../../types';
+import { UseHistoricalResultsFetch } from '../../index_check_flyout/types';
+
+export interface HistoricalResultsValue {
+  historicalResultsState: {
+    results: HistoricalResult[];
+    total: number;
+    isLoading: boolean;
+    error: Error | null;
+  };
+  fetchHistoricalResults: UseHistoricalResultsFetch;
+}
diff --git a/...roperties/index_check_fields/constants.ts → ...hooks/use_historical_results/constants.ts b/...roperties/index_check_fields/constants.ts → ...hooks/use_historical_results/constants.ts
@@ -5,8 +5,4 @@
  * 2.0.
  */
 
-export const ALL_TAB_ID = 'allTab';
-export const ECS_COMPLIANT_TAB_ID = 'ecsCompliantTab';
-export const CUSTOM_TAB_ID = 'customTab';
-export const INCOMPATIBLE_TAB_ID = 'incompatibleTab';
-export const SAME_FAMILY_TAB_ID = 'sameFamilyTab';
+export const GET_INDEX_RESULTS = '/internal/ecs_data_quality_dashboard/results/{indexName}';
diff --git a/.../data_quality_details/indices_details/pattern/hooks/use_historical_results/index.test.tsx b/.../data_quality_details/indices_details/pattern/hooks/use_historical_results/index.test.tsx
@@ -0,0 +1,158 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+import { act, renderHook } from '@testing-library/react-hooks';
+
+import { mockHistoricalResult } from '../../../../../mock/historical_results/mock_historical_results_response';
+import { TestDataQualityProviders } from '../../../../../mock/test_providers/test_providers';
+import * as fetchHistoricalResults from './utils/fetch_historical_results';
+import { useHistoricalResults } from '.';
+
+describe('useHistoricalResults', () => {
+  beforeEach(() => {
+    jest.restoreAllMocks();
+  });
+
+  it('should return initial historical results state and fetch historical results function', () => {
+    const { result } = renderHook(() => useHistoricalResults(), {
+      wrapper: TestDataQualityProviders,
+    });
+
+    expect(result.current.historicalResultsState).toEqual({
+      results: [],
+      total: 0,
+      isLoading: true,
+      error: null,
+    });
+
+    expect(result.current.fetchHistoricalResults).toBeInstanceOf(Function);
+  });
+
+  describe('when fetchHistoricalResults is called', () => {
+    it('should fetch historical results and update historical results state', async () => {
+      const fetchResultsSpy = jest
+        .spyOn(fetchHistoricalResults, 'fetchHistoricalResults')
+        .mockResolvedValue({
+          results: [mockHistoricalResult],
+          total: 1,
+        });
+
+      const { result } = renderHook(() => useHistoricalResults(), {
+        wrapper: TestDataQualityProviders,
+      });
+
+      const abortController = new AbortController();
+
+      await act(() =>
+        result.current.fetchHistoricalResults({
+          abortController,
+          indexName: 'indexName',
+          size: 10,
+          from: 0,
+          startDate: 'now-7d',
+          endDate: 'now',
+          outcome: 'pass',
+        })
+      );
+
+      expect(fetchResultsSpy).toHaveBeenCalledWith({
+        indexName: 'indexName',
+        httpFetch: expect.any(Function),
+        abortController,
+        size: 10,
+        from: 0,
+        startDate: 'now-7d',
+        endDate: 'now',
+        outcome: 'pass',
+      });
+
+      expect(result.current.historicalResultsState).toEqual({
+        results: [mockHistoricalResult],
+        total: 1,
+        isLoading: false,
+        error: null,
+      });
+    });
+  });
+
+  describe('when fetchHistoricalResults fails', () => {
+    it('should update historical results state with error', async () => {
+      const fetchResultsSpy = jest
+        .spyOn(fetchHistoricalResults, 'fetchHistoricalResults')
+        .mockRejectedValue(new Error('An error occurred'));
+
+      const { result } = renderHook(() => useHistoricalResults(), {
+        wrapper: TestDataQualityProviders,
+      });
+
+      const abortController = new AbortController();
+
+      await act(() =>
+        result.current.fetchHistoricalResults({
+          abortController,
+          indexName: 'indexName',
+          size: 10,
+          from: 0,
+          startDate: 'now-7d',
+          endDate: 'now',
+          outcome: 'pass',
+        })
+      );
+
+      expect(fetchResultsSpy).toHaveBeenCalledWith({
+        indexName: 'indexName',
+        httpFetch: expect.any(Function),
+        abortController,
+        size: 10,
+        from: 0,
+        startDate: 'now-7d',
+        endDate: 'now',
+        outcome: 'pass',
+      });
+
+      expect(result.current.historicalResultsState).toEqual({
+        results: [],
+        total: 0,
+        isLoading: false,
+        error: new Error('An error occurred'),
+      });
+    });
+  });
+
+  describe('during fetchHistoricalResults call', () => {
+    it('should set isLoading to true', async () => {
+      jest.spyOn(fetchHistoricalResults, 'fetchHistoricalResults').mockImplementation(() => {
+        return new Promise(() => {});
+      });
+
+      const { result } = renderHook(() => useHistoricalResults(), {
+        wrapper: TestDataQualityProviders,
+      });
+
+      const abortController = new AbortController();
+
+      act(() => {
+        result.current.fetchHistoricalResults({
+          abortController,
+          indexName: 'indexName',
+          size: 10,
+          from: 0,
+          startDate: 'now-7d',
+          endDate: 'now',
+          outcome: 'pass',
+        });
+      });
+
+      expect(result.current.historicalResultsState).toEqual({
+        results: [],
+        total: 0,
+        isLoading: true,
+        error: null,
+      });
+    });
+  });
+});
diff --git a/...panel/data_quality_details/indices_details/pattern/hooks/use_historical_results/index.tsx b/...panel/data_quality_details/indices_details/pattern/hooks/use_historical_results/index.tsx
@@ -0,0 +1,85 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+import { useReducer, useCallback } from 'react';
+
+import { GET_RESULTS_ERROR_TITLE } from '../../../../../translations';
+import { useDataQualityContext } from '../../../../../data_quality_context';
+import { useIsMountedRef } from '../../../../../hooks/use_is_mounted_ref';
+import { fetchHistoricalResults } from './utils/fetch_historical_results';
+import { FetchHistoricalResultsReducerState, UseHistoricalResultsReturnValue } from './types';
+import { UseHistoricalResultsFetchOpts } from '../../index_check_flyout/types';
+import { fetchHistoricalResultsReducer } from './reducers/fetch_historical_results_reducer';
+
+export const initialFetchHistoricalResultsReducerState: FetchHistoricalResultsReducerState = {
+  results: [],
+  total: 0,
+  isLoading: true,
+  error: null,
+};
+
+export const useHistoricalResults = (): UseHistoricalResultsReturnValue => {
+  const [state, dispatch] = useReducer(
+    fetchHistoricalResultsReducer,
+    initialFetchHistoricalResultsReducerState
+  );
+  const { httpFetch, toasts } = useDataQualityContext();
+  const { isMountedRef } = useIsMountedRef();
+
+  const fetchResults = useCallback(
+    async ({
+      abortController,
+      indexName,
+      size,
+      from,
+      startDate,
+      endDate,
+      outcome,
+    }: UseHistoricalResultsFetchOpts) => {
+      dispatch({ type: 'FETCH_START' });
+
+      try {
+        const { results, total } = await fetchHistoricalResults({
+          indexName,
+          httpFetch,
+          abortController,
+          size,
+          from,
+          startDate,
+          endDate,
+          outcome,
+        });
+
+        if (isMountedRef.current) {
+          dispatch({
+            type: 'FETCH_SUCCESS',
+            payload: {
+              results,
+              total,
+            },
+          });
+        }
+      } catch (error) {
+        if (isMountedRef.current) {
+          toasts.addError(error, { title: GET_RESULTS_ERROR_TITLE });
+          dispatch({ type: 'FETCH_ERROR', payload: error });
+        }
+      }
+    },
+    [dispatch, httpFetch, toasts, isMountedRef]
+  );
+
+  return {
+    historicalResultsState: {
+      results: state.results,
+      total: state.total,
+      isLoading: state.isLoading,
+      error: state.error,
+    },
+    fetchHistoricalResults: fetchResults,
+  };
+};
diff --git a/...ls/pattern/hooks/use_historical_results/reducers/fetch_historical_results_reducer.test.ts b/...ls/pattern/hooks/use_historical_results/reducers/fetch_historical_results_reducer.test.ts
@@ -0,0 +1,66 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License
+ * 2.0; you may not use this file except in compliance with the Elastic License
+ * 2.0.
+ */
+
+import { mockHistoricalResult } from '../../../../../../mock/historical_results/mock_historical_results_response';
+import { fetchHistoricalResultsReducer } from './fetch_historical_results_reducer';
+
+const getInitialState = () => ({
+  results: [],
+  total: 0,
+  isLoading: true,
+  error: null,
+});
+
+describe('fetchHistoricalResultsReducer', () => {
+  describe('on fetch start', () => {
+    it('should return initial state', () => {
+      expect(fetchHistoricalResultsReducer(getInitialState(), { type: 'FETCH_START' })).toEqual({
+        results: [],
+        total: 0,
+        isLoading: true,
+        error: null,
+      });
+    });
+  });
+
+  describe('on fetch success', () => {
+    it('should update state with fetched results', () => {
+      const results = [mockHistoricalResult];
+      const total = 1;
+
+      expect(
+        fetchHistoricalResultsReducer(getInitialState(), {
+          type: 'FETCH_SUCCESS',
+          payload: { results, total },
+        })
+      ).toEqual({
+        results,
+        total,
+        isLoading: false,
+        error: null,
+      });
+    });
+  });
+
+  describe('on fetch error', () => {
+    it('should update state with error', () => {
+      const error = new Error('An error occurred');
+
+      expect(
+        fetchHistoricalResultsReducer(getInitialState(), {
+          type: 'FETCH_ERROR',
+          payload: error,
+        })
+      ).toEqual({
+        results: [],
+        total: 0,
+        isLoading: false,
+        error,
+      });
+    });
+  });
+});