Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Secuity Solution][DQD] add historical results (Phase 1) #191898

Merged
merged 21 commits into from
Oct 11, 2024

Conversation

kapral18
Copy link
Contributor

@kapral18 kapral18 commented Sep 2, 2024

addresses #185882

leverages changes introduced in #188468

Data Quality Dashboard Historical Results (Phase 1)

This PR introduces new functionality to the Data Quality Dashboard

History tab (new):

  • view last 30 days of check results by default:
  • filter by historical checks by outcome (PASS/FAIL/ALL)
  • paginate all results (10 per page by default)
  • each result can be viewed in individually and independently expandable/collapsible accordion panel (collapsed by default)
  • each result contains an extended index stats panel with (custom, ecs and all fields counts)
  • each result contains index properties tabs (incompatible and same family)
  • check now checks and redirects to latest check tab with latest check info
  • switching from initial historical tab to latest check tab triggers latest check
  • subsequent switching back and forth between already open history or latest check tabs doesn't trigger a check
  • legacy data (before release of this [Security Solution][DQD] Persist new fields in results storage #185025) is supported with degraded view (same family tab is disabled with warning tooltip), incompatible tab tables are statically rendered from markdown

Latest checks list view (changes):

  • remove check index button icon from list view
  • add historical results button icon instead of check index button
  • historical results button icon directly opens history tab without going through latest check

UI Changes (before/after):

ESS Changes

Latest check expand icon

  • expand icon is replaced with check now icon (functionality is the same)
  • tooltip text is updated
  • this new icon is still opening the index check flyout tab (latest check tab)
    ess_before_after_0

Historical check icon (NEW)

  • inline check now functionality is removed
  • view history icon is added in its stead to open a flyout with history tab
  • tooltip text is updated
    ess_before_after_1

Flyout Header

  • "checked at" subheader is now shorter (milliseconds are removed)
  • Tabline with Latest check and History tabs is added (NEW)
    ess_before_after_2

History tab (NEW)

  • top left: filter by check outcome
  • top right: filter by date range
  • list of checks collapsed by default (individually separately controlled, multiple can be open at a time)
  • pagination (10,25,50). 10 by default
    ess_after_3

Individual check result view (NEW)

  • topline: extended index stats including new "custom", "ecs compliant" & "all fields".
  • incompatible fields and same family fields view (custom, ecs compliant and all fields view is unavailable in history tab)
    ess_after_4

Legacy check result view (NEW)

  • before this PR went to production [Security Solution][DQD] Persist new fields in results storage #185025 check result data contained information allowing to recreate detailed view of incompatible fields from markdown only (without same family fields)
  • we recreate incompatible field tables in degraded view from markdown
  • same family tab is permanently disabled with an explanation tooltip
  • action buttons still work as is for incompatible fields view
  • index stats panel is showing as for non-legacy result
    SCR-20241009-lmcu

No results

SCR-20241009-llzw

Loading view

ess_after_8

Error view

ess_after_9

Serverless Changes

Empty checks result badge (FIX)

  • previously empty pattern check result badge was marked as PASS which was incorrect. It was removed.
    serverless_before_after_0

Latest check expand icon

  • expand icon is replaced with check now icon (functionality is the same)
  • tooltip text is updated
  • this new icon is still opening the index check flyout tab (latest check tab)
    serverless_before_after_1

Historical check icon (NEW)

  • inline check now functionality is removed
  • view history icon is added in its stead to open a flyout with history tab
  • tooltip text is updated
    serverless_before_after_2

Flyout Header and Body Topline

  • "checked at" subheader is now shorter (milliseconds are removed)
  • Tabline with Latest check and History tabs is added (NEW)
  • Index Stats Panel is now also showing here just like in latest check tab (but without phase label as ilm is not available in serverless) (NEW)
    serverless_before_after_3

History tab (NEW)

  • top left: filter by check outcome
  • top right: filter by date range
  • list of checks collapsed by default (individually separately controlled, multiple can be open at a time)
  • pagination (10,25,50). 10 by default
    serverless_after_4

Individual check result view (NEW)

  • topline: extended index stats including new "custom", "ecs compliant" & "all fields" but excluding ilm phase label section.
  • incompatible fields and same family fields view (custom, ecs compliant and all fields view is unavailable in history tab)
    serverless_after_5

Legacy check result view (NEW)

  • before this PR went to production [Security Solution][DQD] Persist new fields in results storage #185025 check result data contained information allowing to recreate detailed view of incompatible fields from markdown only (without same family fields)
  • we recreate incompatible field tables in degraded view from markdown
  • same family tab is permanently disabled with an explanation tooltip
  • action buttons still work as is for incompatible fields view
  • index stats panel is showing as for non-legacy result
    SCR-20241009-lkhi

No results

SCR-20241009-ljwg

Loading view

serverless_after_9

Error view

serverless_after_10

@kapral18 kapral18 self-assigned this Sep 2, 2024
@kapral18 kapral18 changed the title feat: restore original flyout tabs with disabled history [Secuity Solution][DQD] add historical results (Phase 1) Sep 2, 2024
@kapral18 kapral18 added 8.16 candidate v8.16.0 Team:Threat Hunting Security Solution Threat Hunting Team Team:Threat Hunting:Explore backport:skip This commit does not require backporting and removed 8.16 candidate labels Sep 2, 2024
@kapral18 kapral18 force-pushed the feat/DQD/185882-add-historical-tab branch from f8fdfe6 to 9aaa871 Compare September 10, 2024 11:46
@kapral18 kapral18 added ci:cloud-deploy Create or update a Cloud deployment ci:cloud-persist-deployment Persist cloud deployment indefinitely ci:project-deploy-security Create a Security Serverless Project ci:project-persist-deployment Persist project deployment indefinitely labels Sep 10, 2024
@kapral18 kapral18 force-pushed the feat/DQD/185882-add-historical-tab branch 6 times, most recently from cd06cc9 to 021edde Compare September 16, 2024 16:01
@kapral18 kapral18 force-pushed the feat/DQD/185882-add-historical-tab branch 3 times, most recently from fa35ca9 to 68a7041 Compare September 24, 2024 00:26
@kapral18 kapral18 force-pushed the feat/DQD/185882-add-historical-tab branch 2 times, most recently from 68f53da to c72f04d Compare September 30, 2024 18:18
@kapral18 kapral18 marked this pull request as ready for review October 1, 2024 14:01
@kapral18 kapral18 requested a review from a team as a code owner October 1, 2024 14:01
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-threat-hunting (Team:Threat Hunting)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-threat-hunting-explore (Team:Threat Hunting:Explore)

@kapral18 kapral18 added backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) and removed backport:skip This commit does not require backporting labels Oct 1, 2024
- add historical results fetching logic with hooks/utils/contexts
- add filtering by outcome
- add filtering by date
- add pagination
- add error handling
- add loading state
- rename index properties into latest results
- add total checks
- add accordion component titles (result badge + check time +
  incompatible count)
- refactor affected parts
- added `EuiEmptyPrompt` for no historical results scenario.
- flattened translations
- flattened tabs/
- removed unused `missing_timestamp_callout` component and related helpers.
- Updated tests to reflect new file structure and imports.
- implement flyout base functionality of historical results showing
incompatible and same family tabs
- add index stats panel at the top
- add check now to latest check redirect
- Introduced constants for HISTORY_TAB_ID and LATEST_CHECK_TAB_ID.
- Lifted historical results hook and context up to accomodate calling
  fetch historical results from pattern level
- Updated components to utilize new tab functionality for viewing historical results and latest checks.
- Added 2 distinct non results empty prompt translations for when there
  is no results vs no results in given range
- Most tests are broken with this commit, updated tests will come in
  next one
@kapral18 kapral18 force-pushed the feat/DQD/185882-add-historical-tab branch from ae8043e to e282dde Compare October 11, 2024 14:05
@elasticmachine
Copy link
Contributor

elasticmachine commented Oct 11, 2024

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
securitySolution 5951 5976 +25

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
securitySolution 20.7MB 20.7MB +23.9KB
Unknown metric groups

ESLint disabled line counts

id before after diff
@kbn/ecs-data-quality-dashboard 3 7 +4

Total ESLint disabled count

id before after diff
@kbn/ecs-data-quality-dashboard 3 7 +4

History

  • 💔 Build #241453 failed ae8043eceb0d8703d75c74a80f1dbf7f9e7c9bb0
  • 💚 Build #240767 succeeded cd619ef28d0586e90fdec26c68f6a9a7f8953cb4
  • 💔 Build #240625 failed 6bff5b6064601825fede5e0a353d42f1c81e9520
  • 💔 Build #240500 failed ea5e543f0339f4a507239536fab460ea9a7d8947

cc @kapral18

@kapral18 kapral18 merged commit e5f7739 into elastic:main Oct 11, 2024
39 of 44 checks passed
@kapral18 kapral18 deleted the feat/DQD/185882-add-historical-tab branch October 11, 2024 16:14
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11295727561

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 11, 2024
)

addresses elastic#185882

leverages changes introduced in
elastic#188468

# Data Quality Dashboard Historical Results (Phase 1)

This PR introduces new functionality to the Data Quality Dashboard

History tab (new):
- view last 30 days of check results by default:
- filter by historical checks by outcome (PASS/FAIL/ALL)
- paginate all results (10 per page by default)
- each result can be viewed in individually and independently
expandable/collapsible accordion panel (collapsed by default)
- each result contains an extended index stats panel with (custom, ecs
and all fields counts)
- each result contains index properties tabs (incompatible and same
family)
- check now checks and redirects to latest check tab with latest check
info
- switching from initial historical tab to latest check tab triggers
latest check
- subsequent switching back and forth between already open history or
latest check tabs doesn't trigger a check
- legacy data (before release of this
elastic#185025) is supported with
degraded view (same family tab is disabled with warning tooltip),
incompatible tab tables are statically rendered from markdown

Latest checks list view (changes):
- remove check index button icon from list view
- add historical results button icon instead of check index button
- historical results button icon directly opens history tab without
going through latest check

# UI Changes (before/after):

## ESS Changes

### Latest check expand icon
- expand icon is replaced with check now icon (functionality is the
same)
- tooltip text is updated
- this new icon is still opening the index check flyout tab (latest
check tab)

![ess_before_after_0](https://github.com/user-attachments/assets/795af721-6867-4f56-882e-2a0f52793560)

### Historical check icon **(NEW)**
- inline check now functionality is removed
- view history icon is added in its stead to open a flyout with history
tab
- tooltip text is updated

![ess_before_after_1](https://github.com/user-attachments/assets/7f2c6009-35c3-488c-87ac-3048f4bded7b)

### Flyout Header
- "checked at" subheader is now shorter (milliseconds are removed)
- Tabline with Latest check and History tabs is added **(NEW)**

![ess_before_after_2](https://github.com/user-attachments/assets/728ff743-500e-435a-a07e-4287647a0af5)

### History tab **(NEW)**
- top left: filter by check outcome
- top right: filter by date range
- list of checks collapsed by default (individually separately
controlled, multiple can be open at a time)
- pagination (10,25,50). 10 by default

![ess_after_3](https://github.com/user-attachments/assets/36fc0cee-b103-483d-ba79-d583bba89acf)

### Individual check result view **(NEW)**
- topline: extended index stats including new "custom", "ecs compliant"
& "all fields".
- incompatible fields and same family fields view (custom, ecs compliant
and all fields view is unavailable in history tab)

![ess_after_4](https://github.com/user-attachments/assets/57e6d5a1-1470-4c4b-9272-ccc872d80dc5)

### Legacy check result view **(NEW)**
- before this PR went to production
elastic#185025 check result data
contained information allowing to recreate detailed view of incompatible
fields from markdown only (without same family fields)
- we recreate incompatible field tables in degraded view from markdown
- same family tab is permanently disabled with an explanation tooltip
- action buttons still work as is for incompatible fields view
- index stats panel is showing as for non-legacy result

![SCR-20241009-lmcu](https://github.com/user-attachments/assets/cd11435e-7335-40f3-a0b8-4e5c6bcc2f38)

### No results

![SCR-20241009-llzw](https://github.com/user-attachments/assets/a942ce8e-6e0e-46d3-9104-c30648a18208)

### Loading view

![ess_after_8](https://github.com/user-attachments/assets/1411ccc2-4978-41f6-a02d-2ca404a01c16)

### Error view

![ess_after_9](https://github.com/user-attachments/assets/adc80e19-0005-46f9-a667-ffd3bf8ecb4f)

## Serverless Changes
### Empty checks result badge **(FIX)**
- **previously empty pattern check result badge was marked as `PASS`
which was incorrect. It was removed.**

![serverless_before_after_0](https://github.com/user-attachments/assets/67e02e9c-cd7f-46d7-9b7a-9bdaa0abfc6c)

### Latest check expand icon
- expand icon is replaced with check now icon (functionality is the
same)
- tooltip text is updated
- this new icon is still opening the index check flyout tab (latest
check tab)

![serverless_before_after_1](https://github.com/user-attachments/assets/dfac9aad-158b-4863-b719-47d50b06bda3)

### Historical check icon **(NEW)**
- inline check now functionality is removed
- view history icon is added in its stead to open a flyout with history
tab
- tooltip text is updated

![serverless_before_after_2](https://github.com/user-attachments/assets/c688c28c-2d86-4669-a9bb-ffc297d21bbf)

### Flyout Header and Body Topline
- "checked at" subheader is now shorter (milliseconds are removed)
- Tabline with Latest check and History tabs is added **(NEW)**
- **Index Stats Panel is now also showing here just like in latest check
tab (but without phase label as ilm is not available in serverless)**
**(NEW)**

![serverless_before_after_3](https://github.com/user-attachments/assets/c3ae4160-d07c-4049-b8b4-4b66faa50320)

### History tab **(NEW)**
- top left: filter by check outcome
- top right: filter by date range
- list of checks collapsed by default (individually separately
controlled, multiple can be open at a time)
- pagination (10,25,50). 10 by default

![serverless_after_4](https://github.com/user-attachments/assets/8b767de3-1ab1-4b9f-b0b8-84754a3776ae)

### Individual check result view **(NEW)**
- topline: extended index stats including new "custom", "ecs compliant"
& "all fields" but **excluding ilm phase label section**.
- incompatible fields and same family fields view (custom, ecs compliant
and all fields view is unavailable in history tab)

![serverless_after_5](https://github.com/user-attachments/assets/d8fdd48f-63f2-48f2-8ede-3613bffaa157)

### Legacy check result view **(NEW)**
- before this PR went to production
elastic#185025 check result data
contained information allowing to recreate detailed view of incompatible
fields from markdown only (without same family fields)
- we recreate incompatible field tables in degraded view from markdown
- same family tab is permanently disabled with an explanation tooltip
- action buttons still work as is for incompatible fields view
- index stats panel is showing as for non-legacy result

![SCR-20241009-lkhi](https://github.com/user-attachments/assets/10adee1c-c11a-428a-9c56-ecc20a37f97f)

### No results

![SCR-20241009-ljwg](https://github.com/user-attachments/assets/8bf48778-98d6-4a96-a713-b49d4cc5165a)

### Loading view

![serverless_after_9](https://github.com/user-attachments/assets/5ba1f2cc-cbd9-4cfa-964c-962be150016f)

### Error view

![serverless_after_10](https://github.com/user-attachments/assets/b5c33ded-4ee5-46ff-9e13-f9e5dfc7546e)

(cherry picked from commit e5f7739)
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Oct 11, 2024
…) (#195962)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Secuity Solution][DQD] add historical results (Phase 1)
(#191898)](#191898)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Karen
Grigoryan","email":"karen.grigoryan@elastic.co"},"sourceCommit":{"committedDate":"2024-10-11T16:14:11Z","message":"[Secuity
Solution][DQD] add historical results (Phase 1) (#191898)\n\naddresses
#185882 \r\n\r\nleverages changes introduced
in\r\nhttps://github.com//pull/188468\r\n\r\n# Data
Quality Dashboard Historical Results (Phase 1)\r\n\r\nThis PR introduces
new functionality to the Data Quality Dashboard\r\n\r\nHistory tab
(new):\r\n- view last 30 days of check results by default:\r\n- filter
by historical checks by outcome (PASS/FAIL/ALL)\r\n- paginate all
results (10 per page by default)\r\n- each result can be viewed in
individually and independently\r\nexpandable/collapsible accordion panel
(collapsed by default)\r\n- each result contains an extended index stats
panel with (custom, ecs\r\nand all fields counts)\r\n- each result
contains index properties tabs (incompatible and same\r\nfamily)\r\n-
check now checks and redirects to latest check tab with latest
check\r\ninfo\r\n- switching from initial historical tab to latest check
tab triggers\r\nlatest check\r\n- subsequent switching back and forth
between already open history or\r\nlatest check tabs doesn't trigger a
check\r\n- legacy data (before release of
this\r\nhttps://github.com//pull/185025) is supported
with\r\ndegraded view (same family tab is disabled with warning
tooltip),\r\nincompatible tab tables are statically rendered from
markdown\r\n\r\nLatest checks list view (changes):\r\n- remove check
index button icon from list view\r\n- add historical results button icon
instead of check index button\r\n- historical results button icon
directly opens history tab without\r\ngoing through latest
check\r\n\r\n# UI Changes (before/after):\r\n\r\n## ESS
Changes\r\n\r\n### Latest check expand icon\r\n- expand icon is replaced
with check now icon (functionality is the\r\nsame)\r\n- tooltip text is
updated\r\n- this new icon is still opening the index check flyout tab
(latest\r\ncheck
tab)\r\n\r\n![ess_before_after_0](https://github.com/user-attachments/assets/795af721-6867-4f56-882e-2a0f52793560)\r\n\r\n###
Historical check icon **(NEW)**\r\n- inline check now functionality is
removed\r\n- view history icon is added in its stead to open a flyout
with history\r\ntab\r\n- tooltip text is
updated\r\n\r\n![ess_before_after_1](https://github.com/user-attachments/assets/7f2c6009-35c3-488c-87ac-3048f4bded7b)\r\n\r\n###
Flyout Header\r\n- \"checked at\" subheader is now shorter (milliseconds
are removed)\r\n- Tabline with Latest check and History tabs is added
**(NEW)**\r\n\r\n![ess_before_after_2](https://github.com/user-attachments/assets/728ff743-500e-435a-a07e-4287647a0af5)\r\n\r\n###
History tab **(NEW)**\r\n- top left: filter by check outcome\r\n- top
right: filter by date range\r\n- list of checks collapsed by default
(individually separately\r\ncontrolled, multiple can be open at a
time)\r\n- pagination (10,25,50). 10 by
default\r\n\r\n![ess_after_3](https://github.com/user-attachments/assets/36fc0cee-b103-483d-ba79-d583bba89acf)\r\n\r\n###
Individual check result view **(NEW)**\r\n- topline: extended index
stats including new \"custom\", \"ecs compliant\"\r\n& \"all
fields\".\r\n- incompatible fields and same family fields view (custom,
ecs compliant\r\nand all fields view is unavailable in history
tab)\r\n\r\n![ess_after_4](https://github.com/user-attachments/assets/57e6d5a1-1470-4c4b-9272-ccc872d80dc5)\r\n\r\n###
Legacy check result view **(NEW)**\r\n- before this PR went to
production\r\nhttps://github.com//pull/185025 check result
data\r\ncontained information allowing to recreate detailed view of
incompatible\r\nfields from markdown only (without same family
fields)\r\n- we recreate incompatible field tables in degraded view from
markdown\r\n- same family tab is permanently disabled with an
explanation tooltip\r\n- action buttons still work as is for
incompatible fields view\r\n- index stats panel is showing as for
non-legacy
result\r\n\r\n![SCR-20241009-lmcu](https://github.com/user-attachments/assets/cd11435e-7335-40f3-a0b8-4e5c6bcc2f38)\r\n\r\n###
No
results\r\n\r\n![SCR-20241009-llzw](https://github.com/user-attachments/assets/a942ce8e-6e0e-46d3-9104-c30648a18208)\r\n\r\n###
Loading
view\r\n\r\n![ess_after_8](https://github.com/user-attachments/assets/1411ccc2-4978-41f6-a02d-2ca404a01c16)\r\n\r\n###
Error
view\r\n\r\n![ess_after_9](https://github.com/user-attachments/assets/adc80e19-0005-46f9-a667-ffd3bf8ecb4f)\r\n\r\n##
Serverless Changes\r\n### Empty checks result badge **(FIX)**\r\n-
**previously empty pattern check result badge was marked as
`PASS`\r\nwhich was incorrect. It was
removed.**\r\n\r\n![serverless_before_after_0](https://github.com/user-attachments/assets/67e02e9c-cd7f-46d7-9b7a-9bdaa0abfc6c)\r\n\r\n###
Latest check expand icon\r\n- expand icon is replaced with check now
icon (functionality is the\r\nsame)\r\n- tooltip text is updated\r\n-
this new icon is still opening the index check flyout tab
(latest\r\ncheck
tab)\r\n\r\n![serverless_before_after_1](https://github.com/user-attachments/assets/dfac9aad-158b-4863-b719-47d50b06bda3)\r\n\r\n###
Historical check icon **(NEW)**\r\n- inline check now functionality is
removed\r\n- view history icon is added in its stead to open a flyout
with history\r\ntab\r\n- tooltip text is
updated\r\n\r\n![serverless_before_after_2](https://github.com/user-attachments/assets/c688c28c-2d86-4669-a9bb-ffc297d21bbf)\r\n\r\n###
Flyout Header and Body Topline\r\n- \"checked at\" subheader is now
shorter (milliseconds are removed)\r\n- Tabline with Latest check and
History tabs is added **(NEW)**\r\n- **Index Stats Panel is now also
showing here just like in latest check\r\ntab (but without phase label
as ilm is not available in
serverless)**\r\n**(NEW)**\r\n\r\n![serverless_before_after_3](https://github.com/user-attachments/assets/c3ae4160-d07c-4049-b8b4-4b66faa50320)\r\n\r\n###
History tab **(NEW)**\r\n- top left: filter by check outcome\r\n- top
right: filter by date range\r\n- list of checks collapsed by default
(individually separately\r\ncontrolled, multiple can be open at a
time)\r\n- pagination (10,25,50). 10 by
default\r\n\r\n![serverless_after_4](https://github.com/user-attachments/assets/8b767de3-1ab1-4b9f-b0b8-84754a3776ae)\r\n\r\n###
Individual check result view **(NEW)**\r\n- topline: extended index
stats including new \"custom\", \"ecs compliant\"\r\n& \"all fields\"
but **excluding ilm phase label section**.\r\n- incompatible fields and
same family fields view (custom, ecs compliant\r\nand all fields view is
unavailable in history
tab)\r\n\r\n![serverless_after_5](https://github.com/user-attachments/assets/d8fdd48f-63f2-48f2-8ede-3613bffaa157)\r\n\r\n###
Legacy check result view **(NEW)**\r\n- before this PR went to
production\r\nhttps://github.com//pull/185025 check result
data\r\ncontained information allowing to recreate detailed view of
incompatible\r\nfields from markdown only (without same family
fields)\r\n- we recreate incompatible field tables in degraded view from
markdown\r\n- same family tab is permanently disabled with an
explanation tooltip\r\n- action buttons still work as is for
incompatible fields view\r\n- index stats panel is showing as for
non-legacy
result\r\n\r\n![SCR-20241009-lkhi](https://github.com/user-attachments/assets/10adee1c-c11a-428a-9c56-ecc20a37f97f)\r\n\r\n###
No
results\r\n\r\n![SCR-20241009-ljwg](https://github.com/user-attachments/assets/8bf48778-98d6-4a96-a713-b49d4cc5165a)\r\n\r\n###
Loading
view\r\n\r\n![serverless_after_9](https://github.com/user-attachments/assets/5ba1f2cc-cbd9-4cfa-964c-962be150016f)\r\n\r\n###
Error
view\r\n\r\n![serverless_after_10](https://github.com/user-attachments/assets/b5c33ded-4ee5-46ff-9e13-f9e5dfc7546e)","sha":"e5f7739b263902bd01aca9fa902be7cc5181e855","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["v9.0.0","Team:Threat
Hunting","release_note:feature","Team:Threat
Hunting:Explore","backport:prev-minor","ci:cloud-deploy","ci:cloud-redeploy","ci:cloud-persist-deployment","ci:build-serverless-image","ci:project-deploy-security","ci:project-persist-deployment","ci:project-redeploy","v8.16.0"],"title":"[Secuity
Solution][DQD] add historical results (Phase 1)
","number":191898,"url":"https://github.com/elastic/kibana/pull/191898","mergeCommit":{"message":"[Secuity
Solution][DQD] add historical results (Phase 1) (#191898)\n\naddresses
#185882 \r\n\r\nleverages changes introduced
in\r\nhttps://github.com//pull/188468\r\n\r\n# Data
Quality Dashboard Historical Results (Phase 1)\r\n\r\nThis PR introduces
new functionality to the Data Quality Dashboard\r\n\r\nHistory tab
(new):\r\n- view last 30 days of check results by default:\r\n- filter
by historical checks by outcome (PASS/FAIL/ALL)\r\n- paginate all
results (10 per page by default)\r\n- each result can be viewed in
individually and independently\r\nexpandable/collapsible accordion panel
(collapsed by default)\r\n- each result contains an extended index stats
panel with (custom, ecs\r\nand all fields counts)\r\n- each result
contains index properties tabs (incompatible and same\r\nfamily)\r\n-
check now checks and redirects to latest check tab with latest
check\r\ninfo\r\n- switching from initial historical tab to latest check
tab triggers\r\nlatest check\r\n- subsequent switching back and forth
between already open history or\r\nlatest check tabs doesn't trigger a
check\r\n- legacy data (before release of
this\r\nhttps://github.com//pull/185025) is supported
with\r\ndegraded view (same family tab is disabled with warning
tooltip),\r\nincompatible tab tables are statically rendered from
markdown\r\n\r\nLatest checks list view (changes):\r\n- remove check
index button icon from list view\r\n- add historical results button icon
instead of check index button\r\n- historical results button icon
directly opens history tab without\r\ngoing through latest
check\r\n\r\n# UI Changes (before/after):\r\n\r\n## ESS
Changes\r\n\r\n### Latest check expand icon\r\n- expand icon is replaced
with check now icon (functionality is the\r\nsame)\r\n- tooltip text is
updated\r\n- this new icon is still opening the index check flyout tab
(latest\r\ncheck
tab)\r\n\r\n![ess_before_after_0](https://github.com/user-attachments/assets/795af721-6867-4f56-882e-2a0f52793560)\r\n\r\n###
Historical check icon **(NEW)**\r\n- inline check now functionality is
removed\r\n- view history icon is added in its stead to open a flyout
with history\r\ntab\r\n- tooltip text is
updated\r\n\r\n![ess_before_after_1](https://github.com/user-attachments/assets/7f2c6009-35c3-488c-87ac-3048f4bded7b)\r\n\r\n###
Flyout Header\r\n- \"checked at\" subheader is now shorter (milliseconds
are removed)\r\n- Tabline with Latest check and History tabs is added
**(NEW)**\r\n\r\n![ess_before_after_2](https://github.com/user-attachments/assets/728ff743-500e-435a-a07e-4287647a0af5)\r\n\r\n###
History tab **(NEW)**\r\n- top left: filter by check outcome\r\n- top
right: filter by date range\r\n- list of checks collapsed by default
(individually separately\r\ncontrolled, multiple can be open at a
time)\r\n- pagination (10,25,50). 10 by
default\r\n\r\n![ess_after_3](https://github.com/user-attachments/assets/36fc0cee-b103-483d-ba79-d583bba89acf)\r\n\r\n###
Individual check result view **(NEW)**\r\n- topline: extended index
stats including new \"custom\", \"ecs compliant\"\r\n& \"all
fields\".\r\n- incompatible fields and same family fields view (custom,
ecs compliant\r\nand all fields view is unavailable in history
tab)\r\n\r\n![ess_after_4](https://github.com/user-attachments/assets/57e6d5a1-1470-4c4b-9272-ccc872d80dc5)\r\n\r\n###
Legacy check result view **(NEW)**\r\n- before this PR went to
production\r\nhttps://github.com//pull/185025 check result
data\r\ncontained information allowing to recreate detailed view of
incompatible\r\nfields from markdown only (without same family
fields)\r\n- we recreate incompatible field tables in degraded view from
markdown\r\n- same family tab is permanently disabled with an
explanation tooltip\r\n- action buttons still work as is for
incompatible fields view\r\n- index stats panel is showing as for
non-legacy
result\r\n\r\n![SCR-20241009-lmcu](https://github.com/user-attachments/assets/cd11435e-7335-40f3-a0b8-4e5c6bcc2f38)\r\n\r\n###
No
results\r\n\r\n![SCR-20241009-llzw](https://github.com/user-attachments/assets/a942ce8e-6e0e-46d3-9104-c30648a18208)\r\n\r\n###
Loading
view\r\n\r\n![ess_after_8](https://github.com/user-attachments/assets/1411ccc2-4978-41f6-a02d-2ca404a01c16)\r\n\r\n###
Error
view\r\n\r\n![ess_after_9](https://github.com/user-attachments/assets/adc80e19-0005-46f9-a667-ffd3bf8ecb4f)\r\n\r\n##
Serverless Changes\r\n### Empty checks result badge **(FIX)**\r\n-
**previously empty pattern check result badge was marked as
`PASS`\r\nwhich was incorrect. It was
removed.**\r\n\r\n![serverless_before_after_0](https://github.com/user-attachments/assets/67e02e9c-cd7f-46d7-9b7a-9bdaa0abfc6c)\r\n\r\n###
Latest check expand icon\r\n- expand icon is replaced with check now
icon (functionality is the\r\nsame)\r\n- tooltip text is updated\r\n-
this new icon is still opening the index check flyout tab
(latest\r\ncheck
tab)\r\n\r\n![serverless_before_after_1](https://github.com/user-attachments/assets/dfac9aad-158b-4863-b719-47d50b06bda3)\r\n\r\n###
Historical check icon **(NEW)**\r\n- inline check now functionality is
removed\r\n- view history icon is added in its stead to open a flyout
with history\r\ntab\r\n- tooltip text is
updated\r\n\r\n![serverless_before_after_2](https://github.com/user-attachments/assets/c688c28c-2d86-4669-a9bb-ffc297d21bbf)\r\n\r\n###
Flyout Header and Body Topline\r\n- \"checked at\" subheader is now
shorter (milliseconds are removed)\r\n- Tabline with Latest check and
History tabs is added **(NEW)**\r\n- **Index Stats Panel is now also
showing here just like in latest check\r\ntab (but without phase label
as ilm is not available in
serverless)**\r\n**(NEW)**\r\n\r\n![serverless_before_after_3](https://github.com/user-attachments/assets/c3ae4160-d07c-4049-b8b4-4b66faa50320)\r\n\r\n###
History tab **(NEW)**\r\n- top left: filter by check outcome\r\n- top
right: filter by date range\r\n- list of checks collapsed by default
(individually separately\r\ncontrolled, multiple can be open at a
time)\r\n- pagination (10,25,50). 10 by
default\r\n\r\n![serverless_after_4](https://github.com/user-attachments/assets/8b767de3-1ab1-4b9f-b0b8-84754a3776ae)\r\n\r\n###
Individual check result view **(NEW)**\r\n- topline: extended index
stats including new \"custom\", \"ecs compliant\"\r\n& \"all fields\"
but **excluding ilm phase label section**.\r\n- incompatible fields and
same family fields view (custom, ecs compliant\r\nand all fields view is
unavailable in history
tab)\r\n\r\n![serverless_after_5](https://github.com/user-attachments/assets/d8fdd48f-63f2-48f2-8ede-3613bffaa157)\r\n\r\n###
Legacy check result view **(NEW)**\r\n- before this PR went to
production\r\nhttps://github.com//pull/185025 check result
data\r\ncontained information allowing to recreate detailed view of
incompatible\r\nfields from markdown only (without same family
fields)\r\n- we recreate incompatible field tables in degraded view from
markdown\r\n- same family tab is permanently disabled with an
explanation tooltip\r\n- action buttons still work as is for
incompatible fields view\r\n- index stats panel is showing as for
non-legacy
result\r\n\r\n![SCR-20241009-lkhi](https://github.com/user-attachments/assets/10adee1c-c11a-428a-9c56-ecc20a37f97f)\r\n\r\n###
No
results\r\n\r\n![SCR-20241009-ljwg](https://github.com/user-attachments/assets/8bf48778-98d6-4a96-a713-b49d4cc5165a)\r\n\r\n###
Loading
view\r\n\r\n![serverless_after_9](https://github.com/user-attachments/assets/5ba1f2cc-cbd9-4cfa-964c-962be150016f)\r\n\r\n###
Error
view\r\n\r\n![serverless_after_10](https://github.com/user-attachments/assets/b5c33ded-4ee5-46ff-9e13-f9e5dfc7546e)","sha":"e5f7739b263902bd01aca9fa902be7cc5181e855"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/191898","number":191898,"mergeCommit":{"message":"[Secuity
Solution][DQD] add historical results (Phase 1) (#191898)\n\naddresses
#185882 \r\n\r\nleverages changes introduced
in\r\nhttps://github.com//pull/188468\r\n\r\n# Data
Quality Dashboard Historical Results (Phase 1)\r\n\r\nThis PR introduces
new functionality to the Data Quality Dashboard\r\n\r\nHistory tab
(new):\r\n- view last 30 days of check results by default:\r\n- filter
by historical checks by outcome (PASS/FAIL/ALL)\r\n- paginate all
results (10 per page by default)\r\n- each result can be viewed in
individually and independently\r\nexpandable/collapsible accordion panel
(collapsed by default)\r\n- each result contains an extended index stats
panel with (custom, ecs\r\nand all fields counts)\r\n- each result
contains index properties tabs (incompatible and same\r\nfamily)\r\n-
check now checks and redirects to latest check tab with latest
check\r\ninfo\r\n- switching from initial historical tab to latest check
tab triggers\r\nlatest check\r\n- subsequent switching back and forth
between already open history or\r\nlatest check tabs doesn't trigger a
check\r\n- legacy data (before release of
this\r\nhttps://github.com//pull/185025) is supported
with\r\ndegraded view (same family tab is disabled with warning
tooltip),\r\nincompatible tab tables are statically rendered from
markdown\r\n\r\nLatest checks list view (changes):\r\n- remove check
index button icon from list view\r\n- add historical results button icon
instead of check index button\r\n- historical results button icon
directly opens history tab without\r\ngoing through latest
check\r\n\r\n# UI Changes (before/after):\r\n\r\n## ESS
Changes\r\n\r\n### Latest check expand icon\r\n- expand icon is replaced
with check now icon (functionality is the\r\nsame)\r\n- tooltip text is
updated\r\n- this new icon is still opening the index check flyout tab
(latest\r\ncheck
tab)\r\n\r\n![ess_before_after_0](https://github.com/user-attachments/assets/795af721-6867-4f56-882e-2a0f52793560)\r\n\r\n###
Historical check icon **(NEW)**\r\n- inline check now functionality is
removed\r\n- view history icon is added in its stead to open a flyout
with history\r\ntab\r\n- tooltip text is
updated\r\n\r\n![ess_before_after_1](https://github.com/user-attachments/assets/7f2c6009-35c3-488c-87ac-3048f4bded7b)\r\n\r\n###
Flyout Header\r\n- \"checked at\" subheader is now shorter (milliseconds
are removed)\r\n- Tabline with Latest check and History tabs is added
**(NEW)**\r\n\r\n![ess_before_after_2](https://github.com/user-attachments/assets/728ff743-500e-435a-a07e-4287647a0af5)\r\n\r\n###
History tab **(NEW)**\r\n- top left: filter by check outcome\r\n- top
right: filter by date range\r\n- list of checks collapsed by default
(individually separately\r\ncontrolled, multiple can be open at a
time)\r\n- pagination (10,25,50). 10 by
default\r\n\r\n![ess_after_3](https://github.com/user-attachments/assets/36fc0cee-b103-483d-ba79-d583bba89acf)\r\n\r\n###
Individual check result view **(NEW)**\r\n- topline: extended index
stats including new \"custom\", \"ecs compliant\"\r\n& \"all
fields\".\r\n- incompatible fields and same family fields view (custom,
ecs compliant\r\nand all fields view is unavailable in history
tab)\r\n\r\n![ess_after_4](https://github.com/user-attachments/assets/57e6d5a1-1470-4c4b-9272-ccc872d80dc5)\r\n\r\n###
Legacy check result view **(NEW)**\r\n- before this PR went to
production\r\nhttps://github.com//pull/185025 check result
data\r\ncontained information allowing to recreate detailed view of
incompatible\r\nfields from markdown only (without same family
fields)\r\n- we recreate incompatible field tables in degraded view from
markdown\r\n- same family tab is permanently disabled with an
explanation tooltip\r\n- action buttons still work as is for
incompatible fields view\r\n- index stats panel is showing as for
non-legacy
result\r\n\r\n![SCR-20241009-lmcu](https://github.com/user-attachments/assets/cd11435e-7335-40f3-a0b8-4e5c6bcc2f38)\r\n\r\n###
No
results\r\n\r\n![SCR-20241009-llzw](https://github.com/user-attachments/assets/a942ce8e-6e0e-46d3-9104-c30648a18208)\r\n\r\n###
Loading
view\r\n\r\n![ess_after_8](https://github.com/user-attachments/assets/1411ccc2-4978-41f6-a02d-2ca404a01c16)\r\n\r\n###
Error
view\r\n\r\n![ess_after_9](https://github.com/user-attachments/assets/adc80e19-0005-46f9-a667-ffd3bf8ecb4f)\r\n\r\n##
Serverless Changes\r\n### Empty checks result badge **(FIX)**\r\n-
**previously empty pattern check result badge was marked as
`PASS`\r\nwhich was incorrect. It was
removed.**\r\n\r\n![serverless_before_after_0](https://github.com/user-attachments/assets/67e02e9c-cd7f-46d7-9b7a-9bdaa0abfc6c)\r\n\r\n###
Latest check expand icon\r\n- expand icon is replaced with check now
icon (functionality is the\r\nsame)\r\n- tooltip text is updated\r\n-
this new icon is still opening the index check flyout tab
(latest\r\ncheck
tab)\r\n\r\n![serverless_before_after_1](https://github.com/user-attachments/assets/dfac9aad-158b-4863-b719-47d50b06bda3)\r\n\r\n###
Historical check icon **(NEW)**\r\n- inline check now functionality is
removed\r\n- view history icon is added in its stead to open a flyout
with history\r\ntab\r\n- tooltip text is
updated\r\n\r\n![serverless_before_after_2](https://github.com/user-attachments/assets/c688c28c-2d86-4669-a9bb-ffc297d21bbf)\r\n\r\n###
Flyout Header and Body Topline\r\n- \"checked at\" subheader is now
shorter (milliseconds are removed)\r\n- Tabline with Latest check and
History tabs is added **(NEW)**\r\n- **Index Stats Panel is now also
showing here just like in latest check\r\ntab (but without phase label
as ilm is not available in
serverless)**\r\n**(NEW)**\r\n\r\n![serverless_before_after_3](https://github.com/user-attachments/assets/c3ae4160-d07c-4049-b8b4-4b66faa50320)\r\n\r\n###
History tab **(NEW)**\r\n- top left: filter by check outcome\r\n- top
right: filter by date range\r\n- list of checks collapsed by default
(individually separately\r\ncontrolled, multiple can be open at a
time)\r\n- pagination (10,25,50). 10 by
default\r\n\r\n![serverless_after_4](https://github.com/user-attachments/assets/8b767de3-1ab1-4b9f-b0b8-84754a3776ae)\r\n\r\n###
Individual check result view **(NEW)**\r\n- topline: extended index
stats including new \"custom\", \"ecs compliant\"\r\n& \"all fields\"
but **excluding ilm phase label section**.\r\n- incompatible fields and
same family fields view (custom, ecs compliant\r\nand all fields view is
unavailable in history
tab)\r\n\r\n![serverless_after_5](https://github.com/user-attachments/assets/d8fdd48f-63f2-48f2-8ede-3613bffaa157)\r\n\r\n###
Legacy check result view **(NEW)**\r\n- before this PR went to
production\r\nhttps://github.com//pull/185025 check result
data\r\ncontained information allowing to recreate detailed view of
incompatible\r\nfields from markdown only (without same family
fields)\r\n- we recreate incompatible field tables in degraded view from
markdown\r\n- same family tab is permanently disabled with an
explanation tooltip\r\n- action buttons still work as is for
incompatible fields view\r\n- index stats panel is showing as for
non-legacy
result\r\n\r\n![SCR-20241009-lkhi](https://github.com/user-attachments/assets/10adee1c-c11a-428a-9c56-ecc20a37f97f)\r\n\r\n###
No
results\r\n\r\n![SCR-20241009-ljwg](https://github.com/user-attachments/assets/8bf48778-98d6-4a96-a713-b49d4cc5165a)\r\n\r\n###
Loading
view\r\n\r\n![serverless_after_9](https://github.com/user-attachments/assets/5ba1f2cc-cbd9-4cfa-964c-962be150016f)\r\n\r\n###
Error
view\r\n\r\n![serverless_after_10](https://github.com/user-attachments/assets/b5c33ded-4ee5-46ff-9e13-f9e5dfc7546e)","sha":"e5f7739b263902bd01aca9fa902be7cc5181e855"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Karen Grigoryan <karen.grigoryan@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) ci:build-serverless-image ci:cloud-deploy Create or update a Cloud deployment ci:cloud-persist-deployment Persist cloud deployment indefinitely ci:cloud-redeploy Always create a new Cloud deployment ci:project-deploy-security Create a Security Serverless Project ci:project-persist-deployment Persist project deployment indefinitely ci:project-redeploy Always create a new Cloud project release_note:feature Makes this part of the condensed release notes Team:Threat Hunting:Explore Team:Threat Hunting Security Solution Threat Hunting Team v8.16.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants