[Index Management] Add indices list columns to the extensions service (…

…elastic#174785)

## Summary

This PR adds a function to the extensions service that allows to add new
columns to the indices list. The new column can display information that
is already available in the index data or a new index data enricher can
be registered. See
[readme](https://github.com/elastic/kibana/blob/b72c03e9b022e30476969b18b9c0ca5344c5d67c/x-pack/plugins/index_management/README.md)
for more details.

### Screenshot
Example of adding an ILM phase column 

elastic@86864fe
<img width="1127" alt="Screenshot 2024-01-15 at 18 09 43"
src="https://github.com/elastic/kibana/assets/6585477/9502a90b-43e8-45a0-aa44-515585865452">




### Checklist

Delete any items that are not applicable to this PR.

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)


### Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.

When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:

| Risk | Probability | Severity | Mitigation/Notes |

|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces&mdash;unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes&mdash;Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |


### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

x-pack/plugins/index_management/README.md

@@ -5,6 +5,16 @@ This service is exposed from the Index Management setup contract and can be used
 - `addBanner(banner: any)`: adds a banner on top of the indices list, for example when some indices run into an ILM issue
 - `addFilter(filter: any)`: adds a filter to the indices list, for example to filter indices managed by ILM 
 - `addToggle(toggle: any)`: adds a toggle to the indices list, for example to display hidden indices
+- `addColumn(column: IndicesListColumn)`: adds a column to the indices list, for example to display an ILM phase
+- `setEmptyListContent(content: EmptyListContent)`: replaces the default empty prompt displayed when there are no indices in the indices list. The empty list content has the following interface:
+
+```ts
+export interface EmptyListContent {
+  renderContent: (args: {
+    createIndexButton: ReturnType<FunctionComponent>;
+  }) => ReturnType<FunctionComponent>;
+}
+```
 
 #### Extensions to the indices list and the index details page
 - `addAction(action: any)`: adds an option to the "manage index" menu, for example to add an ILM policy to the index
@@ -44,6 +54,14 @@ interface IndexContent {
 ```
 - `setIndexMappingsContent(content: IndexContent)`: adds content to the mappings tab of the index details page. The content is displayed in the right bottom corner, below the mappings docs link. 
 
+## Index data enrichers
+The extensions service that allows to render additional UI elements in the indices list and on the index details page often
+relies on additional index data that is not available by default. To make these additional data available in the response of 
+the `GET /indices` request, an index data enricher can be registered. A data enricher is essentially an extra request that is
+done for the array of indices and the information is added to the response. Currently, 3 data enrichers are registered 
+by the ILM, Rollup and CCR plugins. Before adding a data enricher, the cost of the additional request should be taken 
+in consideration (see [this file](https://github.com/elastic/kibana/blob/main/x-pack/plugins/index_management/server/services/index_data_enricher.ts) for more details).
+
 ## Indices tab
 
 ### Quick steps for testing

x-pack/plugins/index_management/__jest__/client_integration/home/indices_tab.test.tsx

@@ -25,10 +25,14 @@ jest.mock('@elastic/eui/lib/components/search_bar/search_box', () => {
 import React from 'react';
 import { act } from 'react-dom/test-utils';
 
-import { API_BASE_PATH, INTERNAL_API_BASE_PATH } from '../../../common';
+import { API_BASE_PATH, Index, INTERNAL_API_BASE_PATH } from '../../../common';
 import { setupEnvironment } from '../helpers';
 import { IndicesTestBed, setup } from './indices_tab.helpers';
-import { createDataStreamPayload, createNonDataStreamIndex } from './data_streams_tab.helpers';
+import {
+  createDataStreamBackingIndex,
+  createDataStreamPayload,
+  createNonDataStreamIndex,
+} from './data_streams_tab.helpers';
 
 import { createMemoryHistory } from 'history';
 import {
@@ -81,30 +85,8 @@ describe('<IndexManagementHome />', () => {
   describe('data stream column', () => {
     beforeEach(async () => {
       httpRequestsMockHelpers.setLoadIndicesResponse([
-        {
-          health: '',
-          status: '',
-          primary: '',
-          replica: '',
-          documents: '',
-          documents_deleted: '',
-          size: '',
-          primary_size: '',
-          name: 'data-stream-index',
-          data_stream: 'dataStream1',
-        },
-        {
-          health: '',
-          status: '',
-          primary: '',
-          replica: '',
-          documents: '',
-          documents_deleted: '',
-          size: '',
-          primary_size: '',
-          name: 'no-data-stream-index',
-          data_stream: null,
-        },
+        createDataStreamBackingIndex('data-stream-index', 'dataStream1'),
+        createNonDataStreamIndex('no-data-stream-index'),
       ]);
 
       // The detail panel should still appear even if there are no data streams.
@@ -195,26 +177,6 @@ describe('<IndexManagementHome />', () => {
       expect(testBed.exists('createIndexMessage')).toBe(true);
     });
 
-    it('displays an empty list content if set via extensions service', async () => {
-      httpRequestsMockHelpers.setLoadIndicesResponse([]);
-      await act(async () => {
-        testBed = await setup(httpSetup, {
-          services: {
-            extensionsService: {
-              _emptyListContent: {
-                renderContent: () => {
-                  return <div>Empty list content</div>;
-                },
-              },
-            },
-          },
-        });
-      });
-      testBed.component.update();
-
-      expect(testBed.component.text()).toContain('Empty list content');
-    });
-
     it('renders "no indices found" prompt for search', async () => {
       const { find, component, exists } = testBed;
       await act(async () => {
@@ -530,4 +492,70 @@ describe('<IndexManagementHome />', () => {
       expect(httpSetup.get).toHaveBeenNthCalledWith(2, '/api/index_management/indices');
     });
   });
+
+  describe('extensions service', () => {
+    it('displays an empty list content if set via extensions service', async () => {
+      httpRequestsMockHelpers.setLoadIndicesResponse([]);
+      await act(async () => {
+        testBed = await setup(httpSetup, {
+          services: {
+            extensionsService: {
+              _emptyListContent: {
+                renderContent: () => {
+                  return <div>Empty list content</div>;
+                },
+              },
+            },
+          },
+        });
+      });
+      testBed.component.update();
+
+      expect(testBed.component.text()).toContain('Empty list content');
+    });
+
+    it('renders additional columns registered via extensions service', async () => {
+      httpRequestsMockHelpers.setLoadIndicesResponse([
+        {
+          ...createNonDataStreamIndex('test-index'),
+          ilm: {
+            phase: 'hot phase',
+            managed: true,
+          },
+        },
+      ]);
+      await act(async () => {
+        testBed = await setup(httpSetup, {
+          services: {
+            extensionsService: {
+              _columns: [
+                {
+                  fieldName: 'ilm.phase',
+                  label: 'ILM column 1',
+                  order: 55,
+                },
+                {
+                  fieldName: 'ilm.managed',
+                  label: 'ILM column 2',
+                  order: 56,
+                  render: (index: Index) => {
+                    if (index.ilm?.managed) {
+                      return <div>ILM managed</div>;
+                    }
+                  },
+                },
+              ],
+            },
+          },
+        });
+      });
+      testBed.component.update();
+
+      const text = testBed.component.text();
+      expect(text).toContain('ILM column 1');
+      expect(text).toContain('hot phase');
+      expect(text).toContain('ILM column 2');
+      expect(text).toContain('ILM managed');
+    });
+  });
 });