[APM] Initial diagnostics tool #157500

sorenlouv · 2023-05-12T14:47:39Z

Pitch: https://github.com/elastic/apm-dev/issues/872 (internal)

Introduces the Diagnostics tool under a hidden page: /app/apm/diagnostics. The Diagnostics tool can help find problems with missing index templates, data streams with incorrect index templates and indices with incorrect mappings.

It will be released in 8.9 but customers can take advantage of it as soon as this PR is merged. They can export a diagnostics bundle from their system by running this script:

node x-pack/plugins/apm/scripts/diagnostics_bundle \ 
--kbHost https://foo.kb.europe-west2.gcp.elastic-cloud.com:9243 \
--esHost https://foo.es.europe-west2.gcp.elastic-cloud.com:9243 \
--username elastic
--password changeme

apmmachine · 2023-05-12T14:47:54Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

/oblt-deploy : Deploy a Kibana instance using the Observability test environments.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

yngrdyn · 2023-05-15T08:24:08Z

/oblt-deploy

x-pack/plugins/apm/common/diagnostics/get_default_index_template_names.ts

x-pack/plugins/apm/server/lib/helpers/create_es_client/create_apm_event_client/index.ts

...gins/apm/server/routes/diagnostics/index_templates/get_default_apm_index_templates_states.ts

x-pack/plugins/apm/public/components/app/diagnostics/index.tsx

yngrdyn · 2023-06-01T11:52:05Z

x-pack/plugins/apm/public/components/app/diagnostics/index_pattern_settings_tab.tsx

+import { FETCH_STATUS } from '../../../hooks/use_fetcher';
+import { useDiagnosticsContext } from './context/use_diagnostics';
+
+type DiagnosticsBundle = APIReturnType<'GET /internal/apm/diagnostics'>;


Would it be useful just to export this type under app/diagnostics folder? I've seen this line in multiple files

x-pack/plugins/apm/public/components/app/diagnostics/summary_tab/data_streams_status.tsx

yngrdyn · 2023-06-01T12:19:02Z

x-pack/plugins/apm/scripts/diagnostics_bundle/main.ts

How can I execute this diagnostics bundle?
Could we add an example in our dev_docs?

yngrdyn

LGTM, just some minor stuff 🎩

kpatticha · 2023-06-01T13:30:44Z

/oblt-deploy

sorenlouv · 2023-06-02T08:11:57Z

/oblt-deploy

kpatticha

LOVE THIS. I edited the pipeline as we have seen that it can be missing and causing issues and the tool reported it 🥳

One "minor" issue that I experienced. Locally without any data, I override the apm-indices with an index that doesn't exist. The diagnostic endpoint failed with 500.

More stuff

Co-authored-by: Yngrid Coello <yngrdyn@gmail.com>

kibana-ci · 2023-06-05T22:42:01Z

💚 Build Succeeded

Buildkite Build
Commit: aad75bb

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`apm`	1355	1369	+14

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`apm`	3.5MB	3.5MB	+13.8KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
`apm`	112	113	+1

Unknown metric groups

ESLint disabled in files

id	before	after	diff
`apm`	11	13	+2

ESLint disabled line counts

id	before	after	diff
`enterpriseSearch`	19	21	+2
`securitySolution`	414	418	+4
total			+6

Total ESLint disabled count

id	before	after	diff
`apm`	84	86	+2
`enterpriseSearch`	20	22	+2
`securitySolution`	498	502	+4
total			+8

History

💔 Build #132541 failed 26df392
💔 Build #132121 failed e91da3220627d4ccbe913e5695019aa89389632b
💔 Build #132071 failed 77177f0d9fb275f1f4ba3ebd22225b18e4c43533
💚 Build #132028 succeeded 11d2a4a5dfa5bceef6334b99161d9b335e7e4a99
💔 Build #132014 failed ce24b5f8ab59b56957c7ccfa3d45a9c1c26b1c10

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

sorenlouv · 2023-06-05T23:13:13Z

💚 All backports created successfully

Status	Branch	Result
✅	8.8
✅	8.7

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

Pitch: elastic/apm-dev#872 (_internal_) Introduces the Diagnostics tool under a hidden page: `/app/apm/diagnostics`. The Diagnostics tool can help find problems with missing index templates, data streams with incorrect index templates and indices with incorrect mappings. It will be released in 8.9 but customers can take advantage of it as soon as this PR is merged. They can export a diagnostics bundle from their system by running this script: ``` node x-pack/plugins/apm/scripts/diagnostics_bundle \ --kbHost https://foo.kb.europe-west2.gcp.elastic-cloud.com:9243 \ --esHost https://foo.es.europe-west2.gcp.elastic-cloud.com:9243 \ --username elastic --password changeme ``` ![image](https://github.com/elastic/kibana/assets/209966/a783d25e-652a-441f-abcf-acc3444876ee) ![image](https://github.com/elastic/kibana/assets/209966/fe60e83e-d559-459a-9fda-e2adb7d583dc) ![image](https://github.com/elastic/kibana/assets/209966/557aebdf-005d-4cfb-bd16-895c9b1b90cb) --------- Co-authored-by: Yngrid Coello <yngrdyn@gmail.com> (cherry picked from commit 8f6469a)

sorenlouv requested a review from a team as a code owner May 12, 2023 14:47

botelastic bot added the Team:APM - DEPRECATED Use Team:obs-ux-infra_services. label May 12, 2023

yngrdyn reviewed May 15, 2023

View reviewed changes

x-pack/plugins/apm/common/diagnostics/get_default_index_template_names.ts Outdated Show resolved Hide resolved

sorenlouv mentioned this pull request May 16, 2023

[APM] Diagnostics Tool (Technical Preview) #157845

Closed

sorenlouv force-pushed the diagnostics-tool branch from 86db339 to 094ce32 Compare May 16, 2023 21:48

sorenlouv added release_note:skip Skip the PR/issue when compiling release notes v8.9.0 labels May 17, 2023

sorenlouv force-pushed the diagnostics-tool branch 5 times, most recently from 8f61c23 to a5b9f16 Compare May 22, 2023 09:14

sorenlouv force-pushed the diagnostics-tool branch 2 times, most recently from 00e900f to de6a4cf Compare May 26, 2023 09:51

kpatticha reviewed May 26, 2023

View reviewed changes

x-pack/plugins/apm/server/lib/helpers/create_es_client/create_apm_event_client/index.ts Outdated Show resolved Hide resolved

kpatticha reviewed May 26, 2023

View reviewed changes

x-pack/plugins/apm/server/lib/helpers/create_es_client/create_apm_event_client/index.ts Outdated Show resolved Hide resolved

kpatticha reviewed May 26, 2023

View reviewed changes

x-pack/plugins/apm/server/lib/helpers/create_es_client/create_apm_event_client/index.ts Outdated Show resolved Hide resolved

kpatticha reviewed May 26, 2023

View reviewed changes

...gins/apm/server/routes/diagnostics/index_templates/get_default_apm_index_templates_states.ts Outdated Show resolved Hide resolved

sorenlouv force-pushed the diagnostics-tool branch from a9d5224 to 139474e Compare June 1, 2023 07:37

yngrdyn reviewed Jun 1, 2023

View reviewed changes

x-pack/plugins/apm/public/components/app/diagnostics/index.tsx Outdated Show resolved Hide resolved

yngrdyn reviewed Jun 1, 2023

View reviewed changes

x-pack/plugins/apm/public/components/app/diagnostics/summary_tab/data_streams_status.tsx Outdated Show resolved Hide resolved

yngrdyn reviewed Jun 1, 2023

View reviewed changes

yngrdyn approved these changes Jun 1, 2023

View reviewed changes

kpatticha approved these changes Jun 2, 2023

View reviewed changes

[APM] Initial diagnostics tool

bd9b807

More stuff

sorenlouv and others added 14 commits June 5, 2023 14:28

Re-enable synthtrace

7a65666

Fix yargs

a5b445c

Added cypress test

69a8735

Fix some bugs

1173e15

Update x-pack/plugins/apm/public/components/app/diagnostics/index.tsx

2265286

Co-authored-by: Yngrid Coello <yngrdyn@gmail.com>

Simplify layout with TabStatus component

ddca201

Fix integration tests

2aa3658

Add empty state test

61a8623

Fix test

3088e9e

Fix api test

0809fad

Fix tests

075134c

Fix bug with missing indices

9aa5835

Ignore unavailable indices

e39984f

Fix e2e tests

101f4aa

sorenlouv force-pushed the diagnostics-tool branch from 8a565d7 to 101f4aa Compare June 5, 2023 12:29

sorenlouv added 2 commits June 5, 2023 14:37

Improve bundle name

26df392

Fix filename casing

aad75bb

sorenlouv enabled auto-merge (squash) June 5, 2023 22:02

sorenlouv merged commit 8f6469a into elastic:main Jun 5, 2023

kibanamachine added the backport:skip This commit does not require backporting label Jun 5, 2023

sorenlouv deleted the diagnostics-tool branch June 5, 2023 22:42

This was referenced Jun 5, 2023

[8.8] [APM] Initial diagnostics tool (#157500) #159073

Closed

[8.7] [APM] Initial diagnostics tool (#157500) #159074

Closed

gbamparop mentioned this pull request Jun 20, 2023

[APM] Document UI diagnostics tool #160025

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[APM] Initial diagnostics tool #157500

[APM] Initial diagnostics tool #157500

sorenlouv commented May 12, 2023 •

edited

Loading

apmmachine commented May 12, 2023

yngrdyn commented May 15, 2023

yngrdyn Jun 1, 2023

yngrdyn Jun 1, 2023

yngrdyn left a comment

kpatticha commented Jun 1, 2023

sorenlouv commented Jun 2, 2023

kpatticha left a comment

kibana-ci commented Jun 5, 2023

ESLint disabled in files

ESLint disabled line counts

Total ESLint disabled count

sorenlouv commented Jun 5, 2023

[APM] Initial diagnostics tool #157500

[APM] Initial diagnostics tool #157500

Conversation

sorenlouv commented May 12, 2023 • edited Loading

apmmachine commented May 12, 2023

🤖 GitHub comments

yngrdyn commented May 15, 2023

yngrdyn Jun 1, 2023

Choose a reason for hiding this comment

yngrdyn Jun 1, 2023

Choose a reason for hiding this comment

yngrdyn left a comment

Choose a reason for hiding this comment

kpatticha commented Jun 1, 2023

sorenlouv commented Jun 2, 2023

kpatticha left a comment

Choose a reason for hiding this comment

kibana-ci commented Jun 5, 2023

💚 Build Succeeded

Metrics [docs]

Module Count

Async chunks

Public APIs missing exports

ESLint disabled in files

ESLint disabled line counts

Total ESLint disabled count

History

sorenlouv commented Jun 5, 2023

💚 All backports created successfully

Questions ?

sorenlouv commented May 12, 2023 •

edited

Loading