Update host metrics docs and refactor files

docs/en/observability/cloud-monitoring/aws/monitor-amazon-intro.asciidoc

@@ -5,7 +5,7 @@
 //for other cloud services.
 
 ++++
-<titleabbrev>Monitor AWS</titleabbrev>
+<titleabbrev>AWS monitoring</titleabbrev>
 ++++
 
 // Kibana description used in all of the monitoring topics in this section

docs/en/observability/index.asciidoc

@@ -76,14 +76,6 @@ include::monitor-infra/analyze-metrics.asciidoc[leveloffset=+1]
 
 include::monitor-infra/view-infrastructure-metrics.asciidoc[leveloffset=+2]
 
-include::monitor-infra/monitor-servers.asciidoc[leveloffset=+3]
-
-include::monitor-infra/monitor-kubernetes-pods.asciidoc[leveloffset=+3]
-
-include::monitor-infra/monitor-docker-containers.asciidoc[leveloffset=+3]
-
-include::monitor-infra/monitor-aws-services.asciidoc[leveloffset=+3]
-
 include::monitor-infra/explore-metrics.asciidoc[leveloffset=+2]
 
 include::monitor-infra/analyze-hosts.asciidoc[leveloffset=+2]
@@ -92,9 +84,11 @@ include::monitor-infra/inspect-metric-anomalies.asciidoc[leveloffset=+2]
 
 include::monitor-infra/configure-metrics-sources.asciidoc[leveloffset=+2]
 
+include::monitor-infra/metrics-reference.asciidoc[leveloffset=+2]
+
 // AWS monitoring
 // Will eventually be replaced by cloud monitoring when other providers are covered
-include::cloud-monitoring/aws/monitor-amazon-intro.asciidoc[leveloffset=+2]
+include::cloud-monitoring/aws/monitor-amazon-intro.asciidoc[leveloffset=+1]
 
 // Synthetics
 include::synthetics-intro.asciidoc[leveloffset=+1]

docs/en/observability/monitor-infra/analyze-hosts.asciidoc

@@ -1,16 +1,10 @@
 [[analyze-hosts]]
-= Analyze and compare hosts on the Hosts page
+= Analyze and compare hosts
 
-++++
-<titleabbrev>Analyze and compare hosts</titleabbrev>
-++++
-
-****
-This functionality is in technical preview and may be changed or removed in a future release. Elastic will apply best effort to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.
+beta[]
 
 We'd love to get your feedback!
 https://docs.google.com/forms/d/e/1FAIpQLScRHG8TIVb1Oq8ZhD4aks3P1TmgiM58TY123QpDCcBz83YC6w/viewform[Tell us what you think!]
-****
 
 The **Hosts** page provides a metrics-driven view of your infrastructure backed
 by an easy-to-use interface called Lens. On the **Hosts** page, you can view
@@ -23,14 +17,16 @@ health and performance metrics to help you quickly:
 * Filter and search the data to focus on the hosts you care about the most.
 
 To access this page from the main {kib} menu, go to
-**{observability} -> Infrastructure**, and then click **Hosts**. If the view is
-not enabled, click **Enable host view** to begin exploring.
-
-include::view-infrastructure-metrics.asciidoc[tag=add-metrics-tip]
+**{observability} -> Infrastructure**, and then click **Hosts**.
 
 [role="screenshot"]
 image::images/hosts.png[Screenshot of the Hosts page]
 
+To learn more about the metrics shown on this page, refer to the
+<<metrics-reference>> documentation.
+
+include::view-infrastructure-metrics.asciidoc[tag=add-metrics-tip]
+
 The **Hosts** page provides a few different ways to view host metrics:
 
 * Overview tiles show the number of hosts returned by your search plus
@@ -51,9 +47,6 @@ load, CPU usage, memory usage, network inbound, network outbound, disk read
 IOPS, and disk write IOPS. Place your cursor over a line to view metrics at a
 specific point in time.
 
-For information about how the metrics on this page are calculated, refer to
-<<analyze-hosts-fields>>.
-
 [discrete]
 [[analyze-hosts-filter-view]]
 == Filter the Hosts view
@@ -78,20 +71,6 @@ filters to set other visualizations on the page to the same time and/or host.
 To learn more about filtering data in {kib}, refer to
 {kibana-ref}/kibana-concepts-analysts.html[{kib} concepts].
 
-[NOTE]
-.Limitation on filtering by service names
-====
-You may see queries where either the expected hosts are not returned, or where returned hosts do not return metrics.
-This may happen when your filter returns events that do not contain all of the fields and/or values specified within your search.
-
-For example, if you search for `service.name : myservice`, you will return all hosts where there are events
-containing both `service.name : myservice` and where `host.name` exists.
-You may find that some metrics show as `0` when there are no events which contain both
-`host.name` and `service.name : myservice` and the metric you are looking for -- like `system.memory.actual.used.pct` for Memory Usage.
-
-TIP: You can configure which indices you are searching in the Settings page.
-====
-
 [discrete]
 [[analyze-hosts-open-in-lens]]
 == Open in Lens
@@ -166,92 +145,3 @@ From the **Actions** menu, you can choose to:
 
 [role="screenshot"]
 image::images/hosts-view-alerts.png[Screenshot showing Alerts view]
-
-[discrete]
-[[analyze-hosts-fields]]
-== Fields used to populate the Hosts page
-
-The **Hosts** page displays metrics from indices that match the index patterns
-specified on the **Settings page**. For example, the default patterns are
-`metrics-*` and `metricbeat-*`.
-
-Log sources are specified on the **Settings page** of the Logs app. For example,
-the default patterns are `logs-*,filebeat-*,kibana_sample_data_logs*`.
-
-The following sections describe the fields used to populate the **Hosts** page
-with data.
-
-[discrete]
-[[analyze-hosts-fields-metrics]]
-=== Metrics
-
-[options="header"]
-|====
-| Metric                    | Description
-
-| **# of CPUs**             | Number of CPU cores
-
-**Field Calculation**: `count(system.cpu.cores)`
-
-| **CPU Usage**             | Percentage of CPU time spent in states other than Idle and IOWait, normalized by the number of CPU cores.
-This includes both time spent on user space and kernel space.
-
-100% means all CPUs of the host are busy.
-
-**Field Calculation**: `avg(system.cpu.user.pct + system.cpu.system.pct) / system.cpu.cores)`
-
-| **Disk Latency**          | Time spent to service disk requests.
-
-**Field Calculation**: `avg(system.diskio.read.time + system.diskio.write.time) / (system.diskio.read.count + system.diskio.write.count)`
-| **Disk Read IOPS**        | Average number of bytes read from the device per second.
-
-**Field Calculation:** `counter_rate(max(system.diskio.read.bytes)`
-| **Disk Write IOPS**       | Average number of bytes written from the device per second.
-
-**Field Calculation**: `counter_rate(max(system.diskio.write.bytes)`
-| **Memory Total**          | Total available memory.
-
-**Field Calculation**: `avg(system.memory.total)`
-| **Memory Usage**          | Percentage of main memory usage excluding page cache.
-
-This includes resident memory for all processes plus memory used by the kernel structures and code apart from the page cache.
-
-A high level indicates a situation of memory saturation for the host.
-For example, 100% means the main memory is entirely filled with memory that can't be reclaimed, except by swapping out.
-
-**Field Calculation:** `avg(system.memory.actual.used.pct)`
-| **Network Inbound (RX)**  | Number of bytes which have been received per second on the public interfaces of the hosts.
-
-**Field Calculation:** `avg(host.network.ingress.bytes / (metricset.period / 1000))`
-| **Network Outbound (TX)** | Number of bytes which have been sent per second on the public interfaces of the hosts.
-
-**Field Calculation:** `avg(host.network.egress.bytes / (metricset.period / 1000))`
-| **Normalized Load**       | 1 minute load average normalized by the number of CPU cores.
-
-Load average gives an indication of the number of threads that are runnable
-(either busy running on CPU, waiting to run or waiting for a blocking IO operation to complete).
-
-100% means the 1 minute load average is equal to the number of CPU cores of the host.
-For example, with a 32 CPU cores host, if 1 min load average is 32, the value reported here is 100%.
-If 1 min load average is 48, the value reported here is 150%.
-
-**Field Calculation:** `average(system.load.1) / max(system.load.cores)`
-|====
-
-[discrete]
-[[analyze-hosts-fields-fields]]
-=== Fields
-
-[options="header"]
-|====
-| Field                | Description
-| **Name**             | Name of host.
-
-**Field Calculation:** `host.name`
-| **Operating System** | Operating System of host.
-
-**Field Calculation:** `host.os.name`
-| **Cloud Provider**   | Cloud Provider host is running on.
-
-**Field Calculation:** `cloud.provider`
-|====

docs/en/observability/monitor-infra/analyze-metrics.asciidoc

@@ -1,14 +1,14 @@
 [[analyze-metrics]]
 = Infrastructure monitoring
 
-The {infrastructure-app} in {kib} enables you to visualize infrastructure metrics
-to help diagnose problematic spikes, identify high resource utilization,
-automatically discover and track pods, and unify your metrics
-with logs and APM data in {es}.
+The {infrastructure-app} in {kib} enables you to visualize infrastructure
+metrics to help diagnose problematic spikes, identify high resource utilization,
+automatically discover and track pods, and unify your metrics with logs and APM
+data in {es}.
 
-Using {agent} integrations, you can ingest and analyze metrics from servers, Docker containers,
-Kubernetes orchestrations, explore and analyze application
-telemetries, and many more.
+Using {agent} integrations, you can ingest and analyze metrics from servers,
+Docker containers, Kubernetes orchestrations, explore and analyze application
+telemetries, and more.
 
 // Conditionally display a screenshot or video depending on what the
 // current documentation version is.
@@ -28,7 +28,32 @@ ifeval::["{is-current-version}"=="true"]
 ++++
 endif::[]
 
-ifeval::["{is-current-version}"=="false"]
-[role="screenshot"]
-image::images/metrics-app.png[{infrastructure-app} in {kib}]
-endif::[]
+To access the {infrastructure-app} from the main {kib} menu, go to
+**Observability -> Infrastructure**. The {infrastructure-app} provides a few
+different views of your data.
+
+[cols="1,1"]
+|===
+| **Inventory**
+|Provides a metrics-driven view of your entire infrastructure grouped by the resources that you are monitoring. 
+
+<<view-infrastructure-metrics,Learn more about the Inventory page > >>
+
+| **Metrics Explorer**
+| Enables you to create time-series visualizations based on aggregation of your metrics, chart them against related metrics, and break them down per the field of your choice.
+
+<<explore-metrics,Learn more about the Metrics Explorer > >>
+
+| **Hosts**
+| Provides a metrics-driven view of your infrastructure backed by an easy-to-use interface called Lens.
+
+<<analyze-hosts,Learn more about the Hosts page > >>
+
+|===
+
+By default, the {infrastructure-app} displays metrics from {es} indices that
+match the `metrics-*` and `metricbeat-*` index patterns. To learn how to change
+this behavior, refer to <<configure-settings>>.
+
+To learn more about the metrics shown in the {infrastructure-app}, refer to
+the <<metrics-reference>> documentation.

docs/en/observability/monitor-infra/monitor-aws-services.asciidoc → docs/en/observability/monitor-infra/aws-metrics.asciidoc

@@ -1,6 +1,11 @@
 [[aws-metrics]]
 = AWS metrics
 
+//TODO: Rewrite this to match new format. Requires input from dev team.
+
+//REVIEWERS: This was taken from the existing docs and moved here. Do we want
+//to publish this info for now until updated desriptions are available?
+
 [IMPORTANT]
 =====
 Additional AWS charges for GetMetricData API requests are generated using this module.

docs/en/observability/monitor-infra/configure-metrics-sources.asciidoc

@@ -1,42 +1,39 @@
 [[configure-settings]]
 = Configure settings
 
-By default, the {kibana-ref}/infrastructure-ui-settings-kb.html[configuration settings] for the 
-{infrastructure-app} are enabled. The configuration uses the `metricbeat-*` and `metrics-*` index patterns
-to query the data. The configuration also defines field settings for things like timestamps
-and container names.
+To configure settings for the {infrastructure-app} in {kib}, go to
+**{observability} -> Infrastructure** and click the **Settings** link at the top
+of the page. The following settings are available:
 
-[discrete]
-[[metrics-config-settings]]
-== Override configuration settings
-
-. To access this page from the main {kib} menu, go to
-*{observability} -> Infrastructure*.
-+
-. From any page in the Infrastructure app, click *Settings*.
-+
-You can override the following configuration settings.
-+
-|=== 
+|===
+| Setting | Description
 
 | *Name* | Name of the source configuration. 
 
-| *Indices* | {ipm-cap} or patterns in the {es} indices to read metrics data from.
+| *Indices* | {ipm-cap} or patterns used to match {es} indices that contain metrics. The default patterns are `metrics-*,metricbeat-*`. 
+
+| *{ml-cap}* | The minimum severity score required to display anomalies in the {infrastructure-app}. The default is 50.
+|===
 
-| *Fields* | Configuring fields input has been deprecated. You should adjust your indexing using the
-<<metrics-app-fields,{infrastructure-app} fields>>, which use the {ecs-ref}/index.html[Elastic Common Schema (ECS) specification].
+Click *Apply* to save your changes.
 
-| *{ml-cap}* | Specify the minimum severity score required to display anomalies in the {infrastructure-app}.
+NOTE: The patterns used to match log sources are configured in the Logs app. The
+default setting is `logs-*,filebeat-*,kibana_sample_data_logs*`. To change the
+default, refer to <<configure-data-sources>>.
 
-|=== 
-+
-. When you have completed your changes, click *Apply*.
-+
 If the fields are grayed out and cannot be edited, you may not have sufficient privileges to change the source configuration.
 For more information see {kibana-ref}/xpack-security-authorization.html[Granting access to {kib}].
-+
+
 [TIP]
 ===============================
 If {kibana-ref}/xpack-spaces.html[Spaces] are enabled in your {kib} instance, any configuration changes you make here are specific to the current space.
 You can make different subsets of data available by creating multiple spaces with different data source configurations.
 ===============================
+
+[discrete]
+== Related settings
+
+You can configure additional metrics settings in the {kib} configuration. For
+more information, refer to
+{kibana-ref}/infrastructure-ui-settings-kb.html[configuration settings] in the
+{kib} documentation.

docs/en/observability/monitor-infra/explore-metrics.asciidoc

@@ -1,9 +1,5 @@
 [[explore-metrics]]
-= Group and analyze metrics in the Metrics Explorer
-
-++++
-<titleabbrev>Group and analyze metrics</titleabbrev>
-++++
+= Explore infrastructure metrics over time
 
 The *Metrics Explorer* page enables you to create time-series visualizations based on
 aggregation of your metrics, chart them against related metrics, and break them down
@@ -16,16 +12,20 @@ your custom dashboards by using the {kibana-ref}/tsvb.html[Time Series Visual Bu
 To access this page from the main {kib} menu, go to
 *{observability} -> Infrastructure*, and then click *Metrics Explorer*.
 
+
+[role="screenshot"]
+image::images/metrics-explorer.png[Metrics Explorer]
+
+To learn more about the metrics shown on this page, refer to the
+<<metrics-reference>> documentation.
+
 include::view-infrastructure-metrics.asciidoc[tag=add-metrics-tip]
 
 By default, the Metrics Explorer page displays the CPU usage for hosts, Kubernetes pods, and Docker containers.
 The initial configuration has the *Average* aggregation selected, the *of* field is populated with the default metrics,
 and the *graph per* dropdown is set to `Everything`.
 
-[role="screenshot"]
-image::images/metrics-explorer.png[Metrics Explorer]
-
-As an example, let’s view the system load metrics for hosts we’re currently monitoring with {metricbeat}.
+As an example, let’s view the system load metrics for hosts we’re currently monitoring.
 
 // lint disable agg
 1. In the *of* field, delete the selected metrics, and then add `system.load.1`, `system.load.5`, and `system.load.15`.