[docs][7.3] Agent configuration documentation (#2366)

_meta/beat.yml

@@ -198,7 +198,7 @@ apm-server:
   #agent.config.cache.expiration: 30s
 
   #kibana:
-    # For agent remote configuration, enabled must be true
+    # For APM Agent configuration in Kibana, enabled must be true
     #enabled: false
 
     # Scheme and port can be left out and will be set to the default (http and 5601)

apm-server.docker.yml

@@ -198,7 +198,7 @@ apm-server:
   #agent.config.cache.expiration: 30s
 
   #kibana:
-    # For agent remote configuration, enabled must be true
+    # For APM Agent configuration in Kibana, enabled must be true
     #enabled: false
 
     # Scheme and port can be left out and will be set to the default (http and 5601)

apm-server.yml

@@ -198,7 +198,7 @@ apm-server:
   #agent.config.cache.expiration: 30s
 
   #kibana:
-    # For agent remote configuration, enabled must be true
+    # For APM Agent configuration in Kibana, enabled must be true
     #enabled: false
 
     # Scheme and port can be left out and will be set to the default (http and 5601)

docs/agent-configuration.asciidoc

@@ -0,0 +1,54 @@
+[[agent-configuration-api]]
+== Agent configuration API
+
+APM Server exposes an API endpoint that allows agents to query the server for configuration changes.
+More information on this feature is available in {kibana-ref}/agent-configuration.html[APM Agent configuration in Kibana].
+
+[[agent-config-endpoint]]
+[float]
+=== Agent configuration endpoint
+
+Send an `HTTP GET` request to the agent configuration endpoint.
+`service.name` is a required query string parameter.
+
+[source,bash]
+------------------------------------------------------------
+http(s)://{hostname}:{port}/config/v1/agents?service.name=SERVICE_NAME
+------------------------------------------------------------
+
+If a <<secret-token,`secret_token`>> has been configured, it should also apply to this endpoint.
+
+[[agent-config-api-response]]
+[float]
+==== Responses
+
+* Successful - `200`
+* Kibana endpoint is disabled - `403`
+* Kibana is unreachable - `503`
+
+[[agent-config-api-example]]
+[float]
+==== Example
+
+Example Agent configuration request including the service name "test-service":
+
+["source","sh",subs="attributes"]
+---------------------------------------------------------------------------
+curl -i http://127.0.0.1:8200/config/v1/agents?service.name=test-service
+---------------------------------------------------------------------------
+
+A sample response to the above curl request:
+
+["source","sh",subs="attributes"]
+---------------------------------------------------------------------------
+HTTP/1.1 200 OK
+Cache-Control: max-age=30, must-revalidate
+Content-Type: application/json
+Etag: "5"
+Date: Fri, 05 Jul 2019 21:47:35 GMT
+Content-Length: 30
+
+{
+  "sampling_rate": "0.12"
+}
+---------------------------------------------------------------------------

docs/configure-kibana-endpoint.asciidoc

@@ -0,0 +1,125 @@
+[[setup-kibana-endpoint]]
+== Configure the Kibana endpoint
+
+Starting with {beatname_uc} 7.3.0, we've introduced a new Kibana API for APM Server.
+Configuring this endpoint is required for 
+{kibana-ref}/agent-configuration.html[APM Agent configuration in Kibana].
+Once the endpoint is configured,
+you'll be able to fine-tune certain agent configurations directly in Kibana without having to redeploy your agents.
+You configure the endpoint in the `apm-server.kibana` section of the
++{beatname_lc}.yml+ config file.
+
+Here's a sample configuration:
+
+[source,yaml]
+----
+apm-server.kibana.host: "http://localhost:5601"
+----
+
+[float]
+=== Considerations
+
+* If your setup uses a <<config-secret-token>> for Agent/Server communication,
+the same token is used to secure this endpoint.
+* It's important to still set relevant defaults locally in each Agent's configuration.
+If APM Server is unreachable, slow to respond, returns an error, etc.,
+defaults set in the agent will apply according to their precedence. 
+
+[float]
+=== Kibana endpoint configuration options
+
+You can specify the following options in the `apm-server.kibana` section of the
++{beatname_lc}.yml+ config file:
+
+[float]
+[[kibana-enabled]]
+==== `apm-server.kibana.enabled`
+
+Defaults to `false`. Must be `true` to use APM Agent configuration.
+
+[float]
+==== `apm-server.kibana.host`
+
+The Kibana host that APM Server will communicate with. The default is
+`127.0.0.1:5601`. The value of `host` can be a `URL` or `IP:PORT`. For example: `http://192.15.3.2`, `192:15.3.2:5601` or `http://192.15.3.2:6701/path`. If no
+port is specified, `5601` is used.
+
+NOTE: When a node is defined as an `IP:PORT`, the _scheme_ and _path_ are taken
+from the <<kibana-protocol-option,apm-server.kibana.protocol>> and
+<<kibana-path-option,apm-server.kibana.path>> config options.
+
+IPv6 addresses must be defined using the following format:
+`https://[2001:db8::1]:5601`.
+
+[float]
+[[kibana-protocol-option]]
+==== `apm-server.kibana.protocol`
+
+The name of the protocol Kibana is reachable on. The options are: `http` or
+`https`. The default is `http`. However, if you specify a URL for host, the
+value of `protocol` is overridden by whatever scheme you specify in the URL.
+
+Example config:
+
+[source,yaml]
+----
+apm-server.kibana.host: "192.0.2.255:5601"
+apm-server.kibana.protocol: "http"
+apm-server.kibana.path: /kibana
+----
+
+
+[float]
+==== `apm-server.kibana.username`
+
+The basic authentication username for connecting to Kibana. If you don't
+specify a value for this setting, {beatname_uc} uses the `username` specified
+for the Elasticsearch output.
+
+[float]
+==== `apm-server.kibana.password`
+
+The basic authentication password for connecting to Kibana. If you don't
+specify a value for this setting, {beatname_uc} uses the `password` specified
+for the Elasticsearch output.
+
+[float]
+[[kibana-path-option]]
+==== `apm-server.kibana.path`
+
+An HTTP path prefix that is prepended to the HTTP API calls. This is useful for
+the cases where Kibana listens behind an HTTP reverse proxy that exports the API
+under a custom prefix.
+
+[float]
+==== `apm-server.kibana.ssl.enabled`
+
+Enables {beatname_uc} to use SSL settings when connecting to Kibana via HTTPS.
+If you configure {beatname_uc} to connect over HTTPS, this setting defaults to
+`true` and {beatname_uc} uses the default SSL settings.
+
+Example configuration:
+
+[source,yaml]
+----
+apm-server.kibana.host: "https://192.0.2.255:5601"
+apm-server.kibana.ssl.enabled: true
+apm-server.kibana.ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
+apm-server.kibana.ssl.certificate: "/etc/pki/client/cert.pem"
+apm-server.kibana.ssl.key: "/etc/pki/client/cert.key
+----
+
+For information on the additional SSL configuration options,
+see <<configuration-ssl>>.
+
+[float]
+=== Agent Config configuration options
+
+You can specify the following options in the `apm-server.agent.config` section of the
++{beatname_lc}.yml+ config file:
+
+[float]
+==== `agent.config.cache.expiration`
+
+When using APM Agent configuration, information fetched from Kibana will be cached in memory.
+This setting specifies the time before cache key expiration. Defaults to 30 seconds.

docs/configuring.asciidoc

@@ -10,7 +10,9 @@ include::./copied-from-beats/shared-configuring.asciidoc[]
 * <<configuring-ingest-node>>
 * <<configuration-ssl>>
 * <<configuration-template>>
+* <<ilm>>
 * <<configuration-logging>>
+* <<setup-kibana-endpoint>>
 * <<configuration-rum>>
 * <<using-environ-vars>>
 * <<configuration-path>>
@@ -31,6 +33,8 @@ include::./ilm.asciidoc[]
 
 include::./copied-from-beats/loggingconfig.asciidoc[]
 
+include::./configure-kibana-endpoint.asciidoc[]
+
 include::./configuration-rum.asciidoc[]
 
 :standalone:

docs/intake-api.asciidoc

@@ -33,4 +33,5 @@ The APM Server exposes endpoints for
 --
 include::./events-api.asciidoc[]
 include::./sourcemap-api.asciidoc[]
+include::./agent-configuration.asciidoc[]
 include::./server-info.asciidoc[]

docs/redirects.asciidoc

@@ -238,21 +238,6 @@ Loading dashboards from APM Server is no longer supported. Please see the {kiban
 
 Loading dashboards from APM Server is no longer supported. Please see the {kibana-ref}/xpack-apm.html[Kibana APM UI] documentation.
 
-[role="exclude",id="setup-kibana-endpoint"]
-=== Set up the Kibana endpoint
-
-Loading dashboards from APM Server is no longer supported. Please see the {kibana-ref}/xpack-apm.html[Kibana APM UI] documentation.
-
-[role="exclude",id="kibana-protocol-option"]
-=== setup.kibana.protocol
-
-Loading dashboards from APM Server is no longer supported. Please see the {kibana-ref}/xpack-apm.html[Kibana APM UI] documentation.
-
-[role="exclude",id="kibana-path-option"]
-=== setup.kibana.path
-
-Loading dashboards from APM Server is no longer supported. Please see the {kibana-ref}/xpack-apm.html[Kibana APM UI] documentation.
-
 [role="exclude",id="load-kibana-dashboards"]
 === Dashboards