documentation for config auto reload feature (#12548)

* add config watcher to the config package

* add logging to watcher

* add test and refactor to add WatcherEvent.

* add all API calls and fix a bug with recreated files

* add tests for watcher

* remove the unnecessary use of context

* Add debug log and a test for file rename

* use inode to detect if the file is recreated/replaced and only listen to create events.

* tidy ups (#1535)

* tidy ups

* Add tests for inode reconcile

* fix linux vs windows syscall

* fix linux vs windows syscall

* fix windows compile error

* increase timeout

* use ctime ID

* remove remove/creation test as it's a use case that fail in linux

* fix linux/windows to use Ino/CreationTime

* fix the watcher to only overwrite current file id

* fix linter error

* fix remove/create test

* set reconcile loop to 200 Milliseconds

* fix watcher to not trigger event on remove, add more tests

* on a remove event try to add the file back to the watcher and trigger the handler if success

* fix race condition

* fix flaky test

* fix race conditions

* set level to info

* fix when file is removed and get an event for it after

* fix to trigger handler when we get a remove but re-add fail

* fix error message

* add tests for directory watch and fixes

* detect if a file is a symlink and return an error on Add

* rename Watcher to FileWatcher and remove symlink deref

* add fsnotify@v1.5.1

* fix go mod

* do not reset timer on errors, rename OS specific files

* rename New func

* events trigger on write and rename

* add missing test

* fix flaking tests

* fix flaky test

* check reconcile when removed

* delete invalid file

* fix test to create files with different mod time.

* back date file instead of sleeping

* add watching file in agent command.

* fix watcher call to use new API

* add configuration and stop watcher when server stop

* add certs as watched files

* move FileWatcher to the agent start instead of the command code

* stop watcher before replacing it

* save watched files in agent

* add add and remove interfaces to the file watcher

* fix remove to not return an error

* use `Add` and `Remove` to update certs files

* fix tests

* close events channel on the file watcher even when the context is done

* extract `NotAutoReloadableRuntimeConfig` is a separate struct

* fix linter errors

* add Ca configs and outgoing verify to the not auto reloadable config

* add some logs and fix to use background context

* add tests to auto-config reload

* remove stale test

* add tests to changes to config files

* add check to see if old cert files still trigger updates

* rename `NotAutoReloadableRuntimeConfig` to `StaticRuntimeConfig`

* fix to re add both key and cert file. Add test to cover this case.

* review suggestion

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* add check to static runtime config changes

* fix test

* add changelog file

* fix review comments

* Apply suggestions from code review

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* update flag description

Co-authored-by: FFMMM <FFMMM@users.noreply.github.com>

* fix compilation error

* add static runtime config support

* fix test

* fix review comments

* fix log test

* Update .changelog/12329.txt

Co-authored-by: Dan Upton <daniel@floppy.co>

* transfer tests to runtime_test.go

* fix filewatcher Replace to not deadlock.

* avoid having lingering locks

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* split ReloadConfig func

* fix warning message

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* convert `FileWatcher` into an interface

* fix compilation errors

* fix tests

* extract func for adding and removing files

* add a coalesceTimer with a very small timer

* extract coaelsce Timer and add a shim for testing

* add tests to coalesceTimer fix to send remaining events

* set `coalesceTimer` to 1 Second

* support symlink, fix a nil deref.

* fix compile error

* fix compile error

* refactor file watcher rate limiting to be a Watcher implementation

* fix linter issue

* fix runtime config

* fix runtime test

* fix flaky tests

* fix compile error

* Apply suggestions from code review

Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>

* fix agent New to return an error if File watcher New return an error

* add a coalesceTimer with a very small timer

* extract coaelsce Timer and add a shim for testing

* set `coalesceTimer` to 1 Second

* add flag description to agent command docs

* fix link

* add Static runtime config docs

* fix links and alignment

* fix typo

* Revert "add a coalesceTimer with a very small timer"

This reverts commit d9db2fc.

* Revert "extract coaelsce Timer and add a shim for testing"

This reverts commit 0ab8601.

* Apply suggestions from code review

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com>
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
Co-authored-by: FFMMM <FFMMM@users.noreply.github.com>
Co-authored-by: Daniel Upton <daniel@floppy.co>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

website/content/docs/agent/options.mdx

@@ -460,7 +460,10 @@ The agent configuration options below are all specified on the command-line.
   "trace", "debug", "info", "warn", and "err". You can always connect to an agent
   via [`consul monitor`](/commands/monitor) and use any log level. Also,
   the log level can be changed during a config reload.
-
+- `-auto-reload-config` ((#\_auto_reload_config)) - This flag set Consul to automatically reload
+  [Reloadable Configuration](#reloadable-configuration) when configuration files change.
+  Consul will also watch certificate and key files set in `cert_file` and `key_file` and reload the configuration
+  if updated.
 - `-log-json` ((#\_log_json)) - This flag enables the agent to output logs
   in a JSON format. By default this is false.
 
@@ -1833,6 +1836,8 @@ There are also a number of common configuration options supported by all provide
 
 - `log_level` Equivalent to the [`-log-level` command-line flag](#_log_level).
 
+- `auto-reload-config` Equivalent to the [`-auto-reload-config` command-line flag](#_auto_reload_config).
+
 - `log_json` Equivalent to the [`-log-json` command-line flag](#_log_json).
 
 - `default_query_time` Equivalent to the [`-default-query-time` command-line flag](#_default_query_time).
@@ -2771,6 +2776,19 @@ items which are reloaded include:
 - Services
 - TLS Configuration
   - Please be aware that this is currently limited to reload a configuration that is already TLS enabled. You cannot enable or disable TLS only with reloading.
+  - To avoid a potential security issue, the following TLS configuration parameters do not automatically reload when [-auto-reload-config](#_auto_reload_config) is enabled:
+    - [encrypt_verify_incoming](#encrypt_verify_incoming)
+    - [verify_incoming](#verify_incoming)
+    - [verify_incoming_rpc](#verify_incoming_rpc)
+    - [verify_incoming_https](#verify_incoming_https)
+    - [verify_outgoing](#verify_outgoing)
+    - [verify_server_hostname](#verify_server_hostname)
+    - [ca_file](#ca_file)
+    - [ca_path](#ca_path)
+
+    If any of those configurations are changed while [-auto-reload-config](#_auto_reload_config) is enabled,
+    Consul will issue the following warning, `Static Runtime config has changed and need a manual config reload to be applied`.
+    You must manually issue the `consul reload` command or send a `SIGHUP` to the Consul process to reload the new values.
 - Watches
 
 <!-- list of reference-style links -->