Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] Streamline beats getting started guides #17007

Merged
merged 42 commits into from
Jun 18, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
9fca58e
Move detail out of getting started guides
dedemorton Mar 14, 2020
78d1868
Changes in progress
dedemorton May 7, 2020
d5edbef
Add widgets
dedemorton May 8, 2020
98871be
Change tag width to 100%
dedemorton May 9, 2020
5c8b18f
remove unused css and red border
bmorelli25 May 11, 2020
aa992f2
breathing room below tab content
bmorelli25 May 11, 2020
ec5ae26
Remove inexplicable tab
dedemorton May 11, 2020
f13e935
Clean up wording
dedemorton May 12, 2020
46bf73c
Cleanup the getting started and move some stuff
dedemorton May 13, 2020
70023ad
Git auditbeat building
dedemorton May 14, 2020
352f9aa
Get all books building
dedemorton May 14, 2020
40c97db
More small changes
dedemorton May 14, 2020
5eac1e8
Fix build errors
dedemorton May 15, 2020
e48e739
Finish adding widgets and cleaning up gs guides
dedemorton May 16, 2020
7773f42
Fix widget filenames
dedemorton May 18, 2020
cde03dd
More cleanup work
dedemorton May 22, 2020
08c7af0
Add keystore tip to cloud example
dedemorton May 27, 2020
7cf5981
Another round of improvements
dedemorton May 28, 2020
58ada33
Delete redundant files
dedemorton May 28, 2020
09a5230
Replace dashboards in quick start topics
dedemorton May 29, 2020
16801a9
Change short titles for quick start guides
dedemorton May 29, 2020
59e2031
Make shared what's next list easier to include
dedemorton May 29, 2020
75b1f34
Remove modules quick start
dedemorton May 29, 2020
c8821a8
More cleanup
dedemorton May 30, 2020
56e06a9
Remove redundant files
dedemorton May 30, 2020
42e3281
Remove Logstash from the Winlogbeat quick start
dedemorton May 30, 2020
b7dcbbc
Remove old comments
dedemorton May 30, 2020
7e7f1d8
More cleanup work
dedemorton May 31, 2020
aee466d
Delete unused graphics
dedemorton May 31, 2020
4a8a003
Use correct spelling for GitHub
dedemorton Jun 4, 2020
6c7ffb6
Add formatting
dedemorton Jun 4, 2020
6514c40
Use correct capitalization for GitHub
dedemorton Jun 4, 2020
f003270
Use time filter instead of date range
dedemorton Jun 4, 2020
2e9f4ba
Change wording of headings as suggested during review
dedemorton Jun 6, 2020
be2a844
Add more fixes from the review
dedemorton Jun 16, 2020
e420b1d
Synchronize widget tab selection and fix bugs
dedemorton Jun 17, 2020
2227dcd
Include javascript only once in a file
dedemorton Jun 17, 2020
a7a2238
Add script and stylesheet to support tabbed panel
dedemorton Jun 18, 2020
cb7a36d
Fix step numbering
dedemorton Jun 18, 2020
be0b7e1
Include code for displaying tabbed panel
dedemorton Jun 18, 2020
0827608
Add javascript and css for tabbed panel
dedemorton Jun 18, 2020
c8d9c7d
Fix code include
dedemorton Jun 18, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
More cleanup
  • Loading branch information
dedemorton committed May 30, 2020
commit c8821a8803f74b38a67595d69657271b089587e1
3 changes: 3 additions & 0 deletions filebeat/docs/configuring-howto.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
include::{libbeat-dir}/shared/configuring-intro.asciidoc[]

* <<configuration-filebeat-options>>
* <<configuration-{beatname_lc}-modules>>
* <<configuration-general-options>>
* <<configuration-path>>
* <<filebeat-configuration-reloading>>
Expand All @@ -32,6 +33,8 @@ include::{libbeat-dir}/shared/configuring-intro.asciidoc[]

include::./filebeat-options.asciidoc[]

include::{docdir}/../docs/filebeat-modules-options.asciidoc[]

include::./filebeat-general-options.asciidoc[]

include::{libbeat-dir}/shared-path-config.asciidoc[]
Expand Down
135 changes: 43 additions & 92 deletions filebeat/docs/filebeat-modules-options.asciidoc
Original file line number Diff line number Diff line change
@@ -1,114 +1,69 @@
:modulename: apache mysql

[id="configuration-{beatname_lc}-modules"]
== Enable and run modules
== Configure modules

++++
<titleabbrev>Modules</titleabbrev>
++++

// REVIEWERS: I've changed the focus of this topic to back to configuration.

NOTE: Using {beatname_uc} modules is optional. You may decide to
<<configuration-{beatname_lc}-options,configure inputs manually>> if you are using
<<configuration-{beatname_lc}-options,configure inputs manually>> if you're using
a log type that isn't supported, or you want to use a different setup.

{beatname_uc} <<{beatname_lc}-modules,modules>> provide a quick way for you to get started
processing common log formats. They contain default configurations,
Elasticsearch ingest node pipeline definitions, and Kibana dashboards to help you
{beatname_uc} <<{beatname_lc}-modules,modules>> provide a quick way for you to
get started processing common log formats. They contain default configurations,
{es} ingest node pipeline definitions, and {kib} dashboards to help you
implement and deploy a log monitoring solution.

{beatname_uc} provides a few different ways to enable modules. You can:

* <<enable-modules-d-configs>>
* <<enable-modules-cli>>
* <<enable-modules-config-file>>

include::{libbeat-dir}/shared-note-file-permissions.asciidoc[]

When you enable modules, you can also
<<specify-variable-settings,specify variable settings>> to change the default
behavior of the modules, and you can specify
<<advanced-settings,advanced settings>> to override input settings.
You can configure modules in the `modules.d` directory (recommended), or in the
{beatname_uc} configuration file.

Before running {beatname_uc} with modules enabled, make sure you also set up the
environment to use Kibana dashboards. See
environment to use {kib} dashboards. See
<<{beatname_lc}-installation-configuration>> for more information.

[float]
[[enable-modules-d-configs]]
=== Enable module configs in the `modules.d` directory
[[configure-modules-d-configs]]
=== Configure modules in the `modules.d` directory

The `modules.d` directory contains default configurations for all the modules
available in {beatname_uc}. You can enable or disable specific module configurations
under `modules.d` by running the
<<modules-command,`modules enable` or `modules disable`>> commands.

For example, to enable the `apache` and `mysql` configs in the `modules.d`
directory, you use:

include::./include/enable-modules-command.asciidoc[]

Then when you run {beatname_uc}, it loads the corresponding module configurations
specified in the `modules.d` directory (for example, `modules.d/apache.yml` and
`modules.d/mysql.yml`).
available in {beatname_uc}. To enable or disable specific module configurations under `modules.d`, run the
<<modules-command,`modules enable` or `modules disable`>> commands. For example:

To see a list of enabled and disabled modules, run:
include::{libbeat-dir}/tab-widgets/enable-modules-widget.asciidoc[]

include::./include/list-modules-command.asciidoc[]
The default configurations assume that your data is in the location expected for
your OS and that the behavior of the module is appropriate for your environment.
To change the default behavior, configure variable settings. For a list of
available settings, see the documentation under <<{beatname_lc}-modules>>.

The default module configurations assume that the logs you’re harvesting are
in the location expected for your OS and that the behavior of the module is
appropriate for your environment. To change the default configurations, you need
to <<specify-variable-settings,specify variable settings>>.

[float]
[[enable-modules-cli]]
=== Enable modules when you run {beatname_uc}
For advanced use cases, you can also override input settings. See
<<advanced-settings>>.

To enable specific <<{beatname_lc}-modules,modules>> when you run {beatname_uc} at the
command line, you can use the `--modules` flag. This approach works well when
you're getting started and want to specify different modules and settings each
time you run {beatname_uc}. Any modules specified at the command line will be loaded
TIP: You can also enable modules at runtime by using the
<<{beatname_lc}-modules,--modules flag>>. This is useful if you're getting started
and want to try things out. Any modules specified at the command line are loaded
along with any modules that are enabled in the configuration file or `modules.d`
directory. If there's a conflict, the configuration specified at the command
line is used.

The following command enables and runs the `nginx`,`mysql`, and `system`
modules.

*deb and rpm:*

["source","sh",subs="attributes"]
----
{beatname_lc} --modules nginx,mysql,system
----

*mac:*

["source","sh",subs="attributes"]
----
./{beatname_lc} --modules nginx,mysql,system
----

*win:*

["source","sh",subs="attributes"]
----
PS > .{backslash}{beatname_lc}.exe --modules nginx,mysql,system
----

The default module configurations assume that the logs you’re harvesting are
in the location expected for your OS and that the behavior of the module is
appropriate for your environment. To change the default configurations, you need
to <<specify-variable-settings,specify variable settings>>.
// REVIEWERS: I've removed the detail about using the --modules flag because
// it seems like an extra detail. Users can look at the command reference to
// learn how to specify the flag.

[float]
[[enable-modules-config-file]]
=== Enable module configs in the +{beatname_lc}.yml+ file
[[configure-modules-config-file]]
=== Configure modules in the configuration file

When possible, you should use the config files in the `modules.d` directory.

However, enabling <<{beatname_lc}-modules,modules>> directly in the config file is a
practical approach if you have upgraded from a previous version of {beatname_uc}
and don't want to move your module configs to the `modules.d` directory. You can
continue to configure modules in the +{beatname_lc}.yml+ file, but you won't be
able to use the `modules` command to enable and disable configurations because
the command requires the `modules.d` layout.
However, configuring <<{beatname_lc}-modules,modules>> directly in the config
file is a practical approach if you have upgraded from a previous version of
{beatname_uc} and don't want to move your module configs to the `modules.d`
directory. You can continue to configure modules in the +{beatname_lc}.yml+
file, but you won't be able to use the `modules` command to enable and disable
configurations because the command requires the `modules.d` layout.

To enable specific modules in the +{beatname_lc}.yml+ config file, you can add
entries to the +{beatname_lc}.modules+ list. Each entry in the list begins with
Expand All @@ -128,12 +83,11 @@ The following example shows a configuration that runs the `nginx`,`mysql`, and
The default module configurations assume that the logs you’re harvesting are
in the location expected for your OS and that the behavior of the module is
appropriate for your environment. To change the default configurations, you need
to <<specify-variable-settings,specify variable settings>>.

[[specify-variable-settings]]
=== Configure variable settings
to set variables in module configuration.

include::./include/set-paths.asciidoc[]
// REVIEWERS: Deleting the section about configuring variable settings because
// it doesn't really add much value to users who are looking at the module
// config docs. It's just an extra hop.

[[advanced-settings]]
=== Override input settings
Expand Down Expand Up @@ -173,6 +127,3 @@ You can also enable `close_eof` for all inputs created by any of the modules:
----------------------------------------------------------------------
-M "*.*.input.close_eof=true"
----------------------------------------------------------------------

:modulename!:

3 changes: 0 additions & 3 deletions filebeat/docs/howto/howto.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@
Learn how to perform common {beatname_uc} configuration tasks.

* <<override-{beatname_lc}-config-settings>
* <<configuration-{beatname_lc}-modules>>
* <<{beatname_lc}-template>>
* <<change-index-name>>
* <<load-kibana-dashboards>>
Expand All @@ -22,8 +21,6 @@ Learn how to perform common {beatname_uc} configuration tasks.

include::override-config-settings.asciidoc[]

include::{docdir}/../docs/filebeat-modules-options.asciidoc[]

include::{libbeat-dir}/howto/load-index-templates.asciidoc[]

include::{libbeat-dir}/howto/change-index-name.asciidoc[]
Expand Down
77 changes: 33 additions & 44 deletions filebeat/docs/howto/override-config-settings.asciidoc
Original file line number Diff line number Diff line change
@@ -1,6 +1,12 @@
[id="override-{beatname_lc}-config-settings"]
== Override configuration settings at the command line

//TODO: Test the new examples.

//TODO: Convert this topic to use platform tabs.

// REVEWERS: This is a mix of new and old content. Please review.

NOTE: If you're running {beatname_uc} as a service, you can't specify
command-line flags. To specify flags, start {beatname_uc} in the foreground.

Expand All @@ -15,72 +21,55 @@ You can specify multiple overrides. Overrides are applied to the currently
running {beatname_uc} process. The {beatname_uc} configuration file is not
changed.

The following sections show examples.

[float]
=== Override configuration settings

Use the `-E` flag to override any setting in the configuration.

//TODO: Add more detail here. Add example that shows how to override the input
// settings

[float]
=== Override module settings

Use the `M` flag to override module configuration settings.

For example, the following configuration sets the path to access logs:

[source,yaml]
----
- module: nginx
access:
var.paths: ["/var/log/nginx/access.log*"] <1>
----
<1> Sets the path for `nginx` access log files.

To override this setting from the command line, use the `-M` flag. The variable
name must include the module and fileset name. For example:

//TODO: Convert this section to use platform tabs.
[[example-override-config]]
=== Example: override configuration in +{beatname_lc}.yml+

*deb and rpm:*
The following configuration sends logging output to files:

["source","sh",subs="attributes"]
----
{beatname_lc} -e -M "nginx.access.var.paths=[/var/log/nginx/access.log*]"
logging.level: info
logging.to_files: true
logging.files:
path: /var/log/filebeat
name: filebeat
keepfiles: 7
permissions: 0644
----

*mac:*
To override the logging level and send logging output to standard error instead
of a file, use the `-E` flag when you run {beatname_uc}:

["source","sh",subs="attributes"]
----
./{beatname_lc} -e -M "nginx.access.var.paths=[/usr/local/var/log/nginx/access.log*]"
-E "logging.to_files=false" -E "logging.to_stderr=true" -E "logging.level=error"
----

*brew:*

["source","sh",subs="attributes"]
----
{beatname_lc} -e -M "nginx.access.var.paths=[/usr/local/var/log/nginx/access.log*]"
----
[float]
[[example-override-module-setting]]
=== Example: override `var.paths` module setting

*linux:*
The following configuration sets the path to Nginx access logs:

["source","sh",subs="attributes"]
[source,yaml]
----
./{beatname_lc} -e -M "nginx.access.var.paths=[/usr/local/var/log/nginx/access.log*]"
- module: nginx
access:
var.paths: ["/var/log/nginx/access.log*"] <1>
----

*win:*
To override this setting from the command line, use the `-M` flag when you run
{beatname_uc}. The variable name must include the module and fileset name. For
example:

["source","sh",subs="attributes"]
----
PS > .{backslash}{beatname_lc}.exe -e -M "nginx.access.var.paths=[c:/programdata/nginx/logs/*access.log*]"
-M "nginx.access.var.paths=[/path/to/log/nginx/access.log*]"
----

You can specify multiple overrides. Each override must start with `-M`.

For information about specific variables that you can set for each fileset,
see the <<{beatname_lc}-modules,documentation for the modules>>.
see the documentation under <<{beatname_lc}-modules>>.

3 changes: 2 additions & 1 deletion filebeat/docs/include/config-option-intro.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,10 @@ Each fileset has separate variable settings for configuring the behavior of the
module. If you don’t specify variable settings, the +{modulename}+ module uses
the defaults.

For more information, see <<specify-variable-settings>>. Also see
For advanced use cases, you can also override input settings. See
<<advanced-settings>>.

TIP: When you specify a setting at the command line, remember to prefix the
setting with the module name, for example, +{modulename}.{fileset_ex}.var.paths+
instead of +{fileset_ex}.var.paths+.

3 changes: 3 additions & 0 deletions filebeat/docs/include/running-modules.asciidoc
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
// REVIEWERS: I am deleting this topic because it is not longer required. We
// are pointing users to the quick start instead.

:has_module_steps:

[float]
Expand Down
Loading