Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Set agent.name to hostname by default #18000

Merged
merged 4 commits into from
May 2, 2020
Merged

Set agent.name to hostname by default #18000

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
22e18a2
Set agent.name to hostname by default
andrewkroh Apr 26, 2020
545f5b7
Update field docs for agent.hostname
andrewkroh Apr 26, 2020
c7a75fa
Update tests
andrewkroh Apr 28, 2020
db5eb1c
Update metricbeat field docs
andrewkroh Apr 30, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -210,6 +210,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Add keystore support for autodiscover static configurations. {pull]16306[16306]
- Add Kerberos support to Elasticsearch output. {pull}17927[17927]
- Add support for fixed length extraction in `dissect` processor. {pull}17191[17191]
- Set `agent.name` to the hostname by default. {issue}16377[16377] {pull}18000[18000]

*Auditbeat*

Expand Down
3 changes: 2 additions & 1 deletion auditbeat/docs/fields.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2458,7 +2458,8 @@ Contains common beat fields available in all event types.
*`agent.hostname`*::
+
--
Hostname of the agent.
Deprecated - use agent.name or agent.id to identify an agent. Hostname of the agent.


type: keyword

Expand Down
2 changes: 1 addition & 1 deletion auditbeat/include/fields.go
View file
Open in desktop

Large diffs are not rendered by default.

3 changes: 2 additions & 1 deletion filebeat/docs/fields.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3112,7 +3112,8 @@ Contains common beat fields available in all event types.
*`agent.hostname`*::
+
--
Hostname of the agent.
Deprecated - use agent.name or agent.id to identify an agent. Hostname of the agent.


type: keyword

Expand Down
2 changes: 1 addition & 1 deletion filebeat/include/fields.go
View file
Open in desktop

Large diffs are not rendered by default.

3 changes: 2 additions & 1 deletion filebeat/tests/system/test_fields.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ def test_custom_fields_under_root(self):
def test_beat_fields(self):
"""
Checks that it's possible to set a custom shipper name. Also
tests that beat.hostname has values.
tests that agent.hostname has values.
"""
self.render_config_template(
path=os.path.abspath(self.working_dir) + "/test.log",
Expand All @@ -80,5 +80,6 @@ def test_beat_fields(self):
output = self.read_output()
doc = output[0]
assert doc["host.name"] == "testShipperName"
assert doc["agent.name"] == "testShipperName"
assert doc["agent.hostname"] == socket.gethostname()
assert "fields" not in doc
2 changes: 1 addition & 1 deletion filebeat/tests/system/test_modules.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -214,7 +214,7 @@ def _test_expected_events(self, test_file, objects):

def clean_keys(obj):
# These keys are host dependent
host_keys = ["host.name", "agent.hostname", "agent.type", "agent.ephemeral_id", "agent.id"]
host_keys = ["host.name", "agent.name", "agent.hostname", "agent.type", "agent.ephemeral_id", "agent.id"]
# The create timestamps area always new
time_keys = ["event.created"]
# source path and agent.version can be different for each run
Expand Down
2 changes: 1 addition & 1 deletion heartbeat/cmd/root.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ var RootCmd *cmd.BeatsRootCmd
func init() {
settings := instance.Settings{
Name: Name,
Processing: processing.MakeDefaultSupport(true, processing.WithECS, processing.WithBeatMeta("agent")),
Processing: processing.MakeDefaultSupport(true, processing.WithECS, processing.WithAgentMeta()),
HasDashboards: false,
}
RootCmd = cmd.GenRootCmdWithSettings(beater.New, settings)
Expand Down
3 changes: 2 additions & 1 deletion heartbeat/docs/fields.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,8 @@ Contains common beat fields available in all event types.
*`agent.hostname`*::
+
--
Hostname of the agent.
Deprecated - use agent.name or agent.id to identify an agent. Hostname of the agent.


type: keyword

Expand Down
2 changes: 1 addition & 1 deletion heartbeat/include/fields.go
View file
Open in desktop

Large diffs are not rendered by default.

3 changes: 2 additions & 1 deletion journalbeat/docs/fields.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@ Contains common beat fields available in all event types.
*`agent.hostname`*::
+
--
Hostname of the agent.
Deprecated - use agent.name or agent.id to identify an agent. Hostname of the agent.


type: keyword

Expand Down
2 changes: 1 addition & 1 deletion journalbeat/include/fields.go
View file
Open in desktop

Large diffs are not rendered by default.

4 changes: 3 additions & 1 deletion libbeat/_meta/fields.common.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,9 @@
fields:
- name: agent.hostname
type: keyword
description: Hostname of the agent.
description: >
Deprecated - use agent.name or agent.id to identify an agent.
Hostname of the agent.

- name: beat.timezone
type: alias
Expand Down
37 changes: 29 additions & 8 deletions libbeat/publisher/processing/default.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,14 +76,14 @@ type builtinModifier func(beat.Info) common.MapStr
// MakeDefaultBeatSupport automatically adds the `ecs.version`, `host.name` and `agent.X` fields
// to each event.
func MakeDefaultBeatSupport(normalize bool) SupportFactory {
return MakeDefaultSupport(normalize, WithECS, WithHost, WithBeatMeta("agent"))
return MakeDefaultSupport(normalize, WithECS, WithHost, WithAgentMeta())
}

// MakeDefaultObserverSupport creates a new SupportFactory based on NewDefaultSupport.
// MakeDefaultObserverSupport automatically adds the `ecs.version` and `observer.X` fields
// to each event.
func MakeDefaultObserverSupport(normalize bool) SupportFactory {
return MakeDefaultSupport(normalize, WithECS, WithBeatMeta("observer"))
return MakeDefaultSupport(normalize, WithECS, WithObserverMeta())
}

// MakeDefaultSupport creates a new SupportFactory for use with the publisher pipeline.
Expand Down Expand Up @@ -139,21 +139,42 @@ var WithHost modifier = builtinModifier(func(info beat.Info) common.MapStr {
}
})

// WithBeatMeta adds beat meta information as builtin fields to a processing pipeline.
// The `key` parameter defines the field to be used.
func WithBeatMeta(key string) modifier {
// WithAgentMeta adds agent meta information as builtin fields to a processing
// pipeline.
func WithAgentMeta() modifier {
return builtinModifier(func(info beat.Info) common.MapStr {
metadata := common.MapStr{
"type": info.Beat,
"ephemeral_id": info.EphemeralID.String(),
"hostname": info.Hostname,
"id": info.ID.String(),
"name": info.Hostname,
"type": info.Beat,
"version": info.Version,
// hostname is deprecated. To be removed for 8.0. It's not in ECS.
// See https://github.com/elastic/beats/issues/16377.
"hostname": info.Hostname,
}
if info.Name != "" {
metadata["name"] = info.Name
}
return common.MapStr{"agent": metadata}
})
}

// WithObserverMeta adds beat meta information as builtin fields to a processing
// pipeline.
func WithObserverMeta() modifier {
return builtinModifier(func(info beat.Info) common.MapStr {
metadata := common.MapStr{
"type": info.Beat, // Per ECS this is not a valid type value.
"ephemeral_id": info.EphemeralID.String(), // Not in ECS.
"hostname": info.Hostname,
"id": info.ID.String(), // Not in ECS.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should "name" also be here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nevermind, I see this has not changed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, I didn't change observer. I had to separate them to avoid introducing changes to it. observer does need some changes, but that's another task - probably for 8.0.

"version": info.Version,
}
if info.Name != info.Hostname {
metadata.Put("name", info.Name)
}
return common.MapStr{key: metadata}
return common.MapStr{"observer": metadata}
})
}

Expand Down
1 change: 1 addition & 0 deletions libbeat/publisher/processing/default_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,7 @@ func TestProcessorsConfigs(t *testing.T) {
"agent": common.MapStr{
"ephemeral_id": "123e4567-e89b-12d3-a456-426655440000",
"hostname": "test.host.name",
"name": "test.host.name",
"id": "123e4567-e89b-12d3-a456-426655440001",
"type": "test",
"version": "0.1",
Expand Down
3 changes: 2 additions & 1 deletion metricbeat/docs/fields.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4631,7 +4631,8 @@ Contains common beat fields available in all event types.
*`agent.hostname`*::
+
--
Hostname of the agent.
Deprecated - use agent.name or agent.id to identify an agent. Hostname of the agent.


type: keyword

Expand Down
3 changes: 2 additions & 1 deletion packetbeat/docs/fields.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -452,7 +452,8 @@ Contains common beat fields available in all event types.
*`agent.hostname`*::
+
--
Hostname of the agent.
Deprecated - use agent.name or agent.id to identify an agent. Hostname of the agent.


type: keyword

Expand Down
2 changes: 1 addition & 1 deletion packetbeat/include/fields.go
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions packetbeat/magefile.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ import (
// mage:import
"github.com/elastic/beats/v7/dev-tools/mage/target/common"
// mage:import
_ "github.com/elastic/beats/v7/dev-tools/mage/target/unittest"
// mage:import
_ "github.com/elastic/beats/v7/dev-tools/mage/target/integtest/notests"
// mage:import
_ "github.com/elastic/beats/v7/dev-tools/mage/target/test"
Expand Down
1 change: 1 addition & 0 deletions packetbeat/tests/system/test_0099_golden_files.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ def clean_keys(obj):
"agent.ephemeral_id",
"agent.hostname",
"agent.id",
"agent.name",
"agent.type",
"agent.version",
"ecs.version",
Expand Down
3 changes: 2 additions & 1 deletion winlogbeat/docs/fields.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,8 @@ Contains common beat fields available in all event types.
*`agent.hostname`*::
+
--
Hostname of the agent.
Deprecated - use agent.name or agent.id to identify an agent. Hostname of the agent.


type: keyword

Expand Down
2 changes: 1 addition & 1 deletion winlogbeat/include/fields.go
View file
Open in desktop

Large diffs are not rendered by default.

3 changes: 2 additions & 1 deletion x-pack/functionbeat/docs/fields.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@ Contains common beat fields available in all event types.
*`agent.hostname`*::
+
--
Hostname of the agent.
Deprecated - use agent.name or agent.id to identify an agent. Hostname of the agent.


type: keyword

Expand Down
2 changes: 1 addition & 1 deletion x-pack/functionbeat/include/fields.go
View file
Open in desktop

Large diffs are not rendered by default.