Skip to content

Commit 00ed51b

Browse files
lucian-ioanlucian-ioan
authored
[vSphere][network] Create network datastream (#10993)
1 parent b20a65f commit 00ed51b

File tree

14 files changed

+464
-3
lines changed

14 files changed

+464
-3
lines changed

‎packages/vsphere/_dev/build/docs/README.md‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ The vSphere integration collects logs and metrics.
1616

1717
Logs help you keep a record of events that happen on your machine. The `Log` data stream collected by vSphere as integration is `log`.
1818

19-
Metrics give you insight into the statistics of the vSphere. The `Metric` data stream collected by the vSphere integration are `cluster`, `datastore`, `datastorecluster`, `host`, `resourcepool` and `virtualmachine` so that the user can monitor and troubleshoot the performance of the vSphere instance.
19+
Metrics give you insight into the statistics of the vSphere. The `Metric` data stream collected by the vSphere integration are `cluster`, `datastore`, `datastorecluster`, `host`, `network`, `resourcepool` and `virtualmachine` so that the user can monitor and troubleshoot the performance of the vSphere instance.
2020

2121
Data Streams:
2222

@@ -27,6 +27,7 @@ Data Streams:
2727
- **`host`**: This data stream collects host metrics from VMware vSphere, including performance statistics such as CPU usage, memory usage, disk I/O, and network activity.
2828
- **`resourcepool`**: This data stream collects metrics from VMware vSphere, such as CPU and memory usage, CPU and memory reservation, and CPU and memory limit.
2929
- **`virtualmachine`**: This data stream gathers virtual machine metrics from VMware vSphere, including performance statistics such as status, uptime, CPU usage, memory usage, and network activity.
30+
- **`network`**: This data stream gathers metrics and status information related to VMware vSphere networks, including network accessibility, connected hosts and virtual machines, configuration health, and network type.
3031

3132
Note:
3233
- Users can monitor and see the log inside the ingested documents for vSphere in the `logs-*` index pattern from `Discover`, and for metrics, the index pattern is `metrics-*`.
@@ -120,6 +121,18 @@ Please refer to the following [document](https://www.elastic.co/guide/en/ecs/cur
120121

121122
{{fields "host"}}
122123

124+
### Network
125+
126+
This is the `network` data stream. This data stream gathers metrics and status information related to VMware vSphere networks, including network accessibility, connected hosts and virtual machines, configuration health, and network type.
127+
128+
{{event "network"}}
129+
130+
**ECS Field Reference**
131+
132+
Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
133+
134+
{{fields "network"}}
135+
123136
### Resourcepool Metrics
124137
Resource pools in vSphere allow for the allocation and management of CPU and memory resources across groups of virtual machines.
125138

‎packages/vsphere/changelog.yml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,13 @@
11
# newer versions go on top
22
- version: "1.15.0-next"
33
changes:
4+
- description: Add new network datastream.
5+
type: enhancement
6+
link: https://github.com/elastic/integrations/pull/10993
47
- description: Add new datastorecluster datastream.
58
type: enhancement
69
link: https://github.com/elastic/integrations/pull/11089
7-
- description: Add additional metrics for virtualmachine datastream
10+
- description: Add additional metrics for virtualmachine datastream.
811
type: enhancement
912
link: https://github.com/elastic/integrations/pull/10942
1013
- description: Add resourcepool datastream.

‎packages/vsphere/data_stream/network/_dev/test/pipeline/test-network.json‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
{
2+
"events": [
3+
{
4+
"vsphere": {
5+
"network": {
6+
"accessible": true,
7+
"config": {
8+
"status": "green"
9+
},
10+
"name": "VM Network",
11+
"status": "green",
12+
"type": "Network"
13+
}
14+
}
15+
}
16+
]
17+
}

‎packages/vsphere/data_stream/network/_dev/test/pipeline/test-network.json-expected.json‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
{
2+
"expected": [
3+
{
4+
"ecs": {
5+
"version": "8.11.0"
6+
},
7+
"vsphere": {
8+
"network": {
9+
"accessible": true,
10+
"config": {
11+
"status": "green"
12+
},
13+
"name": "VM Network",
14+
"status": "green",
15+
"type": "Network"
16+
}
17+
}
18+
}
19+
]
20+
}

‎packages/vsphere/data_stream/network/_dev/test/system/test-default-config.yml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
service: vsphere-metrics
2+
vars:
3+
hosts:
4+
- "{{Hostname}}:8989/sdk"
5+
period: 10s
6+
username: "user"
7+
password: "password"
8+
# If insecure is true, don't verify the server's certificate chain
9+
insecure: true

‎packages/vsphere/data_stream/network/agent/stream/stream.yml.hbs‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
metricsets: ["network"]
2+
hosts:
3+
{{#each hosts}}
4+
- {{this}}
5+
{{/each}}
6+
period: {{period}}
7+
username: {{username}}
8+
password: {{password}}
9+
# If insecure is true, don't verify the server's certificate chain
10+
insecure: {{insecure}}
11+
12+
tags:
13+
{{#each tags as |tag|}}
14+
- {{tag}}
15+
{{/each}}
16+
17+
{{#if processors}}
18+
processors:
19+
{{processors}}
20+
{{/if}}

‎packages/vsphere/data_stream/network/elasticsearch/ingest_pipeline/default.yml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
---
2+
description: Pipeline for network
3+
processors:
4+
- set:
5+
field: ecs.version
6+
value: '8.11.0'
7+
on_failure:
8+
- set:
9+
field: error.message
10+
value: '{{ _ingest.on_failure_message }}'

‎packages/vsphere/data_stream/network/fields/agent.yml‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
- name: host
2+
title: Host
3+
group: 2
4+
description: 'A host is defined as a general computing instance. ECS host.* fields should be populated with details about the host on which the event happened, or from which the measurement was taken. Host types include hardware, virtual machines, Docker containers, and Kubernetes nodes.'
5+
type: group
6+
fields:
7+
- name: name
8+
level: core
9+
type: keyword
10+
dimension: true
11+
ignore_above: 1024
12+
description: 'Name of the host. It can contain what `hostname` returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use.'
13+
- name: containerized
14+
type: boolean
15+
description: >
16+
If the host is a container.
17+
- name: agent
18+
title: Agent
19+
type: group
20+
fields:
21+
- name: id
22+
type: keyword
23+
ignore_above: 1024
24+
dimension: true

‎packages/vsphere/data_stream/network/fields/base-fields.yml‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
- name: data_stream.type
2+
type: constant_keyword
3+
description: Data stream type.
4+
- name: data_stream.dataset
5+
type: constant_keyword
6+
description: Data stream dataset.
7+
- name: data_stream.namespace
8+
type: constant_keyword
9+
description: Data stream namespace.
10+
- name: '@timestamp'
11+
type: date
12+
description: Event timestamp.
13+
- name: event.module
14+
type: constant_keyword
15+
description: Event module
16+
value: vsphere
17+
- name: event.dataset
18+
type: constant_keyword
19+
description: Event dataset
20+
value: vsphere.network

‎packages/vsphere/data_stream/network/fields/ecs.yml‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
- name: service.address
2+
external: ecs
3+
dimension: true
4+
- external: ecs
5+
name: cloud.instance.id
6+
dimension: true
7+
- external: ecs
8+
name: cloud.provider
9+
dimension: true
10+
- external: ecs
11+
name: container.id
12+
dimension: true
13+
- external: ecs
14+
name: cloud.account.id
15+
dimension: true
16+
- external: ecs
17+
name: cloud.region
18+
dimension: true
19+
- external: ecs
20+
name: cloud.availability_zone
21+
dimension: true

0 commit comments

Comments
 (0)