[vSphere][virtualmachine] Add support for additional metrics #10942

lucian-ioan · 2024-08-30T08:04:07Z

Proposed commit message

Following beats code changes in elastic/beats#40485, the following metrics need integration update:

Metrics	API	Field Mappings
VirtualMachine.Summary.QuickStats.UptimeSeconds	Summary	virtualmachine.uptime
VirtualMachine.Summary.OverallStatus	Summary	virtualmachine.status
VirtualMachine.Summary.Datastore	Summary	virtualmachine.datastore.names

Alerts
triggerd_alarms.name	Name of the alarm.
triggerd_alarms.id	Unique key that identifies the alarm.
triggerd_alarms.status	Overall status of the alarm object.
triggerd_alarms.triggered_time	Time the alarm triggered.
triggerd_alarms.description	Description of the alarm.
triggerd_alarms.entity_name	alarm happened in respective entity

Snapshot
snapshot.info.id	Unique ID of the snapshot.
snapshot.info.name	Name of the snapshot.
snapshot.info.description	Description of the snapshot.
snapshot.info.createtime	Time the snapshot was created.
snapshot.info.state	State of the snapshot (e.g., poweredOn).
snapshot.count	Total number of snapshots.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

Create Snapshot of Metricbeat changes.
Use Snapshot to make the changes in the vSphere Integration Host datastream.
Update the system and pipeline changes.
Add versioning to changelog.
Test the Integration for metric collection.
Test Integration from TSDB Perspective.
Code review.

How to test this PR locally

Related issues

Closes https://github.com/elastic/obs-integration-team/issues/35

Screenshots

elasticmachine · 2024-08-30T08:26:31Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

lalit-satapathy · 2024-08-30T10:08:30Z

packages/vsphere/data_stream/virtualmachine/fields/fields.yml

    - name: cpu.free.mhz
      type: long
-      metric_type: gauge


Any reason to remove these metric_types?

No, reverting, tested first with beats code directly.

ishleenk17 · 2024-09-02T07:06:15Z

packages/vsphere/data_stream/virtualmachine/fields/fields.yml

    - name: name
      type: keyword
-      # Reason to add as a dimension field: Each VM has Unique name assigned
-      dimension: true


@lucian-ioan : I suppose you are using the fields.yml directly from the beats code.
That would not be correct as we don't consider tsdb there.
Also, for any new fields please keep TSDB into consideration.

ishleenk17 · 2024-09-02T07:07:49Z

packages/vsphere/data_stream/virtualmachine/sample_event.json

@@ -59,37 +59,58 @@
    },
    "vsphere": {
        "virtualmachine": {


Are these events autogenerated by system tests ?
Looks like these are being used from beats code ?

Tested with beats code, regenerated it now.

ishleenk17 · 2024-09-19T06:10:01Z

packages/vsphere/changelog.yml

@@ -1,4 +1,9 @@
 # newer versions go on top
+- version: "1.14.0-next"


Lets update this to latest 1.15.0-next

ishleenk17 · 2024-09-19T06:13:48Z

packages/vsphere/data_stream/virtualmachine/fields/fields.yml

+          type: long
+          metric_type: gauge
+          description: The number of snapshots of this virtualmachine.
+    - name: triggerd_alarms.*


This needs to incorporate the latest change of this name change yesterday.
"triggered_alamrs"

ishleenk17 · 2024-09-19T06:14:03Z

packages/vsphere/data_stream/virtualmachine/manifest.yml

+      - name: period
+        type: text
+        title: Period
+        description: The vSphere performance API allows for collecting host and datastore metrics at various intervals, including real-time (every 20 seconds), 5 minutes, or longer durations. Properly configuring the "period" parameter is crucial to manage usage restrictions and ensure accurate data collection. For more information on optimizing your data collection setup, refer to the [Data Collection Intervals](https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.monitoring.doc/GUID-247646EA-A04B-411A-8DD4-62A3DCFCF49B.html) and [Data Collection Levels](https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.monitoring.doc/GUID-25800DE4-68E5-41CC-82D9-8811E27924BC.html) documentation.


Change the description as discussed yesterday

…hine_integration

niraj-elastic · 2024-09-19T12:55:15Z

@lucian-ioan Please add tags in hbs file.

...vsphere/data_stream/virtualmachine/_dev/test/pipeline/test-virtualmachine.json-expected.json

ishleenk17 · 2024-09-19T14:09:22Z

Update description of commit PR with the snapshot fields as well

ishleenk17

Looks good!
Let the CI pass

Addressed changes, merging today.

elastic-sonarqube · 2024-09-20T05:39:23Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

elasticmachine · 2024-09-20T05:39:28Z

💚 Build Succeeded

Buildkite Build
Commit: a009454

History

💔 Build #16212 failed ba0636d
💚 Build #16177 succeeded 4aa0c34
💔 Build #16172 failed f546aa8
💔 Build #16170 failed 6b27dc4
💔 Build #16169 failed d2f467e
💔 Build #16168 failed 649c864

cc @lucian-ioan

elastic-vault-github-plugin-prod · 2024-09-26T14:30:33Z

Package vsphere - 1.15.0 containing this change is available at https://epr.elastic.co/search?package=vsphere

add additional fields

8e782c6

lucian-ioan added enhancement New feature or request Integration:vsphere VMware vSphere Team:Obs-InfraObs Label for the Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] labels Aug 30, 2024

lucian-ioan requested a review from ishleenk17 August 30, 2024 08:04

lucian-ioan self-assigned this Aug 30, 2024

lucian-ioan marked this pull request as ready for review August 30, 2024 08:16

lucian-ioan requested a review from a team as a code owner August 30, 2024 08:16

lalit-satapathy reviewed Aug 30, 2024

View reviewed changes

ishleenk17 reviewed Sep 2, 2024

View reviewed changes

lucian-ioan and others added 10 commits September 2, 2024 12:33

revert fields metric types

3e86570

redo system test

260ef06

add changelog entry

953b653

add snapshot field

e1af4af

use group for host

56c61bd

update info field

28978e0

add triggered alarms

6c72eb4

update manifest file

9d3b1a0

Merge branch 'main' into update_vsphere_virtual_machine_integration

3f909da

add more metric types

a782210

lucian-ioan requested a review from ishleenk17 September 18, 2024 09:43

lucian-ioan added 3 commits September 18, 2024 13:03

change snapshot info to group

8e01de8

update readme

d6b9916

update type

eca6813

ishleenk17 reviewed Sep 19, 2024

View reviewed changes

lucian-ioan added 4 commits September 19, 2024 15:31

default period description

649c864

update version to 1.15.0-next

d2f467e

update triggered alarms

6b27dc4

add system tests

f546aa8

lucian-ioan requested a review from ishleenk17 September 19, 2024 12:43

lucian-ioan added 2 commits September 19, 2024 15:46

Merge remote-tracking branch 'origin' into update_vsphere_virtual_mac…

9c318ec

…hine_integration

update readme

75b3a08

add tag in stream.yml.hbs

4aa0c34

lucian-ioan requested a review from niraj-elastic September 19, 2024 13:02

niraj-elastic previously requested changes Sep 19, 2024

View reviewed changes

...vsphere/data_stream/virtualmachine/_dev/test/pipeline/test-virtualmachine.json-expected.json Show resolved Hide resolved

lucian-ioan and others added 2 commits September 20, 2024 08:07

update IPs and domains

ba0636d

Merge branch 'main' into update_vsphere_virtual_machine_integration

a009454

lucian-ioan requested review from lalit-satapathy and niraj-elastic September 20, 2024 05:15

ishleenk17 approved these changes Sep 20, 2024

View reviewed changes

lucian-ioan merged commit 69dc708 into elastic:main Sep 20, 2024
5 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[vSphere][virtualmachine] Add support for additional metrics #10942

[vSphere][virtualmachine] Add support for additional metrics #10942

lucian-ioan commented Aug 30, 2024 •

edited

Loading

elasticmachine commented Aug 30, 2024

lalit-satapathy Aug 30, 2024

lucian-ioan Sep 2, 2024

ishleenk17 Sep 2, 2024

ishleenk17 Sep 2, 2024

lucian-ioan Sep 2, 2024

ishleenk17 Sep 19, 2024

ishleenk17 Sep 19, 2024

ishleenk17 Sep 19, 2024

niraj-elastic commented Sep 19, 2024

ishleenk17 commented Sep 19, 2024

ishleenk17 left a comment

elastic-sonarqube bot commented Sep 20, 2024

elasticmachine commented Sep 20, 2024

elastic-vault-github-plugin-prod bot commented Sep 26, 2024

		@@ -1,4 +1,9 @@
		# newer versions go on top
		- version: "1.14.0-next"

[vSphere][virtualmachine] Add support for additional metrics #10942

[vSphere][virtualmachine] Add support for additional metrics #10942

Conversation

lucian-ioan commented Aug 30, 2024 • edited Loading

Proposed commit message

Checklist

Author's Checklist

Author's Checklist

How to test this PR locally

Related issues

Screenshots

elasticmachine commented Aug 30, 2024

🚀 Benchmarks report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

niraj-elastic commented Sep 19, 2024

ishleenk17 commented Sep 19, 2024

ishleenk17 left a comment

Choose a reason for hiding this comment

elastic-sonarqube bot commented Sep 20, 2024

Quality Gate passed

elasticmachine commented Sep 20, 2024

💚 Build Succeeded

History

elastic-vault-github-plugin-prod bot commented Sep 26, 2024

lucian-ioan commented Aug 30, 2024 •

edited

Loading