Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cherry-pick #20812 to 7.x: [Metricbeat] Add billing metricset into googlecloud module #20923

Merged
merged 3 commits into from
Sep 4, 2020
Merged

Cherry-pick #20812 to 7.x: [Metricbeat] Add billing metricset into googlecloud module #20923

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ef2c8d8
[Metricbeat] Add billing metricset into googlecloud module (#20812)
kaiyan-sheng Sep 2, 2020
eefafd7
Merge remote-tracking branch 'upstream/7.x' into backport_20812_7.x
kaiyan-sheng Sep 3, 2020
b427288
Merge branch '7.x' into backport_20812_7.x
kaiyan-sheng Sep 3, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -746,6 +746,7 @@ field. You can revert this change by configuring tags for the module and omittin
- Add host inventory metrics to googlecloud compute metricset. {pull}20391[20391]
- Add host inventory metrics to azure compute_vm metricset. {pull}20641[20641]
- Request prometheus endpoints to be gzipped by default {pull}20766[20766]
- Add billing metricset into googlecloud module. {pull}20812[20812] {issue}20738[20738]

*Packetbeat*

Expand Down
424 changes: 212 additions & 212 deletions NOTICE.txt
View file
Open in desktop

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ go 1.14
require (
4d63.com/tz v1.1.1-0.20191124060701-6d37baae851b
cloud.google.com/go v0.51.0
cloud.google.com/go/bigquery v1.0.1
cloud.google.com/go/pubsub v1.0.1
cloud.google.com/go/storage v1.0.0
code.cloudfoundry.org/go-diodes v0.0.0-20190809170250-f77fb823c7ee // indirect
Expand Down
42 changes: 42 additions & 0 deletions metricbeat/docs/fields.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19866,6 +19866,48 @@ type: object

--

[float]
=== billing

Google Cloud Billing metrics


*`googlecloud.billing.cost_type`*::
+
--
Cost types include regular, tax, adjustment, and rounding_error.

type: keyword

--

*`googlecloud.billing.invoice_month`*::
+
--
Billing report month.

type: keyword

--

*`googlecloud.billing.project_id`*::
+
--
Project ID of the billing report belongs to.

type: keyword

--

*`googlecloud.billing.total`*::
+
--
Total billing amount.

type: float

--

[float]
=== compute

Expand Down
Binary file added metricbeat/docs/images/metricbeat-googlecloud-billing-overview.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
27 changes: 26 additions & 1 deletion metricbeat/docs/modules/googlecloud.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -149,9 +149,20 @@ For example, if Compute Metricset fetches 14 metrics (which is the number of met

[float]
== Metricsets
Currently, we have `compute`, `loadbalancing`, `pubsub`, `metrics` and
Currently, we have `billing`, `compute`, `loadbalancing`, `pubsub`, `metrics` and
`storage` metricset in `googlecloud` module.

[float]
=== `billing`
This metricset fetches billing metrics from https://cloud.google.com/bigquery[GCP BigQuery]
Cloud Billing allows users to export billing data into BigQuery automatically
throughout the day. This metricset gets access to the daily cost detail table
periodically to export billing metrics for further analysis.

The `billing` metricset comes with a predefined dashboard:

image::./images/metricbeat-googlecloud-billing-overview.png[]

[float]
=== `compute`
This metricset fetches metrics from https://cloud.google.com/compute/[Compute Engine]
Expand Down Expand Up @@ -275,13 +286,25 @@ metricbeat.modules:
- "instance/cpu/usage_time"
- "instance/cpu/utilization"
- "instance/uptime"

- module: googlecloud
metricsets:
- billing
period: 24h
project_id: "your project id"
credentials_file_path: "your JSON credentials file path"
dataset_id: "dataset id"
table_pattern: "table pattern"
cost_type: "regular"
----

[float]
=== Metricsets

The following metricsets are available:

* <<metricbeat-metricset-googlecloud-billing,billing>>

* <<metricbeat-metricset-googlecloud-compute,compute>>

* <<metricbeat-metricset-googlecloud-loadbalancing,loadbalancing>>
Expand All @@ -292,6 +315,8 @@ The following metricsets are available:

* <<metricbeat-metricset-googlecloud-storage,storage>>

include::googlecloud/billing.asciidoc[]

include::googlecloud/compute.asciidoc[]

include::googlecloud/loadbalancing.asciidoc[]
Expand Down
24 changes: 24 additions & 0 deletions metricbeat/docs/modules/googlecloud/billing.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
////
This file is generated! See scripts/mage/docs_collector.go
////

[[metricbeat-metricset-googlecloud-billing]]
[role="xpack"]
=== Google Cloud Platform billing metricset

beta[]

include::../../../../x-pack/metricbeat/module/googlecloud/billing/_meta/docs.asciidoc[]


==== Fields

For a description of each field in the metricset, see the
<<exported-fields-googlecloud,exported fields>> section.

Here is an example document generated by this metricset:

[source,json]
----
include::../../../../x-pack/metricbeat/module/googlecloud/billing/_meta/data.json[]
----
3 changes: 2 additions & 1 deletion metricbeat/docs/modules_list.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,8 @@ This file is generated! See scripts/mage/docs_collector.go
.2+| .2+| |<<metricbeat-metricset-golang-expvar,expvar>>
|<<metricbeat-metricset-golang-heap,heap>>
|<<metricbeat-module-googlecloud,Google Cloud Platform>> beta[] |image:./images/icon-yes.png[Prebuilt dashboards are available] |
.5+| .5+| |<<metricbeat-metricset-googlecloud-compute,compute>> beta[]
.6+| .6+| |<<metricbeat-metricset-googlecloud-billing,billing>> beta[]
|<<metricbeat-metricset-googlecloud-compute,compute>> beta[]
|<<metricbeat-metricset-googlecloud-loadbalancing,loadbalancing>> beta[]
|<<metricbeat-metricset-googlecloud-metrics,metrics>> beta[]
|<<metricbeat-metricset-googlecloud-pubsub,pubsub>> beta[]
Expand Down
1 change: 1 addition & 0 deletions x-pack/metricbeat/include/list.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 10 additions & 0 deletions x-pack/metricbeat/metricbeat.reference.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -556,6 +556,16 @@ metricbeat.modules:
- "instance/cpu/utilization"
- "instance/uptime"

- module: googlecloud
metricsets:
- billing
period: 24h
project_id: "your project id"
credentials_file_path: "your JSON credentials file path"
dataset_id: "dataset id"
table_pattern: "table pattern"
cost_type: "regular"

#------------------------------- Graphite Module -------------------------------
- module: graphite
metricsets: ["server"]
Expand Down
10 changes: 10 additions & 0 deletions x-pack/metricbeat/module/googlecloud/_meta/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,13 @@
- "instance/cpu/usage_time"
- "instance/cpu/utilization"
- "instance/uptime"

- module: googlecloud
metricsets:
- billing
period: 24h
project_id: "your project id"
credentials_file_path: "your JSON credentials file path"
dataset_id: "dataset id"
table_pattern: "table pattern"
cost_type: "regular"
13 changes: 12 additions & 1 deletion x-pack/metricbeat/module/googlecloud/_meta/docs.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,9 +139,20 @@ For example, if Compute Metricset fetches 14 metrics (which is the number of met

[float]
== Metricsets
Currently, we have `compute`, `loadbalancing`, `pubsub`, `metrics` and
Currently, we have `billing`, `compute`, `loadbalancing`, `pubsub`, `metrics` and
`storage` metricset in `googlecloud` module.

[float]
=== `billing`
This metricset fetches billing metrics from https://cloud.google.com/bigquery[GCP BigQuery]
Cloud Billing allows users to export billing data into BigQuery automatically
throughout the day. This metricset gets access to the daily cost detail table
periodically to export billing metrics for further analysis.

The `billing` metricset comes with a predefined dashboard:

image::./images/metricbeat-googlecloud-billing-overview.png[]

[float]
=== `compute`
This metricset fetches metrics from https://cloud.google.com/compute/[Compute Engine]
Expand Down
Loading