Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add connection metricset of nats module #10095

Merged
merged 3 commits into from
Jan 17, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,6 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
*Journalbeat*

*Metricbeat*
- Add nats module with stats metricset. {pull}9825[9825]

*Packetbeat*

Expand Down Expand Up @@ -102,6 +101,8 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d

- Add `key` metricset to the Redis module. {issue}9582[9582] {pull}9657[9657] {pull}9746[9746]
- Add `socket_summary` metricset to system defaults, removing experimental tag and supporting Windows {pull}9709[9709]
- Add `nats` module with `stats` metricset. {pull}9825[9825]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice find :-) Could you perhaps move both of these lines to CHANGELOG.next.asciidoc. We recently introduced this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure! I'm in for addressing this in follow up PRs, in order to have it merged and add new metricsets on top of schema changes introduced here.

cc: @skatsaounis, @MichaelKatsoulis

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi both @ruflin, @ChrsMark. I will apply this change to the second commit I am preparing for the new pr regarding routes metricset #10149

cc: @MichaelKatsoulis

- Add `connections` metricset to the `nats` module. {pull}10095[10095]

*Packetbeat*

Expand Down
35 changes: 26 additions & 9 deletions metricbeat/docs/fields.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -17232,33 +17232,50 @@ nats Module



[float]
== stats fields
*`nats.server_id`*::
+
--
type: string

Contains nats var related metrics
The server ID


--

*`nats.stats.server_id`*::
*`nats.now`*::
+
--
type: keyword
type: date

The server ID
Server time of metric creation


--

*`nats.stats.now`*::
[float]
== connections fields

Contains nats connection related metrics



*`nats.connections.total`*::
+
--
type: date
type: integer

Server time of metric creation
The number of currently active clients


--

[float]
== stats fields

Contains nats var related metrics



*`nats.stats.uptime`*::
+
--
Expand Down
7 changes: 6 additions & 1 deletion metricbeat/docs/modules/nats.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -29,18 +29,23 @@ in <<configuration-metricbeat>>. Here is an example configuration:
----
metricbeat.modules:
- module: nats
metricsets: ["stats"]
metricsets: ["connections", "stats"]
period: 10s
hosts: ["localhost:8222"]
#stats.metrics_path: "/varz"
#connections.metrics_path: "/connz"
----

[float]
=== Metricsets

The following metricsets are available:

* <<metricbeat-metricset-nats-connections,connections>>

* <<metricbeat-metricset-nats-stats,stats>>

include::nats/connections.asciidoc[]

include::nats/stats.asciidoc[]

23 changes: 23 additions & 0 deletions metricbeat/docs/modules/nats/connections.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
////
This file is generated! See scripts/docs_collector.py
////

[[metricbeat-metricset-nats-connections]]
=== Nats connections metricset

experimental[]

include::../../../module/nats/connections/_meta/docs.asciidoc[]


==== Fields

For a description of each field in the metricset, see the
<<exported-fields-nats,exported fields>> section.

Here is an example document generated by this metricset:

[source,json]
----
include::../../../module/nats/connections/_meta/data.json[]
----
3 changes: 2 additions & 1 deletion metricbeat/docs/modules_list.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,8 @@ This file is generated! See scripts/docs_collector.py
.2+| .2+| |<<metricbeat-metricset-mysql-galera_status,galera_status>> experimental[]
|<<metricbeat-metricset-mysql-status,status>>
|<<metricbeat-module-nats,Nats>> experimental[] |image:./images/icon-no.png[No prebuilt dashboards] |
.1+| .1+| |<<metricbeat-metricset-nats-stats,stats>> experimental[]
.2+| .2+| |<<metricbeat-metricset-nats-connections,connections>> experimental[]
|<<metricbeat-metricset-nats-stats,stats>> experimental[]
|<<metricbeat-module-nginx,Nginx>> |image:./images/icon-yes.png[Prebuilt dashboards are available] |
.1+| .1+| |<<metricbeat-metricset-nginx-stubstatus,stubstatus>>
|<<metricbeat-module-php_fpm,PHP_FPM>> beta[] |image:./images/icon-no.png[No prebuilt dashboards] |
Expand Down
1 change: 1 addition & 0 deletions metricbeat/include/list.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion metricbeat/metricbeat.reference.yml
Original file line number Diff line number Diff line change
Expand Up @@ -518,10 +518,11 @@ metricbeat.modules:

#-------------------------------- Nats Module --------------------------------
- module: nats
metricsets: ["stats"]
metricsets: ["connections", "stats"]
period: 10s
hosts: ["localhost:8222"]
#stats.metrics_path: "/varz"
#connections.metrics_path: "/connz"

#-------------------------------- Nginx Module -------------------------------
- module: nginx
Expand Down
3 changes: 2 additions & 1 deletion metricbeat/module/nats/_meta/config.reference.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
- module: nats
metricsets: ["stats"]
metricsets: ["connections", "stats"]
period: 10s
hosts: ["localhost:8222"]
#stats.metrics_path: "/varz"
#connections.metrics_path: "/connz"
5 changes: 3 additions & 2 deletions metricbeat/module/nats/_meta/config.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
- module: nats
metricsets: ["stats"]
metricsets: ["connections", "stats"]
period: 10s
hosts: ["localhost:8222"]
stats.metrics_path: "/varz"
#stats.metrics_path: "/varz"
#connections.metrics_path: "/connz"
8 changes: 8 additions & 0 deletions metricbeat/module/nats/_meta/fields.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,11 @@
description: >
`nats` contains statistics that were read from Nats
fields:
- name: server_id
type: string
description: >
The server ID
- name: now
type: date
description: >
Server time of metric creation
21 changes: 21 additions & 0 deletions metricbeat/module/nats/connections/_meta/data.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"@timestamp":"2016-05-23T08:05:34.853Z",
"beat":{
"hostname":"beathost",
"name":"beathost"
},
"metricset":{
"host":"localhost",
"module":"nats",
"name":"connections",
"rtt":44269
},
"nats":{
"server_id": "bUAdpRFtMWddIBWw80Yd9D",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you expect more info about the server then just the id? If yes, perhaps we should make this server.id.

Copy link
Member Author

@ChrsMark ChrsMark Jan 17, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had similar thoughts about merging server_id and now fields in something like:

{
 "server": {
    "id": "foobar",
    "reporting_time": ....
 }
}

How about introducing this too in follow up PRs?

cc: @skatsaounis, @MichaelKatsoulis

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea. Is it reporting_time or is this actually the time of the server? server.time?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically, it's the same thing. So let's keep it as sever.time for simplicity.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ChrsMark Can you follow up on this and also do the change for the changelog?

"now": "2018-12-28T12:33:53.026865597Z",
"connections":{
"total": 10
}
},
"type":"metricsets"
}
1 change: 1 addition & 0 deletions metricbeat/module/nats/connections/_meta/docs.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
This is the connections metricset of the module nats.
9 changes: 9 additions & 0 deletions metricbeat/module/nats/connections/_meta/fields.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
- name: connections
type: group
description: >
Contains nats connection related metrics
fields:
- name: total
type: integer
description: >
The number of currently active clients
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"server_id": "bUAdpRFtMWddIBWw80Yd9D",
"now": "2018-12-28T12:33:53.026865597Z",
"total": 10
}
98 changes: 98 additions & 0 deletions metricbeat/module/nats/connections/connections.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
// Licensed to Elasticsearch B.V. under one or more contributor
// license agreements. See the NOTICE file distributed with
// this work for additional information regarding copyright
// ownership. Elasticsearch B.V. licenses this file to you under
// the Apache License, Version 2.0 (the "License"); you may
// not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.

package connections

import (
"github.com/elastic/beats/libbeat/common/cfgwarn"
"github.com/elastic/beats/libbeat/logp"
"github.com/elastic/beats/metricbeat/helper"
"github.com/elastic/beats/metricbeat/helper/elastic"
"github.com/elastic/beats/metricbeat/mb"
"github.com/elastic/beats/metricbeat/mb/parse"
)

const (
defaultScheme = "http"
defaultPath = "/connz"
)

var (
hostParser = parse.URLHostParserBuilder{
DefaultScheme: defaultScheme,
DefaultPath: defaultPath,
PathConfigKey: "connections.metrics_path",
}.Build()
)

// init registers the MetricSet with the central registry as soon as the program
// starts. The New function will be called later to instantiate an instance of
// the MetricSet for each host defined in the module's configuration. After the
// MetricSet has been created then Fetch will begin to be called periodically.
func init() {
mb.Registry.MustAddMetricSet("nats", "connections", New,
mb.WithHostParser(hostParser),
mb.DefaultMetricSet(),
)
}

// MetricSet holds any configuration or state information. It must implement
// the mb.MetricSet interface. And this is best achieved by embedding
// mb.BaseMetricSet because it implements all of the required mb.MetricSet
// interface methods except for Fetch.
type MetricSet struct {
mb.BaseMetricSet
http *helper.HTTP
Log *logp.Logger
}

// New creates a new instance of the MetricSet. New is responsible for unpacking
// any MetricSet specific configuration options if there are any.
func New(base mb.BaseMetricSet) (mb.MetricSet, error) {
cfgwarn.Experimental("The nats connections metricset is experimental.")

config := struct{}{}
if err := base.Module().UnpackConfig(&config); err != nil {
return nil, err
}

http, err := helper.NewHTTP(base)
if err != nil {
return nil, err
}
return &MetricSet{
base,
http,
logp.NewLogger("nats"),
}, nil
}

// Fetch methods implements the data gathering and data conversion to the right
// format. It publishes the event which is then forwarded to the output. In case
// of an error set the Error field of mb.Event or simply call report.Error().
func (m *MetricSet) Fetch(r mb.ReporterV2) {
content, err := m.http.FetchContent()
if err != nil {
elastic.ReportAndLogError(err, r, m.Log)
return
}
err = eventMapping(r, content)
if err != nil {
elastic.ReportAndLogError(err, r, m.Log)
return
}
}
Loading