Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redesign the Jolokia input plugin (as Jolokia2) #2278

Merged
merged 2 commits into from
Sep 27, 2017

Conversation

dylanmei
Copy link
Contributor

@dylanmei dylanmei commented Jan 16, 2017

Required for all PRs:

This PR is an attempt to deprecate the initial Jolokia input plugin. The shortcomings of the initial Jolokia plugin include:

  • inability to extract more than one measurement from a single input
  • inability to capture tags from mbean names #2014

I'll gladly add other target "shortcomings" should they come up and be in scope.

Taking inspiration from the win_perf_counters input, the core change here is to tie measurements to mbeans, rather than folding all mbean requests to a single "jolokia" measurement.

  • bulk requests
  • proxy support
  • tls support
  • error handling
  • readme and examples

Feedback is welcome.
I'm available for discussion on the gophers #influxdb channel as @dylanmei.

Checkout quay.io/nordstrom/telegraf:latest to test-drive this.

  • CHANGELOG.md updated (we recommend not updating this until the PR has been approved by a maintainer)
  • Sign CLA (if not already signed)
  • README.md updated (if adding a new plugin)

@sparrc
Copy link
Contributor

sparrc commented Jan 17, 2017

cc @phemmer @mainvoid @saiello @subhachandrachandra could you take a look at this?

@sparrc
Copy link
Contributor

sparrc commented Jan 17, 2017

@dylanmei does this support bulk requests? (see #2253) nevermind, I see that the implementation is not there yet, just design of how the requests are made and structured.

Can you comment on how this compares with the Cassandra plugin, as proposed in #2242?

@dylanmei
Copy link
Contributor Author

dylanmei commented Jan 17, 2017

I don't understand #2242 exactly. But I am making a Jolokia "gatherer" that implements this input plugin and is usable by other inputs.

That way, the Cassandra input plugin could be a thin veneer on top of the Jolokia gatherer. I think this satisfies #2242.

If I was implementing #2242 after finishing this PR, I would consider doing these two things together:

  • Extract JVM specific measurements into a JVM input plugin that leverages the Jolokia "gatherer" so nobody ever again has to extract java.lang: based mbeans. Anything related to newJavaMetric could be removed from the Cassandra input: https://github.com/influxdata/telegraf/blob/master/plugins/inputs/cassandra/cassandra.go#L51
  • Delete the Cassandra input metrics config and grab everything from Cassandra. An input config based on these changes would look like a Jolokia config without any mbeans sections.

The final config, to get all the JVM and Cassandra measurements from a Jolokia agent running on Cassandra:

[[inputs.jvm]]
  name_prefix = "cassandra_"

  [inputs.jvm.jolokia_agents]
    urls = ["http://cassandra:8080/jolokia"]

[[inputs.cassandra2]]
  [inputs.cassandra2.jolokia_agents]
    urls = ["http://cassandra:8080/jolokia"]

@mainvoid
Copy link

#2242 is quite simple. Replace existing Jolokia plugin with Cassandra plugin. Both hook into jolokia and grab data out of it. However, the Cassandra plugin get the data out in a simpler and smarter way than the jolokia plugin. It eases how one configures it, compared to the jolokia plugin, as well as apparently this PR. I'm not sure why re-inventing the wheel is a necessity, but hey. I'd say, if you want to replace the jolokia plugin, make it better than the Cassandra one if possible. I see little need to have a shim layer on top of a plugin in order to grab specific metrics. All of them are presented as it is. A side effect of the merge is also, one code base to maintain. Only reason the Cassandra plugin doesn't grab other metrics out is because it has been coded to only grab java.lang and cassandra specific metrics. Remove that and it'll do what the Jolokia plugin does, and more.. Like name_drop, field_pass, etc. etc. which doesn't exist in the Jolokia plugin today. It's already there for the taking.

Also, replacing jolokia with Cassandra only needs three things. 1/ Removing 10 lines of code. 2/ Renaming it to something suitable. 3/ Profit.

It has also already proven itself in production environments.

@dylanmei
Copy link
Contributor Author

dylanmei commented Jan 17, 2017

Cassandra plugin get the data out in a simpler and smarter way than the jolokia plugin

Cassandra input plugin does not do proxying. It does not do bulk HTTP. It does not handle TLS. It does not produce measurements other than the ones it knows about: javaSomething and cassandraSomthing. And unless I'm mistaken, it requires the dozens of lines of canned config printed on the README.

It does take key-property names and turn them into tags, which is something we want with the tag_keys idea in this PR proposal.

Improvements made to Jolokia gathering could knock on to any plugin interacting with Jolokia agents.

@dylanmei dylanmei changed the title Redesign the Jolokia input plugin (as Jolokia2) WIP: Redesign the Jolokia input plugin (as Jolokia2) Jan 19, 2017
@dylanmei dylanmei force-pushed the jolokia2 branch 4 times, most recently from e63d114 to ed52c88 Compare January 23, 2017 16:16
@sparrc sparrc added this to the 1.3.0 milestone Jan 25, 2017
@dylanmei dylanmei force-pushed the jolokia2 branch 3 times, most recently from 5a51530 to d01034f Compare February 2, 2017 16:28

func (jc *Jolokia) SampleConfig() string {
return fmt.Sprintf(`
# %s
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you don't need to put the Description in your SampleConfig, telegraf will add that automatically

[[inputs.jolokia2.metric]]
name = "jvm_memory_pool"
mbean = "java.lang:name=*,type=MemoryPool"
paths = ["Usage", "PeakUsage, "CollectionUsage"]
Copy link
Contributor

@sparrc sparrc Feb 3, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you know why github is interpreting this as invalid toml?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's subtle, but there's a missing quote after PeakUsage.

url = "service:jmx:rmi:///jndi/rmi://targethost:9999/jmxrmi"
#username = ""
#password = ""
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you put sample plain-text metrics here?

Copy link

@asko asko Feb 16, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, curious if the problem of collapsing metrics with long paths is fixed.
ATM we're using Cassandra jolokia plugin and it seems that plugin collapses LastGcInfo for all heap spaces into one. For instance:

metrics = [
"/java.lang:type=GarbageCollector,name=G1%20Young%20Generation/LastGcInfo/memoryUsageAfterGc/G1%20Eden%20Space",
"/java.lang:type=GarbageCollector,name=G1%20Young%20Generation/LastGcInfo/memoryUsageAfterGc/G1%20Survivor%20Space",
"/java.lang:type=GarbageCollector,name=G1%20Young%20Generation/LastGcInfo/memoryUsageAfterGc/G1%20Old%20Gen",
"/java.lang:type=GarbageCollector,name=G1%20Young%20Generation/LastGcInfo/memoryUsageAfterGc/Metaspace",
"/java.lang:type=GarbageCollector,name=G1%20Young%20Generation/LastGcInfo/memoryUsageAfterGc/Compressed%20Class%20Space",
"/java.lang:type=GarbageCollector,name=G1%20Young%20Generation/LastGcInfo/memoryUsageAfterGc/Code%20Cache"
]

result in only following 6:

"javaGarbageCollector,mname=G1\\ Young\\ Generation,cassandra_host=test LastGcInfo_used=0,LastGcInfo_init=4513071104,LastGcInfo_committed=4160749568,LastGcInfo_max=-1 1487259680000000000\n"
"javaGarbageCollector,mname=G1\\ Young\\ Generation,cassandra_host=test LastGcInfo_max=-1,LastGcInfo_used=352321536,LastGcInfo_init=0,LastGcInfo_committed=352321536 1487259680000000000\n"
"javaGarbageCollector,mname=G1\\ Young\\ Generation,cassandra_host=test LastGcInfo_init=12666798080,LastGcInfo_committed=12666798080,LastGcInfo_max=17179869184,LastGcInfo_used=4759843152 1487259680000000000\n"
"javaGarbageCollector,mname=G1\\ Young\\ Generation,cassandra_host=test LastGcInfo_used=39882464,LastGcInfo_init=0,LastGcInfo_committed=40714240,LastGcInfo_max=-1 1487259680000000000\n"
"javaGarbageCollector,mname=G1\\ Young\\ Generation,cassandra_host=test LastGcInfo_init=0,LastGcInfo_committed=4587520,LastGcInfo_max=1073741824,LastGcInfo_used=4326520 1487259680000000000\n"
"javaGarbageCollector,mname=G1\\ Young\\ Generation,cassandra_host=test LastGcInfo_init=2555904,LastGcInfo_committed=44236800,LastGcInfo_max=251658240,LastGcInfo_used=41900864 1487259680000000000\n"

whereas they should (perhaps) have path in their names providing metric names sort of like (or in any way where path

"javaGarbageCollector,mname=G1\\ Young\\ Generation,cassandra_host=test memoryUsageAfterGc_Metaspace_LastGcInfo_init=2555904,memoryUsageAfterGc_Metaspace_LastGcInfo_committed=44236800,memoryUsageAfterGc_Metaspace_LastGcInfo_max=251658240,memoryUsageAfterGc_Metaspace_LastGcInfo_used=41900864 1487259680000000000\n"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes we are collapsing paths, and coalescing series as much as we can so we don't produce nonsensical measurements.

For example, if you were to set it up so that Jolokia returned this:

foo,gc="G1 Garbage Collector" x=1
foo,gc="G1 Garbage Collector" y=2

These would get compacted into: foo,gc="G1 Garbage Collector" x=1,x=2

That happens at this point in the code is for: https://github.com/dylanmei/telegraf/blob/c9a251e7f219c326c30e48140bcffcc4e302f1d8/plugins/inputs/jolokia2/gatherer.go#L138

LastGcInfo in particular has been tricky, since Young Gen returns an object and Old Gen always returns null. I'm super confused by the design of this output!

With this updated plugin, you can create a very finely tuned "young gen" measurement for yourself, or what we've been doing is punting on it entirely and deriving just what we need from CollectionTime/CollectionCount:

  [[inputs.jolokia2.metric]]
    name     = "java_garbage_collector"
    mbean    = "java.lang:name=G1*,type=GarbageCollector"
    paths = ["CollectionTime", "CollectionCount"]

Copy link

@asko asko Feb 17, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dylanmei still not sure I get how it is possible to configure plugin for getting stats for utilization of different heap spaces (Eden, SurvivorSpace and OldGen etc which are only in paths which as I understand are collapsed completely). Perhaps I am missing something really-really basic 😆

@sparrc
Copy link
Contributor

sparrc commented Feb 3, 2017

@dylanmei can you explain some of the changes that you made to config.go? Not saying they're wrong but I'd like to know why they are included in this PR.

@dylanmei
Copy link
Contributor Author

dylanmei commented Feb 3, 2017

@sparrc regarding config.go: This plugin is very complex. It is my feeling that if I can simply leverage the TOML parsing and plugin creation in config.go that I can create more readable test fixtures, and the test fixtures can enhance the documentation.

This can lead to an fixture file like this: https://github.com/dylanmei/telegraf/blob/jolokia2/plugins/inputs/jolokia2/testdata/field_substitutions.toml

I realize I may be off on a tangent here. Writing tests this way has helped me immensely as I was progressing. I may decompose these fixtures files back into code now that the changes are settling down. But I think being able to pass a plain string as well as a file to config.go is a handy thing.

@dylanmei dylanmei force-pushed the jolokia2 branch 4 times, most recently from b98ecaa to c9a251e Compare February 7, 2017 14:52
@dylanmei
Copy link
Contributor Author

dylanmei commented Feb 8, 2017

I've begun running this "in production" for a Kafka cluster. The conf is largely based on this project: https://github.com/dylanmei/compose-confluent and uses a publicly available Docker image quay.io/nordstrom/telegraf:latest.

In addition to iterating on the tests,

  • The docs need polishing (including plain-text metrics output examples)
  • TLS support is on my hitlist
  • Proxy support is another TODO item
  • I'd like to provide a few common confs in an examples directory
  • I feel like tagging the mbean domain would satisfy the SOLR case that @phemmer raised

@sparrc
Copy link
Contributor

sparrc commented Feb 8, 2017

good to know @dylanmei, have any plans for implementing the proxy support? I'm not sure we can merge it until then since the current jolokia plugin does support jolokia proxies.

@dylanmei
Copy link
Contributor Author

dylanmei commented Feb 8, 2017

Yes, I do plan to put the proxy support back into place.

As noted in the README, we'll express targets with the full url (i.e, service:jmx:rmi:///jndi/rmi://targethost:9999/jmxrmi) which will address #2005.

Meaning, the config will just channel what's shown in the Jolokia docs: url, password, user.

{
  "type": "READ"
  "mbean": "java.lang:type=Threading",
  "attribute": "ThreadCount",
  "target": { 
    "url":"service:jmx:rmi:///jndi/rmi://jboss-as:8686/jmxrmi",
    "password":"admin",
    "user":"s!cr!t"
  },
}

@dylanmei
Copy link
Contributor Author

@danielnelson the changes to internal/config.go have been reverted.

Related to naming, I want to call out that there are two plugins here.

The original jolokia plugin handles both agent mode and proxy mode in one plugin. I found this very difficult to sustain in the toml configuration and so this PR as it stands has them as separate plugins inputs.jolokia2 (agent mode) and inputs.jolokia2_proxy using the same underlying implementation.

We can

  1. take it as it is
  2. work around toml issues to roll them back into one inputs.jolokia2 plugin
  3. be very explicit about there being two plugins, one for each of the Jolokia modes: inputs.jolokia_agent and inputs.jolokia_proxy

Copy link
Contributor

@danielnelson danielnelson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The two agent and proxy configs are quite similar, what were the issues you ran into with the toml config? If it has to do with associating the proxy url to the connection url, perhaps we just need the urls to be in the same table:

[[inputs.jolokia2]]
  # default_tag_prefix      = ""
  # default_field_prefix    = ""
  # default_field_separator = "."
  
  response_timeout = "5s"

  ## Add agents to query
  [[inputs.jolokia2.servers]]
    url = "http://localhost:8080/jolokia"
    username = ""
    password = ""

    proxy_url = "service:jmx:rmi:///jndi/rmi://targethost:9999/jmxrmi"
    proxy_username = ""
    proxy_password = ""

  ## Optional SSL config
  # ssl_ca   = "/var/private/ca.pem"
  # ssl_cert = "/var/private/client.pem"
  # ssl_key  = "/var/private/client-key.pem"
  # insecure_skip_verify = false

  ## Add metrics to read
  [[inputs.jolokia2.metric]]
    name  = "java_runtime"
    mbean = "java.lang:type=Runtime"
    paths = ["Uptime"]

Two plugins is fine as well. I'll try to finish a proper review this week, but here are a couple minor things I noticed:

# default_field_separator = "."

## Add agents to query
agents = ["http://localhost:8080/jolokia"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be urls

## Add targets to query
# default_target_username = ""
# default_target_password = ""
[[inputs.jolokia_proxy.target]]
Copy link
Contributor

@danielnelson danielnelson Aug 1, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can there be more than one target per url?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the proxy can make requests to multiple targets, and each of those targets could potentially have different user/pass. This would be in a world where, for example, a distributed application's JMX endpoints are completely inaccessible except to the proxy. The proxy can have its own user/pass, and the individual targets can have their own user/pass. It's quite difficult to express all these possibilities in a single table.

[[inputs.jolokia2.metric]]
name = "kafka_log"
mbean = "kafka.log:name=*,partition=*,topic=*,type=Log"
field_name = "$1"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't see a spot where the $1 syntax was explained. Do these correspond to the * in the mbean options?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll explain this more clearly.

})
}

func (jc *JolokiaProxy) Gather(acc telegraf.Accumulator) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The method receiver is sometimes named jc and sometimes jp, lets always use the same variable name.


for _, config := range jc.Metrics {
metrics = append(metrics, NewMetric(config,
jc.DefaultFieldPrefix, jc.DefaultFieldSeparator, jc.DefaultTagPrefix))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is done every gather, but only needs to be done once. Lets put it into an initialize function and only run it once. Should be done also in Jolokia struct.

}

gatherer := NewGatherer(acc, metrics)
proxy, err := jc.buildProxy()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Building the proxy and agent is also done every gather when it could be done once. This will create a new transport each Gather which has the potential to cause problems. Also needs change in Jolokia struct

return err
}

err = gatherer.Gather(agent)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be ran in a goroutine to avoid earlier urls blocking later ones. It is not a problem in the JolokiaProxy since there is only one server, unless there are multiple targets.

You could get around this by saying there can only be one url per plugin, but the flip side of that is if you have many servers with the same configuration it would be a lot of config file to duplicate.

@danielnelson danielnelson modified the milestones: 1.4.0, 1.5.0 Aug 14, 2017
@danielnelson danielnelson added the feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin label Aug 24, 2017
@dylanmei
Copy link
Contributor Author

@danielnelson I believe I've addressed your awesome feedback, the big ones were: less mysterious documentation and also a groroutine around fetching agent metrics from multiple agents.

@danielnelson
Copy link
Contributor

Thanks @dylanmei, this will be in 1.5.0.

@dylanmei dylanmei deleted the jolokia2 branch December 3, 2017 02:37
@nemosupremo
Copy link

Sorry if this is the incorrect place for this - I was getting started with the jolokia2_agent in telegraf 1.5 to monitor kafka 1.0.0. I'm having trouble parsing the metrics I'm seeing reported when using the example configuration.

For example for per topic metrics:

  [[inputs.jolokia2_agent.metric]]
    name         = "topic"
    mbean        = "kafka.server:name=*,topic=*,type=BrokerTopicMetrics"
    field_prefix = "$1."
    tag_keys     = ["topic"]

What I'm noticing in InfluxDB are fields for .Count, MeanRate, that seem to be aggregated across Mbeans (namely AllTopicsBytesOutPerSec, AllTopicsBytesInPerSec, AllTopicsMessagesInPerSec). In the example, does tag_keys need to have name in it as well?

@dylanmei
Copy link
Contributor Author

dylanmei commented Dec 16, 2017

@nemosupremo

In the example, does tag_keys need to have name in it as well?

The declaration field_prefix = $1. is capturing name=*.

If name=Foo and it has attributes Count and MeanRate then you should see Foo.Count=x,Foo.MeanRate=y

I'm not clear what you think the problem is -- I need more info. Can you create an issue and show the line-protocol output?

@nemosupremo
Copy link

nemosupremo commented Dec 16, 2017

I should have actually included what is being sent to influxdb, with stdout:

Here are two different items, both with the same field name, but being reported with the same tags in Telegraf:

kafka_topic,host=kafka01,jolokia_agent_url=http://localhost:8778/jolokia,topic=prod-topic .MeanRate=1.175280709120574,.RateUnit="SECONDS",.OneMinuteRate=1.1678508713989282,.EventType="messages",.Count=81880,.FifteenMinuteRate=1.1179524347826097,.FiveMinuteRate=1.1277911148442847 1513442250000000000
kafka_topic,jolokia_agent_url=http://localhost:8778/jolokia,topic=prod-topic,host=kafka01 .Count=0,.FifteenMinuteRate=0,.FiveMinuteRate=0,.MeanRate=0,.RateUnit="SECONDS",.OneMinuteRate=0,.EventType="bytes" 1513442270000000000

You can see one is EventType "messages" and the other is EventType "bytes". I believe they have different JMX names.

I would expect to either see a tag (metric_name=foo) or the Fields to actually have completed names (foo.count=0)

@dylanmei
Copy link
Contributor Author

I see a problem here, which is that your fields are starting with just . and not the result of $1.

@nemosupremo
Copy link

nemosupremo commented Dec 16, 2017

The output from the jolokia server seems correct:

curl  http://localhost:8778/jolokia/read/kafka.server:name=*,topic=*,type=BrokerTopicMetrics
{
  "request": {
    "mbean": "kafka.server:name=*,topic=*,type=BrokerTopicMetrics",
    "type": "read"
  },
  "value": {
    "kafka.server:name=TotalFetchRequestsPerSec,topic=prod-topic,type=BrokerTopicMetrics": {
      "RateUnit": "SECONDS",
      "OneMinuteRate": 2.964393875e-314,
      "EventType": "requests",
      "Count": 63,
      "FifteenMinuteRate": 4.174759510437766e-37,
      "FiveMinuteRate": 1.4223410020833352e-106,
      "MeanRate": 0.000760766356235835
    },
    "kafka.server:name=FailedFetchRequestsPerSec,topic=prod-topic,type=BrokerTopicMetrics": {
      "RateUnit": "SECONDS",
      "OneMinuteRate": 0,
      "EventType": "requests",
      "Count": 0,
      "FifteenMinuteRate": 0,
      "FiveMinuteRate": 0,
      "MeanRate": 0
    },
    "kafka.server:name=MessagesInPerSec,topic=prod-topic,type=BrokerTopicMetrics": {
      "RateUnit": "SECONDS",
      "OneMinuteRate": 1.1654669777738194,
      "EventType": "messages",
      "Count": 96684,
      "FifteenMinuteRate": 1.1598221110942293,
      "FiveMinuteRate": 1.1526658491904729,
      "MeanRate": 1.1675227624956321
    },
    "kafka.server:name=BytesInPerSec,topic=prod-topic,type=BrokerTopicMetrics": {
      "RateUnit": "SECONDS",
      "OneMinuteRate": 24247.62725889159,
      "EventType": "bytes",
      "Count": 1717869132,
      "FifteenMinuteRate": 23861.129588429572,
      "FiveMinuteRate": 23573.155350780762,
      "MeanRate": 20744.397382482304
    },
    "kafka.server:name=BytesRejectedPerSec,topic=prod-topic,type=BrokerTopicMetrics": {
      "RateUnit": "SECONDS",
      "OneMinuteRate": 0,
      "EventType": "bytes",
      "Count": 0,
      "FifteenMinuteRate": 0,
      "FiveMinuteRate": 0,
      "MeanRate": 0
    },
    "kafka.server:name=BytesOutPerSec,topic=prod-topic,type=BrokerTopicMetrics": {
      "RateUnit": "SECONDS",
      "OneMinuteRate": 2.964393875e-314,
      "EventType": "bytes",
      "Count": 12445845,
      "FifteenMinuteRate": 8.418860443606704e-32,
      "FiveMinuteRate": 3.295931974580797e-101,
      "MeanRate": 150.2917479255084
    },
    "kafka.server:name=ProduceMessageConversionsPerSec,topic=prod-topic,type=BrokerTopicMetrics": {
      "RateUnit": "SECONDS",
      "OneMinuteRate": 1.1654669777738194,
      "EventType": "requests",
      "Count": 96684,
      "FifteenMinuteRate": 1.1598221110942293,
      "FiveMinuteRate": 1.1526658491904729,
      "MeanRate": 1.1675227688593328
    },
    "kafka.server:name=FetchMessageConversionsPerSec,topic=prod-topic,type=BrokerTopicMetrics": {
      "RateUnit": "SECONDS",
      "OneMinuteRate": 2.964393875e-314,
      "EventType": "requests",
      "Count": 652,
      "FifteenMinuteRate": 4.0730516115215564e-36,
      "FiveMinuteRate": 1.5654916456104026e-105,
      "MeanRate": 0.007873328005193672
    },
    "kafka.server:name=FailedProduceRequestsPerSec,topic=prod-topic,type=BrokerTopicMetrics": {
      "RateUnit": "SECONDS",
      "OneMinuteRate": 0,
      "EventType": "requests",
      "Count": 0,
      "FifteenMinuteRate": 0,
      "FiveMinuteRate": 0,
      "MeanRate": 0
    },
    "kafka.server:name=TotalProduceRequestsPerSec,topic=prod-topic,type=BrokerTopicMetrics": {
      "RateUnit": "SECONDS",
      "OneMinuteRate": 1.1406883382817614,
      "EventType": "requests",
      "Count": 91950,
      "FifteenMinuteRate": 1.1162715461060453,
      "FiveMinuteRate": 1.111917792093389,
      "MeanRate": 1.1103566089201538
    }
  },
  "timestamp": 1513455392,
  "status": 200
}

Do you think this is a problem with the plugin or with my environment?

@dylanmei
Copy link
Contributor Author

@nemosupremo Let's take this to issue #3597

maxunt pushed a commit that referenced this pull request Jun 26, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin new plugin
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants