Closed
Description
When using metricbeat-7.6.0-darwin-x86_64, commenting out any of the metricsets in /etc/metricbeat/modules.d/elasticsearch-xpack.yml, such as:
- module: elasticsearch
metricsets:
- cluster_stats
- index
- index_recovery
- index_summary
- node_stats
- shard
# - ccr
- enrich
- ml_job
results in an error message:
2020-02-24T12:38:35.193+0100 ERROR instance/beat.go:933 Exiting: The elasticsearch module with xpack.enabled: true must have metricsets: [ccr enrich cluster_stats index index_recovery index_summary ml_job node_stats shard]
Exiting: The elasticsearch module with xpack.enabled: true must have metricsets: [ccr enrich cluster_stats index index_recovery index_summary ml_job node_stats shard]
Related issues:
- [6.8] Difficulty configuring Elasticsearch module in Metricbeat #14963
- metricbeat. elasticsearch-xpack module failed to start. #14984
For some context, this came up while trying to use the module with an Elastic Cloud deployment which apparently doesn't support the CCR stats endpoint. Instead, metricbeat logs:
2020-02-24T12:47:19.888+0100 INFO module/wrapper.go:252 Error fetching data for metricset elasticsearch.ccr: HTTP error 405 in : 405 Method Not Allowed
Interestingly, I do see documents created in the metricbeat index for this document with:
{
"@timestamp" : "2020-02-24T11:33:47.592Z",
"agent" : {
"type" : "metricbeat",
"ephemeral_id" : "$$$",
"hostname" : "$$$",
"id" : "c912fcc4-90fa-4b77-8d12-0587ec040ed5",
"version" : "7.6.0"
},
"event" : {
"duration" : 730521798,
"dataset" : "elasticsearch.ccr",
"module" : "elasticsearch"
},
"metricset" : {
"name" : "ccr",
"period" : 10000
},
"error" : {
"message" : "HTTP error 405 in : 405 Method Not Allowed"
},
"service" : {
"address" : "$$$",
"type" : "elasticsearch"
},
"ecs" : {
"version" : "1.4.0"
},
"host" : {
"name" : "$$$",
"architecture" : "x86_64",
"os" : {
"build" : "17G10021",
"platform" : "darwin",
"version" : "10.13.6",
"family" : "darwin",
"name" : "Mac OS X",
"kernel" : "17.7.0"
},
"id" : "$$$",
"hostname" : "$$$"
}
}
However, I don't see documents generated for any of the other metricsets.
GET metricbeat-7.6.0/_search?filter_path=aggregations.history
{
"size": 0,
"query": {
"range": {
"@timestamp": {
"gte": "now-10m",
"lte": "now/m"
}
}
},
"aggs": {
"history": {
"date_histogram": {
"field": "@timestamp",
"fixed_interval": "1m"
},
"aggs": {
"metricset": {
"terms": {
"field": "metricset.name",
"size": 10
}
}
}
}
}
}
results in:
{
"aggregations" : {
"history" : {
"buckets" : [
{
"key_as_string" : "2020-02-24T11:47:00.000Z",
"key" : 1582544820000,
"doc_count" : 6,
"metricset" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "ccr",
"doc_count" : 6
}
]
}
},
{
"key_as_string" : "2020-02-24T11:48:00.000Z",
"key" : 1582544880000,
"doc_count" : 6,
"metricset" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "ccr",
"doc_count" : 6
}
]
}
},
{
"key_as_string" : "2020-02-24T11:49:00.000Z",
"key" : 1582544940000,
"doc_count" : 6,
"metricset" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "ccr",
"doc_count" : 6
}
]
}
},
{
"key_as_string" : "2020-02-24T11:50:00.000Z",
"key" : 1582545000000,
"doc_count" : 6,
"metricset" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "ccr",
"doc_count" : 6
}
]
}
},
{
"key_as_string" : "2020-02-24T11:51:00.000Z",
"key" : 1582545060000,
"doc_count" : 6,
"metricset" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "ccr",
"doc_count" : 6
}
]
}
},
{
"key_as_string" : "2020-02-24T11:52:00.000Z",
"key" : 1582545120000,
"doc_count" : 6,
"metricset" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "ccr",
"doc_count" : 6
}
]
}
},
{
"key_as_string" : "2020-02-24T11:53:00.000Z",
"key" : 1582545180000,
"doc_count" : 6,
"metricset" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "ccr",
"doc_count" : 6
}
]
}
},
{
"key_as_string" : "2020-02-24T11:54:00.000Z",
"key" : 1582545240000,
"doc_count" : 4,
"metricset" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "ccr",
"doc_count" : 4
}
]
}
}
]
}
}
}
I wonder if the failure collecting one metricset prevents the reporting of the other metricsets.
Activity