Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Telegraf 0.2.2 panic #389

Closed
rvrignaud opened this issue Nov 23, 2015 · 6 comments · Fixed by #391
Closed

Telegraf 0.2.2 panic #389

rvrignaud opened this issue Nov 23, 2015 · 6 comments · Fixed by #391

Comments

@rvrignaud
Copy link

Hello,

I'm using telegraf to scrape prometheus' kubernetes endpoints.
I tried to upgrade my telegraf from 0.1.9 to 0.2.2.
It panicked after complaining a lot about NaN. Please find the logs below.
I'm happy to provide more informations and help in debugging if needed.
I did a rollback to 0.1.9: no complain about NaN and no panic.

2015/11/23 17:49:21 Loaded plugins: elasticsearch exec prometheus
2015/11/23 17:49:21 Tags enabled:
2015/11/23 17:49:21 Agent Config: Interval:2m0s, Debug:false, Hostname:"telegraf-5-6n0e2", Flush Interval:10.183959985s
2015-11-23 17:49:22,055 INFO success: telegraf entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error in plugin [elasticsearch]: elasticsearch: got unexpected type <nil> with value <nil> (network)
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_containers_per_pod_count]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_worker_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_pod_start_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_rest_client_request_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_container_manager_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Error adding point [prometheus_kubelet_docker_operations_latency_microseconds]: NaN is an unsupported value for field value
2015/11/23 17:50:00 Gathered metrics, (2m0s interval), from 3 plugins in 623.598043ms
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x4ffa14]

goroutine 213 [running]:
github.com/influxdb/influxdb/client/v2.(*client).Write(0xc8201c2b40, 0x7f1e71ca7268, 0xc82171f920, 0x0, 0x0)
    /home/telebuild/go/src/github.com/influxdb/telegraf/Godeps/_workspace/src/github.com/influxdb/influxdb/client/v2/client.go:354 +0xbe4
github.com/influxdb/telegraf/outputs/influxdb.(*InfluxDB).Write(0xc820148b40, 0xc821b22000, 0x42ac, 0x4c00, 0x0, 0x0)
    /home/telebuild/go/src/github.com/influxdb/telegraf/outputs/influxdb/influxdb.go:145 +0x3aa
github.com/influxdb/telegraf.(*Agent).writeOutput(0xc8201863f0, 0xc821b22000, 0x42ac, 0x4c00, 0xc820137a60, 0xc820146a20, 0xc82164be50)
    /home/telebuild/go/src/github.com/influxdb/telegraf/agent.go:332 +0x116
created by github.com/influxdb/telegraf.(*Agent).flush
    /home/telebuild/go/src/github.com/influxdb/telegraf/agent.go:372 +0x117

goroutine 1 [select]:
github.com/influxdb/telegraf.(*Agent).Run(0xc8201863f0, 0xc820146a20, 0x0, 0x0)
    /home/telebuild/go/src/github.com/influxdb/telegraf/agent.go:487 +0x98b
main.main()
    /home/telebuild/go/src/github.com/influxdb/telegraf/cmd/telegraf/telegraf.go:159 +0x1712

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1696 +0x1

goroutine 5 [syscall]:
os/signal.loop()
    /usr/local/go/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
    /usr/local/go/src/os/signal/signal_unix.go:28 +0x37

goroutine 31 [select, locked to thread]:
runtime.gopark(0x10921b8, 0xc820023728, 0xec5110, 0x6, 0x7f1e71c93918, 0x2)
    /usr/local/go/src/runtime/proc.go:185 +0x163
runtime.selectgoImpl(0xc820023728, 0x0, 0x18)
    /usr/local/go/src/runtime/select.go:392 +0xa64
runtime.selectgo(0xc820023728)
    /usr/local/go/src/runtime/select.go:212 +0x12
runtime.ensureSigM.func1()
    /usr/local/go/src/runtime/signal1_unix.go:227 +0x353
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1696 +0x1

goroutine 169 [IO wait]:
net.runtime_pollWait(0x7f1e71c93488, 0x72, 0xc820014140)
    /usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc8202659c0, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc8202659c0, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc820265960, 0xc8203c9000, 0x1000, 0x1000, 0x0, 0x7f1e74aea050, 0xc820014140)
    /usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc82002a1e8, 0xc8203c9000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/net.go:172 +0xe4
net/http.noteEOFReader.Read(0x7f1e71c938b8, 0xc82002a1e8, 0xc8201a5de8, 0xc8203c9000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/http/transport.go:1370 +0x67
net/http.(*noteEOFReader).Read(0xc8202438a0, 0xc8203c9000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    <autogenerated>:126 +0xd0
bufio.(*Reader).fill(0xc820019560)
    /usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc820019560, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc8201a5d90)
    /usr/local/go/src/net/http/transport.go:876 +0xf7
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:685 +0xc78

goroutine 33 [select]:
github.com/influxdb/telegraf.(*Agent).flusher(0xc8201863f0, 0xc820146a20, 0xc820146c60, 0x0, 0x0)
    /home/telebuild/go/src/github.com/influxdb/telegraf/agent.go:389 +0x448
github.com/influxdb/telegraf.(*Agent).Run.func1(0xc82021fbe0, 0xc8201863f0, 0xc820146a20, 0xc820146c60)
    /home/telebuild/go/src/github.com/influxdb/telegraf/agent.go:448 +0x7f
created by github.com/influxdb/telegraf.(*Agent).Run
    /home/telebuild/go/src/github.com/influxdb/telegraf/agent.go:452 +0x41f

goroutine 32 [chan receive]:
main.main.func1(0xc820146a80, 0xc820146a20)
    /home/telebuild/go/src/github.com/influxdb/telegraf/cmd/telegraf/telegraf.go:139 +0x36
created by main.main
    /home/telebuild/go/src/github.com/influxdb/telegraf/cmd/telegraf/telegraf.go:141 +0x1031

goroutine 114 [IO wait]:
net.runtime_pollWait(0x7f1e71c93548, 0x72, 0xc820014140)
    /usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc8202ea060, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc8202ea060, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc8202ea000, 0xc8202ee000, 0x1000, 0x1000, 0x0, 0x7f1e74aea050, 0xc820014140)
    /usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc8202ec000, 0xc8202ee000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/net.go:172 +0xe4
net/http.noteEOFReader.Read(0x7f1e71c938b8, 0xc8202ec000, 0xc8201a5918, 0xc8202ee000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/http/transport.go:1370 +0x67
net/http.(*noteEOFReader).Read(0xc8202e2240, 0xc8202ee000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    <autogenerated>:126 +0xd0
bufio.(*Reader).fill(0xc8202e60c0)
    /usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc8202e60c0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc8201a58c0)
    /usr/local/go/src/net/http/transport.go:876 +0xf7
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:685 +0xc78

goroutine 115 [select]:
net/http.(*persistConn).writeLoop(0xc8201a58c0)
    /usr/local/go/src/net/http/transport.go:1009 +0x40c
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:686 +0xc9d

goroutine 202 [IO wait]:
net.runtime_pollWait(0x7f1e71c94e80, 0x72, 0xc820014140)
    /usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc82024ef40, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc82024ef40, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc82024eee0, 0xc82045e000, 0x1000, 0x1000, 0x0, 0x7f1e74aea050, 0xc820014140)
    /usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc8202ec120, 0xc82045e000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/net.go:172 +0xe4
net/http.noteEOFReader.Read(0x7f1e71c938b8, 0xc8202ec120, 0xc8201a59c8, 0xc82045e000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/http/transport.go:1370 +0x67
net/http.(*noteEOFReader).Read(0xc820397000, 0xc82045e000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    <autogenerated>:126 +0xd0
bufio.(*Reader).fill(0xc8202e7380)
    /usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc8202e7380, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc8201a5970)
    /usr/local/go/src/net/http/transport.go:876 +0xf7
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:685 +0xc78

goroutine 140 [IO wait]:
net.runtime_pollWait(0x7f1e71c94dc0, 0x72, 0xc820014140)
    /usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc8202eac30, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc8202eac30, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc8202eabd0, 0xc82038e000, 0x1000, 0x1000, 0x0, 0x7f1e74aea050, 0xc820014140)
    /usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc8202ec0d0, 0xc82038e000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/net.go:172 +0xe4
net/http.noteEOFReader.Read(0x7f1e71c938b8, 0xc8202ec0d0, 0xc8201a5b28, 0xc82038e000, 0x1000, 0x1000, 0xc820316000, 0x0, 0x0)
    /usr/local/go/src/net/http/transport.go:1370 +0x67
net/http.(*noteEOFReader).Read(0xc8202e3a80, 0xc82038e000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    <autogenerated>:126 +0xd0
bufio.(*Reader).fill(0xc8202e6ae0)
    /usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc8202e6ae0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc8201a5ad0)
    /usr/local/go/src/net/http/transport.go:876 +0xf7
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:685 +0xc78

goroutine 161 [IO wait]:
net.runtime_pollWait(0x7f1e71c930c8, 0x72, 0xc820014140)
    /usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc82024c7d0, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc82024c7d0, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc82024c770, 0xc8203cb000, 0x1000, 0x1000, 0x0, 0x7f1e74aea050, 0xc820014140)
    /usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc82012e2f8, 0xc8203cb000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/net.go:172 +0xe4
net/http.noteEOFReader.Read(0x7f1e71c938b8, 0xc82012e2f8, 0xc8201a5d38, 0xc8203cb000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/http/transport.go:1370 +0x67
net/http.(*noteEOFReader).Read(0xc8203617e0, 0xc8203cb000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    <autogenerated>:126 +0xd0
bufio.(*Reader).fill(0xc8202b3ec0)
    /usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc8202b3ec0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc8201a5ce0)
    /usr/local/go/src/net/http/transport.go:876 +0xf7
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:685 +0xc78

goroutine 170 [select]:
net/http.(*persistConn).writeLoop(0xc8201a5d90)
    /usr/local/go/src/net/http/transport.go:1009 +0x40c
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:686 +0xc9d

goroutine 211 [IO wait]:
net.runtime_pollWait(0x7f1e71c93608, 0x72, 0xc820014140)
    /usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc82024c920, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc82024c920, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc82024c8c0, 0xc8203cd000, 0x1000, 0x1000, 0x0, 0x7f1e74aea050, 0xc820014140)
    /usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc82012e300, 0xc8203cd000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/net.go:172 +0xe4
net/http.noteEOFReader.Read(0x7f1e71c938b8, 0xc82012e300, 0xc8201a5a78, 0xc8203cd000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/http/transport.go:1370 +0x67
net/http.(*noteEOFReader).Read(0xc820361860, 0xc8203cd000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    <autogenerated>:126 +0xd0
bufio.(*Reader).fill(0xc8203d0060)
    /usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc8203d0060, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc8201a5a20)
    /usr/local/go/src/net/http/transport.go:876 +0xf7
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:685 +0xc78

goroutine 158 [select]:
net/http.(*persistConn).writeLoop(0xc8201a5c30)
    /usr/local/go/src/net/http/transport.go:1009 +0x40c
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:686 +0xc9d

goroutine 168 [select]:
net/http.(*persistConn).writeLoop(0xc8201a5ef0)
    /usr/local/go/src/net/http/transport.go:1009 +0x40c
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:686 +0xc9d

goroutine 159 [IO wait]:
net.runtime_pollWait(0x7f1e71c92dc8, 0x72, 0xc820014140)
    /usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc82024c840, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc82024c840, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc82024c7e0, 0xc8203bd000, 0x1000, 0x1000, 0x0, 0x7f1e74aea050, 0xc820014140)
    /usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc82012e2f0, 0xc8203bd000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/net.go:172 +0xe4
net/http.noteEOFReader.Read(0x7f1e71c938b8, 0xc82012e2f0, 0xc820328058, 0xc8203bd000, 0x1000, 0x1000, 0x16, 0x0, 0x0)
    /usr/local/go/src/net/http/transport.go:1370 +0x67
net/http.(*noteEOFReader).Read(0xc820361760, 0xc8203bd000, 0x1000, 0x1000, 0xc820216c80, 0x0, 0x0)
    <autogenerated>:126 +0xd0
bufio.(*Reader).fill(0xc8202b3d40)
    /usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc8202b3d40, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc820328000)
    /usr/local/go/src/net/http/transport.go:876 +0xf7
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:685 +0xc78

goroutine 201 [select]:
net/http.(*persistConn).writeLoop(0xc8201a5e40)
    /usr/local/go/src/net/http/transport.go:1009 +0x40c
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:686 +0xc9d

goroutine 190 [IO wait]:
net.runtime_pollWait(0x7f1e71c92d08, 0x72, 0xc820014140)
    /usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc82024c990, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc82024c990, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc82024c930, 0xc8203d6000, 0x1000, 0x1000, 0x0, 0x7f1e74aea050, 0xc820014140)
    /usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc820252118, 0xc8203d6000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/net.go:172 +0xe4
net/http.noteEOFReader.Read(0x7f1e71c938b8, 0xc820252118, 0xc820328108, 0xc8203d6000, 0x1000, 0x1000, 0xc8202f6400, 0x0, 0x0)
    /usr/local/go/src/net/http/transport.go:1370 +0x67
net/http.(*noteEOFReader).Read(0xc820398a00, 0xc8203d6000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    <autogenerated>:126 +0xd0
bufio.(*Reader).fill(0xc8202391a0)
    /usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc8202391a0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc8203280b0)
    /usr/local/go/src/net/http/transport.go:876 +0xf7
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:685 +0xc78

goroutine 193 [select]:
net/http.(*persistConn).writeLoop(0xc82033e000)
    /usr/local/go/src/net/http/transport.go:1009 +0x40c
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:686 +0xc9d

goroutine 167 [IO wait]:
net.runtime_pollWait(0x7f1e71c92f48, 0x72, 0xc820014140)
    /usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc820265950, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc820265950, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc8202658f0, 0xc8201cb000, 0x1000, 0x1000, 0x0, 0x7f1e74aea050, 0xc820014140)
    /usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc82002a1e0, 0xc8201cb000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/net.go:172 +0xe4
net/http.noteEOFReader.Read(0x7f1e71c938b8, 0xc82002a1e0, 0xc8201a5f48, 0xc8201cb000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/http/transport.go:1370 +0x67
net/http.(*noteEOFReader).Read(0xc8202435c0, 0xc8201cb000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    <autogenerated>:126 +0xd0
bufio.(*Reader).fill(0xc8200192c0)
    /usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc8200192c0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc8201a5ef0)
    /usr/local/go/src/net/http/transport.go:876 +0xf7
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:685 +0xc78

goroutine 210 [select]:
net/http.(*persistConn).writeLoop(0xc8201a5ce0)
    /usr/local/go/src/net/http/transport.go:1009 +0x40c
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:686 +0xc9d

goroutine 200 [IO wait]:
net.runtime_pollWait(0x7f1e71c92c48, 0x72, 0xc820014140)
    /usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc82024edf0, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc82024edf0, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc82024ed90, 0xc8203d4000, 0x1000, 0x1000, 0x0, 0x7f1e74aea050, 0xc820014140)
    /usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc8202ec118, 0xc8203d4000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/net.go:172 +0xe4
net/http.noteEOFReader.Read(0x7f1e71c938b8, 0xc8202ec118, 0xc8201a5e98, 0xc8203d4000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/http/transport.go:1370 +0x67
net/http.(*noteEOFReader).Read(0xc820396e60, 0xc8203d4000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    <autogenerated>:126 +0xd0
bufio.(*Reader).fill(0xc8202e71a0)
    /usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc8202e71a0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc8201a5e40)
    /usr/local/go/src/net/http/transport.go:876 +0xf7
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:685 +0xc78

goroutine 192 [IO wait]:
net.runtime_pollWait(0x7f1e71c94c40, 0x72, 0xc820014140)
    /usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc8202eb170, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc8202eb170, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc8202eb110, 0xc820422000, 0x1000, 0x1000, 0x0, 0x7f1e74aea050, 0xc820014140)
    /usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc820252120, 0xc820422000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/net.go:172 +0xe4
net/http.noteEOFReader.Read(0x7f1e71c938b8, 0xc820252120, 0xc82033e058, 0xc820422000, 0x1000, 0x1000, 0xc8202f6400, 0x0, 0x0)
    /usr/local/go/src/net/http/transport.go:1370 +0x67
net/http.(*noteEOFReader).Read(0xc820398a80, 0xc820422000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    <autogenerated>:126 +0xd0
bufio.(*Reader).fill(0xc820239320)
    /usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc820239320, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc82033e000)
    /usr/local/go/src/net/http/transport.go:876 +0xf7
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:685 +0xc78
@sparrc
Copy link
Contributor

sparrc commented Nov 23, 2015

@rvrignaud What plugins and outputs are you using? Only the prometheus plugin & influxdb output? Are the values actually NaN? How do they appear in influxdb?

@rvrignaud
Copy link
Author

I have only influxdb output (I use influxdb 0.9.4.1) and only prometheus plugin (an array with 13 urls).
The prometheus urls do seem to have NaN values (not checked all of them) for the complaining tags, I can PV you an output if needed.
I'm not sure what exactly do you mean by how do they appear in influxdb. I can throw a bunch of influxdb queries if you ask them.

@sparrc
Copy link
Contributor

sparrc commented Nov 23, 2015

@rvrignaud I'm just trying to understand what the NaN metrics actually look like when they reach InfluxDB.

When using Telegraf 0.1.9, could you make a query on a NaN metric and show me how it looks in the DB? Something simple like SELECT * FROM prometheus_kubelet_pod_start_latency_microseconds would do

@rvrignaud
Copy link
Author

> SELECT * FROM prometheus_kubelet_pod_start_latency_microseconds;
ERR: invalid character 'j' looking for beginning of value

Others queries are working fine

@sparrc
Copy link
Contributor

sparrc commented Nov 23, 2015

@rvrignaud I see, it looks like InfluxDB 0.9.5 has begun explicitly banning NaN values, because it has never actually supported them anyways.

I think the fix is going to be for Telegraf to just drop the NaNs before trying to even create points out of them. It looks like the panic is coming from the influxdb client so that should probably be fixed as well in the influxdb repo.

sparrc added a commit that referenced this issue Nov 23, 2015
sparrc added a commit that referenced this issue Nov 23, 2015
@sparrc
Copy link
Contributor

sparrc commented Dec 1, 2015

@rvrignaud 0.2.3 is now available and this is fixed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants