Update statsd storage - issue #724 #798

jmaitrehenry · 2015-07-05T19:16:13Z

Hi, I just finish a statsd storage for cadvisor, if you wish try it and send me some feedback about how I did it, you can test it by running my docker image jmaitrehenry/cadvisor or by checking this PR

This is how I run it:

docker run \
  --volume=/:/rootfs:ro \
  --volume=/var/run:/var/run:rw \
  --volume=/sys:/sys:ro \
  --volume=/var/lib/docker/:/var/lib/docker:ro \
  --publish=8080:8080 \
  --detach=true \
  --name=cadvisor \
  jmaitrehenry/cadvisor \
  -storage_driver=statsd \
  -storage_driver_host=192.168.59.3:8125 \
  -storage_driver_db=docker_001

With storage_driver_db is use as a prefix for all stats, I use it for prefixing stats with the docker node hostname for example.

This is a sample of gauge receive by statsd :

  gauges: 
   { [...]
     'docker_001.lonely_yonath.memory_working_set': 47902720,
     'docker_001.lonely_yonath.rx_bytes': 8915403,
     'docker_001.lonely_yonath.rx_errors': 0,
     'docker_001.lonely_yonath.tx_bytes': 1473345,
     'docker_001.lonely_yonath.tx_errors': 0,
     'docker_001.lonely_yonath.cpu_cumulative_usage': 15310631766,
     'docker_001.lonely_yonath.memory_usage': 61747200,
     'docker_001.lonely_yonath.-dev-sda1.fs_limit': 19507089408,
     'docker_001.lonely_yonath.-dev-sda1.fs_usage': 12288,
     'docker_001.lonely_yonath.fs_summary.fs_limit': 19507089408,
     'docker_001.lonely_yonath.fs_summary.fs_usage': 12288,
     'docker_001.cadvisor.tx_bytes': 7236437,
     'docker_001.cadvisor.tx_errors': 0,
     'docker_001.cadvisor.cpu_cumulative_usage': 106884766265,
     'docker_001.cadvisor.memory_usage': 26087424,
     'docker_001.cadvisor.memory_working_set': 26087424,
     'docker_001.cadvisor.rx_bytes': 5218,
     'docker_001.cadvisor.rx_errors': 0,
     'docker_001.cadvisor.-dev-sda1.fs_limit': 19507089408,
     'docker_001.cadvisor.-dev-sda1.fs_usage': 28672,
     'docker_001.cadvisor.fs_summary.fs_limit': 19507089408,
     'docker_001.cadvisor.fs_summary.fs_usage': 28672,
     [...]
 },

cadvisorJenkinsBot · 2015-07-05T19:17:20Z

Can one of the admins verify this patch?

rjnagal · 2015-07-06T20:28:10Z

ok to test

rjnagal · 2015-07-06T20:30:26Z

storage/statsd/statsd.go

+	}
+	return nil
+}
+


Can you move open, close, and Send to a separate client file? We are trying to change the metrics format and merge some of the container stats to values methods. It will make the conversion a bit easier.

No problem, i will update it soon, maybe today.

rjnagal · 2015-07-06T20:31:13Z

Thanks for the PR, @jmaitrehenry

It looks great overall. Will try it out soon !

rjnagal · 2015-07-28T17:39:09Z

LGTM.

This works fine' thanks @jmaitrehenry (and sorry for the delay).

Let's squash the PRs and its ready to merge.

cadvisorJenkinsBot · 2015-07-28T18:55:01Z

Build finished. No test results found.

Add Fs stats to statsd Update import for pointing to official repository Update statsd storage for respecting fmt coding style Create a separate client file Fix gofmt

jmaitrehenry · 2015-07-28T18:55:58Z

Squash done!

rjnagal · 2015-07-28T18:58:20Z

Thanks, merging.

Update statsd storage - issue #724

cadvisorJenkinsBot · 2015-07-28T19:02:10Z

Build finished. No test results found.

scalp42 · 2016-03-04T19:20:46Z

@jmaitrehenry do you know if there's a way to use a Docker env variable as the docker container name ? Kinda like what was done with Prometheus backend: #546

When running in a Mesos environment, containers are just IDs:

4f7c0aa8d03a        java:8                   "/bin/sh -c 'JAVA_OPT"   4 hours ago         Up 4 hours          0.0.0.0:31926->8080/tcp   mesos-29e183be-f611-41b4-824c-2d05b052231b-S6.3dbb1004-5bb8-432f-8fd8-b863bd29341d
66f2fc8f8056        java:8                   "/bin/sh -c 'JAVA_OPT"   4 hours ago         Up 4 hours          0.0.0.0:31939->8080/tcp   mesos-29e183be-f611-41b4-824c-2d05b052231b-S6.60972150-b2b1-45d8-8a55-d63e81b8372a
f7382f241fce        java:8                   "/bin/sh -c 'JAVA_OPT"   4 hours ago         Up 4 hours          0.0.0.0:31656->8080/tcp   mesos-29e183be-f611-41b4-824c-2d05b052231b-S6.39731a2f-d29e-48d1-9927-34ab8c5f557d
880934c0049e        java:8                   "/bin/sh -c 'JAVA_OPT"   23 hours ago        Up 23 hours         0.0.0.0:31371->8080/tcp   mesos-29e183be-f611-41b4-824c-2d05b052231b-S6.23dfe408-ab8f-40be-bf6f-ce27fe885ee0
5eab1f8dac4a        java:8                   "/bin/sh -c 'JAVA_OPT"   44 hours ago        Up 44 hours         0.0.0.0:31500->8080/tcp   mesos-29e183be-f611-41b4-824c-2d05b052231b-S6.5ac75198-283f-4349-a220-9e9645b313e7
b63740fe56e7        java:8                   "/bin/sh -c 'JAVA_OPT"   44 hours ago        Up 44 hours         0.0.0.0:31382->8080/tcp   mesos-29e183be-f611-41b4-824c-2d05b052231b-S6.5d417f16-df24-49d5-a5b0-38a7966460fe
5c7a9ea77b0e        java:8                   "/bin/sh -c 'JAVA_OPT"   2 days ago          Up 2 days           0.0.0.0:31186->8080/tcp   mesos-29e183be-f611-41b4-824c-2d05b052231b-S6.b05043c5-44fc-40bf-aea2-10354e8f5ab4
53065e7a31ad        java:8                   "/bin/sh -c 'JAVA_OPT"   2 days ago          Up 2 days           0.0.0.0:31839->8080/tcp   mesos-29e183be-f611-41b4-824c-2d05b052231b-S6.f0a3f4c5-ecdb-4f97-bede-d744feda670c

You can pass Docker environment variables though, so you could technically inspect the container to grab a certain variable to use as name. Thoughts?

vishh · 2016-03-05T00:27:09Z

cAdvisor should expose container env variables as part of its API. Have you
looked at it?

On Fri, Mar 4, 2016 at 11:20 AM, Anthony Scalisi notifications@github.com
wrote:

@jmaitrehenry https://github.com/jmaitrehenry do you know if there's a
way to use a Docker env variable as the docker container name ? Kinda like
what was done with Prometheus backend: #546
#546

When running in a Mesos environment, containers are just IDs:

4f7c0aa8d03a java:8 "/bin/sh -c 'JAVA_OPT" 4 hours ago Up 4 hours 0.0.0.0:31926->8080/tcp mesos-29e183be-f611-41b4-824c-2d05b052231b-S6.3dbb1004-5bb8-432f-8fd8-b863bd29341d
66f2fc8f8056 java:8 "/bin/sh -c 'JAVA_OPT" 4 hours ago Up 4 hours 0.0.0.0:31939->8080/tcp mesos-29e183be-f611-41b4-824c-2d05b052231b-S6.60972150-b2b1-45d8-8a55-d63e81b8372a
f7382f241fce java:8 "/bin/sh -c 'JAVA_OPT" 4 hours ago Up 4 hours 0.0.0.0:31656->8080/tcp mesos-29e183be-f611-41b4-824c-2d05b052231b-S6.39731a2f-d29e-48d1-9927-34ab8c5f557d
880934c0049e java:8 "/bin/sh -c 'JAVA_OPT" 23 hours ago Up 23 hours 0.0.0.0:31371->8080/tcp mesos-29e183be-f611-41b4-824c-2d05b052231b-S6.23dfe408-ab8f-40be-bf6f-ce27fe885ee0
5eab1f8dac4a java:8 "/bin/sh -c 'JAVA_OPT" 44 hours ago Up 44 hours 0.0.0.0:31500->8080/tcp mesos-29e183be-f611-41b4-824c-2d05b052231b-S6.5ac75198-283f-4349-a220-9e9645b313e7
b63740fe56e7 java:8 "/bin/sh -c 'JAVA_OPT" 44 hours ago Up 44 hours 0.0.0.0:31382->8080/tcp mesos-29e183be-f611-41b4-824c-2d05b052231b-S6.5d417f16-df24-49d5-a5b0-38a7966460fe
5c7a9ea77b0e java:8 "/bin/sh -c 'JAVA_OPT" 2 days ago Up 2 days 0.0.0.0:31186->8080/tcp mesos-29e183be-f611-41b4-824c-2d05b052231b-S6.b05043c5-44fc-40bf-aea2-10354e8f5ab4
53065e7a31ad java:8 "/bin/sh -c 'JAVA_OPT" 2 days ago Up 2 days 0.0.0.0:31839->8080/tcp mesos-29e183be-f611-41b4-824c-2d05b052231b-S6.f0a3f4c5-ecdb-4f97-bede-d744feda670c

—
Reply to this email directly or view it on GitHub
#798 (comment).

scalp42 · 2016-03-05T01:13:21Z

I did, but I can't find how you can use it in the statsd backend unless I'm mistaken the storage backend need to support it no?

vishh · 2016-03-05T01:15:27Z

Yeah. The backend needs to export it.

jmaitrehenry mentioned this pull request Jul 5, 2015

Statsd/Graphite Support #724

Closed

rjnagal reviewed Jul 6, 2015
View reviewed changes

jmaitrehenry force-pushed the master branch from 083be8a to abf204c Compare July 28, 2015 18:52

Update statsd storage - cpu, memory and network done

1e35331

Add Fs stats to statsd Update import for pointing to official repository Update statsd storage for respecting fmt coding style Create a separate client file Fix gofmt

jmaitrehenry force-pushed the master branch from abf204c to 1e35331 Compare July 28, 2015 18:55

rjnagal added a commit that referenced this pull request Jul 28, 2015

Merge pull request #798 from jmaitrehenry/master

843fc13

Update statsd storage - issue #724

rjnagal merged commit 843fc13 into google:master Jul 28, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update statsd storage - issue #724 #798

Update statsd storage - issue #724 #798

jmaitrehenry commented Jul 5, 2015

cadvisorJenkinsBot commented Jul 5, 2015

rjnagal commented Jul 6, 2015

rjnagal Jul 6, 2015

jmaitrehenry Jul 6, 2015

rjnagal commented Jul 6, 2015

rjnagal commented Jul 28, 2015

cadvisorJenkinsBot commented Jul 28, 2015

jmaitrehenry commented Jul 28, 2015

rjnagal commented Jul 28, 2015

cadvisorJenkinsBot commented Jul 28, 2015

scalp42 commented Mar 4, 2016

vishh commented Mar 5, 2016

scalp42 commented Mar 5, 2016

vishh commented Mar 5, 2016

Update statsd storage - issue #724 #798

Update statsd storage - issue #724 #798

Conversation

jmaitrehenry commented Jul 5, 2015

cadvisorJenkinsBot commented Jul 5, 2015

rjnagal commented Jul 6, 2015

rjnagal Jul 6, 2015

Choose a reason for hiding this comment

jmaitrehenry Jul 6, 2015

Choose a reason for hiding this comment

rjnagal commented Jul 6, 2015

rjnagal commented Jul 28, 2015

cadvisorJenkinsBot commented Jul 28, 2015

jmaitrehenry commented Jul 28, 2015

rjnagal commented Jul 28, 2015

cadvisorJenkinsBot commented Jul 28, 2015

scalp42 commented Mar 4, 2016

vishh commented Mar 5, 2016

scalp42 commented Mar 5, 2016

vishh commented Mar 5, 2016