Skip to content
This repository has been archived by the owner on Nov 8, 2022. It is now read-only.

Changed format of metrics published to Kafka as a JSON #21

Merged
merged 1 commit into from
Jul 29, 2016

Conversation

IzabellaRaulin
Copy link
Contributor

@IzabellaRaulin IzabellaRaulin commented Jul 22, 2016

Summary of changes:

  • added formatting "the metrics to publish" based on incoming metrics types
  • adjusted integration tests

Before:

Before the published data looks like below:

[{"namespace":[{"Value":"intel","Description":"","Name":""},{"Value":"mock","Description":"","Name":""},{"Value":"foo","Description":"","Name":""}],"last_advertised_time":"0001-01-01T01:22:00.597278575+01:22","version":0,"config":{"name":"bob","password":"secret","user":"root"},"data": 68,"tags":{"plugin_running_on":"devmachine"},"Unit_":"","description":"","timestamp":"2016-07-20T14:00:34.597278575+02:00"}]

Notice that namespace is represented by core.Namespace struct and could be illegible to publish it in that form.

After changes:

  • publish namespace as a single string
  • do not publish config items (can contain passwd and so on; moreover, tags should be used to expose an additional information)
  • do not publish metric's description (overhead)
  • changed the order of published metric's fields
[{"timestamp":"2016-07-22T13:58:33.193748361+02:00","namespace":"/intel/mock/foo","data":68,"unit":"","tags":{"plugin_running_on":"devmachine"},"version":0,"last_advertised_time":"2016-07-22T13:58:28.68173158+02:00"}]

FYI, To consume the data from Kafka the following command was used:
docker run --rm ches/kafka kafka-console-consumer.sh --topic snap --from-beginning 
> --zookeeper <zookeeper_ip>:2181 > kafka_output.json

Tests done:

  • unit tests
  • integration tests

@intelsdi-x/plugin-maintainers

@andrzej-k
Copy link
Contributor

@IzabellaRaulin Does this change requires documentation and unit tests update?

@IzabellaRaulin IzabellaRaulin force-pushed the format_json_items branch 3 times, most recently from f6e1c5d to 5ce1a18 Compare July 25, 2016 11:52
@IzabellaRaulin
Copy link
Contributor Author

Thanks @andrzej-k for review. I added in README.md section presenting the format of published data + test which covers func formatMetricTypes().

@andrzej-k
Copy link
Contributor

Nicely done @IzabellaRaulin 👍

@IzabellaRaulin IzabellaRaulin merged commit aa27129 into intelsdi-x:master Jul 29, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants