Allow for Tags on non base level JSON format #4284
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
maxunt
force-pushed
the
bugfix/json-parser
branch
from
cc76d9c
to
c9269b8
Compare
plugins/parsers/json/parser.go
Outdated
| case bool: | ||
| tags[name] = strconv.FormatBool(fields[name].(bool)) | ||
| delete(fields, name) | ||
| case int: |
There was a problem hiding this comment.
I think it is not possible to have ints in parsed JSON, so we can remove this case. Let's also add a warning though if we hit the default case:
log.Printf("E! [parsers.json] Unrecognized type %T", value)
plugins/parsers/json/parser.go
Outdated
| for _, name := range p.TagKeys { | ||
| //switch any fields in tagkeys into tags | ||
| if fields[name] != nil { | ||
| switch fields[name].(type) { |
There was a problem hiding this comment.
To avoid needing to type assert in each case, you can do this:
switch value := fields[name].(type) {
Then in each case the value will already be in the type of the case label.
plugins/parsers/json/parser.go
Outdated
| func (p *JSONParser) switchFieldToTag(tags map[string]string, fields map[string]interface{}) (map[string]string, map[string]interface{}) { | ||
| for _, name := range p.TagKeys { | ||
| //switch any fields in tagkeys into tags | ||
| if fields[name] != nil { |
There was a problem hiding this comment.
Nitpick, instead of nesting the switch statement in this check, I would check and continue:
if fields[name] == nil {
continue
}
switch ....
This help with readability because the scope of the conditional is reduced.
plugins/parsers/json/parser.go
Outdated
| @@ -119,6 +159,9 @@ func (f *JSONFlattener) FlattenJSON( | |||
| if f.Fields == nil { | |||
| f.Fields = make(map[string]interface{}) | |||
| } | |||
|
|
|||
| //bool values to be changed to allow strings, bools as fields | |||
| //changed to true true, func switchFieldToTag will remove values | |||
There was a problem hiding this comment.
I would remove this comment
plugins/parsers/json/parser_test.go
Outdated
| parser := JSONParser{ | ||
| MetricName: "json_test", | ||
| TagKeys: []string{"total_devices", "total_threads", "shares_tester", "shares_tester3"}, | ||
| //TagKeys: []string{"mytag", "a", "b_c", "b_d"}, |
There was a problem hiding this comment.
Remove any commented out code
plugins/parsers/json/parser_test.go
Outdated
| t.Logf("error: %v", err) | ||
| t.Logf("num of metrics: %v", len(metrics)) | ||
| t.Logf("metric tags: %v", metrics[0].Tags()) | ||
| t.Logf("metric fields: %v", metrics[0].Fields()) |
There was a problem hiding this comment.
Instead of logging the values, use require to assert that the values are as expected.
| if err != nil { | ||
| return nil, err | ||
| } | ||
|
|
||
| metric, err := metric.New(p.MetricName, tags, f.Fields, time.Now().UTC()) | ||
| tags, nFields := p.switchFieldToTag(tags, f.Fields) |
There was a problem hiding this comment.
Now that we are adjusting the values after flattening, is it still required to keep the loop above on line 47?
|
closes #4260 |
danielnelson
pushed a commit
that referenced
this pull request
Jun 14, 2018
(cherry picked from commit 8482c40)
mkboudreau
added a commit
to mkboudreau/telegraf
that referenced
this pull request
Jun 22, 2018
* origin: (39 commits) Document path tag in tail input Update changelog Added path tag to tail input plugin (influxdata#4292) Run windows tests with -short Fix postfix input handling of multi-level queues (influxdata#4333) Update changelog Add support for comma in logparser timestamp format (influxdata#4311) Update vendoring to dep from gdm (influxdata#4314) Update changelog Add new measurement with results of pgrep lookup to procstat input (influxdata#4307) Update changelog Add valuecounter aggregator plugin (influxdata#3523) Update changelog Update docker input documentation for container status Add container status tag to docker input (influxdata#4259) Drop CI support for Go 1.8 (influxdata#4309) Update changelog Fix selection of tags under nested objects in the JSON parser (influxdata#4284) Update changelog Add owner tag on partitions in burrow input (influxdata#4281) ...
maxunt
added a commit
that referenced
this pull request
Jun 26, 2018
codesmith14
pushed a commit
to signalfx/telegraf
that referenced
this pull request
Jul 5, 2018
…data#4284) (cherry picked from commit 8482c40)
otherpirate
pushed a commit
to otherpirate/telegraf
that referenced
this pull request
Mar 15, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Required for all PRs: