Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

parser json_v2 multiple_timestamps bug #10160

Closed
covered opened this issue Nov 24, 2021 · 2 comments · Fixed by #10221
Closed

parser json_v2 multiple_timestamps bug #10160

covered opened this issue Nov 24, 2021 · 2 comments · Fixed by #10221
Labels
bug unexpected problem or unintended behavior

Comments

@covered
Copy link

covered commented Nov 24, 2021 •
edited by powersj
Loading

Relevent telegraf.conf

# Example taken from: https://github.com/influxdata/telegraf/issues/5940

[[inputs.file]]
    files = ["./testdata/multiple_timestamps/input.json"]
    data_format = "json_v2"
    [[inputs.file.json_v2]]
        [[inputs.file.json_v2.object]]
            path = "events"
            timestamp_key = "time"
            timestamp_format = "unix_ms"

System info

telegraf 1.20.4 / telegraf 1.20.0

Docker

No response

Steps to reproduce

  1. input json：
{
    "events": [
        {
            "name1": "fire1",
            "name": "fire",
            "time": "0"
        },
        {
            "name": "flood",
            "time": "1"
        }
    ]
}

Expected behavior

file name="fire" 0
file name="flood" 1

Actual behavior

file name="fire", name1="fire1" 1577811661000000000
file name="flood" 1

Additional info

No response
@covered covered added the bug unexpected problem or unintended behavior label Nov 24, 2021
@covered
Copy link
Author

covered commented Nov 24, 2021

modify:
telegraf v1.20.4/plugins/parsers/json_v2/parser.go:
line 440 : results = cartesianProduct(results, r)
modify to:
results = cartesianProduct(r, results)

powersj added a commit to powersj/telegraf that referenced this issue Dec 6, 2021
@powersj
fix: json_v2 parser timestamp setting
867dfff 
The json_v2 parser does a carterisan product during parsing. As part of
this product, it will attempt to merge metric fields and tags. However,
timestamps are not part of this merge process. In the current state,
the custom timestamp will always get overriden if the timestamp field is
the last value in the JSON.

Adds a test and does modify one of the tests, which produces the exact
same metrics, but in a slightly different order.

Resolves: influxdata#10160
@powersj powersj mentioned this issue Dec 6, 2021
Merged
@powersj
Copy link
Contributor

powersj commented Dec 6, 2021

Thanks for the bug and the potential solution. I have put up #10221 with a fix and test case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug unexpected problem or unintended behavior
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: json_v2 parser timestamp setting powersj/telegraf
2 participants
@covered @powersj